74295 Romania Functional Review HIGHER EDUCATION SECTOR Final Report [Type a quote from the document or the summary of an interesting point. [ [ [ [ [ [ You can position the text box anywhere in the document. Use the Text T T T T T T May 12, 2011 Box Tools tab to change the formatting of the pull quote text box.] y y y y y y p p p p p p e e e e e e a a a a a a q q q q q q u u u u u u o o o o o o t t t t t t e e e e e e f f f f f f r r r r r r o o o o o o mmm m m m t t t t t t h h h h h h e e e e e e d d d d d d o o o o o o The World Bank Europe and Central Asia Region c c c c c c u u u u u u mmm m m m e e e e e e n n n n n n Table of Contents I. Executive Summary ............................................................................................................. 1 II. Sector Performance ............................................................................................................ 24 III. Performance Management .................................................................................................. 37 IV. The Structure of the Higher Education Sector ................................................................... 50 V. Strategic Management in the Sector ................................................................................... 64 VI. Financial Management in the Sector .................................................................................. 84 VII. Assessing MERYS‘ Capacity for General Management of the Sector ............................ 105 VIII. Annexes ............................................................................................................................ 117 Annex for Tables and Figures ................................................................................................. 117 Annex 1: National Approaches to Learning Outcomes Assessment in Higher Education ..... 136 Annex 2: Government Decision on Student Loan Agency ..................................................... 150 Annex 3: Ministerial Order on Student Loan Agency ............................................................ 154 IX. References ........................................................................................................................ 157 Tables: Table 1: Number of Universities and Teaching Staff per Population Unit by Development Regions ......................... 59 Table 2: The Growth of Private Higher Education in Romania ................................................................................... 61 Table 3: Structure of Program Budget for Higher Education ...................................................................................... 96 Annex tables: Annex table 1: Income of state universities ............................................................................................................... 117 Annex table 2: Educational attainment of 25-29 year olds, by location and by income quintile, 2002 and 2009 ............................................................................................................................................................. 118 Annex Table 3. List of Education Forms and Their Equivalence Coefficients for Year 2010 (as per MER Decision) ..................................................................................................................................................... 123 Annex Table 4: Distribution of Cycle I (Bachelor) Programs in Romanian Universities in the Academic Year 2010-2011 by Field of Study, Number of Programs and Number of Universities Offering them ............................................................................................................................................................. 125 Annex Table 5: The Level of Autonomy in the Romanian State Higher Education System Compared with Other European Countries ........................................................................................................................... 127 Annex Table 6: Indicators monitored in annual report on education ......................................................................... 133 Annex Table 7: Estimates of rates of return in education .......................................................................................... 135 Annex Table 8: National Approaches to Learning Outcomes Assessment in Higher Education -- Assessment Processes / Organizations involved ......................................................................................... 136 Annex Table 9: National Approaches to Learning Outcomes Assessment in Higher Education -- Assessment Format and Design .................................................................................................................. 140 i Annex Table 10: National Approaches to Learning Outcomes Assessment in Higher Education -- Participants .................................................................................................................................................. 142 Annex Table 11: National Approaches to Learning Outcomes Assessment in Higher Education -- Outcomes assessed and results ..................................................................................................................................... 146 Annex Table 12: Personnel Establishment of the Student Loans and Scholarships Agency ..................................... 155 Boxes: Box 1: The Difficulty of Measuring Competencies at the Tertiary Level ................................................................... 39 Box 2: How would performance information about students‘ skills look a nd what questions could such information help answer? .............................................................................................................................. 44 Box 3: Strategic Projects for Romanian Higher Education ......................................................................................... 68 Box 4: Using External Watchdogs to Shed Light on Integrity Problems .................................................................... 77 Box 5: Internationalization at Ion Mincu University of Architecture and Urbanism (UAUIM) ................................. 78 Box 6: Granada university‘s strategy for internationalization ..................................................................................... 81 Box 7: How are state universities receiving their funding from the state? .................................................................. 88 Box 8: Quality indicators used in formula financing of state HE in Romania (in 2010) ............................................. 90 Box 9: Common budgeting and financial management problems across the Ministries of Romania .......................... 98 Box 10: Status of the student loan discussion in Romania ........................................................................................ 104 Box 11: Summary of findings on Human Resource Management from Functional Review of Pre-university sector ........................................................................................................................................................... 116 Figures: Figure 1: Tertiary Enrollment in Romania and the EU ................................................................................................ 26 Figure 2: Private Tertiary Enrollment in Romania and the EU ................................................................................... 26 Figure 3: Tertiary Graduates in Romania and the EU ................................................................................................. 27 Figure 4: Distribution of Tertiary Graduates by Field in EU Countries, 2008 ............................................................ 28 Figure 5: Gap between ―Soft‖ and ―Hard‖ Fields in Romania and the EU ................................................................. 29 Figure 6: Percent of 25-29 year olds who have completed a tertiary degree, by income quintile and by location, 2002 and 2009 ................................................................................................................................ 30 Figure 7: Percent of 25-29 year olds who have completed a tertiary degree, by ethnicity, 2009 ................................ 31 Figure 8: Estimate of the change in supply and demand ............................................................................................. 32 Figure 9: Unemployment rates by highest level of education attained 2002-2009 (%) ............................................... 33 Figure 10: Gross Tertiary Entry Rate in EU Countries, 2007 ...................................................................................... 34 Figure 11: Student-Faculty Ratio in Tertiary Education, Romania and the EU........................................................... 35 Figure 12: Three core performance dimensions of a higher education system ............................................................ 41 Figure 13: Four steps in creating information about what competencies students acquire .......................................... 49 Figure 14: Study-abroad rates are among the lowest in the EU ................................................................................... 80 Figure 15: Share of Students Fluent in Two Foreign Languages, 2005-2008 ............................................................. 80 Figure 16: Spending by state higher education institutions (as a share of GDP) ......................................................... 85 Figure 17: Income of state universities ........................................................................................................................ 86 Figure 18: Cost per student in OECD (2005), EU19 (2005) and Romania's state and private universities (2008) ............................................................................................................................................................ 87 Figure 19: World Bank estimates of full-time equivalent enrollment in state (fee-paying and state- sponsored) and private universities (based on CNFIS and Eurostat figures) ................................................ 91 Annex Figures: Annex Figure 1: Initial budget as % of final budget .................................................................................................. 124 Annex Figure 2: Initial budget as % of actual expenditures ...................................................................................... 124 ii Abbreviations ACPART University Qualifications Agency and Partnership with Economic and Social Environment ARACIS The Romanian Agency for Quality Assurance in Higher Education AHELO Assessment of Higher Education Learning Outcomes CCCDI Advisory Board for Research, Development and Innovation CEMU University Ethics and Management Board CNATDCU National Council for Attestation of University Titles, Diplomas and Educational Certificates CNBU National Council of University Libraries CNCS National Council of Scientific Research CNECSDTI National Council of Ethics for Research, Technological Development and Innovation CNFIS National Council of Higher Education Funding CNSPIS National Council of Statistics and Forecasting of Higher Education EC European Commission EU European Union HEI Higher Education Institution MERYS Ministry of Education, Research, Youth, and Sports OECD Organization for Economic cooperation and Development PIAAC Programme for International Assessment of Adult Competencies UEFISCDI The National Council for Financing Higher Education, Scientific Research, Technological development and Innovation iii Acknowledgments This Report on the Functional Review of the Higher Education sector is part of the broader strategic and functional review of Romania‘s central public administration being undertaken by the World Bank on behalf of the European Commission and the Government of Romania (GoR) as per their June 2009 Memorandum of Understanding. The report was prepared by a core team led by Bojana Naceva and Lars Sondergaard, and comprising Mariana Doina Moarcas, Eliezer Orbach, Igor Kheyfets and Mihai Korka, with inputs from Mehtabul Azam and Kristin Sinclair. Raluca Marina Banioti, Tanta Duta, and Andreea Silvia Florescu provided invaluable logistical and administrative support. The team would like to express its gratitude to the officials of the Ministry of Education, Research, Youth and Sport and various government agencies for their constructive collaboration. In particular, the team would like to thank to the Minister of Education, Mr. Daniel Funeriu, and Mr. Sorin Gabriel Popescu, General Director, and Mr. Iona Ciuca, Deputy Director, for their effective leadership of the functional review process. The team worked in close consultation with Mr. Bernard Myers (Task Team Leader of the Functional Review Project) and Donato De Rosa ( Task Team Leader of the R&D Functional Review), and under guidance from Peter Harrold (Country Director), William Dorotinsky (Sector Manager, Public Sector and Institutional Reform), Alberto Rodriguez (Sector Manager, Education) and Mr. François Rantura, (Country Manager, Romania). The report was peer reviewed by Jamil Salmi( Tertiary Education Coordinator, Human Development Network), Ana Maria Sandi (International consultant), Sorin Ionita, (Director, Romanian Academic Society) and Gelu Calacean ( Policy Officer, DG Education and Culture, European Commission). iv I. Executive Summary 1. The need for improved effectiveness and efficiency stems from recent changes in the world economy where technology has become the driver of growth and where knowledge, skills and competencies are key to its application. A skilled workforce today is critical to development and the competitiveness of nations depends on its quality and availability. This has made higher education critical to growth and the expansion of higher education, as well as an improvement in its relevance and quality, a top development priority. Many countries, therefore, are designing new visions for their higher education sectors and are devoting much attention both to planning in, and to the management of, these sectors. 2. Not surprisingly, the importance of improving competitiveness through innovation and faster labor productivity growth is a centerpiece of the EU2020 strategy (EU Commission 2010). The strategy explicitly recognizes that such improvements critically depend on improving the skills of the workforce. In fact, four of the strategy‘s six targets depend on improving the education and training of the workforce, and four of EU Commission‘s seven ―flagship initiatives‖ aimed at supporting the implementation of the EU2020 Strategy are directly or indirectly related to building skills.1 3. In Romania improving the education and training system is critical not only to meeting the two EU2020 targets which are directly related to education (i.e. raising enrollment rates in higher education and reducing the rate of early school leavers) but also to turning the country into a growth engine in Europe. 4. Romania has undertaken important reforms in the past decade, in terms of both policies and size. Recent policy changes include: (i) shifts in the structure of most study programs as a result of Romania‘s participation in the Bologna process; (ii) a reform of higher education finance (in 1999) involving a move away from input-based funding toward enrollment-based funding; (iii) a substantial increase in overall funding in recent years; and (iv) since 2001, the introduction of provisions allowing state universities to admit fee-paying students, which provide a major new 1 There are six headline targets (actually there are five but the target focusing directly on education is actually two separate ones: one on higher education enrollment and one on school dropout). The targets which require improving the education of the population are: (1) If 75 % of the population aged 20-64 are going to be employed, a country will need to make sure that its citizens are employable and remain so for their entire life; (2) If a country is going to invest 3% of its GDP on R&D, it will need able researchers, skilled scientists and engineers to make good use of these resources; (3) If a country wants to reduce early school leavers to less than 10%, it needs to make sure that no one is left behind in the early grades; and (4) if 40% of a country‘s younger generation is to graduate with a tertiary diploma, it should ensure that these youngsters learn useful skills in the process. And the four EU2020 ―Flagship initiatives‖ which all have large elements of education and training in them are: ―Youth on the move‖ ; ―Innovation Union‖; ―An Agenda for new skills and jobs; and ―European Platform against Poverty‖. 1 revenue source for university expansion and quality improvement. Furthermore, the education law recently adopted, if implemented successfully can direct the sector towards improved management of Higher education institutions and sector‘s outcomes. 5. Growth in enrollment has been impressive (total enrollment in state and private universities more than doubled between 2000 and 2008), but there is a concern that the quality of education has likely suffered and the equity dimension was not addressed comprehensively2. Although the absence of data on learning and employment outcomes makes it difficult to draw firm conclusions about the quality or relevance of the education which Romanian student receive, some indirect evidence is worrisome. In 2006, 53 percent of Romania's students failed to attain basic proficiency levels in reading and mathematics on the PISA international assessments of 15- year-olds, while nearly 90 percent scored below the OECD average. Due to very high transition rates from secondary to tertiary education in Romania, there is reason to believe that many students in this cohort entered university in 2009 lacking basic skills expected of university entrants in most OECD and EU countries3.Secondly, a large part of the expansion of tertiary education has taken place in newly established (often private) institutions, in part-time (or weekend) and long-distance learning programs. The format of these programs is new, and their quality unknown. It is impossible to determine whether these students are receiving the same quality education as their peers in full-time, regular programs. 6. This review provides a discussion of these challenges in more details and provides policy options on how to tackle them. 7. The review focuses on the higher education sector as a whole and how it is managed as a system by the Ministry of Education, Research, Youth and Sport (MERYS). The team assessed how the system is structured, led and managed by MERYS and not how each and every university is structured, led and managed internally. Some internal issues that confront higher education institutions were, however, reviewed since they are common to so many institutions that, by reason of commonality, they deserve to be addressed so that the Ministry can develop appropriate responses. 8. The report includes a large number of findings, conclusions and recommendations. Some of the findings and conclusions require no recommendations. They indicate that the review team is happy with what it found in specific areas and they are included in the report since the team was required to cover these areas and since the team wants to give credit where credit is due. There are other findings and conclusions, indicating areas where improvement is necessary and possible and the team is providing recommendations, but on their own, are not likely to bring about major improvement; then, there are findings and conclusions in areas where the team believes that the potential for the greatest impact lies. 2 More than 50 percent of the richest quintile of Romania‘s youth (25 -29 year olds in 2009) hold a tertiary degree, whereas less than 5 percent of the poorest quintile do (see Figure 6 and Annex Table 2) 3 This hypothesis was further corroborated by a panel of university rectors convened for this study. 2 9. The team will present first main messages including recommendations on what needs to be done in each area. The main messages section is followed by the action plan which contains the recommendations and policy actions regarding all areas of the Review in table format for the consideration of the Ministry. Main Messages and Recommendations 1. Introduce performance data that measure what competencies students acquire and whether graduates find jobs. The Review argues that to assess performance in higher education, three dimensions need to be assessed and monitored: are students acquiring the right competencies during the course of their study; are graduates finding jobs; and is quality research being produced? Currently, of the three dimensions, data are only being collected on faculty research output (and data show that research output is on the increase). Romania is making rapid progress in adding a performance indicator on whether graduates are finding jobs. Until now, only ad hoc studies of graduates‘ employment outcomes are available. The most important performance data which is missing from the discussion in Romania‘s HE system is whether students learn during the course of their studies, and arguably this performance indicator is the most important of the three. These gaps in performance data constraint the ministry‘s ability to manage for performance: without clarity on what performance means, and indicators to measure progress, strengths and weaknesses, it is impossible to assess the performance of the sector, to strategize effectively, it is difficult to design policies (and mobilize resources) to remedy potential problems; and it is impossible to hold actors in the sector accountable for what really matters, namely results. Recommendations: Making full use of tracer study results as they become available (1) Help institutions use results to get feedback from the labor market. Such information can be used to: a. Help institutions design better study programs, including better aligning their offerings with demand; b. Help improve the marketing of their programs (vis-à-vis students) (including with regard to international students); and c. Be used as an engagement tools to strengthen their alumni network. (2) Make students voting with their feet a more powerful force for improving the quality of education 3 a. Make publicly available easy-to-read information about prospective earnings, employment rates, types of jobs typically acquired upon graduation, and job satisfaction rates; b. Organize regional ―future job‖ fairs for high school students; c. Distribute pamphlets with easy-to-read to last year high school students; (3) Embed information in system-wide quality assurance a. Use information to differentiate between universities and programs of study; b. Use information in accreditation decisions; c. Use it to prioritize public spending, e.g. by identifying priority fields of study that should be supported by public resources. (4) Use launch of tracer study results as a vehicle to bring employers, education institutions and policy makers around the table to discuss skills needs. a. Involve sector councils in discussion preliminary results, get them involved in questionnaire design and analysis; b. Host an annual conference to present findings (of tracer study) and, every three years, a ―skills forecast‖ (e.g. similar to what CEDEFOP is doing on an EU-wide basis). Developing the tools to assess whether students acquire the desired competencies during the course of their study  Build on top of what has already been achieved in the context of the National Qualifications Framework project: university representatives and sector specialists have already defined competencies for all bachelor programs. Have the same specialists work with experts in developing standardized assessment tools, pilot them, host conference to discuss findings, and further refine instruments.  Start where there is least resistance. Conduct preliminary discussions with universities and sector experts to pick a field of study where there is strong interest from universities in getting started. For this to succeed, champions within universities who can help cajole others to participate have to be found. The more universities participate, the higher the value will be to students – and to outsiders wanting to gauge what skills students acquire during the course of their studies.  Recognize that success will take years but that the important thing is to start a process that will gradually lead to the development of assessment tools. o Define/refine competencies that graduates should master o Develop/refine assessment tools 4 o Test students (collecting background information about the students to allow researchers to compare performance of similar students) o Analyze and disseminate results to students; institutions; employers and policy makers with the aim to:  Help students become better at voting with the feet  Help institutions design better programs  Discuss findings with employers o Compare findings with results from tracer studies: does it look like the competencies defined are the ―important ones‖ in terms of who finds jobs? Is the sector producing graduates within the right fields of study?  Learn from the few international examples that have started this process, including the Collegiate Learning Assessment in the United States (see discussion below).  Sign up to participate in AHELO and make active participation in all OECD working group meetings a high priority. During these meetings, the world‘s foremost experts on the field brainstorm and discuss pitfalls. And by participating in AHELO, substantial technical expertise in administering and analyzing such test results would be transferred to Romania.  Sign up to participate to Programme for International Assessment of Adult Competencies (PIAAC). (for the same reasons as above)  Emphasize that this test is an assessment, not an examination. The primary point is to gather information (to be used both by students and universities – and by the Ministry to assess performance) on the quality of the teaching and learning which takes places in the Romanian HE sector; the purpose is not to institute a national ―graduation‖ exam. Measure intermediate results: are students engaged and is the problem of unethical behavior being reduced?  Conduct annual survey of students  Finance civil society watchdogs to annual track (and report on) unethical behavior in higher education. 2. Make the financing of Higher Education more policy-oriented and less discretionary 10. The Review finds several features of the Romanian the system of financing which are very strong. These include (i) the long tradition of using formula financing to allocate core financing for staff and recurrent expenditures to state universities; (ii) the fact that research grants (approx. lei 1.8 bn in 2008) are allocated on a competitive basis, with universities competing and 5 receiving an increasing proportion; and (iii) the fact that the financing of higher education has gradually become more diverse, reducing the dependence on constrained public resources. 11. There are several aspects of the financing of higher education, however, which can be improved upon. This review focuses on four aspects that seem particularly important to address: the large of amount of discretionary decision making in budget allocations; the lack of policy orientation in the budget; the lack of a vision and a strategy for mobilizing more resources for higher education; and the inadequacy of the reporting formats which are used to monitor spending in higher education. Recommendations: 12. The review‘s recommendations focus on addressing each of these four weaknesses and emphasize  the importance of bringing more meaningful performance indicators (see discussion on knowledge gaps) into the budget discussion;  relying more on students voting with their feet as a source for allocating public resources (equipped with better information about employment and learning outcomes);  shifting public resources from mostly supporting the best (and, likely most well-off) students to mostly financing the students most in need of financial support;  more analysis of future skills needs conducted and discussed with employers‘ association;  more strategic use of the private sector as a vehicle to meet the sector‘s objectives  more transparency and better reporting. 3. Expand autonomy in exchange for results-focused accountability 13. The performance of many Romanian universities leaves much to be desired. These universities are not being managed well enough strategically and operationally. Although data to assess their performance in terms of learning or employment outcomes of their students and graduates does not exist, it appears that their productivity in both teaching and research is low. One of the most important reasons for this poor performance has been an almost complete lack of accountability for delivering students that can demonstrate they have mastered certain competencies and skills and graduates that find jobs; until now accountability has been focusing on financial accountability (i.e. are they spending their resources in accordance with the public finance law?). Up to now, these universities suffered no consequences if their students did not acquire competencies, skills and knowledge and their graduates did not find jobs – to a large extent because no one defined ―performance‖ in this way and no one is measuring the extent to which universities perform according to these dimensions. These universities also suffered no consequences for poor performance. In line with global norms of the time,the Government 6 continued to finance them on the basis of a formula based on the number of students enrolled and not on the quality of the education received by them. 14. Alongside this lack of accountability for performance, the autonomy of Romanian universities expanded during the period 1995-2005 which was a period that saw decreased state regulation, increased academic self-governance, and greater managerial governance at the university level (Stensaker, de Boer, and Enders, 2006). The problem was that these increases in autonomy (and more flexibility in financing) happened very quickly—before nascent measures aimed at holding institutions accountable could take hold. For instance, the rapid expansion in enrollment that began in 1990 occurred when the newly established Romanian Council for Accreditation (CNEAA, established in 1993), was still developing its mandate and trying to establish its institutional credibility. Many new (mostly private) institutions were established during the boom years, but CNEAA simply did not have the capacity to review and accredit every new program and/or institution; that task was, in some instances, outsourced to state universities, which—in exchange for a fee—were charged with taking a private, growing institution under their wings. By 2005, the shortcomings of CNEAA had become apparent and it was replaced with a new quality assurance agency—the Romanian Agency for Quality Assurance in Higher Education (ARACIS)—modeled on the European Standards and Guidelines (ESG) for quality assurance (see Korka 2008). 15. In response to this perceived imbalance, the government has realigned autonomy and accountability norms for universities, particularly through its new law on education (2011). It imposed many restrictions on administrative autonomy, financial autonomy, staffing autonomy and academic autonomy. For the universities, the most important restrictions are in the areas of staffing and financing, where the restrictions are imposed not only by education laws, but also the laws of public employment and public finance. Universities cannot freely decide on the recruitment and appointment of academic and non-academic staff; they depend on the government for the bulk of their budgets and have serious limitations on spending even of money allocated to them. 16. Moreover, in the new Education Law which was enacted earlier this year, the government introduced key measures to strengthen accountability for performance. For instance, the law requires extensive reporting by the universities and much transparency in their operations. Importantly, the law also introduces for the first time two powerful and realistic sanctioning tool: the provision, or withholding of financial support with respect to state universities and the provision or withholding of recognition with respect to private universities. State universities that do not meet minimal operational standards will no longer receive public funds as before, while universities that excel will receive more than they have received before. Private universities that fail to perform at minimal standards shall not get State recognition of the diplomas they grant to their students, and those that perform well will not only get recognition from the State, but will also qualify to compete for research and institutional development funds provided by the State. 7 17. The team believes that the managers of institutions are most effective when they have the freedom to make decisions on how to manage their institutions; it believes further that neither direct control with strong micro-management by the Government, nor full autonomy at the universities to do what they deem fit, without accountability, is desirable. There is significant evidence in the research literature that university autonomy is positively correlated with performance and that both in Europe and the USA more autonomous universities that also need to compete more for resources are more productive. The evidence suggests that centralized government control may be much less effective at enhancing performance than making universities more autonomous and surrounding them with a competitive environment. As was reported by one group of researchers,4 European countries, such as the United Kingdom (U.K.) and Sweden, have unusually autonomous universities which are also unusually productive, and in the USA, universities that are more autonomous and face more competition have higher research outputs than those that are less autonomous, and are facing less competition. One of the main findings reported by this group is that universities with higher Shanghai rankings enjoy greater autonomy and draw a higher percentage of their budgets from competitive resources. Recommendations: 18. The conclusion is that there needs to be a balance between autonomy and accountability and that when full accountability is in place, and when competition among universities is strengthened much more autonomy can be given to the universities. The government has now introduced through the new Education Law significant competition (through the classification and ranking of universities) as well as many of the necessary measures of accountability. Clearly, these measures are not going to automatically and immediately translate into accountability on the ground. But once the Ministry gains experience with the system of competition, rewards and sanctions and sees an improvement in accountability on the ground, it is recommended that the universities increasingly regain autonomy. Group sanctions are not a good idea. There is no reason to restrict the autonomy of universities that are performing well and are taking full accountability for their performance. The team recommends that the Ministry will selectively give more autonomy to universities that are consistently well-performing and consistently providing full and transparent reports on their performance. The amount of control can be adjusted to the level of performance and the quality of reporting. Furthermore, in dealing with accountability, it is important to define clearly ‗accountability for what‘. The Ministry, in collaboration with the universities, is advised to formulate precisely the elements of performance for which universities should be held accountable. 4 Philippe Aghion, Mathias Dewatripont, Caroline M. Hoxby, Andreu Mas-Colell, André Sapir, The Governance and Performance of Research Universities : Evidence from Europe and the U.S., Working Paper 14851, National Bureau of Economic Research, 2009 8 4. Optimize Differentiation in the Sector 19. The sector consists of the Ministry, 108 higher education institutions and 12 intermediary organizations, of which nine are predominantly consultative, while three are fully operational, carrying out specific functions on behalf of the Ministry. This structure of the sector makes sense: all of the roles and functions that need to exist in it do exist, there are no functional gaps and, with one exception (see Chapter IV on Sector Structure, p. 53), no functional overlaps. 20. The level of differentiation among the 108 universities, however, is not optimal. On one hand, universities are not specializing enough in areas such as main line of business, mode of delivery or clientele; on the other, they seem to specialize too much in study programs. They do not clearly distinguish among themselves by key line of business, such as teaching and research. Most of them claim to be involved in both. They do not attempt to focus on key types of clientele; most are trying to serve every type, whether high-school graduates, working adults, full-time students or part-time students. They do not attempt to focus on key modes of delivery; most are attempting to deliver instruction both on site, in the classroom, and off-site through the internet, or by correspondence. The Ministry has started addressing the issue of differentiation in one area: key line of business. With the new education law it is introducing a mechanism to distinguish between research–intensive, teaching and research and predominantly-teaching universities. This is an important step that will also help the Ministry bring about a gradual change from a system where research is done predominantly in dedicated research institutes into a system where universities play an equally important role in research. But, except for this step the Ministry is yet to tackle the issue of differentiation. 21. Of potentially key importance is the differentiation among universities by fields of study and the proliferation of offerings in these fields of study. Here the question asked by the team was whether there is over-differentiation and too high a level of proliferation. There are in Romania a large number of specialized, single-domain (sometimes even single-field) state and private universities, dedicated to fields such as medicine, pharmacology and health care, engineering, agriculture and veterinary, economics and business administration, architecture, music and performing arts. Also, there are currently 2,562 study programs on offer at the Bachelor level and 2,306 study programs on offer at the Master level in 73 fields of study. The team‘s concern that these numbers represent over-differentiation and over-proliferation emanates mainly from the severe financial constraints faced by the Government in supporting higher education. One way to alleviate financial constraints is to increase overall efficiency through economies of scale. Both differentiation and proliferation reduce the ability to achieve such economies. Still, the idea is not to achieve economies of scale at the expense of needed differentiation and reasonable proliferation. What needs to be achieved is a balance between the two. For the Ministry, achieving such a balance – not by decree, but by a system of incentives – is a strategic matter. It was not possible for the team within the scope of this review to treat this strategic matter in 9 more detail, but the team recommends that the Ministry conduct an in-depth study on the pros and cons of having highly specialized universities and on the desirable number of study programs and, if necessary, consolidate or close some of them. 22. Focusing on geographical differentiation, the supply of study places in different regions does not match the need for higher education in these regions. The ratio of people to university varies greatly across the regions reflecting potential weaknesses in the economic and social development potential of these regions and potentially also an inequitable access to higher education. Not enough information is available in this area and the review team recommends that the Ministry conduct an in-depth assessment of the need for the expansion of higher education services in the currently under-served regions, bearing in mind the need to avoid excessive proliferation of study programs, and develop a plan to address this need. 5. Harness the Power and Potential of Private Higher Education 23. The most significant differentiation that occurred in the Romanian higher education sector over the last 15 years has been the emergence and growth of private institutions. There are today 52 private institutions and (in 2009/10) 322,337 enrolled students and 91,803 graduates. This represents 48% of all institutions, 41.5% of all enrolment and 42.2% of all graduates. Most of the institutions got full accreditation during 2006/8 and most of the students come from two groups of clientele: (a) people who are not being admitted to state universities in the country and (b) people who cannot study on a full time basis, or cannot physically attend university even on a part-time basis. With respect to the second, it is important to note that private higher education institutions are carving for themselves two very important, closely-linked and growing niches in the market: the niche of people who do not have time to attend regularly because they are employed and the other is the niche of people who, employed or not, cannot attend because of distance and/or the need to stay home. The institutions are offering them both part-time programs and distance learning programs. Of the 322 337 enrolled students in 2009/10, 32.5% are part time and 6.2% are distance students – together 38.7%. 24. The development of the private higher education sub-sector and the direction it takes are matters of strategic importance to the MERYS. Both the President‘s Commission and the Ministry have recognized the importance of the sub-sector. While subjecting it to all the requirements and conditions aimed at ensuring quality education, they have already opened the door for private universities to enjoy some of the privileges enjoyed by state universities. This is a step in the right direction. It helps the higher education sector align itself better with conditions in the market for higher education. But more needs to be done. 10 Recommendations: 25. The review team recommends that the Ministry conduct an in-depth review of the present status and performance of private higher education institutions in Romania in order to develop a comprehensive strategic framework for the next steps. The strategic framework should chart more clearly the future role of private universities in the education sector of the country, as well as the kind of public sector environment that will be most conducive to its success in playing this role. This should include strategies to bring about a complementary division of labor, to enhance collaboration between state and private universities and to help the private ones bring the quality of their graduates to the level of state universities (which needs to improve too). 6. Improve capacity of key intermediary organizations 26. The Ministry is managing the higher education sector strategically, using the intermediate organizations as its key instrument in steering it. Through these instruments, the Ministry has already made impressive progress. Nine of the intermediary organizations are councils and boards that meet a few times a year mostly to deliberate on policy and strategy issues and advice the Ministry on them. They are small legal entities, consisting of a relatively small number of professionals from the academic community who volunteer their time and are not paid for their advice. Three organizations are implementation agencies carrying out important and laborious functions on behalf of the Ministry: one is the University Qualifications Agency and Partnership with Economic and Social Environment. It is developing the National Qualifications Framework and the required qualifications; one is the Romanian Agency for Quality Assurance in Higher Education whose role is to assess universities and study programs and provide accreditation and the third one is The Executive Unit for Financing HE and Research, Development and Innovation. This agency (referred to as the Executive Unit in short throughout this report) is serving as the administrative and logistical arm of six councils and boards and is also carrying out the Ministry‘s actual work in the areas in which they provide policy and strategy advice. For example, it processes all applications for research grants within the policy framework provided by the Ministry‘s Advisory Board for Research, Development and Innovation and under the new Education Law will now do all the work of providing attestation and habilitation within the framework provided by the National Council for Attestation of Titles, Diplomas and University Certificates. The Huge Operational Work Load of the Implementing Organizations 27. The workloads carried out by these three organizations are huge. UECNCFPA - the qualifications agency is dealing with 3,850 occupations So far, 250 occupations have been validated on a pilot basis. The bulk of the work will come after finishing this summer the DOCIS project which is providing mainly for the setting up of the National Qualifications Framework and the installation of the electronic platform that will host the Romanian Register of 11 Qualifications. ARACIS - the accreditation agency is going to temporarily authorize and/or fully accredit 521 study programs in 2011 and 624 in 2012. It will also conduct assessments of 16 higher education institutions in 2011 and 13 institutions in 2012 for the purpose of accreditation. In each of these assessments, at least 20% of all programs have to be reviewed. The Executive Unit, which is serving six advisory organizations, is dealing with an extremely large and varied work program. The attestation work of the National Council for Attestation of Titles, Diplomas and University Certificates alone will require it to review between 2000 to 4000 requests for attestation in 2012 and some 2000 request annually thereafter. The habilitation work will require it to review some 500 applications annually. Key factors that constrain the capacity of the three organizations 28. The three intermediary organizations don't have built-in capacity to match this work load. They were established, structured and staffed with little or no regard to expected workloads. Their organization structures, staff numbers and other operational characteristics – all of which are usually matters left to Management at facility level – were fixed by law, denying managers the discretion and flexibility needed to manage efficiently. With one exception, they are financially critically dependent on external European funding. 29. Organization structures - The structures that were reviewed by the team do not exist any longer. These structures were inadequate, ignoring key principles of structuring. They were not fully supportive of the functions of the three organizations. The characteristics of the structures that impeded performance in the past are discussed in the chapter on capacity. There is no need to discuss them here. One of the recommendations made by the team is that the Ministry gets technical assistance to help design the new structures. 30. Financial Resources - without the help of European funds, none of the three organizations would have been able to do the initial ground work done so far. The three organizations have built or are completing now the building of basic, and very important, infrastructure in the areas of qualifications, accreditation, attestation and habilitation. In some areas, such as accreditation and attestation they have already switched to a fully operational mode, but the funds received from European sources were aimed at the development of the infrastructure and not the regular on-going operations thereafter. No financial planning has been done by these organizations for these operations so far. None of them was clear at the time this review was conducted how much money will be needed in the future and none, except for ARACIS, knew for sure what will be the sources of this money. ARACIS is the only one that is charging institutions for accreditation services and is self-sufficient in this line of business, but it is getting new work that is not likely to be funded by European funds, such as the classification of universities and the rating of study programs. Currently, it and the other two organizations get most of their funds from project 12 work, and the need to get money from projects is pushing some of them to undertake work that is not in their core business. 31. Human resources - Two key factors are constraining, or will soon constrain, the capacity of the three agencies. The first has to do with a heavy reliance on external expertise. In all four areas of work – qualification, accreditation, attestation and habilitaton – these agencies use panels of experts to do the needed assessment. The panels consist of between 3, 5, 11 and some even 19 experts, depending on the specific assessment. All of the agencies expect the experts to come from the Romanian academic community and increasingly also from outside the country. 32. These expectations are unrealistic. To take one example, as indicated above, the Council for Attestation and the Executive unit are planning to have between 2000 and 4000 requests for attestation in 2012 and about 2000 requests annually thereafter. Since its panels consist of three experts, this translates into 6000 to 12000 ‗units‘ of participation by expert evaluators in 2012 and 6000 units annually thereafter. It is expecting also to have 500 requests for habilitation annually, for which it is planning to engage 5 experts per panel. This will translate into 2,500 units of participation annually. This represents the demand for experts coming from just one of the three agencies. The team believes that the pool of experts in Romania is not large enough to provide such large numbers of participation units, each lasting between 3 to 7 days regularly, whether on a voluntary basis, free of charge, or even at a reasonable fee. In each of the organizations, work processes and procedures are being considered or reconsidered now. In each, target dates for the completion of the first round of work are being set now and – in the case of accreditation - decisions are being made on the frequency of re-assessment and re-accreditation. These processes and procedures have great implications for the work load and once adequate analysis is done it may be necessary to go back to them and change them so as to lighten the load. 33. The second constraining factor has to do with the number of full time staff available to the intermediary organizations either internally or within the Ministry. Although the bulk of the work load is done by the external experts, these experts have to be mobilized and managed and the number of full-time internal staff is too small whether relative to the number of qualifications, attestations, habilitations and accreditations to be done, or relative to the numbers of external experts to be mobilized and managed. A very small number of internal staff is operating a force of external experts that runs into the hundreds and thousands. 34. Management - The huge and sometimes complex operations of the organizations delivering accreditation, attestation, habilitation and qualifications services require high-level operational management. There are few, if any, professional managers in the three organizations. The practice of taking university professors to fill key management positions in the intermediary organizations is widespread. The practice raises two issues: the first has to do with their ability to 13 fully discharge their responsibilities in two full-time jobs, since there is not enough time in the day; the second has to do with their ability to manage well, since most, if not all, come from disciplines that have little to do with management and don‘t have significant management experience in the running of large-scale operations. Both the lack of financial resources planning and the lack of human resources planning point at this lack of experience in the management of large operations. 35. The Ministry and all of the intermediary organizations are currently busy developing the secondary legislation, the methodologies, that will clarify and specify the many changes that were promulgated in the new Education Law with respect to their roles, functions and operations. It is recommended that once this is done, each of the three implementing agencies will embark on a major organization development exercise with the objective of ensuring that it will have full capacity to carry out its role in the system most effciently and effectively. It will be very useful for the Ministry to obtain technical assistance funds with which to acquire management consulting services for this purpose. 36. The exercise should focus on the following:  Developing a tool that can be used to assess methodically the unit-based work load that the organization will have to deal with in the short, as well as, medium term and then conducting the actual assessment  Reviewing the design of all types of assessments with the aim of simplifying processes and procedures and optimizing the mix of resources used; much attention should be paid to the following: (a) the impact of current recognition and accreditation norms and standards on the volum of work, (b) the balance between full-time regular staff and external experts, (c) the balance between Romanian and international experts and (d) increasing the use of information technologies so as to reduce the time required of internal staff as well as external experts  Optimizing the organization structure of each of the three intermediary organizations  Conducting human resources planning both for the immediate future and for the next 10 years to determine optimal staffing needs  Conducting financial planning both for the immediate future and the next 10 year to establish clearly funding requirements as well as sources of funds and ensure a stable flow of funds  Designing a management development training program and training of all managers Recommendations: 37. The team recommends further the following four recommendations: 14  The organization structure of each of the intermediate organizations will have a position for a full-time professional General Manager as well as positions for full-time unit managers for all organizational units  The Ministry appoints full-time professional managers for all management positions disallowing them undertake additional full-time jobs simultaneously.  The Ministry avoids specifying organization structures, staffing levels and other operational matters in primary and secondary laws; and  The Ministry uses the new student loan scheme as a pilot to develop and introduce a performance contract, based on measurable performance indicators, between it and the Student Loan Agency. 7. Strengthen Efforts at Internationalization 38. Internationalization is a strategic issue in higher education all over the world including Europe today, considered to be the third most important change driver in European higher education in the past three years and expected to move to first place within the next five years, according to the European Universities Association. Encompassing student and faculty flows across borders, internationalization provides opportunities for collaborative research work as well as experiential and academic learning that increases the contribution of higher education to individuals and society. With regard to student mobility, the global number of international students rose by more than 75% since 2000, reaching 3.43 million in 2010 and projected to reach 8 million by 2025. 39. The potential benefits of internationalization are great: from helping to improve the quality of education at home to enhancing both the production and the quality of research, through the enhanced mobility of students and professors and through exposure to better or best practice and through competition. Internationalization can also have financial and economic benefits when net flows are positive, since international students pay fees and foreign providers pay taxes. The risks are not as great, but are important nevertheless, and chief among them is the risk of brain drain. 40. The Ministry of Education has not yet developed its thinking regarding internationalization. It has no strategy aimed at reaping the potential benefits of internationalization or addressing the risks. Ideally, there should be an organizational unit in the Ministry to deal with internationalization and the team recommends that the Ministry establish such a unit in the context of its forthcoming reorganization. The first item on the agenda of this unit needs to be developing a comprehensive strategy for internationalization. Part of this strategy needs to aim at reversing the imbalance between Romanian students abroad (22,432 in 2008) and foreign students in Romania (13,875). 15 Recommendations: 41. Universities could be encouraged and helped by the Ministry to: (a) Design more programs especially for undergraduates from other countries (some have already be designed). These need to be in English and other international languages, and can take place during the regular academic year and also on holidays. Some may be structured around specific disciplines as is done by the German DDA. (b) Conduct well-planned, systematic marketing campaigns abroad (c) Widen student services to help foreign students who are planning to come to Romania. Such services could include for example help in the locating and renting of apartments or actually building residences for students; and (d) Develop cooperation mechanisms through regional clustering and networking at the European level and use institutional and academic partnerships particularly to improve doctoral education. 16 DRAFT REFORM ACTION PLAN HIGHER EDUCATION SECTOR Objective Sequenced Actions Impact/ Implementa Responsibi Progress/Output Target/ Financial Priority tion Period lity Outcome Resource s 1. Introduce performance data that measure what competencies students acquire and whether graduates find jobs 1.1 Making full use Use launch of tracer study results as a Critical < 6 months MERYS Tracer studies are conducted MERYS better positioned vis –a- vis €€ of tracer study vehicle to bring all key stakeholders Intermediate annually., with more and more different interest group and able to around the table to discuss skills needs Agencies institutions participating and more effectively manage the system for results as they performance. Closer linkages between and utilization of results, embed with the response rate (of become available study programs offered and labor information in system-wide quality graduates) increasing over time. market needs. assurance, make publicly available disaggregated results aimed at helping Results are analyzed and data students to make informed decisions and analysis widely When surveyed, students increasingly disseminated. report using tracer study results when making decisions about what and where to study Skills forecasts are used in decisions about allocation about public resources for higher education. 1.2 Developing the Build on top of what has already been Critical >18 months MERYS Standardized tests administered Romania Higher Education sector €€€ tools to assess achieved in the context of the National HEIs to students tested (and collects and monitor on a regular basis Qualifications Framework project. Have background information about information about core performance whether students the students are collected) indicators (besides graduates the same specialists work with experts in acquire the desired employability and research outputs) developing standardized assessment Results analyzed and which facilitates taking informed competencies tools, pilot them in a field of study where disseminated to students; decisions and improving the quality of during the course there is interest from universities in institutions; employers and the sector of their study participating, host conference to discuss policy makers findings, and further refine instruments. Emphasize that the use of tests is to assess, not to introduce another layer of exams. The idea is to track performance of the HE sector, identifying strengths and weaknesses. 17 Objective Sequenced Actions Impact/ Implementa Responsibi Progress/Output Target/ Financial Priority tion Period lity Outcome Resource s 2 Make the financing of Higher Education more policy-oriented and less discretionary 2.1. Reduce the Reduce the role of the Ministry in High < 18 months MERYS Discretionary element in Increased transparency in spending € amount of mobilizing and distributing capital allocating capital expenditures is decisions and better aligned spending to discretionary investments through more transparency, eliminated. priorities more analysis of needs and priorities, decision making in and by developing a strategy for Allocation of budget financed budget allocations allowing HEIs to tap private credit places based on analysis and markets to finance capital projects. sector needs. Similarly, develop new methodology to help prioritize the allocation of budget places, relying on forecast of skills, and of students voting with their feet. 2.2 Improve policy Create demand for a more performance Critical >18 months MERYS Performance data (generated Budget discussions involve a discussion €€ orientation in the –oriented budget from the very top. This under 1.1 and 1.2) used in of what results the Higher Education budget by using set of activities relies heavily on the CNFIS budget discussed. sector delivers in terms of students data on activities under 1.1 and 1.2. Once learning outcomes (as measured by new performance in performance data start being generated, standardized tests) and in terms of budget discussion train MERYS staff to use analysis in graduation rates (as measured by tracer budget discussion. studies). 2.3 Improve policy Also, train MERYS staff in the usefulness Enabling <12 months MERYS A rigorous impact evaluation is Impact evaluation results are widely € orientation in the of using randomized controlled conducted of the scholarship discussed and impact policy discussion budget by building experiments to assess impact of policy program, and the new student on scholarship and student loan loan program is rolled out with capacity to under initiatives (e.g. the scholarship program program. an impact evaluation embedded when to use impact or the student loan program to be in the design. evaluations developed) 2.4 The need of a Develop medium –term strategy for High <12 months MERYS Tuition fees increased. Policy of Improved financial viability of higher € vision and a mobilization of more resource for higher selection of students in state education sector education. Increase the number of CNFIS financing quota revised. strategy for Scholarship policy revised. students paying, and tuition fees. mobilizing more Student loan program in place. resources 18 Objective Sequenced Actions Impact/ Implementa Responsibi Progress/Output Target/ Financial Priority tion Period lity Outcome Resource s 2.5 Implementing a To ensure that students have support Enabling <18 months MERYS First cohort of students to An impact evaluation will be used to €€€ student loan mechanisms in place when raising tuition receive support from a student document improvements (of recipients fees, introduce a student loan program loan program. of program compared to a control program (as mentioned in the new education law) group) in the following indicators: Drop out rates; Time to complete studies; Grades; Time it takes to find employment upon graduation 2.6 Improve the Create a single data base of budgets and High <6 months MERYS Single data base created and Data available for system monitoring € adequacy of the expenditures and make this data base access to data is available to the and informed policy making. reporting system to publicly available. Develop a “functional public. Core functions of each classification” for activities of the agency classified and better monitor agencies. Issue “performance contract “ performance contracts issued. with intermediate agencies. 3. Improving differentiation and harnessing the Power and Potential of Private HEIs 3.1 Improve the Conduct an in-depth study on the pros High >9 MERYS Study report, including a Higher levels of efficiency in the sector, € Ministry’s strategic and cons of having highly specialized proposed strategy, completed if consolidation of small specialized Potential universities and available universities with other specialized or €€€ stance with respect comprehensive universities is found to to differentiation be desirable and is implemented; same including the role with respect to numbers of study programs of private HEIs 3.2 Address Conduct an in-depth assessment of the High >9 MERYS Study report, including a Improved access to higher education in € excessive need for the expansion of higher proposed strategy, completed under-served areas and improved equity Potential education services in the currently and available €€ differentiation of under-served regions and develop a plan study programs to address this need. 19 Objective Sequenced Actions Impact/ Implementa Responsibi Progress/Output Target/ Financial Priority tion Period lity Outcome Resource s Conduct an in-depth review of the High >12 MERYS Study report, including a Competition between public and private € present status and performance of proposed strategy, is completed universities will push public ones to private higher education institutions in and available improve their services; will increase innovation; Romania in order to develop a Private universities will attract more 3.3Harness the comprehensive strategic framework for people who can pay, thus reducing the power of private the next steps financial burden of the Government HEIs Review the number of study programs by High >12 MERYS Number of programs closed and A higher level of efficiency €€ field of study, determine size and number consolidated efficiency criteria and provide incentives to universities to consolidate or close programs that do not meet these criteria Conduct a joint pilot/demonstration Enabling >24 MERYS Program in implementation The ‗best of breed‘ institutions serve as project with 3 to 5 private HEI to real models of excellence for the develop „best of breed‟, or models, of private higher education sector excellent private institutions 4. Expanding autonomy in exchange for results-focused accountability 4.1 Increase the Work closely with activities 1.1 and 1.2 Critical >18 months MERYS Universities participate in Patterns of application for admission by € accountability of to use results generated from tracer designing standardized students change in favour of the better studies and develop assessment tests to assessment tools of students‘ universities; pressure on universities to universities and competencies, and in tracer improve performance increases; measure students‟ competencies. match it with an studies. Ministry‘s ability to monitor and hold Highlight the importance of measuring universities accountable for specific increase in “performance” (or results) by including Future result: analysis of results results improves; gradual improvement autonomy assessment results (when they become becomes part of the performance follows. available) in the Ministry‟s annual contracts that universities have report. with MERYS. Make further expansion in university autonomy conditional on universities participating in tracer studies and in measuring students‟ learning outcomes (using standardized tests). 20 Objective Sequenced Actions Impact/ Implementa Responsibi Progress/Output Target/ Financial Priority tion Period lity Outcome Resource s Conduct a pilot granting autonomy in High >24 months MERYS Pilot in operations; first Increased satisfaction at management € selected areas of financial and human monitoring report levels, improved management and resources management to a sample of improved accountability universities (who are furthest ahead in terms of participating in tracer studies and in piloting standardized assessment tests to measure their students‟ competencies); monitor & evaluate results 4.2 Strengthen civil Under the auspices of the new University High >Base line MERYS First survey completed and € society ‘s voice in Ethics and Management Board, develop survey, 6 published; base line established. and conduct annual ethics survey and months Noticeable decrease in unethical holding university publish results. behaviour in the universities over time accountable € Provide financial support to civil society Competitive grants scheme watch dogs (on a competitive basis) Enabling 18 months MERYS established involved in holding universities accountable. Encourage more policy analysis and debates on the higher education sector 21 5. Strengthen the capacity of key intermediary Agencies 5.1 Improve the Engage management consultants to help Critical 18 months MERYS, Each of the three agencies will Each agency will produce its required € structures, these agencies improve all aspects of ARACIS have new vision and mission outputs fully in line with required operations and operation (accreditatio statements, improved standards/specifications, within n), organization structure, and reasonable, specified time and at lowest management of UEFISCDI medium term strategic plans and cost possible these agencies and (attestation, rolling short-term operational their ability to habilitation plans conduct large-scale and more), operations ACPART including the (qualification setting of s) qualification and the granting of accreditation, attestation and habilitation 5.2. Use the roll-out Work with Agency and the Ministry of High <18 months MERYS Performance contract in place New student loan scheme developed by €€ of the student-loan Public Finance to develop a performance with Student Loans and a professional and businesslike agency, scheme to pilot a contract between the Agency and Scholarship Agency free from political patronage, MERYS. The performance contract interference and working under a different way of establishes measurable performance performance contract with the Ministry managing agencies indicators (that can gradually be which clearly monitors performance developed from “activities” in year one indicators and provides Agency with of the contract to more output or even more managerial and financial freedom. outcome-oriented indicators in later years) in exchange for more managerial and financial freedom. 22 6. Harness the power of internationalization to help improve the quality of education and research and to mobilize resources 6.1 Facilitate Establish organizational unit in the enabling <24 MERYS Unit established and strategy in Over time greater scholarly and scholarly and Ministry to deal with internationalization place and first steps in scientific exchange results in improved scientific exchange in the context of its forthcoming implementation taken. teaching and research in Romanian Number of doctoral students of Romanian reorganization. Develop comprehensive universities; higher standards of supervised jointly by Romanian universities with strategy to enhance the exchange. Use doctoral degrees awarded and foreign professors increased European and other institutional and academic partnerships universities; particularly to improve doctoral education 6.2 Attract foreign Design x programs in English for enabling >24 HEIs, X study programs running with Campus life enriched with foreign €€ students and undergraduates from other countries. MERYS y number of students students; income generated through professors to Conduct well-planned, systematic attraction of foreign students exceeds marketing campaigns abroad. costs of providing the special courses; Romanian Romanian professors improve foreign universities language skills as well as teaching skills Widen and improve student services to enabling >24 Student services widened. Conditions for foreign students more help ease the orientation and attractive; more come accommodation of foreign students. 23 II. Sector Performance A. Context and Purpose of the Review 42. The functional review of the higher education sector in Romania is part of the broader functional review of the country‘s public administration carried out by the World Bank at the request of the Government of Romania and the European Commission. The request is based on a Memorandum of Understanding signed in June 2009, according to which the objective of the Review is to develop an action plan that the Government can use over the short and medium term to strengthen effectiveness and efficiency in the education sector. 43. The need for improved effectiveness and efficiency stems from recent changes in the world economy where technology has become the driver of growth and where knowledge, skills and competencies are key to its application. A skilled workforce today is critical to development and the competitiveness of nations depends on its quality and availability. This has made higher education critical to growth and the expansion of higher education, as well as an improvement in its relevance and quality, a top development priority. Many countries, therefore, are designing new visions for their higher education sectors and are devoting much attention both to planning in, and to the management of, these sectors. 44. The higher education sector in Romania has undergone major changes in the past decade, in terms of both policies and size. Recent policy changes include (i) shifts in the structure of most study programs as a result of Romania‘s participation in the Bologna process; (ii) a reform of higher education finance (in 1999) involving a move away from input-based funding toward enrollment-based funding; (iii) a substantial increase in overall funding in recent years; and (iv) since 2000, the introduction of provisions allowing state universities to admit fee-paying students, which provide a major new revenue source for university expansion and quality improvement. 45. Meanwhile the growth in enrollment has been impressive (total enrollment in state and private universities more than doubled between 2000 and 2008), but there is a concern that the quality of education has likely suffered. Although the dearth of data on learning and employment outcomes makes it difficult to draw firm conclusions about the quality or relevance of Romania‘s tertiary education, the indirect evidence is worrying. The rapid expansion of the sector has stretched available resources beyond the point of where they can be expected to provide quality education; it has brought in large numbers of new students, many of whom are below the baseline levels of basic competence proficiency; and it has seen a growth in the number of newly established (often private) institutions whose quality is unknown. 24 46. The following sections discuss in greater detail the recent trends in Romanian higher education and outline the key challenges for sector performance, viewed from a comparative perspective within the EU. 47. It is important to note here that performance trends in research which are an integral part of trends in higher education are not reviewed in this report since it was decided to separate the HE functional review from the R&D functional review. The decision was made by the Government decision for reasons that are unknown to the team. Nevertheless, the team has taken great care to ensure that the two reports complement each other. B. Key Trends in Romanian Higher Education 48. An important message of this review is that critical data gaps on students learning and graduates‘ employment outcome make it difficult to manage this sector. Most critically, there are no objective data on how much students learn during the course of their study. As such, it is impossible to take stock on where the Romanian higher education sector is in 2011, and provide guidance on how far it has improved (or regressed) in recent years. 49. This is not to say that data are not available. Data are available on how many students (out of a particular age cohort) enroll and how many of them graduate. Data are also available on their socio-economic characteristics (using household survey data) and what they study. All of these data are analyzed below. However, it is important to remember that the study of enrollments and number of graduates is, ultimately, the study of how many diplomas are issued, not the study of the skills and competencies the sector produces. a. Rapid sector expansion 50. Over the past decade, Romania‘s higher education sector has experienced tremendous growth. Between 2000 and 2008, total tertiary enrollment increased by 133%—a rate of growth six times higher than the EU average. In fact, at a time when the country‘s population was declining, Romania‘s higher education sector accounted for one-fifth of all tertiary growth in the EU. The sharpest increase in the number of students in Romanian universities occurred after 2004, while enrollment in the EU as a whole remained flat during this period (see Figure 1). 25 Figure 1: Tertiary Enrollment in Romania and the EU (a) Increase in Tertiary Enrollment, 2000-2008 (b) Gross Tertiary Enrollment Ratio, 2008 Source: Authors‘ calculations based on EdStats. 51. Private universities have led the way in the sector‘s unprecedented growth. Since 2004, 7 out of 10 new places in Romanian universities were added by private institutions; during this time, private-sector enrollment nearly tripled. By 2008, the private tertiary sector accounted for 39% of total enrollment in Romania—one of the highest private enrollment rates in the EU (see Figure 2). An important part of this growth in the private sector has involved distance and part- time learning: 41% of all students in private universities in 2010 were enrolled in part-time (35%) or distance-learning (6%) programs. By contrast, the proportion of students in non- traditional (part-time/evening/distance-learning) programs in state universities was 17%.5 Figure 2: Private Tertiary Enrollment in Romania and the EU (a) Share of Enrollment in Private Institutions, (b) Share of Enrollment in Private Institutions, 2000-2008 2008 Source: Authors‘ calculations based on EdStats. 5 Authors‘ calculations based on data from Romania‘s National Institute of Statistics. 26 52. As a result of the expanding access to higher education, the annual number of graduates produced by Romania‘s universities increased fourfold since 2000. By 2008, Romania had one of the highest gross tertiary graduation rates in the EU (see Figure 3). Starting from one of Europe‘s lowest tertiary enrollment rates only a decade earlier, Romania has in a few short years succeeded in enrolling nearly half of its high school graduates in university. With the EU2020 goal of having 40% of its young adults complete a tertiary degree in sight, Romania must now focus on examining the quality of the education that these graduates will receive. Figure 3: Tertiary Graduates in Romania and the EU (a) Increase in Tertiary Graduates, 2000-2008 (b) Gross Tertiary Graduation Ratio, 2008 Source: Authors‘ calculations based on EdStats. b. Shift toward ―soft‖ disciplines6 53. The sector‘s rapid growth has been led by programs in the social sciences, while engineering and science-related fields have seen shrinking demand from Romania‘s students in recent years. Since 2000, 63% of the new university places added in Romania have been in the fields of social science, business and law. 54. Though similar trends have been observed in many European countries, Romania currently has the highest proportion of graduates in the social sciences anywhere in the EU. In 2008, 58% of Romania‘s graduates received degrees in the social sciences, while 72% graduated with degrees in all ―soft‖ disciplines. By contrast, only 24% received diplomas in the ―hard‖ sciences. This dramatic tilt toward the social sciences by Romanian students is one of the most pronounced among all EU countries (see Figure 4). 6 For the purpose of this section, ―soft‖ disciplines include social science, business, law, education, humanities and arts; meanwhile, ―hard‖ disciplines include engineering, manufacturing, construction, science and health. 27 Figure 4: Distribution of Tertiary Graduates by Field in EU Countries, 2008 100 other fields 80 science and engineering 60 health and (%) education 40 humanities 20 and arts social science, 0 business, law "soft" fields Ireland Italy Austria Denmark France Hungary Spain Malta Finland Bulgaria Estonia Belgium Germany Netherlands Greece Portugal Cyprus Poland Slovenia Sweden Romania Latvia United Kingdom Lithuania Czech Republic Slovak Republic Note: ―Soft‖ fields include: social science, business, law, education, humanities and arts. ―Hard‖ fields include: engineering, manufacturing, construction, science and health. Source: Authors‘ calculations based on EdStats. 55. This shift toward ―soft‖ disciplines is closely related to the booming private sector, which has specialized in these fields. Because the state universities have struggled to meet the growing demand for degrees in the social sciences and related fields, and because such programs can be offered at a relatively lower cost than those in engineering and the exact sciences (which require more capital investments), 74% of programs offered by private institutions are i n the ―soft‖ disciplines. 56. In 2008, the gap between the share of graduates in the ―soft‖ fields and those in the ―hard‖ fields was 48 percentage points in Romania, second only to Latvia‘s 52 percentage points. This gap, however, has grown by 25 percentage points since 2000, by far the sharpest increase in the share of degrees granted in the social sciences and related fields anywhere in the EU (see Figure 5). This rapid scale-up of tertiary education in the ―soft‖ fields provided largely by new and private institutions sets Romania apart from most EU member countries and raises additional questions about the quality of education provided to Romanian students. 28 Figure 5: Gap between “Soft” and “Hard” Fields in Romania and the EU (a) Gap in Share of Tertiary Graduates, 2008 (b) Increase in Gap of Tertiary Graduates, 2000-2008 Note: ―Soft‖ fields include: social science, business, law, education, humanities and arts. ―Hard‖ fields include: engineering, manufacturing, construction, science and health. Source: Authors‘ calculations based on EdStats. c. Equity in higher education 57. As is the case in all countries, there is a strong socio-economic dimension to who enrolls and graduates from tertiary education: more than 50 percent of the richest quintile of Romania‘s youth (25-29 year olds in 2009) holds a tertiary degree, whereas less than 5 percent of the poorest quintile do (see Figure 6 and Annex Table 2). While the situation has improved somewhat between 2002 and 2009 (with rapid growth in the number of graduates from the poorest quintile), students from poorer (and rural) households continue to lag substantially behind their more well-off peers. As will be discussed in the section on financing, this is not a problem that has a straightforward solution. But the fact that higher education, an activity with very high private returns, is mainly an activity for the wealthier segments of the population does raise important questions regarding whether public money is being used equitably. 58. These structural inequalities are, in part, a function of the large proportion of the Romanian population (45 percent in 2005/06) living in rural areas, where heads of household, on average, 29 have completed fewer years of schooling, and have lower average incomes. In terms of preparedness for higher education, only 37 percent of 19-21 year olds coming from rural households have completed a high school degree, compared to 68 percent of their peers in urban households. This is partly the result of there being very few upper secondary schools in rural areas, and therefore, almost all rural students who go on to upper secondary education, need to go to urban schools and are prevented to do so by the prohibitively high living costs.7 Figure 6: Percent of 25-29 year olds who have completed a tertiary degree, by income quintile and by location, 2002 and 2009 Note: See Annex Table 2 for education attainment of this age group for all educational levels. Source: Authors‘ calculations based on Household Budget Survey, 2002 and 2009. 59. Additional concerns about equity in higher education focus on the inclusion of disadvantaged and vulnerable populations. In particular, concerns about Roma exclusion from formal education channels have led the Romanian government to adopt a set of policies aimed at expanding access to quality education for Roma students.8 Though exact estimates vary,9 less than 1 percent of 7 World Bank education note (2007). 8 See: http://www.anr.gov.ro/html/Politici_Publice.html. 9 See, for example:  Fleck, Gábor and Cosima Rughiniş, eds. (2008), Come Closer: Inclusion and Exclusion of Roma in Present-Day Romanian Society, p. 166. Available online at: http://books.google.com/books?id=ck_kFYKjeBkC &printsec=frontcover#v=twopage&q&f=true.  Open Society Foundation (2007), Roma Inclusion Barometer, pp. 74-75, 81-82. Available online at http://www.soros.ro/en/program.php?program=16.  Open Society Institute (2006), Monitoring Education for Roma 2006: A Statistical Baseline for Central, Eastern, and South Eastern Europe, p. 21. Available online at: http://www.soros.org/initiatives/roma/articles_publications/publicationations/monitoring_20061218. 30 Roma complete higher education in Romania—a rate up to 40 times lower than that of the general population (see Figure 7). Figure 7: Percent of 25-29 year olds who have completed a tertiary degree, by ethnicity, 2009 30 24.2 25 18.7 20 Percent 15 10 5 0.6 0 Romanian Hungarian Roma Source: Authors‘ calculations based on Household Budget Survey, 2009. 60. Despite the increase in affirmative action efforts of reserving several hundred university places for Roma students each year, the underlying issue is one of lacking qualifications among Roma students at lower educational levels. While less than 1 in 10 Roma children is able to complete high school, worries of exclusion in tertiary education ought to be addressed through expanded access and greater equity of opportunity at the primary and secondary levels. d. Supply and demand for graduates 61. The only jobs that were created (net) during the 2002-2009 period were ones that usually require a tertiary education: professionals, technicians, and managers. In fact, while total employment shrank by more than 500,000 jobs, employment opportunities for tertiary graduates grew by around 400,000 positions. This implies a decline in the number of people employed with anything less than a tertiary degree (see Figure 8). 31 62. Given this rapid growth in new jobs for graduates, virtually all graduates during this period found jobs: Figure 7 shows that the number of individuals in the population with a tertiary degree grew slightly more than the number of those employed. Moreover, most graduates found jobs in occupations traditionally reserved for tertiary graduates (professionals, etc). There is some evidence that a small but growing number of graduates have to resort to jobs as service workers (see Annex Figure 2). This could reflect either that the demand for tertiary graduates is slowing or that the skills that tertiary graduates are acquiring is too poor (or not suited for the higher-end jobs). Figure 8: Estimate of the change in supply and demand for graduates with different levels of education, 2002-2009 600 400 Changes in individuals (thousand) 200 0 Supply -200 Demand -400 -600 -800 -1,000 Upper secondary and Pre-primary, primary and lower Tertiary education post-secondary non-tertiary secondary education education Source: Authors‘ calculations based on Eurostat. e. Earnings premium and unemployment 63. Wages grew for all occupations during the period 2002-2009, but the growth especially fast in the occupations typically reserved for tertiary graduates. In fact, the earnings premium (i.e., the additional earnings that a graduate of tertiary education makes compared to lower levels of education) has risen between 2002 and 2009. After controlling for other characteristics of workers—such as age, level of experience, and geographic location—a graduate with a tertiary degree, on average, made 41 percent more than an upper secondary graduate in 2009, compared to a premium of just 33 percent in 2002. 32 64. Despite the recent economic crisis, unemployment levels for tertiary graduates have remained below those with upper secondary and lower levels of education (see Figure 9). The unemployment rate for tertiary graduates (of all ages) in Romania stood at 4.1 percent in 2009, while the rates for those with upper secondary education and those with lower levels were 7.3 and 8.9 percent, respectively. The same pattern was evident for young Romanians in their late 20s and 30s: those with university degrees were half as likely to find themselves unemployed as those who had lower secondary education or less. Figure 9: Unemployment rates by highest level of education attained 2002-2009 (%) Source: Authors‘ calculations based on Eurostat. C. Challenges for Sector Performance 65. As outlined above, Romania has made impressive gains in terms of expanding its system of higher education. The pace of growth, however, is slowing somewhat (in fact 2010 enrollments are, reportedly lower than 2008), providing an opportunity to shift more attention toward improving the quality and relevance of the education provided. As will be discussed in this Functional Review, there are no direct measures of quality (or whether graduates find jobs) but there are indirect signs that the higher education faces major challenges related to the quality and relevance provided. These signs include the large numbers of (often unqualified) entrants into higher education, doubts regarding the quality provided in the many new private institutions, and signs that Romania‘s graduates are less likely to gain work experience during studies, less likely to study in another country (as part of their study program) and less likely to master a foreign language. 33 a. Large number of (often unqualified) entrants overwhelm the system 66. In 2006, 53 percent of Romania's students failed to attain basic proficiency levels in reading and mathematics on the PISA international assessments of 15-year-olds, while nearly 90 percent scored below the OECD average. Due to very high transition rates from secondary to tertiary education in Romania, there is reason to believe that many students in this cohort entered university in 2009 lacking basic skills expected of university entrants in most OECD and EU countries. This hypothesis was further corroborated by a panel of university rectors convened for this study. In recent years, Romania‘s universities have struggled to absorb the sizable influx of entrants associated with Europe‘s highest tertiary entry rate (see Figure 10). Because the rapid growth in enrollment has not been accompanied by a corresponding expansion of the teaching force, Romania‘s student-faculty ratio has doubled since 2000. At 33 students per teacher, it is now by far the highest in the EU (see Figure 11). Figure 10: Gross Tertiary Entry Rate in EU Countries, 2007 (Tertiary Entrants as a Percentage of Qualifying Secondary Graduates in the Previous Year) Source: EQUNET (2010), ―Evolving diversity: An overview of equitable access to HE in Europe,‖ (p. 40). 34 Figure 11: Student-Faculty Ratio in Tertiary Education, Romania and the EU (a) Student-Faculty Ratio, 2000-2008 (b) Student-Faculty Ratio, 2008 Source: Authors‘ calculations based on EdStats. b. Little emphasis is placed on practical work experience 67. Romanian students have the lowest levels of pre-university work experience in the EU. Only 14% of Romanian students report having a job prior to entering higher education; this rate is half the EU average and one-quarter of the level reported by students in Sweden and Finland in a 2005-2008 survey. Once in university, only 36% of Romanians report having a job and just 15% of those who work say that their job is ―very closely related‖ to their studies—both rates are among the lowest in Europe. 68. One of the reasons for the Romanian students‘ low rates of labor market participation is their heavy study load: Romania leads the EU in the average time its students report spending on study-related activities with 41 hours per week. Not surprisingly, little time remains for pursuing outside employment, thus reducing the students‘ experience in university to a purely academic one. Little room remains for valuable hands-on learning that students can obtain by participating in the labor market.10 10 EUROSTUDENT (2008), ―Social and Economic Conditions of Student Life in Europe,‖ Synopsis of indicators final report: Eurostudent III 2005–2008 (pp. 45, 124-125). 35 c. Low levels of mobility hamper student preparation for EU labor market 69. In addition to low levels of labor market exposure, Romanian students rarely take advantage of their country‘s membership in the EU by studying abroad. Mobility rates are among the lowest in Europe: only 6% of Romanian students report having study-abroad experience and only 2% have been enrolled for credit at a foreign university. As a result, foreign language fluency is low by EU standards: just 11% of Romanian students are fluent in at least two foreign languages. By comparison, the corresponding fluency rates for Bulgaria, Latvia, and Slovenia are 15%, 31%, and 46%, respectively. And all of these students will be competing for the same jobs upon graduation. 11 70. To conclude, the challenges for Romanian higher education can be characterized as follows:  No information exists on the most important performance aspects of this sector: are students learning during the course of their study, and are graduates finding jobs?  Rapid expansion has left the sector with inadequate resources to meet rising demand and some institutions—particularly private ones—without adequate quality control.  Insufficient emphasis on student preparation through work experience and foreign study threatens to leave Romanian graduates behind their EU counterparts in the increasingly competitive regional labor market.  Students from poorer (and rural) households continue to lag substantially behind their more well-off peers when it becomes to enrollments. And likely (although the lack of data on students’ employment and learning outcomes makes it impossible to quantify) the inequities in outcomes may be even larger. 11 Ibid., pp. 132, 138, 144. 36 III. Performance Management 71. Ministers, parliamentarians and outside stakeholders need guidance on what ―results‖ or ―performance‖ mean within a sector. Without such guidance, their discussions on the sector will either not focus on data at all, or focus on the data which are available – which may or may not be what performance in the sector is about. And without such data, politicians more easily become capture for special interests seeking to influence policies or attract money in one particular direction. Only the Ministry of Education can provide this clarity and start measuring whether or not the sector performs. 72. Without clarity on what performance means, and indicators to measure progress, strengths and weaknesses, it is impossible to assess the performance of the sector (e.g. there is no discussion on ―results‖ in the Ministry‘s Annual Report on the Education Sector12), to strategize effectively (because the task of strategizing involves taking stock of where you are, and clearly defining where you want to go, both of which cannot be done without such clarity) 13, it is difficult to design policies (and mobilize resources) to remedy potential problems (because confusion will reign regarding what the problems and their magnitude are); and it is impossible to hold actors in the sector accountable for what really matters, namely results. In the absence of performance indicators, accountability relationship usually revolves around inputs and processes (usually done by excessively detailed norms and regulations) instead of devolving freedom to actors to manage inputs and processes in exchange for demonstrated results. 73. Policy makers are not the only actors that would benefit from better information on results. Employers, students, and parents also need objective, standardized data to make informed decisions. The market for tertiary education is a classic example of the seller (i.e., universities) being vastly better informed than their consumers (i.e., students, parents, and future employers), or even their financiers and donors (i.e., governments, for the most part). When students make choices about future universities and degrees, they usually do so without having met the professors who will teach them and with little to no understanding of the skills that they will acquire. This asymmetry places a great deal of power in the hands of the seller, which can contribute to worsening quality and higher costs for the consumer. In other parts of an economy characterized by similar informational asymmetries, policy makers usually play a strong regulatory role and/or mandate that providers provide better information to consumers. 12 The fact that performance indicators are missing in higher education is clearly understood by the authors of the annual education report. In this report, a glaring hole is left in the table summarizing ―results‖ in the higher education system. 13 For instance, the Pact on Education and the education strategy was clearly constrained in its ability to set targets on results. It set goals in equity in enrollments (at best an intermediate result) and on the opaque target of getting three Romanian universities in the top 500 universities in the world (a worthwhile goal but hardly a goal to use to steer a sector of more than 100 universities and almost 1 million students). No targets or goals were set on employability of graduates, or on the quality (as measured by standardized assessments) of students. 37 74. The good news is that Romania is leading Europe in terms of adding an important performance indicator on higher education, namely, ―are graduates finding jobs after they graduate?‖ The Romanian Tracer Study project – one of Romania‘s ―Strategic Projects‖ (financed with EU funds) will generate objective, standardized information on whether graduates (from different study fields or program, and different universities) find jobs, including how long it takes them to find these jobs, what type of jobs are they, and what are the starting salaries. And alongside these data, it will collect socio-economic and other background information about the graduates (e.g. parents‘ educational background, grades in high school etc) to allow analysts to take such differences into account when comparing employment outcomes. If correctly used, this type of information can help students and parents make more informed decisions; it can help university managers better design programs in demand; it can help employers in their search for high quality graduates and it can help policy makers identify particularly (un)successful programs. 75. Several aspects make this project particularly promising, including the fact that Romania is not only undertaking a once-off data collection exercise, but is going to do it regularly and is going to embed the tracer data as part of the information used to manage the system (e.g. the data is envisioned to play a role in accreditation decisions, in the distribution of study places, etc). Only one other new EU Member State (Hungary) is as advanced in this field as Romania and only a handful (at most) of EU‘s countries have as sophisticated systems (examples include the Netherlands and Italy). 76. Three additional important changes will help built solid foundations for creating more performance data from this sector: first, the new law establishes a new council (of Higher Education Statistics and Forecast) (see Article 195 (1)) whose main responsibilities are the development and continuous updating of higher education indicators for monitoring and forecasting its evolution in relation to labor market dynamics. The council will receive support from a new unit to be established within Executive Unit for Financing Higher Education and Scientific Research, Technological Development and Innovation (UEFISCDI) (which the law will convert into a National Council Secretariat).. Second, another one of the strategic projects financed by the EU is building a registrar for all higher education students. This will greatly facilitate the process of keeping track of students and cohorts as they move through and obtain credits in the higher education system (some of them moving between fields and across universities). Third, the National Higher Education Qualifications Registry project is in the process of defining the competencies that have to be acquired in every field of study, and within every study program, linking each of them to potential occupations. By the end of 2011 Romania will have 325 registered qualifications for almost all bachelor degree in 72 study fields. Later, the qualifications for 175 master level programs will be registered. This is an important first step in clarifying and reaching consensus on what competencies students (within a particular field of study) should have at the end of their study. 38 77. These initiatives are extremely important because, at the current juncture, the lack of performance data is a critical constraint for the ministry in effectively managing the sector: it prevents it from strategizing effectively, mobilizing resources to problem areas; reporting to stakeholders on the success and failures of the sector; and holding actors in the sector accountable for results. This section discusses in more details what is missing, and proposes ideas for how to address the gaps. Not surprisingly, the first recommendation is to continue in the direction Romania is already heading, with its tracer study project, its student registrar project, and its national qualification project. 78. The argument for introducing performance indicators (made in this Review) is not that Romania is behind the rest of the world; rather, this Review‘s argument is that such data are critical to better manage the sector. Romania‘s higher education sector is not alone by not having a clear definition of what performance means and the tools to measure it. In fact, most countries in the world are struggling on this front: for a number of reasons (see Box 1 and Carey 2010 for some of the reasons behind this gap14) no country is systematically measuring what their tertiary students are learning, and very few countries are systematically tracking whether tertiary graduates are finding jobs. Box 1: The Difficulty of Measuring Competencies at the Tertiary Level It is perhaps understandable that higher education institutions around the world have been reluctant to let governments interfere in designing tests and measuring competencies of their students, arguing that such attempts would interfere with academic freedom. Any attempt to introduce such tests should accordingly be carried out in cooperation with higher education institutions. Moreover, it is critically important to draw a clear distinction between examinations (that have the purpose of judging whether a student should receive a degree or not) vs. assessments (that have the purpose of assessing the quality of learning). In fact, there are relatively few examples of such testing worldwide. In the United States, for example, the Collegiate Learning Assessment (CLA) was developed in the 2000s by a subsidiary of the RAND Corporation, but is currently used only by roughly 400 institutions. Recognizing that students choose very different academic specialties in college, ―the CLA tests the higher-order thinking skills that all college graduates should possess: critical thinking, analytic reasoning, and communication. The exam is given to a sample of freshmen and seniors to estimate how much students learn in college‖ (Carey 2010, 16–17). Similarly, since 1949, the Graduate Record Examination (GRE) has been used as an admission requirement by numerous graduate programs in the United States. It is a standardized test that seeks to measure verbal and quantitative reasoning, analytical writing, and critical thinking skills. The OECD is currently preparing an assessment of tertiary students called the ―Assessment of Higher Education Learning Outcomes,‖ or AHELO, a tool that will ―assess learning outcomes on an international 14 Carey, Kevin. 2010. ―That Old College Lie: Are Our Colleges Teaching Students Well? No. But Here‘s How to Make Them.‖ Democracy 15 (Winter): 8–20. 39 scale by creating measures that would be valid for all cultures and languages‖a However, the OECD does not expect a full-scale AHELO to be launched before 2016. Source: Carey 2010; and authors‘ research. Note: a. See the AHELO Web site for more details: www.oecd.org/edu/ahelo (accessed September 2010) 79. It is the assessment of this Review that Romania is already on track for being a world leader in the field of tracking performance of its higher education (by virtue of its rapid progress on developing a Graduate Tracer Study) and it could develop this leadership further. The rest of Europe (and Central Asia) will soon be flocking to Romania to learn the details of how to design tracer studies and, more importantly, institutionalize such surveys (as opposed to just having them on an ad hoc basis) and embed the information generated into the overall quality assurance and accountability relationships in the sector. In addition, Romania is also making good progress on defining what skills and competencies graduates from each field of study should have (as part of the development of its national qualifications framework). The authors of this review have every reason to suspect that with Ministerial leadership (and EU funds) Romania could become a leader in designing assessment tools to start measuring whether its graduates are acquiring the desired competencies. a. What performance information is missing? 80. To assess performance in higher education, several dimensions need to be assessed but the most important ones are: are students acquiring the right competencies during the course of their study; are graduates finding jobs; and is quality research being produced?15 And, as part of assessing whether graduates are finding jobs and whether students are acquiring knowledge, skills and competencies during the course of their study, it is important to assess the access and equity dimension of these performance criteria: was is the performance gap vis-à-vis the socio- economically weak (and ethnic minority) students? And, in terms of enrollments, what parts of the population is not being enrolled? Of the three dimensions, data are only being collected on faculty research output. As the Functional Review on Research and Innovation discusses, the data show that, on this dimension, performance is improving (i.e. more research is being produced, although less is known about whether this is quality research). 15 A word of caution: by including whether graduates find jobs as a performance criteria, the Review is not proposing having a target ―employment rate‖ that universities would be measured again. This would be ill -advised since labor market conditions vary across sectors, geographical areas and across time. And graduates from some fields of study (e.g. humanities) have a narrower job market than others (e.g. business students). Thus, one would need to exercise care in defining performance around the notion of whether graduates find jobs. For instance, one might look at the employment performance of graduates (with similar socio-economic backgrounds) from a similar field of study entering the labor market the same year but from different universities. 40 Figure 12: Three core performance dimensions of a higher education system 81. As discussed above, Romania is making rapid progress in adding a performance indicator on whether graduates are finding jobs. So far, only ad hoc studies of graduates are available (e.g. a survey of graduates were conducted by ARACIS in 2010) and, of course, labor force surveys (LFS) which can be used to gauge the aggregate situation regarding whether or not tertiary graduates find jobs (as was done in the introduction to this Functional Review). That is, LFS can be used to look at the type of jobs tertiary graduates (within a particular age group) take (usually by broad occupational fields) and what their unemployment rates are. But LFS cannot provide any information on are the details: how many of the graduates that graduated last fall found jobs within 12 months? How many found full-time and permanent jobs? And what, exactly, did the graduate study and at which institution? These questions will only be answered once the Tracer Study results start being analyzed. 82. The most important performance data which is missing completely from the discussion in Romania‘s HE system is whether students learn during the course of their studies. That is, on average, are they acquiring competencies (and skills and knowledge)? And, where in the system – e.g. by fields of study, by type of university – are students on a particular steep learning curve compared to other places? In part, such information exists for certain professions where the profession itself has developed ―professional exams‖ to certify that graduates are qualified. Such professional exam exists for lawyers, for teachers, for medical doctors and for architects but not for the majority of what Romania‘s students study, namely, economics, business, political science, or the humanities (except if the graduate decides to become a teacher). 83. However, even these ―professional exams‖ have their limitations in terms of the performance data they generate: they only generate for graduates who decide to take the exam; and they only provide a pass-fail score, not an assessment of the mastery each graduate has vis-à-vis particular desired competencies (e.g. in the form of a score on a scale from 1-100). And, alongside the exam, no socio-economic and other background information about the students are collected, 41 making it impossible for a researcher to parse out whether a graduate‘s performance on the exam is due to the value-added of the university, or whether the student‘s background characteristics (e.g. parental education background, high school graduation marks etc). As such, even for the 39 faculties of law (22 of which are located in private universities), very little is known about how much their graduates‘ competencies develop during the course of their studies. 84. Take the following concrete example of how this gap in performance data limit the ability of key actors in taking informed decisions and improving the quality of the sector: there are 41 HE institutions teaching finance in Romania: 22 private and 19 state:  Student‘s perspective: which institution does a better job at teaching me the skills I need to succeed? Without such information, the powerful force of students ―voting with the feet‖ is significantly limited.  Employer‘s perspective: which graduates outside of the prestigious institutions (which may already have found jobs) have acquired mastery of skills and competencies that my firm needs?  Higher education institution‘s (teaching finance) perspective: how does my program compare to other programs? Are my students acquiring as solid mastery in the core competencies in this field as other institutions? If so, how do I demonstrate that to prospective students, and potential employers of my graduates?  Policy maker‘s perspective: how does the performance (in terms of quality and relevance of the education provided) compare with other fields? Which institutions are doing a good teaching imparting skills, and which are not? Which institutions need more support/guidance? 85. The lack of performance data also make it impossible to assess the efficiency of spending in the higher education sector since an efficiency analysis is a comparison of spending (for which data exists) and some meaningful and desired output or outcome (for which no data exists). Certainly it is possible to calculate the cost of enrolling or graduating a student (and compare this ―unit cost‖ between, say, state and private institutions (something which is done in the section on financing in this review) but without a measure of the quality of the education provided (or whether graduates find jobs), it is easy to draw exactly the wrong conclusions about ―efficiency‖ by looking at such measures. For instance, as the section on financing will show, the average unit cost of educating a student in the private sector is significantly lower than in the state sector (something which is likely true even if one compared the unit cost educating a student in similar fields of study). However, the cost of graduating a student is not the same as the cost of graduating a student with a desired level of competencies and/or graduating a student who finds a job. It is the latter two measures that give rise to meaningful efficiency analysis but, again, the performance data to allow for such analysis are missing. 42 86. Developing and using performance data that measure results (as described above) will take time but there are more intermediate results that the Ministry could start collecting. For instance, the Ministry could start an annual survey of students, focusing such survey on measuring their ―engagement‖ (while collecting socio-economic background information to allow researchers to compare answers from students with similar characteristics). Moreover, given the extent of unethical behavior reported in the Romanian higher education system, the Ministry could introduce an annual survey (of both students and faculty) to measure the various dimensions of unethical behavior. A good starting point for such survey is the work done by the Coalition for Clean Universities (see box 4). 43 Box 2: How would performance information about students’ skills look and what questions could such information help answer? Standardized tests are the best available indicator for measuring student performance, especially when socioeconomic characteristics are collected for each student (allowing the results of different socioeconomic groups to be compared). Such data are especially useful if organized to track the progress of individual students over time. These tests provide data on differences in learning outcomes between weaker and stronger students and institutions. Without this information, policy makers have no way of identifying—and thereby rewarding—institutions that do the best job of giving the largest number of students a quality education. Imagine that a sample of students in both state and private universities had taken test in March 2011 to measure a number of broad competencies that are usually assumed to be sharpened during the course of a tertiary education. Specifically, imagine that the test was designed to measure the core competencies that economics graduates should acquire. That is, the competencies being tested are the ones developed by specialists within the particular field (as part of the National Qualification Framework). Moreover, the assessment tools would be tools developed and defined by sector experts. In this hypothetical example, suppose these competencies are the broad competencies that most tertiary graduates are assumed to sharpen during the course of their studies: ―analytical reasoning skills‖, ―crit ical thinking skills‖ and ―communication skills‖. Below is an example of what such results might reveal and how they could be used to reveal particular weaknesses which, in turn, could be targeted with policies. Box figure 1: Gaps between state and private universities teaching economics, marketing, business and related fields (of which there are 260 faculties, 134 state and 126 private) 50 on standardized assessment Gaps in skills as measured 45 40 35 30 25 20 Analytical reasoning skills 15 Critical thinking skills 10 Communication skills 5 0 No Yes Controlling for differences in student population? The hypothetical results show that: Significant gaps exist between state and private universities when it comes to students‘ analytical reasoning skills. For the sake of argument, suppose that the numerical difference shown in the graph amounting to as much as one academic year of study (ie a fourth year private university student performs as well a third year student in a state university). In part, the graph shows that this gap is due to differences in the underlying characteristics of the student population in state vs. private universities. For instance, it might be the case that state university students tend to have better high school graduation exams, more educated and socio- 44 economically more well-off parents. However, the graph shows that even when such differences are taking into account, the gap in analytical reasoning skills persists. Importantly, the gaps with regard to critical and communication skills are much smaller, suggesting that institutions should focus their attention on the analytical reasoning skills. If such assessments were conducted across several institutions and across time, each institution would have additional dimensions to benchmark its performance: how well does its students compare with students in institutions with which it compares itself (again, after controlling for differences in student characteristics); and how has its performance changed over time? Undoubtedly, such information could help institutions better market themselves (including to international students) and help them design better programs. As Annex 1 outlines, several countries around the world have started to rigorously assess the competencies, knowledge and skills of their tertiary students (and improvements in these competencies during the course of their study and over time). Source: Authors Recommendation: 87. The principles behind the recommendations presented below are as follows:  Build on the work which has already been begun, making sure that two year action plans are in place to ensure each of these efforts become game changing forces for improving the quality and relevance of higher education (and not peter out when EU money runs out, or new ministerial leadership takes over). Many of the potential game changers are still in their infancy in terms of the potential; they still have to demonstrate their usefulness and ability to continue to mobilize financing.  Introduce the performance data (initially) with the main purpose of ―assessing‖ the system and providing more information to institutions and students, not as a national graduation exam, or a sanctioning tool. Given how many practical issues have to be worked on in developing the instruments to measure performance (and understand the results), it would be ill-advised to rush to using such information to punish/reward universities (budget-wise or otherwise). Moreover, the main point of conducting this test is to assess the system, not introducing a new examination (to determine whether a particular student should be allowed to pass or fail). However, as argued throughout this Review, as the quality, reliability and credibility of the performance information improves, the information can gradually be used as a core element in the accountability relationship.  Introducing more performance data is controversial (because it sheds light on the quality of what goes on inside the lecture halls), requiring the Ministry to delicately steer this 45 process. Just as the Ministry successfully has done in getting universities to buy into the Tracer Study project, the Ministry should use its softer tools to achieve its goal, including the power of persuasion, and through voluntary participation.  Providing incentives for participating, of which a very large range of options (many of which are underutilized by the Ministry): o through financial incentives by incorporate participation as a quality indicator; o by only providing capital expenditures to universities participating in effort; o by changing eligibility criteria for applying to EU funds (requiring participation in the effort); or o (eventually) only hiring graduates in the state sector who have attending a study program which are being assessed using standardized assessment tools.  Improving performance information is not just about developing more data; it is about expanding the access to such data, and the use of such data. Thus, making improvements on this front involve making improvements on ―creating data‖, ―expanding access‖, and ―improving usage‖.  Recognizing that better data alone does not imply better management of the sector; to make a difference data need to be analyzed, shared and used by policy makers and other actors in the sector. The reason this review places much emphasis on the ―knowledge gap‖ is that at the current juncture, the lack of data on performance is a roadblock for moving ahead with a range of different other initiatives, including making the budget more policy-oriented (see discussion in financing section); shifting accountability toward performance and, in exchange, providing universities with more autonomy to manage their day-to-day operations; and, perhaps most importantly, for students to become a more powerful force in demanding performance from this sector, they need data showing which universities impart knowledge and skills, and which universities excel in placing their graduates in jobs. Policy recommendation: Making full use of tracer study results as they become available (1) Help institutions use results to get feedback from the labor market. Such information can be used to: a. Help institutions design better study programs, including better aligning their offerings with demand; b. Help improve the marketing of their programs (vis-à-vis students) (including with regard to international students); and c. Be used as an engagement tools to strengthen their alumni network. 46 (2) Make students voting with their feet a more powerful force for improving the quality of education a. Make stately available easy-to-read information about prospective earnings, employment rates, types of jobs typically acquired upon graduation, and job satisfaction rates. b. Organize regional ―future job‖ fairs for high school students c. Distribute pamphlets with easy-to-read to last year high school students (3) Embed information in system-wide quality assurance a. Use information to differentiate between universities and programs of study b. Use information in accreditation decisions c. Use it to prioritize state spending, e.g. by identifying priority fields of study that should be supported by state resources. (4) Use launch of tracer study results as a vehicle to bring employers, education institutions and policy makers around the table to discuss skills needs. a. Involve sector councils in discussion preliminary results, get them involved in questionnaire design and analysis b. Host an annual conference to present findings (of tracer study) and, every three years, a ―skills forecast‖ (e.g. similar to what CEDEFOP is doing on an EU-wide basis).16 Policy recommendation: Developing the tools to assess whether students acquire the desired competencies during the course of their study  Build on top of what has already been achieved in the context of the National Qualifications Framework project: university representatives and sector specialists have already defined competencies for all bachelor programs. Have the same specialists work with experts in developing standardized assessment tools, pilot them, host conference to discuss findings, and further refine instruments. In terms of the big picture, the four steps involved in designing and generating information on students‘ competencies are described in Figure 13.  Start where there is least resistance. Conduct preliminary discussions with universities and sector experts to pick a field of study where there is strong interest from universities in getting started. For this to succeed, champions within universities who can help cajole 16 See http://www.cedefop.europa.eu/EN/about-cedefop/projects/forecasting-skill-demand-and-supply/index.aspx for more details. 47 others to participate have to be found. The more universities participate, the higher the value will be to students – and to outsiders wanted to gauge performance.  Recognize that successful will take years but that the important thing is to start a process that will gradually lead to the development of assessment tools. o Define/refine competencies that graduates should master o Develop/refine assessment tools o Test students (collecting background information about the students to allow researchers to compare performance of similar students) o Analyze and disseminate results to students; institutions; employers and policy makers  Help students become better at voting with the feet  Help institutions design better programs  Discuss findings with employers o Compare findings with results from tracer studies: does it look like the competencies defined are the ―important ones‖ in terms of who finds jobs? Is the sector producing graduates within the right fields of study? o Start with one field of study (and universities interested and willing to participate), and gradually expand to include more universities and more fields of study (working with sector specialists within each field of study to develop their own assessment tools).  Learn from the international examples that have started this process, including the Collegiate Learning Assessment in the United States, the Graduate Skills Assessment in Australia, the Exame Nacional de Cursos in Brazil, or the Exámenes Generales para el Egreso del Técnico Superior Universitario in Mexico (see Annex 1 for a list of international examples in assessing students‘ skills and competencies, including a description of what agency is behind the test, how the test is being used, and what incentives universities and students have to participate)  Sign up to participate in AHELO and make active participation in all OECD working group meetings a high priority. During these meetings, the world‘s foremost experts on the field brainstorm and discuss pitfalls. And by participating in AHELO, substantial technical expertise in administering and analyzing such test results would be transferred to Romania.  Sign up to participate to Programme for International Assessment of Adult Competencies (PIAAC). (for the same reasons as above) 48  Emphasize that this test is an assessment, not an examination. The primary point is to gather information (to be used both by students and universities – and by the Ministry to assess performance) on the quality of the teaching and learning which takes places in the Romanian HE sector; the purpose is not to institute a national ―graduation‖ exam. Policy recommendation: Measure intermediate results: are students engaged and is the problem of unethical behavior being reduced?  Conduct annual survey of students  Finance civil society watchdogs to annual track (and report on) unethical behavior in higher education. 88. As more data start being generated, it is important to emphasize that all analysis does not have to be carried out within the Ministry and its agencies but the Ministry has to demand the analysis and use it. In fact, it is unrealistic to suspect the current staff of the Ministry to become research-oriented, working with large data sets, using statistical methods to look at correlations, and writing analytical papers. They do not have to for the sector to become more evidence-based. Rather, for this to happen, this Ministry need to coordinate such research, by demanding more analysis, by mobilizing resources for it, by organizing conferences and workshops, and, most importantly, by making available (proactively and aggressively, not only by responding to requests for information) entire datasets. Figure 13: Four steps in creating information about what competencies students acquire Have experts and university community develop standards: what competencies should graduates have? Develop assessment tools to measure whether graduates acquire these competencies Collect data and analyze results. Collect socio- economic background data on students to enable you to control for differences in student population. Use assessment results to track performance of the HE sector, identifying strengths and weaknesses. Source: Authors‘ conceptual framewor 49 IV. The Structure of the Higher Education Sector 89. The term ‗structure‘ refers to the way the sector is set up - the configuration of inter-related organizations, the roles and functions carried out by each and the relationships among them. In focusing on the configuration of organizations the objective is to determine whether they differ by type and why there is a need for each type. In focusing on roles and functions, the objective is to see if any key functions are missing, overlapping, unclear or sub-optimally located in the structure and, in focusing on the relationships among them the objective is to identify interdependencies and assess the need for coordination. Finally, in the higher education sector ‗structure‘ refers also, and importantly, to the level of differentiation among higher education institutions – to the extent to which these institutions are divided by roles and functions. Differentiation is a key issue in higher education today, and much attention will be paid to it in this chapter. A. The Configuration of the Higher Education Sector 90. The sector consists of the Ministry of Education, Research, Youth and Sports, intermediary organizations and higher education institutions. The Ministry is the lead organization in the sector – the organization that controls it. In the Ministry there is one General Directorate for Higher Education headed by a Director General, and (usually) a Secretary of State for Higher Education. The intermediary organizations are distinct organizations set up by law as individual legal entities, separate from but subordinated to the Ministry. There are 12 such organizations, divided into two major sub-types: ‗specialized bodies‘ bearing the names ‗council‘ and ‗board‘, and ‗agencies‘ bearing the names ‗agency, ‗authority‘ and ‗executive unit‘. There are nine (9) specialized bodies, of which two are new, having been promulgated by the new Education Law and are yet to be established. Their roles are predominantly consultative: to provide regulatory, policy and strategy advice, recommend the allocation of financial resources, and intermediate between the Ministry and the higher education institutions in cases where an independent third party may help resolve potential conflict. There are three (3) agencies and their roles are not advisory roles; rather, they are regulatory implementation roles. They are to carry out specific on-going functions and tasks on behalf of the Ministry. Agencies are further sub-divided by name and by legal status into ‗agencies‘, ‗authorities‘ and ‗executive unit‘. 91. The list of specialized bodies includes the following: - National Council for Attestation of University Titles, Diplomas and Educational Certificates (CNATDCU), 50 - National Council of Scientific Research (CNCS), - Advisory Board for Research, Development and Innovation (CCCDI), - National Council of Higher Education Funding (CNFIS), - National Council of University Libraries (CNBU), - National Council of Ethics for Research, Technological Development and Innovation (CNECSDTI). - National Council of Statistics and Forecasting of Higher Education (CNSPIS),[New] - University Ethics and Management Board (CEMU) [New]; and - The National Council for Financing Higher Education, Scientific Research, Technological development and Innovation (UEFISCDI). 92. The list of agencies includes: - The Romanian Agency for Quality Assurance in Higher Education (ARACIS) - University Qualifications Agency and Partnership with Economic and Social Environment (ACPART) - Student loan and scholarship agency 93. There are 108 higher education institutions which may be divided into two main groups: state universities and private universities. 94. Altogether, there are in the sector six types of organizations by legal status, but only four by type of function: the controlling body, i.e., the Ministry, the specialized regulatory/advisory bodies, the regulatory implementation agencies and the universities. One of the questions faced by the review team concerned the location of key functions and the number of organizations that are responsibile for them. The question was not whether or not there is a need for these key functions, but whether or not it is necessary to carry them out (a) outside the Ministry and (b) in so many organizations. The team concluded that, from this point of view, the structure of the sector makes sense. With respect to (a), in the current civil/state sector environment in Romania, functions placed outside a Ministry, but subject to its control, are carried out more effectively than inside the Ministry. This was already indicated in the team‘s earlier report on pre-university education in Romania. 95. With respect to question (b) above, the team believes that the number of councils and boards is indeed large. There are two pittfalls in having so many organizations: administrative overhead costs are multiplied and coordination is rendered too complex. To the credit of the Ministry, the first pittfall is already being effectively addressed: the Ministry moved the administrative and logistical oeprations of most councils and boards to the National Council for Higher Education Secretariat which is now managing all these operations. The Unit is not yet optimally organized 51 to carry out the operations of so many councils and boards, and the team recommends providing it technical assistance to develop an optimal and efficient structure, but the arrangement is promising. Once an effective organization structure is in place for this Secretariat, the first pittfall will be fully addressed. 96. The second pittfall has not been addressed yet. It is well known in management that the larger the number of units of anything that need to be coordinated, the more complex is the coordination among them. Together with the Ministry itself, 12 organizations are involved in guiding, directing, supporting and controlling the 108 universities. They need to be well coordinated. Coordination requires authority: the coordinating organization must have authority over the coordinated ones, or else its effectiveness will depend on good will. The Ministry is the only organization among the 12 to whom all the other 11 report. Therefore, coordination is one function that cannot be moved outside. There is a difference between strategic coordination and operational coordination. The optimal place for strategic coordination in the Ministry is in the offices of the Minister and the Secretary of State for Higher Education. This position is not filled today. The optimal place for operational coordination is in the Directorate General for Higher Education. In the absence of a Secretary of State for Higher Education, the Directorate General could carry some of the burden of strategic coordination too, but the capacitry of this Directorate is very weak. This matter will be discussed in detail later on in Chapter IV; still, the team is of the opinion that the capacity of the Directorate General is a matter of high importance and recommends dealing with it urgently. 97. Focusing on roles and functions now, the review team has concluded that all of the roles and functions that need to exist in a higher education sector exist in Romania and are covered by the 12 organizations referred to above. There are no functional gaps in the Romanian higher education sector. Equally, with one exception, there seem to be no functional overlaps among the many organizations. Each has a distinct role to play and the only potential overlap may evolve between the two councils that deal with ethics: the one that deals with ethics of research and the other that deals with the ethics of university management. The potential for overlap exists because universities play a major role in research. The National Council of Ethics for Research, Technological Development and Innovation (CNECSDTI) has the role of advising the Minister of Education on matters related to the moral and professional standards of research institutions and researchers in Romania. It addresses ethical and legal issues that arise in the course of scientific research, promotes the development of codes of conduct for researchers and serves as an appeals board for researchers who disagree with the decisions of institutional ethics committees. The exact role of the University Ethics and Management Board (CEMU) according to the new law is to serve as a last resort in resolving academics ethics disputes. According to Article 195(3) in the new law its main duties are ―(a) monitoring the achievement of academic ethics at the higher education system‘s level‖ and ―(b) auditing ethics committees of universities and presenting an annual report on academic ethics.‖ 52 B. The Level of Differentiation among Institutions in the Higher Education Sector 98. Higher education systems diversify in response to several forces in society. Chief among them are (a) an increased quantitative demand for such education which brings to the universities a growing number of students, (b) an on-going process of specialization in professions and occupations which is resulting in differentiation in the types of knowledge and skills needed and taught, and now also (c) the revolution in information technologies which has opened the door to new and different modes of instructional delivery. The growing number of student is coming with increased heterogeneity in socio-economic and cultural backgrounds and therefore increasingly different needs as well as aspirations. The on-going, and in some cases rapid, process of specialization comes with an increase in new fields of study as well as significant changes in the level of knowledge and skills required in existing fields. Finally, the advent of new modes of delivery is greatly influencing access and is further intensifying the increased demand. These forces combine and interact in many ways, creating different client populations for higher education and dividing the higher education market into a growing number of market segments and niches. 99. From a national, societal point of view there is a need to cover all of the existing and emerging client groups. Two criteria by which higher education systems are measured are therefore (a) how complete is the coverage and (b) how good it is. Effective higher education systems manage to respond to these market conditions through a process of differentiation in what they offer academically, and how they deliver their offered programs organizationally. New higher education institutions are created to absorb the growing demand; new programs and courses are created to cover emerging areas of knowledge, and new occupations and professions; and new modes of delivery are introduced to take advantage of information technologies. 100. The most important forms of differentiation are by: 1. Key line of business (i.e., teaching and research) 2. Fields of Study (i.e., academic discipline) 3. Geographic area (i.e., regions in the country) 4. Clientele (e.g., high-school graduates, working adults, people who can or cannot pay) 5. Ownership (state, public, private - for gain and not for gain); and 6. Modes of delivery (i.e., full-time residential, part-time residential, virtual/distance delivery) 101. Basically, to have an effective higher education system, there needs to be an alignment between the demand for and supply of: 1. Graduates in each field and level of study at the national level 53 2. Study places in various fields of study at regional levels 3. Study places in state, public and private institutions 4. Tailor-made services to different client groups where content, delivery methods and student support services (including payment arrangements) are adjusted to fit their needs 102. It is important to note, however, that no single institution can serve the entire market for higher education services and deliver all of the services to all of the client groups. The larger the number of clients and the larger the number of services, the more difficult it is to maintain high standards in all services, or to serve equally well all client groups. The larger the number, the higher the likelihood that some clients are neglected and some services suffer. Yet, many higher education institutions that may have started with a well-focused mission to serve a reasonable number of clients in a reasonable number of fields, and in a reasonable number of ways are now exhibiting mission-creep and diversifying into a growing number of fields, clients and modes of delivery. 103. Two points need to be made at this juncture: firstly, all higher education systems struggle to create a system that flexibly creates new supply in the face of changing demand but the closer a system of higher education is to such an alignment and the more flexibly it can respond, the more effective it can be. Secondly, in countries where the level of inertia among higher education institutions is high, where these institutions do not respond to the different demands and opportunities that exist in the market fast enough, it is one of the strategic roles of ministries of education to provide incentives that will speed up and improve the alignment between demand and supply. As will be seen below, Romania is one of these countries. The Ministry has started addressing the issue of differentiation with the introduction of a mechanism to distinguish between research –intensive, teaching and research and predominantly-teaching universities. More can be done in the area of differentiation and this is discussed below. 104. When thinking about a more differentiated higher education system, it is also important to consider expanding the role of short-cycle programs (i.e. what is internationally classified as ISCED 5B). The development of more short – 2-3 years – occupationally oriented programs has played a key role in creating a more flexible (and labor-market responsive) tertiary sector in many other countries. One striking example is Finland, where the creation of tertiary-B institutions contributed to the doubling of higher education enrolments in 10 years (1990-2000) (OECD, 2006). a. The differentiation among higher education institutions by main line of business 54 105. It is necessary to note here that Romania has a whole sector outside the higher education sector whose main line of business is research. This is the Romanian Academy which conducts research in a large number of areas in the Sciences, Arts and Letters. These include, for example, Philology and Literature, History and Archeology, Mathematics, Physics and Engineering, Agriculture and Forestry, Medical Sciences, Economics, Law, Sociology and Psychology. The Romanian Academy is not included in this review, since there is a separate review dedicated to research and innovation. 106. On paper, universities in Romania do not differentiate themselves by line of business. All claim to be in the business of teaching as well as research. In practice, most universities engage almost exclusively in teaching and only a few conduct significant research activities. The review team analyzed the mission statements of a sample of universities, consisting of 21 state universities and 11 private ones. All 32 universities state that their professors are engaged in both teaching and research. The team assumes that most or all other mission statements as well assert that they are engaged in teaching and research as two main business lines. The evidence does not support this assertion. The total number of research papers published by Romanian authors in 2009 was 11,190. There were108 universities in the country at that time with 624 faculties and 31 103 teaching staff. Whether or not the total of 11,190 includes all papers published by the Academy, the average number of publications per university, per faculty and per teacher was very low Close to 91% of all these research papers were in the domains of engineering, exact sciences, mathematics and information technologies. The remaining 9% were spread over the domains of natural sciences, human sciences, social and political sciences, economic sciences, legal sciences, agriculture, theology, architecture and arts. 107. The Ministry has recognized this situation and is addressing it. Its strategy is to promote a more efficient division of labor among universities and encourage them to focus on main lines of business, so that they can deliver better services in these main lines. The strategy was introduced into the recently passed education law and higher education institutions will now be divided into three groups – research intensive, teaching and research and teaching-intensive universities. Funding will be adjusted to better meet the requirement of each type. Universities will be able to move from one group to the other, but such moves will require much more than the expression of aspirations in mission statements. The review team believes that this is indeed a significant step in the right direction. It will help the Ministry bring about a gradual change from a system where research is done predominantly in dedicated research institutes into a system where universities play an equally important role in research. It will also help align the supply of research services by Romanian universities with the need for such services in society and the economy. But the team is also concerned by the risk that this classification may lead to the creation of second class and third class institutions both in the eyes of the public and in reality. The team therefore recommends that the Ministry explain and demonstrate by words and by deeds that the classification is meant to be a functional classification, that it is designed to ensure that each 55 university focus on a main line of business rather than try to do everything and become a jack of all trades and master of none. The team recommends further that the Ministry encourage universities in the second and third categories to become ‗best of class‘ each in its own category, both through the financing formula and by rating them within their categories. b. The differentiation among higher education institutions by fields of study 108. The higher education system in Romania seems to be too differentiated, particularly at the Bachelor level, relative to the need and demand at this level. Two phenomena have greatly influenced demand for graduates at the Bachelor level – and to a lesser degree at the Master level - in the economy: the first a recognition that students of engineering, technology, exact sciences and other non-humanities and non-social-science need to understand better the broader contexts within which they will operate once out of school. For this to happen, it is necessary to provide them structured opportunities to understand topics such as human behavior, creativity and the arts, the nature of social, economic and political conflicts, and global issues of importance such as health and poverty. The second is the rapid development of knowledge, as well as its application in the production of all goods and services. Technologies are changing so fast that in many cases, specific methods and techniques, specific ways of doing things learned at university, are outdated by the time students reach the labor market, or soon thereafter. Because of these two forces, the demand for graduates at the Bachelor level is changing towards a richer and broader education, not only within a domain of study fields, but also across several domains. Employers want education that is broad and versatile enough to make it easy for their employees to rapidly acquire new skills when these are needed. 109. In Romania there are a large number of specialized, single-domain (sometimes even single- field) state and private universities. There are universities dedicated to fields such as medicine, pharmacology and health care, engineering, agriculture and veterinary, economics and business administration, architecture, music and performing arts. In comprehensive universities it is easy to expose students of engineering and technology to the humanities, the social sciences and the arts. It is also easy to expose students of humanities, social sciences and arts to the role of technology in society. The question is whether this is equally possible in the specialized universities. Are these universities well aligned with the emerging demand in the market and in society? The scope of this functional review did not allow the review team to investigate this question deeply enough and come up with an answer. 56 Recommendation: 110. The team recommends that the Ministry conduct an in-depth study on the pros and cons of having highly specialized universities. Such a study can also focus on issues of economies of scale and efficiency, exploring the question whether less specialized universities are inherently more efficient. 111. Another reason why the review team is suggesting that the level of differentiation by field of study may be too high has to do with the number of study programs being offered by Romanian universities. The number may be too high in view of the change in patterns of need and demand discussed above. As can be seen in Annex table 4, there are currently 2,562 programs in 73 fields at the Bachelor level, an average of 35 programs per field. It is doubtful that Romanian society and economy today require such a variety of programs at this level. In the ACPART survey it was found that 45% of the students, who obtained employment in the last five years, obtained it in fields other than those that they studied at university. What this implies is that there is no need to diversify study programs so much at this level. This conclusion is supported by two findings reported in ARACIS‘s Quality Barometer – 2010: the first was that some employers actually complain about the too many programs at the Bachelor level; the second was that 72%-85% of the employers surveyed do not give any priorities to students coming from given, specific study programs. A similar situation exists with respect to study programs at the Master level. The number of such programs is 2306, raising the question whether here too there may be too many programs. 112. It is difficult for this review to provide guidance on what the appropriate number of study programs ought to be. However, the review can provide recommendations on how to build in mechanisms in the system – mostly through providing more information to students and institutions – to ensure that universities respond quicker to changing demand. At the current juncture, it looks as if virtually all tertiary graduates find jobs (see discussion in section 1) but this is no guarantee that they will in the future. Moreover, at the current juncture, students are already voting with their feet in terms of selecting the university and field of study they want to attend but without information about prospective earnings and employment outcomes (and with near-certain job guarantee upon graduation because of the strong (past) demand for tertiary graduates), students may not have been as demanding as they would have been in a tighter labor market, and with more information at their disposal. 113. As noted earlier, the scope of this functional review did not allow the team to delve more deeply into questions of differentiation. Here too, no definite conclusion can be reached on whether or not there are too many study programs at the Bachelor and Master levels, but here too the team believes that there is sufficient evidence on which to base a recommendation that the 57 Ministry conduct a thorough, in-depth study to review the need for all programs and, if necessary, consolidate or close some of them. Recommendation: 114. Conduct a thorough, in-depth study to review the need for all programs and, if necessary, consolidate or close some of them. c. The differentiation among higher education institutions by geographic/development region 115. The distribution of higher education institutions in Romania across geographic development regions suggests that the supply of places at higher education institutions in some regions may not be aligned with the size of the population that has to be served in these regions. There is not sufficient data to conclude that the supply is not aligned with the demand coming from the population for these services, since the extent of demand is not known, but from an equity point of view, based on the premise that the needs of the people in all regions are not different in terms of the acquisition of knowledge, this finding provides further support to the conclusion reached earlier in the chapter of sector performance that the provision of higher education in these regions is not fully equitable. 116. As can be seen in Table 1 below, the ratio of people to university varies greatly across the regions from the low of around 66,000 in Bucharest to the high of around 828000 in South Muntenia. The spread between these two extremes is quite wide too. In South Muntenia, one university has to serve a population that is on average almost 13 times larger than must be served by a university in Bucharest, or 11 times than West Oltenia. 58 Table 1: Number of Universities and Teaching Staff per Population Unit by Development Regions Ratio to No. Ratio Macro No. No. Teaching Region Population* Teaching People to Region Universities Faculties Staff per Staff University University Total Country 21584365 108 624 31103 199855 288 1 North 2730132 17 97 5616 160569 330 West Central 2530818 13 75 2875 194678 221 2 North East 3732583 15 75 3994 248838 266 South East 2837834 7 53 1694 405404 242 3 South 3312342 4 37 1248 828086 312 Muntenia Bucharest/ 2293895 34 166 10726 67467 315 Ilfov 4 South 1927229 4 42 1653 481807 413 West 1039704 14 79 3297 74265 236 Oltenia *National Institute of Statistics figures for 2006 117. The large differences in the provision of higher education services may reflect weakness in the economic and social development potential of the regions in question. It is a strategic issue that has to be addressed by the Ministry. The issue is much more complex than what has been presented here briefly. Clearly, adequate higher education services need to be provided to all people in all regions, but the extent of demand need to be verified first before an attempt is made to increase the provision. The excessive proliferation of study programs in all existing universities has to be taken into consideration in any plans for the expansion of services. Expansion in services does not necessarily mean a growth in the number of universities; it may also be achieved through an increase in the sizes of existing universities. Finally, some fields of education are more expensive than others and a less expensive alternative to the provision of these in the regions may be to increase the capacity of existing universities wherever they are and provide residential facilities. 59 Recommendation: 118. The review team recommends that the Ministry conduct an in-depth assessment of the need for the expansion of higher education services in the currently under-served regions and develop a plan to address this need. d. The differentiation among higher education institution by Ownership and Clientele 119. The most significant differentiation that occurred in the Romanian higher education sector over the last 15 years has been the emergence and growth of private institutions. These institutions absorbed much of the unmet demand for tertiary education, have proven to be resilient and self-sustaining and are now a force to be reckoned with – a force that must not be seen as a threat to the system, but rather as an opportunity for it. 120. The first private sector higher education institutions entered the scene in the early 1990s. As can be seen in Table 2 below there was an initial flurry of activity once the higher education sector opened up. In 1995/6 there were already 36 institutions in operation with 85305 students and by 2002/3 the number of institutions reached 70 and the number of students reached 139 038. Since then, the number of institutions started dropping. There was a sharp drop between 2002/3 and 2005/6, but the number then settled down around 50 institutions and is standing today at 52 institutions and 322337 enrolled students. Most of these institutions got full accreditation during 2006/8. The number of enrolled students kept growing during that time while the number of institutions went down – an indication that the drop in institutions was not a matter of demand, rather a business failure due to organizational and management issues. The relative stability in the number of private institutions coupled with the growth in enrollment since then indicates that there is now stable demand for this type of provision. It comes predominantly from two groups of clientele: (a) people who are not being admitted to state universities in the country and (b) people who cannot study on a full time basis, or cannot physically attend university even on a part-time basis. The first group consists of a mix of people, some of whom are perhaps not university material and some who are, but who have not been lucky enough to be admitted. There seems to be some flow of people from the second sub-group away from the big urban centers to state universities in the smaller ones, but many prefer to stay in the big urban centers and go to the private universities. 60 Table 2: The Growth of Private Higher Education in Romania 1995/6 2002/3 2009/10 Private State Private Private State Private Private State Private % of % of % of Total Total Total Number 36 59 37.9 70 35 56 52 56 48% Institutions Number 85305 250836 25.4 139038 45725 23.3 322337 452982 41.5 Enrolled 9 +62.9% +131.8 -1.0% over +55.4 % over over 2002/3 % 2002/3 2003/3 over 2002/ 277.8% +80.6 3 over % over 1995/6 1995/6 Number 7184 50176 12.5 26006 77399 25.1 91803 123023 42.7 Graduates +261.9 +54.3 +253% +58.9 % % over % over over over 2002/3 2002/3 2002/3 2002/ 3 +1177% +145.2 % over 1995/6 121. The emergence of private institutions of higher education was perceived by the state academic community initially, and is still perceived today, as a threat – not in the sense of competition over students, but in the sense of loss of reputation for the entire sector. Quality was, and seems to be still an issue, and the concern has been that since the MERYS signs and issues all diplomas, the issuance of diplomas for graduates of private institutions will devalue all diplomas. Again, the sector is being constrained by the lack of performance data. There is a strong perception that quality in the private sector is a problem but because there are no tests of students‘ competencies, the discussion has not evolved beyond anecdotes, perceptions and inputs standards. No one knows how big the learning gap is, whether it has improved over time, and whether it varies by field of study? 122. The perception of employers with respect to the quality of education provided by the private institutions is similar to that of the state universities‘ academic community. In a survey carried 61 out in 2010, ARACIS found that employers rate the quality of higher education received at private universities as lower by 1 to 1.5 points on a scale of 1 to 10, where 1 is very poor and 10 is excellent. 123. One sign that the power of students voting with their feet (toward better quality) is not being harnessed is that students keep enrolling in private universities, irrespective of the perception of poorer quality. As can be seen in the table above, the number of graduates of private universities in 2009/10 was close to four times larger than the number of graduates in 2002/3. 124. It is important to note that private higher education institutions are carving for themselves two very important, closely-linked and growing niches in the market: one is the niche of people who do not have time to attend regularly because they are employed and the other is the niche of people who, employed or not, cannot attend because of distance and/or the need to stay home. The institutions are offering them both part-time programs and distance learning programs. Of the 322 337 enrolled students in 2009/10, 32.5% are part time and 6.2% are distance students – together 38.7%. This is undoubtedly a major line of business addressing a rapidly growing niche in the market. Because of its centrality, the universities will focus on it, trying to align their services even better with the needs of the people in this niche. Some of the state universities too have part time and distance learning, but for them it is a side business with more nuisance value than added value. They are much less likely to focus on it. In time, the quality of the service provided in this area, not only in terms of organizational arrangements but also in terms of the content and standard of instruction, will improve in the private universities sufficiently to present strong competition to the state ones. 125. The development of the private higher education sub-sector and the direction it takes are matters of strategic importance to the MERYS, because by now 41.5% of all enrolled students are found in private institutions and 42.7% of all graduates come from them. Both the President‘s Commission and the Ministry have recognized the importance of the sub-sector. While subjecting it to all the requirements and conditions aimed at ensuring quality education which as is the case with state universities, they have already opened the door for private universities to enjoy some of the privileges enjoyed by state universities, such as access state funds. Private universities can now compete alongside state universities for research grant funds and institutional development funds. Also in its recommendation that the state provide scholarships to foreign students in order to attract them to Romania the President‘s Commission stated that such scholarships will be allocated to the universities and course programs that meet the highest quality standards, whether state or private. 126. This is a step in the right direction. It helps the higher education sector align itself better with conditions in the market for higher education. 62 Recommendation: 127. The review team recommends that the Ministry conduct an in-depth review of the present status and performance of private higher education in Romania in order to develop a comprehensive strategic framework for the next steps. The strategic framework should chart more clearly the future role of private universities in the education sector of the country, as well as the kind of state sector environment that will be most conducive to its success in playing this role. This should include strategies to bring about a complementary division of labor, to enhance collaboration between state and private universities and to help the private ones bring the quality of their graduates to the level of state universities (which needs to improve too). a. Other Forms of Differentiation 128. The growth in tertiary education all over the world has been remarkable. Attendance at tertiary institution, which not too long ago was a privilege of the few, has become a requirement for the many, and the trend is for tertiary education to become universal. This trend is leading to further diversification in the needs and demands of students as well as employers, which in turn require the tertiary education system to further diversify too. This is true for Romania as well. Here are two further examples of differentiation the need for which has already become, or may soon become, evident: 129. There is a difference between state universities and public universities. State universities are funded predominantly and controlled by the state, while public universities are supported by the state but are not fully controlled by it. One of the key differences has to do with the status of employment of university teaching and other staff – in state universities they are considered state employees, while in public universities they are not. The implications are very important, particularly with respect to the issues of university autonomy discussed earlier. Public universities have the freedom to determine the number of professors they employ and their salaries. In Romania, public universities do not exist. According to the Education Law, there are three types of universities: state, private and confessional. The ability of universities to attract top-level professors and researchers, in line with the Ministry‘s, and their own, vision of excellence, is limited because of this situation and is weakening further now with the introduction of the Unitary Salary Law. The team recommends that, in the context of developing further its strategy for higher education in Romania, the Ministry consider the question whether or not to legally and operationally open the door for some state universities to formally change their status from state to public institutions. 63 130. In the period that followed the country‘s gaining of independence, when many new education institutions were established at the tertiary level, they could become only fully-fledged universities. The system of accreditation that was created, addressed only universities. Tertiary- level, shorter duration, technical and vocational institutions could not legally exist as institutions of higher education. Having observed the rapidly increasing growth in enrolment and diversification in students‘ backgrounds, as well as the diversification in demand for graduates, the team recommends that the Ministry bring this matter too, particularly the need usher in shorter-cycle institutions, into its planning for the future of higher education in Romania. V. Strategic Management in the Sector 131. Three conditions must be met by government to manage the higher education sector strategically. It must (a) know where the sector is today, (b) determine where it wants the sector to be tomorrow, and (c) chart a clear way showing how it will get the sector to move from where it is today to where it wants it to be tomorrow. The first requires it to do a situation analysis, clarifying what is the sector‘s performance today and identif ying its most important issues and opportunities. The second requires the government to specify the most important goals for the future that it will seek to achieve in the sector. This produces its vision/mission. The third requires the government to identify all the possible strategies through which it can move the sector from the present situation to the desired future, assessing their feasibility and selecting the best and most feasible among them. If it does all this, then the government is managing the sector strategically well. Clearly, there is a sequence to be followed here. Where the government wants the sector to be tomorrow must not be determined before the government knows clearly where it is today. Likewise, how the government wants to get to a given destination tomorrow must not be determined before both the starting point and the destination point are known. The same applies individually to each and every university in the sector. In analyzing strategic management in the sector, the review team followed the same logic, asking whether a good situation analysis was carried out and was followed by a solid visioning and strategizing exercise. The team concluded that this was indeed done for the sector as a whole and that the MERYS is now managing the higher education sector fully in a strategic way. The same cannot be said about many of the individual higher education institutions. Based on the definition of strategic management given above, and based on their mission statements, the management of these institutions seems to be focused more on operational issues and less on strategic ones. 132. As discussed in detail in the team previous report on the Pre-University sub-sector, the vision, as well as the strategies of the education sector evolved over the last four years in a process that started with the setting up of a Presidential Commission in January 2007 to conduct the situation analysis, define the long-term goals and identify the best ways to achieve these goals. The Commission‘s report was made state in July of that year and was followed by a 64 National Education Pact signed on March 5, 2008 by all political parties, as well as 22 organizations with an important stake in the education sector. Having elaborated on the report, the Commission then produced the core strategy for the education sector later in 2008. With respect to the higher education sector, the Commission identified in its situation analysis five broad areas of concern. It formulated a broad vision in each of them and then produced altogether around 30 strategic measures to address them. The five areas were:  Mediocrity and lack of excellence in teaching  Slow growth in, and too low quality of, research  Poor quality of student services  Inadequate organizational and governing structures at the universities; and  Lack of accountability for performance by the universities Here is an example of the excellent work done by the Commission: Situation Analysis Vision Lack of excellence in both teaching and A system where the required outcomes will be research, due to (a) lack of concentration of clearly defined on the basis of competences resources (human, physical, financial), (b) poor and skills and the curriculum will be well- quality of a good part of the teaching and designed to produce these outcomes, where research staff of universities, (c) wrong criteria smarter resource allocation, better criteria for for determining seniority in teaching and the selection and promotion of professors, zero research, (d) inadequate ethical behavior of tolerance for unethical behavior will result in university management and professors and (e) higher academic quality, reaching excellence need for better assuring of quality and in many areas. relevance both at system level and at institutional level. Strategies 1. Assess all higher education institutions, both state and private, externally. Rank study programs and make the results state. 2. In the allocation of state money concentrate resources, including study grants, , giving priority to quality institutions and programs and reducing or eliminating state funding to state universities that do not meet the minimal operational standards 3. Enhance differentiation of universities by classifying them into research-intensive, reaching and research and teaching only universities. 4. Require universities under the terms of the institutional contract to assess and rank departments by performance and to prioritize allocation of institutional resources to high performing departments. Monitor low performing departments over 2years and if they fail to improve close them down. 5. Establish an institutional development fund for allocations to top universities, whether state or private. 6. Give universities full autonomy and responsibility to define teaching and research loads. Give universities full autonomy and responsibility for selection, evaluation, motivation, training and dismissal of their teaching and research staff 7. Fill vacancies at universities not only with Romanian professors, but also foreign ones. 65 8. Assess and re-accredit higher education institutions as PhD program providers and allow high- performing associate professors to guide doctoral students 9. Streamline teaching grades and discard seniority as a criterion for teaching and research positions. 10. Implement effectively a professional code of ethics at university level. 11. Strengthen the role of the Commission of Ethics as an appellate court on internal code-of- ethics matters of universities. 12. Make the Code of Ethics and the Quality Assurance Code annexes to the institutional contracts between universities and the ministry. Report compliance annually and publish the report. Make it a condition for issuing next year‘s contract. 13. Develop a National Qualifications Framework for Higher Education and require universities to accurately define the qualification they give in terms of competences/skills and education outcome. 133. One weakness of the strategy, though, is that it lacked data on performance to assess the current situation of the higher education sector, and to set targets for the future. One clear example of this gap in performance data is on the targets set by the strategy: in the section on pre-university, some of the targets clearly focus on the quality of the education provided (i.e. a performance indicator), with a goal to close the learning gap between Romania‘s 15 year olds and that of the OECD, as measured by the OECD‘s PISA assessment. By contrast, the targets on higher education focus on enrollments (to create more equity in enrollments) and the broad (but opaque) goal of having three Romanian universities in the top 500 in the world. 134. A large number of the strategies developed by the President‘s Commission was adopted by the MERYS and introduced into the new Education Law. The law was passed several months ago, and with the three conditions for strategic management met, the Ministry began implementing in earnest key strategies in it. Strategic management means not only the development of strategies on paper, but also their actual implementation on the ground. This further supports the team‘s conclusion that the Ministry is indeed managing the higher education sub-sector strategically. 135. Until 1997 only a few Romanian universities felt the need to develop medium to long-term visions, missions and strategies for themselves and express them in state statements. The universities‘ role was conceived as serving the state, and universities fulfilled their role by implementing the directives of the Ministry of Education. Identifying and meeting the needs of students or actors in the labor market was not expected of them – neither by the state, nor by the students and the market. There was no need for state vision and mission statements. In 1998, the Ministry issued a Ministers‘ Order requiring all universities for the first time to develop university charters. The Order stated that charters will have to be reviewed every four years coinciding with the term of each newly elected governing team of rector, vice-rectors and deans 66 (to which the position of administrative general director was added in 2000). This time period was not yet conducive to long-term thinking and strategizing. Romanian society as a whole and each of the universities individually were in a period of trial and error, characterized by frequent changes of strategy at the national and at the institutional levels. Universities focused on immediate survival and did not get to the point of focusing on longer-term visions, missions and strategies. 136. During this time, in the absence of institutional strategic thinking, it was quite typical for the universities to simply go with the flow – to let their direction be determined by the interests of their teaching staff. Study programs were designed to meet the preferences of the professorship. It was only around 2005 that some universities started to proactively think of longer-term strategies and to re-design their curricula accordingly aiming, among others, to address needs and expectations of their students and the labor market. The analysis of university charters by the team suggests, however, that the bulk of the universities are not yet being managed strategically. Longer-term visions, missions and strategies are not yet there. The current charters which relate to the years 2008-2012 contain many common platitudes and only a few thoughtful and substantive bits of information differentiating among universities and reflecting longer-term direction and planning. All of them are addressing young, non-working high school graduates and do not reflect thinking on the targeting of different student population groups. Few of them address the effectiveness of investment in higher education, learning outcomes and the impact they would like to make on the personal and professional life of graduates. None speaks of accountability. The impression is that many universities use generic statements with some ―fashionable‖ components taken out of the Charters of other universities. According to Article 128 of the new Education Law, universities have to up-date their charters and the law lists ten compulsory elements that must be included in the charters. The inclusion of a well-developed vision/mission, proving that the university is thinking, and being managed, strategically is not among them. In conclusion, the team believes that having selected important key areas of concern for immediate and forceful action, the Ministry is now managing the sector strategically. Its vision and strategies are not different from those expressed by the President‘s Council and the strategies it selected too are well aligned with the most important issues in the sub-sector. One piece of evidence of strategic management is the existence of 6 strategic projects briefly described in Box 3 67 Box 3: Strategic Projects for Romanian Higher Education Project 1: Quality and Leadership for Romanian Higher Education Project The Quality and Leadership Project for the Romanian Higher Education has its objective to develop a vision and a strategy for the Romanian higher education system (within year 2025 time horizon), in order to ground medium and long term policies, so contributing to improving financial and strategic management of Universities. (see ―http://www.edu2025.ro/home.aspx for more details) Project 2: Doctorate in Universities of Excellence - Research Assessment and Support for Scientific Publishing Project The goal of this project is to develop the national framework for quality assessment of a national research and supports excellence schools along with increased individual and institutional publishing capacity. (see ―http://www.ecs-univ.ro/home.aspx for more details) Project 3: Doctoral Studies in Romania - Organization of the Doctoral Schools Project The aim of the project is to promote a unified strategic approach to reforming Romanian doctoral studies in order to make sure that doctoral and postdoctoral programmes contribute effectively to the training of researchers with internationally recognized achievements, and thus ensure the visibility and recognition of the talent of young researchers trained in the country (see http://www.studii-doctorale.ro/home.aspx for more details) Project 4: Improving University Management Project This project aimed at providing support for responsible and effective university governance by promoting techniques, knowledge and modern management instrument of a higher education institutions .The project (see http://www.management-universitar.ro/home.aspx for more details) Project 5: The National Student Enrollment Registry Project The aim of the project is establishment of an integrated, unique IT system on a national level for the Higher education participants, contributing to improvements of the administrative management in Romanian Higher Education System and providing the necessary tools for the inform policy making in the public domain (see http://www.rmu.ro/home.aspx for more details) Project 6: The Graduates and Labour Market Project The aim of the project is to generate objective, standardized information on whether graduates (from different study fields or program, and different universities) find jobs, including how long it takes them to find these jobs, what type of jobs are they, and what are the starting salaries. (see http://www.absolvent-univ.ro/home.aspx for more details) 137. The universities are lagging behind the Ministry in this area. Most state universities do not have corporate management and are, at least by the content of their charters, are not being managed strategically. One of the actions contemplated in the President‘s Commission report is to ―set up a National Center for Higher Education Manager Training to provide managerial training not only to holders of executive positions, but also to all academics contemplating management as a career development option.‖ 68 Recommendation: 138. The team recommends that the Ministry follow up on this proposed action (of setting up a National Center for Higher Education Manager Training) to see how best to address the issue. 139. The Ministry‘s objective needs to be not just making sure that all universities manage themselves strategically. It should also be to ensure that their strategies are aligned with its overall strategy. University charters are among the data collected by ARACIS in its assessment and accreditation of universities. Recommendation: 140. The team recommends that ARACIS bring these two concerns into its assessment process. A. Expanding autonomy in exchange for accountability that focuses on results 141. The autonomy and accountability of universities have been among the top items on the agenda of higher education in Europe. The issue – how much autonomy universities should have and how far are they going to be accountable in return – has been recognized as a strategic issue by the European Commission17, the Council of the European Union18 , the European Universities‘ Association (EUA)19 and a significant number of governments in Europe. 142. Direct control with strong micro-management by a central lead agency and autonomy for organizations to do what they think is effective are two ends of a management spectrum. In management sciences the prevailing theory is that being close to either end may be the right thing, depending on circumstances, but direct control requires a heavy involvement in operational matters from the central agency, taking the time of its managers away from their core business which is strategic work. The theory is, furthermore, that organizational autonomy improves the performance of organizations, but is also risky. The organizations that receive autonomy need to have capable managers, whose thinking and core values are aligned with those of the central lead agency. These managers need to be equipped to handle autonomy and must be given performance targets. The role of the central agency is to guide them, to correct the courses of action, if necessary, and in the end to hold them accountable for their performance. The ideal situation is to achieve a balance between autonomy and accountability. 17 ―Delivering on the Modernization Agenda for Universities: Education, Research and Innovation‖ (May 2006) 18 ―The European Research Area: New perspectives‖ (April 2007) 19 The Prague Declaration (2009) 69 143. The belief shared by the European Commission, the Council of Europe and the EUA is that the ability of universities to respond to society expectations, to act quickly in the rapidly changing national and global environment, to balance the growing flows of talent, and to raise much needed funding will improve with autonomy. University autonomy is seen as crucial for the achievement of the European Higher Education Area (EHEA), as well as the European Research Area (ERA). In its 2009 publication on University Autonomy in Europe, the EUA reports the responses of members of the National Rectors‘ Conference, to the question how important were some changes to their institutions. Autonomy was among the top three important changes and 43% indicated that more autonomy is needed. There is consensus among them, however, that national frameworks and regulatory environments need to balance autonomy with accountability, and that neither the exercise of total freedom by universities, nor micromanagement by the state is desirable. 144. There is significant evidence in the research literature that university autonomy is positively correlated with performance. Philippe Aghion et al.20, for example, have shown in a 2009 publications that both in Europe and the USA more autonomous universities that need to compete more for resources are more productive. They maintain that ―centralized government control may be much less effective, as a form of governance, than making largely autonomous organizations compete with one another for resources and prizes.‖21 They show that in Europe countries, such as the United Kingdom (U.K.) and Sweden that have unusually autonomous universities also have unusually productive universities and that in the USA, universities' output is higher in the states in which the universities are more autonomous and face more competition. One of the main findings they report is that universities with higher Shanghai rankings enjoy greater autonomy and draw a higher percentage of their budgets from competitive resources. 145. As stated by Aghion et al., referred to above, it is important to keep in mind that ― autonomy and competition are intertwined and that there is little point and possibly some danger in giving universities great autonomy if they are not in an environment disciplined by competition for research funding, faculty, and students. There is [equally] little point in promoting competition among universities if they do not have sufficient autonomy to respond with more productive, inventive, or efficient programs.‖ 22 a. Autonomy - The Concept and the Practice 146. The review team found it useful to use the EUA‘s conceptual framework for university autonomy.23 In that framework autonomy refers to the degree of freedom universities have in making key decisions on one hand and the degree of control exerted by the state over these 20 Ibid. 21 Ibid., p. 4 22 Aghion et al., pp. 3-4 23 Estermann, Thomas and Terhi Nokkala (2009): ―University Autonomy in Europe I‖ Exploratory Study, European University Association. 70 decisions on the other. The framework separates the various components of autonomy as follows: • Organisational Autonomy – refers to the ability to make decisions regarding the leadership and management structures of universities, particularly about the governing bodies and the reporting relationships among them. • Financial Autonomy – refers to the ability of universities to make decisions regarding the mobilization, allocation and management of funds – decisions such as whether or not to charge tuition fees, accumulate surplus, or borrow money; whether or not to procure goods and services or invest in infrastructure. • Staffing Autonomy – refers to the ability to make decisions regarding the number and qualifications of academic and non-academic staff, when to recruit and what terms of employment to offer. The autonomy to make decisions on financial matters related to staffing such as the overall salary costs and individual salary levels, are very important in this context. • Academic Autonomy – refers to the ability to make decisions regarding the vision, mission and the desired academic profile of the university, regarding the introduction or termination of degree programs, their structure and content, and regarding matters such as students‘ admissions or how to assure the quality of programs and degrees. Also, the ability to decide on the areas, scope, aims, and methods of research are an important sub- components of academic autonomy. 147. One of the general conclusions of the EUA study (see Esterman and Nokkala 2009) was that although there was a trend away from direct state control, governments in many countries continued to exert much control through a variety of indirect steering mechanisms, such as agencies for quality assurance. Romania participated in the study, but the study was carried out in 2008/9 before the new National Education Law was enacted. The team extracted all the types of decisions reflecting level of autonomy that were included in the study and attempted to determine who makes these decisions in Romania today, following the enactment of the law. The team then compared the level of autonomy in Romania with the level of autonomy in 32 European countries where autonomy was reviewed by EUA. The results are found in Annex Table 5. Twenty eight (28) types of decisions are listed in the table. Next to each type there is an indication of who makes the decision in Romania, as well as who is making it in other European countries today. The table reflects restrictions on autonomy in Romania with respect to 10 of the 28 types of decisions and actually demonstrates that many other countries in Europe have similar, if not even more stringent restrictions on autonomy. It is, however not the number of decisions made by the Government, the Ministry, or its intermediate agencies that reflects the severity of restrictions on autonomy, because not all decisions are of equal importance. The sense of restrictions on autonomy in Romania emanates more from the types of decisions made by the government or its agencies, some of which do not even have much to do with the new Education Law. 71 148. The most important restrictions on the autonomy of universities in Romania are in the areas related to staffing and financing – areas in which restrictions imposed by the Public Employment Law and the Public finance Law are added to restrictions imposed by the Education Law. With respect to staffing, universities cannot freely decide on the recruitment, appointment and pay of staff – both academic and non-academic. With respect to financing, universities depend on the government for the bulk of their budgets and, as discussed in the chapter on Financing, a form of centralized procurement, including the oddity of having to obtain special approval from MERYS for any expenditure above a 500 EUR spending limit. b. Accountability – the Concept and the Practice 149. Whereas consensus is beginning to emerge on the various dimensions of what university autonomy means and how to measure it (in large part due to the world of EUA), the discussion on accountability is still constrained by a lack of consensus regarding how to define and measure it. Notwithstanding this lack of consensus, it is important to distinguish first between ―accountability‖ and ‖responsibility‖. Both are obligations: ‖responsibility‖ is the obligation of an organization or a person to take action; ‖accountability‖ is the obligation of the organization, or person, firstly to answer for a responsibility that it accepted and secondly to bear the consequences. The obligation to answer is basically a reporting obligation. The responsible organization or person must report on how they have carried out their responsibilities, and the reports provided by them must be validated. In effective systems there is full clarity on the questions ‗accountability to whom?‘ and ‗responsibility for what?‘ and there is also full clarity on the consequences of poor performance. 150. There are many different instruments to bring about more accountability in a sector, depending on which accountability relationship is targeted. Between the financier (in this case the MERYS and institutions), the instruments can be: reporting, transparency and the power to sanction. Moreover, policy makers can strengthen accountability by empowering the client (in this case student, parents and employers) vis-à-vis the provider (in this case HE institutions). It can do so in many ways, including providing the client with better and more clear information about the product being purchased. Such information could include data and analysis on employment and learning outcomes, to let students make more informed decisions about which university and field of study to pursue. Students can also be ―trained‖ and provided with better incentives to become tougher clients, for instance by providing them with better counseling when selecting university, or by making more of them pay (and pay more). Arguably, a paying customer is a customer who will seek more information about the product, and demand better quality. 151. In analyzing accountability in the area of higher education in Romania the questions are then: to whom are Romanian universities accountable, for what are they responsible and what 72 consequences might have to be borne by them, should they fail to carry out their responsibilities. The answer to the first question is that they are accountable to the students and their parents (as the main recipient of their services), to society (as both the tax payer and an intended beneficiary) and the government (as the founder). This Review‘s section on ―performance management‖ provided an answer to the second question: in terms of results, universities are responsible for delivering graduates that are employable, students that acquire skills, knowledge and competencies during the course of their study, and for producing quality research. There is a tendency today to add to these two major categories of obligations a third one: to deliver services to the communities where they operate. 152. The gaps in data on the performance of Romanian universities are discussed elsewhere in this report. Here, suffice it to say that according to the President‘s Commission report, the Ministry‘s analysis there is a perception that performance in the sector leaves much to be desired (but, again, no data exists to quantify this perception of show whether it has worsened or improved over time). There is little doubt, however, that the Ministry perceives performance to be unacceptable and that the new Education Law aim at addressing this perceived poor performance by strengthening accountability. Moreover, the new law utilizes a range of difference accountability tools but the tools seem to focus mostly on the relationship between the Ministry and the Institutions, with less emphasis being placed on making students more informed consumers (to achieve accountability for results through that venue). 153. In details, the law requires extensive reporting and much transparency. Importantly, the law also introduces for the first time two powerful and realistic sanctioning tool: the provision, or withholding of financial support with respect to state universities and the provision or withholding of recognition with respect to private universities. State universities that do not meet minimal operational standards will no longer receive public funds as before, and universities that excel will receive more than they have received before. Private universities that fail to perform at minimal standards shall not get State approval for the diplomas they grant to their students, and those that perform well will not only get recognition from the State, but will also qualify to compete for research and institutional development funds provided by the State. c. More Autonomy in Exchange for an Accountability Relationship that Focuses on Results 154. Coming back to management sciences, the theory is that the best situation is one where there is a balance between autonomy and accountability. The more difficult it is to hold organizations and individuals accountable, the less autonomy they should get and vice versa: the easier it is to hold them accountable, the more autonomy they can get. At the current juncture, expanding autonomy (which is what universities desire) should be done in exchange for an accountability relationship (e.g. in the context of the performance contracts that MERYS signs with universities) that reports on whether graduates find jobs (as measured by tracer studies), 73 whether students acquire skills, knowledge and competencies during the course of the study (as measured by standardized tests to be delivered) and whether quality research is being produced. More Autonomy Better Reporting on Performance More Transparency Ability/ willingness to sanction 155. It is the perceived poor performance of many Romanian universities and the lack of data and analysis of whether students learn and graduates find jobs that led the MERYS to introduce into the new Education Law measures that limit the autonomy of universities. However, together with these measures, the Ministry has introduced into the law two additional sets of clear and strong measures: one set relating to accountability and one relating to competition (which is expressed in the classification and ranking of universities). The team reviewed both sets of measures and grappled with two questions: firstly, why is it necessary to limit autonomy when all the necessary measures to have accountability and competition are in place? The rationale for this question is that taking the power to make decisions away from the universities moves the responsibility for making these decisions – and therefore for important elements of performance - from the universities to the decision-makers, that is, the Ministry and its intermediate agencies. Once the responsibility moves away from the universities, they cannot be held accountable for performance which is dependent on these decisions. Accountability moves away too and now the Ministry and its agencies are accountable for much of the performance of the universities. The effect of introducing measures that limit autonomy is thus to reduce accountability. Secondly, as is stated by Aghion et al., referred to above, ―there is little point in promoting competition among universities if they do not have sufficient autonomy to respond with more productive, inventive, or efficient programs.‖ 24 Thus, the objective of enhancing competition for better research performance among universities is not supported by the restrictions on autonomy. 156. The team appreciates that the measures introduced into the new Education Law regarding accountability do not automatically and immediately translate to accountability on the ground. In particular, sanctioning public bodies seem to be difficult in all countries and all sectors. It is necessary to see how successful will the new reporting requirements be and how effectively will the Ministry be at applying the rewards and the sanctions. The team recommends however that once the Ministry gains experience with the system of rewards and sanctions and sees an improvement in accountability on the ground, it will let the universities regain autonomy. There is no need to give back full autonomy in all areas to all universities in one step. The Ministry can 24 Aghion et al., pp. 3-4 74 selectively give more autonomy to universities that firstly are consistently well-performing and secondly consistently providing full and transparent reports on their performance. The amount of control can be adjusted to the level of performance and the quality of reporting. 157. In the review team‘s view, a much more promising avenue for strengthening the accountability of the sector is through more and better information on performance and through the empowerment of students (and stakeholders more broadly) to put pressure on the sector to improve the quality of the learning taking place, and creating a more flexible higher education system (with a better alignment of program choices and demand) where graduates find jobs. Another promising instrument for better accountability is more internationalization: many of the Romanian students returning from studying abroad will bring home a clearer picture of what higher education should provide and will help increase demand for better performance. Similarly, to attract foreign students and faculty, Romanian universities need to demonstrate their value added. In the list of recommendations, many of these will be focusing on how to strengthen this avenue for more accountability. 158. The correlations shown in Section 2 are merely suggestive. They do not necessarily indicate that university autonomy and competition cause higher output. Reverse causality is quite plausible: Perhaps governments allow very productive universities to be more autonomous and such universities campaign for resources to be allocated by competition, rather than rules. Recommendations: 159. The recommendations below recognize that strengthening accountability for performance likely involves a number of different interventions, including:  Making students vote with their feet a more powerful force for demanding quality o Place better information in their hands: are graduates finding jobs, what can they expect to earn? Are graduates using their skills and happy in their future jobs? o Educate and counsel high school students; o Increase the number of students who pay and increase what they pay: paying students are more demanding students (for more on this recommendation, see section on financing). o Make it clear that a diploma is not a guarantee for a job. The public sector can begin doing so by starting to differentiate between graduates from different universities when hiring for public jobs. o Through internationalization: students who have been studying outside return with a clearer picture of what the quality looks like in other countries; and foreign students studying in Romania, too, will be a voice for change.  Strengthen civil society‘s voice in holding universities accountable for performance 75 o Close gaps in performance data (see section on performance data) to enable outsiders to see gaps and demand better value for public money o Encourage more policy analysis on the higher education sector (through competitive grants to researchers and civil society, hosting conference etc) o Submit data to Eurostat to allow comparison between Romania‘s HE system and the rest of the EU o Support ―watch dogs‖ who shed light on integrity problem, corruption etc (e.g. ―Coalition for Clean Universities‖, see Box 4) o Conduct an annual survey of students and faculty to gauge the magnitude of unethical behavior  Revamp the discussion on performance-based budgeting to empower MFP and Parliament to provide more challenge to the budget discussion (i.e. to shift the discussion away from historical budgeting to a discussion revolving around spending and results), along the lines suggested in the financing section: o Include more meaningful non-financial performance indicators (again, see recommendation in section on performance gaps) o Start evaluating policy initiatives (using impact evaluations) to generate the capacity in the ministry to start asking more policy-oriented questions (e.g. what results are we achieving? At what cost? Are our programs effective? Etc)  Extensive use of foreigners in helping to benchmark the sector and push for better quality o As foreign peer reviewers o As visiting professors o In helping to design assessments of students‘ competencies (see section on performance gaps) 76 Box 4: Using External Watchdogs to Shed Light on Integrity Problems In 2007 in Romania, a group of 14 NGO‘s combined forces into a ―Coalition for Clean Universities‖ to monitor the Romanian state university regarding integrity issues and establish some norms for good governance. To that end, they piloted a methodology and decided to assess the academic integrity of Romania‘s state universities. Specifically, a questionnaire was designed and teams of external evaluators – composed equally of experts and students – set about requesting information from the universities. Mungiu-Pippidi (2009) describes the methodology in details. In their widely disseminated report, the Coalition emphasized three findings: First, they found evidence of increased tolerance for the phenomenon of plagiarism. Despite numerous scandals regarding plagiarism, universities have no tools to control this phenomenon. With the exception of a single university, evaluators could not identify procedures for combating plagiarism, both at diploma level and at the level of papers elaborated by research staff. Second, they found evidence of extended nepotism. In a number of universities, evaluators identified the existence of so-called ―academic families‖. This state of affairs raises serious questions regarding the objectivity of promotions and evaluations among colleagues. In one of the universities CUC evaluated, there were eight pairs of academic families, three husband- wives and five father-sons. Taking into account the total number of teachers (45), the incidence ―academic families‖ is very high. Third, they found lack of transparency in decisions and carrying out academic process. For example, the competitions for certain positions as teachers or academics are kept quasi- secret, the most common cases being those in which inside the competition there is a single candidate. The procedure for the approval of the budget of revenues and expenditure is often carried without a real consultation and the decision is taken without any provision in the minutes from the respective meetings of the University‘s Senate. Promotions and pay increases are also governed by non-transparent procedures, without the existence of clear benchmarks. Evaluators have also witnessed problems with the publishing of asset and interest declarations and lack of declarations on cooperation with the former secret police. Restrictions in accessing public records with regards to public acquisitions are also a major problem. Lack of transparency regarding internal procedures or administrative/academic results can be identified simply by trying to access websites that universities have and which frequently do not include information relating to employment contests, teaching jobs, the performance of teachers, content of university programs, decisions of the internal governing structures of universities and so on. Source: Coalition for Clean Universities (2007) 77 B. The Internationalization of Higher Education 160. Internationalization is a strategic issue in higher education all over the world, including Europe and Romania today. According to the EUA it has been the third most important change driver in European higher education in the past three years, immediately following the Bologna process and the quality assurance reforms, and is expected to move to first place within the next five years. It presents many challenges, but also offers many opportunities. Some Romanian universities have already incorporated important aspects on internationalization into their strategies and plans as can be seen in the example below, but the Ministry is not yet geared organizationally to dealing with internationalization and has yet no strategy to deal with it; there is a need to urgently get organized and develop the strategy. Box 5: Internationalization at Ion Mincu University of Architecture and Urbanism (UAUIM) Under its forward looking leadership, the UAUIM has been one of the first universities to recognize the importance of internationalism since the early 1990‘s. As stated in many of its documents, the UAUIM regarded internationalization as a key to the quality of architectural education in Romania and its modernization. It viewed international cooperation as a tool to attract both foreign students and professors and thus bring in new approaches and best practice. The University has appealing, well- designed promotional materials in Romanian and English describing in detail its history, study programs, faculties, instructional and other facilities as well as its research centers. Here are some of the activities it has conducted over the last 10 years in the area of internationalization:  Developed relations with more than 40 schools of architecture in Europe, Asia, South America and the United States  Sought and received recognition for its diplomas from the French Ministry of Culture and Communications, the European Commission‘s expert group of coordinators for professional qualifications. Since 2005, its ‗architect‘ diploma is automatically recognized by the EC. It has also sought and received accreditation by the Royal Institute of British Architects.  Developed student exchange programs with 30 schools in Europe. Many of its students participate in study programs abroad and are admitted for practical training in design offices in these countries  Developed teachers‘ exchange programs with these schools and is regularly inviting guest jurors from around the world as judges in its diploma projects  Became a member of six international associations and is playing a very significant role in the European Association of Architectural Education and the International Association for the Exchange of Students for Technical Experience  Since 2004, organized biennially an International Competition for Students in Architecture 78 161. The recent trend for internationalization in higher education is an offshoot of globalization. Student from many countries are going to other countries for their higher education at all levels and a world market for higher education has emerged in recent decades. This market is consolidating and growing rapidly. According to the UNESCO Institute of Statistics, the global number of international students rose by more than 75% since 2000. Working on an update of its statistics as recently as March 2011, UNESCO reported that the final figures for 2009 are expected to show the number of internationally mobile students rising from 2.96 million in 2008 to 3.43 million in 2010. Its projection for the year 2025 is over 8 million international students worldwide.25 162. The latest figures publicly available for Romania, according to the Institute of Statistics‘ Global Education Digest are from 2008. In 2008, 22 432 Romanian students were studying abroad and 13 875 foreign students studied in Romania – a net flow of -8 575.26 By comparison to its EU neighbors, Romania is lagging behind in terms of having students studying abroad (see Figure 14) which partly explains why fewer Romanian students are proficient in foreign languages compared to their peers in other EU countries (Figure 15) The top five destinations were France, Germany, USA, Hungary and Italy. Romania started using the European Credit Transfer System (ECTS) in 1998, and since 2005 it has fully implemented it in expressing student workload and achievements. This will further enhance the exchange of students between Romania and other European countries. 25 UNESCO Institute of Statistics, Record Number of International Students, Bulletin dated March 10, 2011 26 UNESCO Institute of Statistics, Global Education Digest, 2010, Tables 9 &10 pp.116-120. 79 Figure 14: Study-abroad rates are among the lowest in the EU Average Mobility Rate by Subject Group, 2005-2008 (% of all students with foreign study-related experience) Source: EUROSTUDENT (2008), "Social and Economic Conditions of Student Life in Europe," Synopsis of indicators final report: Eurostudent III 2005–2008 (p. 138). Figure 15: Share of Students Fluent in Two Foreign Languages, 2005-2008 Source: EUROSTUDENT (2008), "Social and Economic Conditions of Student Life in Europe," Synopsis of indicators final report: Eurostudent III 2005–2008 (p. 144). 80 163. The Ministry‘s objective is to harness the power of the internationalization tide, and the Ministry needs to strategize on how to do it, but the initiative and the bulk of the action needs to come from the universities. The tide is so powerful that universities which are not gearing themselves to be part of it are likely to be left behind even by Romanian students and professors. Gearing themselves means formulating visions for internationalization, developing strategies for achieving them and aligning their organization structures as well as study programs with these strategies. An excellent example of how one university in Spain has done all of this is presented in the box below: Box 6: Granada university’s strategy for internationalization The University of Granada (UGR), a public university located in Granada, Spain, enrolls approximately 80,000 students. A key goal in the university‘s vision of growth and development is to become a top-rated international university. To achieve this goal the university has developed, among others, a strategic plan for the internationalization of its student body, teaching and research and has aligned its organizations and management structure with it. The main components of its strategic plan focus on the following:  enhancing the mobility of students, faculty and administrative staff  providing administrative and other services to international students and faculty as well as international partner universities  providing international experience to members of the university community who cannot go abroad  promoting the linguistic and intercultural competences of the entire university community  promoting international forums, associations and networks  promoting, and participation in, international inter-university projects  promoting bilateral and multilateral agreements for inter-university collaboration The University has Office for International Relations at the level of Vice Rector and within it four secretariats:  Secretariat of International Student Mobility  Secretariat of International Networks, Associations and Projects  Secretariat for International Staff Mobility  Secretariat of Internationalisation and Marketing Information for foreign students in the University‘s home page on the web is conspicuously presented and is very detailed, and in recent years, the university increased its effort to promote academic exchanges, among others, by increasing the financial support provided by it to international students. The University participates in over 800 bilateral and multilateral exchanges with institutions around the world. It has one Master‘s program taught completely in English and 7 officially bilingual Masters Pro grams in Spanish and English. The university's Center for Modern Languages has agreements with 20 universities and study abroad organizations in the U.S. and in Canada. Not surprisingly, thirteen percent (13%) of the University‘s students are international students. Every year, over 2,000 European students enroll in it through the EU‘s Erasmus Program, making it the most popular destination within this program. In 2007, it was awarded the program's "Gold Star" by the European Commission. The Center for Modern Languages receives over 10,000 international students every year. Source: Official website of the University of Granada 81 164. The movement of students among countries is, of course, only one element of the process of internationalization. Other elements include the standardization of structures, programs and qualifications, the movement of professors across borders and the pursuit of collaborative research and university development programs. The Bologna Process has been one of the main drivers of this phenomenon. 165. The potential benefits of internationalization are great: 1. It can help improve the quality of education delivered at home by enhancing mobility and thus competition, by bringing talent into the country, by exposing local students to multicultural thinking/learning and giving them exposure abroad and experience which they can bring back home. 2. It can help increase the quantity of research done by Romanian academics and improve its quality, by letting them gain access to EU or other international institutions providing grants for scholarly exchange and work abroad; and 3. It can have financial and economic benefits when net flows are positive, since international students pay fees, usually higher than local students and since foreign providers pay taxes 4. Create better foreign language skills 5. Create a more (internationally) mobile workforce 166. The challenges are great too. Internationalization and open borders can result in a brain drain. There is already today said to be a large number of Romanian academics abroad, including some top-rated researchers in a number of fields. Much will have to be done to bring some of them back home. There is also a potential for loss of investment in graduates who go to other countries and do not return home. The loss can be reduced if Romania will manage to turn the tide in student flows from the current net loss to a net gain. Also, gains from remittances made by those who do not return will also help offset the economic loss, but the more serious challenge that cannot be ameliorated in this way is loss of talent. Other challenges which can be met more easily include, for example the simplification of visa requirements and the removal of language obstacles by offering a growing number of courses in international languages. Recommendations 1. To reap the potential benefits of internationalization and to address the risks and the challenges if presents the Ministry needs to conceive of a new function and house it in a new organizational home. In other words, there needs to be an organizational unit in the Ministry to deal with internationalization. 82 2. The first item on the agenda of this unit will be to develop a comprehensive strategy for internationalization, aiming to create in the higher education sector an environment that is conducive to international collaboration between universities and to the mobility of students, teaching staff and researchers. Part of this strategy needs to aim at reversing the balance between Romanian students abroad and foreign students in Romania. 3. Universities should be encouraged and helped to: (e) Design more programs especially for undergraduates from other countries (some have already be designed). These need to be in English and other international languages, and can take place during the regular academic year and also on holidays. Some may be structured around specific disciplines as is done by the German DDA. (f) Conduct well-planned, systematic marketing campaigns abroad (g) Widen student services to help foreign students who are planning to come to Romania. Such services could include for example help in the locating and renting of apartments or actually building residences for students; and (h) Develop cooperation mechanisms through regional clustering and networking at the European level and use institutional and academic partnerships particularly to improve doctoral education. 4. One recommendation made earlier in this report is to provide management training to rectors, deans and all high-level administrators in universities. It is recommended further here that such training will include internationalization, including international marketing as a subject. 83 VI. Financial Management in the Sector 167. A well-functioning Ministry should have people who tasked with regularly investigating how the sectors it finances functions, whether or not it allocates funds to its priorities, how it implements its policy, how much value a particular intervention creates and whether public funds are used in regularity and probity. The nature of such analysis is to be critical of the environment it analyzes. Its ultimate objective is to provide recommendations to the Minister on how to improve the situation. The methodology requires such analysis to substantiate its conclusions with in-depth analysis. 168. The authors of this review were unable to find any evidence of such analysis or investigation of the higher education sector – focusing on financing and on what is achieved in exchange for such financing– taking place in Romania – both inside and outside of the Ministry. With no one seeking to ask and answer questions related to financing of the sector and what such financing achieves, the risk for the analysis and recommendation of this section is that they do not have an audience and a champion to implement the changes. 169. This section provides a brief overview of the financing of higher education, discusses its strengths and weaknesses, and provides recommendations – in the form of an action plan – for how to address the weaknesses. The objective of these recommendations is to equip the sector – not just the Ministry – with more demand for the type of analysis and investigations which can help policy makers implement better policies, improve the quality of the outcomes this sector produces, and, ultimately, improve value for money. 170. Public spending on higher education (as a percentage of GDP or as a share of total public expenditure) is roughly similar to that of EU neighbors. Total spending on higher education by state universities stood at 0.91 percent of GDP in 2009, a dip from 2007 and 2008 when spending was at 1.12 percent of GDP. Compared to its neighbors, Romania‘s spending on higher education was in line with the EU average during the boom years of 2007-2008 but not in the years before the boom years (see Figure 16). 84 Figure 16: Spending by state higher education institutions (as a share of GDP) 3.0 2007 2.5 2001 2.0 % of GDP 1.5 1.0 0.5 0.0 Italy Spain France Belgium Estonia Croatia United Kingdom Netherlands Malta Poland Cyprus Greece Denmark Hungary Sweden Germany Latvia Portugal Ireland Bulgaria Finland Austria Slovenia Slovakia Czech Republic EU-27 average Romania Lithuania Luxembourg Source: Eurostat (extracted January 25, 2011) 171. The financing of higher education provided by state universities can be split into three parts: (i) core financing based on a formula (which include both quality indicators and the number of students, weighted to reflect differences in the cost of providing students to, say, medical vs. law students). Core financing is used by universities to contribute toward salaries and recurrent expenditure; (ii) complementary financing which include research grants distributed on a competitive basis, social expenditure for students (incl. scholarships) as well as capital expenditure; (iii) universities‘ own source revenue, mostly from charging tuition fees for students that do not have high enough test scores to receive budget-financed places. 85 Figure 17: Income of state universities 1.4% 1.2% 1.0% Percent of GDP 0.8% 0.6% 0.4% 0.2% 0.0% 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 Core funding Complementary funding Own source revenue Total Source: CNFIS 172. The nominal amounts disbursed in these groupings (as well as the size of state universities own sources of revenues) are reported in Figure 17 and Box 7 shows the amounts as a share of GDP. Moreover, Box 6 discusses core funding versus complementary funding in more details. The numbers in Figure 17 are averages across all state universities but they mask large variations between universities, with some universities relying more on tuition fees (―tax -paying students‖) and earnings from research than others.27 173. Reflecting the lower level of GDP (and lower overall salaries) in Romania, spending on a per student basis is still significantly lower than in the OECD and in the EU, especially in the private sector (Figure 18). Part of the reason for the lower per student costs in the private sector is the fact that the private sector mainly focuses on the lower-cost programs (the ―soft‖ fields of study discussed in the opening section of this report). However, another reason is that the faculty-student ratio is significantly larger in the private sector. According to 2010/11 data from the National Statistics Institute, the private institutions now have a student-teacher ratio of 74.2, compared to 16.9 in state institutions. 174. Without data on the quality of the education provided it is impossible to draw conclusions on the efficiency of spending in the private vs. public sector (or in the Romanian HE system vs. its peers). That is, the graph below shows that it is cheaper to enroll a student in the private sector but the only way to know if the private sector is more efficient would be to compare the 27 See World Bank 2009: ―Introducing a Student Loan Scheme in Romania. A Discussion Paper‖ for a more detailed discussion of the differences in sources of income of universities. 86 competencies that similar students acquire in the public vs. private sector but, again, no data on the quality of learning are available. 28 Figure 18: Cost per student in OECD (2005), EU19 (2005) and Romania's state and private universities (2008) 9,000 (US$, PPP, constant 2005 prices) 8,000 7,000 6,000 5,000 4,000 3,000 2,000 1,000 0 OECD average (US$, EU19 average (US$, Romania, public Romania, private PPP, constant 2005 PPP, constant 2005 university (US$, PPP, university (US$, PPP, prices) prices) constant 2005 prices) constant 2005 prices) Source: Davidson (2007), IMF WEO data base (for PPP adjustment factors), MERY and World Bank estimates 28 Davidson (2007) provides unit cost estimates for tertiary institutions in OECD and EU19 for 2005. The estimate of the unit cost in state universities in Romania was obtained by dividing total public spending by the number of students in state universities, and converting the figure into constant US$ PPP terms (using an adjustment factor calculated from IMF WEO data base). The estimate for private universities was calculated using an estimate of total costs of running private universities (estimated as described in the box on total spending in HE) and dividing total cost by the number of students in private universities (and converting the number into US$ PPP terms). 87 Box 7: How are state universities receiving their funding from the state? State universities receive their funding under two broad groupings: core funding vs. complementary funding. Below, each of these is explained in details. 1. Base (core) finance is used by universities to cover part of their personnel expenditures (teaching and non-teaching staff payroll expenditures and travel expenditures) and material expenditures (for maintenance and administration, for functional materials and services, inventory goods, recurrent repairs, books and publications, and staff training, protocol, labor protection etc.). Seventy percent of the core finance is based on the number of physical students eligible by law for state budgeted financing (i.e. eligible for the study grant) as reported by the university in each form and field of education and calculated using a cost coefficient determined as the ratio of the financial effort required to train a student in a undergraduate program to the financial effort required to train an undergraduate student in economics, which is taken as the standard. The cost coefficient for social sciences and humanities, for example, is 1, while the cost coefficient for mathematics is 1.65 and for agronomy is 1.75. The student enrollment number is multiplied by the cost coefficient to get the unit equivalent student number which is then multiplied by the budget allocation for base finance as approved by the Budget law less a reserve fund. The enrollment quota to be financed from state budgets for all education levels is established yearly through Decisions of the Government (MERY negotiates with the MPF). The placements (called ―study grants‖) financed from state budget are then allocated to state higher education institutions through a Ministerial Order. Thirty percent of the core finance is based on quality indicators that include teaching staff quality, teaching staff development potential, research work performance, research capacity use, quality of physical resources, quality of documentation, quality of academic, administrative and financial management and quality of social and administrative services provided to students. Each of these indicators is given a weight based on importance. The individual university allocations are computed separately using mathematical formulas involving the whole set of variables. 2. Complementary funding from the government is used by universities to cover among other things: i. Accommodation and food subsidies for students ii. Funds allocated on priority bases for capital expenditures, other investment expenditures and capital repair work; iii. University research funds allocated on competitive basis; iv. Social expenditures for students (scholarships, travel expenditures, etc.) distributed according to numbers of eligible students in each university; v. Procurement of materials and equipment Complementary funding is allocated to universities using formulas that are based on the numbers of eligible students. Source: CNFIS 88 175. Some features of the Romanian the system of financing are very strong. These include the long tradition of using formula financing to allocate core financing for staff and recurrent expenditures to state universities (approx. 1/3 of state universities‘ total income, as shown in Figure 17 and Box 7. The formula – which was introduced in 1999 – is based on the principle of financing on a per student basis but has gradually involved to include ―quality indicators‖ – indicators that measure processes, inputs and management practices argued to be related to providing higher quality of education. When the quality indicators were introduced in 2004, there were only four of them with a weight in the formula of only three percent (with 97 percent of core financing based on the notion of ―equivalent students‖ – i.e. different per student amounts allocated to students in different fields and programs of study to reflect that the cost of educating a medical student is higher than that of an economist). By 2010, there were 16 quality indicators (some of them ―composite indicators‖ with additional sub-indicators), with a weight of 30 percent in the formula (see Box 8 for a description of the quality indicators used in 2010). Some of the benefits of having had this formula (and its evolution to include quality indicators) in place for more than a decade include the following:  Transparency in how the core financing of HE is allocated once the number of budget places has been allocated (more on this below).  Capacity has been built in CNFIS and MERYS‘ HE department to monitor quality indicators and enrollments, as well as students‘ progression toward graduation. With money being tied to students and quality indicators, a healthy need was created to request and work with data. As discussed elsewhere in the report, there is still room for improving how information is being (or not being) used but one thing is certain: if HE had not been financed by a formula (with indicator), there would have been significantly less information available, and significantly fewer people in MERYS and CNFIS with detailed knowledge of how public money is being used.  A group of people within MERYS and the buffer agencies are clearly thinking about how to use the financing of HE education as a policy tool (as opposed to just an accountancy exercise) to improve the performance of the sector. This is evidence in the increased sophistication of the quality indicators and, perhaps even more importantly, in the fact that it is CNFIS who is taking the lead on institutionalizing results from a graduate tracer study.29 The individuals involved in developing quality indicators realized that the only true ―performance indicator‖ (according to the discussion in the section of this report on performance management) is faculty input; all other indicators are related to inputs and processes. And, therefore, they applied for and received EU financing to develop the mechanisms to regularly track whether graduates find jobs. 29 Not everyone the team spoke to agreed with this assessment. Opinions were expressed that, in fact, the Council for Financing HE tended to represent the views of the best universities. And the increased importance of ―quality indicators‖ (which, as discuss, mainly reflect processes and inputs) has been the main instrument for the best universities to ensure that they receive higher per student financing than less prestigious universities. 89 Box 8: Quality indicators used in formula financing of state HE in Romania (in 2010) Quality indicators for the didactical process IC1 - The ratio of the number of full-time teaching staff to the total number of unitary equivalent students IC2 - The ratio of the number of full-time professors, to the total number of unitary equivalent students IC3 - The ratio of the number of full-time lecturers, to the total number of unitary equivalent students IC4 - The ratio of the number of full-time teaching staff, with doctoral academic title, to the total number of unitary equivalent students IC5 - The ratio of the number of full-time teaching staff, under 35 years, to the total number of unitary equivalent students Quality indicators for the scientific research impact IC6 - The level of the scientific research performances IC7 - The ratio of the number of unitary equivalent students from master and doctoral study programs to the total number of unitary equivalent students IC8 - The ratio of the research contracts to the total incomes of the university Quality indicators for infrastructure IC9 – The ratio of the expenditures for development to the total number of students IC10 – The ratio of the material expenditures to the total number of students IC11 – The ratio of the expenditures for books and publications to the total number of students Quality indicators for university management IC12 - The ratio of the expenditures from university‘s budgetary allocation destinatated for investments IC13 - The total quality of the academic and administrative management IC14 - The ratio of the extra budgetary incomes to the total incomes IC15 - The ratio of the expenditures from extra budgetary incomes for development to the total incomes IC16 - The quality of the social and administrative services for students Quality indicators for lifelong learning development [planned to be included] IC17 – Development of lifelong learning A second strength of the financing to universities is the fact that research grants (approx. lei 1.8 bn in 2008) are allocated on a competitive basis, with universities competing and receiving an increasing proportion. By 2008, 18 percent of state universities‘ income (and 40 percent of ―complementary funding‖ shown in Figure 17 came from budget-funded research grants distributed on a competitive basis. For a more in-depth discussion of research grants (and some 90 of the challenges facing these grants), please consult a parallel functional review focusing on Research and Innovation. 176. A third strength is that the financing of higher education has gradually become more diverse, reducing the dependence on constrained public resources. The clearest sign of this is the growing reliance on the private sector and fee-paying students, shown in Figure 19 below. Unfortunately, no institution in Romania gathers or estimates to spending (from public and private sources) on higher education but the trends in enrollment in the private sector (where all students are fee-paying) as well as the trends in fee-paying students in state universities suggest that more resources are being mobilized from paying students. Figure 19: World Bank estimates of full-time equivalent enrollment in state (fee-paying and state-sponsored) and private 30 universities (based on CNFIS and Eurostat figures) (enrolled students, part-time and full-time) 900,000 800,000 700,000 600,000 500,000 400,000 300,000 200,000 100,000 0 2003 1998 1999 2000 2001 2002 2004 2005 2006 2007e Public, state-sponsored Public, fee-paying Private Total, public and private Source: World Bank estimates based on CNFIS and Eurostat figures 177. There are several aspects of the financing of higher education, however, which can be improved upon. This review focuses on four aspects that seem particularly important to address: the large of amount of discretionary decision making in budget allocations; the lack of policy orientation in the budget; the lack of a vision and a strategy for mobilizing more resources for 30 Enrollment in private universities is based on Eurostat figures (for 2001-2007). Figures for full-time equivalent enrollments for 1998, 1999 and 2000 are World Bank estimates based on full-time and part-time enrollments. Enrollments in state universities are using CNFIS figures for 2004-2007 and growth rates in Eurostat figures for 1998-2003 to estimate historical data consistent with CNFIS figures for 2004-2007. The split between fee-paying and state-sponsored is available for 2004-2007 (from CNFIS). For the years 1998-2003, it is known that there were no fee-paying students prior to 2001. Moreover, it is assumed that the number of state-sponsored students stayed roughly constant at around 260,000 (full-time equivalent students) during the years 2001, 2002 and 2003. 91 higher education; and the inadequacy of the reporting formats which are used to monitor spending in higher education. 178. First, there are still a lot of discretionary (and non-transparent) elements in how public money is being distributed. For instance, the capital expenditure budget (totaling almost as much as the amounts for core financing during the boom years 2007 and 2008) is distributed without any underlying analysis or central databases keeping track of investment needs and almost entirely at the Minister‘s discretion. Given the sizes of the amounts being distributed this way (more than 1 billion lei in 2008), is it not surprising that rectors of state universities can be found roaming the hallways of the Ministry, pitching their particular needs to directors (or above). 179. Similarly, the distribution of budget-sponsored study places is also largely discretionary – or at best, driven mostly by historical trends. According to the old law, the distribution was supposed to be based on an annual study conducted jointly by the Ministry of Education and Ministry of Labor. However, such analysis was only done on an ad hoc basis and never had any impact on the distribution of study places. Instead, the decision was based more on a ―historical approach‖ (ie, based on what it was in the years before) than any research about future demand. In practical terms, via an ordinance the Ministry decides how many state-sponsored places each state university gets. Interviewed counterparts report that it uses a number of factors to distribute the places: the historical data of state funded places allocations; the state of universities‘ academic facilities (i.e. do they have room to accommodate more students?); and universities‘ own proposals for how many students they want to accept (as budget financed vs. tuition paying).31 180. Admittedly, there is scope for confusion on this topic: core funding is formula-based with the number of students being the most important factor in the formula (which makes the distribution of core funding transparent) and, at the same time, the distribution of study places (which is what money is being tied to in the formula) is discretionary (which makes the core funding non-transparent). So, is the distribution of ―core funding‖ transparently distributed or not? The answer is that money is transparently distributed once the Ministry has gone through its non-transparent process of deciding on the distribution of study places. 181. Second, there is very little policy-orientation in the budget process. It is still largely based on historical budgeting, the program structure and the associated medium-term expenditure discussion are largely ceremonial with no signs of it being taken seriously or having any impact 31 The team‘s assessment is that historical factors and universities‘ wishes play the most dominant role in this distribution. The team base this assessment on what the team has been presented in terms of research and analysis (or lack thereof) and discussions with rectors, and staff within the Ministries (of Education and Public Finance) and agencies. Although most of the people interviewed on this aspect acknowledge that this is, indeed, the case, some people interviewed added that, in addition to historical trends there is some analysis behind the distribution. For instance, one counterpart explained that the Ministry uses the ―quality indicators‖ produced by CNFIS to rank universities and, based on this, allocate more spaces to those that are better performing. Another counterpart explained that the distribution is based also based on the evolution of labor market demands for various qualified personnel, and the variation of number of high school graduates. 92 on or relationship to the sector‘s strategic goals. Examples of policy questions which are not discussed in either the annual or medium budget discussions:  In terms of priorities, which fields of study should tax payers‘ money finance? (are we supporting the ―right‖ fields?). And, related to this, what is the best way to use public resources to create a more flexible HE system that is response to the labor market? Is the best way to transfer the money to state universities or would the system be more responsive if the money were allocated to students (e.g. in the form of vouchers?)  What type of students should be financed by tax-payers (core financing): the best or those most in need? (are we supporting the ―right‖ students?)  What objectives are the resources allocated for scholarship – both needs-based and merit- based – and these objectives being achieved? And, is the most effective way to support students through a scholarship program (which, because it is costly, can only reach a limited number of students) or by providing students with loans (which can reach more students)?  Role of private universities in meeting sector‘s objectives (e.g. there is no mentioning of private universities at all in any budget documents, and, as mentioned earlier, there is no estimates of total education (private plus public) spending on higher education).  Role of external (EU) financing: how much is being mobilized, what is it supporting, what complementarities (if any) does it have with state money, and what is the scope for increasing external financing? In practice, there is virtually no communication taking place between the units involved in managing EU funds, and the budget directorate. 182. The usefulness of creating a more performance-oriented (and more policy oriented) budget is not new. In fact, Romania (not just the Ministry of Education) has a relatively long history of being exposed to international consultants providing guidance on creating a more performance- oriented budget and on using non-financial indicators in the budget discussion.  The first external consultants (from the US treasury) started making presentations and providing training 12 years ago. More recently, consultants from the Netherlands have provided further training in how program should be prepared, and what indicators could be used.  A methodology (in the form of a ministerial order issued by the Ministry of Public Finance in 2004) describes how programs should be designed and what information should be submitted as part of the budget discussion.  MERYS was selected as a pilot ministry to develop more performance-oriented budgets and has been submitting budgets in this form for three years. 183. The question, though, is why has the budget not gradually become more performance (and policy)-oriented? In fact, according to officials in the budget directorate in the MFP, the quality of the budget proposals submitted by the MERYS has worsened (with more opaqueness, and less useful non-financial indicators). Below are some possible explanations but there are likely many: 93  Preparing the budget in a new way takes time which most of the actors involved in the process do not have. Director-level staff and above – the very staff who are needed to steer such shift – are pre-occupied with too many day-to-day operational tasks (e.g. one director spends a large part of his day sorting letters from stakeholders with inquiries, complaints and requests to enable him to distribute the task of responding to the letters to his staff).  The demand for a more performance-oriented budget from the very top – e.g. the Ministers, Cabinet or Parliamentary Committee on Education– does not appear to be very strong (e.g. the Parliamentary Committee – on which several universities professors sit – reportedly are content to push for spending to reach 6 percent of GDP, without asking questions about performance). Partly, the problem may be the frequent change of the Minister of Education (e.g. since 2006, there has been five different ministers) and with each Minister being in place, on average, for 1 ½ budget cycle, there simply is not time to push for changes of this magnitude.  Likely, the MFP could have played a more pro-active role in shepherding this shift including using the less formal instruments it has at its disposal (e.g. naming and shaming/praising budget submission from line ministries). MFP officials interviewed for this review explained that they were powerless in mandating MERYS to submit more performance-oriented budgets; MFP has issued a Ministerial order explaining what each line Ministry has to do but MFP has no sanctioning mechanisms to punish those in non- compliance (or reward those ministries which comply). Although its formal sanctioning tools may be missing, it could help if MFP rated the proposals it received (a process which would involve the useful exercise of identifying specific desired features, examples and so forth) and kept track of the quality of the submissions (across time and across Ministries). Such exercise would allow Cabinet with a bird‘s eye view of how far this process has moved along, and whether it has regressed or not. And, for each line Minister, it would provide information as to the quality of his/her budget vis-à-vis other line ministries.  Moreover, part of the problem is the lack of performance information: with data focusing on enrollments (and with no analysis taking place to investigate the linkages (or lack thereof) between spending and performance, it is very difficult to replace the current programs (of which there are six with the largest one called ―creation of a flexible university education, in accordance with the requirements of the labor market‖) with more meaningful ones (see Table 3) below with current program structure as well as indicators used).  Finally, the problem may be one of translating training received into practice. Several actors reported that it would be useful to receive outside hands-on training in creating more performance-oriented budgets during the actual budget preparation. 184. A third area where there is scope for improvement is in the area of providing a vision and strategy for mobilizing more funds for higher education. Currently, there is clearly a recognition 94 (within the counterparts interviewed) that more resources will be needed for higher education beyond what the public sector can mobilize but there is neither a vision nor a strategy with a road map on how such mobilization can be done. Moreover, there is little attempt to integrate the mobilization and use of EU funds with the overall budget process. In fact, there seems to be little discussion about the complementarities (or lack thereof) between activities financed out of state resources vs. activities financed by EU resources. 95 Table 3: Structure of Program Budget for Higher Education Programme General Objectives Lines of action Results Indicators name 6.1.1.Basic financing for state universities, based on the average equivalent cost per Average cost for basic financing per Basic financing student, per field, per cycles of studies, per equivalent student 6.1.Creation of a flexible language of teaching university education, in accordance with the Scholarship funds for Average cost for scholarships per students eligible student requirements of the labor 6.1.2. Efficient granting of scholarships, Average cost for transportation per market Transport allowance transportation facilities, subsidies for physical student hostels and canteens for students. Average cost for subsidy for Subsidies for hostels – hostels-canteens per equivalent canteens student 6. University and 6.2.1. Integration of young people from 6.2.Ensuring equality of changes Roma students registered Number of places assigned for postgraduate underprivileged groups in university level with regard to education for bachelor education Roma students education in training programs. Romania 6.3.Development of tertiary 6.3.1.Consolidating doctoral and Doctoral schools Number of doctoral schools lifelong education postdoctoral education in doctoral schools Increase in the number of 6.4.1.Attracting foreign students in the Number of foreign students who foreign students who study Romanian education system study in Romania 6.4.Internationalization of higher in Romania education in Romania 6.4.2.Granting scholarships for young Increase of Romanian Number of Romanian ethnical people from the Republic of Moldova and ethnic students students from the neighboring area of Romania 6.5.Increase of the scientific Increase of the funds potential, innovation, transfer and 6.5.1.Total value of research agreements coming from scientific Value of income from scientific capitalization of the research carried out by the higher education research, innovation and research, innovation and results carried out by specialized institution per year; technological transfer technological transfer activities structured within HEIs activities 96 Programme General Objectives Lines of action Results Indicators name Budget source: state, 2008 2009 2010 2011 2012 2013 local, European funds Creation of a flexible education system, State budget 2,296 2,297 adapted to the requirements of the labor market Own income 1,895 1,601 Ensuring equality of chances to education University and Development of lifelong tertiary postgraduate education 6 education Internationalization of higher education FEN 0 120 in Romania Increase in the scientific potential, innovation, transfer and capitalization of the results of researches made in specialized structures of higher education institutions TOTAL BUDGET 4,191 4,018 97 Box 9: Common budgeting and financial management problems across the Ministries of Romania As mentioned in the functional review of the pre-university sector, there were a number of financial and budgeting weaknesses that were common across the various ministries. The detailed description of these problems and recommendations on how to address them can be found in the Functional Review of the Ministry of Public Finance but are summarized below:  The process of cash release by the Budget Finance General Directorate (BFGD) to the ‗spending authorities‘ through the monthly ‗credit opening‘ is often delayed, negatively affecting the management of expenditure in the Ministry. The delays result from the large number of checks and verifications which take a long time to go through and sometimes from limited availability of overall funds at the Treasury. The Ministry has experienced cases where cash arrears were generated as a result of late release or insufficient consideration of payment schedules related to existing commitments.  There is no effective system to advise the Ministry and its subordinated agencies that funds have been released. The agencies have to rely on frequent informal checks by telephone with the financial staff in the Ministry and they have to rely, in turn, on frequent checks with Treasury.  The transmission of payment order to Treasury is manual. The Law on Public Finance (500/2002) requires spending authorities to bring paper payment orders (ordonantare) to the Treasury branch offices. The process is cumbersome and time consuming, involving significant paperwork and many signatures. The process of could be automated and done on computer, but problems related to the granting of legal authorization for electronic signature are preventing the automation.  Each main spending authority and all public institutions with a budget of more than € 100,000 must have an internal audit unit. The Ministry has such a unit, but the number of auditors in it is too low and many positions for internal auditors are not filled. The unit is engaged in excessive ad-hoc internal audit missions that impair its effectiveness. Management at the Ministry has a low interest in the unit. It is not clear how efficient and effective is the unit‘s monitoring and control of expenditure.  One peculiarity in the higher education system is the existence of a EUR 500 threshold: for all expenditures above this threshold, state universities need prior approval from the Ministry. Three staff members in the Directorate for Patrimony Modernization, Investments and Informatics of the MERYS are occupied mainly with going through stacks of paper received from universities, requesting permission to purchase items the price of which is above 500 EUR. 185. Fourth, financial reports are not presented in a manner that top level managers (and the Minister) need to manage for performance; budgets and expenditure data are presented the way accountants would organize them (by spending units) – which is suitable when the main purpose of such data is to exert effective control on expenditure – but these reports are ill-suited to ask more policy-oriented questions. Below are some examples of these weaknesses: 98  From an accountancy perspective, it is possible to obtain the details budgets and expenditures of spending made out of the state budget for each agency, and have this total broken down by economic use. But no functional break-down exists for how agencies spent their resources (e.g. spending broken down by ―administrative overhead‖; ―quality assurance of higher education‖; ―monitoring‖ etc).32  Without clarity on the total amount spent on various functions (across different sources of financing and across different buffer institutions), the ministry is severely constrained in its ability to assess whether or not there is scope of making changes or improvements in what the buffer agencies do and how resources are spent across the agencies.  The team was unable to obtain data on total expenditure – from both state resources as well as EU funds – made by each agency.  No centralized data base is kept on budgets and expenditures of each state university. Excel files exists both in the General Directorate of HE in the Ministry and in CNFIS but no one has attempted to create one database that harmonizes how these data are organized and shared within (and outside) of the Ministry. As such, it is impossible for anyone – without direct access to the handful of people who have these excel files on their computers – to conduct detailed analysis of the trends in HE financing in Romania.  To a large extent, the problem of budget and expenditure data is that they are incredibly difficult to get access to. Most glaringly, it requires a Herculean effort to obtain detailed expenditure data needed to analyze the sector‘s efficiency of spending. Data are available on the Ministry of Public Finance website but in pdf files and not in a machine-readable format. The data are presented in such a way that only users with intimate knowledge of Romania‘s budget classification can make sense of it. Similarly, data on enrollments and staff are available on the Bureau of Statistics‘ website but, again, in pdf format, and not separately for each institution. This needs to change if outsiders – whether academics, civil society watchdogs or NGOs – are going to exert effective pressure for better performance.  Finally, Romania seems to lag far behind its EU neighbors in terms of submitting data on higher education to Eurostat. For instance, in Eurostat‘s dataset, only one observation (for 2005) exists for Eurostat indicator ―public and private per student spending on higher education‖ for the period 1998-2007. By comparison, 18 of the EU27 members have submitted such data for either the entire period or for all but one year. Needless to say, this lack of readily available data increases the difficultly with which outsiders – 32 Here is an illustration of the point that budget data are mainly used from an accountancy perspective, not for analysis: the functional team received most of the expenditure data from agencies as pdf files with a stamp and a signature on them; not in excel. That is, the sender wanted to illustrate that the data had been checked, controlled, and that someone was accountable for how it was spent (ie the signature was on the paper). Time and again, the team tried to explain that budget and expenditure could be used for analysis and that, sometimes, data had to be grouped and re-organized in a different way (than the accountants might have organized it) to tease out policy messages. 99 including the authors of this review – can assess performance of the sector, and provide useful policy recommendations. 186. Taken together, these four problems suggest that there is likely much scope for improving value-for-public-money in the sector. Consider the following examples:  Without prioritization in terms of what fields of study to support with public money, money is being spread thinly across too many specializations. There are more than 2,000 programs being supported by public money, at the bachelor‘s level alone (and another 2,500 at the MA level). Without analysis of future demand for workers, without information about employability of graduates, and without information about the quality of the learning that takes place (see section on performance information for more details), there is no reason to suspect that students are a particular strong force in ―voting with their feet‖.  Virtually all public money (with the exception of a small needs-based scholarship program) is merit-based which, likely, implies that the more well-off students (since socio-economic status is closely related to academic achievements) receive more public support than the lesser well-off students (who make it to higher education). With all likelihood, the well-off students would have attended higher education without the state support (because they have the parental support to pay, and because of the high private returns to completing a tertiary education), raising the question: why use 1.5 billion lei per year supporting them? Why not allocate more of these resources toward weaker students and mobilize more resources for students who can pay?  Support to students is provided in the form of ―free places‖ and scholarships which, because such support mechanisms are expensive, can only reach a limited number of students. Likely, a more effective way to reach more students (while simultaneously increasing the number of fee-paying students, as well as tuition fees) would be to provide public support in the form of subsidies student loans. Such discussions have been ongoing since 2007 (with World Bank participation) and an agency has been established but little progress has been made in introducing such loan scheme.  Public money finance students that migrate immediate after they graduate. Although no data exists, interviewed counterparts reported that more than 90 percent of graduates from maritime fields of study migrate after graduation, as well as a large number of doctors. Why use public resources (at all) for programs where the vast majority leaves the country immediately after graduation? For certain fields of study, why not leave the provision of graduates completely in the hand of the private sector (i.e. fully privatize the university)? 100 Recommendations: Recommendations for problem area 1: The large of amount of discretionary decision making in budget allocations (i) Discretionary decision making with regard to capital budgets  Reduce role of Ministry in mobilizing and distributing capital expenditures. Develop a medium-term strategy to let universities tap credit markets to finance capital projects.  When distributing capital funds, do so with transparency and clear policy objectives (beyond ―address capital investment needs‖). For instance, o if the Ministry accepts this review‘s recommendation that assessments of students learning outcomes are needed, the capital investment funds could be used to incentivize universities to participate (i.e. no participation, no capital funds). o Similarly, the capital investment funds could be used to incentivize universities to merge. (ii) Discretionary decision making with regard to distribution of budget places  Develop a methodology to help prioritize the allocation of budget classes. The objective is to move the allocation from a discretionary decision (based largely on historical trends and universities‘ bargaining power) toward a decision based on analysis and future needs. There is no unique way of doing this but the following activities would help on this endeavor: o Commission a skills-forecasts (every three years), working closely with universities and employers associations and sector specific councils. Use the forecast to identify priority sectors. The results from the tracer study (and surveys of employers) would be an important contribution to this discussion. o Distribute public money, not on the basis of the number of ―budget students enrolled‖ but, rather, based on the total number of students enrolled (i.e. include tax-paying students). Needless to say, when doing so, the per student allocation to each university would need to be proportionally smaller. Doing so, would make students voting with their feet a more powerful signaling device. o A more radical approach would be to distribute the core financing in the form of vouchers to be distributed to students. Upon choosing a university and field of study, the student would give the voucher to the university. o Use assessment data on students‘ learning outcomes (if/when these are developed) to help identify priority and high performing sectors to focus on. o Conduct analysis of how many graduates leave the country (by field of study) and use such information to reduce public spending in these sectors. 101 Recommendations for problem area 2: The lack of policy orientation in the budget 187. The objective is to have a budget discussion of a more medium-terms perspective, with more policy-orientation through the use of more non-financial performance data in the discussion (on employment and learning outcomes). Achieving this will involve working simultaneously on addressing several problems:  Freeing up time of director-level staff for more strategic planning (see recommendations in the functional review on pre-university on how to engineer a change in management practices within the ministry).  Creating demand for a more performance-oriented budget from the very top – e.g. the Ministers, Cabinet or Parliamentary Committee on Education. The instruments of achieving this are several fold: o Study tours o Providing concrete examples of how a more policy-oriented budget would look o Several years of hands-on help during budget preparation and discussions from international consultants.  Have MFP (or general secretariat) play an overall guiding and shepherding role by tracking progress over time (a task which would involve developing indicators to measure progress), and naming and shaming laggard ministries.  Develop better performance information (see discussion in section on knowledge gaps on performance data).  Training to MERYS staff in how to use impact evaluations to rigorously evaluate the impact of policy initiatives. Several programs would be suitable to evaluate immediately, including the scholarship program; and there are opportunities for embedded randomized experiments in the design of new policy initiatives (e.g. in the design of the student loan program).  Create more analysis and pressure to perform in the higher education sector by mobilizing outside researchers to analyze existing data (and put pressure on the development of more data). Such pressure can be created by hosting conferences, issuing calls for abstracts, distributing competitive grants to researchers, and inviting more international researchers to examine the Romanian higher education sector. Recommendations for problem area 3: The lack of a vision and a strategy for mobilizing more resources for higher education  A medium-term strategy on how to mobilize more resources for higher education is needed. Irrespective of how what such strategy might recommend, there are some aspects that will surely need tweaking and MERYS could start addressing these immediately: o Increasing cost sharing: having even more students contribute, and increasing the rate of tuition. 102 o For this to work without further worsening equity in higher education, a student loan scheme has to be put in place and the entire system of public support for students (i.e. not only the scholarship scheme but also the decision regarding which students receive budget places) need to be revised. See box for a summary of the recommendations provided by a World Bank TA on designing a student loan scheme in Romania. o Harnessing the potential of the private sector. The new law rightly opens up the possibility for private universities to apply for research grants but this is only a very small step in the right direction. Currently, private universities‘ niche is not research, it is teaching. While they may one day become more research-oriented, creating a level playing field today and fully utilizing the potential in the private sector would imply making, private universities eligible to receive public money for core financing. Needless to say, in exchange for public money, private universities need to demonstrate performance. Again, demonstrating performance requires developing better data on whether graduates find jobs, and what competencies students acquire during the course of their studies. 103 Box 10: Status of the student loan discussion in Romania Three recent steps have made it clear that a student loan scheme will be established in Romania:  The education law which was passed in December 2010 now explicitly mentions that student loans will be made available.33  An agency has been established (with 34 approved positions) to be in charge of implementing it (see  Annex 2 and Annex 3)  A draft methodology for how such scheme might look has been prepared by the agency. However, as will be discussed the next section (VI. Assessing MERYS‘ Capacity for General Management of the Sector), in its current form, the recently established Student Loan and Scholarship Agency does not have the capacity (or managerial flexibility) to implement such scheme. More specifically, it does not have staff with the skills to discuss and weigh the policy implications of designing a scheme in different ways (e.g. what are pros and cons of having a scheme that funds only the best students vs. a scheme that funds the most needy students); calculate the fiscal costs of different schemes; and – if the scheme is designed as a public guarantee of private loans – negotiate terms with private banks. Recommendations: Carrying the work on student loans forward in Romania should involve additional technical support to the Student Loan and Scholarship Agency to:  clarify the type of student loan program (government run versus government guaranteed) that will be pursued.  identify the main short-, medium- and long-term objectives of the scheme. If the objective of the loan program in Romania is to increase participation rates and improve access to higher participation among lower income students who would not otherwise be able to go on to higher education (as stated in the Romanian Student Loan Program Design Work Session Schedule), it has to be designed with a recognition that the existing student financial assistance system is biased to economically better off families (see Discussion Paper) and that access to loans for tuition fees alone is probably not enough for disadvantaged students from rural areas confronting living costs that exceed the costs of tuition. Therefore, in a winner- takes-all system such as Romania, where access to subsidized student housing is merit based, the loans for students from lower income families may need to cover both tuition fees and living expenses.  strategically involve private capital, if a government guaranteed loan program is being pursued (as appeared to be the case following the first TA and in the first stages of the present TA project). It may be decided, for example, that notwithstanding the equity and access objectives of student loans, a smaller pilot project aimed at a particular type of student is needed to allow the banks to gain experience with student lending and to build a track record of repayments. In this case, however, alternative financial assistance programs aimed at low-income students would need to be developed to address their needs in the short term.  design the program with a rigorous impact evaluation embedded in the program (e.g. by having applications fill out an application and use a threshold to select receipients (ie. the treatment group) vs. non-receipients (ie the control group)) to assess the impact of the program on desired outcomes, including drop-out and repetition rates, time it takes to graduate, and employment outcomes.  roll out the scheme slowly – initially focusing on students close to graduation – to quickly gather experience in collecting loan repayments. The hard part of getting a loan scheme to work is not disbursing money; it is collecting them. 33 Article 204 of the new law states: (1) Students coming from low income families benefit from a system of banking study loans, guaranteed by the state, under the conditions of the laws in force, through the Agency for Loans and Scholarship Grants. Loans may cover study taxes and the cost of life during the period of study. (2) The graduates that will practice their profession for a minimum of 5 years in rural environment shall be exempt from the payment of 75% of the loan, the respective amount being taken over by the state, in the maximum amount of 5000 lei. (3) The Agency for Loans and Scholarship Grants proposes appropriate regulations in order to grant loans. 104 Recommendations for problem area 4: The inadequacy of the reporting formats which are used to monitor spending in higher education  Create a single data base of budgets and expenditures (using the same budget classification as employed by the MFP) to track expenditure of every agency and every state university.  Make this data base publicly available on MERYS website.  Develop a ―functional classification‖ for the activities of the agencies to generate more meaningful budget reports (e.g. what core functions does ARACIS carry out and how much does it spend on each)? Having such break-down by function would make a functional analysis significantly easier to conduct on a more regular basis.  A positive spillover by issuing ―performance contracts‖ with buffer agencies (as recommendation elsewhere in this review) is that such contracts can serve as a vehicle to generate more meaningful budget discussions. Eventually, such contracts would necessitate spelling out core functions and outputs and have to involve costing them and tracking them.  In the absence of anyone analyzing budget and expenditure data, it can be difficult to identify gaps. Therefore, host a conference on the financing of higher education and ask domestic and foreign researchers to analyze the financing of higher education.  Submit data to Eurostat  Use household budget surveys to annually estimate private spending on higher education. VII. Assessing MERYS’ Capacity for General Management of the Sector A. Introductory Comment 188. The most important tasks of leadership are (a) to provide direction for action, (b) to mobilize resources for the action, (c) to convince people/organizations to act and then (d) to oversee the action. The capacity of the Ministry to lead the higher education sector depends on the extent to which it can formulate and develop clear and good vision, mission and strategies. The capacity to develop and use these tools of directions, as well as to oversee the results of action (which implies monitoring) depends, in turn, on the Ministry‘s ability to ask the right questions, gather the right data, carry out the right analysis and use the information in the 105 determination of direction. All of these have been dealt with in two previous chapters: the chapter on The Management of Information and the chapter on Strategic Management of the Education Sector. The capacity of the Ministry to mobilize resources for the action it wants universities to take depends, among others, on the types of resources in question. There is a difference between its capacity to mobilize financial resources and its capacity to mobilize human resources. The capacity to mobilize financial resources refers not only to the Ministry‘s ability to get a proper share of the national budget, or see a proper share of GDP devoted to higher education, but also to its ability to allocate its share of the national budget (or influence the self-allocation process of GDP) in line with its vision, mission and strategies. Both have been dealt with in the Chapter on Financial Management in the Higher Education Sector. The capacity to mobilize human resources refers to the Ministry‘s ability to obtain for itself and for its subordinated agencies, councils, boards and authorities the number of staff, the skills and competencies needed to lift the work load. This will be discussed in this chapter. Finally, the capacity of the Ministry to convince universities to act in line with its sector vision, mission and strategies, as well as to generally manage the sector, will be discussed in this chapter too. 189. The instruments that the Ministry has to steer and manage the higher education sector are its twelve subordinated, intermediary organizations and the Directorate General for Higher Education in the Ministry. Its capacity depends totally on them. Therefore, the discussion in this chapter will focus on their capacity. It is important to note, however, that all of the existing intermediary organizations are being transformed now, and two new ones are being created, based on the new Education Law. There are significant changes in the roles, organization structures, personnel, financial resources, management structures, as well as work processes of all existing organizations. Methodologies, which are the tools that shape work processes, are being developed now. Because of this, all of the review team‘s findings and conclusions are based on a situation that does not exist any longer making it impossible to assess the capacity of these organizations now. The team has therefore focused on findings and conclusions that are common to all, rather than on each organization individually, and is presenting these nevertheless since being aware of them the Ministry will be able to take them into account in designing new roles, structures and processes. Because of this, it is also not necessary to relate to each and every intermediary organization separately. Only general findings and conclusions will be presented. The discussion will be split into two sections: a section on the intermediary organizations‘ capacity and a section on the capacity of the Directorate General for Higher Education. 190. One further introductory comment needs to be made at this juncture. It relates to the definition of capacity applied by the review team in assessing the capacity of the intermediary organizations and the directorate general in the Ministry. The definition is as follows: Capacity is the quantity of goods/services that can be produced in a given unit of time, to given specifications or standards, with given resources, when all of these resources are fully and efficiently utilized 106 191. Capacity is not the actual quantity of goods and services being produced in a given time at the required standard. Organizations often have unutilized capacity. In other words, they can produce more if they utilize their resources fully and if their work practices are well designed so that when they do use their resources, they use them efficiently. This depends very much on three factors: the way the organizations are organized, the way they manage themselves and the skills, competencies and motivation of their staff. The findings and conclusion discussed below relate to these three factors. 192. To illustrate this conceptual framework concretely, here is a definition of capacity as it relates to one of the key intermediary organizations: The capacity of ARACIS to accredit study programs is the number of programs that it can process for accreditation in a given unit of time (say, a year)at the required standard of assessment with the current quantity of human resources and financial resources when these resources are fully and efficiently utilized. 193. ‗Capacity‘ does not equate with ‗work load‘. Work load refers to the number of units of a good or a service that an organization has to produce within a given time and at given standards/specifications. When the number of units required exceeds the number of units that the organization can produce, one of two reasons explains it: either the organization has capacity but is not fully utilizing it – usually, because of poor organization, work processes, management etc. - or the organization does not have sufficient capacity – usually, not enough people or money. Sometimes, both of these reasons explain why an organization cannot cope with its work load. Work Load Capacity B. Factors that have Influenced the Capacity of Intermediary Organizations so far and May Influence their Capacity in the Coming Years 194. As indicated in the chapter on sector structure, there are currently 11 intermediate organizations. Three of them are conducting large-scale on-going operations in the higher education sector. These are the organizations that define qualifications for all professions and occupations - the University Qualifications Agency (UECNCFPA), the organization that provides accreditation to universities - the Romanian Agency for Quality Assurance in Higher 107 Education (ARACIS), and the organization that grant attestations and habilitation to persons who obtained doctoral degrees and reached a level of expertise allowing them to supervise doctoral students. This used to be ACPART, but its operations have moved now to the Executive Unit for Financing Higher Education, Scientific Research, Technological Development and Innovation (UEFISCDI). The Executive Unit now helps to all councils and boards. 195. The operational work load of these organizations is huge. The qualifications agency is dealing with 3,850 occupations, of which 2,670 require education below university level and 1180 require tertiary education at ISCD levels 4 and 5. So far, 250 occupations have been validated on a pilot basis. The bulk of the work will be validation work and will come after finishing this summer the DOCIS project which is providing mainly for the setting up of the National Qualifications Framework and the installation of the electronic platform that will host the Romanian Register of Qualifications. The accreditation agency is going to temporarily authorize and/or fully accredit 521 study programs in 2011 and 624 in 2012. It will also conduct assessments of 16 higher education institutions in 2011 and 13 institutions in 2012 for the purpose of accreditation. In each of these assessments, at least 20% of all programs have to be reviewed. The Executive Unit, which is serving many advisory organizations, is dealing with an extremely large and varied work program. The attestation work of the National Council for Attestation of Titles, Diplomas and University Certificates alone will require it to review between 2000 to 4000 requests for attestation in 2012 and some 2000 request annually thereafter. The habilitation work will require it to review some 500 applications annually. 196. In reviewing the history of the intermediate organizations, the review team found that all of them were established, structured and staffed without regard to expected workloads. Organization structures, staff numbers and many other operational details that, under any management theory are left to managers at facility level, were determined at the time of enactment on the basis of factors not related to work load, or to accepted good management practices, and were fixed by law. For an example of this, consider the Government Decision ( Annex 2) which describes the organization and role of the ―Student Loans and Scholarship Agency‖: the Government Decision establishes that the agency must have one General Director and two Directors and, ―at most 34 positions‖ (Article 6). Even more restrictive, the Ministerial Order that follows it lays out an organizational structure and a detailed list of positions the agency can have (Annex 3). In interviews, it was explained to the team that such details are needed to make spending of the agency conform to available budgets (and ensure probity of spending). However, there are other ways to assure conformity of spending with budgets which do not deny managers the operational discretion and flexibility needed to design and manage work and to respond to changing circumstances, including changes in roles, functions and work volume, efficiently. To take this example a step further, one way that this can be done is by introducing a performance contract between the Ministry and the Student Loan Agency, where indicators of performance as well as the Agency‘s accountability for the use of financial and 108 other resources are specified. Neither the Government Decision nor the Ministerial Order refers to such a performance contract. 197. Organization Structures – the structures are inadequate. In two of the three organizations they ignor in one way or another some proven principles and practices in organization structuring. The grouping together of tasks into functions and functions into organizational units is sub-optimal in some cases and fully inefficient in others. The hierarchical arrangements among the units are not right, and in some cases the lines of authority are too long and complex, making it difficult to establish the needed unity of command and the necessary control. Every superior may of course have a number of subordinates, but in most cases every subordinate should have only one clear superior, but in some intermediary organizations this principle is not followed. Also the span of control at least in one organization is much too long. 198. Financial Resources - When the Government decided to go ahead with the plans to introduce the national qualifications framework, the system for attestation and habilitation and the system for accreditation, it did not assess the longer-term financial implications of its decision. No financial planning for the running of these systems, once developed, was done at that time. The Ministry relied heavily on European project funds in developing and putting in place the three big systems. The systems are now built or reaching completion and the infrastructure is in place. In the areas of accreditation and attestation the systems have been in full operational mode for some time, under pilot conditions. However, the funds received from European sources were aimed at the development of the infrastructure and not the regular on- going operations thereafter. European financial support is not going to be available for on-going operations. Funding has to come from other sources and the two possible sources are user charges and the Ministry‘s budget. When the team reviewed with the relevant agencies their financial capacity, none had yet developed financial plans for the future. With the exception of accreditation - one service provided by ARACIS - neither unit costs, nor the number of units, or workload, were calculated by the organizations involved. It was not clear how much money will have to come from the national education budget, whether or not it will be possible to charge clients and how much could be charged. Financial planning is still lacking. 199. ARACIS is the only one that is charging institutions for its mandatory accreditation services and is self-sufficient in this line of business, but it is getting new work that may not be funded by European funds, such as the classification of universities and the rating of study programs. Even if ARACIS does get European project funds to put the systems for classification and ranking in place, maintaining the systems thereafter will not be so funded. 200. Human resources - The work of the three agencies is constrained, or could soon be constrained, by two factors. The first has to do with a heavy reliance of the three agencies on external expertise. The second has to do with the small number of full-time regular staff that has to mobilize and manage the external experts. In all four areas of work – qualification, 109 accreditation, attestation and habilitaton – these agencies use panels of experts to do the needed assessments. The experts are needed for a short time, but for many assessments. The Executive Unit, following procedures set up by the National Council for Attestation of Titles, Diplomas and University Certificates requires 3 evaluators for every case of attestation and 5 evaluators for every case of habilitation. ACPART, the accreditation agency requires 11 to 19 experts for each institutional evaluation and 4 to 5 experts for each assessment of a study program. UECNCFPA, the Qualifications Agency, requires 3 experts for the formulation and registration of each new qualification. All of these agencies expect the experts to come predominantly from the Romanian academic community (and, except for accreditation, to do this work free of charge.) 201. These expectations are unrealistic. To take one example, the Council for Attestation and the Executive unit are planning to have between 2000 and 4000 requests for attestation in 2012 and about 2000 requests annually thereafter. This translates into 6000 to 12000 ‗units‘ of participation – each at least 3-days long - by expert evaluators in 2012 and 6000 units annually thereafter. They are expecting also to have 500 requests for habilitation annually, for which they will need to get 2,500 units of participation, each up to 7-days long. This represents the demand for experts coming from one of the three agencies only. The numbers are going to be two to three time higher when the demand from the other two agencies is added. The team believes that the pool of experts in Romania is not large enough to provide such large numbers of participation units, each lasting between 3 to 7 days regularly, whether on a voluntary basis, free of charge, or even at a reasonable fee. The pool can be much larger of course if experts from other countries are invited, but the likelihood that they will come regularly free of charge is very low – and even if they do, the costs of bringing them over will be considerable. As was indicated earlier, no financial planning has been carried out so far to calculate such (and other) costs. The analysis in this paragraph suggests that planning for human resources too has not been carried out yet. 202. Most of the external experts at the moment are professors in Romanian universities. This opens the door to conflict of interest. There is a great need to involve experts from abroad in sufficient numbers and strengthen their advisory role so as to improve the quality and integrity of the evaluations and assessments. Each of the organizations involved has already started, or is planning to, get more international experts involved, but this renders the process even more complex logistically. 203. The number of full time staff available to the intermediary organizations either in the Ministry or internally is very small relative to their workloads. Although the bulk of the work load in accreditation, attestation, habilitation and the design of qualifications is done by the external experts, these experts have to be mobilized and managed. The number of full-time internal staff is too small whether relative to the number of qualifications, attestations, habilitations and accreditations to be done, or relative to the numbers of external experts to be 110 mobilized and managed to do them. A very small number of internal staff is operating a force of external experts that runs into the hundreds and thousands. 204. Work Processes and Management - In each of the organizations, work processes and procedures are being considered or reconsidered now. In each, target dates for the completion of the first round of work are being set now and – in the case of accreditation - decisions are being made on the frequency of re-assessment and re-accreditation. 205. Although the operations of the three organizations delivering accreditation, attestation, habilitation and qualifications services are logistically huge and complex, requiring high-level operational management, there are few, if any, professional managers in the them. In one organization there is even no position for a general manager. There is a council of 15 people that meets monthly and a board of five persons that manages daily operations. All are full time university professors and they take turns daily at managing the organization. 206. The practice of taking university professors to fill key management positions in the intermediary organizations is widespread. Almost all management positions in all of these organizations are filled by university professors. The practice raises two issues: the first has to do with their ability to fully discharge their responsibilities in two full-time jobs, since there is not enough time in the day to do this well; the second has to do with their ability to manage well, since most, if not all, of them come from disciplines that have little to do with management and don‘t have significant management experience – particularly not in the running of large-scale operations. Recommendations 207. The Ministry and all of the intermediary organizations are currently busy developing the secondary legislation, the methodologies, that will clarify and specify the many changes that were promulgated in the new Education Law with respect to their roles, functions and operations. It is recommended that once this is done, each of the three implementing agencies will embark on a major organization development exercise with the objective of ensuring that it will have full capacity to carry out its role in the system most effciently and effectively. It will be very useful for the Ministry to obtain technical assistance funds with which to acquire management consulting services for this purpose. 208. The exercise should focus on the following:  Developing a tool that can be used to assess methodically the unit-based work load that the organization will have to deal with in the short, as well as, medium term and then conducting the actual assessment 111  Reviewing the design of all types of assessments with the aim of simplifying processes and procedures and optimizing the mix of resources used; much attention should be paid to the following: (a) the impact of current recognition and accreditation norms and standards on the volum of work, (b) the balance between full-time regular staff and external experts, (c) the balance between Romanian and international experts and (d) increasing the use of information technologies so as to reduce the time required of internal staff as well as external experts  Optimizing the organization structure of each of the three intermediary organizations  Conducting human resources planning both for the immediate future and for the next 10 years to determine optimal staffing needs  Conducting financial planning both for the immediate future and the next 10 year to establish clearly funding requirements as well as sources of funds and ensure a stable flow of funds  Designing a management development training program and training of all managers 209. The team recommends further the following four recommendations:  That the organization structure of each of the intermediate organizations will have a position for a full-time professional General Manager as well as positions for full-time unit managers for all organizational units  That the Ministry appoints full-time professional managers for all management positions disallowing them undertake additional full-time jobs simultaneously.  That the Ministry avoids specifying organization structures, staffing levels and other operational matters in primary and secondary laws; and  That the Ministry uses the new student loan scheme as a pilot to develop and introduce a performance contract, based on measurable performance indicators, between it and the Student Loan Agency. C. The Capacity of the General Directorate for Higher Education in the Ministry 210. The answer to the question ‗does the General Directorate (GD) have capacity?‘ depends on the answer to another question: ‗capacity to do what?‘ The reason is that the role being played by the GD today is not necessarily the role it should and could play in the future. Today the GD‘s role is predominantly a low to mid-level operational role, while in the future it can be a combination of mid-level and high-level operational role combined with a role in the development of policies and strategies. The capacity to play a strategic role is different to the capacity to play an operational role. The answer to the question is then that for the role being 112 played today, the DG has almost sufficient capacity, but for the role it should play in the future, its current capacity is inadequate and will have to be built in a significant way. 211. The reason why the GD is playing a relatively low-level operational role today is complex. Part of it has to do with the general tendency in the Ministry to shift some of its key strategic functions, both at pre-university and at university levels to intermediary organizations. Another part of it has to do with the structure of the Ministry, where a significant role is being played by Secretaries of State. When there is in the Ministry a Secretary of State for Higher Education, the tendency is for strategic matters to be dealt with by the Minister and the Secretary of State and to be kept at that level. But, the reason why much of the discussion is kept at that level is the capacity of the GD for Higher Education. In other words, its capacity to provide strategic inputs into the discussion is very limited. 212. The GD does have one extremely important ingredient of capacity to play a role in strategic matters: a huge data base on higher education in the country. But it does not have enough people to maintain it – let alone to do much analysis of the data. The data base is being managed and run by one person who is overwhelmed with the data. If the reports received by the review team are correct, only that one person knows well enough how to work with it, how to do anything more than the simple extraction of data from it. This is a highly undesirable situation from a risk management point of view, but it is also undesirable from a capacity management point of view, since it means that the ability of the GD to do some analysis proactively or to respond to request reactively is very limited. 213. Currently, then, most of the work carried out by the GD is routine, operational work consisting of the checking, recording and tracking of documents and responding to requests for information. The GD consists of two directorates: one called Directorate for Institutional Funding Strategies and the other called National Center for Recognition and Equivalence of Diplomas. The operational work in this Center is different to that of the first-mentioned directorate, requiring more decision-making, but the decisions are usually simple and the rules for making them have been determined at higher levels and are fixed. According to the DG, the Center is not managed by the Directorate General, but ―only coordinated‖. In discussing the work of the Directorate General, the Director General reported to the review team that in 2010, the Directorate for Institutional Funding Strategies accomplished the following activities:  Responded to 3100 external and internal written requests for information and 2,400 external complaints - a total of 5500 items  Responded to 4000 telephone requests for information and explanation  Dealt with, made comments on, 120 normative acts  Processed 5000 PhD theses  Processed the records of contests for professorship or associate professorship in 1 500 positions  Processed the records of 500 persons requesting the authority to lead PhD students 113  Processed the files of 100 requests for doctoral school status (IOSUDs) 214. The National Center for Recognition and Equivalence of Diplomas accomplished the following activities:  Dealt with 7835 requests for equivalence  Dealt with 5772 requests for recognition  Checked the records of 2886 foreign students requesting visas to enter Romania  Commented on 260 Ministry Orders and 11 legislative proposals 215. The constraint on the GD‘s capacity is only partly a matter of numbers. The review team believes that it is also, and primarily, a matter of skills and competencies. There are 25 staff in the Directorate for Institutional Funding Strategies and 21 staff in the National Center for Recognition and Equivalence of Diplomas. Of the 46 persons, 12 are full-time university professors. According to the DG, ―only some 5 persons are currently strong enough to do strategic work and 10 could be trained to do it‖. Recommendations 216. The review team believes that there is a need to completely reform the role of the DG for Higher Education in the Ministry and that the Ministry can benefit from technical assistance in this area. The starting point needs to be a review of all its routine operations - the rationale for and utility of these operations, the variety of processing procedures and the parts played by the DG in these procedures. Once this is done, it is necessary to re-design the role of the General Directorate, bringing into it more strategic functions. Again, the starting point here may be to bring in the function of inter-agency coordination in the higher education sector. As was discussed earlier in the chapter on the structure of the sector, there are now 11 intermediary organizations in the sector, each with a variety of operations that need to be coordinated not only internally, but also externally with the operations of the other organizations. The large number of intermediary organizations has made coordination a strategic function – a function that cannot be shifted outside the Ministry. This strategic function needs to have a dedicated organizational home in the Ministry and in theory the home should be the General Directorate for Higher Education. In practice, adding intra-sector coordination to the functions of this directorate can take place only when the capacity of the directorate is greatly improved. The team recommends that once the role and functions of the directorate are re-designed, the Ministry will also design and then implement a capacity building program for it, which will allow the Ministry to move the coordination function to the directorate. Key elements in this capacity building program will have to be the strengthening of leadership, a major improvement in accessibility to, and user- 114 friendliness of, the data base as well as a major improvement in the ability of the directorate to do analytic work. 217. While the Directorate General for Higher Education is the main directorate dealing with higher education in the Ministry, a number of other directorate are carrying out functions which cover partly higher education too. When the role of the General Directorate is redesigned it will be necessary to keep them in mind as well and ensure that no functional overlaps are created. One operation in the action plan for the Pre-University Functional Review, which is to be launched in a month or two, is a thorough examination of the structure of the Ministry of Education in its entirety. The objective of this operation is to come up with a better organizational structure and the outputs will include TORs (or ROFs) for each organizational unit as well as job descriptions for each key type of a job. In this context, all roles and functions including those focusing on higher education, and all jobs related to them – in all of the Ministry‘s directorates - will be analyzed; gaps and overlaps will be identified and TORs will be re-written. 115 Box 11: Summary of findings on Human Resource Management from Functional Review of Pre-university sector The situation is not as clear with respect to the mix of skills and levels of competency in the Ministry and its external agencies. The Ministry and its agencies do not have strategies for training and development. No skills- needs analyses have been conducted in them in recent years. All managers are university trained; most have a master degree and some have a Ph.D. But most, if not all, come from a background of teaching and few, if any, have education, training or experience in administration, general management, project implementation, customer relations, etc. This ingredient – the knowledge, skills and experience - is missing in the mix. Competency can be acquired through experience on the job and some of these managers may have acquired a reasonable level of competency in management; but the responses of managers to the Management Practices Questionnaire indicate significant shortcomings in this skill area. Among the rest of the staff, most people are university educated and at least one third has a master degree. The most serious skills gaps are in two areas: general management and project management and implementation. The second has been discussed earlier in this report. There are simply not enough project managers, procurement specialists and accountants in the mix and those that are there do not possess the required levels of competency. The first is discussed below. The roles of the managers in the Ministry, as well as the entire management culture, are in need of re-definition. There has been a general move in management cultures all over Eastern Europe from the authoritarian models of the past to the more modern HR model of the present. This model which emphasizes decentralization, participation, teamwork, flexibility and performance management, is regarded in most countries today as ‗best practice‘. In the Ministry, more than 2/3 of the mana gers and the staff are 40 years old or older. This means that they were educated before the fall of Communism in a very rigid, non-free-thinking school environment. They have been in the Ministry a long time in a highly centralized, hierarchical and legalistic culture, in which bureaucratic formalism and conformity prevailed. Roles, functions and jobs were enshrined in legal documents and the people performing them were not able, or did not wish, to take any action that would seem to go beyond their narrowly-interpreted responsibilities. Today, new knowledge, high-level thinking, conceptualization and analysis are required. Attitudes relating to responsibility, initiative and risk-taking need to change too. This requires fresh blood, but will have to be accomplished also with current personnel through intensive staff development, as well as important changes in the rules, procedures and practices surrounding managers. Based on the Management Culture and Practices Questionnaire and information collected in the interviews, managers in the Ministry are talking about staff development, but they do not link staff development to performance appraisal and actually do not get opportunities to go on, or send their staff for, training. The situation in the external agencies is somewhat similar, though not as pronounced. In these agencies there is a general shortage of people in all professional areas and specific skills‘ gaps in some, such as statisticians, curriculum specialists and test-writing experts. In the Institute of Education Sciences, only 26 of 56 professionals are PhDs; some are in the process of getting the degree. Being a research institute, where the required levels of competency are usually acquired at the Ph.D. level, this suggests that there is room for improvement. Based on the Management Practices Questionnaire, as well as the interviews, it may be deduced that management cultures in these agencies are changing faster than in the Ministry. One can observe a higher level of energy and creativity in them. Yet even in these agencies there is still a need for people with higher-level analytical skills in whatever they do. As one of the directors summarized, ―for me, the lack of skills relates to the ability to conceptualize…, I need people who can be more analytic and can develop policies - people who can respond to European initiatives, prepare proposals, join European/international working meetings and represent us well... Right now there are only two of us who can do this job. I need more.‖ Source: World Bank (2010): Functional Review of the Romanian Pre-University Sector 116 VIII. Annexes Annex for Tables and Figures Annex table 1: Income of state universities 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 Core funding 175 270 380 477 633 847 1,041 1,181 1,730 1,913 Complementary funding 94 157 299 363 350 372 483 1,496 2,040 2,671 Own source revenue 109 167 237 385 489 618 723 860 967 1,169 Total income 378 595 915 1,226 1,472 1,837 2,248 3,537 4,737 5,753 Share of total income Core funding 46% 45% 42% 39% 43% 46% 46% 33% 37% 33% of which: based on equivalent students 46% 45% 42% 38% 42% 45% 40% 27% 27% 23% of which: based on quality indicators 0% 0% 0% 1% 1% 1% 6% 7% 9% 10% Complementary funding 25% 26% 33% 30% 24% 20% 22% 42% 43% 46% Own source revenue 29% 28% 26% 31% 33% 34% 32% 24% 20% 20% of which: student fees 18% 21% 20% 25% 26% 27% 25% 20% 16% 16% Total income 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% Share of GDP Core funding 0.32% 0.34% 0.33% 0.32% 0.32% 0.34% 0.36% 0.34% 0.42% 0.38% Complementary funding 0.17% 0.20% 0.26% 0.24% 0.18% 0.15% 0.17% 0.43% 0.49% 0.53% Own source revenue 0.20% 0.21% 0.20% 0.25% 0.25% 0.25% 0.25% 0.25% 0.23% 0.23% Total income 0.69% 0.74% 0.78% 0.81% 0.75% 0.75% 0.78% 1.03% 1.15% 1.14% 117 Annex table 2: Educational attainment of 25-29 year olds, by location and by income quintile, 2002 and 2009 2002 Middle High Post- Primary school school secondary or No formal (grades (grades Vocational/ (grades foremen's Higher schooling 1-4) 5-8) Apprentice 9-12) school education Total Rural 1.8 4.1 27.5 26.2 35.6 2.0 2.8 100.0 Urban 0.6 1.4 6.8 17.3 47.5 6.8 19.7 100.0 Total 1.1 2.6 15.7 21.1 42.4 4.7 12.4 100.0 Quintile 1 (Poorest) 3.8 8.4 32.7 22.5 31.0 0.5 1.0 100.0 2 0.6 3.3 23.5 26.5 41.2 1.8 3.2 100.0 3 0.5 0.8 13.7 24.5 50.1 3.9 6.5 100.0 4 0.7 0.4 8.0 21.0 48.3 7.7 13.8 100.0 Quintile 5 (Richest) 0.0 0.1 2.8 12.6 41.6 9.1 33.9 100.0 Total 1.1 2.6 15.7 21.1 42.4 4.7 12.4 100.0 2009 Middle High Post- Primary school school secondary or No formal (grades (grades Vocational/ (grades foremen's Higher schooling 1-4) 5-8) Apprentice 9-12) school education Total Rural 2.1 7.4 33.3 25.4 22.9 1.7 7.1 100.0 Urban 1.5 1.5 7.8 14.7 35.4 5.6 33.4 100.0 Total 1.8 3.8 17.7 18.8 30.6 4.1 23.3 100.0 Quintile 1 (Poorest) 5.5 11.1 40.4 23.0 15.4 0.9 3.8 100.0 2 2.8 5.7 27.0 27.9 28.5 1.8 6.4 100.0 3 0.4 2.2 14.6 21.6 39.0 3.2 19.0 100.0 4 0.4 0.7 8.2 17.4 41.2 6.3 25.8 100.0 Quintile 5 (Richest) 0.0 0.2 2.4 8.3 29.6 7.1 52.4 100.0 Total 1.8 3.8 17.7 18.8 30.6 4.1 23.3 100.0 Source: Authors‘ calculations based on Household Budget Survey, 2002 and 2009 118 A: Evolution in per student financing in Romanian HE 119 Source: CNFIS powerpoint entitled ―Higher Education Funding System in Romania‖ 120 B: Example of sub-indicator behind main indicator: details of IC6 1. The capacity to obtain funding for the scientific activity 1.1 The initiative to obtain research funding, from national and international level 1.2 Number of projects gained through national and international competitions 1.3 Funds obtained from national and international competitions, through projects/research contracts, consultancy/technical and technological services (at national and international level, including the contracts signed directly with international companies) 2. The capacity to train human resource highly developed for the scientific research activity 2.1 The degree of implication into training the human resource highly developed for the scientific research activity 2.2 The efficiency of the training the human resource highly developed for the scientific research activity 3. Relevance and visibility of the scientific research activity results 3.1 Article published in journal, recognized at the international level – indexed by ISI Web of Science, from the principal flux of publications without an impact factor, BDI indexed, published in the conferences volumes of the indexed ISI conferences and/or those organized by international professional societies 3.2 Article published in journal, recognized at the national level, by CNCSIS, the B, B+ şi C- categories 3.3 Books published at national and international publishing houses (printed and/or electronic forms) 4. The capacity of the universities to design/develop products – innovative technologies for the economy market 5. Institutional capacity of the universities to organize and maintain the performance of the scientific research activity C: Explaining the notion of an “equivalent student” (translation of MERY document which describes the base finance distribution methodology)The notion of equivalent student and equivalence coefficients by form of education. The notion of equivalent student was introduced in order to express, in mathematical terms, the fact that a student‘s higher education incurs different costs, depending on the form of education (program of studies) one is enrolled in. Moreover, the aim was from the outset to foster advanced programs of studies (for master‘s and doctor‘s degrees) by putting the financial conditions in place for a better quality of such programs. The notion of equivalent student is also aimed at other forms of university training or activities that financial resources are allocated to, namely: initial teacher training (Rom. seminar 121 pedagogic), preliminary training of foreign students in Romania or activities related to awarding teacher grades in pre-university education. These are special student categories which are treated separately, regardless of the field of study. Equivalent student numbers are determined from the number of physical students eligible by law for State-budgeted financing (in the broader sense specified above) as stated by universities for each form and field of education at the reference time. Equivalent student numbers are computed for each university and field of education as a weighted total of physical student numbers enrolled in the forms of education provided by the university in the respective field. The weights carried by the forms of education are called equivalence coefficients and are the ratio of the financial effort required to train a student enrolled in education form f to the financial effort needed to train a student enrolled in a long- term undergraduate (or 1st cycle) form of education, in Romanian, in the same field of study. Mathematically, this is conveyed by the following relation: (1) Where: SEUd = equivalent student numbers in field d at university U SEUfd = physical student numbers in field d, education form f enrolled at university U at reference time Ef = equivalence coefficient for education form f F = total number of state-budgeted forms of education in Romania‘s universities (for this year F = 19) 122 Annex Table 3. List of Education Forms and Their Equivalence Coefficients for Year 2010 (as per MER Decision) No. Form of education – f - Equivalence coefficient - ef - I. Bachelor‘s degree (undergraduate) programs 1 Higher education studies in Romanian (short and long-term) 2 Higher education studies in Hungarian – as a mother tongue (short and long- term) 3 Higher education studies in German – as a mother tongue (short and long- term) 4 Higher education studies in widely spoken foreign languages 5 Higher education studies in languages of limited circulation 6 Higher education studies provided as part of extension courses – abroad 7 Higher education studies – evening classes 8 Higher education studies – part-time classes (viz. limited attendance requirements -T.N.) 9 Distance higher learning* II. Further training or master‘s degree programs** 10 Master degree or further studies in Romanian 11 Master degree or further studies in widely circulated foreign languages 12 Master degree or further studies as part of extension courses – abroad III. Doctoral programs 13 Doctoral degree classes (except for engineering, agronomy, science and medicine) 14 Doctoral degree classes in engineering, agronomy, science and medicine 15 Extramural doctoral studies (begun prior to 2006/2007 academic year) IV. Other forms of training 16 Resident term (only for 1st cycle, years 1-2) 17 Foreign students‘ preliminary training (preparatory year) 18 Additional teacher classes 19 Activities related to awarding teacher grades in pre-university education Note:*) Only for fee-paying students, to compute the quality indicators covering all students (budgeted and fee-paying) **) For last-term students enrolled in a 1 ½ -year master‘s program, the last two months of this year are considered. For students starting their second year of a master‘s program in October, corrections will be determined under the existing legislation. (1) Coefficient applied to studies taken in German throughout (2) Coefficient applied to the fields of study the cost coefficient of which is under 1.75. As from October, the correction was also applied for the 1 ½ -year master‘s degree program. 123 Initial budget as % of actual expenditure Initial budget as % of final budget -10% 20% 40% 60% 80% 100% 120% 140% 160% 180% 200% 10% 30% 50% 70% 90% 110% 130% 150% 0% 85 Communications 81 Fuels and energy 81 Fuels and energy 61 Public order and national… 61 Public order and national… 68 Social security and welfare 68 Social security and welfare 84 Transport 84 Transport 67 Culture, recreation and… 67 Culture, recreation and… 70 Housing, services and… Average across main… Average across main… 51 Public authorities and… 83 Agriculture, forestry, fish… 83 Agriculture, forestry, fish… 86 Economic research &… 82 Mining industry,… 54 Other general public… Annex Figure 1: Initial budget as % of final budget 86 Economic research &… 51 Public authorities and… 53 Basic research and… 82 Mining industry,… 70 Housing, services and… 74 Environmental protection 124 Annex Figure 2: Initial budget as % of actual expenditures 74 Environmental protection 53 Basic research and… 60 Defence 65 Education 87 Other economic actions 66 Health 66 Health 60 Defence 54 Other general public… 80 General economic,… 65.06 Higher education 87 Other economic actions 65 Education 65.06 Higher education 80 General economic,… 85 Communications 2006 2008 2006 2008 Annex Table 4: Distribution of Cycle I (Bachelor) Programs in Romanian Universities in the Academic Year 2010-2011 by Field of Study, Number of Programs and Number of Universities Offering them Number of Cycle I Number of No. Study Field (Bachelor) Study Universities Programs Offering Them 1 Business administration 86 53 2 Accounting 63 48 3 Administration sciencies 78 46 4 Finance 62 46 5 Law 53 43 6 Management 51 39 7 Engineering and management 94 36 8 Communication sciences 66 34 9 Marketing 40 33 10 Philology 109 32 11 Environmental engineering 47 30 12 Informatics 41 28 13 Social assistance 28 27 14 Physical education and sports 56 27 15 History 50 25 16 Theology 85 25 17 Psychology 26 23 18 Education sciences 66 23 19 Intl. relations and European studies 27 22 20 Cybernetics, statistics and ec informatics 29 22 21 Computers & information technology 34 22 22 Sociology 29 21 23 International business and economics 26 21 24 Mechanical engineering 67 21 25 Industrial engineering 78 20 26 Systems engineering 24 20 27 Electronical engineering & telecom 43 20 28 Electronical engineering & telecom 43 20 29 Mathematics 49 20 30 Electrical engineering 47 19 31 Electrical engineering 47 19 32 Political sciences 23 18 33 Modern applied languages 21 18 34 Music 49 18 35 Food product engineering 36 17 36 Geography 47 17 37 Environment sciences 41 16 38 Environment sciences 41 16 125 Number of Cycle I Number of No. Study Field (Bachelor) Study Universities Programs Offering Them 39 Environment sciences 41 16 40 Visual and decorative arts & design 66 16 41 Biology 20 16 42 Mecatronics and robotics 25 15 43 Energetical engineering 28 15 44 Engineering in geodesy 15 15 45 Physics 35 15 46 Performing arts (Theater) 28 14 47 Philosophy 17 13 48 Chemistry 23 13 49 Economics 19 12 50 Chemical engineering 40 12 51 Chemical engineering 40 12 52 Engineering of materials 26 12 53 Civil engineering 39 12 54 Horticulture 17 11 55 Engineering of autovehicules 15 11 56 Cultural studies 19 10 57 Agronomy 16 9 58 Forestry 13 8 59 Engineering and instalations 10 7 60 Engineering of transportation 7 7 62 Military sciences and information 18 6 63 Zootechnics 10 6 64 Cinema and media 6 5 65 Bio-technologies 10 5 66 Geological engineering 6 5 67 Aerospatial engineering 9 4 68 Naval engineering and navigation 9 4 69 Forest engineering 5 3 70 Mining, oil and gas engineering 7 4 71 Geology 3 3 72 Urbanism 3 1 73 Engineering of ammunition & armament 5 1 Total = 2562 126 Annex Table 5: The Level of Autonomy in the Romanian State Higher Education System Compared with Other European Countries Category Are the Universities Free Decisions are Made Today by The University The University Makes Decisions The Government, [including Faculty] based on rules prescribed by Ministry [by No Government Government, or subject to primary or approval required approval of Government secondary law] 1. Organizational 1.1 To make decisions on internal In Romania Autonomy academic structures, e.g., number/type of X [GD] faculties In Europe 21 10 3 1.2 To make decisions on internal In Romania structures of governing bodies, e.g., X Senate, Council – a general statement In Europe 29 1.2.1 Specifically, to decide whether to In Romania have dual or unitary governing body – x Council/ Board and Senate In Europe In most countries it is a dual structure of board and senate which is a more representative body 1.2.2 If dual, to decide what will be the In Romania division of power between the X Council/Board and the Senate In Europe Usually one of the two is primary decision maker and the other advisory 1.2.3 To decide who should be X represented in Council/Board,Senate 1.2.4 To decide what external bodies X should be in governing bodies and how they should be selected 127 Category Are the Universities Free Decisions are Made Today by The University The University Makes Decisions The Government, [including Faculty] based on rules prescribed by Ministry [by No Government Government, or subject to primary or approval required approval of Government secondary law] 1.2.5 To decide whether they can take X part in decisions – and if so, what decisions 1.3 To decide what will be the X composition of the executive body of university and what competences will be required of members 1.4 To elect/appoint the rector In Romania X In Europe Minority 22 have provision on required qualifications 1.5 To determine the rector‘s term of In Romania office X [No distinct provision in the law] In Europe 8 24 1.6 To dismiss a rector In Romania X x 1.7 To decide whether the rector is part In Romania of the governing body or is external but x accountable to this body. In Europe In 21 the rector is part of the governing body; in 9 not 2. Financial 2.1 To allocate/distribute [block] funds In Romania Autonomy internally, according to own needs X 128 Category Are the Universities Free Decisions are Made Today by The University The University Makes Decisions The Government, [including Faculty] based on rules prescribed by Ministry [by No Government Government, or subject to primary or approval required approval of Government secondary law] In Europe 26 7 2.2 To decide whether or not to charge In Romania tuition fees and administrative fees from X whom and how much In Europe 10 2.3 To keep and build reserves from self- x34 generated funding 2.4 To accumulate surpluses from state In Romania funding. X Europe 26 7 2.5 To borrow money on the financial In Romania markets X35 In Europe 22 12 2.6 To issue shares and bonds In Romania Not allowed In Europe 34 It is not forbidden by the law and therefore means that universities can build reserves. However, all the financial resources of state universities have to be kept in public treasury accounts, which impose high charges and bear no interest on deposits. 35 State universities cannot use their physical inventory to guarantee loans, so practically they cannot borrow. For state entities, the Ministry of Public Finance could agree to guarantee a loan as it is doing for state companies borrowing from abroad. 129 Category Are the Universities Free Decisions are Made Today by The University The University Makes Decisions The Government, [including Faculty] based on rules prescribed by Ministry [by No Government Government, or subject to primary or approval required approval of Government secondary law] 11 19 2.7 To own the land and buildings they In Romania occupy X In Europe 18 8 2.8 To sell the land and buildings they In Romania occupy x In Europe 8 12 1 3. Staffing Autonomy 3.1 To recruit staff Romania x Europe 12 16 6 3.2 To appoint senior academic staff Romania x36 Europe 28 4 3.3 To determine individual salary levels Romania 36 Senior academic staff (bellow age 65) can be hired by universities after the habilitation by CNATDCU. That means that universities do not have to provide professional files of the habilitated staff in CNATDCU/MERYS for each promotion in their career. The publication in the Official Gazette is, however compulsory for the state universities. In order to publish that announcement the HEI has to get the (formal) visa from MERYS. After retirement at the age of 65, senior academic staff can be hired only for 1 year. The labor contract has to be renewed annually. 130 Category Are the Universities Free Decisions are Made Today by The University The University Makes Decisions The Government, [including Faculty] based on rules prescribed by Ministry [by No Government Government, or subject to primary or approval required approval of Government secondary law] of academic staff X Europe 4 decide fully + 6 15 8 decide some categories 4. Academic 4.1 To introduce a degree program Romania Autonomy X Accreditation required Europe 2 32 require some type of accreditation 4.2 To terminate a degree program Romania X37 Europe 4.3 To determine overall numbers of Romania students X ARACIS Europe 6 9 by negotiation with authority; 5 5 split 37 Universities have to inform the MERYS when its Senate decides to terminate a degree program 131 Category Are the Universities Free Decisions are Made Today by The University The University Makes Decisions The Government, [including Faculty] based on rules prescribed by Ministry [by No Government Government, or subject to primary or approval required approval of Government secondary law] 4.4 To decide numbers of students per Romania field of study or domain, or program x38 Europe 10 13 2 + 6 where the state may decide Romania 4.5 To set admission criteria X Europe 16 9 admission free by law + 9 based on grades by law 38 ARACIS accredits every study program. It gives a temporary authorization for the first 3 years following a review and then a permanent one following another review. 132 Annex Table 6: Indicators monitored in annual report on education A. The whole of the education system B. Preschool education C. Primary and Gymnasium Education I. Costs of education 1. Public expenditure for education, as % of the GDP 1. Share of expenses for preschool education from the 1. Share of expenses for primary total expenses for education and gymnasium education from the 2. Public expenditure for education, as % from the total 2. Staff expenses 2. Personnel expenses public expenditure 3. Public expenditure for education, as % from the total 3. Average cost per child 3. Average cost per pupil expenses for education 4. Staff expenses, as % din total expenses for education 5. Average cost on pupil /student II. Human resources 1. Share of qualified teaching staff of the education 2. Share of full time teaching staff system 3. Proportion of female teaching staff 4. Number of children per teacher III. Participation to 1. Gross rate of school coverage for all levels of education 1. Gross coverage rate in preschool education 1. Gross school coverage rate in education and primary and gymnasium education internal efficiency 2. Average duration of education attendance 2. Average duration of preschool education attendance 2. Average duration of primary and of the education gymnasium education attendance system 3. School life expectancy 3. Specific rate of school coverage on ages in preschool 3. Specific rate of school coverage education on ages 4. Specific rate of school coverage on ages and sexes 4. Share of pupils entered for the first time in 1st 4. Rate of school abandon grade which attended the preschool education 5. Rate of early abandon of the education system IV. Results a. Re sults of pupils at inte rnational e valuations 1. Graduation rate b. Re sults of e ducation on the labor marke t 2. T he Results of pupils at national evaluations (evaluation for 4th grade) 1. Share of active population (15-64 years), on levels of 3. T he results of pupils at national education evaluations (exam of capacity/evaluation 8th grade) 2. Occupation rate of population aged 15-64 years, on levels of education 3. Occupation rate of young people aged 15-24 years, on levels of education 4. Unemployment rate for young people aged 15-24 years, on levels of education 5. Insertion rate of graduates of various levels of education and vocational training on the labor market 133 D. Secondary and Vocational Education E. Post-secondary Non – tertiary/ F. Continuous Vocational (SAM) tertiary Education training I. Costs of education 1. Share of expenses for secondary /vocational education 1. Share of expenses for post-secondary / higher education 1. Share of expenses for FPC from the from the total expenses for education from the total expenses for education total expenses for education 2. Personnel expenses 2. Personnel expenses in post-secondary / higher 3. Average cost per pupil education 3. Average cost per pupil /student II. Human resources of the education 1. Gross ratio of school coverage in secondary /vocational 1. Gross ratio of school coverage in post-secondary / 1. Adult participation rate (25-64 system education higher education years) to education and vocational 2. Average attendance duration of secondary /vocational 2. Share of students enrolled for mathematics, sciences training education and technology 3. Specific rate of school coverage on ages 3. Average attendance duration of post-secondary /higher education 4. T ransition rate in secondary /vocational education 4. Specific rate of school coverage on ages 5. Rate of school abandon 5. T ransition rate in higher education 6. Access rate to higher education 7. Rate of school abandon in post-secondary education III. Participation to education and 1. Graduation Rate 1. Graduation Rate internal efficiency 2. Results of pupils at school-leaving exam /final exams 2. Share of graduates in the areas of mathematics, of the education science and technology system IV. Results Source: MERYS 2009: Annual Report on the Education System Note: Indicators are monitored and tracked in the annual report on the education system. The analysis presented in this report (which has been written in the same way since 2005) is done according to 32 basic indicators, grouped along the following four dimensions: i. education expenses; (i) ii. human resources of the education system; (ii) iii. participation to education and internal efficacy of the education system; (iii) iv. results: a. pupils‘ results; b. results of education on the labor market. 134 Annex Table 7: Estimates of rates of return in education (1) (2) (3) (4) table1_2009 table2_2009 table1_2002 table2_2002 VARIABLES lwage lwage lwage lwage years of education 0.0533*** 0.0426*** (0.000893) (0.000908) vocational 0.135*** 0.0858*** (0.0106) (0.0111) lower high school 0.0806*** 0.0237 (0.0167) (0.0184) high high school 0.233*** 0.169*** (0.0103) (0.0112) special post-secondary 0.388*** 0.296*** (0.0149) (0.0140) college 0.470*** 0.379*** (0.0206) (0.0238) university 0.645*** 0.494*** (0.0121) (0.0130) expert 0.0149*** 0.0151*** 0.0237*** 0.0250*** (0.00102) (0.00103) (0.00106) (0.00107) Expert 2 -0.0253*** -0.0256*** -0.0436*** -0.0474*** (0.00226) (0.00228) (0.00255) (0.00258) female -0.174*** -0.181*** -0.180*** -0.186*** (0.00561) (0.00569) (0.00554) (0.00559) married 0.0378*** 0.0398*** 0.0532*** 0.0535*** (0.00692) (0.00687) (0.00707) (0.00704) capital 0.140*** 0.132*** 0.0604*** 0.0535*** (0.0125) (0.0125) (0.0126) (0.0125) Other urban 0.0686*** 0.0620*** 0.0628*** 0.0584*** (0.00642) (0.00643) (0.00654) (0.00654) Constant 5.635*** 6.056*** 13.56*** 13.91*** (0.0200) (0.0176) (0.0193) (0.0178) Observations 18304 18304 17892 17892 R-squared 0.315 0.323 0.244 0.252 Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 Note: Omitted in this table is that fact that regional and monthly dummies were added to the regression Follows: Yemtsov, Cnobloch and Mete 2006 specification 135 Annex 1: National Approaches to Learning Outcomes Assessment in Higher Education Annex Table 8: National Approaches to Learning Outcomes Assessment in Higher Education -- Assessment Processes / Organizations involved Single, cross- Test Name, Responsibility for Frequency Initiator of Test Administration of sectional or Country Introduction test design and of Reporting of assessment results Development assessment longitudinal Date analysis assessment testing Australia Graduate Skills Federal Government: Australian Council for Administered on HEI Twice a year, Cross-sectional Personalized reports for students, containing Assessment Department of Educational Research campus, supervised by once for entry assessment of total score on each component and relative (GSA), 2000 Education, Science and (ACER) faculty and once for students at entry achievement compared to all other Training (DEST), exit level and participants of the same year. HEIs receive formerly Department of examinations. students at data on student performance and institutional Education, Training and graduation reports. Youth Affairs (DETYA) level. Australia Course Federal Government: Graduate Careers Questionnaires are Annually Single testing For each HEI, the GCCA analyses and Experience Department of Council of Australia supplied by GCCA and reports data to DEST. The GCCA also Questionnaire Education, Science and (GCCA); Australian sent out by HEIs reports back to each HEI. Results are made (CEQ), part of Training (DEST), Council for public in a variety of aggregations and levels the Graduate formerly Department of Educational Research of detail by universities, GCA, ACER and destination Education, Training and (ACER) the government. The press and a commercial survey since 1993 Youth Affairs (DETYA) publication ("The Good Universities Guide") draw on results to establish rankings for public consumption Australia Graduate Federal Government: Graduate Careers Questionnaires are Annually. Single testing For each HEI the GCCA analyses and reports Destination Department of Council of Australia supplied by GCCA and data to DEST. The GCCA also reports back Survey (GDS), Education, Science and (GCCA), Australian sent out by HEIs to each HEI. Results are made public in a 1972 Training (DEST), Council of Educational variety of aggregations and levels of detail by formerly Department of Research (ACER), universities, GCA, ACER and the Education, Training and University of government. A national file, national tables, a Youth Affairs (DETYA) Melbourne’s media release and GradStats (a four-page Information summary of results) are publicly available. Technology Service (ITS) 136 Single, cross- Test Name, Responsibility for Frequency Initiator of Test Administration of sectional or Country Introduction test design and of Reporting of assessment results Development assessment longitudinal Date analysis assessment testing Brazil Exame Nacional Federal government CESGRANRIO Organized by Instituto Annually. Single testing Results were disclosed through technical de Cursos (ENC (specialized Nacional de Estudos e reports, institutional bulletins (per area of or "Provão"), assessment agency) Pesquisas Educacionais study and course), and student bulletins. The 1995-2003 'Anísio Teixeira' (INEP) students' individual bulletins were only on a nation-wide testing available to the students themselves. Annual date, administered by course classifications were made public and municipalities rankings were taken up by the media. Brazil Exame Nacional Federal government CESGRANRIO and Organized by Instituto Once every Cross-sectional Results are disclosed through technical de Desempenho CESPE (specialized Nacional de Estudos e three years. assessment of reports, institutional bulletins (per area of dos Estudantes assessment agencies) Pesquisas Educacionais freshmen and study and course), and student bulletins. The (ENADE), 2004 'Anísio Teixeira' (INEP) senior students students' individual bulletins are only on a nation-wide testing available to the students themselves. Results date, administered by are reported in a discrete manner that draws municipalities little attention from the media. Canada National Federal Government: Statistics Canada Answers are collected Periodically. Longitudinal Individual student or HEI information is not Graduate Survey Human Resources (StatCan) for HRDC by regional StatCan NGS in 1978, survey of the disclosed. HEIs may obtain their own (NGS), 1978. Development Canada offices through 1984, same students aggregated institutional results. Public-access Follow-up (HRDC) is the primary Computer Assisted 1988,1992, two years after data files are only identified by the type of Survey of sponsor and funder Telephone Interviews 1997, 2002; graduation and record (HEI type and region). Graduates, 1987 (CATI) Follow-up five years after surveys three graduation years later on the original respondents (since 1987) Chile Exit exam for medical studies Colombia ECAES In February 2008, Constitutional Court declares unconstitutional 137 Single, cross- Test Name, Responsibility for Frequency Initiator of Test Administration of sectional or Country Introduction test design and of Reporting of assessment results Development assessment longitudinal Date analysis assessment testing Jordan Mexico Exámen Nacional Asociación Nacional de Centro Nacional de Administered by Individual Single testing Individual student results are disseminated de Ingreso al Universidades e Evaluación para la CENEVAL on HEI students or electronically and can be consulted by the Posgrado Instituciones de Educación Superior campus. institutions students themselves and by the institution (EXANI-III), Educación Superior (CENEVAL) sign up for pre- they applied for. 1997 (ANUIES), set national testing dates. Mexico Exámen General Asociación Nacional de Centro Nacional de Administered by Individual Single testing Results are confidential. HEIs receive Para el Egreso de Universidades e Evaluación para la CENEVAL on HEI students or aggregate results of their students. Individual la Licenciatura Instituciones de Educación Superior campus. institutions students receive bulletins indicating their (EGEL), 1994 Educación Superior (CENEVAL) sign up for pre- absolute test score. Certificates of (ANUIES) set national achievement are provided for students testing dates. scoring at or above national average Mexico Exámenes Coordinación General de Centro Nacional de Administered by Institutions Single testing Individual student results are confidential. Generales para el Universidades Evaluación para la CENEVAL on HEI sign up for pre- HEIs receive aggregate results of program Egreso del Tecnológicas (CGUT) Educación Superior campus. set national and institutional performance. Individual Técnico Superior (CENEVAL) testing dates. students receive bulletins indicating their Universitario absolute test score. Certificates of (EGETSU), 2000 achievement are provided for students scoring at or above national average UK Destinations of Federal Government: Higher Education Questionnaires are Annually. Single testing Aggregate results are publicly available on Leavers from Department for Statistics Agency supplied by HESA and the internet. HEIs receive information on the Higher Education Education and Skills (HESA) commissioned administered by the results of their graduates. (DLHE), 2002 (DfES) and other expert group Careers Offices of each (replaced the government bodies HEI. "First Destination Supplement") 138 Single, cross- Test Name, Responsibility for Frequency Initiator of Test Administration of sectional or Country Introduction test design and of Reporting of assessment results Development assessment longitudinal Date analysis assessment testing USA Collegiate Not applicable ACT HEIs order materials Flexible Multiple forms Institutional Summary Report, student roster Assessment of and administer them to of each module reports, student score reports, certificates of Academic students allow for pre- achievement for students scoring at or above Proficiency and post-testing national average, up to three previously (CAAP), 1988 (cross-sectional specified subgroup reports (e.g. by gender, or longitudinal ethnicity or major). Sub-scores are given for assessment) each test component. USA Measure of Not applicable Educational Testing HEIs order materials Flexible Multiple forms Aggregated student score reports, student Academic Service (ETS), The and administer them to allow for pre- roster reports, institutional summary score Proficiency and College Board students and post-testing reports. Sub-scores are given for each Progress (cross-sectional component. National data available by class (MAPP), 2006 or longitudinal level and by Carnegie classification (replaced the ETS assessment) (institution type). MAPP scores are fully "Academic comparable to Academic Profile scores. Profile" test, 1992-2006) USA Tasks in Critical Not applicable Educational Testing HEIs order materials Flexible Single testing Scores are reported as the percentage of Thinking, 1992 Service (ETS) and administer them to students demonstrating proficiency in each of students the three skill areas as measured by the tasks. No national data available. USA Major Field Not applicable Educational Testing HEIs order materials Flexible Single testing Reports include individual proficiency Tests, 1990 Service (ETS) and administer them to scores, departmental summary with (based on the students department mean-scaled scores, and GRE Subject demographic information. Percentile tables Tests) for all seniors taking the current form of each test are published each year. Departments may purchase subscores or group assessment indicators that the tests may support. USA and National Survey The Pew Charitable National expert team Questionnaires are sent Annually Cross-sectional Institutional Reports for individual HEIs. Canada of Student Trust chaired by Peter Ewell out by NSSE (a joint survey of National comparisons by academic level and Engagement (NCHEMS) venture between the freshmen and by Carnegie classification (institution type) (NSSE), 2000 (in Indiana University and senior students. publicly available. Canada since NCHEMS) 2004) 139 Annex Table 9: National Approaches to Learning Outcomes Assessment in Higher Education -- Assessment Format and Design Criterion- Duration of Country Test Name, Introduction Date Instrument Format Number of items referenced or Assessment norm-referenced Australia Graduate Skills Assessment (GSA), Examination Multiple-choice test One multiple-choice test and two writing tasks Three hours. (2 hours Norm 2000 and writing tasks (one reporting task and one argument task) for the multiple-choice test and one hour for writing tests) Australia Course Experience Questionnaire Survey Survey 25 items. Questions rating student satisfaction Five to ten minutes Not applicable (CEQ), part of the Graduate and generic skills development on a scale of 1-5 destination survey since 1993 (where 1 = Strongly disagree and 5 =Strongly agree). Australia Graduate Destination Survey (GDS), Survey Survey 19 items on employment, 6 items on further Five to ten minutes Not applicable 1972 study Brazil Exame Nacional de Cursos (ENC or Examination Information not Information not available. Four hours Norm "Provão"), 1995-2003 available. Brazil Exame Nacional de Desempenho dos Examination Objective questions 30 field-specific questions and ten general study Four hours Norm Estudantes (ENADE), 2004 and essay questions questions Canada National Graduate Survey (NGS), Survey Survey The CATI questionnaire of the NGS is 113 pages Not available Not applicable 1978. Follow-up Survey of long and contains 18 sections. Most respondents Graduates, 1987 answer only a portion of the questions within each section. Some respondents skip entire sections that are not applicable to them. Mexico Exámen Nacional de Ingreso al Examination Multiple-choice test 120 test items. 54% (66 items) test general Four hours Norm Posgrado (EXANI-III), 1997 intellectual abilities, 46% (54 items) test competencies in information use Mexico Exámen General Para el Egreso de la Examination Multiple-choice test Depending on the subject area, 100 to 300 items. Several sessions on a Criterion Licenciatura (EGEL), 1994 Up to 20% of the items may be pilot items that weekend (in total, are not counted in the student evaluation. between eight and ten hours on average) Mexico Exámenes Generales para el Egreso Examination Mutliple-choice test 250 test items. 35% (87 items) constitute the Six hours (two Criterion del Técnico Superior Universitario general area component. 65% (163 items) sessions of three hours (EGETSU), 2000 constitute the specific area component. each) UK Destinations of Leavers from Higher Survey Survey 56 items Approximately 15 Not applicable Education (DLHE), 2002 (replaced minutes the "First Destination Supplement") 140 Criterion- Duration of Country Test Name, Introduction Date Instrument Format Number of items referenced or Assessment norm-referenced USA Collegiate Assessment of Academic Examination Multiple-choice test Users can choose among six different skill 40 minutes for each Norm Proficiency (CAAP), 1988 and essay questions modules. Each module has up to 72 questions. module Users may add up to nine additional items. USA Measure of Academic Proficiency Examination Multiple-choice test, Long form contains 108 multiple-choice Two standard forms Criterion- and norm- and Progress (MAPP), 2006 optional essay questions and takes 100 minutes. Short form (two-hour tests) and referenced (replaced the ETS "Academic question contains 36 questions. Optional essay is six abbreviated forms Profile" test, 1992-2006) available. Users may add up to 50 multiple- (40-minute tests) choice skill or knowledge questions USA Tasks in Critical Thinking, 1992 Examination Open-ended questions Information not available. 90 minutes for each Norm and problem-solving task tasks USA Major Field Tests, 1990 (based on Examination Multiple-choice test Information not available. Two hours (three Norm the GRE Subject Tests) hours for MBA) USA and National Survey of Student Survey Survey About 90 questions in five broad areas of Approximately 15 Not applicable Canada Engagement (NSSE), 2000 (in engagement minutes Canada since 2004) 141 Annex Table 10: National Approaches to Learning Outcomes Assessment in Higher Education -- Participants Test Name, Target Selection of Coverage of test Country Introduction Incentives for Participation Population participants application Date For test takers For institutions Australia Graduate Skills Students at entry Limited, self- About 2000 students from about Students may add their At entry level: results help identify poorly Assessment level and students at selected sample 20 universities from a variety of GSA score to their performing students to follow up and offer (GSA), 2000 graduation level (Voluntary fields of study participate each curriculum vitae when assistance. At graduation level: results may be assessment) year. they apply for a job. used as an additional criterion for graduation or for entry into post-graduate courses. Results provide information on "value-added" by the institution and on general education learning quality across courses and HEIs Australia Course Experience Graduates who General Surveys are sent to all recent HEIs receive feedback on their students' Questionnaire completed assessment of all graduates of Australian HEIs results. HEIs may analyze and report on their (CEQ), part of the requirements for qualified (including international students). results for internal purposes. Graduate any higher candidates Response rates are around 60- destination survey education 65%. since 1993 qualification in the previous calendar year Australia Graduate All graduates who General Surveys are sent to all recent HEIs receive feedback on their students' Destination Survey completed assessment of all graduates of Australian HEIs results. HEIs may analyse and report on their (GDS), 1972 requirements for qualified (including international students). results for internal purposes. any higher candidates Response rates are around 60- education 65%. qualification in the previous calendar year Brazil Exame Nacional Students at General All graduating students from Participation was Participation was mandatory. Courses and de Cursos (ENC or graduation level assessment of all courses within previously defined mandatory: Reception of institutions receive feedback regarding the "Provão"), 1995- qualified areas of study. In 2003, testing graduation diploma is performance of their students compared to 2003 candidates covered more than 460,000 conditional to students in the same field enrolled in other students enrolled in 6,500 participation. Awards were courses/HEIs. courses totaling over 70% of all given to students with the graduating students best performances. 142 Test Name, Target Selection of Coverage of test Country Introduction Incentives for Participation Population participants application Date Brazil Exame Nacional Students at entry A random sample Students at entry level and Participation is mandatory Participation is mandatory. Courses and de Desempenho level (having is selected from a students at graduation level from for the randomly sampled institutions receive feedback regarding the dos Estudantes covered between list including all courses within previously defined students: Reception of performance of their students in absolute and (ENADE), 2004 7% and 22% of qualified students. areas of study. In 2004, the graduation diploma is relative terms. As opposed to the "Provão", curriculum) and If courses have randomly selected sample conditional to ENADE does not divulge results via the students at less than 20 included 140,340 students participation. Awards are media. Results are communicated more graduation level students, all enrolled in 2,184 courses, given to students with the discretely through bulletins on the internet. (having covered at students are tested. totaling to 51.4% of freshmen best performances. least 80% of students and 68.5% of graduating curriculum) students. 2,830 students that were not randmonly chosen singned up for voluntary participation Canada National Graduate Graduates from all A sample is Sample sizes range from 35,000 Individual HEIs are not the focus of Survey (NGS), public HEIs selected using a to over 40,000 graduates. assessment, thus no possible negative impact. 1978. Follow-up (universities, stratified design to Participation is voluntary. The HEIs receive feedback on their graduates' Survey of colleges, trade provide accurate average response rate is around labor market outcomes. HEIs may analyse and Graduates, 1987 schools). NGS: two estimates by 65% report on their results for internal purposes years after province, program, graduation. Follow- and field of study up Survey: five years after graduation (same respondents) Mexico Exámen Nacional Graduates applying General All graduates applying for Participation is mandatory Individual HEIs are not the focus of de Ingreso al for entry into post- assessment of all admission or scholarships in for entry into some post- assessment, thus no possible negative impact. Posgrado graduate study students applying post-graduate institutions that use graduate programs. Allows Post-graduate HEIs receive reliable, objective (EXANI-III), 1997 programs for post-graduate the test. 13,604 applicants took students to provide and comparable results on student admission or the test in 2006. evidence of their performance. scholarships in proficiency to apply for institutions that scholarships. use the test 143 Test Name, Target Selection of Coverage of test Country Introduction Incentives for Participation Population participants application Date Mexico Exámen General Students from non- HEIs decide Students may sign up voluntarily. Gives a complementary HEIs receive feedback on their students' Para el Egreso de technical HEIs whether to assess HEIs may sign up sample groups and nationally-comparable results. HEIs may analyse and report on their la Licenciatura having covered students. HEIs of their students. qualification to students. results for internal purposes. Results may be (EGEL), 1994 100% of the decide on size and Certificates of used as an additional criterion for graduation curriculum characteristics of achievement are provided or for higher-level course entry (graduated or not) the student sample. for students scoring at or Individual students above national average may sign up for voluntary participation. Mexico Exámenes Students from General All graduate level students of all Gives a complementary HEIs receive feedback on their students' Generales para el technical HEIs assessment of all 48 technical HEIs throughout the and nationally-comparable results. HEIs may analyse and report on their Egreso del Técnico having covered qualified country qualification to students. results for internal purposes. Results may be Superior 100% of the candidates Certificates of used as an additional criterion for graduation Universitario curriculum achievement are provided or for higher-level course entry (EGETSU), 2000 (graduated or not) for students scoring at or above national average UK Destinations of Recent graduates General Questionnaires are sent to all Individual HEIs are not the focus of Leavers from from publicly- assessment of all qualified students. In 2002/03, assessment, thus no possible negative impact. Higher Education funded HEIs who qualified 77% of the full-time qualifiers HEIs receive feedback on their graduates' (DLHE), 2002 obtained a relevant candidates (251,300 students) and 70% of labor market outcomes. HEIs may analyse and (replaced the "First qualification and the part-time qualifiers (60,900 report on their results for internal purposes Destination who studied full- students) responded. Supplement") time or part-time (approximately six months after graduation) USA Collegiate All types of students States or HEIs Between 1988 and 2001, the test Incentives for test takers HEIs receive feedback on their students' Assessment of decide whether to has been used by more than 600 may be provided by some results. HEIs may analyse and report on their Academic assess students and HEIs and more than 450,000 HEIs. Motivation results for internal purposes. Results may Proficiency determine the size students questions on the objective provide information on "value-added" by the (CAAP), 1988 and characteristics tests help determine how institution and on general edcuation learning of the student seriously students took the quality across courses and HEIs. sample. test. 144 Test Name, Target Selection of Coverage of test Country Introduction Incentives for Participation Population participants application Date USA Measure of All types of students States or HEIs The Academic Profile (1992- Incentives for test takers HEIs receive feedback on their students' Academic decide whether to 2006) on which the MAPP is may be provided by some results. HEIs may analyse and report on their Proficiency and assess students and based has been used by 375 HEIs HEIs. results for internal purposes. Results may Progress (MAPP), determine the size and 1 million students provide information on "value-added" by the 2006 (replaced the and characteristics institution. Results show student performance ETS "Academic of the student in general edcuation compared to students Profile" test, 1992- sample. from other courses/HEIs. 2006) USA Tasks in Critical All types of students States or HEIs Between 1992 and 2001, the test Incentives for test takers HEIs receive feedback on their students' Thinking, 1992 decide whether to has been administered at 35 may be provided by some results. HEIs may analyse and report on their assess students and institutions to 200-500 students HEIs. results for internal purposes. Results may determine the size at each institution provide information on "value-added" by the and characteristics institution. Results show student performance of the student in critical thinking compared to students from sample. other courses/HEIs. USA Major Field Tests, Senior students (4- States or HEIs More than 500 colleges and Test is often given as a HEIs may incorporate the assessment into 1990 (based on the year colleges) decide whether to universities employ the test per capstone course or in the their course syllabi and make the exam a GRE Subject assess students and year. In the 1999-2000 academic last semester of study as graduation requirement. HEIs may analyse and Tests) determine the size year, more than 1,000 part of a graduation report on results for internal purposes. Results and characteristics departments from 606 HEIs requirement. show student performance in the specific area of the student administered the test to nearly of study compared to students from other sample. 70,000 students. HEIs. USA and National Survey of First-year and A random sample Since 2000, almost 1,000 HEIs receive feedback on their students' Canada Student senior students (4- is selected from a different North American results. HEIs may analyze and report their Engagement year colleges) list including all universities and colleges have results for internal purposes. Results allow to (NSSE), 2000 (in first-year and administered NSSE to more than identify aspects of the undergraduate Canada since senior students. 1,160,000 students. Minimum experience that can be improved through 2004) sample sizes are determined by changes in policies undergraduate enrollment (sample sizes vary between 450 and 1,000 students per HEI) 145 Annex Table 11: National Approaches to Learning Outcomes Assessment in Higher Education -- Outcomes assessed and results Test Name, Focus of Type of results Country Introduction Type of outcomes assessed Use of assessment results assessment yielded Date Australia Graduate Skills Broad abilities: Critical Thinking, Problem General education HEIs: At entry level: identify poorly performing students to follow up Assessment Solving, Written Communication (ACER is results of entry-level and offer assistance. At graduation level: use results as an additional (GSA), 2000 currently considering modifications such as students and graduation- criterion for entry into post-graduate courses. Information on "value- the addition of basic skills, management skills, level students added": benchmark and analyze trends, document/demonstrate IT skills, research skills). Domain-specific program effectiveness and improvement over time, compare students' knowledge and abilities: (Not yet included achievement levels with national user norms, develop and improve but ACER is currently considering the curricula, determine student eligibility for upper-division studies. possibility of testing elements within various Government: Collect information on the quality of learning outcomes broad Field of Study groups) Non-cognitive across HEIs for national and potentially international benchmarking of outcomes: Interpersonal understanding graduate skills. Employers: The Government promotes the test to employers and supports its use as a standard recruitment tool. Australia Course Students, Generic academic abilities: Problem Solving, Graduate satisfaction HEIs: Benchmarking, trend analysis, evaluation of programs, Experience Programs, Analytic Skills, Written Communication with teaching and curriculum development and improvement. Provide national Questionnaire Institutions Skills. Non-cognitive outcomes: Teamwork learning. Self-reported accountability data. Government: Ensure quality and performance (CEQ), part of skills, Confidence in tackling unfamiliar gains in academic skills management within HEIs. Inform student choice. Assess and plan for the Graduate situations, Ability to plan work. Student related to the HEI the needs of the HE sector. Since 2005, results from the CEQ are used destination satisfaction with the following: Teaching, experience. for performance-based incentive funding through the national survey since Goals and Standards, Workload, Assessment. "Learning and Teaching Performance Fund (LTPF)" 1993 Australia Graduate Programs Behavioral outcomes: Employment outcomes Information on HEIs: Benchmarking, trend analysis, evaluation of programs, Destination approximately 4 months after graduation: employment and further curriculum development and improvement to optimize labor market Survey (GDS), availability for employment, sectors of study and further study outcomes. Provide national accountability data. 1972 employment, average annual salaries, Government: Ensure quality and performance management within graduates' job search activities. Further study HEIs. Inform student choice. Assess and plan for the needs of the HE activities, such as mode of study (full/part- sector. Since 2005, results from the GDS are used for performance- time), levels of study, fields of education, and based incentive funding through the "Learning and Teaching institution. Performance Fund (LTPF)" Brazil Exame Nacional Programs Domain-specific knowledge and abilities Inter-institutional HEIs: Good scores were widely used for commercial purposes de Cursos (ENC that are considered essential and common to performance (advertisements & publicity). Results often served to mobilize or "Provão"), all HEI curricula in the specific domain. comparisons between students and professors to make a joint effort to maintain good 1995-2003 Available for 26 subject areas. students from the same scores/improve bad ones. Government: Since 2001, the test served as field of study a guidance for accreditation and re-accreditation, but punitive measures were only taken in extreme cases. General Public: Provão results were widely divulged by the media to inform prospective students and society at large about the quality of learning across HEIs 146 Test Name, Focus of Type of results Country Introduction Type of outcomes assessed Use of assessment results assessment yielded Date Brazil Exame Nacional Programs and Domain-specific knowledge and abilities Differences in cognitive Students: Prove their performance according to national standards to de Desempenho institutions that are considered essential and common to results between entry- potential employers. HEIs: Benchmarking, trend analysis, evaluation dos Estudantes all HEI curricula in the specific domain. level students and of programs, curriculum development and improvement. Provide (ENADE), 2004 Available for 13 study areas. General content graduation-level national accountability data. Government: ENADE is one aspect of a knowledge: Among the assessed themes are students combination of performance indicators used for HEI evaluations. biological and social diversity, public policies, Employers: may ask for a candidate's test results as objective social networks, citizenship, and current evidence of proficiency in the professional area. events and problems. Generic academic abilities: Ability to infer, interpret poetic texts, establish common points, identify associations, reflect, deduct, and understand graphics. Canada National Students, Behavioral outcomes: Graduate satisfaction Information on the HEIs: Benchmarking, trend analysis, evaluation of programs, Graduate Programs and with their HEI experience. Employment integration of graduates curriculum development and improvement to optimize labor market Survey (NGS), institutions outcomes two years and five years after into the labor market. and further study outcomes. Provide national accountability data. 1978. Follow-up graduations: Information on the number, Links between Government: Assess and plan for the needs of the HE sector. Survey of characteristics, and duration of all jobs held education and labor Graduates, 1987 since graduation, on the length of job search, market outcomes. the match between education and occupation. Mexico Exámen Institution types General academic abilities: Verbal and Mastery of the generic HEIs: Use individual student results to compare performance of Nacional de and regions mathematic reasoning, Capacities to infer, academic skills required applicants and to facilitate decision-making on student admission Ingreso al analyze and synthesize. Competencies in to undertake post- and/or scholarship attribution Posgrado information use: organize, obtain and graduate study (EXANI-III), understand information 1997 Mexico Exámen Students Domain-specific knowledge and abilities Cumulative results Students: Prove their performance according to national standards to General Para el that are considered essential and common to (specific to the area of potential employers. HEIs: Benchmarking, trend analysis, evaluation Egreso de la all HEI curricula in the specific domain. study). Mastery of the of programs, curriculum development and improvement. Some HEIs Licenciatura Available for 33 different subject areas. essential knowledge and use results as an additional criterion for certification or graduation. (EGEL), 1994 abilities required to start Employers: may ask a candidate's test results as objective evidence of professional practice proficiency in the professional area. 147 Test Name, Focus of Type of results Country Introduction Type of outcomes assessed Use of assessment results assessment yielded Date Mexico Exámenes Students, Domain-specific knowledge and abilities: Cumulative results Students: Prove their performance according to national standards to Generales para Programs, or Specific technical area component: (specific to the area of potential employers. HEIs: are free to decide how results are used. el Egreso del Institutions comprehension levels and problem-solving study). Mastery of the HEIs may want to assess their students for purposes of benchmarking, Técnico skills needed in the student's major field. Tests essential knowledge and trend analysis, evaluation of program analysis, curriculum Superior are available for all 19 areas of the Technical abilities required to start development and improvement. Results may be used by individual Universitario University Track. Common area component: professional practice HEIs as a criterion for certification or graduation. Employers: may (EGETSU), general academic skills, knowledge and ability ask a candidate's test results as objective evidence of proficiency in the 2000 necessary for all careers, namely social and professional area. economic knowledge, IT, and English. UK Destinations of Students, Behavioral outcomes: Employment and Information on HEIs: Benchmarking, trend analysis, evaluation of programs, Leavers from Programs, or further study outcomes six months after employment and further curriculum development and improvement to optimize labor market Higher Institutions graduation: how many graduates are in study and further study outcomes. Provide national accountability data. Education employment, the types of jobs they go into, Government: Assess and plan for the needs of the HE sector. (DLHE), 2002 and how many go onto further study. (replaced the "First Destination Supplement") USA Collegiate Programs and Generic academic abilities: Writing Depending on simple or HEIs: Satisfy accreditation and accountability reporting requirements, Assessment of institutions (objective and essay), reading, mathematics, value-added benchmark and analyze trends, document/demonstrate program Academic science reasoning, critical thinking, curricular administration: effectiveness and improvement over time, compare students' Proficiency content drawn from all fields Cumulative outcomes in achievement levels with national user norms, develop and improve (CAAP), 1988 general academic skills curricula, determine student eligibility for upper-division studies. or growth in general States: the CAAP has been used to track entire systems over a period academic outcomes of time, using test results as a benchmark for progress within a public while at college accountability framework. State-wide results may be published in a "public report card" and compared with the results of other states. USA Measure of Students, Broad disciplinary abilities: Reasing and Growth in generic HEIs: Satisfy accreditation and accountability reporting requirements, Academic Programs, critical thinking are measured in the context of academic abilities while benchmark and analyse trends, document/demonstrate program Proficiency and Institutions humanities, social sciences, or natural at college effectiveness and improvement over time, compare students' Progress sciences. Generic academic abilities: achievement levels with national user norms, develop and improve (MAPP), 2006 reading, writing, critical thinking, mathematics curricula, determine student eligibility for upper-division studies, (replaced the counsel individual students for academic achievement. ETS "Academic Profile" test, 1992-2006) 148 Test Name, Focus of Type of results Country Introduction Type of outcomes assessed Use of assessment results assessment yielded Date USA Tasks in Critical Students Generic academic abilities: College-level Proficiency in college- HEIs: Satisfy accreditation and accountability reporting requirements, Thinking, 1992 higher order thinking skills: inquiry, analysis, level higher order benchmark and analyse trends, document/demonstrate program communication skills thinking skills effectiveness and improvement over time, compare students' achievement levels with national user norms, develop and improve curricula, determine student eligibility for upper-division studies. USA Major Field Students, Domain-specific knowledge and Mastery of concepts, Students: Test is often given as a capstone course or in the last Tests, 1990 Programs disciplinary abilities that are considered most principles, and semester of study as part of a graduation requirement. HEIs: Scores (based on the important within each major field of study: knowledge expected of may be used for medium to high-stakes decisions. Document GRE Subject factual knowledge, ability to analyze and solve students at the proficiency in the specific area in the last semester of study to measure Tests) problems, ability to understand relationships, conclusion of an effectiveness of departmental curricula. Satisfy accreditation and ability to interpret material including graphs, academic major in accountability reporting requirements, benchmark and analyze trends, diagrams, and charts based on material related specific subject areas. document/demonstrate program effectiveness and improvement over to the field. Available for 15 undergraduate time, compare students' achievement levels with national user norms, disciplines and for MBAs. develop and improve curricula, determine student eligibility for upper- division studies. USA and National Survey Students, Behavioral outcomes: information on student Secondary indicators of HEIs: Develop and improve curricula and services to enhance student Canada of Student Programs, engagement: how undergraduates spend their learning: Information on engagement. Satisfy accreditation and accountability reporting Engagement Institutions time and what they gain from courses, student participation in requirements (NSSE provides an "Accreditation Toolkit" facilitating (NSSE), 2000 extracurricular activities, and HEI services. learning opportunities the use of NSSE results for regional accreditation). Benchmark and (in Canada since during the college analyse trends, document/demonstrate program effectiveness and 2004) experience. Self- improvement over time, compare students' achievement levels with reported gains in national user norms. General Public: Results are public and provide academic skills related information about what students gain from their HEI experiences. to the college Government: Data can be used as an indicator of institutional experience. effectiveness in accrediting processes. Data supports national and sector benchmarking processes. 149 Annex 2: Government Decision on Student Loan Agency Government of Romania DECISION on Establishment, Organization and Functioning of Student Loans and Scholarships Agency Under Article 108 of the (republished) Romanian Constitution and Article 5 para.(1) of the Law No.329/2009 on reorganization of some authorities and public institutions, streamlining of public expenditures, support of business environment and respect of framework agreements with the European Commission and the International Monetary Fund, The Government of Romania adopts this decision. Article 1 – (1) The Student loans and and legal persons), by various states in the scholarships Agency, a specialized body of framework of bilateral collaboration central public administration, is established as agreements or unilaterally, as well as of legal person, under the Ministry of Education, scholarships offered by governmental and Research and Innovation, following the merge international programs. of the following structures, which are abolished: the Loan Agency for Students from (2) Sending the selected candidates to State and Private Accredited Higher Education studies abroad is done on nominal basis, based Institutions and the National Center for on a decree of the Minister of Education, Scholarships Abroad. Research and Innovation. (2) The Student Loans and Scholarships Article 3 – (1) The Student Loans and Agency has its headquarters in Bucharest Scholarships Agency has the following municipality, 1, Caransebes str., 7th floor, competences: Sector 1. a) launches publicly and manages the Article 2 – (1) The Student Loan and scholarship programs offered to the Ministry of Scholarships Agency has the following goals: Education, Research and Innovation by the Government of Romania, public authorities, a) to ensure the access of Romanian foundations, donors (natural and legal persons), students to education abroad, for the purpose of by different states in the framework of bilateral their training and specialization; collaboration agreements or unilaterally; b) to ensure the management of the loan b) launches publicly and manages the system for state and private accredited higher scholarships allocated to the Ministry of education institutions that offer Bachelor Education, Research and Innovation from university studies and Master university governmental and international programs; studies; c) organizes national competitions to c) to provide access to students and award scholarships for studies abroad; graduates of accredited education institutions, the tenure-track teaching staff and other d) develops and publishes its own categories of beneficiaries of scholarships evaluation and selection procedures from offered to the Ministry of Education, Research national competitions for the award of and Innovation by the Government of Romania, scholarships for studies abroad; public authorities, foundations, donors (natural 150 e) at the proposal of the Board of the k) ensures the management of student Student Loans and Scholarships Agency, loan system; establishes on annual basis the number and fields/specializations for which scholarships for l) may submit and implement project studies abroad are awarded, scholarships with external funding from reimbursable or funded from currency payments collected by non-reimbursable funds. the Ministry of Education, Research and Innovation or from state budget allocations (2) The Student Loans and Scholarships within limits of approved budget; a person may Agency may also have other competences, benefit only once in a lifetime from a complementing those provided in para.(1) that scholarship for studies abroad funded from the are set in the regulation on organization and currency payments collected by the Ministry of functioning, approved by decree of the Minister Education, Research and Innovations or from of Education, Research and Innovation. state budget allocations; (3) In exercising its functions, the Student f) makes proposals for drafting the Loans and Scholarships Agency is entitled to: decrees of the Minister of Education, Research and Innovation on awarding the scholarships a) use external personnel, specialists in for studies abroad, in collaboration with the the field, as experts, remunerated in accordance Board of Student Loans and Scholarships with the law; Agency, based on meeting the criteria formulated by the provider of funds, the results b) use external personnel from pre- of candidate selection competitions, as well as university and university education to evaluate in accordance with the general reform strategy the files of applicants for scholarships, of university education and research, promoted expressed in scores; by the Ministry of Education, Research and Innovation; c) select based on a transparent methodology and using competence criteria its g) establishes and publishes the own evaluators, which will be contractual regime of scholarships financed by methodologically prepared and will be asked to the Ministry of Education, Research and act as evaluators of the files in competition. Innovation and other providers of funds, a regime approved by decree of the Minister of Article 4 – (1) The activity of the Student Education, Research and Innovation; Loans and Scholarships Agency is conducted in cooperation with the Board of the Student h) publishes guides and syntheses on best Loans and Scholarships Agency, a national practices in the Member States of the European consultative entity, directly subordinated to the Union and other developed countries regarding Minister of Education, Research and Innovation the award of scholarships for studies abroad; and which consists of 4 tenure-track university professors, with teaching and scientific i) manages the financial resources experience renowned in the country and allocated from state budget or resulted from its abroad, as well as with moral behavior, a own income; Secretary of State from the Ministry of Education, Research and Innovation, the j) develops and proposes for Government Director General of the General Directorate for approval, via the Ministry of Education, Budget, Finance, Assets and Investments, the Research and Innovation, methodological Director General of the General Directorate for norms on establishing the loan fund, the Higher Education within the Ministry of methodology concerning the amount and Education, Research and Innovation, 2 tenure- conditions of loan granting and reimbursement, track professors from pre-university education other regulations necessary for the organization and the Director General of the Student Loans and functioning of the loan system; and Scholarships Agency, appointed by decree of the Ministry of Education, Research and 151 Innovation, at the request of the Student Loans it, based on a methodology, in accordance with and Scholarships Agency, for a period of 4 the law. years. (2) The personnel of the Loan Agency for (2) The Board of the Student Loans and Students from State and Private Accredited Scholarships Agency may have in its Higher Education Institutions and the National composition other members representing the Center for Scholarships Abroad, existing at the education in national minority languages, moment of reorganization, will be taken over proposed by the universities and respectively by the Student Loans and Scholarships Agency, the school inspectorates, at the request of the within the limits of approved positions, based Student Loans and Scholarships Agency. on the evaluation of professional competences, in accordance with the law. (3) With regards to awarding the scholarships, the Student Loans and (3) The tenure-track teaching personnel Scholarships Agency collaborates with the enjoy the reservation of the chair during their Board of Student Loans and Scholarships employment at the Student Loans and Agency. Scholarships Agency. (4) The Board of Student Loans and (4) The remuneration of the personnel of Scholarships Agency, in cooperation with the the Student Loans and Scholarships Agency is Student Loans and Scholarships Agency, done in accordance with the law. monitors the valorization of the activity conducted after accomplishing the scholarship Article 8 – (1) The funding of current and training abroad, by requesting reports from the capital expenditures of the Student Loans and beneficiary institutions/ universities. Scholarships Agency is carried out from its own income and the subventions allocated from Article 5 – The leadership of the Student the state budget, through the budget of the Loans and Scholarships Agency consists of one Ministry of Education, Research and Director General, assisted by 2 Directors, Innovation. appointed on competition or examination basis by decree of the Minister of Education, (2) Student scholarships and loans Research and Innovation in accordance with the programs are funded from state budget law. allocations, currency payments collected by the Ministry of Education, Research and Article 6 – The Student Loans and Innovation, from external funds, as well as Scholarships Agency has at most 34 positions. from other sources, in accordance with the law and within limits of approved budget. Article 7 – (1) The staffing of the number of positions approved in accordance with the (3) The own income of the Student Loans law is carried out within 60 days from the date and Scholarships Agency originates from the of entry into force of this decision, respecting organization of national and international the provisions of Article 6 of Law No.329/2009 events, fees collected from candidates for on reorganization of some authorities and processing and academic evaluation of public institutions, streamlining of public applications in competition, fees collected for expenditures, support of business environment the issue of confirmation documents and fees and respect of framework agreements with the collected in the implementation of national European Commission and the International and/or international projects, from services Monetary Fund. The personnel dismissed in the provided to various beneficiaries, donations, result of reorganization enjoy the social sponsorships and any other resources, except protection measures provided by the law. The state resources, in accordance with the law. staffing of the Student Loans and Scholarships Agency in public and contractual functions is (4) The Student Loans and Scholarships done by means of a competition organized by Agency collects from the beneficiaries of 152 provided services amounts calculated based on PRIME MINISTER the rates established by decree of the Minister EMIL BOC of Education, Research and Innovation, in accordance with the law, and these collections form its own income. Countersigned: (5) In accordance with the law, the Student Minister of Education, Research and Loans and Scholarships Agency is provided Innovation, ad interim with 3 cars. Catalin Ovidiu Baba, Secretary of State Article 9 – The personnel structures, the related assets, as well as any other rights and Minister of Labor, Family and Social duties resulting from the enforcement of Protection, ad interim provisions of this decision are taken over on the Mihai Constantin Seitan, basis of the deed of acceptance concluded in Secretary of State, accordance with Article 8 para.(5) of the Law No.329/2009. Minister of Public Finances, Gheorghe Pogea Article 10 – The regulation on organization and functioning, staff and organizational structure is developed by the Student Loans and Scholarships Agency and approved by decree of the Minister of Education, Research and Bucharest, 18 November 2009 Innovation, within 30 days from the date of No.1402 entry into force of this decision. Article 11 – On the date of entry into force of this decision, the Government Decision No.1861/2005 on Establishment of the National Center for Scholarships for Studies Abroad, by reorganization of the National Office of Scholarships for Studies Abroad, published in the Official Bulletin of Romania, Part I, No.29 of 12 January 2006, is repealed. 153 Annex 3: Ministerial Order on Student Loan Agency ROMANIA MINISTRY OF EDUCATION, RESEARCH, YOUTH AND SPORTS MINISTER’S BUREAU ORDER On approving the organizational structure and the personal establishment of the Student Loans and Scholarships Agency On the basis of Government Decision no. 1402/18.11.2009 on establishing, organizing and functioning of the Student Loans and Scholarships Agency, published in the Official Gazette of Romania, Part I, 821/30.11.2009, On the basis of Decree no. 1902/23.12.2009 on appointing the Government of Romania, published in the Official Gazette of Romania, Part I, no. 908/23.12.2009, On the basis of Government Decision no. 51/2009 on organizing and functioning of the Ministry of Education, Research and Innovation, On the basis of Government Ordinance no.10/2008 on the amount of basic salaries and other rights of the remunerated budget personnel according to Emergency Government Ordinance no. 24/2000 on the system of establishing basic salaries for contractual personnel from the budgetary sector and remunerated personnel according to annexes no. II and III to Law no. 154/1998 on the system of establishing basic salaries for the budgetary sector and indemnities for people in state dignity positions, as well as certain measures to regulate salary rights and other rights of contractual personnel remunerated basing on specialized laws. THE MINISTRY OF EDUCATION, RESEARCH, YOUTH AND SPORTS Issues the following order: Article 1. To approve the organizational structure of the Student Loans and Scholarships Agency presented in Annex 1, which is an integral part of this order. Article 2. To approve the personnel establishment of the Student Loans and Scholarships Agency presented in Annex 2, which is an integral part of this order. Article 3. To abrogate OMEdC no. 3277/15.02.2006 with subsequent modifications and amendments. Article 4. The Student Loans and Scholarships Agency shall implement the provisions of this order. MINISTER [signature and seal] DANIEL PETRU FUNERIU Bucharest No. 6266 Date: 30.12.2009 154 Annex 4 to the Order of MERYS no. 6266/30.12.2009 Annex Table 12: Personnel Establishment of the Student Loans and Scholarships Agency Management Number of indemnity as Department Level of Basic salary-lei No. Name of position Management position positions percentage of basic education salary minimum maximum 1 Expert IA General Director 1 H 765 1739 55% 2 Expert IA Director 1 H 765 1739 50% 3 Expert IA Director 1 H 765 1739 50% 4 Legal advisor IA 1 H 765 1739 5 Legal advisor I 1 H 697 1523 6 Expert IA Financial Service – Accounting, Administrative, Human Head of Financial Service-Accounting, 1 H 765 1739 30% Resources Administrative, Human Resources 7 Expert IA Loans, Scholarships and State Relations Service Head of Loans, Scholarships and State 1 H 765 1739 30% Relations Service 8 Expert I Loans, Scholarships and State Relations Service 13 H 697 1523 9 Expert I Financial Service – Accounting, Administrative, Human - 10 H 697 1523 Resources 10 Referent IA Financial Service – Accounting, Administrative, Human 1 SSD 630 1523 Resources 11 Referent IA Loans, Scholarships and State Relations Service 1 S 630 889 12 Worker I Financial Service – Accounting, Administrative, Human - 1.5 S 693 894 Resources 13 Caretaker I Financial Service – Accounting, Administrative, Human - 0.5 - 641 675 Resources TOTAL Number of Positions 34 155 Annex 5 to the Order of MERYS no. 6266/30.12.2009 Maximum number of staff = 34 The organizational structure of the Student Loans and Scholarships Agency presented in Annex 1, which is an integral part of this order. GENERAL DIRECTOR LOANS AND SCHOLARSHIPS LEGAL DEPARTMENT ECONOMIC DEPARTMENT DEPARTMENT LOANS, SCHOLARSHIPS FINANCIAL SERVICE – AND STATE RELATIONS ACCOUNTING, SERVICE ADMINISTRATIVE AND HUMAN RESOURCES Loans Division Financial Division- Accounting Scholarships Division Administrative Division State Relations Division Human Resources Division 156 IX. References 1. Carey, Kevin. 2010. ―That Old College Lie: Are Our Colleges Teaching Students Well? No. But Here‘s How to Make Them.‖ Democracy 15 (Winter) 2. Coalition for Clean Universities (2007) 3. Estermann, Thomas and Terhi Nokkala (2009): ―University Autonomy in Europe I‖ Exploratory Study, European University Association. 4. Delivering on the Modernization Agenda for Universities: Education, Research and Innovation‖ (May 2006) 5. EUROSTUDENT (2008), ―Social and Economic Conditions of Student Life in Europe,‖ Synopsis of indicators final report: Eurostudent III 2005–2008 6. European Commission (2010),‖Europe 2020, A European Strategy for Smart, Sustainable and Inclusive Growth.‖ 7. The European Research Area: New perspectives‖ (April 2007) 8. Fleck, Gábor and Cosima Rughiniş, eds. (2008), Come Closer: Inclusion and Exclusion of Roma in Present-Day Romanian Society, p. 166. Available online at: http://books.google.com/books?id=ck_kFYKjeBkC &printsec=frontcover#v=twopage&q&f=true. 9. Jamil Salmi ( 2009): ― The Growing Accountability Agenda in Tertiary Education : Progress or Mixed Blessing?, World Bank 10. Open Society Foundation (2007), Roma Inclusion Barometer, pp. 74-75, 81-82. Available online at http://www.soros.ro/en/program.php?program=16. 11. Open Society Institute (2006), Monitoring Education for Roma 2006: A Statistical Baseline for Central, Eastern, and South Eastern Europe, p. 21. Available online at: http://www.soros.org/initiatives/roma/articles_publications/ publications/monitoring_20061218. 12. Philippe Aghion, Mathias Dewatripont, Caroline M. Hoxby, Andreu Mas-Colell, André Sapir, The Governance and Performance of Research Universities : Evidence from Europe and the U.S., Working Paper 14851, National Bureau of Economic Research, 2009 13. Sybille Reichert.(2009): ―Institutional Diversity in European Higher Education‖, European University Association 14. UNESCO Institute of Statistics, Global Education Digest, 2010 15. UNESCO Institute of Statistics, Record Number of International Students, Bulletin dated March 10, 2011 16. World Bank 2009: ―Introducing a Student Loan Scheme in Romania. A Discussion Paper 17. World Bank (2010): Functional Review of the Romanian Pre-U 157