94446 Benchmarking the Governance of Tertiary Education Systems 1. Introduction Comparisons of tertiary education systems across the world reveal wide variations in performance, despite often-similar funding levels and common country characteristics. They demonstrate that certain tertiary education systems consistently out-perform the others in many critical areas. Among the various factors influencing the results of tertiary education systems and the performance of tertiary education institutions, recent research has identified governance as a key determinant (Aghion, 2008; Salmi, 2009). A good governance structure and favorable regulatory conditions can promote innovative behavior among tertiary education institutions, enable the development of strong quality assurance systems, and facilitate the design of effective financing mechanisms, while the opposite is not necessarily the case. This paper presents a benchmarking approach for analyzing and comparing governance in tertiary education as a critical determinant of system and institutional performance. This methodology is tested through a pilot survey in East Asia and Central America. The paper is structured in the following way: (i) the first part highlights the link between good governance practices and the performance of tertiary institutions (ii) the second part introduces the analytical approach underpinning the governance dimensions of the benchmarking tool and the specific indicators that could be used to develop relevant governance databases; (iii) the third and final part presents the results of two pilot surveys carried out by the World Bank in the East Asia and Central America regions in relation to the governance of the tertiary education systems and institutions of the countries concerned.1 2. How does good governance affect performance? A series of recent case studies on emerging research universities indicates that institutions that have complete autonomy are not constrained by externally imposed standards and can, as a result, manage their resources (whether financial, or human) with more flexibility and agility in order to respond to changes in the global marketplace for education. (Altbach and Salmi, 2011) In a comparative study of European and U.S. universities, Harvard economist Philippe Aghion observed that governance was, along with funding, the main determinant of rankings. Commenting on the poor performance of European universities in international rankings, Aghion notes that “European universities suffer from poor governance, insufficient autonomy and often perverse 1 This paper, prepared under the overall guidance of Ariel Fiszbein and Dena Ringold, benefitted from insightful suggestions from Mamta Murthi and Halsey Rogers (peer reviewers) as well as helpful comments from Nina Arnold, Andreas Blom, Erik Bloom, Emmanuela DiGropello, Adriana Jaramillo, Sachiko Kataoka, and Benoit Millot. 1 incentives” (Aghion et al. 2007, 1). A subsequent paper reporting on a survey of European universities found that research performance was positively linked to the degree of autonomy of the universities in the sample, especially with regard to budget management, the ability to hire faculty and staff, and the freedom to set salaries (Aghion et al. 2008). With respect to the composition of university boards, the report concludes that “having significant outside representation on the board may be a necessary condition to ensure that dynamic reforms taking into account long-term institutional interests can be decided upon without undue delay.” It should be noted that autonomy by itself is not sufficient for good governance. Accountability is also required to ensure that institutions perform in a way that supports national objectives for quality, relevance, transparency and equity within the overall system. Aghion’s finding of the close link between autonomy and the research performance of institutions in Europe is also borne by a clear relationship between these two variables in East Asian countries. The survey of the governance status of tertiary education systems in East Asia described in this article shows that Hong Kong, Japan, Singapore, and South Korea have features of a solid governance system, and that this is goes hand in hand with the good performance of these countries’ tertiary education system. This can be measured with two indicators of performance: (i) the number of universities of a given country appearing in the Shanghai Jiao Tong University’s Academic Ranking of World Universities (ARWU) divided by the total population of that country, and (ii) the country’s research output measured by the number of annual citations per population (Figure 1). Figure 1 - SJTU ranked institutions / million people 0.5 0.4 0.4 0.4 0.3 0.3 0.2 0.2 0.2 0.2 0.1 0.1 0.03 0.0 Singapore South Korea Japan China Source: ARWU, 2011 In 2011, only 42 countries had universities appearing in the ARWU rankings of which four were East Asian countries: Singapore, South Korea, Japan and China (including 2 Taiwan and Hong Kong). The first three among them have been highlighted in this paper for relatively strong governance. Figure 2 shows the relative performance of East Asian countries in terms of research output, that is, the number of scientific articles published in the following fields; physics, biology, chemistry, mathematics, clinical medicine biomedical research, engineering and technology, and earth and space sciences divided by the total population in each country. Again, as is the case in the evaluation of country performance according to SJTU ranking, high income countries such as Japan, Singapore and South Korea which have good governance also fare well on research output. Figure 2 - Citations relative to population of East Asian countries (for 1 million inhabitants - 2007) Singapore 826.5 Japan 414.0 Korea, Rep. 381.1 China 43.1 Malaysia 29.9 Thailand 25.5 Mongolia 8.2 Vietnam 3.4 Philippines 2.2 Lao PDR 2.1 Cambodia 1.9 Indonesia 0.8 0.00 100.00 200.00 300.00 400.00 500.00 600.00 700.00 800.00 900.00 Source: National Science Foundation, Science and Engineering Indicators In the Central American case, while it is difficult to demonstrate a causal relationship between governance and results, it is clear that the weak governance of the tertiary education systems in the sub-region is associated with low results overall, compared to 3 other regions of the world or even with the rest of Latin America.2 No Central American university has ever made it into a major global ranking so far. Even in the Webometrics ranking, which measures the online presence of universities’ research outputs, among the top 100 universities in Latin America, Central American universities appear just once: in the 16th position with the University of Costa Rica, and in the 492nd position in the world ranking.3 Similarly, when analyzing the results of the Scimago Institutional Ranking of 2011, which ranks all Spanish and Portuguese speaking universities in Spain, Portugal and Latin America, the University of Costa Rica is the only one in Central America appearing in the top 200.4 It comes in as number 126 overall and 72 if only Latin American universities are considered. In terms of country ranking, Costa Rica is the 9th in Latin America (Table 1). 2 It should be noted that, with the exception of Costa Rica, countries in Central America are poorer than in the rest of the continent. Weak governance, thus, is not the only, (nor the main) reason for the low ranking within the Latin American region of Central American universities. 3 Four indicators are considered: “Size: Number of pages recovered from: Google, Yahoo, Live Search and Exalead. Visibility: The total number of unique external links received (inlinks) by a site can be only confidently obtained from Yahoo Search. Rich Files: Adobe Acrobat (.pdf), Adobe PostScript (.ps), Microsoft Word (.doc) and Microsoft Powerpoint (.ppt). and Scholar: Google Scholar provides the number of papers and citations for each academic domain. These results from the Scholar database represent papers, reports and other academic items”. At: http://www.webometrics.info/methodology.html 4 The Scimago Ranking measures the research capacity of tertiary education institutions in Ibero-American countries, through four indicators: “1. Scientific output (An institution's publ ication output reveals its scientific outcomes in terms of published documents in scholarly journals. Output gives an overall idea of HEIs size. When a publication is coauthored, a score is assigned to each HEI). 2. International collaboration (IC index shows the institution's output ratio that has been produced in collaboration with foreign institutions. These values are computed by analyzing the institution's output whose affiliation includes foreign country addresses.),3. Normalized Impact (This indicator reveals the ratio between the average scientific impact of an institution and the world average for publications of the same period and subject area. The world average is set to score 1, --i.e. a score of 0.8 means the institution is cited 20% below average and 1.3 means the institution is cited 30% above average). 4. High Prestige Journal (Ratio of scientific documents, an institution publishes in the most influential scholarly journals of the world. Journals considered for this indicator are those ranked in the first quartile (25%) in their categories as ordered by SCImago Journal Rank SJR indicator). Consulted on xxxx at : http://www.scimagoir.com/pdf/ranking_iberoamericano_2011_en.pdf 4 Table 1 - SCImago Journal and Country Rank (Latin America only) Citations Citable Self- per Country Documents Citations H index documents Citations Docume nt 1 Brazil 325,549 315,102 2,203,616 716,178 8.98 253 2 Mexico 124,320 120,830 926,074 206,334 8.92 193 3 Argentina 93,075 90,135 814,586 185,945 9.85 183 4 Chile 49,931 48,479 468,897 91,514 12.03 162 5 Colombia 22,764 22,068 138,167 21,538 9.37 105 6 Venezuela 21,589 21,031 149,208 23,519 7.64 111 7 Uruguay 7,297 7,050 75,682 11,436 12.74 90 8 Peru 6,241 5,929 61,065 7,070 13.09 87 9 Costa Rica 5,002 4,843 58,687 7,262 13.35 88 10 Ecuador 3,291 3,158 30,236 4,156 11.76 67 11 Panama 2,643 2,507 50,970 5,557 26.32 91 12 Bolivia 2,039 1,979 20,632 2,411 12.11 52 13 Guatemala 1,122 1,055 10,488 654 10.96 42 14 El Salvador 794 772 4,833 137 7.93 32 15 Nicaragua 709 688 6,344 548 11.44 37 16 Paraguay 650 624 5,983 366 11.42 37 17 Honduras 514 502 4,924 282 10.86 33 18 Belize 171 168 1,513 96 11.28 22 When considering research output, Central American countries lag behind most other LAC countries. Here again, Costa Rica is the only Central American country appearing in the ranking measuring the number of research papers produced between 2000 and 2010 (Table 2), and it comes in the last position (85th rank in the world). 5 5 This ranking was developed with data provided by Thomson Reuters from its Essential Science Indicators. Consulted on xxx at http://www.globserver.com/en/press/latin-american-nations-ranked-research-paper-output 5 Table 2 – Research Output of Latin American Universities Latin American nations ranked by research paper output Data provided by Thomson Reuters from its Essential Science Indicators, 1 January 2000-31 October 2010 Region World Country Top field by Papers World Citation Citation Top field by rank rank share share (%) s impact relative impact Agricultural 1,246,01 1 15 Brazil 195,459 2.00 6.37 Engineering sciences 3 Plant and 2 28 Mexico animal 72,403 0.74 507,316 7.01 Physics sciences Plant and 3 35 Argentina animal 55,872 0.57 449,864 8.05 Medicine sciences Ecology/envir 4 43 Chile 31,837 0.33 282,292 8.87 Astronomy onment Plant and 5 55 Colombia animal 12,013 0.12 78,067 6.50 Medicine sciences Plant and 6 56 Venezuela animal 11,748 0.12 77,877 6.63 Medicine sciences 7 65 Cuba Immunology 7,192 0.07 42,913 5.97 Chemistry Plant and 8 75 Uruguay animal 4,513 0.05 40,919 9.07 Medicine sciences 9 77 Peru Immunology 3,866 0.04 40,257 10.41 Medicine Plant and 10 85 Costa Rica animal 3,192 0.03 37,688 11.81 Medicine sciences Source: Retrieved on March 10, 2011 from: http://www.globserver.com/en/press/latin-american-nations-ranked-research-paper-output LAC countries produce barely 5 per cent of the scientific articles indexed by Thomson Reuters in the internationally influential journals surveyed for its Web of Science database, and the contribution of Central American countries to this percentage is insignificant. Thus, there is evidence of a link between tertiary systems with good governance mechanisms and high performance of institutions within those systems. What then are the key characteristics of good governance, and how can countries adopt these features within their own systems? 6 3. Proposed Indicators of Governance Quality Personally, I like the university. They gave us money and facilities, and we didn’t have to produce anything. You’ve never been out of college. You don’t know what it’s like out there. I’ve worked in the private sector. They expect results. Dan Akroyd talking to Bill Murray after both lost their jobs as university researchers, in the movie Ghostbusters (Quoted in Penn 2007) Fielden defines tertiary education governance as “all those structures, processes and activities that are involved in the planning and direction of the institutions and people working in tertiary education” (2008). Good governance has been identified as one of the main drivers of system health (Aghion et al., 2008; Salmi, 2009; Altbach and Salmi, 2011). The governance benchmarking framework presented here is designed as a tool to guide policy makers in identifying the specific governance aspects that can be reformed in order to improve the performance of tertiary education systems. Shift from ‘state-controlled’ to ‘state-supervised’ model Several models of the state – institution governance relationship have been proposed by higher education scholars. Neave and Van Vught (1994) describe a continuum at one end of which is the “state-control model” where the government seeks to control its universities closely, and at the other end is the “state-supervising model” where it monitors and regulates them at a distance. As shown by Fielden (2008), countries are moving from the control model to the supervisory model in all aspects of their relationship with their universities given the growth in demand for tertiary education and the impracticality of a central body effectively managing day-to-day operations of university institutions. As systems become more complicated and modern management principles are adopted in institutions, the need for autonomy grows. “Higher education systems are… getting more complex due to the growth in the number of public and private institutions, so that the task of managing and monitoring the sector is becoming more specialized and demanding. As a result, the old model of total control from a central ministry of education is proving unsustainable in the long term and is being replaced throughout the world by other models. These alter the mode of central involvement from one of detail to that of strategy and rely on more sophisticated forms of monitoring and performance review.” (Fielden, 2008) A review of recent reforms in East Asian countries confirms the trend toward the “state - supervised” model of governance where governments award tertiary education institutions increased autonomy to enable them to manage their operations independently (Reza, 2010). The push for decentralization of tertiary management came in the mid and late 1990s, beginning with the Republic of Korea and followed by Indonesia and Thailand. A second wave of reforms happened in the mid 2000s, when Japan and Singapore extended autonomy to their public tertiary education institutions. 7 This pattern of decentralization of control from governments to institutions has not only been witnessed in emerging economies but it has been reviewed in depth by the OECD. In 2008, the OECD prepared a three-volume synthesis of the main findings and lessons learned on the basis of the twenty-four reviews of national tertiary education systems conducted in the previous five years (Santiago et al., 2008). The report showed that countries differed widely in their economic and social characteristics, as well as their approaches to tertiary education. Together they permitted a comprehensive analysis of key policy issues in a comparative perspective. Main components of governance The following figure can be used to visualize the dynamic elements of governance which affect the performance of the tertiary education system as a whole as well as that of all the institutions that operate in it. Figure 3 – Tertiary Education Governance Framework State Accountability Autonomy Vision, regulations & incentives Tertiary Institutions Source: Elaborated by J. Fielden and J. Salmi In this framework, the main responsibility of the state is to establish the regulations and incentives that create autonomy and accountability within tertiary institutions. It does so by elaborating a vision for the future of tertiary education, providing overall guidance and policy-making, establishing a higher education law that that sets a level-playing field for all tertiary institutions operating in the system, creating a favorable regulatory framework for private tertiary education institutions, developing and maintaining a quality assurance 8 system, encouraging tertiary education institutions to develop a strategic plan by which they set their own policies and direction, and allocating public resources to tertiary institutions and students in the system on the basis of transparent performance criteria. Through these mechanisms, the state engenders a context for and culture of autonomy of tertiary education institutions. A highly autonomous institution is characterized by the presence of an independent university board with external representation, the appointment of the leadership team according to professional criteria, the ability to set human resource policy around recruitment and retention of staff as well including salary levels, academic autonomy and financial autonomy. While tertiary education institutions are given autonomy by the State, in return they must be accountable for their use of public resources, the alignment of their operations with public policy goals, and their overall performance. In the words of John Millett, former Senior Vice President of the Academy for Educational Development, “Accountability is the responsibility to demonstrate that specific and carefully defined outcomes result from higher education and that these outcomes are worth what they cost.” (MOHE, 2008). For universities and their leaders, accountability represents the ethical and managerial obligation to report on their activities and results, explain their performance, and assume responsibility for unmet expectations. At the very minimum, all tertiary education institutions should be legally required to fulfill the following two basic dimensions of accountability: (i) integrity in the delivery of education services, and (ii) honesty in the use of financial resources. In addition, many stakeholders have a legitimate claim to expect a cost-effective use of available resources and the best possible quality and relevance of the programs and courses offered by these tertiary institutions (Salmi, 2007). Tertiary institutions maintain accountability specifically through their internal quality assurance mechanisms, regular reporting on academic results and relevance of programs, financial audits, and by putting in place appropriate instruments to prevent and punish corruption. A joint publication by the International Academy of Education and the International Institute for Educational Planning (Anderson, 2005) proposes three dimensions of accountability in education—compliance, professional standards, and results— that need to be fully aligned in order to order to achieve high levels of performance. Table 3 illustrates how these three complementary forms of accountability would pan out in the case of tertiary education. 9 Table 3- Accountability in Tertiary Education Type of Accountability Accountable for what? Accountable to whom? Observance of Higher Compliance Education Law, government Government, Parliament regulations Adherence to quality Professional standards assurance norms and QA agencies, Government standards Quality, relevance, equity, Government, students, research production, Results families, employers, society knowledge and technology at large transfer, values These dimensions of accountability have been captured in the proposed framework at two levels. First, they are embedded in the set of system-wide governance indicators through indicators assessing the strength of a country’s management information system for tertiary education institutions, design of its funding allocation mechanisms, and strength of its quality assurance system. Second, accountability measures are a key feature of the set of indicators relating to institutional autonomy. The governance of tertiary education systems involves two key dimensions: system-wide governance and institutional governance. x System-wide governance relates to the macro level laws, policies, and processes that are in place with the purpose of providing a favorable context in order for tertiary institutions to be able to operate efficiently and effectively. These institutions are the actors that, by working in an integrated and articulated fashion, can produce the education outcomes that are most important to any nation- attainment, quality and relevance, equity, research, technology transfer, and positive values and behaviors. x The second layer of analysis is institutional governance. This dimension answers the question- what policies and practices should be in place to enable tertiary institutions to operate autonomously? As noted earlier in this paper, research shows that systems that promote high autonomy are also better placed to produce positive results. 10 System-wide Governance: definition and justification of indicators There are five key features of system-wide governance. These cover (i) the capacity of the State to guide and direct the development of tertiary education, (ii) the presence of a strong and favorable regulatory framework, (iii) reliance on performance-based funding allocation mechanisms, (iv) a comprehensive quality assurance system for public and private institutions, and (v) a solid accountability framework. These five features of system-wide governance were selected after a review of research evaluating the common policies, laws and practices in place leading to high performance institutions and systems that promote regional economic development. In terms of support for the first three features of system-wide governance, an OECD study (OECD 2008) focused on the factors restricting tertiary institutions from contributing more significantly to regional economic development. The study showed that key barriers included (i) lack of explicit orientation of public policy toward that goal; (ii) inadequate incentives structures for regional engagement; (iii) limits to autonomy and leadership and (iv) the limited capacity of local and regional actors to determine the strategic direction of tertiary institutions. (Raza, 2010) This finding broadly supports the need for the State to play a specific role in guiding and directing tertiary education, to set an appropriate regulatory framework for new institutions to enter and existing institutions to flourish, and to align financial rewards with performance. The fourth feature of system-wide governance is quality assurance. There is a wealth of literature emphasizing the necessary role for quality assurance mechanisms in a well- functioning tertiary education system. Henard (2009) rightly notes that as the role of higher education in contributing to economic growth becomes more important, so does the attention on quality assurance: The rise of the new economy in the 1990s has made research and innovation key to countries‘ competitive edge in the global economy. This has been central to the Lisbon Strategy, which stresses the importance of excellence in research and development in order to turn the European Union into the most competitive and dynamic knowledge-driven economy by 2010 (European Council, 2000). (Morley et al., 2006; Geiger, 2004) The fifth feature of system-wide governance is the accountability framework adopted to ensure the financial, academic and informational integrity of the tertiary education system. The key components of accountability framework proposed in this text are supported by Huisman and Currie in their 2004 study focusing on the shift from input focused, rigid state-controlled systems of accountability to those that are broader, outcome based and allow for independence of institutions in day-to-day decision-making. This push for more outcome-based accountability has not only evolved from the changing relationship between government and universities, but also because of the increased focus on efficiency, value for money spent and globalization of education requiring greater accountability of many new cross-border HEIs (Huisman and Currie, 2004: 532-533). 11 Thus, the inclusion of indicators tracking use of financial audits, corruption (whether financial, academic or information) and transparency align with this research. Does government have the capacity to lead and guide the tertiary education sector? This refers to the role of the State in setting a strategic vision, planning for the future development of tertiary education, implementing reforms and providing technical assistance to promote effective leadership and management at the institutional level. A related question that needs to be asked in this context is whether the country has a comprehensive tertiary education management information system in order to drive policy decisions? Direction setting based on regular “stock taking” is important as it enables the country to adapt its tertiary education system to changes in the economy and society. Regulatory framework- This refers to the ‘policy and regulatory infrastructure’ of the tertiary education system. Does the country have a positive higher education law in place and how recently has it been updated? Are the rules and regulations in place favorable to the establishment and operation of private tertiary education institutions? These are features that show the extent to which a tertiary education system is able to develop in a flexible way and adapt to change. Funding allocation modality- This refers to the public funding mechanisms in place to encourage tertiary education institutions to operate in an effective and innovative manner. QA system for public and private institutions – A quality assurance system is important to ensure accountability of all tertiary institutions to the primary goal of educating students with the relevant skills and attitudes required to prepare them for further study or the workforce. Accountability Framework- this refers to the mechanisms in place to ensure that tertiary education institutions are responsible to the state and society for financial integrity, lack of fraud and corruption, and transparent reporting of the main dimensions of performance. Institutional Governance: definition and justification of indicators There are four principal features of institutional governance to measure the degree of autonomy of tertiary education institutions. These are (i) organizational autonomy, (ii) financial autonomy, (iii) human resources autonomy, and (iv) academic autonomy. The grouping of autonomy factors into these four categories was first introduced by the European University Association (EUA) in its Lisbon declaration of 2007. This declaration results from the discussions that took place among 700 universities and partners during the fourth EUA Convention of Higher Education Institutions hosted by the five Lisbon universities and the Portuguese Rectors’ Conference. 12 The selection of these four dimensions as the principle elements of institutional governance is further supported empirically by Aghion et al. in various studies focusing on the importance of autonomy for successful research and innovation at universities in Europe and America. (2007, 2008, 2009) The authors define tertiary institution performance as a ranking in the Shanghai Jiao Tong University (SJTU) ranking of world universities. In attempting to identify the factors that correlate with being ranked in the SJTU, the authors rely on factor analysis using a survey of European universities. They find that the autonomy factor is maximized for those European universities that share a number of characteristics, including that they (i) do not need to seek government approval of their budget; (ii) select their baccalaureate students in a manner independent of the government; (iii) pay faculty flexibly rather than based on a centralized seniority/rank based scale; (iv) control their hiring internally; (v) have low endogamy; (vi) own their own buildings; (vii) set their own curriculum; (viii) have a relatively low percentage of their budget from core government funds; and (ix) have a relatively high percentage of their funds from competitive research grants (Aghion et al., 2009). Organizational autonomy and institutional oversight – this refers to the role and functions of the governing board of the university, and the selection modalities of the leadership team. Financial autonomy- this refers to the institution’s capacity to mobilize resources, manage its resources, and use them efficiently. By having control over their resources and facilities, tertiary institutions are in a better position to adapt and improve to meet the changing demands of society and the market. Human resources autonomy- this refers to the capacity and responsibility of the institution to recruit staff and set salaries, usually directly related to the legal status of staff (whether civil servant or employee of the university itself). Academic autonomy- refers to the degree to which an institution is at liberty of directing its own education and research strategy, for example choice in specialization in research or focus on teaching of specific topics and targeting of specific populations for enrolment. It also includes the ability of the institution to manage its academic profile and admission policy (including qualifications and size of student enrolment). Last but not least, it encompasses the degree of academic freedom enjoyed by a tertiary education institution in a given country. Governance Indicators Table 4 below presents the list of indicators proposed to measure each dimension of governance. 13 Table 4- Governance Indicators SYSTEM-WIDE GOVERNANCE ACTIONS TO IMPROVE ON THE INDICATORS 1. Country has no vision / plan. 2. Federal government initiating Country has a partial development / a vision and translating it into reform plan but is not acting on it. 3. a concrete plan Strategic vision / Country has partial vision / reform development plan plan and is acting on it. 4. Country has a strategic vision / plan and is acting on it. 1. Inexistent or weak tertiary education authority. 2. Small tertiary education authority with limited capacity and influence. 3. Reasonably Reform capacity of the well staffed tertiary education tertiary education authority able to implement limited authority reforms. 4. Highly credible and capable tertiary education authority Capacity of able to implement controversial government to plan reforms the development of Monitoring and tertiary education accountability: a and implement Tertiary Education 1. No TEMIS exists. 2. Incomplete reforms Management system exists. 3. Complete system Information System exists but is not frequently used for (TEMIS) exists, covering policy purposes. 4. Comprehensive all tertiary education TEMIS exists and is frequently used institutions and used for for policy purposes. policy guidance and formulation purposes) 1. The tertiary education authority Capacity building: the has no qualified staff for capacity national or provincial building purposes. 2. The tertiary authorities have the education authority has a few know-how and programs in place for building the competencies to management capacity of tertiary strengthen institutional education institutions. 3. The tertiary capacity in the sector education authority has extensive programs for building the 14 management capacity of tertiary education institutions. 1. No higher education law exists. 2. Federal government The country has a higher education establishing a higher HE law, policies and law, but it has not been updated in education law and either establishing clear regulation regulation the past 10 years. 3. The higher on private provision or education law has been updated in delegating this responsibility past 10 years to provinces/ states Regulatory 1. Private providers not allowed or framework Opportunities for entry heavily restricted. 2. Reasonable of private providers regulations to control entry and operation of private providers. 1. No clear demarcation between for- Regulation of private profit and not-for-profit private providers institutions. 2. Clear definition of for- profit and not-for-profit status Proportion of recurrent budget Higher Education Authority Funding Allocation allocated on the basis of performance identifies performances goals Modalities measures and uses financing as a lever extent to which the to achieve these goals. Proportion of research budget State uses public allocated on a competitive basis subsidies to encourage improved Proportion of investment funding performance, allocated on a competitive basis innovation and accountability Proportion of recurrent budget allocated through the students (grants, loans, vouchers) 15 a) Government authority negotiates performance agreements with Options: no performance agreements institutions b) results are are in place and there is no plan to do tracked annually c) so/ Country has feature 1/ Country rewards/incentives are has feature 1&2/ Country has all 3 in place for those features institutions that meet/exceed targets 1. No QA agency in place. 2. Less Government and than ¼ of institutions or programs stakeholders establish and (public and private) are evaluated or maintain QA agency accredited 3. Between ¼ and ¾ of Existence of QA system programs (public and private) are evaluated or accredited. 4. More QA system for public than ¾ of all programs (public plus and private private) are evaluated or accredited. institutions Level of independence1. Part of a government agency 2. Autonomous Degree of independence public agency 3. Autonomous, non- of QA agency(ies) public agency that is independently funded) Financial audits (public Free access to information 1. No audit conducted .2. Audit law established by institutions are conducted annually but results are government and adopted by independently audited not publicly available. 3. Audit tertiary institutions annually and results of conducted annually and results audit are publicly publicly available. Accountability available) Framework Global corruption barometer score: % of users of services reporting they Monitoring of paid a bribe to receive attention from corruption (financial, at least one of 9 different service academic and providers in the past 12 months information fraud) Percentage of students who report academic or information fraud Operation of an anti- 1. No such agency exists. 2. Agency corruption agency exists but does not cover tertiary competent for tertiary education. 3. Agency exists and education covers tertiary education. 16 Participation in accreditation and publication of results: 1. No publication of evaluation / accreditation results. 2. Very few institutions publish evaluation / accreditation results. 3. Most institutions publish evaluation / accreditation results. Transparency and Participation in student engagement openness of tertiary and tracer surveys and publication of institutions results: 1. Very few institutions participate in surveys 2. Most institutions participate in surveys but they do not publish the results 3. Most institutions participate in surveys and publish the results Publication of deliberations of governing boards (Yes/No) Sharing of deliberations of governing boards internally (with faculty and students) (Yes/No) Existence of fair/transparent admission policies and examinations Regular research reports on industry Higher Education Authority and employment trends are released and Government collaborate Labor market advisory to public along with Employment rate to collect and publish results service of recent graduates in relevant to public occupations (Yes/No) ACTIONS TO IMPROVE ON THE INSTITUTIONAL AUTONOMY INDICATORS 1. University faculty and staff only 2. Higher Education Authority Organizational University faculty, staff, and student establishes policy on role and autonomy / representatives 3. University faculty, functions of board institutional oversight Composition of Board staff, students, and external participants (Role and functions of consisting only of government officials the Board) 4. University faculty, staff, students, and majority external participants 17 including members from private sector/ non-profit arena. 1. Election by faculty and staff 2. Selection of Board Appointment by external committee on basis of search committee Independence of boards of 1. No independence, 2. Low degree of public universities vis-à-vis independence, 3. high degree of Higher Education Authority independence (score 1, 2 or 3) 1. Board is purely consultative. 2. Board approves strategic plan. 3 Board Authority of Board approves strategic plan and annual (strategic plan, budget and budget. 4. Board approves strategic selection of leader) plan, annual budget and is responsible for recruitment of University leader 1. Selection of university leader is based on political appointment. 2. Selection Selection of university of leader is based on internal election. leader 3. Selection of leader is based on competitive search (score 1, 2 or 3) 1. Leader can be only an internal candidate. 2. Leader can come from Eligibility of external another academic institution in the candidates country. 3. Leader can come from any Organizational academic institution in the world. autonomy: Leadership 1. Leader can serve only one term and cannot serve elsewhere afterwards. 2. Leader can serve 2 terms and cannot Leadership term limits serve elsewhere afterwards. 3. No limits on terms either within or in other institutions. 1. Based on outside political appointment. 2. Based on elections. 3. Selection of leadership At discretion of university leader. 4. By team (deans, etc.) university leader based on competitive search Freedom to borrow from Score: 1. (1. No 2. Yes with restrictions Higher Education Authority commercial banks 3. Yes, no restrictions) establishes policy on resource Financial autonomy: 1. No fee-setting authority. 2. Fee- mobilization Freedom to set level of resource mobilization setting authority, but with a ceiling on tuition and fees fees 3. Unlimited freedom to set fees. Institutions are allowed to 1. No. 2. Yes with restrictions. 3. Yes, issue bonds in the financial no restrictions. markets Institutions are able to retain surplus from annual budget and / or 1. No. 2. Partially true. 3. Yes. Institutions are able to retain self-generated funds Institutions are allowed to 1. No. 2. Partially true. 3. Yes. constitute and use an 18 endowment Flexibility in use of available resources: public budget allocated as block 1. No. 2. Partially true. 3. Yes. grant (rather than line items) Property management: Financial autonomy: universities own and have resource utilization 1. No. 2. Partial ownership and selling authority to sell own rights. 3. Yes full ownership. buildings, facilities and equipment 1. Public universities are subject to restrictive rules and procedures. 2. Procurement Procurement rules and procedures are as flexible as for private universities. Legal status of faculty (civil servant) Civil service status (Yes/No) 1. Universities have no freedom to dismiss non performing staff. 2. HR autonomy Staffing: recruitment and Universities have the authority to dismissal of faculty dismiss non performing staff, but it is very difficult in practice. 3. Universities can dismiss non performing staff. 1 set by civil service conditions. 2. Freedom to set salaries limited margin of movement, 3. total freedom) Score 1-3 Freedom to define academic structure, programs and course content (Score 1- Academic autonomy Institutional strategy 3; no freedom, some freedom, high degree of freedom) Restrictions to academic freedom (Score Academic profile (Degree 1-3; no freedom, some freedom, high programs) degree of freedom) Freedom to recruit students (enrolment size and academic qualifications) (Score Student admission policy 1-3; no freedom, some freedom, high degree of freedom) 19 Types of Indicators and Survey Methodology Benchmarking these governance dimensions implies relying on three types of indicators: quantitative indicators, objective qualitative indicators, and subjective qualitative indicators. x Quantitative indicators provide users with a tangible measure to compare performance across various dimensions of country systems and institutions. Data for these indicators are relatively easier to collect than qualitative data. Unfortunately, few governance indicators come under this category, for example the proportion of accredited institutions or programs in a country, or the proportion of budget allocated based on performance criteria, or the proportion of students who report corruption or academic fraud. x Objective qualitative indicators describe key dimensions of governance in a non numeric way. For example, qualitative indicators can capture, in an objective manner, the main characteristics of a tertiary education system’s quality assurance set up (existence of an accreditation system) and key dimensions of organizational autonomy (existence of an independent Board, mode of selection of university leaders). x Subjective qualitative indicators are constructed on the basis of expert judgments on key dimensions of system and institutional governance. For example, one of the important governance drivers of system health is the degree of management autonomy that tertiary education institutions enjoy, which is difficult to measure and compare across institutions / countries with full objectivity. These governance indicators have been applied to assess the state of affairs in university governance in East Asia and Central America. The East Asia survey instrument involved use of a questionnaire developed by education sector staff with feedback provided by UNESCO Institute for Statistics (UIS). UIS Bangkok assisted World Bank staff in the identification and communication with expert consultants in each country under study. Experts then completed questionnaires based on their review of policies and laws as well as through communication with relevant government officials where necessary. A similar methodology was used by the authors of the Teacher Policies and Student Assessments chapters in the book SABER- System Assessment and Benchmarking Education for Results: Strengthening Education Quality in East Asia. (Patrinos, 2012) The Central America survey was coordinated by a higher education expert from the region. The original questionnaire was translated and adapted to Central American tertiary education systems. The survey was then sent to experts in each country and was followed up by telephone calls in Honduras, Guatemala, Panama and Costa Rica. Field visits took place in Belize and in El Salvador. Participants in the survey included representatives from Universities Council of Central America - CSUCA and 20 representatives of universities of each one of the Central American countries considered in the survey. 4. Results of the World Bank East Asia and Central America Pilot Surveys Governance Characteristics of East Asian Countries The first set of examples analyzes elements of system-wide governance among East Asian countries based on the results of an expert survey conducted by the World Bank in 20106 and Central American countries based on a an expert survey conducted in 2010 (Cajas, 2010)7. Most data are available only for the Spanish-speaking countries of the sub-region as it proved very difficult to gather data on Belize. System-wide governance The survey results on the first three dimensions of system-wide governance, include the capacity of the government to plan and make reforms, the strength of the regulatory framework, and the funding allocation modalities in the country highlight the marked difference in the higher education policy context in East Asia and Central America. The assessment of East Asian countries on these features is reflected in Figure 4, Figure 5 and Figure 6. 6 Participants in the expert survey included representatives of SJTU University (China) the Asian Development Bank, the National Economic University (Vietnam), the University of Hong Kong, the Korean Institute of Education , professors of education from Japan and a number of experienced consultants to the World Bank. 7 The expert surveys were coordinated by Fernando Cajas, a Guatemalan higher education specialist. Participants in the survey included representatives from Universities Council of Central America - CSUCA, Carlos Pérez Brito from the World Bank in Guatemala; as well as representatives of the universities of Belize, Guatemala, Honduras, El Salvador, Nicaragua, Costa Rica and Panama. 21 Figure 4: Capacity of the government to plan the development of tertiary education and implement reforms 5 4 4 4 4 4 3 3 3 3 2 1 1 0 Lao PDR Malaysia Thailand Indonesia Vietnam China Hong Kong Japan 1. Country has no plan. 2. Country has a partial reform plan but is not acting on it. 3. Country has partial reform plan and is acting on it. 4. Country has a strategic plan and is acting on it. Figure 5: Regulatory framework of East Asia tertiary systems 3.5 3 3 3 3 2.5 2 2 2 2 2 2 1.5 1 1 1 1 0.5 0 1. No higher education law exists. 2. The country has a higher education law, but it has not been updated in the past 10 years. 3. The higher education law has been updated in past 10 years 22 Figure 6 - Regulatory framework of East Asia tertiary systems- transparency and openness 3.5 3 3 3 3 3 3 2.5 2 2 2 2 2 1. No publication of evaluation / accreditation results. 2. Very 1.5 few institutions publish 1 1 evaluation / accreditation 1 results. 3. Most institutions publish evaluation / 0.5 accreditation results. 0 In East Asia, figures 4, 5 and 6 show a marked difference in the degree of development of the system-wide policy capacity across countries. For example, Hong Kong developed a strategic vision for the tertiary education system as early as 1965 and has updated it regularly since then with the most recent update being in 2004. Japan also has many of these tools- with a strategic vision having been updated in 2008 and a new legal framework that transforms public universities into corporations enacted in 2004. By contrast, some among the low income countries in the region have limited policies or processes set up to enable tertiary institutions to act autonomously. These systems are largely characterized by either centralization or limited policies on tertiary education development. For example, Lao PDR does not have a strategic vision, or a legal framework for its tertiary education sector. Instead, as noted in a World Bank 2010 study on higher education development in East Asia, the “legislative basis for the higher education system is found in a series of Prime Ministerial Decrees and other initiatives.” (World Bank, 2010) Further, the quality assurance body, although established in 2008 is yet to be functioning and there is no Tertiary Education Management Information System. A review of tertiary education policies in Vietnam shows similar results. While the country did issue a strategic vision in 2004 and has invested enormously to expand and improve its tertiary education system, its legal framework is still under development, governance is relatively centralized, and there is no functional TEMIS yet. 23 Figures 7, 8, 9, 10 show the results of the Central America survey on these dimensions of system-wide governance. Figure 7: Capacity of government to plan the development of tertiary education and implement reforms – Strategic vision / development plan Figure 8: Capacity of government to plan the development of tertiary education and implement reforms – Reform capacity of the tertiary education authority 24 Figure 9: Capacity of government to plan the development of tertiary education and implement reforms – Monitoring capacity 4 3 1. No TEMIS exists 2. Incomplete system exists 3. 2 Complete system exists but 2 is not frequently used for policy purposes. 4. 1 1 1 Comprehensive TEMIS exists 1 and is frequently used for policy purposes. 0 Belice Nicaragua Honduras Costa Rica Figure 10 - Regulatory Framework of Central America tertiary education systems 4 3 3 1. No higher education law 2 2 2 2 exists. 2. The country has a 2 higher education law, but it has not been updated in the past 10 years. 3. The higher 1 education law has been updated in past 10 years 0 Costa Rica Honduras Nicaragua Panama El Salvador The survey results on these dimensions denote the critical situation in Central America, where there are no national Tertiary Education Authorities as such, nor national strategic visions or development plans for tertiary education. The capacity of the government to monitor the performance of tertiary education is inexistent or weak, as are the Quality Assurance systems. Indeed there is no government ministry or public agency formerly responsible for guiding or managing tertiary education in any of the Central American 25 countries. By constitution, the public universities in all six Spanish-speaking countries are entitled to a fixed share of the national budget without any control by the Ministry of Finance or the Ministry of Education. Within the sub-region, Costa Rica stands out with better governance, relatively, as it has developed a monitoring capacity at least for the public universities and has a quality assurance system in place. CONARE, the entity that coordinates Costa Rica’s public universities, is the only in the region that has elaborated a strategic vision linked to the country’s development goals. However, CONARE is not a government agency but the Council of Rectors of the four leading public universities. Interestingly, it has refused so far to include the newly-created public Technology University for fear of having to share public resources with it. Quality Assurance Quality assurance (QA) is a key component of the system’s ability to hold institutions accountable to a certain standard of performance. The last two indicators of system-wide governance reflected in Figures 11 and 12, look at the strength of the quality assurance system. The first indicator measures the existence of a QA system for public and private institutions and groups countries on the basis of the proportion of programs that are accredited as shown in Figure 11 below. Countries received a score of one if there was no QA system in place or there was no data available. They received a score of two if less than ¼ of tertiary institutions or programs (public and private) are evaluated or accredited. Countries received a score of three if between ¼ and ¾ of programs (public and private) are evaluated or accredited. Finally, they received a score of four if between ¾ and all programs (public plus private) are evaluated or accredited. 26 Figure 11 – Proportion of Accredited Tertiary institutions in East Asia 4.5 4 4 4 4 4 4 4 4 3.7 3.5 3 3 2.5 2 2 1.5 non-existent 1 0.5 0 Note: Data in this figure are being updated The second indicator measures the degree of independence of the quality assurance agency (Figure 12). 27 Figure 12- Degree of Independence of the Quality Assurance Agency 4 3.5 Degree of 3 3 3 3 3 independence of 3 the QA agency: 1) Part of a 2.5 government agency 2 2 2) Autonomomous 2 public agency 3) Autonomous non- 1.5 public agency that is funded 1 1 1 1 1 independently 0.5 0 These two indicators confirm the steady progress in establishing a quality assurance culture in most East Asian nations, boosted in the past decade by the rapid development of the Asian and Pacific Quality Assurance Network (APQN) that was initially sponsored by a World Bank grant. In Central America, as mentioned only Costa Rica and El Salvador have a QA system in place, however the proportion of programs and Institutions that have received accreditation is still insignificant. One of the explanations is that for the universities there are no specific benefits in receiving the accreditation (or sanction for not participating). Besides, there is a regional entity called Consejo Superior Centro Americano (CSUCA) that promotes self-evaluation and provides accreditation to programs and institutions in Central American countries. However, the number of accreditations that it has offered is still very low. 28 Figure 13 - QA system for public and private institutions 4 3 1. No QA system in place. 2. Less than ¼ of institutions or 2 2 programs (public and 2 private) are evaluated or accredited 3. Between ¼ and 1 1 1 ¾ of institutions or programs 1 (public and private) are evaluated or accredited. 4. Between ¾ and all 0 institutions or p All the countries have a national entity responsible for the monitoring and control of public funds, however there is no specific agency for tertiary education institutions that includes private providers. Figure 14: Accountability Framework – Anti corruption agency Overall, the survey results from East Asia show a wide discrepancy among countries in the region around activity of QA agencies and their independence, though most surveyed countries have 29 some QA mechanism in place. In Central America, only Costa Rica and El Salvador have QA agencies in place. Institutional Autonomy The next set of examples analyzes elements of institutional autonomy among East Asian and Central American countries. As noted in the earlier discussion of university autonomy, there are four major components of autonomy- organizational autonomy, financial autonomy, staffing or human resources autonomy, academic autonomy.8 The World Bank report on East Asia underscores the point that historically academic autonomy has been restricted in many countries in the region. This is due to the fact that governments have had the desire to control academic programs, including content, for both public and private universities. But this general tendency is not true for the most developed economies in the region9. For example, Hong Kong, Japan and Singapore guarantee high levels of academic autonomy. Even in Singapore, where until recently the government prescribed enrollment levels in each public tertiary education institution, the three main universities now have the flexibility to enroll up to 10% more students than the prescribed bands set by governments at their own discretion (World Bank, 2011). By contrast, Chinese tertiary institutions are not allowed to accept students outside a pre- determined range. Another area where tertiary institutions have partial autonomy is in terms of staffing. For example, in Indonesia, institutions are allowed to hire and fire faculty and staff that are self-financed. This does not apply, however, to the majority of university staff who continue to be paid by the public budget. These trends and others are shown graphically in Figures 9 through 15 as the degree of autonomy of tertiary education institutions in a number of East Asian countries was evaluated according to the benchmarking indicators. 8Note that survey question on the degree of organizational autonomy in East Asian institutions was not included in the questionnaire. 9 The exception to this rule is South Korea, where autonomy is still relatively restricted (perhaps explaining the weaker research performance compared to its developed country counterparts.) 30 Figure 15 - Financial autonomy – resource mobilization ability of East Asian tertiary institutions (borrowing from commercial banks) 3.5 3 3 3 3 2.5 2 2 Freedom to borrow from commercial banks Score: 1. (1. 1.5 No 2. Yes with restrictions 3. Yes, no restrictions) 1 0.5 0 China Hong Kong Indonesia Japan Figure 16 -Financial autonomy – resource mobilization capacity of East Asian tertiary institutions (setting of tuition fees) 4 3 3 3 3 2 2 2 2 2 2 Set level of tuition and fees 1. No fee-setting authority 2. Yes 1 but with a ceiling on fees 3. Yes, absolute freedom 1 0 31 Figure 17 - Financial autonomy- property management of East Asian tertiary institutions 4 3 3 3 3 3 2 2 2 Universities own and have authority to sell own buildings, 2 facilities and equipment (1. No 2. Partial ownership and selling 1 rights 3. Yes full ownership) 1 0 China Hong Kong Indonesia Japan Thailand Figure 18 - HR autonomy- recruitment of staff 1 1 1 1 1 1 1 1 Universities have the freedom 0 to recruit and dismiss faculty (Yes/No- Score 1 or 0) 0 0 0 0 0 0 0 32 Figure 19 - HR autonomy- freedom to set salaries in various countries 4 3 3 3 3 3 2 2 Freedom to set salaries (1.set by civil service conditions 2. 2 limited margin of movement, 3. total freedom) Score 1-3 1 1 0 Indonesia Hong Kong Japan Singapore Figure 20 - Academic autonomy: freedom to define academic programs 4 3 3 3 3 3 2 2 2 2 2 2 2 2 2 Freedom to define academic 2 structure, programs and course content (Score 1-3; no freedom, 1 some freedom, high degree of freedom) 1 0 33 Figure 21 - Academic autonomy- student admission policy 4 3 3 3 3 2 2 Freedom to recruit 2 students (enrolment size and academic 2 qualifications) (Score 1-3; no freedom, some 1 freedom, high degree of freedom) 1 0 China Vietnam Hong Kong Japan The figures hereafter illustrate the degree of institutional autonomy in Central American countries. In general, tertiary education institutions in this sub-region have the autonomy to set the level of tuition and fees, but in practice public universities do not charge any significant fees as free public education is widely considered as a constitutional right by the students and their families. Financial autonomy varies among private and public tertiary institutions. Private providers tend to enjoy greater autonomy than public institutions. Figure 22 - Financial Autonomy – Resource mobilization – Borrowing from Commercial banks 3 3 3 2.5 2 Freedom to borrow 1.5 from commercial banks 1 Score: 1. (1. No 2. Yes 1 with restrictions 3. Yes, 0.5 no restrictions) 0 Panama Nicaragua El Salvador 34 Figure 23 - Financial Autonomy – Resource Mobilization – Tuition and Fees Figure 24 - Financial Autonomy – Resource mobilization – Bonds 3 3 3 2 Institutions are allowed 1 1 to issue bonds in the 1 financial markets (1. No 2. Yes with restrictions 0 3. Yes, no restrictions) Panama Nicaragua El Salvador Costa Rica Financial autonomy indicators may vary among private and public tertiary institutions. In general, private providers have greater autonomy, as can be seen in the case of Panama. 35 Figure 25 - Financial Autonomy – Resource mobilization in Panama. Private vs. Public Universities Note: 1. No. 2. Yes with restrictions. 3. Yes, no restrictions In general, the community has no representation on the institutional board. Public universities democratically elect the board and their leaders, while private universities have their own selection process since many of them are family enterprises. Finally, it should be noted that meritocracy is not considered when selecting leaders. Figure 26 - Organizational autonomy / institutional oversight – Board Composition 36 Figure 27 - Organizational autonomy / institutional oversight – Selection of Board 2 1 1 1 1 1. Election by faculty and 1 staff 2. Appointment by external committee on basis of search committee 0 Costa Rica El Salvador Panama Guatemala Figure 28 - Organizational autonomy / institutional oversight – Selection of University Leader 3 2 2 2 2 1. Selection of university 2 leader is based on political appointment. 2. Selection of leader is based on 1 internal election. 3. Selection of leader is based on competitive search (score 1, 2 or 3) 0 Costa Rica El Salvador Panama Guatemala With regards to human resources, all the public universities report civil service status of their faculty while some of them have autonomy in recruitment and dismissal of staff. 37 Figure 29 - HR autonomy – Civil Service Status 1 1 1 1 1 Legal status of faculty (civil servant) (Yes=1 /No=0) 0 Costa Rica El Salvador Nicaragua Panama Figure 30 - HR autonomy – Staff Recruitment 2 2 2 2 1 Universities have the 1 freedom to recruit and dismiss faculty (Yes/No- Score 1 or 0) 0 Panama El Salvador Nicaragua Costa Rica In most countries public universities report that they enjoy full academic autonomy: they can set new programs and course content. In some countries, such as Costa Rica, El Salvador and Panama, the private universities have less academic autonomy, since they need the approval of other entities—usually the public universities themselves-- to create new academic programs. Besides, public and private universities have autonomy when it comes to student admissions. 38 Figure 31 - Academic Autonomy – Institutional Strategy 3 3 3 3 3 2 Freedom to define academic structure, programs and course 1 content (Score 1-3; no freedom, some freedom, high degree of 0 freedom) Costa Rica El Nicaragua Panama Salvador Figure 32 - Academic Autonomy – Institutional strategy – Private vs. Public Universities 3 3 3 3 2 2 2 2 Private Universities Public Universities 1 0 Costa Rica El Salvador Panama Note: freedom to define academic structure, programs and course content (Score 1- 3; no freedom, some freedom, high degree of freedom) 39 Figure 33 - Academic Autonomy – Student admission policy 7. Conclusion No good book was ever written on command, nor can good teaching occur under duress. And yet, conceding this, the fact remains that left entirely to their own devices academic communities are no less prone than other professional organizations to slip unconsciously into complacent habits, inward-looking standards of quality, self-serving canons of behavior. To counter these tendencies, there will always be a need to engage the outside world in a lively, continuing debate over the university’s social responsibilities. Bok 1990 The governance of tertiary education, a key dimension of system health, is considered to have a strong impact on the performance of tertiary education systems. Where a sound governance system is in place, decisions at the national and institutional levels are aligned, resulting in optimal resource allocation, use and outputs. This paper has set out a framework for beginning to analyze and understand governance trends and good practices in a more systematic way. Through the formulation of indicators that measure the key components of governance, this benchmarking framework can support policy- makers in assessing the strength of tertiary education governance in their own countries and compare it against reference countries in other parts of the world. The benchmarking of governance dimensions can also be used to assess the relative merits of various types of institutions and governance modalities, for example when comparing states or provinces within a large federal system, or public and private universities within the same country. In Germany, for instance, the impact of the Excellence Initiative is likely to play out differently in each Land (State), depending on the scope and degree of governance reform under implementation in each particular State. In Chile, the absence of a level-playing field in terms of government regulations 40 imposed on the public and private universities that receive public subsidies explains to a large extent the fact that the [private] Catholic University of Chile seems to be outperforming the [public] University of Chile (Bernasconi, 2011). This evaluation of the governance characteristics of tertiary education systems focuses primarily on governance policies and as a result, the majority of indicators are qualitative indicators subject to diverging interpretations. As noted by Fiszbein et al (2011) there is a distinction between governance performance and governance policies, where the former is measured quantitatively, while the latter is measured qualitatively. This limits the possibility of measuring governance structures, policies and processes and assessing with certainty what aspects affect overall system performance and in which way their influence can be felt. It presents a challenge for analyzing the strength of the governance system, unlike other elements of benchmarking of tertiary education systems. For example, when looking at sustainable expansion or resource mobilization strategies, all indicators are of the quantitative type, making it easy to observe progress and carry out comparative analyses in an objective manner. Further work is therefore needed to test the validity of the proposed governance indicators and assess the impact of various governance structures and modalities on the performance of tertiary education systems and institutions. Three final considerations are important with respect to the desirable alignment among the main accountability dimensions of the governance benchmarking. First, one should look carefully at the gap between theory and practice. It is indeed not sufficient to ascertain what governance structures and processes exist on paper in a country. The analysis of tertiary education governance must always distinguish between what is expected to happen according to the prevailing laws and regulations and what actually occurs in the system as a whole and in individual institutions. Enforcement of and compliance with existing rules are also a critical dimension of accountability. Second, the performance of tertiary education institutions may be negatively affected by divergences between the regulations and the results components of the accountability framework. Governance reforms in various parts of the world have tried to address this lack of alignment by encouraging greater institutional autonomy and putting more emphasis on results than on compliance with regulations per se, as summarized by the following quote illustrating recent developments in Canada. The greater interest in accountability has played out differently by sector and province, recognizing the quite different relationships between governments and the institutions and changes over time in the perceived intent and value of accountability initiatives. Initially seen as intrusive and a recipe for government micro-management with a single goal of containing expenditures, the value of good accountability frameworks is now generally recognized as an important ingredient in the overall management and operation of post-secondary institutions. Moreover, over time, the emphasis has shifted from a more narrow view of adherence to policies and procedures and financial accountability, to a more comprehensive view of accountability with an onus on multi-year plans and 41 performance measures—often developed jointly (or at least with some consultation) by government and the institutions. Snowdon (2005) Thirdly, it is worth noting that the comparative approach taken in this article tracks only the average performance of institutions in the system and gives no way of measuring in a global manner how different one system is from another. That is, the current analysis tells which countries are doing better than others on particular indicators but does not give a single measure of how much better it is doing. An alternative approach would be to calculate the mean and standard deviation from the mean as a way to group systems into high, medium and low performers to observe how different they are within and outside the group. As emphasized at the beginning of this paper, this is an initial exploration of benchmarking issues and the set of indicators proposed need to be tested and expanded further. An important part of this process is to map proposed indicators to the evidence base. One of the challenges of this type of mapping exercise is that there is currently no comprehensive database on most governance aspects discussed in this paper. As a result, much of the data gathered in this study required specific country surveys relying on expert judgments. Further research linking university governance to performance and the datasets accompanying this research would be valuable for the next phase of this study. 42 Annex 1 - Existing Databases of Governance Indicators for Benchmarking Purposes Examples of systematic use of indicators to assess the governance characteristics of tertiary education are few and far between. To date, there is no comprehensive database on the state of governance of tertiary education systems and institutions. However, a small number of organizations have conducted surveys on the topic including the OECD, the EUA, the EURYDICE Network and the World Bank. In 2003, the OECD published a study titled Changing Patterns of Governance in Higher Education, based on surveys of university governance in 15 member institutions of the OECD’s Institutional Management in Higher Education Program (IMHE). Table 5 illustrates some key dimensions of autonomy enjoyed by universities in 14 countries. Table 5 - Extent of Autonomy Experienced by Universities in 14 OECD countries Source: OECD, 2003, 63 The table shows that, overall, institutions have autonomy over expenditure of budgets and management of academic staff. This study confirms the trends described at the beginning of Section 3 of this Paper, namely that government have been withdrawing from their traditional role of direct management of universities, allowing for greater institutional 43 autonomy. At the same time, however, governments are introducing new mechanisms (i.e. through funding and quality assurance) to ensure that institutions remain accountable for their performance. In 2009, the European University Association, an organization that represents and supports more than 800 universities in 47 countries, published the results of an exploratory study on the enabling conditions for university autonomy (Estermann and Nokkala, 2009). The purpose of the study was to provide the foundation for a Europe- wide database of comparable information on different aspects of university governance and autonomy. It also aimed to bring the institutional perspective (i.e. what autonomy really means in practice) into the European debate in order to influence reforms at the policy level. The EUA identified the same four key dimensions driving institutional autonomy as the ones used in the benchmarking framework, including (i) organizational autonomy, (ii) financial autonomy, (iii) staffing autonomy, and (iv) academic autonomy. Within each category, the EUA identified features determining each dimension. So for example, in the case of the first dimension of organizational autonomy, the key features are internal academic and administrative structures, the composition of the institution’s governing bodies, and the ability of the institution to select its executive leadership. The study, which covered 34 universities, relied on an online questionnaire addressed to the National Rector’s Conferences (NRC) in countries with universities belonging to EUA. The questionnaire focused on the legal status of institutions, institutional strategies, management and governing structures, financial issues, students, human resources, intermediary bodies and overall autonomy. The study revealed that the framework and conditions under which European universities operate vary greatly between and sometimes within countries, with some noteworthy trends. First, in terms of organizational autonomy, universities in almost all the systems under review had regulations in place allowing for institutional autonomy with regulations requiring accountability to the State. In the majority of countries, universities are relatively free to decide on administrative structures. In addition, there is a trend toward the inclusion of external members in the university decision-making processes, especially where universities have dual governance structures (academic and corporate) (Estermann and Nokkala, 2009). In terms of staffing autonomy, the analysis reveals that, in some countries, universities are gaining a greater flexibility in their staffing autonomy, in particular as staff is generally directly paid and/or employed by the university rather than by the government, as the following examples illustrate: x Czech Republic- Individual tertiary education institutions determine the number of academic staff in each rank. Institutions also now have the authority to set the length of contract for staff (i.e. fixed term versus indefinite) x Denmark – The government’s policy on Strategy in the Global Economy calls for universities to have increased freedom in attracting highly talented researchers through monetary incentives 44 x France- Introduced a new law in 2007 that gives institutions autonomy in human resource management x Hungary- Introduced a law in 2005 titled the Act on Higher Education which gave increased scope for autonomy in staffing. As a result, institutions now “decide on matters of employment, freely select staff, and designate their duties based on institutional regulation and expectations regarding performance and quality of work” (Euridyce, 2008, 22) x Finland, Romania, Austria have established performance criteria and/ or introduced performance-based pay systems in their tertiary education institutions. However, many universities are still restricted in their ability to set salaries for individual staff members. This power continues to lie in the government’s domain. Furthermore, in almost half of the countries surveyed, all or the majority of staff continue to have civil servant status. In terms of financial autonomy, in the majority of countries institutions receive funding via block grants. Most universities across Europe have the autonomy to borrow funds, but there are restrictions on the amount allowed. In the majority of countries analyzed, institutions can collect tuition fees and administrative fees from students. The study noted the general finding that Western European countries had more autonomy in the use of funding that they receive, but less autonomy in setting tuition and administrative fees in comparison to their Eastern European counterparts. Finally in terms of academic autonomy, the main issues are institution’s ability to determine their own academic profile. In general, most governments across Europe require that institutions obtain some type of approval before establishing new degree programs. In a third of the countries surveyed, universities are able to regulate the number of students per discipline. Similarly in 2009, the Eurydice Network published a report titled “Governance in Higher Education” which provides information on and analyses of 33 European tertiary education systems. The paper provides a useful collection of data on the governance framework of the countries surveyed. In particular, the document provides in depth description and comparative analysis on the following: x national strategic policies for tertiary education with a focus on funding and staffing of institutions; x scope of external regulation (i.e. at national /international level) and discussion on the state of institutional autonomy as it relates to academics and human resource issues; x analysis of public funding mechanisms and their influence on performance, competition and accountability of institutions to society; and x resource mobilization in the context of private financing. Overall, the study concludes that there is a trend toward less prescription regulatory frameworks at the national level, leading to the strengthening of academic self- 45 governance while maintaining accountability to the state. Countries have developed a range of models suiting the demands and traditions of their nations. In 2011, the EUA published a follow up to this report, titled University Autonomy in Europe II- The Scorecard which elaborates on the original report by presenting a country- wise ranking on each of the four dimensions of autonomy. Countries receive a score out of 100 based on the relative “freedom from restriction” enjoyed by institutions in specific areas of organizational, financial, staffing and academic autonomy and are classified into four groups (high, medium high, medium low, and low) . The purpose of this scoring card is to focus attention on the issue of university autonomy and to inspire policy reform on specific regulations identified as being “restrictive” to university autonomy. (Estermann et al, 2011) References Aghion, P. M. Dewatripont, C. Hoxby, A. Mas-Colell, and A. Sapir, 2007. “Why Reform Europe’s Universities.” Bruegel Policy Brief. Brussels: Bruegel. Available via the Internet: aei.pitt.edu/8323/ Aghion, P. M. Dewatripont, C. Hoxby, A. Mas-Colell, and A. Sapir (2008). Higher Aspirations: an Agenda for Reforming European Universities”. Brussels. (http://www.bruegel.org/publications/publication-detail/publication/1-higher- aspirations-an-agenda-for-reforming-european-universities/). Aghion, P., M. Dewatripont, C. Hoxby, A. Mas-Colell, and A. Sapir (2009). “The Governance and Performance of Research Universities: Evidence from Europe and the U.S.” National Bureau of Economic Research. Working Paper No. 14851, April 2009. Altbach, P. and J. Salmi (2011). The Road to Academic Excellence: the Making of World-Class Research Universities. Directions in Development; World Bank, Washington, D.C. Anderson, J.A. (2005). Accountability in Education. IAE and IIEP Education Policy Series No. 1. Paris and Brussels. Baye, A. and Mainguet, C. (2006) “Defining a framework of indicators to measure the social outcomes of learning.” Paper in “Measuring the effects of education on health and civic engagement: Proceedings of the Copenhagen Symposium). Bernasconi, A. (2011). “Private and public pathways to world class research universities: the case of Chile”. Altbach, P. and J. Salmi, eds. “The Road to Academic Excellence: Emerging Research Universities in Developing and Transition Countries”. Washington DC: Directions in Development. The World Bank. (forthcoming). 46 Campbell, David. (2006) “What is education’s impact on civic and social engagement?” Measuring the effects of education on health and civic engagement: Proceedings of the Copenhagen Symposium). OECD. Cajas, F. (2010). “Gobernabilidad de la educación superior en Centro América”. Report produced for the World Bank. Unpublished. Education at a Glance. OECD. 2009. Estermann, Thomas, Nokkala, Terhi, Steinal, Monika. (2011). University Autonomy in Europe II. European University Association. Estermann, Thomas and Nokkala, Terhi. (2009) “University Autonomy in Europe I.” European University Association. Eurydice. Higher Education Governance in Europe. (2008). Eurydice: Belgium. European University Association, The EUA Lisbon Declaration – Europe’s Universities beyond 2010: Diversity with a Common Purpose, EUA, Brussels, May 2007 Fiszbein, A., Ringold, D. and F. Halsey Rogers (2011). “Making Services Work: Indicators, Assessments and Benchmarking of the Quality and Governance of Human Service Delivery in the Human Development Sectors.” Policy Research Working Paper No. 5690. Washington DC: The World Bank. Geiger, R. (2005), Knowledge and Money: Research Universities and the Paradox of the Marketplace, Stanford University Press, Stanford. Henard, Fabrice, 2009. Good Governance and the Transformation of Higher Education Institution: A review of Higher Education Governance Literature, Governance Guidelines and Quality Assurance Guidelines. Paris: OECD Huisman, J. and J Currie, 2004. “Accountability in Higher Education: Bridge Over Troubled Water? “ Higher Education 48: 529-551. Johnstone, B and Marcucci, P. (2010). Center for Comparative and Global Studies in Education of the State University of New York at Buffalo. International Comparative Higher Education Finance and Accessibility Project (ICHEFAP). Minnesota Office of Higher Education (MOHE). 2008. Minnesota Measures: 2008 Report on Higher Education Performance. St. Paul, MN: Minnesota Office of Higher 47 Education, http://www.ohe.state.mn.us/pdf/MinnesotaMeasures2008.pdf. Accessed January 2009. Morley, L., M. Eraut, S. Aynsley, D. MacDonald and J. Shepherd (2006), “Establishing the Needs of Employers and Related Organisations for Information about Quality and Standards of Higher Education Provision and Student Achievement in England”, Higher Education Funding Council for England (HEFCE), Bristol. OECD. (2003). Education and Policy Analysis. Paper 3- Changing patterns of governance in higher education. < http://www.oecd.org/dataoecd/0/20/35747684.pdf> OECD, 2008. Tertiary Education for the Knowledge Society Volume 1 ,Paris: OECD Patrinos, Harry Anthony. 2012. Chapter 4 “Student Assessments” and Chapter 5 “Teacher Policies” in Strengthening Education Quality in East Asia. Washington DC: The World Bank. Penn, J. 2007. “Assessment for ‘Us’ and Assessment for ‘Them.’” Inside Higher Education. June 26, 2007. Raza, Reehana (2010). “Higher Education Governance in East Asia,” A paper prepared for the East Asia Flagship Report on Higher Education. World Bank, Washington D.C. Salmi, J. (2011). “The Road to Academic Excellence: Lessons of Experience”. Altbach, P. and J. Salmi, eds. (2001). The Road to Academic Excellence: Emerging Research Universities in Developing and Transition Countries. Washington DC: Directions in Development. The World Bank. Salmi, J., (2009). The Challenge of Establishing World-Class Universities. Directions in Development; World Bank, Washington, D.C. Salmi, J., and A. Saroyan (2007). “League Tables as Policy Instruments: Uses and Misuses.” Higher Education Management and Policy. Volume 19, No. 2. OECD, Paris. Salmi, J. (2007). “The Growing Accountability Agenda: Progress or Mixed Blessing?”. International Higher Education. Number 50, Winter 2007. Santiago, Paulo, Tremblay, K , Basri, Ester and Elena Arnal. (2008) Tertiary Education for the Knowledge Society: Thematic Review of Tertiary Education. OECD Paris. 48 Snowdon, K. (2005). “Without a Roadmap: Government Funding and Regulation of Canada’s Universities and Colleges.” Research Report W31. Canadian Policy Research Networks, Ottawa, Canada. The World Bank. (2011) “Skills and Research for Productivity and Growth: Higher Education in East Asia Main Report.” Human Development Department, East Asia Region. 49