This document was prepared by: Centre for Financial Reporting Reform (CFRR) Governance Global Practice, The World Bank Praterstrasse 31 1020 Vienna, Austria Web: www.worldbank.org/cfrr Email: cfrr@worldbank.org Phone: +43-1-217-0700 This report was prepared by Alfred Jean-Marie Borgonovo and Lejla Begtasevic Rudalija. © 2018 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. CONTENTS Introduction ............................................................................................................................... 1 Tertiary education in Serbia................................................................................................. 2 ECTS Units ............................................................................................................................ 3 Program goals ...................................................................................................................... 3 Syllabi and curricula ............................................................................................................. 4 1. UNIVERSITY OF BELGRADE, FACULTY OF ECONOMICS .................................................. 6 1.1. Faculty ....................................................................................................................... 6 1.2. Study programs ......................................................................................................... 7 1.3. Faculty body ............................................................................................................ 11 1.4. Student body ........................................................................................................... 13 1.5. Syllabi analysis ......................................................................................................... 15 2. KRAGUJEVAC UNIVERSITY, FACULTY OF ECONOMICS ................................................ 19 2.1. Faculty ..................................................................................................................... 19 2.2. Programmes ............................................................................................................ 20 2.3. Faculty body ............................................................................................................ 25 2.4. Student body ........................................................................................................... 25 2.5. Syllabi analysis ......................................................................................................... 27 3. UNIVERSITY OF NOVI SAD, SUBOTICA FACULTY OF ECONOMICS ................................ 34 3.1. Study programs ....................................................................................................... 34 3.2. Faculty body ............................................................................................................ 39 3.3. Student body ........................................................................................................... 40 3.4. Syllabi analysis ......................................................................................................... 41 4. UNIVERSITY OF NIŠ, FACULTY OF ECONOMICS ........................................................... 47 4.1. Programmes ............................................................................................................ 48 4.2. Faculty body ............................................................................................................ 53 4.3. Student body ........................................................................................................... 53 4.4. Syllabi analysis ......................................................................................................... 55 5. PROFESSIONAL ACCOUNTANCY ORGANIZATIONS (PAOs) .......................................... 58 5.1. Chamber of Authorized Auditors (Chamber) .......................................................... 58 5.2. Serbian Association of Accountants and Auditors (SAAA) ...................................... 63 5.3. Continuing Professional Development (CPD) requirements................................... 66 Appendix 1. Constructive Alignment ....................................................................................... 69 Intended learning outcomes (ILOs) ................................................................................... 69 Program ILOs ...................................................................................................................... 69 Course ILOs ........................................................................................................................ 70 Teaching and learning activities......................................................................................... 71 Assessment ........................................................................................................................ 73 Appendix 2. Developing ILOs ................................................................................................... 75 INTRODUCTION High quality financial reporting is the cornerstone of a well-functioning market economy and a robust financial system. Improved quality of financial reporting strengthens domestic financial architecture and lowers the high cost of information. This allows investors to better evaluate corporate prospects and make informed investment and voting decisions, which in turn results in a lower cost of capital and a better allocation of resources. Such enhancement of the investment climate represents the central development objective underlying the Serbia Technical Assistance for the Reform of Corporate Financial Reporting (STAR-CFR) project. The project’s five technical components are: 1. Aligning statutory framework with the acquis communautaire and best practice; 2. Promoting better formulation, application, understanding and use of corporate accounting and auditing standards; 3. Building effective programs of monitoring and enforcement of corporate accounting & auditing standards; 4. Improving accounting and auditing curricula for university and professional education programs; 5. Institutional strengthening and monitoring of SOEs. Sustainable achievement of these outcomes also requires simultaneously ensuring that current and aspiring accounting and auditing professionals have significant competence in financial accounting and reporting (based upon the International Financial Reporting Standards (IFRS) and other reliable reporting standards), auditing and internal controls (through the application of the International Accounting Standards (ISAs)), and a number of other broader business domains such as management, accounting, finance, taxation, strategy, and governance. It is also imperative that professional competence extends to non-technical areas such as problem-solving and decision-making, critical thinking and integration, ethical judgment, team leadership and development, and communications. Component 4 of the project will contribute to the development of the accountancy profession by building capacity in the essential competence areas. The focus is on both improving the quality of accounting and auditing education and assessment to enhance the capabilities and professional judgment of Serbia’s accountants and auditors, as well as strengthening the institutional frameworks that support certification and continuing professional development. In doing so, the project is underpinned by the International Education Standards (IESs) maintained by the International Accounting Education Standards Board (IAESB), and the Directives of the European Union (EU Directives). 1 The project is working to help higher education providers equip their students with the knowledge, skills, and experience they will need for a successful career in an increasingly global marketplace. In a rapidly changing world, economic forces, demographics and other trends create new needs. Higher educational institutions (HEIs) can be old fashioned and slow to change but they present a huge area for improvement and great capabilities of doing so. The four state Faculties of Economics: Belgrade, Kragujevac, Subotica and Nis, together with two professional accountancy organizations, were chosen to be supported by the STAR-CFR. A detailed curriculum evaluation and assessment was made of the quality of the accounting and audit stream within its Bachelor’s and professional programs. The results are summarized in this report, providing an insight into how their accounting and audit education compares with the best higher education systems globally that are setting standards of what can be achieved. This should provide a unique tool to help facilitate improvements that will better prepare students for their future careers. The report is based on the assessment conducted in 2017 covering the following main parts: i) Faculty ii) Study program iii) Faculty bodies iv) Student body v) Syllabi analysis Recommendations are made, based on the assessment results, that could help guide the institution towards achieving the IESs, as well as standards and criteria contained in educational standards described by the international business school accreditation agencies. While the different accreditation agencies have their own sets of standards and criteria for institution and/or program assessment, they are all generally similar in nature. Compliance with any set of standards or criteria is an inevitable part of the accreditation procedure, usually described as self-assessment. The assessment, feedback, and recommendations contained in this report therefore focus on key relevant and vital elements of the programs that can be applied to any individual standard or accreditation agency request. TERTIARY EDUCATION IN SERBIA Serbia joined the Bologna process in 2003. The Bologna process aims to create the European higher education area by harmonizing academic degree standards and quality assurance standards throughout Europe. The objectives of the Bologna process are the introduction of undergraduate and postgraduate level degrees in all countries, with first degrees no shorter 2 than 3 years; the elimination of obstacles to the mobility of students and teachers; and the introduction of a European Credit Transfer System (ECTS). According to the Serbian Law on Higher Education, every study programme is defined through the number of ECTS. Each course within the study programme is valued through the number of ECTS. Total sum of ECTS varies from a minimum of 180 for three-year programmes to a minimum of 240 ECTS for four-year programmes at the bachelor’s level. At the master’s level a number of ECTS may vary from a minimum of 60 to 120 ECTS depending on the length of the programme (one or two years). Programmes at the PhD level comprise at least 180 ECTS. Accounting and auditing education falls within the Economics faculty of most universities, at the undergraduate (bachelor’s) and potentially graduate (master’s) level. All students therefore graduate with a degree in Economics. ECTS UNITS Following the Bologna concept, student workload is measured using ECTS. The full-time workload for one academic year is 60 ECTS. One ECTS represents 25-30 hours of study activity for an average student. A workload should include all the time an average student spends preparing and performing different tasks and activities (not only contact hours) in order to pass one course. A course with five ECTS, for example, means that the average student needs to spend 125- 150 hours preparing for the exam. 1 An average course might have five contact hours per week (3 hours of lectures and 2 hours of tutorials), which over an average 15-week semester will total 75 hours. The further 50 hours represents time spent on all other required activities such as assignments, research, and presentations, field work, reading, learning, revision for midterm or final exams, preparing for quizzes and tests, etc. ECTS cannot be linked to the amount of course reading or any other similar measure (such as numbers of set text books, lectures, or exam questions). PROGRAM GOALS The review of the content of the bachelor’s programs is based on John Biggs’ “Concept or Theory of Constructive Alignment” (adapted in 2003, based on the first theory from 1999). According to this theory, constructive alignment has two aspects. The constructive aspect refers to what the learner does, which is to construct meaning through relevant learning 1 In social sciences it is usually 25 hours per 1 ECTS, while in natural sciences and medicine it can be 30 hours. 3 activities. The alignment aspect refers to what the teacher does, which is to set up a learning environment that supports the learning activities appropriate to achieving the learning outcomes. The focus is to determine the desired outcomes first and then decide how these should be taught and assessed. It is important to clearly define the purpose and role of the study program. Improved goals should be developed which include the outcomes of learning, and the specific knowledge, skills, and competences that will be developed. Program goals should explain the purpose of the education, giving an answer to the question: why study this course? The distinction between course goals and intended learning outcomes (ILOs) should be obvious. The goal of the course should be oriented towards the ILOs, while the ILOs are oriented towards students, describing the knowledge, skills, and competences that students will have after the successful completion of the course. The goals should be clearly linked to the mission statement and vision, and should: • be relevant, real, understandable, and achievable; • guide how the course is delivered; • be adapted to the level of knowledge appropriate for the level of the course; • be linked to the ILOs; • form the basis for the final examination. Any linkage to future jobs should be stated with care. Descriptions and requirements of specific jobs are set by the national qualifications framework, which should be in compliance with the European qualification framework. Based on this, governments develop qualifications and occupational standards (based on agreed descriptors) for certain professions. SYLLABI AND CURRICULA Curricula include study structure, a list and structure of compulsory and elective courses and their descriptions, course schedule, structure and content of the study program regarding the relationships between academic and general education, theoretical, methodological, scientific and professional subjects. Student workload is expressed by ECTS for each course. All courses should include the same syllabus elements (regardless of specialization and/or stream). These are: title of the course, course code, number of ECTS, faculties delivering the course, course type, in-class activities per week, prerequisites, goal of the course, intended learning outcomes, course content, teaching methods, assessment methods, and specify compulsory and supplemental literature. 4 Linkages between courses are very important; the same structure enables student’s knowledge and improvement/progression to be tracked throughout their studies. The content of individual courses should be aligned both horizontally and vertically with all other courses in the same year, and within the same specialism and/or stream. Overlapping of course content should be avoided, except in individual cases where a slight overlap is deemed necessary. The content of each course should take into account knowledge gained in previous courses, the knowledge gained during the year for a given course, and the knowledge to be gained which is relevant to a specific specialization/stream. Staying mindful of legal obstacles, including a wide range of stakeholders, as illustrated in the diagram below, in the curriculum revision procedure would help improve both the content and the structure of the curriculum. Their different input could be very valuable, for example: the Government to define insufficient occupations in the market; faculties (professors and teaching assistants) to give insight as those responsible for implementing curricula; the business community to help define required skills and competences; alumni to provide feedback on previous curricula; and current students to articulate their ongoing needs from the course. Figure 1. Curriculum impact elements Qualification framework (EQF and local) Local and Priorities international indicators Curriculum Previous Academics and experience students Professsional Busines standards community 5 1. UNIVERSITY OF BELGRADE, FACULTY OF ECONOMICS 1.1. FACULTY The Faculty of Economics of the University of Belgrade (EFBG) is a public higher education institution (HEI) established in 1937 that has been a constituent part of the University of Belgrade since its founding. Generally thought to be the leading school in Serbia, it is also the oldest and largest university in the country, dating back to 1808. EFBG is partially financed from the state budget which covers approximately 35% of total annual average revenue. Students’ fees contribute 55% to the total revenue, and all other sources of income contribute to the total by approximately 10% (projects, consultant work, non-degree education, etc.). EFBG has the autonomy to propose the structure of curriculum but the proposal is subject to the approval by the University Senate and Accreditation Committee of the Republic of Serbia. EFBG has full autonomy to introduce new programs, however the curricula of the program have to go through the above-mentioned approval procedure. The student enrolment quotas are subject to the decision by the Government of Serbia, and EFBG has no autonomy to determine the size of the quotas. EFBG has the autonomy to determine admission requirements, however that autonomy will only last until the 2018 school year when the Ministry of Education, through the state high school graduation exam, will take over the responsibility of admission requirements. EFBG has both local and international professional accounting bodies’ exemptions and they are the only faculty of Economics in Serbia that has achieved all these exemptions. The Association of Chartered Certified Accountants (ACCA) provides exemptions from the following exams to students completing the Accounting, Auditing and Corporate Governance module (4 out of 9): • F1 Accountant in Business • F2 Management Accounting • F3 Financial Accounting • F4 Corporate and Business Law The Association of Accountants and Auditors of Serbia provides exemption from the following exams, for the professional title of Certified Accountant, to students completing the Accounting, Auditing and Financial Corporate Governance module (5 out of 9): 6 • Preparation of Financial Statements • Financial Information for Management • Managing People • Information Systems • Corporate and Business Law The Chamber of Authorized Auditors provides exemption from the following exams, for the professional qualification of Certified Auditor, to students completing the Accounting, Auditing and Corporate Governance module (9 out of 13): • Theory and Principles of Accounting • Financial Analysis • Management Accounting • Risk Management and Internal Control • Theory of Auditing • Business Information Systems • Microeconomics, General and Financial Economics • Financial Mathematics and Statistics • Financial Management EFBG has prepared and started implementing a Strategic Plan for managing its interface with the corporate world, for the period 2015-2020. The plan contains directions of cooperation with the corporate world in terms of internship programs for students, visiting lecturers from the business field, student case study contest in cooperation with the corporate world, and business simulations. The Faculty has some engagement with local audit firms, and an active engagement with international audit firms, business community, and government bodies. 1.2. STUDY PROGRAMS In 2003, EFBG started the transition to the Bologna concept and introduced the European Credit Transfer and Accumulation System (ECTS). It adopted the Bologna process as follows: 4 years of undergraduate study (240 ECTS), 3 years of undergraduate study (180 ECTS), 1 year of graduate study (60 ECTS), and 3 years of doctoral study. The six departments of EFBG are organized according to areas of expertise: Economic Theory and Analysis; Economic Policy and Development; Business Economics and Management; 7 International Economic Relations; Accounting and Business Finance; Statistics and Mathematics. For the first four semesters, all students follow the same basic curriculum. This provides a good foundation for later specializations. The studies are revised periodically (approximately every four years) to ensure that the program is meeting requirements; slight changes are allowed between revisions, such as changes in updated course materials or changes in assessment methods. In the third year, students opt for one of the nine available specialisms: Economic Analysis and Policy; Marketing; Accounting, Audit and Corporate Governance; Trade Management and Marketing; Finance, Banking and Insurance; Statistics, Computer Science and Quantitative Finance; Tourism; Management; or International Economics and Foreign Trade. Students opting for the Accounting, Audit and Corporate Governance module make a further choice in the seventh semester between Accounting and Audit or Corporate Governance. This study focuses on the accounting and audit stream of the Bachelor’s degree course in Accounting, Audit and Corporate Governance. The courses taught in the Accounting, Audit and Corporate Governance concentration are listed in Table 1. Table 1. Belgrade: Accounting, Audit and Corporate Governance concentration courses No of classes per week Year Subject ECST Theory Practice 1st Semester Foundations of Economy 4 4 10 Foundations of Statistical Analysis 4 4 10 Mathematics 4 4 10 1st Semester Total 30 1 2nd Semester Sociology 4 0 4 Macroeconomics 4 3 8 Enterprise Economics 4 3 8 Financial Accounting 4 4 10 2nd Semester Total 30 8 No of classes per week Year Subject ECST Theory Practice 3rd Semester Theory of Prices 4 4 10 Economic Mathematics Methods and Models 4 4 10 National Economy 4 3 9 Foreign language 1 (English, German, Russian, 4 0 4 French) 3rd Semester Total 33 4th Semester International Economy 4 2 7 Strategic Management 4 4 10 Foreign language for economists 2 (English, 3 0 3 German, Russian, French) 2 Elective (choose 1 out of 10): 1. Methods of Economic Analysis 4 2 7 2. Marketing 4 2 7 3. Management Accounting 4 4 10 4. Marketing Channels 4 2 7 5. Monetary Economy 4 2 7 6. Economics of Tourism 4 2 7 7. Business informatics 4 2 7 8. Enterprise Organisation 4 2 7 9. International Finances 4 2 7 10. International Trade 4 2 7 4th Semester Total 27 5th Semester Business Finances 4 4 10 Cost Accounting 4 3 9 3 Financial and Actuarial Mathematics 4 2 7 Theory and Policy of Financial Statements 4 4 10 5th Semester Total 36 9 No of classes per week Year Subject ECST Theory Practice 6th Semester Economic Statistics 4 2 7 Enterprise Organization 4 2 7 Business Law 4 2 7 Foreign language for economists 2 (English, 3 0 3 German, Russian, French) 6th Semester Total 24 7th Semester Special Financial Statements 4 3 8 Public Finances 4 2 7 Auditing 4 4 10 Comparative Accounting 2 2 5 7th Semester Total 30 4 8th Semester Monetary Economy 4 2 7 Accounting Information Systems 4 3 8 Special accounting 3 2 6 Internal accounts 2 2 5 Graduation theses 0 0 4 8th Semester Total 30 The ECTS are currently assigned to specific courses mainly based on the number of contact hours. EFBG should ensure that contact hours are not the sole measurement of ECTS. Workload should include all the students’ efforts and time spent writing, learning, reading, preparing for class, as well as contact hours. It can also be seen that the workload is not equally distributed throughout the semesters, but totals 60 ECTS per academic year. Curriculum review is done on a 5-year basis and is linked to the national accreditation process. Only minor changes are done within the 5-year accredited period which ensures consistency of program delivery. The curriculum creation procedure should be improved. Since the EFBG is a state owned faculty, legal obstacles regarding the procedures need to be taken into consideration. However, in order 10 to improve the content and structure, the Faculty should include different stakeholders in the curriculum creating process. Based on the program structure, it does not appear that EFBG has any collaboration with the business community when it comes to program creation and/or delivery. In the curriculum, it is not prescribed that students need to have an internship and/or any kind of similar project. Internship is considered as a voluntary activity, so if students complete an internship, it will be written in the diploma as a supplement, but no specific ECTS will be assigned for that activity. The manner of defining the intended learning outcomes (ILOs) varies throughout the Faculty. Namely, some ILOs are defined properly considering all elements that should be taken into account when defining the learning outcome of the degree course. On the other hand, some ILOs are very similar to the course goal, so no significant distinction can be made. In the case of EFBG, the teaching methods mentioned in the courses’ syllabi are very similar. Those encompass ex-cathedra lecturing in majority of cases. There is no evidence that a variety of teaching methods exist, such as seminar, team work, or field work/trips. Also, the presence of guest lecturers is not evident. There is no evidence that the ILOs are aligned with the assessment. Depending on the course, different assessment methods are applied (midterm, presentations, homework). However, the final exam usually carries 70% of the final grade in a great number of cases. The Bologna concept prescribes the method of knowledge testing during the semester in order to give students an incentive to learn, or else risk missing the required grade to pass the course. The midterm carries a small portion of the final grade in the majority of cases. In some cases, the oral exam carries a large portion of the final grade. It is good practice to have an oral exam as different levels of knowledge can be assessed; however, this type of exam risks being subjective. When determining the weight of the oral exam, it is not recommended that it constitute a huge portion of the final assessment. Humans are prone to making errors, so subjectivity needs to be taken into account as an external factor that can influence the final result. What experience has shown is that the oral exam should not carry more than 15% of the final grade. 1.3. FACULTY BODY The total teaching/lecturing staff in the Accounting/Audit department comprises of 16 professors. There are 6 full time professors, 1 associate professor, 3 assistant professors, 5 assistants and 12 associates in education. All of them are full-time employed except 1 associate professor that is part-time employed. They are also expected to employ two additional lecturing staff shortly. 11 Figure 2. Belgrade: Age profile of the Figure 3. Belgrade: Gender profile of the professors professors 61-70 2 51-60 2 Women 41-50 5 7 9 Men 31-40 5 under 30 2 Student/staff ratio has decreased in 2015, from 41.9 to 33.75. Given the number of students, the Faculty deems that hiring at least 5 more teaching staff is necessary in order to be comfortable with the student/professor and courses/professor rate. According to the accreditation rules, full professors need to realize 120-180 teaching hours, associate professors and assistant professors 180-240 teaching hours, and teaching assistant 240- 300 hours. Salaries are thought to be competitive in the market, even though it is not considered a primary motive to work for the Faculty. One of the most important motives to join the Faculty is the ranking and importance of the Institution. Professors are also allowed to earn additional income, through research and consultancy projects at the Faculty. Figure 4. Belgrade: Composition of the Accounting faculty with regard to their scientific degree Graduate 1 Masters 5 Doctors 10 10 members of the Accounting faculty received local doctorate degrees in Accounting/Auditing, 5 local master’s in Accounting/Auditing, 1 holds international professional qualification (CFA), 2 are Certified Public Accountant and 1 Certified Accountant (by Association of Accountants and Auditors of Serbia). 12 1.4. STUDENT BODY Looking at students’ admissions in the period of 2013 to 2015, there is a slight increase of students’ enrollment in 2014. The results in 2015 showed that there is an increase of students choosing the accounting program. There is a lack of available data to calculate the drop-out rate, however the general estimate for the Faculty is very low, and 0% for the Accounting, Auditing and Corporate Governance concentration. Table 2. Belgrade: Number of undergraduate students’ admissions and number of graduates per year 2 Number of admissions 1372 Number of admissions in AACG 246 Number of graduates 826 Figure 5. Belgrade: Number of students 24%, 324 2015 1372 22%, 309 2014 1432 22%, 309 2013 1394 % choosing A&A Students admission A total of seven students, in the period from 2013 to 2015, have taken advantage of the ERASMUS, CEEPUS and JOSZEF programs. English language proficiency of the students has been rated 3 out of 5. The average grade for all students (not just Accounting) on English language proficiency is 7.46 out of 10 for the year 2014/2015. 2 Data refers to academic 2015/2016 13 Figure 6. Belgrade: Gender profile of students 21% Women Men 79% The business community is the largest internship provider as well as employer of the Faculty’s graduates. Figure 7. Belgrade: Internship and employment Non-profit organizations 5 5 Government bodies 15 15 Busines community 40 40 Financial sector 15 15 International audit firms 10 10 Local audit firms 15 15 Employment Internship The Faculty is using a survey questionnaire after graduation to determine the employability of graduates at the overall Faculty level. Based on these reports, the students of the Accounting, Auditing and Corporate Governance concentration are in the most favorable position among all student groups, as more than half of these students find a job within six months of graduation. The Faculty has established tracking surveys to measure graduate employment rates, the average unemployment period after graduation, and the average number of years required to complete a degree. 14 1.5. SYLLABI ANALYSIS 1.5.1. Syllabus profile against ACCA profile The syllabi generally show strong coverage of technical audit and assurance topics (96% exceeding the exemption threshold of 80%), technical financial reporting topics (84% exceeding the exemption threshold of 80%), and technical financial management topics (83% exceeding the exemption threshold of 80%). With regards to the content of the courses, the Faculty is ready to submit an exemption claim to ACCA; however, some changes will need to be introduced to the assessment methods in order to achieve these exemptions. The F7 Financial Reporting syllabus is covered by 5 courses at the Faculty. The majority of the syllabus is covered by Theory and Policy of Financial Statements, Financial Accounting, and Special Purpose Financial Accounting. A small percentage (5-10%) of topics is covered by Management Accounting and Corporate Finances. Figure 8. Belgrade: ACCA Benchmark F9 Financial Management 83% F8 Audit and Assurance 96% F7 Financial Reporting 84% F5 - Performance Management 78% F3 Financial Accounting 73% F2 Management Accounting 93% F1 Accountant in Business 84% Belgrade ACCA benchmark The F8 Audit and Assurance syllabus is covered by one auditing course, and F9 Financial Management is covered by five different courses; the majority of topics in Corporate Finance, and small percentage of topics in Strategic Management, Principles of Economics, Monetary Economics, and International Economics. At a somewhat deeper level, there are no ACCA subjects in which the Faculty is showing significant weakness in its syllabi (generally lower than 60% coverage against the ACCA program benchmark). F5 Performance Management achieving 78% is close to the ACCA 15 benchmark of 80%. The Faculty needs to introduce minor changes in order to achieve this benchmark. Table 3 contains some suggestions. Table 3. Topic areas with weaknesses (Belgrade, ACCA) Topic Covered in course Environmental Accounting Coverage: 0% Discuss the issues business face in the management of - environmental costs Describe the different methods a business may use to - account for its environmental costs Quantitative Analysis in Budgeting Coverage: 33% Analyse fixed and variable cost elements from total cost data Cost Accounting using high/low method Estimate the learning rate and learning effect - Apply the learning curve to a budgetary problem, including - calculations on steady states Discuss the reservations with the learning curve - Apply expected values and explain the problems and benefits Management Accounting Explain the benefits and dangers inherent in using - spreadsheets in budgeting Covering Environmental Accounting and Quantitative Analysis in Budgeting, the benchmark would go to 83%, therefore exceeding the 80% ACCA benchmark. The F5 Performance Management syllabus is covered by approximately 99% in the course Management Accounting with the remaining percentage covered in Cost Accounting and Economic- Mathematical Methods and Models. 1.5.2. Syllabus profile against CIPFA profile When it comes to the Chartered Institute of Public Finance and Accountancy (CIPFA) profile, significant weaknesses are noted in two subject areas, namely PS Public Sector Financial Reporting (0%) and MO Managing Organisations (57%). 16 Figure 9. Belgrade: CIPFA Benchmark MF Managing Finance 63% MO Managing organizations 57% AA Audit and Assurance 90% PS Public Sector Financial Reporting 0% MA Management Accounting 95% FA Financial Accounting 73% Belgrade CIPFA benchmark One of the reasons that CIPFA results are generally lower than ACCA is that public sector education is considered as an advanced and separate specialization, particularly public sector financial reporting, and is covered either by other concentrations (not subject of this assessment) or at the master’s level education. Some of the weaknesses, relating to MO Managing Organizations, are listed in Table 4. Table 4. MO Managing Organization weaknesses (Belgrade, CIPFA) Topic Coverage Describe procurement, commissioning and typical tendering 0% processes in the public sector The role of proper systems of procurement in minimizing waste, 0% fraud and corruption Procurement or commissioning strategy 0% Contract and service specifications 0% Pre-qualification 0% Invitation to tender 0% Evaluation and refinement of tenders 0% Contract awards. 0% Evaluate the benefits and risks of outsourcing 0% Types of outsourcing 0% Outsourcing management process 0% 17 Topic Coverage Insourcing and bringing back 0% Capacity 0% Key success criteria. 0% Discuss and evaluate typical issues relating to a contract 0% management process in the public sector: Service level agreements 0% Total acquisition cost and total cost of ownership 0% Competitive tendering, procurement and contract processes 0% Contract pricing and cost escalation 0% Contract monitoring systems. 0% Evaluate the following project management approaches: 0% Critical path approach 0% PRINCE2 0% Open workbench 0% Agile management approach 0% Gateway reviews 0% Project management software. 0% 18 2. KRAGUJEVAC UNIVERSITY, FACULTY OF ECONOMICS 2.1. FACULTY The Faculty of Economics in Kragujevac (the Faculty) was established in 1960 as a department of the Faculty of Economics of the University of Belgrade. In 1975, the Faculty grew into an independent institution and became a constituent part of the University of Kragujevac. The Faculty is accredited by the State to offer Bachelor’s, Master’s and PhD degrees. The Faculty is partially financed by the State. Financing from the state budget constitutes 60% of the total revenue while students’ fees constitute 40% of the total revenue. The Faculty has no other incomes contributing to the total (such as projects, non-degree education, consultant work, etc.). Faculty has the autonomy to design and propose the structure of the curriculum, new programs, types of courses, assessment of students and format of exams, and academic partnerships with other institutions. It also has the autonomy to determine the number of students per program and the admission mechanism. However, the Faculty does not have the autonomy to decide on the number of hours per program per year nor the overall number of students admitted in the Faculty. The Serbian Association of Accountants and Auditors (SAAA) offers exemptions from their exams to the Faculty’s graduates. The total number of exemptions range from 4 to 5 out of 9. The Faculty does not have an explicit policy or strategy for managing its interface with the corporate world. However, it has established the Centre for Economic Research with the aim of fostering cooperation with the corporate world. Cooperation agreements have been signed with firms, banks, financial and other institutions, in order to provide Master’s degree students an opportunity to gain knowledge and skills in the real business world. Cooperation agreements with the corporate world should be extended to provide the same opportunities to undergraduate students. Undergraduate students should also be exposed to practical work as this improves the level of their knowledge and skills, as well as helps them establish contacts prior to graduation which could eventually make the employment process easier. This cooperation could be organized as an internship and/or another type of project. The Faculty has no engagement with local audit firms, international audit firms nor with government bodies. There is some engagement with the corporate world, through the agreements mentioned above, and some with professional accounting organizations. 19 The Faculty has introduced an accounting club and other student activities. In the accounting club, students compete in a case competition to help them develop broader critical thinking, judgment and communication skills. 2.2. PROGRAMMES The Faculty implements the Bologna concept, using the European Credit Transfer System (ECTS). The studies are typically organized as 4 years of undergraduate study, 1 year of graduate study, and 3 years of doctoral study. Programs are conducted in the Serbian language. There are two departments that are organized according to area of expertise: • Economics • Business Economics and Management Table 5. Kragujevac: Number of undergraduate student admission and number of graduates per year 3 Department Admissions Graduates Economics 138 192 Business Economics and Management 245 234 There is a total of four concentrations: Accounting and Business Finance; Marketing; Management; and Tourism and Hotel Management. Students opt for the desired concentration at the beginning of the second year or third semester. Table 6. Kragujevac: Accounting and Business Finance concentration courses No of classes per week Year Subject ECST Theory Practice Other 1st Semester Mathematics in Economy 3 2 1 8 1 Foundations of Statistics 3 2 1 8 Foundations of Macroeconomics 3 2 1 7 Foreign language 1 3 2 1 7 3 Data refers to academic 2015/2016 20 No of classes per week Year Subject ECST Theory Practice Other 1st Semester Total 30 2nd Semester Introduction to Management 3 2 1 7 Enterprise Economics 3 2 1 7 Financial Accounting 3 2 1 8 Elective (choose 1 out of 3): 1. National Economy 3 2 1 7 2. Sociology 3 2 1 7 3. Information Technology 3 2 1 7 Seminar thesis 1 0 0 0 1 2nd Semester Total 30 3rd Semester Financial and Actuarial Mathematics 3 2 1 7 Enterprise Organization 3 2 1 8 Management Accounting 3 2 1 7 Elective 1: 1. Information systems 3 2 1 7 2. Economic doctrines 3 2 1 7 3. Data bases 3 2 1 7 3rd Semester Total 29 2 4th Semester Foreign Language 2 3 2 1 7 Theory and Policy of Prices 3 2 1 7 Cost Accounting 3 2 1 7 Elective 2: 1. International Economic Relations 3 2 1 7 2. Operational research 3 2 1 7 Seminar theses 2 1.5 Seminar thesis 3 1.5 4th Semester Total 31 21 No of classes per week Year Subject ECST Theory Practice Other 5th Semester Financial Institutions Accounting 3 2 1 7 Foundations of Stock Exchange Trade 3 2 1 7 Monetary Finances 3 2 1 7 Elective 1: 1. Insurance 3 2 1 7 2. Strategic Management 3 2 1 7 3. Human Resource Management 3 2 1 7 5th Semester Total 28 6th Semester 3 Financial Reporting 3 2 1 7 Auditing 3 2 1 7 Financial Markets 3 2 1 7 Elective 2: 1. Public Finances 3 2 1 7 2. International Trade 3 2 1 7 3. Market communication 3 2 1 7 Seminar thesis 4 2 Seminar thesis 5 2 6th Semester Total 32 7th Semester Budget and Tax Accounting 3 2 1 7 Business Law 3 2 1 7 Banking 3 2 1 7 4 Elective 1: 1. Payment System 3 2 1 7 2. Market Research 3 2 1 7 3. E-Business 3 2 1 7 7th Semester Total 28 22 No of classes per week Year Subject ECST Theory Practice Other 8th Semester Business Finance 3 2 1 7 Accounting Information Systems 3 2 1 7 International Finances 3 2 1 7 Elective 2: 1. Industry Economics 3 2 1 7 2. Economics and organization of 3 2 1 7 Trade Enterprises Graduation thesis 4 8th Semester Total 32 The Faculty should review its allocation of ECTS, as it is unfeasible that the same workload is expected from all the courses, for example sociology, financial accounting and foreign language. It is good to make a distinction between the different categories of courses (at least courses that contribute to general knowledge and courses that contribute to specific knowledge), but some of the categories should be clearly explained in terms of their purpose and contribution to the overall program intended learning outcomes (ILOs). The status of the item described as other classes is not clear, so it would be good to explain the way it contributes to the students’ knowledge. The item seminar thesis is not clear, and it bears the total of 8 ECTS (which is approximately 200 hours of workload). This item needs to be elaborated in a way that it influences the students’ achievements. The graduation thesis has a rather low number of ECTS. Graduation thesis by itself is a way to check ILOs at the program level, which implies that it is a demanding document and requires more effort (especially in specific disciplines as Accounting and Audit). Students’ workload has to be unified at the level of the semester. The official workload for one semester is defined in the amount of 30 ECTS. Most of the semesters have more or less than that. On one hand, this restricts students’ mobility. On the other hand, a workload greater than 30 ECTS can be very demanding for students. This is a sign that ECTS are probably not calculated properly. The Faculty has a good practice of establishing elective courses in order to expand the range of students’ knowledge. However, it would be good practice to increase the number of 23 elective courses that are related to the main stream that students opt for (Accounting and Business Finance). Namely, there is no single elective course that is related to accounting and/or audit. Curriculum review is done on a 4-year basis and is linked to the national accreditation process. Only minor changes are done within the 4-accredited period which ensures consistency of program delivery. For the curriculum review, the Faculty has adopted a Rulebook on self-evaluation. The self- evaluation and quality assessment is conducted by the Commission for Quality Assurance. The members of the Commission are representatives of teaching and non-teaching staff, as well as representatives of students. The inclusion of students is good practice but other different important stakeholders are missing, such as the business community and alumni. Based on the provided program structure, it can be seen that the Faculty does not collaborate with the business community when it comes to program creation and/or delivery. The curriculum does not prescribe the obligation for an internship and/or any kind of project. Internship exists only as a voluntarily activity, and no specific ECTS will be assigned for that activity. The curriculum creation procedure should be improved. The Faculty should include different stakeholders in curriculum creating to improve its content and structure. For example, the Government to define insufficient occupations in the market; faculties (professors and teaching assistants) to give insight as those responsible for implementing curricula; the business community to help define required skills and competences; alumni to provide feedback on previous curricula; and current students to articulate their ongoing needs from the course. The teaching methods mentioned in the courses’ syllabi are very similar. Those encompass ex- cathedra lecturing in majority of cases. There is no evidence that a variety of teaching methods exist, such as seminar, team work, or field work/trips. Also, the presence of guest lecturers is not evident. Depending on the course, different assessment methods are applied (midterm, presentations, homework). However, the final exam usually carries 70% of the final grade in a great number of cases. The midterm carries a small portion of the final grade in the majority of cases. In some cases, the oral exam carries a large portion of the final grade. It is good practice to have an oral exam as different levels of knowledge can be assessed; however, this type of exam risks being subjective. When determining the weight of the oral exam, it is not recommended that it constitute a huge portion of the final assessment. 24 2.3. FACULTY BODY The total teaching/lecturing staff in the Accounting/Audit department comprises of 13 professors. There is 1 full time professor, 3 associate professors, 6 assistant professors and 3 are teaching assistants. Figure 10. Kragujevac: Age profile of the Figure 11. Kragujevac: Gender profile of professors the professors 61-70 2 51-60 2 5 Women 41-50 1 8 Men 31-40 8 under 30 0 Student/staff ratio has decreased in 2014, from 19:1 to 16:1, and remained the same in 2015. The norm for teaching hours for full and associate professors is 6 to 12 hours per week during the academic year, and 10-20 hours per week for assistant professors and teaching assistants. The salaries paid are high enough to keep the current faculty and attract new members. Figure 12. Kragujevac: Composition of the Accounting faculty with regard to their scientific degree Graduate 1 Masters 1 Doctors 11 2.4. STUDENT BODY Looking at the students’ admission in the period of 2013 to 2015, there is a decrease of students’ enrollment both in the percentages of total number of students and in students choosing the 25 accounting and auditing stream. The percentage of student choosing the accounting and auditing stream is an estimate as the exact statistics were not available at the time of writing this report. Figure 13. Kragujevac: Number of students 0,7%, 3 2015 12%, 47 383 1,2%, 6 2014 12%, 59 482 1,7%, 9 2013 15%, 79 519 Drop out rate % choosing A&A Students admission The decrease of 26% in student numbers needs to be carefully considered. This is a significant decrease particularly when compared to the total size of the cohort. The Faculty needs to be very cautious and try to explore the reasons for this decrease that are coming from inside the institution. Only 2 students have taken advantage of the ERASMUS program in 2015, and none in 2013 and 2014. Figure 14. Kragujevac: Gender profile of students 27% Women Men 73% Looking at the student employment statistics, a difference between Kragujevac and other Faculties under this assessment, can be noticed. The business community is not the largest 26 employer of the Faculty’s graduates, as the employment is evenly spread between the financial sector, business community and government bodies. Figure 15. Kragujevac: Internship and employment Non-profit organizations 10 Government bodies 20 Busines community 20 Financial sector 20 International audit firms 10 Local audit firms 10 Employment The Faculty does not use tracking surveys to measure graduate employment rates, average unemployment period after graduation, and average number of years required to complete a degree. The best guess estimate is that around 30-50% of students find a job within 6 months of graduation. The estimate is also that approximately 15-20% of total number of students work in the accounting/audit sphere, and 80-90% graduate from the Accounting and Business Finance module. There is no evidence of a quality assurance system available. In order to track the program and to be able to improve it (together with the Institution), a quality assurance system should be established. 2.5. SYLLABI ANALYSIS 2.5.1. Syllabus profile against ACCA profile The syllabi generally show a strong coverage of technical audit and assurance topics (94% exceeding exemption threshold of 80%). F8 Audit and Assurance is covered by one course, Auditing, in the Faculty’s syllabus. 27 With regards to the content of the courses, the Faculty is ready to submit an exemption claim to ACCA, however some changes will need to be introduced to the assessment methods in order to achieve these exemptions. Figure 16. Kragujevac: ACCA Benchmark F9 Financial Management 73% F8 Audit and Assurance 94% F7 Financial Reporting 24% F5 - Performance Management 34% F3 Financial Accounting 75% F2 Management Accounting 48% F1 Accountant in Business 69% Kragujevac ACCA benchmark At a somewhat deeper level, there are a number of ACCA subjects that the Faculty is showing significant weakness in its syllabi (generally lower than 60% coverage against the ACCA program benchmark). The Faculty has a strong technical coverage in the advanced paper F9 Financial Management, and at the same time weak coverage of the foundations such as F2 Management Accounting, F5 Performance Management and F7 Financial Reporting. It is difficult for the students to cover some of the advanced financial management techniques without mastering techniques covered in performance management and financial reporting. The Faculty should review its syllabus and make necessary changes to evenly progress though all technical topics. F7 Financial Reporting needs to be reviewed by the Faculty as it only covers 24% of the ACCA benchmark. Some of the weaknesses in the syllabus are: • Account for transactions in accordance with IFRS is only covered by 16%. The distribution of percentage per specific topic area is shown in Figure 16. • Preparation of single entity financial statements is covered 15%. The distribution of percentage per specific topic area is shown in Figure 17. 28 Figure 17. Account for transactions in accordance with IFRS weaknesses (Kragujevac, ACCA) Government grants 0% Revenue 0% Reporting financial performance 0% Taxation 0% Provisions and events after the reporting 0% period Leasing 0% Financial instruments 0% Inventory and biological assets 50% Impairment of assets 0% Intangible non-current assets 83% Tangible non-current assets 57% Figure 18. Prepare and present financial statements for single entities and business combinations in accordance with IFRS weaknesses (Kragujevac, ACCA) Preparation of consolidated financial 0% statements including an associate Preparation of single entity financial 67% statements F5 Performance Management needs to be reviewed by the Faculty as it only covers 34% of the ACCA benchmark. F2 Management Accountant has also significant weakness in coverage (48%). Some of the weaknesses in the syllabus of both these subjects are: • Explain and apply cost accounting techniques are covered 50%. The distribution of percentage per specific topic area is shown in Figure 19. 29 Figure 19. F2 and F5 Explain and apply cost accounting techniques weaknesses (Kragujevac, ACCA) F5 Environmental accounting 0% F5 Throughput accounting 25% F5 Life cycle costing 100% F5 Target costing 100% F5 Activity based costing 100% F2 Explaining alternative cost 100% accounting techniques F2 Cost accounting methods 0% F2 Absorption and marginal costing 100% F2 Accounting for material, labour and 65% overheads 2.5.2. Syllabus profile against CIPFA profile When it comes to the CIPFA profile, significant weaknesses are noted in three subject areas, namely PS Public Sector Financial Reporting (0%), MO Managing Organisations (30%), and MA Management Accounting (52%). Figure 20. Kragujevac: CIPFA Benchmark MF Managing Finance 39% MO Managing organizations 30% AA Audit and Assurance 80% PS Public Sector Financial Reporting 0% MA Management Accounting 52% FA Financial Accounting 68% Kragujevac CIPFA benchmark 30 It has to be noted that in response to the growing market for public sector audit and the importance of state-owned enterprises (SOEs) in the country, the university has partnered with an Erasmus+ initiative to introduce a new Master’s level program in Public Sector Accounting. Therefore, the majority of the topics covered by this analysis are included at the master’s level education. Some of the weaknesses, relating to MO Managing Organizations, are in the specific topic areas – see Table 7. Some of the weaknesses, relating to MA Management Accounting, are in the specific topic areas – see Table 8. Table 7. MO Managing Organizations weaknesses (Kragujevac, CIPFA) Coverage Subject ? % Explain and apply the following performance management tools: 3 43% Key Performance Indicators 3 100% Benchmarking 0 0% Explain and interpret the Balanced Scorecard, and the Building Block 0 0% model proposed by Fitzgerald and Moon Explain and apply the Business Excellence Model 0 0% Explain the following efficiency improvement techniques: 11 85% Cost reduction versus cost control 2 100% Explain business process re-engineering 0 0% Indicate the usefulness and problems with different budget types (including fixed, flexible, zero-based, activity- based, incremental, rolling, 0 0% top-down, bottom up, master, functional) Life cycle costing 3 100% Target costing 3 100% Activity Based Management 3 100% Describe the types and uses of the following types of information 2 40% systems: E-business 0 0% E-government 0 0% Management Information Systems (MIS) 0 0% Integrated Financial Information Systems (IFMIS) 1 100% Business Information Systems (BIS). 1 100% 31 Coverage Subject ? % Evaluate the following project management approaches: 0 0% Critical path approach 0 0% PRINCE2 0 0% Open workbench 0 0% Agile management approach 0 0% Gateway reviews 0 0% Project management software. 0 0% Table 8. MA Management Accounting weaknesses (Kragujevac, CIPFA) Coverage Subject ? % Explain the importance of sustainability: 0 0% Definition of sustainability 0 0% Needs and expectations of key stakeholders 0 0% Describe ways in which businesses can operate more efficiently and 0 0% effectively to limit damage to the environment Explain the scope and importance of budgetary control systems: 2 17% Explain why organizations use budgeting 0 0% Describe the planning and control cycle in an organization 0 0% Define and describe the principles of budgetary control in the public sector 0 0% Budget building processes 0 0% Explain top down, bottom up approaches to budgeting 0 0% Budgeting models 0 0% Explain the use of standard costs 1 100% Explain the link between budgeting and financial planning and 1 100% management Prepare and interpret the following budgets for private sector and public 7 44% service organizations: Flexible budget 0 0% 32 Coverage Subject ? % Prepare functional budgets (production, raw materials usage and 0 0% purchases, labor, variable and fixed overheads) Prepare master budgets (statement of profit or loss and statement of 0 0% financial position) Prepare cash budgets 0 0% Prepare profiled budgets 0 0% Prepare capital budgets 7 100% Prepare service budgets 0 0% 33 3. UNIVERSITY OF NOVI SAD, SUBOTICA FACULTY OF ECONOMICS The Subotica Faculty of Economics (the Faculty), founded in 1960, is a public higher education institution that has been a constituent part of the University of Novi Sad. Subotica serves as the main campus for the Faculty, which is about to celebrate its 50th anniversary. Its strategic orientation is to focus equally on developing educational, scientific, research, expertise and consulting activities. The Faculty has been offering education in the field of accounting and auditing since 1978. The Faculty is partially financed by the State which amounts to approximately 47% of total annual average revenue. Students’ fees contribute 53% to the total revenue. The Faculty has no other sources of income contributing to the total (such as projects, non-degree education, consultancy work…) even though their strategic orientation, in addition to educational, scientific and research activities, is to focus on expertise and consulting services as well. The Faculty has the autonomy to design and propose the structure of curriculum, new programs, types of courses, number of hours per program per year, assessment of students and format of exams, and academic partnerships with other institutions. They also have the autonomy to decide on the number of students per program and the admission mechanisms, but the overall number of students admitted must be approved by the Government. The Faculty does not have an explicit policy or strategy for managing its interface with the corporate world. The Faculty has no engagement with local audit firms, business community, professional accounting/audit organizations nor with government bodies. There is some engagement with international audit and assurance firms. 3.1. STUDY PROGRAMS The Faculty implements the Bologna concept, using the European Credit Transfer System (ECTS). The studies are typically organized as 4 years of the undergraduate study, 1 year of the graduate study and 3 years of doctoral study. The 6 departments are organized according to areas of expertise: Agri-Economics and Agri- Business; Business Informatics and Quantitative Methods; European Economics and Business; Finance, Banking, Accounting and Auditing; Management; and Trade, Marketing and Logistics. 34 Table 9. Novi Sad: Number of undergraduate student admission and number of graduates per year 4 Department Admissions Graduates Undergraduate 650 550 Students opt for desired concentration at the beginning of 2nd semester. Table 10. Novi Sad: Accounting and Auditing concentration courses No of classes per week Year Subject ECST Theory Practice 1st Semester Macroeconomics 3 2 8 Microeconomics 3 2 8 Mathematics for Economists 4 3 8 Information Technology 2 3 6 1st Semester Total 30 2nd Semester Accounting 3 2 8 Enterprise Economics 2 2 6 1 Elective 1a: 1. Sociology 2 2 6 2. Philosophy 2 2 6 Financial Markets 2 2 6 Elective 1b: Foreign language 1. English 1 1 4 2. German 1 1 4 3. Russian 1 1 4 4. French 1 1 4 2nd Semester Total 30 4 Data refers to academic 2015/2016 35 No of classes per week Year Subject ECST Theory Practice 3rd Semester Principles of Management 2 2 6 Statistics 3 3 8 Economic Systems and Policies 3 2 8 Finances 3 2 8 3rd Semester Total 30 4th Semester Management Accounting 2 2 6 Banking 2 2 6 Elective 2a: 1. Theory and Policy of Financial 2 2 6 Statements 2 2. Financial Stock Exchanges 2 2 6 Management of Investments 2 2 6 Elective 2b: Foreign language 1. English 1 1 4 2. German 1 1 4 3. Russian 1 1 4 4. French 1 1 4 Elective 2c: Sport 1. Collective sports 1 2 2 2. Individual sports 1 2 2 3. Activities in nature 1 2 2 4th Semester Total 30 5th Semester International Economics 3 2 6 Business Law 3 2 8 3 Corporate Finances 3 2 8 Political Economy 2 2 6 5th Semester Total 28 36 No of classes per week Year Subject ECST Theory Practice 6th Semester Theory of Organization 3 2 8 Auditing 2 2 6 Principles of Marketing 2 2 6 Elective 3/1 1. Financial Organizations Accounting 2 2 6 2. Public Sector Accounting 2 2 6 3. Agro-Organizations Accounting 2 2 6 Elective 3/2: 1. Financial Organizations Accounting 2 2 6 2. Public Sector Accounting 2 2 6 3. Agro-Organizations Accounting 2 2 6 Internship (80 hours) 0 6th Semester Total 32 7th Semester Quantitative Methods in Economy 4 4 8 Analysis of Financial Statements 3 3 8 Consolidated and Special Purpose Financial 2 2 6 Statements 7th Semester Total 22 8th Semester E-business 3 3 6 Auditing of Computer Information Systems 2 2 6 4 Elective 4a: 1. Corporate and Investment Banking 2 2 6 2. Public Sector Economy 2 2 6 Strategic Management 3 3 8 Elective 4b: 1. Tax Systems and Policy 2 2 6 2. Business Ethics 2 2 6 Graduation thesis 6 8th Semester Total 38 37 The Faculty has a good practice of establishing elective courses to expand the range of students’ knowledge. It is also good practice to have a number of elective courses that are related to the main stream that students opt for. A logical order of courses (from fundamentals to specific ones) should be established. Courses that are too challenging and requires students’ prior knowledge should be postponed to later semesters. The Faculty should review its allocation of credits. Courses that contribute to the overall program intended learning outcomes (ILOs) should require more work by students than courses that are academic (general knowledge) and theoretically methodological courses. Some of the semesters have more than 30 ECTS which can be very demanding for the students. This is also a sign that ECTS are probably not calculated properly. The Faculty, compared to other Faculties under this assessment, allocates the largest number of ECTS, total of 6, to the graduation thesis which is a good practice. However, they should consider increasing it even more. Graduation thesis by itself is a way to check ILOs at the program level, which implies that it is a demanding document and requires more effort (especially in specific disciplines as Accounting and Audit). A general agreement is that graduation theses require between 9 to 10 ECTS. Curriculum review is done on a 5-year basis and is linked to the national accreditation process. Only minor changes are done within the 5-year accredited period which ensures consistency of program delivery. Curriculum review is an internal process with only the faculty bodies involved and its objective is to achieve industry and community satisfaction with courses/programs and to achieve adequate learning outcomes. The curriculum creation procedure should be improved. The Faculty should include different stakeholders in curriculum creating to improve its content and structure. For example, the Government to define insufficient occupations in the market; faculties (professors and teaching assistants) to give insight as those responsible for implementing curricula; the business community to help define required skills and competences; alumni to provide feedback on previous curricula; and current students to articulate their ongoing needs from the course. The Faculty does not collaborate with the business community when it comes to program creation and/or delivery. The curriculum prescribes that students need to have an internship in the amount of 80 hours at the end of their third year of studies. However, the internship is a voluntarily activity for which no specific ECTS will be assigned. The Faculty should consider having internship as a mandatory activity with specific ECTS assigned. 38 Depending on the course, different assessment methods are applied (midterm, participation during classes, project assignments). The final exam carries 30% of the total grade and in some cases, that is an oral exam. It is good practice to have an oral exam as different levels of knowledge can be assessed, but this type of exam can be subjective. When determining the weight of the oral exam, it is not recommended that it constitute a big portion of the final assessment. 3.2. FACULTY BODY The total teaching/lecturing staff in the Accounting/Audit department comprises of 10 professors. There are 5 full time professors, 2 assistant professors, and 3 teaching assistants. Figure 21. Novi Sad: Age profile of the Figure 22. Novi Sad: Gender profile of the professors professors 61-70 3 51-60 1 3 Women 41-50 2 Men 31-40 3 7 under 30 1 The student/staff ratio has been stable over the last three years with an average of 43,00. The teaching hour norm for full and assistant professors is 6 hours per week during the academic year and 10 hours per week for teaching assistants. The salaries paid are high enough to keep the current faculty and attract new members. Figure 23. Novi Sad: Composition of the Accounting faculty with regard to their scientific degree Graduate 0 Masters 0 Doctors 10 39 3.3. STUDENT BODY Looking at the students’ admission in the period of 2013 to 2015, there is a decrease of students’ enrollment both in the percentages of total number of students and in the those choosing to study Accounting and Auditing. Figure 24. Novi Sad: Number of students 0,3%, 17 2015 8%, 387 4903 0,3%, 15 2014 8%, 424 5286 0,3%, 19 2013 8%, 478 5668 Drop out rate % choosing A&A Students admission Figure 25. Novi Sad: Gender profile of students 18% Women Men 82% The Faculty does not use tracking surveys to measure graduate employment rates, average unemployment period after graduation and average number of years required to complete a degree. They also have no data on internship or employment statistics. The interface between the Faculty and corporate world for employment and career support is weak as they only occasionally organize career days. 40 There is no evidence of a quality assurance system available. In order to track the program and to be able to improve it (together with the Institution), a quality assurance system should be established. 3.4. SYLLABI ANALYSIS 3.4.1. Syllabus profile against ACCA profile The syllabi generally show a strong coverage of technical audit and assurance topics (96% exceeding exemption threshold of 80%). F8 Audit and Assurance is covered by one course, Auditing, in the Faculty’s syllabus. Regarding the content of the courses, the Faculty is ready to submit an exemption claim to ACCA, however some changes will need to be introduced to the assessment methods in order to achieve these exemptions. Figure 26. Novi Sad: ACCA Benchmark F9 Financial Management 66% F8 Audit and Assurance 96% F7 Financial Reporting 64% F5 - Performance Management 68% F3 Financial Accounting 87% F2 Management Accounting 87% F1 Accountant in Business 78% Novi Sad ACCA benchmark At a somewhat deeper level, the Faculty of Economics Subotica is showing no significant weakness in its syllabi (generally lower than 60% coverage against the ACCA program benchmark). The Faculty should apply for F1 to F3 exemptions as it has content coverage (minor difference in F1 and should be simple to close the gap). F5, F7 and F9 show some gaps ranging in between 12-16%. Some of the specific topics not covered are shown in Figures 27 – 32. 41 Figure 27. F5 Explain and apply cost accounting techniques weaknesses (Novi Sad, ACCA) F5 Environmental accounting 0% F5 Throughput accounting 0% F5 Life cycle costing 100% F5 Target costing 67% F5 Activity based costing 100% Figure 28. F5 Identify and discuss performance management information and measurement systems weaknesses (Novi Sad, ACCA) External considerations and behavioural 0% aspects Performance analysis in not for profit 0% organizations and the public sector Divisional performance and transfer pricing 50% Performance analysis in private sector 67% organizations Management reports 50% Sources of management information 80% Performance management information 67% systems Figure 29. F7 Analyse and interpret Financial Statements weaknesses (Novi Sad, ACCA) Specialised, not for profit and public sector 0% entities Limitations of interpretation techniques 0% Calculation and interpretation of accounting 86% ratios Limitations of financial statements 0% 42 Figure 30. F7 Account for transactions in accordance with IFRS weaknesses (Novi Sad, ACCA) Government grants 0% Revenue 0% Reporting financial performance 100% Taxation 100% Provisions and events after the reporting 9% period Leasing 100% Financial instruments 100% Inventory and biological assets 100% Impairment of assets 50% Intangible non-current assets 100% Tangible non-current assets 100% Figure 31. F9 Explain and apply risk management techniques in business weaknesses (Novi Sad, ACCA) Hedging techniques for interest rate risk 50% Hedging techniques for foreign currency risk 25% Causes of exchange rate differences and 30% interest rate fluctuations The nature and types of risk and 60% approaches to risk management 43 Figure 32. F9 Identify and evaluate alternative sources of business finance weaknesses (Novi Sad, ACCA) Efficient Market Hypothesis and practical 25% considerations in the valuation of shares Capital structure theories and other 25% financial assets Sources of finances and their relative costs 80% Estimating cost of capital 100% Sources of and raising business finance 65% 3.4.2. Syllabus profile against CIPFA profile When it comes to the CIPFA profile, significant weaknesses are noted in three subject areas, namely PS Public Sector Financial Reporting (10%), MO Managing Organisations (48%), and MA Managing Finances (45%). Figure 33. Novi Sad: CIPFA Benchmark MF Managing Finance 45% MO Managing organizations 48% AA Audit and Assurance 85% PS Public Sector Financial Reporting 10% MA Management Accounting 86% FA Financial Accounting 80% Novi Sad CIPFA benchmark Audit and Assurance, Management Accounting and Financial Accounting are either meeting or exceeding the CIPFA benchmark. Some of the weaknesses, relating to MO Managing Organizations, are in the specific topic areas – see Table 11. 44 Table 11. MO Managing Organizations weaknesses (Novi Sad, CIPFA) Coverage Subject ? % Discuss and evaluate typical issues relating to a contract management 0 0% process in the public sector: Service level agreements 0 0% Total acquisition cost and total cost of ownership 0 0% Competitive tendering, procurement and contract processes 0 0% Contract pricing and cost escalation 0 0% Contract monitoring systems. 0 0% Describe the public services procurement and commissioning environment: 0 0% Impact of management concepts 0 0% Outsourcing 0 0% Role of Central Government 0 0% Impact of ethics, sustainability and corporate social responsibility 0 0% Supply chain management. 0 0% Explain and apply the following performance management tools: 4 57% Key Performance Indicators 1 33% Benchmarking 2 100% Explain and interpret the Balanced Scorecard, and the Building Block model 0 0% proposed by Fitzgerald and Moon Explain and apply the Business Excellence Model 1 100% Some of the weaknesses, relating to MF Managing Finances, are in the specific topic areas – see Table 12. Table 12. MF Managing Finances weaknesses (Novi Sad, CIPFA) Coverage Subject ? % Describe the environment in which financial management is practised 0 0% within the public services: The role of the budget in strategic and policy-related aspects of financial 0 0% management 45 Coverage Subject ? % International Architecture of Public Financial Management and its key role 0 0% players Public stakeholders and sources of funding 0 0% Managing donor funding support to achieve results: the Paris Declaration, 0 0% the Accra agenda for Action and similar initiatives Debt relief – principles and conditionality 0 0% The importance and impact of collaboration amongst stakeholders 0 0% Impact of the economic environment, government policies, financial 0 0% markets and other external factors on financial strategy Alignment of financial strategy with overall organisational strategy 0 0% Public Internal Financial Control (PIFC) Framework. 0 0% Discuss the responsibilities of the accountant in relation to financial and 1 11% treasury management: Role and responsibilities of the Chief Financial Officer 0 0% Financial planning 1 100% Budget setting, monitoring and control 0 0% Risk management 0 0% Internal control systems maintenance 0 0% Asset stewardship 0 0% Treasury and debt management 0 0% Effective anti-fraud controls 0 0% Effective internal audit. 0 0% Describe the process of appraising the viability of capital projects: 15 83% Revenue v capital: budgets and expenditure 2 100% Identify and explain important non-financial factors in investment decisions 0 0% Cash flows, benefits, risk identification and risk attitude 1 50% Funding options and financial structure 12 100% Capital expenditure and the use of investment and financing plans 0 46 4. UNIVERSITY OF NIŠ, FACULTY OF ECONOMICS The Faculty of Economics (the Faculty), founded in 1960, is a public higher education institution (HEI) that has been a constituent part of the University of Niš. Until 1973, the Faculty was a part of the Faculty of Law and Economics, after which it became an independent University. The Faculty is partially financed from the state budget which covers approximately 54% of the total annual average revenue. Students’ fees contribute 46% to the total revenue. The Faculty has no other sources of income contributing to the total (such as projects, non-degree education, consultancy work, etc.). The Faculty has the autonomy to design and propose the structure of curriculum, new programs, types of courses, assessment of students and format of exams, but the ultimate approval is with the University of Niš. The Faculty has the autonomy to decide on the overall number of the students admitted, the number of students per program but the number must be in accordance with the National Accreditation Policy. The Serbian Association of Accountants and Auditors (SAAA) offers exemptions from their exams to the Faculty’s’ graduates. The total number of exemptions is 6: Financial Accounting; Management Accounting; Cost Accounting; Audit; Financial Statements Theory and Analysis; and Business Finances. The Faculty does not have an explicit policy or strategy for managing its interface with the corporate world. However, there are activities that are conducted jointly with the corporate world. Within its Centre for Economic Research (CER), there have been several projects implemented together with business firms. CER’s employees also provide various services tailored to meet the needs of businesses and other organizations. This activity does not contribute to the funding of the Faculty. The second interface is through lectures, where representatives of business firms are occasionally invited as lecturers in specially organized events or regular classes. The Faculty has some engagement with local audit firms but none with international ones. The Faculty has active engagement, with regards to the interface described above, with the business community, professional accounting associations, and government bodies. 47 4.1. PROGRAMMES The Faculty implements the Bologna concept, using the European Credit Transfer System (ECTS). The Studies are typically organized as 4 years of the undergraduate study, 1 year of the graduate study and 3 years of doctoral study. The 4 departments are organized according to area of expertise: General Economics; Accounting, Audit and Financial management; Finance, Banking and Insurance; and Business Management. Table 13. Niš: Number of undergraduate student admission and number of graduates per year Department Admissions Graduates General Economics 70 23 Accounting, Auditing and Financial Management 100 56 Finance, Banking and Insurance 250 126 Business Management 280 135 The Business Management department has the largest number of student enrolling per year, approximately 40% of total students. The Accounting, Auditing and Financial Management department has approximately 14% of total students. The percentage of graduates compared to the number of admissions per year is 56% which is the highest percentage of graduates when compared to other departments. The pass rate and the number of graduates seems to be rather low and this can imply some issues with the program, such as: the program is challenging, the entrance quality is low, the constructive alignment is not achieved, etc. Table 14. Niš: Accounting, Audit and Financial Management concentration courses Year Subject No of classes per week ECST 1st Semester Financial Accounting 6 7 Foundation of Economics 6 7 1 Essentials of Management 6 7 Enterprise Resource Management 6 7 Seminar work 2 1st Semester Total 30 48 Year Subject No of classes per week ECST 2nd Semester Mathematics 6 7 Sociology 6 7 Statistics 6 7 Elective: 1. Economic Geography 6 7 2. Industrial Management 6 7 3. Agrarian Management 6 7 Seminar work 2 2nd Semester Total 30 3rd Semester Public Finances 6 7 Financial Institution Accounting 6 7 Budget Accounting 6 7 Elective: 1. Financial and Actuarial Mathematics 6 7 2. Human Resource Management 6 7 3. Philosophy of Economics 6 7 Foreign language I (English, German, Russian, 2 2 2 French) 3rd Semester Total 30 4th Semester Economic Policy 6 7 Banking Management 6 7 Management Accounting 6 7 Business Law 6 7 Foreign language II (English, German, Russian, 2 2 French) 4th Semester Total 30 49 Year Subject No of classes per week ECST 5th Semester Marketing 6 7 Business Finances 6 7 Informatics 6 7 Elective: 1. Trade Management 6 7 2. Public Sector Economics 6 7 Foreign language III (English, German, Russian, 2 2 French) 5th Semester Total 30 3 6th Semester Cost Accounting 6 7 Financial Statements Theory and Analysis 6 7 Theory and Policy of Prices 6 7 Elective: 1. Economic Functions 6 7 2. Economic Doctrines 6 7 Foreign language IV (English, German, Russian, 2 2 French) 6th Semester Total 30 7th Semester Special Purpose Financial Statements 6 7 Operational Research 6 7 Elective: 1. International Economics 6 7 4 2. Business Performance Management 6 7 3. Managing Risk and Insurance 6 7 4. E-Business 6 7 Elective (same as above) 6 7 Professional practice 2 7th Semester Total 30 50 Year Subject No of classes per week ECST 8th Semester Financial Markets 6 7 Auditing 6 7 Elective: 1. Economics of Transition 6 7 2. Strategic Management 6 7 3. Strategic Logistic Management 6 7 4. Capital Economics and Development 6 7 Financing Elective (same as above) 6 7 Graduation theses 2 8th Semester Total 30 A logical order of courses (from fundamentals to specific ones) should be established. Courses that are too challenging and that requires students’ prior knowledge should be postponed to later semesters. Also, both vertical and horizontal alignment should be achieved, meaning that courses should logically contribute to the overall program intended learning outcomes (ILOs) as well as contribute to each other in the same semester and year with no significant overlapping. The curriculum includes an internship which is a good means of connection between students and the corporate world. However, only 2 ECTS are allocated to the internship, which means that it requires approximately 50 hours of the students’ workload (approximately one working week). The number of ETCS should be increased as the contribution of knowledge gained within only one week is questionable. The graduation thesis is assigned with 2 ECTS (the same as internship) and the aim of the thesis should be to check if the ILOs are achieved at the program level and to what extent. Students only have approximately 50 hours to prepare the thesis including doing research, writing the thesis and defending it, and it is questionable whether the amount of time allocated to it is sufficient. The Faculty should review its allocation of credits, as it is unfeasible that the same workload is expected from all of the courses, for example sociology and financial accounting. It is good to make a distinction between different categories of courses (at least courses that contribute to 51 general knowledge and courses that contribute to specific one), but some of the categories should be clearly explained in terms of their purpose and contribution to the overall program ILOs. Curriculum review is done on a 5-year basis and is linked to national accreditation process. Only minor changes are done within the 5-year accredited period which ensures consistency of a program delivery. For the curriculum review, the Faculty uses results from the following surveys: • Students satisfactory survey – distributed at the end of each semester • Graduate satisfactory survey – implemented since 2015 • Employee satisfaction survey – distributed to employers of Faculty’s graduates and implemented since 2015. It is evident that the Faculty collaborates with the business community when it comes to program creation and/or delivery as internship is a mandatory activity. The Faculty also uses results from the employee satisfaction survey where the employers’ recommendations are embedded into the curricula to ensure that the education is aligned with actual market needs. Teaching methods mentioned in the courses’ syllabi are very similar. Those encompass ex- cathedra lecturing in majority of cases. There is no evidence that a variety of teaching methods exists, such as seminar, team work or field work/field trips. Also, the presence of guest lecturers is not evident even though the interface with the business community states that they are invited occasionally to make lectures. Depending on the course, different assessment methods are applied (midterm, participation during classes, project assignments). However, the final exam usually carries a high percentage of the final grade in a number of cases. The midterm carries a rather small percentage of the final grade in the majority of cases. In some cases, the oral exam carries a high percentage of the final grade. It is good practice to have an oral exam as different levels of knowledge can be assessed, but this type of an exam can be subjective. The evidence of a quality assurance system is partially available. To track the program and to be able to improve it (together with the Institution), a quality assurance system should be established. The use of surveys is one of the number of possible quality instruments, so dispersion is recommended. 52 4.2. FACULTY BODY There is a total of 12 professors teaching Accounting/Financial Reporting/Audit courses. Figure 34. Niš: Age profile of the Figure 35. Niš: Gender profile of the professors professors 61-70 3 51-60 2 4 Women 41-50 Men 31-40 8 3 under 30 0 Student/staff ratio shows a decreasing trend in the period of three years: 6.85 in 2013 and 6.25 in 2015. Salaries are adequate to retain current faculty, but it is hard to attract new members. Out of the total 12 teaching professors, 5 are full professors, 4 are associate professors and 3 are assistant professors. Their teaching norm is 6 hours per week. All 12 of them are PhD and full- time faculty employees. The Faculty faces the issue of inadequate staffing particularly relating to hiring assistants. Since 2014, by the decision of the Government, recruitment in the public sector is prohibited. The Faculty has found a temporary solution, by engaging doctoral student to assist professors and work with students in practical classes. 4.3. STUDENT BODY Looking at students’ admissions in the period of 2013-2015, there is a decrease in the number of students, and at the same time a slight percentage increase of them choosing the Accounting, Audit and Financial Management concentration (2013: 15.45%, 2014: 15.77%, 2015: 15.92%). The drop-out rate is low, decreasing in 2014 and then remaining stable. The decrease of 18% in student numbers is not insignificant, particularly when considering the total size of the cohort. The Faculty needs to be cautious and explore the reasons for this decrease (perhaps the program is too challenging or not adapted to market needs). 53 Figure 36. Niš: Number of students 2%, 10 2015 16%, 75 471 2%, 11 2014 16%, 82 520 3%, 18 2013 15%, 89 576 Drop out rate % choosing A&A Students admission Figure 37. Niš: Gender profile of students 23% Women Men 77% Only two students, in the period from 2013 to 2015, have taken advantage of the ERASMUS program. English language proficiency of the students has been rated 3 out of 5, which is low despite the fact that the students have language classes in the second and third year of their studies. The business community is the largest internship provider as well as employer of the Faculty’s graduates. The Faculty estimates that the percentage of employment on graduation and 6 months after graduation is approximately 40%. This is a best guess estimate as the Faculty does not use tracking surveys to measure the graduate employment rate, average unemployment period after graduation and average number of years required to complete a degree. 54 Figure 38. Niš: Internship and employment Government bodies 15 15 Busines community 60 50 Financial sector 20 30 International audit firms 5 5 Local audit firms 2 2 Employment Internship 4.4. SYLLABI ANALYSIS 4.4.1. Syllabus profile against ACCA profile The syllabi generally show a relatively strong coverage of technical audit and assurance topics (84% exceeding exemption threshold of 80%), with more variable coverage in other areas. Figure 39. Niš: ACCA Benchmark F9 Financial Management 60% F8 Audit and Assurance 84% F7 Financial Reporting 73% F5 - Performance Management 51% F3 Financial Accounting 71% F2 Management Accounting 74% F1 Accountant in Business 73% Niš ACCA benchmark 55 At a somewhat deeper level, there is only one ACCA subject where the Faculty is showing significant weakness in its syllabi (generally lower than 60% coverage against the ACCA program benchmark), F5 Performance Management. The specific topic areas where the weakness is, are shown in Table 15. Table 15. F5 Performance Management weaknesses (Niš, ACCA) Topic Coverage Target costing 0% Life cycle costing 0% Environmental costing 0% Material mix and yield variances 0% Sales mix and quantity variances 0% Planning and operational variances 0% Performance analysis and behavioural aspects 25% Divisional performance and transfer pricing 25% Limiting factors 40% Dealing with risk and uncertainty in decision making 0% Other subjects are close to the 80% exemption threshold, with difference varying from 6% to 20% difference. For most of these subjects, the difference is small and minor changes and/or introduction of new topics is necessary to reach the exemption threshold. 4.4.2. Syllabus profile against CIPFA profile The Faculty of Economics of Nis, does cover public sector financial reporting that is missing entirely from the other Faculties’ curriculum covered by this assessment. When it comes to the CIPFA profile, significant weaknesses are noted in three subjects, namely PS Public Sector Financial Reporting, MO Managing Organisations, and MF Managing Finances. Even though the results have shown some weaknesses, the Faculty of Economics Nis is still showing best CIPFA results out of all the Faculties under this assessment. One of the reasons that CIPFA results are generally lower than ACCA results is that public sector education is considered as an advanced and separate specialization and is covered at the master’s level education. 56 Figure 40. Niš: CIPFA Benchmark MF Managing Finance 46% MO Managing organizations 39% AA Audit and Assurance 76% PS Public Sector Financial Reporting 24% MA Management Accounting 69% FA Financial Accounting 60% Niš CIPFA benchmark Table 16. Topic areas with weaknesses (Belgrade, CIPFA) Topic Coverage Ethical requirements 0% Objectives of external financial reporting within the public sector - Fraudulent behaviour 0% Performance measurement in public sector organisations - Costing methods 0% Procurement, tendering, outsourcing in public sector - 57 5. PROFESSIONAL ACCOUNTANCY ORGANIZATIONS (PAOS) 5.1. CHAMBER OF AUTHORIZED AUDITORS (CHAMBER) Under the Law on Auditing (2013), individuals wishing to perform statutory audit work are required to be licensed by the Ministry of Finance. To acquire a license, a person must meet all of the following conditions5: 1) obtain a university degree; 2) pass (or receive exemptions from) the professional certification exams administered by the Chamber; 3) complete at least three years of practical working experience in statutory audits, out of which at least two years must be under the supervision of a licensed authorized auditor; and 4) not have been convicted of a criminal act that would make them unworthy to be an auditor. Membership in the Chamber is mandatory for all licensed auditors and audit firms. The Law on Auditing assigns duties to the Chamber, including: • maintaining and administering the program of exams for acquiring the title of certified auditor, which ensure new auditors have achieved the minimum competence requirements; • issuing certificates for the title certified auditor (but not audit licenses); • enacting the conditions and procedures for exemptions from exams; • maintaining the mandatory CPD program and organizing CPD training for licensed auditors; • maintaining and administering the quality assurance systems that serve as the auditing profession’s principal means of demonstrating to the public and to regulators that auditors are performing at a level that meets the established auditing standards and ethical rules; • maintaining the registers of audit firms and auditors; 5 The Law on Auditing stipulates requirements for recognized auditors from other countries to pass tax and law exams; it also details requirements for audit firms. 58 • determining fees, including registration and membership fees, exam fees, and special contributions for performing quality control of audit firms, independent auditors and licensed certified auditors. The Chamber has 9 full-time employees. The Secretary General is appointed by the Council and manages and organizes the day-to-day business of the Chamber. Other employees include 3 employees responsible for professional education and examinations, 3 staff members involved in quality assurance activities, and 2 administrative staff. Relevant data related to the Chamber is provided in Table 17. Table 17. Chamber of Authorized Auditors Year established 2006 (by law) IFAC Membership status Associate Member since November 2014 Number of members (official title is ~ 300 “licensed authorized auditor”) Number of audit firms ~ 65 Number of active students ~ 350 (with ~ 100 applying to write one or more exams each year) Average pass rate for exams 58% Average time to complete all exams 4.25 years Average number qualifying as certified ~ 40 (not licensed) auditors per year 5.1.1. Chamber Certification The list of required exams to become a Certified Auditor is shown in Table 18. Table 18. Certified Auditor exams 6 Level 1 Level 2 1 Theory and principles of accounting 1 Law for certified auditors 2 Financial reporting framework and IFRS 2 Tax systems 3 Financial analysis 3 Business information systems ROSC, supra note 1 at para 77. 6 59 Level 1 Level 2 4 Managerial Accounting 4 Microeconomics, general and financial economics 5 Risk management and internal control 5 Financial mathematics and statistics 6 Audit theory 6 Financial management of undertakings 7 Audit methodology There is no prescribed order of taking the exams and no prerequisites for exams. The Chamber offers optional exam preparation seminars/consultations, but does not provide a formal education program. Although some exam preparation sessions have been offered by the Chamber, candidates have not seen the value in these sessions, and they have not been well attended. Most candidates writing the Chamber’s exams come from a few major universities and prepare for the exams independently. An insufficient preparation material for candidates limits the attractiveness of the program and reduces the success rate. Currently, exemptions from some exams are available for university students, but there is no standard map or accreditation guide – exemptions are determined on a case-by-case basis. There is an opportunity to develop accreditation standards to increase the efficiency of the exemption process, and to either develop a professional program or partner with one or more of the top universities to provide additional courses to bridge the gap between university and professional level education. SAAA courses are not recognized for Chamber exam exemptions. Each written exam requires a pass grade of 60%. There is no restriction on the number of attempts. Although there is no published exemption guide, exemptions from some exams are available based on university courses completed. The Chamber’s Commission for Examination, Training and Continuous Professional Development provides overall governance and supervision of the exam process, with full support of the Chamber’s education department. Examiners and markers for each exam are elected from the Commission. A maximum of nine papers at the Fundamentals level of the ACCA Qualification can be awarded for exemption. In the Audit and Assurance paper, the Chamber has gone beyond the exemption threshold, however other papers are below 80%. Tax and Law papers were not subject of review. When compared to the professional level, the Chamber has approximately 90% coverage of Advanced Audit and Assurance, with lower scores in other subjects. When comparing 60 professional subject, the 80% exemption threshold does not apply as ACCA does not award exemptions at the professional level. Figure 41. Chamber of Authorized Auditors: ACCA Benchmark at professional level F9 Financial Management 60.1% F8 Audit and Assurance 97.7% F7 Financial Reporting 73.6% F5 - Performance Management 63.9% F3 Financial Accounting 87.2% F2 Management Accounting 73.8% F1 Accountant in Business 51.5% Chamber of Authorized Auditors ACCA benchmark Figure 42. Chamber of Authorized Auditors: ACCA Benchmark at professional subject P7 Advanced Audit and Assurance 89% P5 Advanced Performance… 36% P4 Advanced Financial Management 35% P3 Business Analysis 36% P2 Corporate Reporting 61% P1 Governance, Risk and Ethics 34% Chamber of Authorized Auditors ACCA benchmark 5.1.2. Compliance with IESs A key element of achieving/maintaining full membership status with the International Federation of Accountants (IFAC) is compliance with the IFAC Statements of Membership Obligations. In education, this includes compliance with the International Education 61 Standards (IESs), maintained by the International Accounting Education Standards Board (IAESB). The IESs have been recently revised, with the updated standards coming into effect in July 2014 (IES 1), July 2015 (IESs 2 to 7) and July 2016 (IES 8). The most fundamental change in the IESs is the addition of “learning outcomes” which are statements that reflect the skills and capabilities expected of a professional at a specific level of proficiency, to be able to perform their role competently. This shift towards a “learning outcomes” approach is an effective means of helping candidates develop competence, by focusing more closely on the specific capabilities, values, ethics and attitudes that professionals need to demonstrate in their roles, and the proficiency levels required to do so (classified as foundation, intermediate or advanced). The IAESB emphasizes that a learning outcomes approach integrates learning outcomes, education program design, assessment activities, and governance in a process of continuous improvement.7 This not only enhances the development of competence in candidates, but also provides better structure and a higher degree of accountability for the education program providers by more closely focusing on what the program needs to achieve. The end result is a better overall program, which enhances both the quality of new professionals and the reputation of the program provider. Chamber exams are currently focused on meeting regulatory requirements. The syllabi largely cover the required topic areas of IES 2 IPD: Technical Competence, but do not follow a learning outcome approach and do not specify proficiency levels for different competencies. Exams would benefit from being re-designed to ensure that the Professional Skills, Values, Ethics, and Attitudes required by IESs 3 and 4 are being covered, and that the requirements of IES 8 Professional Competence for Engagement Partners are also being met. A competency- based approach that includes more integration and judgment will best facilitate this re- design. This will also help ensure that the exams more directly consider and address the needs of local employers and businesses. With respect to IES 6, the Chamber’s exams are developed and administered with strong consideration for security and reliability in grading, for example, using two independent graders. More attention needs to be given to ensuring validity and reliability with respect to the overall competence of passing candidates. More formal review processes and statistical analysis from a psychometric perspective would assist in this regard. As noted previously, more emphasis needs to be placed on including ethics and professionalism in the syllabi of universities and PAOs alike. There is also more that can be done to ensure that practical experience requirements develop 7 See International Accounting Education Standards Board (IAESB), The Value of Implementing a Learning Outcomes Approach (January 2016) online: IAESB . 62 and assess the required competencies needed by professional auditors, such as monitoring the suitability of supervisors and monitoring progression of competence. Enhanced mentoring and more structured competence assessment during practical experience can help alleviate resource constraints on the exams side. Currently, the universities take part in the SAAA conferences, and the University of Belgrade hosts SAAA exams, but there is only an informal faculty-level relationship with the Chamber. Much more can and should be done to collaborate between both the Chamber and SAAA, and the universities. 5.2. SERBIAN ASSOCIATION OF ACCOUNTANTS AND AUDITORS (SAAA) Membership in the SAAA is voluntary, as there are no laws or regulations in Serbia with respect to any requirements or protection of professional titles in the area of corporate accounting. SAAA is registered with state authorities for education and continuing professional development (CPD) activities, and for providing professional qualifications. Although it has no statutory authority, SAAA is one of the oldest professional bodies in the region and offers a multi-level accounting education program (Accountant, Certified Accountant - CA, and Certified Public Accountant - CPA). Members are expected to comply with CPD requirements and the IESBA Code of Ethics for Professional Accountants. SAAA has standing arrangements with several foreign PAOs, generally using comparative program benchmarks, and requiring verification of ethics and CPD requirements. Normally, additional exams in local tax and law are also required. Full professional recognition has been achieved with the Federation of Bosnia & Herzegovina, Montenegro, and the Republic of Srpska. Partial recognition exists with some of the PAOs in EU and Commonwealth countries and some global PAOs (such as ACCA, ICAEW, ICAS, and AAT). SAAA also has a mutual recognition agreement with CIPFA that allows full CIPFA membership for SAAA’s experienced CA and CPA graduates. SAAA has standardized accreditation agreements with many state colleges, universities, and business/accounting schools, as well as with the majority of private education institutions. Based on its Joint Education Committee assessment of academic curriculum and teaching quality, the SAAA awards certain exam exemptions. Relevant data related to the Association is provided in Table 19. There are no legislated educational requirements for accountants. Despite this lack of legal status, the SAAA offers an education program based heavily on the ACCA program. The 63 program includes three levels: Accountant, Certified Accountant, and Certified Public Accountant. Table 19. Serbian Association of Accountants and Auditors Year established 1955 IFAC Membership status Member since 1997 Accountancy Europe (previously Member since 2013 Federation of European Accountants) Number of members > 4,000 Number of firms > 600 Number of students > 3,000 total Accountant program ~ 850 CA program ~ 2,350 CPA program ~ 80 Average time to complete each Accountant program: ~ 3 years program CA program: ~ 3 years CPA program: ~ 3 years (requires completing CA program as pre-requisite) Average age at completion Accountant program: 24 CA program: 28 CPA program: 36 Table 20. SAAA syllabus 8 Accountant Certified Accountant Certified Public Accountant 1 Recording Financial 1 Preparing Financial 1 Audit and Assurance Transactions Statements Services 2 Information for 2 Financial Information for 2 Performance Management Control Management management 3 Preparing Financial 3 Managing People 3 Strategic Business Statements Planning and Development 4 Cost Accounting 4 Information Systems 4 Complex Corporate Reporting Ibid at para 81. 8 64 Accountant Certified Accountant Certified Public Accountant 5 Audit procedures 5 Corporate and Business 5 Strategic Financial Law Management 6 Tax System 6 Tax System 7 Managing People and 7 Financial Management and Systems Control 8 Financial Statements 8 Financial Reporting Compilation 9 Planning, Control and 9 Audit and Internal Review Performance Management Note: Qualification as a Certified Accountant is a necessary condition for acquiring the Certified Public Accountant certification. The minimum academic education requirements for an individual seeking to begin professional studies with the SAAA differ by the qualification they are seeking: • Accountant (Accounting Technician) – Secondary school • Certified Accountant (CA) – High (college) school (2 or 3 years), University • Certified Public Accountant (CPA) – Economic University and CA professional qualification The SAAA program includes education materials, written exams, and a requirement for three years of practical experience. Exams are held twice per year, and there is a 10-year limit on completing the programs. Exemptions are available based on university courses taken. For example, a University of Belgrade graduate who completed the accounting and audit module at the Faculty of Economics will receive more exemptions than the graduates from other universities. The Education Board has an overall responsibility for the exams process, supported by the External Certification Quality Control Committee, which performs a periodic review of the examination process and papers. There are 14 Examination Committees, responsible for the preparation of exam papers. The process is monitored by the Joint Supervisory Commission, consisting of members from the academia. Practical experience is required to be logged based on area and level of competence achieved, and monitored by the employer and a mentor/supervisor who must be a member in good standing of SAAA or another member body of IFAC. SAAA verifies results on a sample basis. 65 The SAAA syllabus is based on the ACCA program syllabus. SAAA also asserts that its exam processes are designed to be competency-based with reference to IES 6, although direct evidence of this has not been evaluated. Periodic reviews and comparisons with ACCA are performed and the syllabus is updated (by the Education Committee) if needed. The most recent updates were made in February 2015 to the tax area, so they are broadly unlikely to be fully compliant with the most current IESs and the newly specified learning outcomes and proficiency levels. 5.3. CONTINUING PROFESSIONAL DEVELOPMENT (CPD) REQUIREMENTS 5.3.1. CPD programs are required and administered by both the Chamber and the SAAA. 5.3.2. Chamber The Law on Auditing contains requirements for licensed authorized auditors to complete a minimum of 120 hours of CPD in the three-year period from date of issuance/renewal of license. Of the 120 hours, 60 hours must be completed through attendance at CPD organized by the Chamber. The remainder can be attained through self-study or attendance at events offered through other relevant providers (e.g. universities, other professional bodies, employers training programs). The Chamber specifies that at least 10% of a member’s CPD hours must relate to each of the following content areas: • International Standards of Auditing an International Standards on Quality Controls (ISA and ISQC); • International Financial Reporting Standards (IFRS); • International Financial Reporting Standards for Small and Medium Entities (IFRS for SME); • IESBA Code of Ethics for Professional Accountants; • Laws and legal regulations/requirements. The main source of achieving CPD for most members is the annual 3-day Chamber Conference held in October. 66 The recent Centre for Financial Reporting Reform (CFRR) Accountancy Education Benchmarking Study has performed the analysis of the content of CPD courses and the percentage of the topics covered is shown in Figure 43.9 Figure 43. Chamber of Authorized Auditors: CPD Courses Auditing standards and 20% updates Professional Ethics 15% 65% Taxes and regulatory updates 5.3.3. SAAA SAAA members are required to fulfill CPD requirements to maintain their voluntary membership in the Association. The program uses a combined input- and output-based approach. Under the input-based element, all SAAA members are required to complete a minimum 120 relevant CPD hours in each rolling three-year period, of which 60 hours must be verifiable. At least 20 CPD hours must be completed in each year. The output-based element requires members to demonstrate the development and maintenance of appropriate professional competence by showing verified evidence. SAAA offers in excess of 200 hours of CPD sessions per year. Periodic reviews of CPD compliance are performed. Members found in breach are informed of their status and provided guidance on potential remedial alternatives. Further non- compliance falls under the jurisdiction of Disciplinary Committee. The recent CFRR Accountancy Education Benchmarking Study has performed the analysis of the content of CPD courses and the percentage of the topics covered is shown in Figure 44.10 9 http://siteresources.worldbank.org/EXTCENFINREPREF/Resources/4152117-1427109489814/9765106- 1487166467531/benchmarking_study.pdf 10 http://siteresources.worldbank.org/EXTCENFINREPREF/Resources/4152117-1427109489814/9765106- 1487166467531/benchmarking_study.pdf 67 Figure 44. SAAA: CPD Courses 9% Financial reporting and accounting 15% Taxes and regulatory updates 42% Professional Ethics and other topics 34% Auditing standards and updates 5.3.4. Challenges and Opportunities in CPD CPD requirements are outlined in IES 7. IFAC encourages an output-based approach, focusing on the actual demonstration of professional competence, ahead of an input-based approach that specifies amounts (such as hours) of learning activity required for professional accountants to develop and maintain professional competence. SAAA’s CPD program has been designed as a combination approach, whereas the Chamber model is input-based but also prescribes 50% of a member’s verifiable hours to fall within 5 specific technical, ethical, and legal domains (at least 10% in each domain). Both CPD program model designs comply with IES 7 assuming they are operating as intended. There is, however, an opportunity to move both programs towards a more output-based approach. In addition, the current CPD system for auditors is very comprehensive, but also very resource intensive. There appear to be opportunities for enhanced efficiency, recognizing that the program is a key means of improving member competence and providing value. 68 APPENDIX 1. CONSTRUCTIVE ALIGNMENT The three elements of constructive alignment (outcome, instruction, and assessment) are inextricably linked, as illustrated below: Figure 45. Constructive alignment influencing elements INTENDED LEARNING OUTCOMES (ILOS) The process of aligning courses begins with establishing ILOs. It should be clear for each program and for each course the knowledge, skills, and competences students will acquire. PROGRAM ILOS ILOs are written statements about student performance, describing what students need to know, understand, and connect to by the end of a period of study. ILOs should be structured to derive the knowledge, skills, and attitudes separately. Knowledge is the lowest and the easiest level to define. Knowledge is the basis for skills; which together are the basis for attitudes. ILOs must be described at the level of the program and for each individual course. Annex A 2 outlines the steps to be taken in developing ILOs. Program ILOs should align with and 69 contribute to the mission statement but be developed independently of the goals of the program. They should be based on occupational standards, qualification standards, educational strategy, and the needs of the labor market in the country. ILOs should be aligned horizontally and vertically (significant overlap is not recommended). Horizontal alignment implies that courses are sequenced in one year in a way that gradually constructs knowledge and skills. The courses cannot be too demanding since the candidates usually have no previous knowledge. Vertical alignment implies that all courses during the program should contribute to the overall learning outcome. A logical order needs to be assured with no repetition of courses or content. The program ILOs then guide development of the curriculum and allow ILOs for individual courses/modules to be defined and described. COURSE ILOS Course ILOs should be defined with special emphasis on vertical alignment, that is knowledge, skills, and attitudes/values and the corresponding knowledge dimensions and domains. They should then be documented, ideally as part of the syllabi. Course ILOs need to be linked to the program, and to the mission statement. ILOs need to be real and achievable. Each course should have seven to ten ILOs – this is very important and should be linked with the number of ECTS that are assigned to the course; this also serves as the benchmark for all courses within the program. Each course ILO should be defined using one or two active verbs from the list of active verbs in Bloom’s Taxonomy (see Annex 2). If there are too many verbs the point of the statement can be lost. The verbs must be assessable and measurable– to be able to assess and measure the extent to which the learning outcomes are achieved. The sentence describing the course ILO should be simple and understandable – it is important to keep in mind the purpose and focus of the ILO. Careful choice of verbs is important, as some verbs are relatively less informative about the depth of understanding and/or knowledge. Some verbs should be avoided since they cannot be measured, or are not concise or explicit. These include: believe, realize, comprehend, listen, to be aware of, to be familiar with, to be interested in, understand, to be knowledgeable about. Comparatives or superlatives should also to be avoided as they are not precise and measurement is not possible (for example better/best or more/most). Course ILOs should be aligned to ensure that the lowest level describes the academic knowledge with active verbs appropriate to that level. The next stage relates to skills (professional knowledge) so the relevant dimension of the appropriate domain should be 70 used. At the most demanding level, attitudes should be described (and therefore the appropriate dimension used). Not all courses will need three levels, depending on content, but it is not recommended to have only one level. Formulating SMART outcomes could be useful: • S = specific (not fuzzy) • M = measurable (must be able to assess if reached) • A = attainable (what is possible given infrastructure and logistics) • R = relevant (linked to the aim of the program) • T= timely (feasible within the module/semester) TEACHING AND LEARNING ACTIVITIES The second element of constructive alignment is teaching and learning activities. The work mapping a curriculum, writing ILOs, and planning what students need to master and how, all comes to life in the classroom – in lectures, tutorials, and students’ own engagement with faculty staff, each other, and with the knowledge. Designing engaging, relevant, and stimulating teaching and learning activities that excite and motivate lecturers, students, and tutors is thus a crucial part of successful aligned and constructive curricula. In constructive alignment the teaching system components, especially teaching methods and assessment, are properly defined and aligned to the learning activities assumed in the intended outcomes. This alignment ensures steady progress towards explicit and measurable outcomes. The focus then becomes facilitating the students’ achievement of the learning outcomes, providing a learning environment in which students are able to show what they know and can do. All assessment tasks, and learning and teaching experiences (and therefore, content and methods), must be linked to the desired learning outcomes. In practical terms, it makes sense to decide on the intended outcomes first, and then to organize course content, teaching methods, learning experiences, and activities to achieve these. A key part of assessments will be based on progress towards study outcomes. Teaching methods and techniques for each course should support the ILOs, guided by the active verbs in the ILOs. This means that when defining ILOs, the faculty needs to identify appropriate teaching methods for each one. Implementing a range of innovative teaching methods would bring benefits to the Faculties, potentially reducing the workload of teaching staff through, for example, guest lecturing, seminars, team work, and video conferences, and offers students exciting new ways to engage with learning. 71 The following table shows some of instructional strategies that are aligned with knowledge/skill dimensions. Table 21. Connection of knowledge dimension, Bloom’s taxonomy and instructional strategy Level Level ILO Dimension Instructional strategy (cognitive) (affective) ILO 1 Knowledge Remembering Receiving Lecture, Reading, Audio/video Demonstration ILO 2 Knowledge / Understanding Responding Discussion, Reflection, Role skill Applying playing, Case studies ILO 3 Knowledge / Analysis Valuing Practice by doing (coaching skill required), Simulations ILO 4 Skill Evaluating Organizing Use in real situations ILO 5 Skill Evaluating Internalizing Normally developed on own Creating (learning by doing), with mentoring that can facilitate the process Not all the methods (and the list above is not a finite one) can be applied to all courses. The content of the course, the course type, the size of the study group, and the year of study must all be considered in determining the application of the appropriate method. As for learning, there is a tremendous set of active learning methods available. They are usually divided into inside the classroom and outside the classroom format. In-class methods can include tutorial groups, quizzes, role-play, mind-maps, and case study analysis. Outside methods are independent projects, group discussion, debates, and teamwork. Massive online open courses can be a valuable source of knowledge for students, providing a great opportunity for international exposure with relatively low cost. Improving connections with the business community could include inviting experienced professionals/specialists in a certain field to teach some classes as a professor of practice. If legal obstacles make this impractical, such individuals could still be encouraged to informally participate in teaching. A set of criteria could be developed to help select appropriate practitioners for specific disciplines. 72 ASSESSMENT The third element of constructive alignment is the assessment of students. Following the previous logic, the ILOs should be supported by appropriate teaching methods and adequate assessment tools. Formative assessment takes place at points of time during a program to check the level of (specific) ILO achievement. Formative assessment builds a cumulative record of student achievement. It usually takes place during day-to-day learning and involves ongoing, (in)formal observations throughout the course. It helps monitor students’ ongoing progress and provides immediate and meaningful feedback. For faculty, it assists in modifying or extending programs or adapting their learning and teaching methods. Some possible tools are one-minute papers, multiple-choice questions (MCQ), quizzes, and presentations during the course. Summative assessment takes place at the end of the semester in order to test the level of ILOs achievement at the course level. Examples include open book exam, MCQ, open questions, short answer questions, final exam, true-false questions, cumulative project, research project, and short essay. It is recommended not to exceed 20% of the total content being assessed using true-false questions and MCQ. Only low levels of knowledge can be assessed using true-false questions or simple MCQ. If MCQ’s are created as examples of real situations, then they can be applied to higher levels (such as skills testing and assessment). The oral exam in many courses is good practice that can be used to assess different levels of knowledge. However, the weight given to the oral exam should be decided with care since this type of assessment can be subjective. To minimize the impact of such subjectivity influencing the final result oral exams should not make up a large portion of the assessment. For some courses attendance is graded. This is not recommended as attendance does not equal learning and it is hard to objectively assess the level of participation and attention to the course, for example during lectures. The best assessment strategy is directly connected to the ILOs of the course and to its specific level. Table 22. Connection of knowledge dimension, Bloom’s taxonomy and assessment method ILO Dimension Level Verb Assessment ILO 1 Knowledge Remembering Describe, Use text readings to identify Recognize basic categories needed 73 ILO Dimension Level Verb Assessment ILO 2 Knowledge Understanding Give example, Use financial statement as Illustrate examples ILO 3 Knowledge / Applying Prepare, Show Use projection, forecast skill ILO 4 Skill Analyzing Compare, Classify Use financial reports, Evaluating Plan, Organize diagrams ILO 5 Skill Evaluating Design, Combine Use financial data for giving Creating Evaluate, Judge the recommendations, valuing and conclusions ILO 6 Attitudes Valuing Criticize, Justify, Provide critique and Argue justification for specific financial position/elements 74 APPENDIX 2. DEVELOPING ILOS The ILOs should describe the additional knowledge, skills and attitudes that students will have after they complete the course: 1. Knowledge - what students should know and understand by the time the course is completed. 2. Skills - what students should be able to do by the time the course is completed and at what level. 3. Attitudes/values11 - what the students’ opinions will be about the subject matter of the course by the time it is completed (according to what values and with what attitudes they will think and act). ILOs are not a list of topics to be covered during the course, a description of the professor’s intention or an explanation of the courses’ importance or point. The following steps should be followed when defining the ILOs. • Step 1 Start by describing the learning opportunity, so the sentence starts with “By the time the student finishes the course, . . .” or “After completion of the course/program…” • Step 2 Next the subject is mentioned, and it is always recommended that a form of address in the first person be used. Therefore, the ILOs start will be “After completion of the course, you/students/candidate will be able to…” • Step 3 In addition, strong active verb(s) should be used in order to express the knowledge, skill, or attitude. After this step, the ILO will be “After completion of the course, you will be able to identify…” • Step 4 Next, the object has to be mentioned. The improved ILO would be “After completion of the course, you will be able to identify specific set of characteristics…” • Step 5 Then it is time to introduce the specific context (framework or circumstances) if there is one. In this case, the ILO will be “After completion of the course, you will be able to identify specific set of characteristics within the specific context…” 11 In local language the term “kompetencije” is widely used. Without prior knowledge, misunderstanding is common. If it would be translated literally to English language, it would be “competencies” which is the term that describes an important skill that is needed to do a job; translated like this it can be very close to skills. The appropriate translation to English language will be attitudes or values. 75 • Step 6 In this last step, one should add the specific set of values and/or norms and/or attitudes with direct influence to the identified context. The final ILO will be “After completion of the course, you will be able to identify specific set of characteristics within the specific context with social sensitivity that is required.” Below is an illustration of gradual creation of a course ILO incorporating practical example that could be applied to a particular course, for example financial reporting. Figure 46. Example of the gradual creation of a financial reporting course ILO After completion of the course, you will be able to… After completion of the course, you will be able to read and to identify… After completion of the course, you will be able to read financial statements and to identify specific set of characteristics… After completion of the course, you will be able to read financial statements and to identify specific set of characteristics within the specific context of non profit organisation... After completion of the course, you will be able to read financial statements and to identify specific set of characteristics within the specific context of non profit organisation with awareness of social sensitivity that is required. 76 Instructors usually use well-defined learning taxonomies12 to create ILOs for a course. The most popular taxonomy of educational outcomes is Bloom's Taxonomy of Intended Learning Outcomes, which contains verbs from three domains: cognitive, affective, and psychomotor. The cognitive and affective domains are the ones mainly used in social sciences. The cognitive domain contains six levels/dimensions of taxonomy (listed from lowest to highest): remembering, understanding, applying, analyzing, evaluating, and creating. All of them can be applied in the undergraduate program that is being reviewed (and in all courses). Each of these dimensions contain active verbs that describe the desired level (for example, “remembering”: know, identify, relate, list, define, name, recognize, etc.). The list of verbs that are being used is huge, and it depends on the language used. In the following table, dimensions will be listed in an appropriate order, along with some of the more pertinent verbs for economics and business. Table 23. Bloom's Taxonomy of Intended Learning Outcome: Cognitive Domain Dimensions Bloom’s definition Verbs used Remembering Exhibit memory of previously learned define, find, label, list, material by recalling facts, terms, basic match, name, relate, select concepts, and answers. Understanding Demonstrate understanding of facts and classify, compare, explain, ideas by organizing, comparing, illustrate, relate, translating, interpreting, giving summarize descriptions, and stating main ideas. Applying Solve problems to new situations by apply, construct, identify, applying acquired knowledge, facts, plan, select, utilize techniques and rules in a different way. Analyzing Examine and break information into parts analyze, assume, classify, by identifying motives or causes. Make compare, distinguish, inferences and find evidence to support examine, relationships, generalizations. survey 12 Among learning taxonomies, the most well-known is Bloom’s Taxonomy of Objectives defined in 1956 and revised in 2000. http://tll.mit.edu/help/two-examples-taxonomies-educational-objectives 77 Dimensions Bloom’s definition Verbs used Evaluating Present and defend opinions by making assess, choose, compare, judgments about information, validity of decide, determine, ideas, or quality of work based on a set of estimate, evaluate, criteria. explain, opinionate, select Creating Compile information together in a change, combine, create, different way by combining elements in a develop, elaborate, new pattern or proposing alternative estimate, maximize, solutions. minimize, test It can be noticed that some verbs can be repeated in neighboring dimensions, but the appropriate definition is up to the instructor’s sense and course specific content. The affective domain is the other one that needs to be defined in order to support the cognitive one. This domain has five levels/dimensions (listed from lowest to highest): receiving, responding, valuing, organization, and characterization. The affective domain is very often unfairly neglected, but in modern times it is an inevitable part of education. In the following table, the dimensions of affective domain will be listed, with some verbs that can be used (list of verbs is not exhaustive). Table 24. Bloom's Taxonomy of Intended Learning Outcome: Affective Domain Dimension Bloom’s definition Verbs used Receive Ability to learn from others. accept, attend, ask, feel, focus, experience, perceive Respond Ability to participate responsibly, respectfully contribute, choose, help and actively as appropriate to the context. team, question, provide, seek to clarify, react Value Ability to associate personal and collective argue, assess, challenge, values with contextual experience and express confront, criticize, justify, value judgments. respect Organize Ability to structure, prioritize and reconcile arrange, contrast, personal and others’ value systems. collaborate, defend, reconcile Characterize Ability to articulate one’s own values and act, conclude, change, belief systems and operate consistently within internalize, resolve, revise, them. 78