Training Assessment Project (TAP) Moldova December 2018 Table of Contents List of abbreviations  4 Acknowledgments  5 Executive Summary  6 Introduction  10 Conceptual framework  13 Theory of change  13 Defining key players  14 Defining key instruments  15 Defining skills  15 Chapter 1: Findings from the mapping exercise of short-term training institutions in Moldova  17 Short-term training providers in numbers  18 Short-term training providers’ profile  21 Chapter 2: Characteristics of short-term training institutions in Moldova based on TAP results  26 Students  28 Teachers/Instructors  30 Facilities and equipment  32 Funding  32 Chapter 3: Actions of short-term training institutions in Moldova  34 Strategic actions of short-term training institutions  34 Action 1: To set a strategic direction  34 Action 2: To develop a demand-driven approach to training  36 Action 3: To establish a sustained relationship with authorities  37 Action 4: To ensure institutional financial viability  39 Action 5: To fulfill national quality standards  41 Action 6: To enable students to pursue education and training opportunities  43 Action 7: To create a teaching/studying experience conducive to learning  44 Action 8: To prepare students for the world of work  48 2  Action 9: To gather and publicize data for informed decision making  50 Perceptions of the performance of short-term training institutions  51 Summary of key findings and policy implications  54 Strengths of short-term training institutions in Moldova  54 Areas of improvement for short-term training institutions in Moldova  55 References  58 Annexes  60 Annex 1: Survey methodology and survey instruments  60 Survey methodology  60 1. Training provider survey  60 1.1. Sample frame of training providers  61 1.2. Quality assurance procedures for the sample frame  61 1.3. Sampling strategy  61 2. Focus groups with students, graduates, and employers  62 2.1. Selection of students, graduates, and employers  62 Survey and focus group instruments  63 1. Training Provider Questionnaire (CAPI)  63 2. Training questionnaire and interviewer guide  64 3. Quality control procedures  64 4. Focus group guides, questionnaire, and discussion agenda  65 Project team for survey  65 Annex 2: Summary of results from focus group discussions with students, graduates, and employers  66 1. Students  67 1.1. Selection of training provider  67 1.2. Plans following graduation  68 1.3. Assistance from training providers to find employment  69 1.4. Experience during training  69 1.5. Students’ perception  70 2. Graduates  70 2.1. Selection of training provider  70 2.2. Employment status and process  72 2.3. Assistance from training providers to find employment  73  3 2.4. Experience during training  74 2.5 Relevance of training to current employment  74 2.6. Graduates’ perception  75 3. Employers  76 3.1. Recruitment process  76 3.2. Perception of graduates’ skills  77 3.3. Partnerships with training institutions  77 3.4. Employers’ perception  79 Figures Figure 1. Student enrollment by level of education and school year  13 Figure 2. Distribution by development region, N = 272  21 Figure 3. Length of operation, N = 272  21 Figure 4. Share of enrolled students by type of short-term training provider, N = 247  22 Figure 5. Core business, N = 272  23 Figure 6. Enrollment by professions in demand on the market (weighted average as share of top three courses), N = 272  24 Figure 7. Enrollment by professions in demand on the market (by cumulative number of enrolled students), N = 272  24 Figure 8. Number of courses, N = 272  24 Figure 9. Duration of top three courses (weighted average), N = 272  24 Figure 10. Overseeing ministry of short-term training providers, N = 272  27 Figure 11. Institution type, N = 50  29 Figure 12. Institution category, N = 50  29 Figure 13. Age distribution of short-term training students, N = 30  30 Figure 14. Average educational attainment of students at the time of enrollment (2017), N = 50  31 Figure 15. Reasons for dropping out in 2017, as a percentage relative to the total dropout rate, N = 22  31 Figure 16. Average industry experience of instructors/teachers, N = 50  32 Figure 17. Average teaching experience of instructors/teachers, N = 50  32 Figure 18. Average education credentials of instructors/teachers, N = 50 (weighted) 33 Figure 19. Average number of computers, N = 47  34 4  Figure 21. Expense composition of training institutions, N = 47  35 Figure 22. Does the institution have a management committee or governance board? (N = 50)  37 Figure 23. Mechanisms to determine the skills to be taught in training programs, N = 50 (Multiple responses)  38 Figure 24. Frequency of training institutions’ participation in government events, N = 50  40 Figure 25. Main communication channels between training providers and government officials, N = 50 (Multiple responses)  41 Figure 26. Personnel preparing the operational budget, N = 50  42 Figure 27. Flexibility in managing financial resources, N = 50  43 Figure 28. Use of competency standards for the delivery of curricula, N = 50 (Multiple responses)  44 Figure 29. Flexible arrangements provided by training institutions, N = 50 (Multiple responses)  46 Figure 30. Autonomy to introduce/close programs, N = 50  47 Figure 31. Percentage of institutions that introduced/closed programs, N = 50  48 Figure 32. Institutions that request feedback from students, N = 50  49 Figure 33. Institutions that request feedback from graduates, N = 50  49 Figure 34. Types of professional development opportunities for instructors, N = 31 (Multiple responses)  50 Figure 35. Frequency of data collection, N = 50  52 Tables Table 1. Mapping methodology of short-term training institutions in Moldova  17 Table 2. Number of short-term training institutions by size and types of training provided  21 Table 3. Domains in which trainings are provided, N = 272  24 Table 4. Number of short-term training institutions by size and types of training provided  27  5 List of abbreviations ANACEC National Agency for Quality Assurance in Education and Research ANOFM National Employment Agency CAPI Computer-Assisted Personal Interview CASEL The Collaborative for Academic, Social, and Emotional Learning CIVIS Centrul de Analiză şi Investigaţii Sociologice, Politologice şi Psihologice EU European Union EMIS Education Management Information System GDP Gross Domestic Product ICT Information and Communication Technology ISIC International Standard Industrial Classification KWPF Korea-World Bank Partnership Facility MoECR Ministry of Education, Culture, and Research MoHLSP Ministry of Health, Labor, and Social Protection NACE Statistical Classification of Economic Activities in Moldova NGO Nongovernmental Organization NQF National Qualifications Framework NBS National Bureau of Statistics PAPI Paper-Assisted Personal Interview PISA Program for International Student Assessment QC Quality Control R&D Research and Development SABER-WfD Systems Approach for Better Education Results Workforce Development TAP Training Assessment Project TAPM Training Assessment Project in Moldova VET Vocational Education and Training USAID U.S. Agency for International Development 6 List of abbreviations Acknowledgments This report was prepared by Lucia Casap (Operations Officer), Koji Miyamoto (Senior Economist), and Minju Choi (Consultant). The firm survey data were collected by CIVIS Moldova under the guidance of Ruslan Sintov. Margo Hoftijzer, Alexandra Valerio, Keiko Inoue, Viviana Venegas Roseth, and Ana María Santillana provided valuable inputs, intel- lectual support, and practical advice throughout the development of this study. The team acknowledges the leadership of Harry Patrinos (Practice Manager), Cristian Aedo (Practice Manager), Omar Arias (Practice Manager), and Anna Akhalkatsi (Country Manager-Moldova). The team thanks Denis Nikolaev, Elizabeth Ninan, and Abla Safir who kindly peer reviewed the study. The team is grateful to the Government of Moldova for sharing feedback and views on findings at an early stage of the study. In particular, the team wishes to thank the Ministry of Education, Culture, and Research and the National Agency for Quality Assurance in Education and Research for insights and guidance throughout the process. The team is indebted to Cristina Denita-Botolin, Head of the Life, Long, Learning Department for continuous collaboration during the process. The team would like to also thank all partic- ipants at the interviews and focus groups for their valuable suggestions and comments. The team is grateful for the administrative support provided by Sujani Eli in Washington, D.C., and Tamara Ursu in Moldova. The study was made possible by the financial support provided by the Korea-World Bank Partnership Facility (KWPF). Acknowledgments 7 Executive Summary Moldova, like many other countries in the region, is facing rapid economic and demo- graphic transitions. Employers report that inadequate levels of workforce skills ham- per their productivity. Education and training policies need to better cater to skills re- quired by employers but undersupplied by workers (Rutkowski, Levin, and Bargu 2017). Recognizing the importance of workforce development to improve the country’s socio- economic prospects, the Government of Moldova approved the educational develop- ment strategies for 2013–2020. The main objectives are to improve the quality of educa- tion and training system and better match its outcome with the labor market needs. Some progress has been made in recent years to better understand the presence and roles of vocational education and training (VET) programs in Moldova. However, there is still much to be learned about the types of training providers available and their effec- tiveness in fostering human capital. In particular, information on short-term training pro- viders, including their characteristics, practices, and outcomes, are not available. Given this lack of information and the government’s strong interest in better understanding specifically the provision of short-term training providers, the World Bank launched the Training Assessment Project (TAP) to identify the landscape and characteristics of these institutions and assess the extent to which they follow practices that are conducive to improved service delivery. In collaboration with Centrul de Analiză şi Investigaţii Sociologice, Politologice şi Psihologice (CIVIS) and with the Ministry of Education, Culture, and Research (MoECR) of Moldova, the World Bank conducted both a mapping exercise and an assessment of short-term training providers in 2017–2018. The mapping exercise identified 272 short- term training providers, including VET institutions and universities. Most of them operate in the urban areas and are small- or medium-size private institutions. While there are few- er public institutions, they are much larger in terms of student size. Driving and language training courses were the most popular among enrolled students in 2016. The main results from TAP, based on a sample of 50 institutions, can be summarized for each of the following 9 areas of institutional actions that drive the quality of service delivery: Action 1: To set a strategic direction Action 2: To develop a demand-driven approach to training Action 3: To establish a sustained relationship with authorities Action 4: To ensure institutional financial viability Action 5: To fulfill national quality standards Action 6: To enable students to pursue education and training opportunities 8 Executive Summary Action 7: To create teaching/studying experience conducive to learning Action 8: To prepare students for the world of work Action 9: To gather and publicize data for informed decision making Action 1: Limited capacity to set demand-sensitive training strategies, monitor progress, and improve accountability. While most short-term training institutions in Moldova have formally developed mission statements, a few of them consider the per- spectives of industry associations, employers, and the community. Although a majority of training providers collect data on current students and instructors, few collect infor- mation on graduates that could help them monitor progress toward the strategic goals. Moreover, many institutions lack internal accountability mechanisms such as manage- ment committees or governance boards. Institutions that have management committees or governance boards are less likely to include members from industries. Action 2: Limited capacity to develop and adapt training programs that reflect skill needs in the labor market. Only about half of short-term training institutions give enough attention to industry demands or skills needs assessments when determining skills developed through the program. Students of training providers in Moldova are more likely to choose the sector of specialization based on their personal interest rather than the information on job demand. Only 30 percent of the training providers have strat- egies to involve employers or industry groups in institutional decision making. Action3: Limited level of relationship with authorities among private training institutions. Private short-term training institutions in Moldova tend to show limited level of engagement in policy dialogues and interactions with government authorities. While about one-third of training institutions do not undergo mandatory inspections or finan- cial auditing, the rest generally follow mechanisms for regulatory compliance and service improvements. Action 4: Some mechanisms to ensure financial viability. Public as well as private training institutions in Moldova have authority over the collection of financial resources and the use and management of financial sources. At the same time, not many institutions seek contributions from government authorities, private donors, or employers. Almost all participating training institutions indicated that they have an operational budget, most of which was prepared by the institutions themselves. These institutions tend to have a space to maneuver the management of the funds. Action 5: Limited engagement, especially among private institutions, to ensure national quality standards. While all public short-term training institutions in Moldova conduct reviews to ensure compliance with national quality standards, many private or not-for-profit institutions do not follow this process. The short-term training programs tend to use curricula and occupational standards based on competencies determined mostly by the institutions themselves or by a national framework. Only about half of the training institutions design, develop, or adapt their curriculum to fit industry association and employer standards and curricula requirements. A significant challenge for many training providers is the lack of national occupational standards that would guide and Executive Summary 9 simplify their work. As the National Quality Assurance Agency has recently started its activity, only one-fifth of the short-term training institutions have been accredited, and a quarter are currently in the process of being accredited. Action 6: Some mechanisms to enable students to pursue education and train- ing opportunities. While less than half of the short-term training institutions in Moldova evaluate student proficiency before admission to the program, they tend to offer flexibil- ity for students to complete the training, mainly through flexible schedule arrangements. Upon completion of the programs, a majority of the training institutions provide certifi- cation and standardized exams to students, but they do not necessarily offer nationally recognized certificates. Action 7: Some mechanisms to create teaching and studying experience con- ducive to learning. Most institutions have autonomy to introduce and close training programs. Over the past three years, more than half of the sampled institutions intro- duced new short-term training programs and a few closed short-term training programs. Three-quarters of training institutions have designed or adapted curricula in the past three years, based on various considerations. Most training institutions request feedback from graduates on the quality of the training services they provide but less from students. Almost three-quarters of the training institutions reported that they evaluate the perfor- mance of instructors, but close to half do not reward or address issues based on perfor- mance. In 2017, short-term training institutions provided opportunities for instructors to participate in professional development activities such as conferences or courses. Action 8: Limited opportunities for students to prepare for the world of work. Many short-term training institutions in Moldova offer limited opportunities for students to make smooth transitions to work. While more than one-third of training providers re- ceive equipment and supplies from firms, very few receive opportunities for research and development (R&D) projects. Training providers also offer limited support for students to find internships and apprenticeship opportunities. Moreover, there is insufficient re- source dedicated to offering career counseling services. Action 9: Limited capacity to gather and publicize data for informed decision making. Most short-term training providers collect a wide variety of data on institutional and program performance, with varying degrees of frequency. However, fewer providers collect information on employment data. The information is used mostly internally, in communication with staff and students. Almost all training institutions use the collected data for internal discussions on institutional or program performance. Moving forward, the results from TAP in Moldova point to three areas in which train- ing providers can continue their efforts to effectively prepare the workforce that meet the skills needs. First, short-term training providers would benefit from improving their capacity to make informed decisions by regularly collecting data on graduate’s training outcomes, including employment and earnings, to improve the programs. Second, short- term private and not-for-profit training providers would benefit from regularly engag- ing with government authorities by maintaining communication channels and following 10 Executive Summary mechanisms to comply with national quality standards. These providers can also improve quality standards by conducting external quality reviews to ensure the quality of train- ing is meeting national standards. Lastly, short-term training providers would be able to better prepare students to the world of work if they have stronger relationships with em- ployers and/or industry associations and provide more information and career advice for students so that they can find internship/apprenticeship opportunities. The government, in turn, is expected to strengthen policies and regulatory mechanisms designed to moti- vate and facilitate the training providers to improve their service delivery. Executive Summary 11 Introduction Moldova is a small lower-middle-income country in Eastern Europe, with a population of approximately 3.55 million. Although the poorest country in Europe, Moldova’s economy has been expanding, with an annual gross domestic product (GDP) growth of 4.1 per- cent in 2016 (World Development Indicators). The poverty rate in Moldova has rapidly declined, with a poverty headcount ratio at national poverty lines decreasing from 22.0 percent of the population in 2011 to 9.6 percent in 2015. The economy has seen signifi- cant structural changes since the 2000s. While the share of the agricultural sector in total GDP accounted for 29 percent in 2000, it declined to 14 percent in 2016. In contrast, the services sector dominates the current economy, increasing its share from 49 percent of GDP in 2000 to over 70 percent in 2016. Despite such recent economic development, the country recognizes that its growth driven by consumption and poverty reduction by remittances and pensions are not sustainable for the future (World Bank 2017b). Moldova is also experiencing demographic changes that affect labor market and pro- ductivity. For over a decade, the annual population growth has been close to −0.1 per- cent. This gradual decline in the population can be attributed to large-scale emigration and low fertility rates (World Bank 2018a). In 2016, over 880,000 Moldovans, or almost one-quarter of the total population, were living outside the country (European Training Foundation 2017), and fertility rates remained low at 1.2 births per woman. With an aging population and anticipated reduction in the share of the working-age population, it is thus necessary for Moldova to equip the existing workforce with strong skills and knowledge. Considering such changes in the economic and demographic landscape, Moldova prioritizes the enhancement of the quality and relevance of education and training for job-relevant skills (World Bank 2018a). The compulsory education system in Moldova includes one year of preschool education followed by nine years of basic education (four years of primary and five years of lower secondary education). After these nine years, students can choose to follow three different education tracks: (a) lyceum/upper second- ary education (three-year general academic stream), (b) colleges (vocational stream with access to higher education), or (c) professional schools (World Bank 2018c). Over the past decade, access to preprimary education has steadily improved, while the enrollment in primary and lower secondary education has shown a reversing trend, largely due to a declining school-age population (Figure 1). Gross enrollment ratio for preprimary education also increased from 70 percent in 2006/2007 school year to 86 percent in 2016/2017 (National Bureau of Statistics [NBS]). In contrast, during the same period, gross enrollment ratio for primary education decreased from 94 percent to 91 percent and from 90.5 percent to 87 percent for the secondary education level. 12 Introduction FIGURE 1. STUDENT ENROLLMENT BY LEVEL OF EDUCATION AND SCHOOL YEAR 280000 260000 240000 220000 200000 180000 160000 140000 120000 100000 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 Preprimary Primary Lower secondary The quality of education in Moldova has been improving, yet its performance is still under the average of other neighboring countries’ outcomes. Moldova participated in the Program for International Student Assessment (PISA) in 2009+ and 2015. Comparing these two years, the PISA scores in all tested subjects (mathematics, reading, and science) of 15-year-olds have improved by the equivalent of one school year (World Bank 2018b). Moldova scored higher than some of the countries with higher-income levels, including Georgia, Kosovo, and the former Yugoslav Republic of Macedonia. However, Moldova performed lower than some of its neighboring countries such as Romania, Bulgaria, and Russian Federation. Despite improvement in student performance, such progress is not uniform across the country, with students from lower-income family backgrounds and those in rural ar- eas facing bigger learning challenges. For instance, results from PISA 2015 show that the gap between students from the lowest 20 percent social, economic, and cultural status and those from the highest 20 percent is equivalent to almost three years of schooling (World Bank 2018b). The gap between students in the rural and urban areas is equal to nearly 1.5 years of schooling. Moreover, declining fertility rates and emigration continue to reduce the number of students in the rural regions. Given these economic and demographic changes affecting the education system, there is an ever increasing perception of skills shortages in the country. Employers in Moldova report that inadequate workforce skills hamper the performance of their firms. In particular, the three biggest skills-related obstacles to firm performance are insuffi- cient occupation-specific technical skills, unsatisfactory work ethic, and poor motivation (Rutkowski, Levin, and Bargu 2017). The skills gaps are more pronounced in the case of middle-skilled (manual and nonmanual) employees than for higher-skilled employees. Middle-skilled workers lack motivation, willingness to learn, analytical and problem-solv- ing skills, and computer skills, while high-skilled workers are more hindered by insuffi- cient foreign language skills. Introduction 13 Employers are also more concerned about the skills of young workers (under the age of 30) than those of older employees. Feedback from employers suggests that university graduates lack the skill sets that the labor market needs (World Bank 2018d). While uni- versity graduates do have better employment and earning prospects than those with low- er educational degrees, nearly half of the Moldovan firms reported problems finding staff with the right skills. Even among employed university graduates, 43 percent reported a mismatch between their education and jobs, including over-education (28 percent) and employment in a field unrelated to their study (9 percent). Similarly, although vocational education and training (VET) programs improve graduates’ job prospects, 40 percent to 50 percent of the graduates choose not to look for work or have been discouraged to participate in the national labor market (World Bank 2018c). According to the Ministry of Health, Labor, and Social Protection (MoHLSP), blue-col- lar jobs (or jobs that require physical work) are the highest in demand across most sectors in Moldova. In particular, seamstresses are in high demand in light industry, drivers in transportation, field workers in agriculture, and production line workers in the food pro- cessing industry (World Bank 2017a). The information and communication technology (ICT) sector is short of low technical-level white-collar jobs, including testers, program- mers, and web designers. Both the ICT and commerce sectors are short of managerial positions. Employee turnover is reported to be high in Moldova, particularly in large firms and blue-collar jobs, with the exception of the ICT sector. A few identified reasons include low salaries and poor investment in blue-collar workforce development by employers. Current understanding of the workforce points to the importance of equipping young people with skills demanded by the market. Education and training policies need to bet- ter focus on developing skills that are needed by employers but undersupplied by work- ers (Rutkowski, Levin, and Bargu 2017). These include not only occupation-specific tech- nical and cognitive skills but also socio-behavioral or socioemotional skills. Recognizing the importance of workforce development to improve the country’s socioeconomic pros- pects, the Government of Moldova approved the VET Development Strategy for 2013– 2020. One of its objectives is to increase the quality of VET programs and better match them with current labor market needs (World Bank 2013). In this context, training pro- viders that serve the working-age population play a crucial role. First, they can provide remediation for those who did not acquire sufficient skills through the formal education system. Second, training providers can offer skills upgrading for those seeking a change of course in their careers. The quality assurance of training providers, therefore, is import- ant for the assessment and improvement of VET programs. Although much progress has been made in recent years to better understand and assess the quality of VET programs, there is still much to be learned about the different types of training providers in Moldova. In particular, there is almost no information col- lected on short-term training providers in the country. Basic data on short-term training providers are not available, and understanding of the characteristics, practices, and be- haviors of short-term training providers is absent. Given this lack of information and the 14 Introduction government’s strong interest in better un- derstanding specifically the provision of SABER-WFD AND TAP short-term training in Moldova, the World TAP uses the conceptual framework Bank launched the Training Assessment of the World Bank’s SABER-WfD. The Project (TAP) initiative to identify the cur- framework identifies the policies and rent conditions and common practices practices that national workforce development systems should have under which short-term training providers in place to move toward the desired operate and those conditions and practic- dynamic alignment between skills es that contribute to good performance. demand and supply. In other words, In collaboration with Centrul de Analiză it captures the policy intent in these şi Investigaţii Sociologice, Politologice systems. TAP covers the same policy areas identified as crucial for success şi Psihologice (CIVIS) Moldova and by SABER-WfD and translates them with Ministry of Education, Culture and into institutional actions that train- Research (MoECR) of Moldova, the World ing institutions should undertake to Bank conducted a mapping and quality impart the skills that the labor market assessment of short-term training provid- needs. In this sense, TAP is part of ers in 2017–2018. SABER-WfD, but with a focus on policy implementation. More information on In 2013, the World Bank conducted SABER-WfD can be found here. the policy intent part of the research in Moldova, using the Systems Approach for Better Education Results-Workforce Development (SABER-WfD) analytical framework and standard procedures to collect and analyze data (link to report). To further the policy dialogue and assess institutional actions, TAP gathers information on the characteristics and institutional values of training providers, as they are important determinants of the extent to which these institutions are able or willing to take action. TAP also collects data on the outcomes of training in- stitutions as an indicator of their performance. To get an accurate picture of the charac- teristics, actions, values, and outcomes of training providers, TAP uses two types of data collection instruments—a questionnaire for training institutions and complementary focus group guides to gather qualitative data on the same topic matters from students, gradu- ates, and employers. The following sections will present findings from the data collected on short-term training providers in Moldova in 2017–2018. The report intends to inform both short-term training providers and the state entities responsible for the governance of these institutions. Introduction 15 Conceptual framework Theory of change Labor productivity in Moldova, measured as GDP per person employed, continues to be much lower than the average of countries in Central Europe and the Baltics. While Moldova’s labor productivity was US$12,990 (constant 2011 purchasing power parity per U.S. dollar) in 2017, the average labor productivity in Central Europe and the Baltics re- gion was US$56,221, more than four times higher than it is in Moldova. Improving labor productivity is essential if the country is to increase overall economic growth and improve workers’ earning opportunities and potential. Labor force flexibility and productivity are reliant on the level of competence of a country’s workforce. Establishment of a skilled labor force makes an important contribution to economic growth and poverty reduction, and adequate assessment of whether education and training are effectively enabling skills development is critical. A prerequisite for skills development programs to achieve their goals, not only for individuals but also for the country as a whole, is assessing the links between the la- bor market, the business community, and the training providers. Education must equip young people with relevant market skills that blend knowledge and technical knowhow with soft and hard skills so that they can be productive and employable members of society (World Bank 2016). Employers look for workers who not only have the technical expertise, but also the communication and collaborative skills that allow them to work in teams and critical thinking skills for analyzing problems. To enhance the youth’s employ- ability and reduce structural unemployment, as well as to support productivity growth and the competitiveness of Moldovan firms, education and training programs should aim to properly address skills gaps of the workforce. To improve the quality of education and training services, a deeper understanding of the characteristics, practices, and behaviors of training providers is necessary. In Moldova, there is currently no publicly available tool that studies these factors in a systematic and standardized manner. TAP was therefore launched to address an important knowledge gap in skills development of the working-age population. This study will particularly focus on short-term training providers in Moldova, who offer courses/programs with a duration of 18 months or shorter.1 A better understanding of short-term training providers will give us insights into the type and quality of services available in Moldova. 1 In Moldova, there is no formal definition for short-term training providers. 16 Introduction Defining key players • MoECR: The MoECR governs VET, higher education, and other type of public and/ or private institutions that provide training and education programs. • MoHLSP: This is responsible for the development of employment policy and moni- toring labor market trends (European Training Foundation 2017). • National Agency for Quality Assurance in Education and Research (ANACEC): This is the administrative authority subordinated to the MoECR, responsible for im- plementing state policies and contributing to development oriented toward inter- national standards in the field of quality assurance in general, technical professional, higher education, continuing vocational training and research (ANACIP 2015). • National Employment Agency (ANOFM): Responsible for implementing labor market policies and producing specific labor market information (labor market fore- cast, profession/trade barometer using methodologies developed with European Union [EU] support) (European Training Foundation 2017). • External partners: World Bank, EU, U.S. Agency for International Development (USAID), and others. Defining key instruments • Constitution of the Republic of Moldova principally regulates the legal framework of education. • Education Code (passed in July 2014 and effective since November 2014, substi- tuting the Education Law of 1995) is the fundamental legislative and regulatory act for education at all levels. It also provided legislation for the modernization of the education and training system, including the establishment of a quality assurance agency for VET and higher education, stronger involvement of social partners in education, implementation of the National Qualifications Framework (NQF), and the extension of compulsory education to the age of 18 from 2018 (European Training Foundation 2017). • National Development Strategy ‘Moldova 2020: Seven Solutions for the Republic of Moldova’ (Moldova 2020) targets a number of cross-sectoral devel- opment priorities, one of which is “aligning the education system to labor market needs in order to enhance labor productivity and increase employment in the econ- omy” (Republic of Moldova 2014). • ‘Strategy for the Development of the Vocational/Technical Education for the Period 2013–2020’ (VET Strategy) clearly analyzes the challenges in the VET sector in Moldova. The VET Strategy aims to modernize and upgrade VET educa- tion to increase competitiveness of the national economy through the training of Introduction 17 a competent and qualified workforce in line with current and future labor market needs (SABER-WfD 2013). • ‘National Employment Strategy 2017–2020’ is currently being developed. The entire VET network will go through an optimization process, where the downsizing of VET schools will be combined with the creation of 11 new centers of excellence. These centers will serve as multifunctioning training providers (European Training Foundation 2017). • The NQF is being developed by the MoECR, in collaboration with relevant minis- tries, sector committees, VET institutions, businesses, and other partners, with ap- proval from the government. Sector skills councils and working groups have con- tributed to developing 33 occupational standards and 43 qualifications, according to the 2017 report by the European Training Foundation. The NQF is also a critical element of the EU-Moldova Association Agreement. Defining skills • Skills: These include competencies, attitudes, beliefs, and behaviors that are mallea- ble (modifiable) during an individual’s development and can be learned and im- proved through specific programs and policies (Guerra, Modecki, and Cunningham 2014). • Cognitive skills: The ability to process information, understand complex ideas, learn from experience, reason, remember, relate, and overcome obstacles through problem solving (for example, literacy and numeracy skills) • Socioemotional skills: Learned knowledge, attitudes, and skills necessary to under- stand and manage emotions, set and achieve positive goals, establish and maintain positive relationships, and make responsible decisions (CASEL 2018). These skills are also referred to as noncognitive, behavioral, soft, or life skills. • Technical skills: Specific knowledge to carry out one’s job and psychomotor or manual dexterity. 18 Introduction Chapter 1: Findings from the mapping exercise of short-term training institutions in Moldova Due to limited information on the market of the short-term training providers in Moldova, before applying the TAP instruments, as a first stage, a mapping of the institutions was conducted. The mapping exercise was conducted by CIVIS in 2017. Table 1 summarizes the methodology for short-term training institutions’ mapping and the basic characteris- tics of short-term training providers. The landscape of short-term training providers in Moldovia was built on a database of 272 entities in total, as a result of screening 1,047 companies. Of the identified 1,047 entities, 308 reported that they do not provide short-term training, 144 refused to partici- pate in the survey, 54 are in the process of closing their short-term training programs, and 269 institutions did not provide any response. Finally, 272 entities confirmed that they provide short-term training and were surveyed for basic information. TABLE 1. MAPPING METHODOLOGY OF SHORT-TERM TRAINING INSTITUTIONS IN MOLDOVA Type of survey Quantitative survey at the national level Data collection method Computer-Assisted Personal Interview (CAPI) Structured questionnaire—mapping tool, provided by the World Research tools Bank and adapted to local context by CIVIS (survey firm) Administrative staff or short-term training providers (director, Target population manager, chief accountant) Operational definition Any entity providing courses/programs with a duration of 18 of short-term training months or less, which are not part of the basic school curricula, providers including VET institutions and universities To adapt to the Moldovan context, the research tool was prelimi- Questionnaire pre-test narily tested and finalized, following the results of pre-test before the main fieldwork was conducted. Language of interviews Romanian or Russian, depending on respondent’s preference Average length of 30 minutes (including callbacks) interviews Fieldwork period October 10–December 20, 2017 Chapter 1: Findings from the mapping exercise of short-term training institutions in Moldova 19 Initial list of training providers was offered by the MoECR, based on the extract of information from the Licensing Chamber. This list was completed from other available official and private sources: • Web page of the MoECR (http://mecc.gov.md/ro/content/ invatare-pe-tot-parcursul-vietii), which includes information about entities providing training programs coordinated with/ by and accredited by the ministry • NBS, which provided the list of all entities corresponding to Resources and steps Statistical Classification of Economic Activities in Moldova taken to identify training (NACE, Rev 2), Section P, Division 85, Class 85.32 to 85.60 institutions • Private databases (yellowpage, compass), as well as Google search engine; terms used for keyword searches were short- term training, training, courses (professional, vocational), professional schools, and craft schools All entities identified from various sources were consolidated to one list and cleaned for duplications. All identified potential short-term training providers were called. Those who responded and confirmed the status of short-term training providers were interviewed. 1,047 entities identified as potential short-term training providers Sample size 272 entities that confirmed they provide short-term training and were interviewed for basic information Results of calls Number Percentage Total 1,047 100 Questionnaire completed 272 26 Does not provide short- 308 29 term training Refused to participate 144 14 Institution is in the process of closing/has ceased its 54 5 activity No response 269 26 Status of nonresponse and 100% response rate to all 20 questions, except Question #8 (Total missing questions (ques- number of students enrolled in short-term programs in 2016), for tionnaire in Annex 1) which response rate was 90.8% Short-term training providers in numbers Most short-term training providers in Moldova operate in the urban areas. Around 70 percent of the short-term training providers are in the municipality of Chisinau, while only about 10 percent of the providers are in the North and Center regions (Figure 2). The least number of training providers are located in the South (6 percent). Ninety-five percent of the providers operate in urban areas, while only 5 percent are in rural areas. 20 Chapter 1: Findings from the mapping exercise of short-term training institutions in Moldova The landscape survey also yielded results that about half of the short-term training providers have online presence, with an active web page. Close to 40 percentage of the providers have a Facebook page, and one out of four providers with a Facebook page does not have an active web page. More than three-quarters of the short-term training providers have been oper- ating for longer than 5 years, with 54 percent of providers operating for more than 10 years (Figure 3). Almost 20 percent of the providers were established during the Soviet Union period (1922 to 1991), while about half of the providers started operating in in the last decade. Short-term training institutions in Moldova vary considerably in student size, with some providers serving thousands of students. The average number of students in short-term training institutions was 468 individuals, with the lowest number of students being 3 and the highest number of students being as high as 30,000. The short-term training providers that were surveyed in 2016 trained a total number of 120,000 stu- dents. This is approximately 6 percent of the 2.15 million total working-age population between 16 and 55 years (NBS). Out of 272 providers, 25 covered 70 percent of the total enrollment. The following short-term training providers had the highest number of en- rollments, covering 39 percent of the total number of students: • Training Center of Specialists for the National Army, providing mainly driv- FIGURE 2. er courses (30,000 students) DISTRIBUTION BY DEVELOPMENT • Car Drivers Union (5,500 students) REGION, N = 272 71% • FOBOS-COOP, providing training in security/safety (4,000 students) • Training and production center, pro- viding trainings for operator profes- sions (4,000 students) 12% 11% • Center for continuous medical edu- 6% cation of medical and pharmaceu- tical staff with secondary education North Center South Chisinau (3,768 students) Most students enrolled in the short- FIGURE 3. term training providers are from the LENGTH OF OPERATION, N = 272 working-age population, between 16 54% and 55 years. A large proportion of the students were in the age range of 25–55 23% years, accounting for about 40 percent of 16% the training providers. This is followed by 7% 29 percent of students ages 20–24 years and 26 percent ages 16–19 years. Only 5 Up to 2 3-5 years 6-10 years More than years 10 years Chapter 1: Findings from the mapping exercise of short-term training institutions in Moldova 21 percent of the training providers had stu- FIGURE 4. dents who were older than 56 years. SHARE OF ENROLLED STUDENTS BY TYPE OF SHORT-TERM TRAINING While most of the short-term train- PROVIDER, N = 247 ing providers are private institutions, Mixed, almost half of the students are enrolled NGO, 5% in public institutions. A majority of the 8% short-term training providers (72 percent) are private, in comparison to 19 percent of public institutions. This is followed by 7 percent of nongovernment training organi- zations and 2 percent of organizations with Public, 46% mixed status, managed and financed both by government and commercial entities. Private, Despite the large number of private train- 41% ing providers, 46 percent of the enrolled students were served by public institutions (Figure 4). In fact, three of these public pro- viders account for over one-third of the total enrolled students. Private entities pro- Sample: 247 respondents reporting vide training services to 41 percent of the enrollment data enrolled students, followed by 8 percent in nongovernmental organizations (NGOs) and 5 percent in mixed status. Almost 90 percent of the short-term training institutions are small- or medi- um-size (fewer than 700 students enrolled). The size of short-term training providers can be grouped by the number of students enrolled.2 The average number of students enrolled in short-term training institutions is 468 students. Using this number, small-size institutions are defined as those with enrollment number equal to or fewer than half of the average (234 students). Medium-size training providers have 235 to 700 students, which are the bottom half and upper half of the average. Finally, large-size training pro- viders are those with more than 700 students enrolled. As Table  2 shows, around 63 percent of the training institutions are small-size providers, followed by 17 percent of medium-size providers. Only 11 percent of the short-term training providers were large- size institutions. 2 Note that 9 percent of the training providers (25 providers) did not provide an enrollment number. 22 Chapter 1: Findings from the mapping exercise of short-term training institutions in Moldova Short-term training providers’ profile TABLE 2. NUMBER OF SHORT-TERM TRAINING INSTITUTIONS BY SIZE AND TYPES OF TRAINING PROVIDED Variables Small Medium Large Unknown Total Training only 110 39 28 17 194 Education (general) and training 34 3 1 3 41 Non-training related activities 27 5 0 5 37 Total 171 47 29 25 272 Almost nine out of ten short-term training providers have training provision as their core business activity. Of these, 71 percent provide (short-term) training only and 15 percent provide both education and training (Figure 5). The remaining 14 percent pro- vide short-term trainings as secondary economic activity. More than half of the public training pro- FIGURE 5. viders (59 percent) have education and short- CORE BUSINESS, N = 272 term training as their main business, with just 71% 35 percent of them providing short-term train- ing services only. In contrast, a greater majority of the private training providers (81 percent), 15% 14% NGOs (68 percent), and mixed providers (83 percent) offer only short-term training services Training only Education Non-training as their core business. and training activities Driving and language training cours- es were the most popular among enrolled students in 2016. When providers were asked to name three courses with the highest enrollment, the weighted average yielded that driving (23 percent), language (16 percent), and make-up and cosmetology (11 per- cent) categories of courses had the highest number of students (Figure 6)3. The catego- ries of courses were also analyzed by the number of students enrolled, where the highest number of students were in driver courses, with a share of 49 percent of the total student enrollment (Figure 7). This is followed by language courses, ‘specialist in electricity, gas, machinery’, ‘safety and security’, and ‘healthcare and pharmaceutics’, which account for 32 percent of the total number of students. A majority of the training institutions provide fewer than 10 courses. On average, a short-term training provider in Moldova offers 6 courses. About 60 percent of the enti- ties provide up to three courses and among these entities (Figure 8), slightly more than 3 Percentages for each mention of profession were cumulated and the resulted cumulative percentage (where total percentage is higher than 100 percent) from three mentions were recalculated and weighted as 100 percent. Chapter 1: Findings from the mapping exercise of short-term training institutions in Moldova 23 FIGURE 6. Driver Language courses 16% 23% ENROLLMENT BY Makeup & Cosmetology 11% PROFESSIONS IN HoReCa 5% DEMAND ON THE Specialist in electricity, gas, machinery 5% MARKET (WEIGHTED Accounting and audit 5% AVERAGE AS SHARE OF Management 4% TOP THREE COURSES), Clothes and footwear manufacturing 4% Safety & security 4% N = 272 Massage 3% IT Programming & design 2% Education & teaching 2% Sales 2% Agriculture & forestry 2% Business 2% Art and sport 1% Healthcare & pharmaceutics 1% Personal development 1% Household services 1% Art objects 1% Basic IT 1% Other 6% FIGURE 7. Driver Language courses 11,770 56,805 ENROLLMENT Specialist in electricity, gas, machinery 9,648 BY PROFESSIONS Safety & Security 9,340 IN DEMAND ON Healthcare & Pharmaceutics 6,038 THE MARKET Accounting and audit 3,146 Education & teaching 2,638 (BY CUMULATIVE Business 2,573 NUMBER OF ENROLLED Makeup & Cosmetology 2,524 STUDENTS), N = 272 Management 2,144 Sales 1,258 HoReCa 991 Household services 840 Personal development 765 Massage 666 Agriculture & forestry 647 Clothes and footwear manufacturing 535 IT Programming & Design 338 Basic IT 86 Art and sport 67 Art objects 29 Other 2,895 FIGURE 8. FIGURE 9. NUMBER OF COURSES, DURATION OF TOP THREE COURSES N = 272 (WEIGHTED AVERAGE), N = 272 10 and more Up to one month 31% courses, 15% between 1 and 3 months 41% between 3 to 6 months 20% 4-9 courses, 1-3 courses, 26% 59% between 6 to 12 months 6% between 12 and 18 months 1% more than 18 months 1% 24 Chapter 1: Findings from the mapping exercise of short-term training institutions in Moldova half of them offer just one course. Only one-quarter of the providers offered 4–9 courses, followed by around 15 percent of them (or 40 providers) providing 10 and more courses. Some training institutions offered more than 100 courses. These providers with the high- est number of courses are the following: • Center for continuous medical education of medical and pharmaceutical staff with secondary education (165 courses) • Center for professional training of medical and pharmaceutical workers with sec- ondary education (132 courses) • Association of professional accountants and auditors from the Republic of Moldova (100 courses) Course duration of short-term training providers is generally less than six months. Training institutions were asked to identify three courses with the highest enrollment in 2016 and the duration of each of the mentioned courses. Courses with a duration of one to three months came with a high demand, with 41 percent (Figure 9). This is followed by 31 percent of training courses that were shorter than one month and 20 percent of cours- es with a duration between three and six months. Training courses that were longer than six months have a share less than 10 percent, pointing to the result that a majority of the courses offered by short-term training providers were shorter than six months. Out of the top five categories of courses with the highest enrollment, driver courses, language training, and healthcare and pharmaceutics are usually trained during a one- to-three-months period, while those training in the category of safety and security are trained in a one-month period. Specialists in electricity, gas, and machinery have a variety of durations, with a balanced distribution among courses, from one to six months. Among the courses offered by short-term training providers, almost half of the trainings are in the domains of transport and storage, as well as education. Consistent with the previous findings that showed a large share of courses focusing on driving and high enrollment in these trainings, the highest percentage of short-term training provid- ers offered trainings related to transportation and storage (23 percent) (Table  3).4 This was followed by education5 (22 percent) and other service activities that were not speci- fied (16 percent). 4 The domains in Table  3 are from the National Classification of Economical Activities Rev. 2, which corresponds with the statistical classification of economic activities in the European Community and International Standard Industrial Classification (ISIC). 5 Education industry includes courses such as pre-university course of preparation for admission to musical specialties, primary and preprimary education, learning centered on the needs of the learner, teacher training on school disciplines, improving the culture workers, community school—a model for comprehensive approach to educational reforms, and psycho-pedagogical—combination of two studies, pedagogy, and psychology—module. Chapter 1: Findings from the mapping exercise of short-term training institutions in Moldova 25 TABLE 3. DOMAINS IN WHICH TRAININGS ARE PROVIDED, N = 272 Transport and storage 23% Education, including language courses 22% Health and social assistance 7% Accommodation and catering 6% Financial and insurance activities 6% Administrative service activities and support service activities 6% Manufacturing industry 5% Professional, scientific, and technical activities 5% Agriculture, forestry, and fishing 4% Production and supply of electric and thermal energy, gas, hot water, and air conditioning 4% Construction 4% Art, recreation, and leisure activities 4% Wholesale and retail trade; maintenance and repair of motor vehicles and motorcycles 3% Information and communications 3% Public administration and defense; compulsory social insurance 3% Extractive industry 1% Water distribution; sanitation, waste management, decontamination activities 1% Real estate transactions 1% Activities of private households as employer of domestic personnel 1% Activities of extraterritorial organizations and bodies 1% Other service activities 16% Almost three-quarters of short-term training institutions report to the MoECR. Nine out of ten training providers report their training activities to the central authori- ties, and one out of five reports to two or three different central authorities. A majority of the short-term training providers are overseen by the MoECR (74 percent), which is followed by the Ministry of Economy and Infrastructure (17 percent) and the MoHLSP (10 percent) (Figure 10). Other authorities mentioned by providers include the Ministry of Agriculture, Regional Development and Environment, the National Center of Public Health, the Ministry of Internal Affairs, and the Ministry of Finance. Most of the public and private training providers report to the MoECR, about 80 per- cent and 75 percent, respectively. Providers with core business in general education and training are also more likely to report to the MoECR (90 percent), while those with core business in non-training related activities are more likely to report to the Ministry of Economy and Infrastructure (MoEI) (30 percent) and the MoHLSP (19 percent). Providers reporting to the MoHLSP have a greater tendency to provide courses in make-up and cosmetology, massage, and safety and security. Providers specializing in accounting and audit are more likely to report to the Ministry of Finance. While a large proportion of short-term training providers have been licensed by the government, a small proportion have been accredited. Following the adoption of 26 Chapter 1: Findings from the mapping exercise of short-term training institutions in Moldova the Education Code of the Republic of Moldova FIGURE 10. (2014) and the creation of the ANACEC, any 6 OVERSEEING MINISTRY OF SHORT-TERM TRAINING economic activity of education service (includ- PROVIDERS, N = 272 ing short-term training) must be authorized by the ANACEC. To be licensed, each training enti- MoHLSP ty has to submit an education plan coordinated 10% and approved by the MoECR. By going through MoEI this process, short-term training institutions are 17% authorized to provide training services to the MoECR 73% public. Accreditation refers to the external evalu- ation process of the program or institution qual- ity, materialized by issuing a certificate with the right to carry out educational process; organize admission to studies and examinations for graduation; and issue diplomas, certificates, and other study documents recognized by the MoECR. Among the surveyed short-term training providers, 61 percent of them are licensed to provide training services. Public and private entities are more likely to be licensed than any other types of institutions (66 percent and 60 percent, respectively), while only one-third of the NGOs and mixed entities are licensed. Providers with core business in short-term training or general education and training are also more likely to be licensed (66 percent) than institutions providing non-training related activities (43 percent). With regard to the accreditation process, only 38 percent of the providers applied, and among those who applied, courses were granted accreditation by the ANACEC about half of the times. Survey results found that providers operating in Chisinau are more likely to apply for accreditation than any other regions. Short-term training institutions offering a greater number of courses have a higher percentage of being licensed or applying for accreditation. More than 70 percent of the providers with 10 or more courses are licensed, in contrast with just 54 percent of the institutions providing 1 to 3 courses. Similarly, 48 percent of those offering 10 or more courses applied for accreditation, while just 35 percent of those providing 1 to 3 courses applied for such process. Providers with a greater number of courses are more likely to have branches elsewhere. Data from the landscape survey show that one out of ten short-term training providers has branches. The vast majority of providers with branches have up to two branches (73 percent). The greatest number of branches reported by a short-term train- ing provider was 20 branches, which was a company offering driver courses. This was followed by 11 branches of a company providing language courses. 6 State entity responsible for authorization, external evaluation, and accreditation of institutions and study programs in preschool, general education, vocational education, and higher and continuing education in the Republic of Moldova. Chapter 1: Findings from the mapping exercise of short-term training institutions in Moldova 27 Chapter 2: Characteristics of short-term training institutions in Moldova based on TAP results Following the mapping exercise, TAP used qualitative and quantitative methods to col- lect in-depth information from three main stakeholder groups: short-term training pro- viders, students and recent graduates of these training providers, and employers collab- orating with short-term training providers. The quality assessment tool was customized to the Moldovan context, under the direction of the World Bank and in consultation with the MoECR. The tool also built on the institutional self-assessment tool developed by the ANACEC. The assessment tool includes (a) an in-depth survey with the training in- stitutions’ representatives, and (b) focus groups with students, graduates, and selected employers.7 An in-depth survey was conducted in 50 identified short-term training pro- viders, and focus group discussions were conducted with 17 groups, 127 participants in total (6 groups with 50 students, 7 groups with 50 graduates, 4 groups with 27 employ- ers). Among the 272 training providers identified during the mapping stage, 50 training institutions were selected based on stratified random sampling. More information on sampling and the survey methodology can be found in Annex 1. TAP begins by identifying the inputs from training institutions. It asks directors or decision makers of short-term training institutions about four different types of inputs that influence the ability of institutions to provide quality training and achieve high per- formance: students, instructors, facilities and equipment, and funding. This information, which is self-reported by training institutions, can help improve the quality of training in Moldova for two reasons. First, as the first instrument to collect nuanced (yet standard) information from a considerable number of institutions, it fills an information gap that is crucial to enhance the management of training providers. Second, by identifying pat- terns, strengths, and weaknesses among institutions by type and status, it can help inform policy responses. Data were collected from a total of 50 identified short-term training institutions, out of which 34 were private entities, 9 were public, and 7 were not-for-profit institutions (Figure  11). More than half of the training institutions were training firms, which were mostly private entities (Figure 12). Regionally, the surveyed training providers are mostly 7 For more information on the research methodology, please see Annex 1. 28 Chapter 2: Characteristics of short-term training institutions in Moldova based on TAP results located in Chisinau (72 percent), followed by 12 percent in North, 12 percent in Center, and 4 percent in the South. On average, training institutions have been operating for about 18 years. A majority of short-term training providers in Moldova are small-size8 institutions, with fewer than 235 students (Table 4). These findings on general character- istics of training providers are consistent with the landscape survey results. TABLE 4. NUMBER OF SHORT-TERM TRAINING INSTITUTIONS BY SIZE AND TYPES OF TRAINING PROVIDED Variables Small Medium Large Unknown Total Private 20 7 3 4 34 Public 7 0 1 1 9 Not-for-profit 3 2 1 1 7 Total 30 9 5 6 50 FIGURE 11. FIGURE 12. INSTITUTION TYPE, N = 50 INSTITUTION CATEGORY, N = 50 14% 18% 68% University/Academy (1) College (1) Professional school (7) Private (34) Business association (4) Public (9) NGO (4) Not-for-profit (7) Training firm (27) Enterprise (training is not main business) (6) 8 The principle used for grouping training providers by size follows the same principle used during the mapping stage: small-size institutions are those with student size smaller than half of the average (234 students). Medium-size providers have 235 to 700 students, which are the bottom and upper half of the average. Large-size institutions have more than 700 students. Chapter 2: Characteristics of short-term training institutions in Moldova based on TAP results 29 Students Short-term training institutions come FIGURE 13. with a wide range in student size. A AGE DISTRIBUTION OF SHORT-TERM TRAINING STUDENTS, N = 30 total of nearly 19,000 students were en- rolled in the 50 training institutions. The 2% number of students enrolled ranges from as small as 3 to as large as 2008. While one-fifth of the institutions enrolled few- 22% 21% er than 30 students, another one-fifth enrolled more than 500 students. On av- erage, the student enrollment number in short-term training institutions is 388, with a median of 132. Of the total 18,995 enrolled students in 2017, 83 percent of 20% them were enrolled in short-term training programs, as opposed to other types of 35% programs or activities provided by insti- tutions. An average of 337 students were enrolled in short-term training programs, 16-19 years old 20-24 years old with a median of 120. 25-35 years old 36-55 years old Short-term training institutions 56 years and older mostly serve students in the work- ing-age population (16 to 55 years). A larger proportion of students are in the 25–35 years age range, with over 30 percent of the total short-term training students (Figure 13). The rest of the age groups (16–19 years, 20–24 years, and 36–55 years) have an almost equal distribution of about 20 percent for each group. Only 2 percent of the short-term training students are 56 years or older. In public training institutions, almost half of the enrolled students are in the 25–35 years range and one-third in the 36–55 years group. Private institutions also have a greater per- centage of students in the 25–35 years range but have a more equal distribution among the different age groups. On average, short-term training institutions have a similar proportion of full- time and part-time students. An average of 49 percent of short-term training students in 2017 were full-time (attending more than 15 hours of instruction per week), while an average of 51 percent were part-time (attending 14 or fewer hours of instruction per week). In terms of gender, male full-time enrollment (51 percent) was very similar to fe- male full-time enrollment (49 percent). Among part-time students, female enrollment (63 percent) was greater than male enrollment (37 percent). Most students enrolled in short-term training institutions have completed at least basic education or a higher level of education. More than a third of the students 30 Chapter 2: Characteristics of short-term training institutions in Moldova based on TAP results attained higher education (32 percent with bachelor’s degree and 3 percent with master’s degree) (Figure 14). Close to one-fifth of the students have completed up to high school and another one-fifth have completed until vocational education. Ten percent of the stu- dents enrolled have no certification or basic education. Not-for-profit institutions have a greater percentage of students with low levels of education than other institution types. Nearly 50 percent of the students enrolled in not-for-profit entities have no education or completed just basic education. In contrast, private entities have students with higher lev- els of education, close to 40 percent having attained bachelor’s or master’s degree. FIGURE 14. AVERAGE EDUCATIONAL ATTAINMENT OF STUDENTS AT THE TIME OF ENROLLMENT (2017), N = 50 Not-for-profit 30 14 13 27 0 16 Public 27 26 9 24 1 13 Private 11 19 23 35 4 8 Total 17 19 19 32 3 10 0% 20% 40% 60% 80% 100% Basic education/primary High school Vocational education Higher education (bachelor degree) Higher education (master degree) No certificate, basic literacy skills In 2017, training institutions reported that an average of 10 students dropped out of training courses. When asked about the reason for dropping out, an average of 25 percent of the total dropouts stated that they could not afford to continue, and anoth- er quarter reported that they had family responsibilities (other than financial) (Figure 15). For those training institutions that identified financial difficulty as a reason for students dropping out, more than half of the institutions reported that this reason applies more frequently to female dropouts. FIGURE 15. REASONS FOR DROPPING OUT IN 2017, AS A PERCENTAGE RELATIVE TO THE TOTAL DROPOUT RATE, N = 22 7% Had family responsibilities (other than financial) 7% 25% Could not a ord to continue 8% Found career opportunity that could not wait Fell sick 10% Found program too di cult Did not find program relevant 24% Other (went abroad) 19% Chapter 2: Characteristics of short-term training institutions in Moldova based on TAP results 31 Teachers/Instructors Almost all institutions have the autonomy to recruit instructors/teachers. Close to 60 percent of the training institutions reported that they have the autonomy, and 32 percent of the providers said they have the autonomy but only by following government criteria. Just 3 out of 50 training institutions could only recruit part-time or contract instructors, and only 2 institutions did not have any autonomy to recruit instructors. Among these 5 training institutions, 3 of them reported that the recruitment of instructors was deter- mined by the national education authority. One institution said it was determined by the subnational education authority, and another reported local education authority. Approximately 1,000 instructors/teachers are working across 50 surveyed train- ing institutions. On average, training providers have about 20 instructors, and 6 of these instructors teach in short-term programs. Among the teachers who teach short-term training programs, 48 percent of them work full-time and 52 percent of them work part- time. On average, the gender breakdown was mostly equal for both full-time instructors (52 percent male and 48 percent female) and slightly less so for part-time teachers (58 percent male and 42 percent female). A majority of instructors/teachers in short-term training programs have more than 10 years of industry and teaching experience. On average, 61 percent of teach- ers have more than 10 years of industry experience (Figure 16) and 58 percent of teach- ers fall within the same range for teaching experience (Figure 17). There are about 24 percent of teachers who have 5–10 years of industry or teaching experience and less than 20 percent of instructors with less than 5 years of industry or teaching experience. In comparison to private or not-for-profit institutions, which have less than 60 percent of instructors with more than 10 years of industry or teaching experience, public institutions have a slightly greater percentage of teachers who are experienced, with 67 percent. FIGURE 16. FIGURE 17. AVERAGE INDUSTRY EXPERIENCE AVERAGE TEACHING EXPERIENCE OF OF INSTRUCTORS/TEACHERS, N = 50 INSTRUCTORS/TEACHERS, N = 50 Total 15% 24% 61% Total 18% 24% 58% Not-for-profit 14% 29% 57% Not-for-profit 29% 14% 57% Public 15% 18% 67% Public 13% 20% 67% Private 15% 25% 60% Private 17% 27% 57% 0% 20% 40% 60% 80% 100% 0% 20% 40% 60% 80% 100% Less than 5 years 5-10 years Less than 5 years 5-10 years More than 10 years More than 10 years 32 Chapter 2: Characteristics of short-term training institutions in Moldova based on TAP results FIGURE 18. AVERAGE EDUCATION CREDENTIALS OF INSTRUCTORS/TEACHERS, N = 50 (WEIGHTED) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Private 9% 30% 38% 23% Public 38% 38% 25% Non-profit 22% 22% 56% Total 6% 31% 36% 28% Certificate Diploma Graduate/Bachelor Master or higher Instructors/teachers have education credentials, with most having completed at least high school. An average of 31 percent of teachers have a diploma, followed by 36 percent with a graduate (bachelor’s) degree and 28 percent with a master’s (or higher) degree (Figure 18). Just 6 percent of the teachers have a certificate. More than half of the teachers in not-for-profit institutions have a master’s or higher degree (56 percent) in comparison to 25 percent in public and 23 percent in private institutions. In both public and private training institutions, close to 40 percent of the teachers have a bachelor’s or graduate degree. Teachers’ salary varies considerably at short-term training institutions with the monthly salary scale ranging from as low as MDL 209 (US$12.30)9 to as high as MDL 12,000 (US$705.96)10. On average, the highest paid teacher received MDL 4,486 (US$264.32) and the low- est paid teacher received MDL 2,697 (US$158.91). It is interesting to note that 54 percent of the highest paid teachers in the 50 training institutions were full-time and 46 percent was part-time, while for the lowest paid teachers, 82 percent were part-time and only 18 percent of them were full-time. The average pay for short-term training instructors in 2017 was MDL 3,612 (US$212.82). There was a slight difference in the average pay for male teachers (MDL 3,833 or US$225.84) and female teachers (MDL 3,523 or US$207.58), with women earning approximately MDL 310 (or US$18) less than men. 9 It is important to note that the training provider who provided this data indicated that the salary value pertains to part-time instructors at the institution. 10 1 Moldovan Leu (MDL) = US$0.059 (June 22, 2018). Chapter 2: Characteristics of short-term training institutions in Moldova based on TAP results 33 Facilities and equipment Training institutions generally have functional and sufficient tools and equipment for classes and workshops. On average, training providers have approximately seven classrooms. Approximately two work- shops are used for short-term training FIGURE 19. programs. With respect to the largest AVERAGE NUMBER OF COMPUTERS, N = 47 workshop used for short-term training, 94 percent of the institutions reported 50 46 45 that the tools and equipment used for the 40 workshops were very functional or func- 35 30 tional enough (46 percent and 48 per- 25 cent, respectively). More than three-quar- 20 15 12 ters of the institutions also said that they 10 10 4 have enough tools and equipment to 5 2 2 1 1 0 accommodate all enrolled students. For Total Private Public Not-for-profit 14 percent of the institutions, it depend- Teacher use Student use ed on the semester and number of en- rolled students, while 10 percent did not have enough tools and equipment for all their students. Training providers have about 17 computers on average, of which 10 were for student use and 4 for teacher use. Public institutions have the greatest average number of computers for student use (Figure 19). For almost half of the institutions in Moldova, Internet speed was functional and over 20 MB per second. Most training institutions do not have dormitories to accommodate short-term students. Out of the six institutions that have dormitories, the dormitory services are accessible on a first-come, first-served basis for four institutions. The other two institu- tions provided dormitory services based on distance of students’ residence. On average, dormitories can accommodate 62 male short-term students and 101 female short-term students, but only an average of nine male and four female students actually reside per year. For four out of six institutions with dormitories, the service is paid by students, while the two other training providers are subsidized by the government. Funding A large proportion of the turnover for training institutions was derived from short- term training programs, as opposed to other types of programs or activities pro- vided by institutions. On average, 68 percent of the income is derived from short-term training only. More than three-quarters of the training institutions reported that their in- come was less than MDL 500,000 (US$29,485) in 2017 (Figure 20), followed by 12 per- cent between MDL 500,000 and MDL 1 million (US$58,970), and 12 percent with more 34 Chapter 2: Characteristics of short-term training institutions in Moldova based on TAP results than MDL 1 million. The total turnover for training institutions was represented by 62 per- cent tuition fees, 15 percent sales of services to private firms, 12 percent sponsorships or donations, 10 percent government funding, and 1 percent membership fees. In 2017, most of the total annual expenses for training institutions comprised salaries, infrastructure, and equipment. Teacher/instructor and staff salaries together represented 46 percent of the total expenses (34 percent and 12 percent respectively) (Figure 21). Over a quarter of the spending was represented by physical infrastructure, which included rent and building maintenance. Equipment maintenance and consum- ables made up 14 percent of the total annual expenses. Public entities spent less on phys- ical infrastructure and building maintenance (17 percent) than private or not-for-profit entities (29 percent and 24 percent, respectively). However, the proportion of spending on teacher/instructor salary was the highest in public institutions (42 percent), followed by private (34 percent) and not-for-profit (21 percent) training institutions. FIGURE 21. EXPENSE COMPOSITION OF TRAINING INSTITUTIONS, N = 47 1% Total 34% 26% 14% 12% 6% 4% 3% Not-for- 4% 21% 24% 10% 17% 6% 16% profit 3% 2% Public 42% 17% 15% 18% 3% 3% 4% Private 34% 29% 14% 9% 7% 2% 0% 20% 40% 60% 80% 100% Teacher/instructor salary Physical infrastructure (including rent)/building maintenance Equipment maintenance and consumables Sta salary (excluding teachers/instructors) Classroom supplies Extracurricular activities Personal and professional development of teachers/sta Debt repayment Chapter 2: Characteristics of short-term training institutions in Moldova based on TAP results 35 Chapter 3: Actions of short-term training institutions in Moldova Acknowledging that the management practices of training institutions have a direct im- pact on students’ learning, TAP identifies the following nine actions that training institu- tions can follow to build a workforce with the right skills for the growing and changing economies. Strategic actions of short-term training institutions Action 1: To set a strategic direction Action 2: To develop a demand-driven approach to training Action 3: To establish a sustained relationship with authorities Action 4: To ensure institutional financial viability Action 5: To fulfill national quality standards Action 6: To enable students to pursue education and training opportunities Action 7: To create teaching/studying experience conducive to learning Action 8: To prepare students for the world of work Action 9: To gather and publicize data for informed decision-making Action 1: To set a strategic direction A key component of effective management is the existence of defined goals and strate- gies to achieve them. For training institutions, such goals are expressed in their long-term vision and corresponding targets. Institutions that engage the perspectives of firms and students in their strategic planning are more likely to be able to respond to the needs of the labor market. To translate intentions into actions, the goal setting must be accompa- nied by mechanisms to monitor progress and ensure accountability. Under this first insti- tutional goal, TAP collects information on whether institutions set and monitor short- and long-term goals and if staff are accountable when it comes to achieving them. 36 Chapter 3: Actions of short-term training institutions in Moldova Many short-term training providers have formally developed mission state- ments, but few consider the perspectives of industry associations, employers, and the community. Eighty percent of the training providers have a formally written or vision statement. Among these institutions, 40 percent engage students and only one-quar- ter involve community members when developing a mission statement. Forty percent of these institutions with a mission statement reported that their statements do not speak of the connection with industry associations and/or employers. Although a majority of the training providers collect data on current students and instructors, few collect information on graduates from the program. Eighty-six percent of institutions collect administrative data, and 78 percent on student perfor- mance. Close to 70 percent of training providers assess teacher/instructor performance. These data are used to evaluate the training program and monitor the attainment of targets. Although approximately three-quarters of training providers collect information on program completion and trainee satisfaction, less than one-third have data on their job placement rates and employee satisfaction. Only 3 out of 50 surveyed training insti- tutions collect information on graduates’ earnings. A large proportion of institutions do not have internal accountability mecha- nisms. Close to 40 percent of the providers reported that they do not have a manage- ment committee or governance board. While a higher percentage of public institutions tend to have such accountability mechanisms (89 percent), almost 50 percent of private or not-for-profit entities do not (Figure 22). Among institutions with internal accountabili- ty mechanisms, fewer than half reported that the members of their management commit- tee or governance board are also members in industries or industry associations. Three- quarters of the providers with a management committee or governance board said their managers are held accountable to the committee or board. FIGURE 22. DOES THE INSTITUTION HAVE A MANAGEMENT COMMITTEE OR GOVERNANCE BOARD? (N = 50) Not-for-profit 57% 43% Public 89% 11% Private 56% 44% Total 62% 38% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Yes No Chapter 3: Actions of short-term training institutions in Moldova 37 Action 2: To develop a demand-driven approach to training In a rapidly changing world, training institutions must assess the relevance of their train- ing programs frequently and adapt accordingly. Although several mechanisms are avail- able, including the formal or informal periodic review of skills needs, connecting with lo- cal employers is likely to be a cost-effective approach. TAP asks training institutions about how, from a strategic point of view, they identify the skills that their programs should foster and how they engage with firms. Only about half of the short-term training institutions do not give enough at- tention to industry demands or skills needs assessments when determining skills developed through the program. Focus group discussions administered through this project reveal that almost all students and graduates believe training institutions should have the responsibility to provide accurate information on labor market and skills need- ed. TAP suggests that all training providers use at least one mechanism to identify the skills demanded by the market. However, only half of the institutions utilize studies and assessments of skill needs (Figure 23). A similar proportion of training providers report- ed taking considerations of employers and industry demands (46 percent), and internal discussion/review (42 percent) to identify skill needs. For private entities, studies and assessments are one of the most important ways to determine the skills to be taught in training program (85 percent). In comparison, a high percentage of public institutions (89 percent) identify skill needs based on the funding received. FIGURE 23. MECHANISMS TO DETERMINE THE SKILLS TO BE TAUGHT IN TRAINING PROGRAMS, N = 50 (MULTIPLE RESPONSES) 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Total Private Public Not-for-profit Studies and assessments of skill 50% 85% 33% 57% needs Through internal discussion/review 42% 24% 43% Based on funding received for the 18% 50% 89% 14% training program Employers and industry demands 46% 21% 22% 14% Government policies 14% 22% 14% Other 2% 85% 33% 57% 38 Chapter 3: Actions of short-term training institutions in Moldova Students of training providers in Moldova are more likely to choose the sector of specialization based on their personal interest rather than information on job de- mand. Ninety percent of current students and over three-quarters of graduates in focus group discussions reported that they chose the training program out of personal interest. However, only half of the students and less than 40 percent of graduates reported that they made decisions based on information on jobs that are in high demand. Only 30 percent of the training providers reported having a strategy to involve employers or industry groups in institutional decision making. Among these 15 train- ing institutions with a strategy, some of the more popular strategies include calling em- ployers for casual conversations (60 percent), hosting events for employers to hear their views (60 percent), and inviting employers to participate in specific tasks, such as curricu- lum advice, instruction, or assessments (53 percent). In most cases, the institution’s direc- tor or another designated person oversees the implementation of such strategies. The employers’ focus group results suggested that most firms observed changes in the train- ing programs offered in their respective sector over the years, and close to 80 percent of the firms believed that they benefited from expressing their needs to training institutions. These positive insights from employers may indicate a need for training providers to better engage with employers or industry groups and develop a more demand-driven approach to training. Action 3: To establish a sustained relationship with authorities Governments play multiple roles in their quest for improving quality and relevance in the education and training programs. As regulators, they create policies and programs for training institutions to perform well. As enforcers, they monitor training institutions and ensure that their services fulfill at least a minimum set of quality standards. As national co- ordinators, they define the vision for the entire VET system and ensure that each training institution understands its role within that vision. Unfortunately, training institutions tend to see government authorities only in their role as enforcers. TAP looks at the existence, nature, and quality of the relationship between training institutions and governments, that is, whether they participate actively in policy dialogue or limit the interaction to reg- ulation compliance. Private short-term training institutions in Moldova tend to show limited level of engagement in policy dialogues and interactions with government authorities. Overall, more than 50 percent of training providers engage with the authorities, of which more than one-third report participation in events to discuss policies regarding training and skills development more than once a year (Figure 24). While close to 90 percent of public institutions engaged in dialogue with government officials, fewer than half the private entities attended such events. Among the training institutions that participate in policy dialogue, 63 percent said the main purpose of the events was to inform training Chapter 3: Actions of short-term training institutions in Moldova 39 institutions about a new regulation or to gather information from training institutions. In the past three years, less than quarter of the training institutions participated in any online public consultations about policies developed by central authorities. FIGURE 24. FREQUENCY OF TRAINING INSTITUTIONS’ PARTICIPATION IN GOVERNMENT EVENTS, N = 50 Total 34% 10% 10% 46% Not-for-profit 57% 43% Public 67% 11% 11% 11% Private 21% 12% 12% 56% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Yes, more than once a year Yes, once a year Yes, once every few years No While many training institutions generally have various communication chan- nels with government officials, their actual engagement is very limited, especially among private institutions. In more than half of the institutions, the director is respon- sible for addressing government communication or requests. Approximately two-thirds of the institutions reported that there is a designated staff for this task. Some of the main communication channels include addressing requests from government officials (30 per- cent), hosting visits from government officials (20 percent), and attending meetings with local government officials (18 percent) (Figure 25). In the last three years, however, 46 percent of the training institutions have not communicated with government officials. All public institutions have communicated with government officials at least once in the past three years, using various communication channels. In contrast, close to 60 percent of pri- vate and not-for-profit institutions have not communicated with government authorities in the last three years. While about one-third of training institutions do not undergo mandatory in- spections or financial auditing, the rest generally follow mechanisms for regulatory compliance and service improvements. Seventy percent of the training providers (35 out of 50) undergo mandatory inspection. Of these 35 institutions, 34 percent received inspections once a year, 26 percent every two to three years, another 34 percent every five years, and 2 percent received quarterly inspections. On average, these mandatory inspections last for about 15 hours. Most of the inspection reports recommend specific priorities to improve institutions (86 percent of the training providers that have manda- tory inspections), and institutions are required to submit an improvement plan following 40 Chapter 3: Actions of short-term training institutions in Moldova FIGURE 25. MAIN COMMUNICATION CHANNELS BETWEEN TRAINING PROVIDERS AND GOVERNMENT OFFICIALS, N = 50 (MULTIPLE RESPONSES) 60% 50% 40% 30% 20% 10% 0% Private Public Not-for-profit Total The institution has not communicated with government o cials in the last three years The institution hosted visits from government o cials at least once The institution addressed requests from government o cials at least once The institution received notifications from government o cials at least twice The institution attended meetings with local government o cials at least twice The institution attended online public consultations the inspection recommendations (71 percent). To ensure compliance with regulations, training institutions also use occasional review of news regarding regulations for train- ing providers (94 percent) and annual formal review of regulations and compliance (86 percent). Just over half of the institutions reported using a quick review of compliance in preparation for official visits. In total, 60 percent of training institutions undergo financial auditing (29 providers through internal reviews and 12 through external reviews). For 80 percent of these institutions, the last audit took place in the last three years (2016–2018). Action 4: To ensure institutional financial viability Regarding financial resources, TAP looks at the autonomy of training institutions to raise funds and their efforts to diversify their sources of revenue. It also studies financial man- agement practices such as the use of financial planning and operational budget. Most institutions have authority over the collection of financial resources, as well as the use and management of financial sources. Public as well as private institu- tions report that they have authority over the collection, use, and management of finan- cial sources. While not-for-profit training entities also have authority over the collection of financial resources, they have limited authority over the use and management of resourc- es. The most common approaches used by training providers to ensure financial stability are closely monitoring the payment of tuition and fees from students (66 percent) and complying with government regulations and requirements (54 percent). Chapter 3: Actions of short-term training institutions in Moldova 41 Not many training institutions receive contributions from government authori- ties, private donors, or employers, while some students receive scholarships from the government and employers. In the last three years, only one-fifth of the institutions received any in-kind or non-cash donations from government authorities or private do- nors. Among the eight institutions that provided an estimated value of these donations, the average was MDL 247,937 (US$14,620). The estimated value of donations ranges from MDL 1,000 (US$58.97) to more than MDL 1 million. None of the training institutions receive contributions from potential employers, but three training institutions reported that students received sponsorship from employers, with an average number of 149 stu- dents. Close to half of the students and a quarter of the graduates interviewed in focus groups also reported that a government program covered the cost of their training. In 2017, three public training institutions said their students received a scholarship—an av- erage of 598 students, of which 431 were male and 168 were female. Only one private training institution reported that their students received a voucher;11 in 2017, 1,000 stu- dents in this institution received an MDL 1,500 voucher. Almost all training institutions have an operational budget, with a majority pre- pared by the institutions themselves. Only 3 out of the 50 training providers do not have an operational budget. Among the 47 providers that have an operational budget, 62 percent of them said the budget was prepared by the institutions’ directors, followed by 36 percent prepared by the staff in charge of finance (Figure 26). One training institu- tion reported having an operational budget prepared by government authorities at the central level. Government authorities prepare the budget for public institutions only, and individual institutions’ directors seem to have a greater role in preparing the operational budget in private and not-for-profit entities. FIGURE 26. PERSONNEL PREPARING THE OPERATIONAL BUDGET, N = 50 Not-for-profit 0 60% 40% Public 11% 33% 56% Private 0 70% 30% Total 2% 62% 36% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Government authorities at the central level Institution's director Sta in charge of finance Many short-term training institutions can manage their funds and use program requests from students or stakeholders to make decisions on budget management. 11 A sum of money (not disbursed as cash to beneficiaries) that a prospective trainee receives to pay for training in an institution of his/her choice. 42 Chapter 3: Actions of short-term training institutions in Moldova Nearly 70 percent of the institutions have the flexibility to shift their budget and make changes without seeking external approval (Figure 27). Institutions have slightly less flex- ibility in reducing expenses, with 58 percent of the providers reporting that they have no constraint. When making decisions on managing funds, more than half of the institutions consider program requests from students or stakeholders as the most important factor. More than one-third of the training institutions consider what is most urgently needed in the institutions as the second most important factor. Fewer institutions use profitability criteria or student assessments to guide fund management decisions. FIGURE 27. FLEXIBILITY IN MANAGING FINANCIAL RESOURCES, N = 50 Find/increase revenue 64% 14% 16% 6% Make investments for the future 64% 16% 20% Reduce expenses 58% 18% 18% 6% Shift its budget 68% 14% 10% 8% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% No constraint Some constraint Heavy constraint Not able to at all Action 5: To fulfill national quality standards Governments set quality standards for training institutions to ensure that those who pur- sue VET are well prepared to find employment and thrive at work. These standards, which constitute a framework to guide the delivery of services, cover a wide array of aspects, from knowledge, attitudes, and competencies that students are expected to develop. TAP asks directors or decision makers of training institutions about the mechanisms they use to ensure compliance with national quality standards, the extent to which they use competency standards to deliver curricula, and whether they hold proof of fulfillment of national quality standards by way of an accreditation. While all public short-term training institutions conduct reviews to ensure com- pliance with national quality standards, many private or not-for-profit institutions do not follow this process. Nearly three-quarters of the institutions use annual inter- nal reviews to ensure the quality of education is on par with national quality standards. A quarter reported using annual external reviews, while one-fifth reported that they do not have such systemic mechanisms in place. This one-fifth of institutions are either pri- vate or not-for-profit institutions (six private and four not-for-profit). All public institutions Chapter 3: Actions of short-term training institutions in Moldova 43 conduct either annual internal or external review to fulfill national quality standards, while most private and not-for-profit entities rely on internal review. Short-term training programs use curricula and occupational standards based on competencies, determined mostly by institutions themselves or by a national framework. In programs with the highest enrollment in 2017, 80 percent of the insti- tutions reported that the curricula use occupational standards based on competencies. Over 40 percent of the training providers said that the occupational standards are in part determined by the institutions themselves or by a national framework of qualifications. However, only 20 percent of the providers determine these standards with a professional association for the corresponding occupation (Figure 28). All public institutions use com- petency standards to deliver the curricula, with a high percentage reporting that these were determined by a national framework of qualifications. More than half of the not- for-profit institutions do not use any occupational standards for the curricula. In private institutions, over 40 percent reported that their competency standards are determined by the institution. FIGURE 28. USE OF COMPETENCY STANDARDS FOR THE DELIVERY OF CURRICULA, N = 50 (MULTIPLE RESPONSES) 80% 70% 67% 60% 57% 50% 47% 43% 44% 42% 41% 40% 33% 30% 21% 22% 20% 20% 18% 20% 14% 14% 10% 0% Private Public Not-for-profit Total Yes, determined by the institution for this program Yes, determined by a professional association for these occupation Yes, determined by a national framework of qualifications No Only about half of the training institutions design, develop, or adapt their curric- ulum to fit industry association and employer standards and curricula requirements. About half of the institutions do not report following national standards as an approach to meet the standards of industry association, employers, and curricula requirements. Only 30 percent of the institutions use methods such as unique or emerging labor market demand supported by market research, enrollment trend in the institutions, and potential employers’ requests. Focus group discussions showed that only half of the employers believe their active participation in the design of curriculum at training institutions is a priority. 44 Chapter 3: Actions of short-term training institutions in Moldova Only one-fifth of the short-term training institutions have been accredited, and a quarter are currently in the process of being accredited. Among the 10 training providers that achieved accreditation, 6 institutions have been accredited sometime be- tween 2015 and 2017. Three of these institutions are public entities, six private, and one not-for-profit entities. All accredited training institutions, except one, are required to re- new their accreditation at least every five years. Action 6: To enable students to pursue education and training opportunities In rapidly changing working environments, the development of skills must extend beyond compulsory education to include skills upgrading through lifelong learning. Governments have a role to play in preparing policies and institutions that open up diverse pathways for skills acquisition at any age. Training institutions, in turn, must recognize individual competencies acquired before enrollment (either formally or informally), offer flexible ar- rangements to facilitate learning among those with competing commitments, and ensure that graduates can use acquired training as a stepping stone to pursue further education or training if they so wish. Less than half of the training institutions evaluate student proficiency before admission to the program. Out of the 50 surveyed training providers, 23 institutions evaluate student proficiency by looking at occupational competency standards. Over 60 percent of these competency standards are determined by the institution, followed by 26 percent determined by a national framework of qualifications, and 13 percent by a professional association for these occupations. Over 85 percent of the not-for-profit insti- tutions assess student proficiency based on occupational competency standards before admission, while only 44 percent of public entities and 50 percent of private entities do so. Short-term training institutions offer flexibility for students to complete the train- ing, mainly through flexible schedule arrangements. Seventy percent of the training providers offer classes in the morning, evening, or weekends (Figure 29). Intensive pro- grams (allowing students to finish faster than the normal track) are offered in half of the institutions. Remote training, credit building options (allowing students get degree by gradually building credit over an undetermined period of time), or online learning plat- forms are not offered as frequently. Generally, private or not-for-profit institutions have greater flexibility in offering different arrangements for training programs than public institutions do. Upon completion of the programs, a majority of the training institutions provide certification and standardized exams to students, but they do not necessarily offer nationally recognized certificates. In three-quarters of the institutions, a nationally rec- ognized certificate is awarded after completing the program. All public institutions offer certification, but one-quarter of private institutions and over 40 percent of not-for-profit Chapter 3: Actions of short-term training institutions in Moldova 45 FIGURE 29. FLEXIBLE ARRANGEMENTS PROVIDED BY TRAINING INSTITUTIONS, N = 50 (MULTIPLE RESPONSES) 80% 74% 70% 71% 70% 59% 60% 56% 50% 50% 43% 40% 29% 30% 22% 20% 16% 15% 14% 12% 11% 10% 10% 0 0% Total Private Public Not-for-profit Intensive program Morning/evening/weekend classes Remote training Credit building options institutions do not offer certification. Most of these training certificates (79 percent) are recognized by the Ministry of Education, Culture, and Research. Eighty percent of training institutions reported that they have standardized exams before offering students certifi- cation. Nearly half of the institutions have the autonomy to design and implement these exams. A third reported that they have the autonomy but must follow the guidelines pro- vided by the authorities, and 15 percent said they have the autonomy to implement, but the exams are designed centrally. In more than half of the institutions, these exams are reviewed only occasionally. In 90 percent of the cases, standardized exams evaluate stu- dent proficiency based on occupational competency standards. Responses from focus groups suggest that most students in Moldova plan to pursue further education or find a job after finishing the training. The highest per- centage of current students (46 percent) reported that they plan to pursue further edu- cation, followed by 40 percent who plan to find a job. A smaller proportion of 18 percent responded that they will continue in their current jobs, and just four students planned to create a business. Upon completion of the training, 40 percent of the graduates in focus groups pursued further education and 38 percent continued in their current jobs. One- fifth of the graduates created a business after graduating from the program. Action 7: To create a teaching/studying experience conducive to learning Perhaps the most crucial aspect of training quality has to do with the extent to which the experience that students and instructors have at training institutions enables learn- ing. From the perspective of training institutions, (a) curricula, (b) student management 46 Chapter 3: Actions of short-term training institutions in Moldova and feedback, and (c) instructor management play a direct role in achieving this goal. Regarding curricula, TAP asks training institutions how they decide to introduce, adapt, and close the programs. On student management, TAP enquires about the size of class- es and workshop sessions, how students spend their time, and if there is any chan- FIGURE 30. nel through which they can provide feed- AUTONOMY TO INTRODUCE/CLOSE PROGRAMS, N = 50 back to management. Regarding the instructor management, TAP gathers in- 100% 14% 9% formation on instructor evaluations and 90% 22% 29% 80% professional development opportunities. 70% 35% Most institutions have the autono- 42% 60% 29% my to introduce and close training pro- 50% 40% 78% grams. Eighty-six percent of the training 30% 56% providers can introduce and close pro- 20% 44% 43% grams at their own discretion, with about 10% half of them following criteria that was 0% 0 set by government authority (Figure 30). All public institutions with the autonomy to open or close programs follow criteria No set by government authority, while only Yes, following criteria set by government authority a third of the private and not-for-profit Yes institutions do so. Private training institu- tions have greater autonomy to introduce and close programs (91 percent), in compari- son to public (78 percent), or not-for-profit institutions (72 percent). Over the past three years, more than half of the institutions introduced new short-term training programs. A greater percentage of public providers newly intro- duced programs compared to private and not-for-profit institutions (Figure 31). Among providers that introduced new programs, most of their funding source came from in- house and less than 15 percent from public funding or private donations. A variety of cri- teria were used to introduce training programs: request from employers (63 percent), an- alytical findings (56 percent), and capacity for training provision (41 percent) were some of the more popular criteria used. Few institutions closed short-term training programs during the past three years. Only 11 out of 50 training institutions closed programs. A higher percentage of not-for- profit institutions closed programs in the past three years than other institutional types. Among the 11 institutions that closed programs, 8 training providers used findings from reviews of programs (such as enrollment trends) to determine whether to close the pro- grams. Four providers used labor market analyses and two held consultations with rel- evant stakeholders to determine the closure of short-term training programs. Only one institution used assessment of system-wide resource utilization and staffing as a criterion for deciding the program’s closure. Chapter 3: Actions of short-term training institutions in Moldova 47 Three-quarters of training institu- FIGURE 31. tions have designed or adapted cur- PERCENTAGE OF INSTITUTIONS THAT INTRODUCED/CLOSED ricula in the past three years, based PROGRAMS, N = 50 on various considerations. The highest 80% percentage of training institutions con- 70% 67% sider availability of teachers/instructors (43 percent) as one of the most import- 60% 54% 53% ant considerations during the design or 50% 43% adaptation of the curricula. Government 40% regulation (41 percent), existing curric- 29% 30% ula at the institution (35 percent), and 22% 21% 22% 20% employer needs (32 percent) were also some of the other considerations for 10% adapting or designing the curricula. 0% Short-term training programs in the areas of transport and storage or driving programs were the most pop- ular for both men and women in 2017. Introduced Closed Among the 50 surveyed training institu- tions, more than one-quarter reported that their programs in the area of transport and storage had the highest enrollment for male students. For female students, transport and storage, as well as education were equally popular. When asked to name the most pop- ular short-term program among male and female in 2017, more than 10 percent of the providers said ‘Driver, category B’. While all the institutions reported that all male students in that cohort graduated, few institutions reported some dropouts among female students. Almost three-quarters of students at the time of enrollment did not have work experience. The duration of popular programs, on average, was slightly less than three months. Students spent around 17 hours of study per week, of which nine were spent on theoretical instruction and eight were devoted to practical, more hands-on instruction. Training institutions reported that, on average, three-quarters of the short-term training programs include both practical and theoretical component equally. Over 80 percent of students and graduates in focus group discussions also thought the ratio between learning theory (classroom) and prac- tice (workshop) was well balanced and maximized the learning process. Moreover, all students and graduates perceive theoretical training as helpful as practical training when looking for a job. Most training institutions request feedback from students and graduates on the quality of the training services they provide. Nearly 50 percent of the institutions in- formally request feedback from students (such as sporadic meetings), and 44 percent request formally (for example, through evaluation forms or surveys) (Figure 32). There is not much difference among different types of institutions in how feedback is requested 48 Chapter 3: Actions of short-term training institutions in Moldova from students. Seventy percent of the FIGURE 32. training providers reported that they re- INSTITUTIONS THAT REQUEST FEEDBACK FROM STUDENTS, N = 50 quest feedback from graduates informal- ly, while just 20 percent request formal 100% 10% 9% 11% 14% feedback (Figure  33). All not-for-profit 90% entities collect feedback from gradu- 80% ates, with a higher percentage of formal 70% 44% 44% 44% 43% feedback than other institutional types. In 60% more than half of the training institutions, 50% no person is designated for the task of 40% receiving and addressing student com- 30% 46% 47% 44% 43% plaints, mainly because students rarely 20% file complaints. 10% Almost three-quarters of the train- 0% ing institutions reported that they evaluate the performance of instruc- tors, but close to half do not reward or address issues based on performance. Yes, informally Yes, formally No Nearly 40 percent of the providers eval- uate instructors quarterly, and one-fifth FIGURE 33. INSTITUTIONS THAT REQUEST do so annually. Among those who evalu- FEEDBACK FROM GRADUATES, N = 50 ate instructors’ performance, 76 percent 100% 0% of the institutions said formal evaluation 10% 12% 11% 90% process was critically important in the lat- 11% 80% 20% 18% 43% est performance evaluation. The evalua- 70% tion form filled out by students at the end 60% of the course was considered important 50% for almost 60 percent of the institutions. 40% 78% Forty-three percent also considered peer 70% 71% 30% 57% assessment as an important criterion for 20% instructor evaluation. Virtual and/or phys- 10% ical suggestion boxes were not deemed 0% important by half of the providers. Almost 50 percent of the institutions reported that they do not reward good perfor- mance of instructors, and close to 40 per- Yes, informally Yes, formally No cent do not address poor performance. In 2017, short-term training institu- tions provided opportunities for instructors to participate in professional develop- ment activities such as conferences or courses. Almost all of these professional devel- opment opportunities were a part of in-house training. Some of the popular professional Chapter 3: Actions of short-term training institutions in Moldova 49 development opportunities offered by institutions were participation in 1–2 weeks cours- es (64 percent) and conferences or forums that lasted 1–2 days (48 percent) (Figure 34). Industry-led, hands-on training and research assignments are rarely offered by short-term training providers. A high percentage of not-for-profit institutions provided opportunities for instructors to attend conferences or forums (83 percent). FIGURE 34. TYPES OF PROFESSIONAL DEVELOPMENT OPPORTUNITIES FOR INSTRUCTORS, N = 31 (MULTIPLE RESPONSES) 90% 83% 80% 75% 70% 67% 64% 59% 60% 48% 50% 50% 40% 35% 30% 30% 19% 20% 12% 12% 10% 3% 0 0 0 0% Total Private Public Not-for-profit Conferences/forum (1-2 days) Courses (1-2 weeks) Industry-led, hands-on training (1-2 weeks) Research assignments Action 8: To prepare students for the world of work The main objective of training institutions is to prepare students to lead independent and prosperous lives. Equipping them with a wide range of cognitive, technical, and socio- emotional skills is crucial but not enough. As students in training institutions are likely to be young and have little to no work experience, they face challenges to navigate through the world of work. To help achieve their goals, training institutions must provide support to students on ways to explore career options. TAP looks at whether training institutions have established partnerships with potential employers, a first step to bringing students closer to job opportunities. It also enquires about the prevalence of internship and ap- prenticeship requirements and the variety of support services for career development that training institutions offer to their students. Training providers receive limited inputs from potential employers in terms of direct investments in equipment as well as on-site training opportunities. Over a third of the institutions seek donations of equipment and supplies or potential job place- ment for students through these relationships. However, in developing interaction with 50 Chapter 3: Actions of short-term training institutions in Moldova employers, creating research and development (R&D) projects are not so important for almost half of the training providers. Even among the rest who reported that R&D proj- ects are important, less than 20 percent of institutions currently have R&D agreements with firms. On average, training institutions have one staff who spends 50 percent or more of his/her time on establishing and maintaining partnerships with employers. Focus groups with students and graduates suggested that over half of the students and grad- uates did not receive any training session at the facilities of a firm, and most have not visited employers as part of the training program. These results indicate that there may not be enough interactions between students and potential employers. Most students in the surveyed short-term training institutions do not participate in internship or apprenticeship, and not enough support is provided by training institutions in finding such opportunities. In 60 percent of the training providers, stu- dents are not expected to participate in internship or apprenticeship. Only one-fifth of the institutions have mandatory internship or apprenticeship requirement for all training programs. In fact, students did not engage in internship or apprenticeship commitments in nearly 80 percent of the training institutions. Providers also reported that, on average, only 18 percent of the students engaged in an internship or apprenticeship in 2017. The focus group discussions suggested that students and graduates perceive the importance of completing an internship or apprenticeship in developing skills in a work environment. In contrast, more than half of the institutions said they do not help students find opportu- nities for internships or employment. Similarly, there is insufficient resource dedicated to offering career counseling services to students. Only 44 percent of the training institutions provided career coun- seling to trainees in 2017, and 63 percent of these institutions that provided counseling reported that they do not have a career counselor among staff. Among those who have career counseling, a popular type of service offered is coaching sessions on professional development, led by instructors/teachers (68 percent). Mentoring sessions with alumni (45 percent) and orientation sessions (41 percent) are also some other ways in which institutions provide career counseling support. Students and graduates in focus group discussions said coaching by teachers and close contact with local public or private em- ployment agencies were also some ways in which training providers helped them find employment. However, over half of the graduates in focus groups reported not receiving any help. Furthermore, in about half of the training providers, less than 25 percent of the students used career counseling services. This points to insufficient support and use of career counseling services in training institutions. Action 9: To gather and publicize data for informed decision making Improvements in the quality and relevance of training rarely happen in the absence of information and data. The practice of collecting basic administrative data is common, but Chapter 3: Actions of short-term training institutions in Moldova 51 such type of data fails to provide a full picture of an institution’s performance. To complete diagnostics that are robust enough to inform changes, institutions must also collect data on student, teacher, and graduate performance, and client feedback. However, while col- lecting data is a necessary first step, it is not enough. Institutions must analyze and curate the data in such a way that they can provide accurate and user-friendly information to all interested parties. Indeed, training institutions must also make efforts to disseminate information and analyses internally, to discuss avenues for improvement, but they must disseminate information externally as well to inform decision making among prospective students, families, and employers. Most training providers collect a wide variety of data on institutional and pro- gram performance, with varying degrees of frequency, but they collect relatively less information on employment data. The highest percentage of data that training in- stitutions collect is administrative data (86 percent), followed by student performance (78 percent), graduation statistics (74 percent), trainee satisfaction (72 percent), and teacher/ instructor performance (68 percent) (Figure 35). There is a varying degree of frequency of data collection, but administrative data and student performance tend to be moni- tored quite regularly among training providers. In contrast, data on employment such as job placement rates (28 percent), earnings of graduates (6 percent), and employer sat- isfaction (30 percent) are not collected in many institutions. Generally, more than half of the training institutions still manage the collected data manually, while about 40 percent manage electronically. FIGURE 35. FREQUENCY OF DATA COLLECTION, N = 50 2% 100% 4% 10% 14% 90% 26% 26% 8% 4% 32% 80% 40% 40% 4% 10% 8% 70% 14% 60% 26% 8% 22% 50% 20% 18% 94% 20% 10% 6% 40% 10% 72% 70% 12% 12% 12% 30% 10% 20% 14% 32% 28% 22% 26% 10% 14% 0% Never Once a year Twice a year Every quarter Monthly 52 Chapter 3: Actions of short-term training institutions in Moldova Almost all training institutions use the collected data for internal discussions on institutional or program performance. Around 80 percent to 90 percent of the provid- ers use the collected data to formally set targets. Most institutions also use this informa- tion to assess or evaluate programs and monitor the attainment of targets. Slightly fewer institutions use data on job placement rates or earnings of graduates because data are not collected in these areas, yet a majority reported actively using the data to process and analyze the program performance. Among these institutions that use the information to assess programs or monitor targets, almost all providers share the results of these assess- ments. However, only three institutions reported that these data are publicly available and easy to access, and 41 institutions share the data only internally. In 87 percent of the training providers, internal meetings are held to discuss staff performance, based on the data collected. In the preceding year, close to 80 percent of the institutions said the main results of these meetings were that everybody became aware of the performance of the institution, and staff/instructors voiced their concerns and constraints. Half of the training institutions reported that the institutions learned about new policies or procedures or agreed on adjustments to policies or procedures. Perceptions of the performance of short-term training institutions TAP in Moldova collected data on the experience of students and graduates during their short-term training program and the training providers’ assistance to students and gradu- ates in finding employment. The extent to which training institutions catered to the needs of the students and graduates partly shows the quality of training and services provided by the training institutions. The employment status of graduates and their perception on the relevance of training they received to the current employment are also informa- tion that was collected in Moldova. Finally, employers were enquired to comment on the graduates’ skills, which offers an understanding of the skills students develop during training. Focus group discussions with students, graduates, and employers have yielded the findings presented in the following paragraphs. Most students and graduates were satisfied with the teachers and the well-bal- anced content of the training program. The average completion rate for the most pop- ular program in training institutions was 97 percent. In both focus groups, over 40 stu- dents and graduates responded that they most appreciate the teachers and the content of the training program. Students and graduates were also generally satisfied with the exposure to practical learning and the connections they made during the training, but a fewer number of students and graduates chose facilities as the most appreciated part of the training experience. Over 80 percent of the students and graduates commented that the average ratio between learning theory and practice in the training program was well balanced and maximized the learning process. Chapter 3: Actions of short-term training institutions in Moldova 53 A majority of the students and graduates did not expect training institutions to help them find employment, and such services were not offered in many institu- tions. Three-quarters of the students and all graduates in the focus group discussions disagreed with the statement “it is not my responsibility to find a job; it is the responsibil- ity of my training institutions.” Accordingly, most students and all graduates perceive job search as their own responsibilities. When asked about the ways in which training institu- tions provided help in finding employment, some of the more popular mechanisms were connecting them to an internship/apprenticeship that allowed them to find employment, coaching by teachers/instructors, and close contact with local public or private employ- ment agencies. However, even these ‘popular’ responses were low in number, with fewer than 10 training institutions offering such services, and more than half of the graduates reported that their institutions did not help at all. Seventy percent of the graduates in focus groups are currently employed. Once graduates completed the training, 41 percent of them pursued further education, fol- lowed by 39 percent that continued in their current employment, and 20 percent that created a business. TAP results also yielded that an average of 53 percent graduates pur- sue further education or training.12 From those who continued working with the same em- ployer, nearly 40 percent of the graduates reported that the training program changed their career prospects by helping them become better at their job. Among 41 respon- dents, 37 percent found their first employment before graduation, 24 percent did within six months of graduation, and 39 percent reported that they still have not found employ- ment in their field. Graduates found their employment through ways like application to a job posting, family connections, and connections made through the training institutions. From the TAP survey results, training institutions reported that an average of 91 percent graduates find a job within six months or create/continue working on their business.13 Seventy percent of the graduates felt that the training they received meets the needs of their field of expertise, and most employers in the focus groups agreed with this sentiment. Graduates responded that the training helped them learn new tech- niques and increase their productivity. More than 90 percent of the employers also be- lieved that their employees who have recently graduated from short-term training courses contribute to productivity improvement at their establishments. Close to three-quarters of the employers in the focus groups thought short-term training helps employees devel- op the skills they lack. Half of these employer respondents said they will likely encourage their employees to enroll in a training program, while the other half reported that they will provide the training themselves. 12 Data were only provided by 15 training providers. 13 Data were only provided by 12 training providers. 54 Chapter 3: Actions of short-term training institutions in Moldova Summary of key findings and policy implications TAP in Moldova describes the landscape and characteristics of short-term training pro- viders and the extent to which their service delivery mechanisms comply with internation- al best practices. The latter was assessed against nine key features of best institutional practices in training provision. This section synthesizes the main findings from this report by highlighting the strengths and weaknesses of short-term training institutions as well as the policy implications to help improve the future of workforce development. Strengths of short-term training institutions in Moldova Most short-term training institutions in Moldova set strategic directions by collect- ing data on student and instructor performance and using these to monitor the quality of their programs. A majority of the short-term training providers regularly gath- er information on student and instructor performance and use these for internal discus- sions on institutional or program performance. To proceed further, short-term training providers can mobilize these data to monitor progress toward the performance targets and make necessary adjustments to improve the quality of their training. Public short-term training institutions regularly collaborate with authorities by maintaining communication channels with government authorities and following mechanisms to comply with national quality standards. All public institutions report- ed that they have communicated with government officials at least once in the past three years, using various communication channels. These providers also conduct internal or external reviews to better comply with national quality standards and use competen- cy standards to deliver the curricula. A high proportion of public institutions reported that these competency standards were determined by a national framework of qualifica- tions. Moldova can further this effort by increasing the number of qualifications covered in the national framework and systematically update them in response to the changing skills needs. Many short-term training providers encourage students to continue studying further by offering flexible timetables and well-balanced content to meet students’ learning needs. Seventy percent of the short-term training institutions in Moldova of- fer classes in the morning, evening, or weekends. Such a flexible arrangement helps Summary of key findings and policy implications 55 accommodate students’ needs in attending and completing the program. A similar pro- portion of the short-term training providers also reported providing well-balanced the- oretical and practical elements in their programs. Most students and graduates in focus group discussions agreed that the well-balanced content maximized their learning ex- perience. These practices of short-term training institutions in Moldova can help create a teaching and studying experience conducive to learning. Areas of improvement for short-term training institutions in Moldova Short-term training providers would benefit from improving their capacity to make informed decisions by regularly collecting data on graduates’ training outcomes to improve the training programs. Most training providers in Moldova collect a variety of information such as administrative data, student performance, teacher/instructor per- formance, graduation statistics, and trainee satisfaction. However, data on training out- comes, including employment, job placement rates, graduates’ earnings, and employer satisfaction, are not collected in many institutions. Collecting these data will allow the training institutions to evaluate the relevance of their training programs in developing skills that are needed in the labor market. Results Based Financing, which is designed to provide financial incentives (for example, grants and tax deductions) to training providers based on compliance to clearly defined outcome indicator goals, provides one way to encourage them to make informed decisions through data collection. Moreover, devel- oping an Educational Management Informational System (EMIS) that allows for tracing students after graduation would help provide valuable information to training providers as well as policy makers. Estonia has an EMIS which is linked with its population register, tax board, unemployment, and health insurance data. Moldova may also benefit from expanding its EMIS by establishing links with other state registers and using the longitu- dinal data to advise training providers and improve policies. Short-term private and not-for-profit training providers would benefit from reg- ularly collaborating with authorities by maintaining communication channels with government authorities and following mechanisms to comply with national quality standards. These providers can also improve quality standards by conducting ex- ternal quality reviews to ensure that the quality of training is meeting national stan- dards. In comparison to public entities, private or not-for-profit training institutions have a limited engagement with government authorities. Close to 50 percent of private and not-for-profit institutions do not participate in government events, and over half of these types of providers have not communicated with government authorities in the recent three years. Most private and not-for-profit entities also rely on internal review to ensure compliance with national quality standards, and more than half of the private providers use competency standards that are determined by the institutions themselves. The newly 56 Summary of key findings and policy implications created sectoral committees may be a useful mechanism to ensure collaboration with government stakeholders. For the sectors where these structures have already been cre- ated, involved parties could work on strengthening links between the training providers and the government. For the sectors where these structures do not exist, the government may take a lead in their creation. Developing clear national quality standards for all professions and conducting ex- ternal reviews in all types of institutions may be useful in holding private and not-for- profit entities accountable for the quality of training. A large majority of 28 EU member countries include quality standards for VET at the national level. Among these countries, more than 85 percent use national quality standards as a condition for accreditation and approval of vocational education and training providers, and over 70 percent use nation- al quality standards as a condition of funding provision (European Training Foundation 2014). In a training system where training providers are governed by multiple ministries and the authorization, external evaluation and accreditation are made by the ANACEC, and the role of regulation and quality assurance mechanism becomes even more crucial to improve the performance of both private and public training providers. Public author- ities can pay better attention to the challenges training providers face in complying with quality standards and provide support to those in priority sectors, without the funding or mechanism to conduct external reviews. Short-term training providers would be able to better prepare students to the world of work if they have (a) stronger relationships with employers and/or indus- try associations and (b) more information and support to find internship/appren- ticeship opportunities as well as career services. Direct involvement of local employ- ers is among the most effective means to improve the relevance of training programs. Currently, training institutions in Moldova do not have enough participation from em- ployers or industry associations in setting performance targets or making institutional decisions. This is in part due to the absence of or relatively weak industry associations and sectoral committees in the country. The government can first facilitate the creation of associations and sectoral committees to strengthen their activities. It can also explore other mechanisms, such as tax incentives, to further encourage partnerships between local employers and training institutions. Establishing formal, or even informal relation- ships with training institutions can be helpful for the employers who can express their skill needs to the training providers. Training institutions can in turn benefit from these partnerships by better understanding the skills in demand and focusing on teaching the most relevant skills. All students and graduates in focus group discussions expressed the importance of completing an internship or apprenticeship in developing work-relevant skills. In contrast to this perception, more than half of the institutions reported that they did not expect students to participate in internship or apprenticeship and do not help students find such opportunities. Furthermore, fewer than half of the training institutions provided career counseling to trainees, and in about half of the training providers that provided career Summary of key findings and policy implications 57 counseling, less than 25 percent of the students used such services. This points to an in- sufficient provision and use of counseling services in training institutions. Short-term train- ing institutions can better prepare students for the world of work by actively looking for internship and apprenticeship opportunities and offering a variety of support services for career development. By partnering with employers and/or industry associations, training providers can help students find on-the-job placements, collaboration projects with em- ployers, mentoring sessions, and other professional learning experiences. A number of European countries have successfully developed a system to link training provision with employers and apprenticeship opportunities. One example is Cooperative Education ini- tiative in Serbia in which 15 secondary vocational schools were formally linked with over 25 companies (European Commission 2017). This cooperative arrangement not only al- lowed training providers to align skills development with demand but also expanded the opportunities for vocational students to engage in work-based learning supervised by trained mentors. 58 Summary of key findings and policy implications References • ANACEC (National Agency for Quality Assurance in Professional Education). 2018. “Press Release. Organization and Functioning of the National Agency for Quality Assurance in Education and Research (ANACEC).”http://anacip.md/index. php/en/comunicare-2/press-releases/307-press-release-organization-and-func- tioning-of-the-national-agency-for-quality-assurance-in-education-and-re- search-anacec. • CASEL (The Collaborative for Academic, Social, and Emotional Learning). 2018. “What is Sel?” https://casel.org/what-is-sel/. • European Commission. 2017. “Business Cooperating with Vocational Education and Training Providers for Quality Skills and Attractive Features.” http://ec.europa.eu/ social/BlobServlet?docId=18591&langId=en. • European Training Foundation. 2014. Quality Assurance in Vocational Education and Training a Collection of Articles https://www.etf.europa.eu/sites/default/ files/m/270970490A6E9327C1257CA800407038_Quality%20assurance%20in%20 VET.pdf. • ———. 2016. Republic of Moldova Education, Training and Employment Developments https://www.etf.europa.eu/webatt.nsf/0/158E8181CC7FB- 661C12580E60051544E/$file/Moldova%202016.pdf. • Guerra, N., K. Modecki, and W. Cunningham. 2014. “Developing Social-Emotional Skills for the Labor Market: The PRACTICE Model.” Policy Research Working Paper 7123, World Bank, Washington, DC. http://documents.worldbank.org/curated/ en/970131468326213915/pdf/WPS7123.pdf. • National Bureau of Statistics of the Republic of Moldova. http://www.statistica.md/ pageview.php?l=en&idc=350 • Republic of Moldova. 2014. “Approval of the Development Strategy of Education for the Years 2014-2020: Education 2020.” http://lex.justice.md/index.php?action=view &view=doc&lang=1&id=355494 • Rutkowski, J., V. Levin, and A. Bargu. 2017. “Missing Skills: Results of the Skills Module of the Moldovan Labor Market Forecast Survey.” World Bank, Washington, DC. From UNESCO Institute for Statistics (database). UNESCO, Montreal. http://uis. unesco.org/. References 59 • World Bank. 2013. SABER Country Report 2013: Moldova - Workforce Development. Washington, DC: World Bank. http://wbgfiles.worldbank.org/documents/hdn/ ed/saber/supporting_doc/CountryReports/WFD/SABER_WfD_Moldova_CR_ Final_2013.pdf. • ———. 2016. SABER Workforce Development. Washington, DC: World Bank. http:// wbgfiles.worldbank.org/documents/hdn/ed/saber/supporting_doc/brief/SABER_ WFD_Brief.pdf. • ———. 2017a. Assessment of Firm-Level Skills Demand and Engagement in Skills Development: Creating a Demand-led Skills Ecosystem in Moldova. Washington, DC: World Bank. http://documents.worldbank.org/curated/en/600671512368522632/ pdf/121811-WP-Moldova-private-sector-skills-Nov-2017-final-PUBLIC.pdf. • ———. 2017b. Country Partnership Framework for the Republic of Moldova for the Period FY18–21. Washington, DC: World Bank. http://documents.worldbank.org/ curated/en/604221499282920085/pdf/MOLDOVA-CPF-FY18-21-20170629- final-07052017.pdf. • ———. 2018a. Country Snapshot. Washington, DC: World Bank. http://pubdocs.world- bank.org/en/195121524127055800/Moldova-Snapshop-Spring2018.pdf. • ———. 2018b. Moldova Preschool and General Education: Transitioning to a Decentralized Service Delivery Model Selected Issues. Washington, DC: World Bank. http://documents.worldbank.org/curated/en/106571523893165988/pd- f/123660-REVISED-PUBLIC-P161676-MD-Decentralization-Analysis-final-for-image- bank-4-2018.pdf. • ———. 2018c. Moldova: Education Sector Public Expenditure Review Selected Issues. Washington, DC: World Bank. http://documents.worldbank.org/curated/ en/897081523894080457/pdf/125331-WP-P161676-MD-Mini-PER-final-for-image- bank4-2018.pdf • ———. 2018d. Special Topic: Higher Education and the Need to Improve its Relevance and Efficiency. Washington, DC: World Bank. http://pubdocs.worldbank.org/ en/793121526539367081/MD-HE-Special-Topic-FinalENG.pdf • World Development Indicators (database). 2016. Washington, DC: World Bank. 60 References Annexes Annex 1: Survey methodology and survey instruments Training Assessment Project in Moldova (TAPM) consists of mapping the training land- scape and applying quantitative and qualitative data collections tools for a more in-depth review of a selected sample of training providers. A survey of training providers was con- ducted, followed by a qualitative analysis of students,14 recent graduates,15 and employ- ers who engage (or would like to engage) with training providers. Section 1 covers the preparation and implementation of the survey; Section 2 describes the preparation and implementation of focus groups discussions. Survey methodology 1. Training provider survey Method Quantitative survey Interviewing CAPI at respondents’ work place technique Target group Selected short-term training providers during mapping stage In-depth structured questionnaire, provided by the World Bank and piloted Survey tool by CIVIS; working language was Romanian (26 respondents) and Russian (24 respondents) Sample size 50 training providers Stratified random sampling Stratification criteria used: (a) core-business (short-term training only, gener- Sample design al education and training, non-training related activities); (b) size by number of students (small, middle, large, and unknown) Length of 90 questions; average length of interviews was 136 minutes. interview 14 Students who are in their last term of training and on the verge of graduating, irrespective of the duration of the program in which the student is enrolled in. 15 Former students of the training institutions that complete the survey, who graduated within three years prior to the application of TAPM. Annexes 61 Method Quantitative survey Fieldwork period May 17–June 9, 2018 67 percent gross response rate (75 training providers have been contacted, Response rate out of which 7 turned out to be inactive by the time of the survey, which results in 74 percent net response rate) 1.1. Sample frame of training providers One of the most important components of this survey is designing a sample that best represents the training institutions in the country. The first step consists of building a sample frame of training providers, which yielded a mosaic of training providers. To gen- erate this mosaic, a preliminary analysis was conducted to identify the different types of formal16 training providers that operate in the country. TAPM focuses on institutions that provide short-term technical and vocational education and training. The sample frame in Moldova was developed using the ‘mapping tool’ provided by the World Bank and was adapted to the Moldovan context. The mapping tool collects not only the name of institutions, their contact information (that is, address/phone numbers), and in- stitutional type (for example, public, private, not-for-profit) but also characteristic variables. Statistical institutes and ministries associated with the operations and regulation of short-term training providers were consulted to consolidate a first sample frame. The generated list of training providers was verified using additional sources of information. The preparation of a usable sample frame for the mapping tool took about 2.5 months. 1.2. Quality assurance procedures for the sample frame To ensure quality, the sample frame of training providers was developed using readi- ly available public information from reliable sources and entities that are classified as part of the education sector. CIVIS contacted all potential short-term training providers through phone to confirm they are currently operating and that their contact information is accurate and also to collect additional information required to complete the mapping tool. Only training institutions that responded agreed to participate in the mapping exer- cise, provided accurate/reliable information was included in the final sample frame. 1.3. Sampling strategy After identifying the universe of training providers, a sample was selected to conduct in-depth interviews and gather information on participants for the focus groups. The planned size of the sample for Moldova was 50 training institutions. This sample frame of short-term training providers is based on information identified during the mapping 16 TAPM does not cover informal provision of training. It is limited to the assessment of formal training providers that are registered as such in various registers of established entities in the country. 62 Annexes stage. The mapping survey first provided basic information on 272 short-term training providers, which was used for the sample design. Besides these entities, there were (a) 111 potential training providers that did not want to participate in phone surveys during the mapping stage, were asked to provide online link with questionnaire, but finally did not respond and (b) 29 entities that refused from the beginning to participate in the mapping survey. In total, these two groups ac- count for 140 potential respondents. Yet, an unknown number of them may not actually provide short-term trainings. The survey used stratified random sampling with multiple stages of sampling. Stratification criteria mainly included core business (short-term training only, general ed- ucation and training, non-training related activities) and size of institutions by number of students (small, medium, large, and unknown). The principle used for grouping is the following: using the average number of students in the landscape survey (468 students), small-size institutions have up to 235 students (half of the average), medium-size institu- tions have between 235 and 700 students (lower half and upper half of the average), and large-size have more than 700 students. 2. Focus groups with students, graduates, and employers Method Qualitative study, based on focus group discussions Interviewing technique Focus group discussions Students (soon-to-be graduates), graduates, employers Target group Participants reflect different socio-demographic, geographic, and training profiles Screening questionnaire, as well as focus group guides provided by Survey tool the World Bank 17 groups in total with 127 participants • 6 student groups, 50 participants (out of 81 invited); average of 8.3 participants per group Sample size • 7 graduate groups, 50 participants (out of 81 invited); average of 7.1 participants per group • 4 employer groups, 27 participants (out of 56 invited); average of 6.8 participants per group Participants were selected from contacts provided by the 50 train- Sampling profile ing providers that participated in the in-depth survey. Length of discussion About 2 hours Fieldwork period June 2–19, 2018 2.1. Selection of students, graduates, and employers Training institutions were asked to provide two lists—one of students on the verge of graduating from different programs and another of graduates who finished their stud- ies within the three years before participating in TAP (either employed or unemployed). CIVIS merged the lists provided by approximately 10 institutions and from the compiled Annexes 63 list, it selected around 10 students and 10 graduates to participate in the first two focus groups. The process was repeated with the remaining 40 institutions to identify the par- ticipants of additional focus group discussions. The selection was done randomly and the list of graduates includes both employed and unemployed individuals. The group of employers corresponds to the institutions that have partnerships with surveyed training providers or with whom these providers would like to establish part- nerships. As such, the employer list includes institutions that either hire or provide intern- ships to the graduates of surveyed training institutions (or collaborate with them in any other way) and others that do not. The survey team asked each surveyed training pro- vider to provide names and contact information of 10 employers (according to criteria mentioned previously). However, between 3 and 4 employer names per institution were identified, as some training providers were not able to identify 10 employers and some employers were listed by more than one provider. CIVIS merged the lists provided by approximately 10 institutions and carried out a random selection of employers to partic- ipate in the first focus group discussion. The process was repeated for the remaining 40 institutions to identify the participants of the remaining four focus groups. Survey and focus group instruments 1. Training Provider Questionnaire (CAPI) The training provider questionnaire is a fundamental part of TAP as it is designed to gath- er information on the inputs, practices, outputs, and outcomes of training institutions as expressed by themselves. The literature17 points that, although traditional assessments of quality are based on the inputs and outputs of education and training institutions, this type of data falls short in providing a full picture of the quality of the services that these institu- tions provide. Acknowledging this shortcoming, an important section of the training pro- vider questionnaire is dedicated to exploring further characteristics, institutional actions, and values that can also have an effect on the quality of education and training services. The training provider questionnaire is structured in five sections: (i) Background Information, which includes basic identifiers, such as name, address, institution type; (ii) Inputs, which covers basic characteristics of students, instructors, facilities, and funding; (iii) Institutional Actions, which examines the practices of training institutions using the SABER-WfD (Policy Intent) conceptual framework; (iv) Institutional Values, which explores the extent to which respondents agree or disagree with statements about the institutions’ role or mandate to undertake certain actions; and (v) Outcomes, which aims to collect data on the employment status, income, and educational status of graduates. 17 See Renaud, Robert, “Measuring Educational Quality in TVET” In: Maclean and Wilson (Eds.) (2009), “International Handbook of Education for the Changing World of Work, Bridging Academic and Vocational Learning.” Springer, Netherland. 64 Annexes The standard training provider questionnaire, designed to be applied in different countries, was slightly adjusted to the Moldovan context. At the beginning of the ex- ercise, a thorough review of the questionnaire was conducted to ensure that all ques- tions are relevant and appropriate to the general characteristics of the Moldovan’s train- ing system. After the English version of the questionnaire was reviewed and adapted, a Paper-Assisted Personal Interview (PAPI) was designed in English and translated by qualified translators into local languages, namely Romanian and Russian. Once the PAPI was finalized, a CAPI tool was developed to match the PAPI English version of the ques- tionnaire. The CAPI version used during the survey contains flags to alert interviewers of any apparent inconsistency in responses and allow corrections to be made accordingly, which would improve the overall quality of the data set in a systematiic manner. The CAPI version also contains all necessary skip patterns, legal value checking, and basic logic checks to verify consistency. 2. Training questionnaire and interviewer guide In addition to the questionnaire, a training guide (to be provided by World Bank) is gen- erated to prepare interviewers and focus groups moderators. The training guide should include the following: • An explanation of the general structure of the questionnaire • General principles and proper conduct expected • An overview of psychological factors that influence the respondent and the inter- viewer in face-to-face interviews • An annotated training questionnaire where the most common difficulties will be identified and addressed, including the provision of hypothetical examples • A glossary of key terms used in the questionnaire The training material has basic quality control (QC) instructions/procedures that should be carried out, whenever possible at the end of each day on exported data taken from CAPI. Both the training materials on the questionnaire and the interviewer guide could be adapted to changes made in the training provider questionnaire. All tools were translated into Romanian, and interviewers had the option to use the tools in both English and the locally spoken language. 3. Quality control procedures The QC procedures correspond to a number of proactive activities designed to control the data collection process and to ensure the highest quality possible of the data set, meaning the highest accuracy of responses. The QCs were designed right after the com- pletion the questionnaire for training providers. These QC procedures can be presented in the following levels: • Level 1: List of all legal values for all variables in the questionnaire, list of all legal skips, and reverse skips (that should be disallowed), completion levels of individual Annexes 65 variables (those that must be filled during interviews, and those for which item nonresponse can be tolerated), list of all variables that are interrelated in strict dependency (for instance, totals that have to yield 100 percent, ‘parts’ that have to be smaller than ‘wholes’ (such as ‘how many students enrolled, of which how many ‘female’, and so on). These tests were introduced directly in the CAPI, but interview- ers were made aware of their existence to be able to smoothly use the CAPI during interviews and advise respondents of any contradiction. • Level 2: List of all semi-strict tests that require flags—these alerts were introduced in the CAPI. • Level 3: Post data entry controls using meta-data if and when they apply (controlling for systematic mistakes coming from the same interviewers, the differences in the duration of interviews, and so on) 4. Focus group guides, questionnaire, and discussion agenda To assess training provider performance from the perspective of students, graduates, and employers, focus group guides have been prepared by the World Bank Group team. These guides cover the following topics: • Screener introduction: Project summary, firm presentation, objectives, and impor- tance of participation in the survey to motivate cooperation of selected respondents • Invitations to participate and validation: Verification that participants meet the criteria to take part in the focus groups, description of session procedures, detailed manner of participation, rules and conditions for receiving attendance incentives and requirements • Questions to determine the conceived notions about the quality of the services pro- vided by training institutions from the perspective of targeted participants • Discussion agenda: Order and roles in the sessions, detailed outlines of the dis- cussions, special techniques or procedures, method and stimulus materials to be presented in the sessions, possible concerns, and proposed solutions to handle the sessions Project team for survey The team for the survey were composed of the following individuals: • Project manager • Project coordinator • Quantitative survey coordinator • Software and data processing manager • Senior-level interviewers (5) 66 Annexes Annex 2: Summary of results from focus group discussions with students, graduates, and employers Basic information on FG participants Students Graduates Employers Sample 50 50 27 Gender Male 28 11 12 Female 22 39 15 15–25 years 26 20 3 Age group 26–40 years 16 13 10 41 years and older 8 17 14 Employed 16 39 Employment Unemployed 23 7 Status Student 11 4 Urban 46 46 27 Type of locality Rural 4 4 0 Driver 6 5 Language courses 17 9 Makeup and cosmetology 5 1 HoReCa 3 2 Specialists in electricity, gas, machinery 10 6 Accounting and audit 7 Field of study Teaching 5 Clothes manufacturing 1 7 Safety and security 1 Massage 3 3 Information technology programming 3 4 and design Other 1 1 Director 9 Manager 4 Job position Head of department 4 Main specialist 6 Other 4 Annexes 67 1. Students 1.1. Selection of training provider Why did you decide to enroll in a training program? (N = 60)18 To find employment 22 37% To improve work conditions in current job (promotion, better salary) 10 17% To change field of work 6 10% To change jobs within the same field of work 1 2% Other 21 35% Total 60 100% How did you decide on what type of training institution you attend? (N = 136) Programs offered 26 19% Friends/family attending training institutions 23 17% Location 23 17% Finances 21 15% Reputation 19 14% Admissibility 11 8% Other 13 10% Total 136 100% So far do your expectations meet your considerations (in choosing the training institution you are attending)? (N = 47) Yes 26 55% No 3 6% Don’t know 18 38% Total 47 100% How did you decide which sector to focus your specialization? (N = 85) Personal interest in field 45 53% Received information that jobs in this industry are in high demand 25 29% Family pressure 8 9% Other 7 8% Total 85 100% 18 N greater than sample size (50 in the case of students) points to multiple responses by focus group participants. 68 Annexes Who has covered the cost of your training? (N = 49) A government program 22 45% Myself 17 35% My employer 7 14% A family member 3 6% Total 49 100% Were you offered a scholarship/voucher? (N = 50) Scholarship 18 36% Voucher 2 4% No 30 60% Total 50 100% 1.2. Plans following graduation What are your plans once you complete this program? (N = 61) Pursue further education 23 38% Find a job 20 33% Continue in your current job 9 15% Create a business 4 7% Other 4 7% Total 61 100% Please explain how you think you will get employment (N = 27) Application to a job posting 10 37% Connections made through your training institution 7 26% Public employment offices 7 26% Through family connections 3 11% Total 27 100% What type of employment will you seek? (N = 50) Full-time 18 36% Permanent 15 30% Self-employment 8 16% Part-time 6 12% Casual or seasonal 3 6% Total 50 100% Annexes 69 1.3. Assistance from training providers to find employment If you plan to find employment/change jobs, is the training institution providing you with assistance to achieve this goal? How are they doing so? (N = 35) Connected me to an internship/apprenticeship that allowed me to find 10 29% employment Coaching by teachers/instructors 7 20% Close contact with local public or private employment agencies 7 20% Orientation on options at the enrolment stage 4 11% Career center available on campus channeling employers hiring requests 2 6% Dedicated counselor available on campus 1 3% Hosting of luncheons (or breakfasts) with potential recruiters 1 3% Other 3 9% Total 35 100% During training, do you work in a research and development lab at your institution’s facil- ities? (N = 49) Yes 41 84% No 8 16% Total 49 100% Have you received any training session at the facilities of a firm/enterprise? (N = 50) Often, at least 50% of sessions 11 22% Sometimes, about one in four sessions 11 22% No 28 56% Total 50 100% If no, have you visited any employers as part of your training program? (N = 18) Yes 2 11% No 16 89% Total 18 100% 1.4. Experience during training Thinking of the average ratio between learning theory (classroom) and practice (work- shop) in your program, how would you characterize this ratio in terms of providing you with the skills set you required to find employment in your field? (N = 50) The ratio was well balanced and maximized the learning process 42 84% There was too much focus on theory 6 12% Program was too practical, did not learn enough theory behind practice 1 2% The ratio was well balanced but what we learned in practice did not reflect 1 2% the skills required to find employment in my field Total 50 100% 70 Annexes Do you feel satisfied with the general experience during the training? If yes, what do you appreciate the most? (N = 164) The teachers 45 27% The content of your training program 40 24% The connections that I made 31 19% The exposure to practical learning 29 18% The facilities 19 12% Total 164 100% 1.5. Students’ perception Do you agree/disagree with the following statements? (N = 50) Agree Disagree Training institutions should provide access to accurate information on labor market to students (i.e. to know what programs are either 47 94% 3 6% the most sought out by employers and/or would lead to a more competitive wage) It is mostly up to employers to approach students and graduates 25 50% 25 50% with job offers My ability to find employment is mostly limited to the services pro- 2 4% 28 96% vided to me by my training institution Completing an internship or apprenticeship is important to develop 45 90% 5 10% skills in a work environment My active participation in discussion pertaining to curriculum and 36 72% 14 28% program development at my training institution is very important It is not my responsibility to find a job, it is the responsibility of my 15 30% 35 70% training institution It is a shared responsibility between the teaching institution and 38 76% 12 24% employers to offer jobs to students and graduates Theoretical training is not essential when looking for a job 0 0% 50 100% The higher the reputation of the training institution the easier it is to 31 62% 19 38% find a job Training providers should use technologies that are similar to those 50 100% 0 0% used in enterprises 2. Graduates 2.1. Selection of training provider Why did you decide to enroll in a training program? (N = 64) To find employment 16 25% To change field of work 12 19% To improve work conditions in current job (promotion, better salary) 10 16% Annexes 71 To change jobs within the same field of work 3 5% To take advantage of opportunity offered by current employer 1 2% Other 22 34% Total 64 100% Which factors did you take into consideration when choosing the training institution attended? (N = 112) Location 22 20% Finances 19 17% Reputation 17 15% Programs offered 16 14% Friends/family attending training institutions 14 13% Admissibility 10 9% Other 14 13% Total 112 100% Please specify if your considerations were met. (N = 50) Yes 48 96% No 2 4% Total 50 100% What made you decide on the sector of specialization you chose? (N = 90) Personal interest in field 38 42% Professional experience 20 22% specify Received information that jobs in this industry are in high demand (­ 19 21% who provided information) Family pressure 7 8% Other 6 7% Total 90 100% Who has covered the cost of your training? (N = 51) Myself 25 49% A government program 12 24% A family member 9 18% My employer 5 10% Total 51 100% Were you offered a scholarship/voucher? (N = 50) Scholarship 12 24% Voucher 7 14% No 31 62% Total 50 100% 72 Annexes 2.2. Employment status and process Once you completed the training, did you _____? (N = 49) Pursue further education 20 41% Continue in your current job 19 39% Create a business 10 20% Total 49 100% If you continued working with the same employer, how do you think that participating in this training has changed your career prospects? (N = 28) I am better at my job 11 39% I was able to change my area of work 5 18% I received a raise 4 14% I received a promotion 3 11% I was able to switch jobs 2 7% Other 3 11% Total 28 100% Are you currently employed? (N = 50) Yes 35 70% No 15 30% Total 50 100% If no, what are the main challenges preventing you from obtaining employment? (N = 12) No jobs in the chosen field 0 0% Jobs in field are mostly temporary in nature 0 0% Don’t know how to contact employers 0 0% There are too many people with the same training/skills as me (i.e. too 1 8% much competition) Never made it past interview 0 0% Other 11 92% Total 12 100% After graduating, how long did it take for you to find your first employment related to the field you studied? (N = 41) Before graduation 15 37% Within 6 months of graduation 10 24% I still have not found employment in my field 16 39% Total 41 100% Did you keep your first employment for at least 1 year? (N = 16) Yes 9 56% No 7 44% Total 16 100% Annexes 73 Did you obtain the type of employment that you wanted upon graduation? (N = 35) Yes 23 66% No 12 34% Total 35 100% If yes, what type of employment did you get? (N = 23) Full-time 14 61% Part-time 4 17% Self-employed 5 22% Total 35 100% How did you find employment upon graduation? (N = 27) Application to a job posting 7 26% Through family connections 5 19% Connections made through your training institution 5 19% I continued working in the same firm 5 19% Other 5 19% Total 27 100% 2.3. Assistance from training providers to find employment To your recollection, did the institution help you find employment? If so, how? (N = 56) Did not help at all 29 52% Coaching by teachers/instructors 13 23% Close contact with local public or private employment agencies 7 13% Connected me to an internship/apprenticeship that allowed me to find employment 4 7% Orientation on options at the enrolment stage 1 2% Mentoring by former students; graduates; alumni 1 2% Career center available on campus channeling employers hiring requests 1 2% Total 56 100% During training, did you work in a research and development lab at your institution’s facilities? (N = 50) Yes 49 98% No 1 2% Total 50 100% During your course, did you receive any training session at the facilities of a firm/enter- prise? (N = 50) Often, at least 50% of sessions 14 28% Sometimes, about one in four sessions 4 8% No 32 64% Total 50 100% 74 Annexes If no, have you visited any employers as part of your training program? (N = 31) Yes 4 13% No 27 87% Total 31 100% 2.4. Experience during training Thinking of the average ratio between learning theory (classroom) and practice (work- shop) in your program, how would you characterize this ratio in terms of providing you with the skills set you required to find employment in your field? (N = 51) The ratio was well balanced and maximized the learning process 42 82% There was too much focus on theory 7 14% Program was too practical, did not learn enough theory behind practice 2 4% The ratio was well balanced but what we learned in practice did not reflect 0 0% the skills required to find employment in my field Total 51 100% Do you feel satisfied with the general experience during the training? If yes what do you appreciate the most? (N = 195) The content of your training program 45 23% The teachers 45 23% The exposure to practical learning 38 19% The connections that I made 36 18% The facilities 31 16% Total 195 100% 2.5 Relevance of training to current employment Was the salary offered to you post-graduation greater than your remuneration prior to your studies at the training institution? (N = 18) Yes 9 50% No 9 50% Total 18 100% If no, why not? (N =12) The remuneration in this field is low 3 25% My employer does not believe the training leads to a raise 5 42% Other 4 33% Total 12 100% Annexes 75 Did the training you receive allow you to meet the needs of the sector in your field of expertise? (N = 50) Yes 35 70% No 15 30% Total 18 100% If yes, please explain how so. (N = 59) I produce more and faster 13 22% I learned new techniques 33 56% I have taught my coworkers what I learned to improve their productivity 9 15% I caught up with technology that the firm recently acquired 3 5% Other 1 2% Total 59 100% If no, please explain why not. (N = 9) I have not used the skills I learned in my current job 1 11% I have not found employment 2 22% Other 6 67% Total 9 100% 2.6. Graduates’ perception Do you agree/disagree with the following statements? (N = Agree Disagree 50) Training institutions should provide access to accurate information on labor market to students (i.e. to know what programs are either 49 98% 1 2% the most sought out by employers and/or would lead to a more competitive wage) It is mostly up to employers to approach students and graduates 22 44% 28 56% with job offers My ability to find employment is mostly limited to the services pro- 8 16% 42 84% vided to me by my training institution Completing an internship or apprenticeship is important to devel- 50 100% 0 0% op skills in a work environment My grades are not an important factor in terms of increasing my 2 4% 48 96% odds of finding a good employment My active participation in discussion pertaining to curriculum and 45 90% 5 10% program development at my training institution is very important It is not my responsibility to find a job, it is the responsibility of my 0 0% 50 100% training institution It is a shared responsibility between the teaching institution and 46 92% 4 8% employers to offer jobs to students and graduates Theoretical training is not essential when looking for a job 0 0% 50 100% 76 Annexes Do you agree/disagree with the following statements? (N = Agree Disagree 50) The higher the reputation of the training institution the easier it is 35 70% 15 30% to find a job Training providers should use technologies that are similar to those 50 100% 0 0% used in enterprises 3. Employers 3.1. Recruitment process Does your company offer financial support for workers who enroll in training relevant to their positions? (N = 27) Yes 25 93% No 2 7% Total 27 100% Does your company offer flexibility for workers who enroll in training relevant to their positions? (N = 27) Yes 25 93% No 2 7% Total 27 100% Does your company incentivize employers to enroll in training programs? (N = 27) Yes 20 74% No 7 26% Total 27 100% In your recruitment process, do you offer preference to candidates who have recently completed short-term training? (N = 27) No 14 52% Yes, for some positions 10 37% Yes, for most positions 3 11% Total 27 100% In your recruitment process, do you offer preference to candidates who have completed short-term training at a particular institution? (N = 27) Yes 4 15% No 23 85% Total 27 100% Annexes 77 3.2. Perception of graduates’ skills Do you believe that employees who have recently completed short-term training courses contribute to productivity improvement at your establishment? (N = 27) Yes 25 93% No 2 7% Total 27 100% If no, why not? (N = 3) They do not have the right competencies 1 33% They generally do not keep their employment long enough 1 33% I don’t know - that is difficult to estimate 1 33% Total 3 100% Do you think that short-term training can help these employees develop the skills they lack? (N = 27) Yes 20 74% No 7 26% Total 27 100% If no, why not? (N = 7) Training institutions offer poor quality training 2 29% Employees are more likely to learn those skills while working in a classroom 2 29% Other 3 43% Total 7 100% If yes, is your company likely to encourage its employees to enroll in a training program? (N = 20) No, we provide them with training ourselves 9 45% Yes, but to do it on their free time and cover the cost 1 5% Yes 10 50% Total 20 100% 3.3. Partnerships with training institutions Do you adopt a specific formal collaboration process with training institution(s) to en- sure that current and potential employees have the skill set required to meet your firm’s needs? (N = 27) Yes 19 70% No 8 30% Total 27 100% 78 Annexes If yes, how best would you describe the process? (N = 24) It is straightforward 18 75% It is somewhat complicated 0 0% Other 6 25% Total 24 100% If no, for what reason? (N = 8) Do not trust training institutions to produce highly skilled persons 1 13% We have never thought of doing this 1 13% It is too complicated 0 0% Other 6 75% Total 8 100% Among the following, please choose the type of links between firms and training institu- tions that your establishment views as best. (N = 79) Offer internships/apprenticeships to students from training institutions 18 23% Mentor graduates or students 14 18% Have a formal planning of contact with training institution 11 14% Create special programs at training institutions 9 11% Participation to colloquiums or forums 6 8% Participation in designing courses/programs 6 8% Publication of technical articles or books by teachers / instructors 4 5% Development of networks of peers across training institutions 4 5% Sponsor scholarships 3 4% Be part of peer reviews 2 3% Create common R&D projects with training institutions 1 1% Support time allocation for research 1 1% Offer awards to reward excellence/success 0 0% Total 79 100% Do you host or participate chamber of commerce/sector association activities to establish contact that will help your firm maintain its needs in terms of human capital develop- ment? (N = 27) Yes 10 37% No 17 63% Total 27 100% Annexes 79 3.4. Employers’ perception Do you agree/disagree with the following statements? (N = Agree Disagree 27) The firm has not observed changes over the years in the training 3 11% 24 89% programs offered directed to firm’ sector of activity The firm does not benefit from expressing their needs to training 5 19% 22 81% institutions Active participation in the design of curriculum at training institu- 14 52% 13 48% tions is not a priority Active participation in the design of programs at training institu- 17 63% 10 37% tions is a priority It is important for the firm to offer internships or apprenticeships to ensure a better fit between the firm’s needs and the training 17 63% 10 37% offered It is primarily the responsibility of graduates of training institutions 20 74% 7 26% to apply for employment at this establishment The firm hires graduates from high school and it provides them 7 26% 20 74% with on-the-job training The firm finds it beneficial to participate in job fairs hosted by train- 14 52% 13 48% ing institution The firm believes in the process of being involved in instructor 11 41% 16 59% training programs at training institutions The firm does not consider participating in mentor training pro- 10 37% 17 63% grams at training institutions Recent graduates from training institutions are not good 17 63% 10 37% employees Recent graduates from training institutions do not know how to work, the firms have to spend a lot of resources to render them 18 67% 9 33% productive Training providers should use technologies that are similar to those 26 96% 1 4% used in enterprises 80 Annexes © 2019 International Bank for Reconstruction and Development/ The World Bank 1818 H Street NW Washington, DC 20433 Phone 202-473-1000 Internet: www.worldbank.org This volume is a product of the staff of the International Bank for Reconstruction and Development/ The World Bank with external contributions. The findings, interpretations and conclusions expressed in this work do not necessarily reflect the view of the World Bank Group, its Board of Directors or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Copyright Statement: The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The International Bank for Reconstruction and Development/ The World Bank encourages dissemination of its work and will normally grant permission to reproduce portions of the work promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA, telephone 978-750-8400, fax 978-750-4470, http://www.copyright.com/. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA, fax 202‑522-2422, e-mail pubrights@worldbank.org.