Armenia Skills toward Employment and Productivity (STEP) Survey Findings (Urban Areas) January 31, 2015 Education Global Practice Europe and Central Asia Region This report was prepared by Alexandria Valerio, Katia Herrera-Sosa, Sebastian Monroy-Taborda, and Dandan Chen. The analyses on employers’ demands for skills draw extensively from the technical reports prepared by Jan Rutkowski using the STEP Employer Surveys in Armenia, and from descriptive analyses prepared by Olga Kupets on the STEP household surveys in Armenia. Martin Moreno, Michelle Loui, and Olga Kupets provided overall research and analytical support. Meskerem Mulatu and Dandan Chen are Task Team Leaders. The team appreciates the strategic leadership and technical support of Alberto Rodriguez (Country Director for Bolivia, Chile, Ecuador, Peru and Venezuela), Andrea Guedes (Acting Sector Manager, Eastern Europe and Central Asia Human Development), and Cristian Aedo (Acting Practice Manger, Education Global Practice) of the World Bank. Helpful peer review and general comments were provided by the following colleagues: Christian Bodewig; Mohamed Ihsan Ajwad, Nistha Sinha, and Vigen Sargsyan. Maria Laura Sanchez Puerta and Tania Rajadel provided useful technical insights. The team is thankful for the overall assistance received from Marie Madeleine Ndaw, Sujani Eli, and Norosoa Andrianarivo. The research for and production of this report received financial support from the Government of Austria through Externally-Financed Output (EFO) for “Promoting Employment through Skills Development”. 2 Table of contents TABLE OF CONTENTS ................................................................................................................................................... 3 EXECUTIVE SUMMARY ................................................................................................................................................. 7 CONTEXT ..................................................................................................................................................................... 7 KEY FINDINGS .............................................................................................................................................................. 8 FUTURE POLICY AND RESEARCH AGENDA ......................................................................................................................... 11 1. ABOUT THE REPORT ............................................................................................................................................... 13 CONTEXT AND OBJECTIVE ............................................................................................................................................. 14 CONCEPTUAL FRAMEWORK AND RESEARCH QUESTIONS ................................................................................................... 16 DATA, TYPE AND DEFINITION OF SKILLS MEASURED, AND METHODOLOGY ........................................................................... 19 2. LABOR MARKET AND DEMAND FOR SKILLS IN ARMENIA ..................................................................................24 UNEMPLOYMENT AND LOW PRODUCTIVITY EMPLOYMENT ................................................................................................ 24 SKILLS IN DEMAND ...................................................................................................................................................... 25 3. STEP 1 | GETTING CHILDREN OFF TO THE RIGHT START IN ARMENIA ................................................................ 31 HAS PARTICIPATION IN EARLY CHILDHOOD EDUCATION PROGRAMS GAINED GROUND OVER TIME? .......................................... 31 IS PARTICIPATION IN EARLY CHILDHOOD EDUCATION PAYING OFF?...................................................................................... 33 HIGHLIGHTS OF FINDINGS .............................................................................................................................................36 4. STEP 2 ǀ ENSURING THAT ALL STUDENTS LEARN IN ARMENIA .......................................................................... 37 IS EDUCATIONAL ATTAINMENT TRANSLATING INTO HIGH QUALITY AND JOB-RELEVANT SKILLS? ............................................... 37 WHAT IS THE ROLE OF PAST SOCIOECONOMIC STATUS IN SKILL ACQUISITION? ......................................................................47 DO EARLY SKILLS GAPS TRANSLATE INTO CURRENT SKILLS GAPS? ...................................................................................... 48 HOW DO PARENTAL EDUCATION AND HOUSEHOLD RESILIENCE TO SHOCKS AFFECT SKILLS OUTCOMES? ................................... 51 HIGHLIGHTS OF FINDINGS ............................................................................................................................................. 53 5. STEP 3 | BUILDING JOB-RELEVANT SKILLS IN ARMENIA......................................................................................54 TO WHAT EXTENT CAN SKILLS HELP EXPLAIN THE LABOR FORCE PARTICIPATION AND EMPLOYMENT? ...................................... 55 HOW ARE DIFFERENT SKILLS SETS, IN CONJUNCTION WITH EDUCATION, CONTRIBUTING TO HIGHER EARNINGS? ....................... 58 IS TRAINING PROVIDING VALUE-ADDED TO WORKERS IN TERMS OF SKILLS? ..........................................................................62 HIGHLIGHTS OF FINDINGS ............................................................................................................................................ 64 6. STEP 4 | ENCOURAGING ENTREPRENEURSHIP AND INNOVATION IN ARMENIA...............................................65 WHO ARE THE ENTREPRENEURS?.................................................................................................................................. 65 HOW DO ENTREPRENEURS AND WAGE-EARNING WORKERS DIFFER IN THEIR USE OF SKILLS?.................................................. 68 WHO IS WORKING IN THE HIGH-INNOVATION SECTORS? ....................................................................................................70 WHAT SKILLS ARE USED IN SECTORS MOST RELATED TO INNOVATION? ................................................................................ 71 HIGHLIGHTS OF FINDINGS ............................................................................................................................................. 72 7. STEP 5 ǀ FACILITATING LABOR MARKET MOBILITY AND JOB MATCHING IN ARMENIA ................................... 74 HOW DOES THE SCHOOL-TO-WORK TRANSITION AFFECT CURRENT LABOR MARKET OUTCOMES AND SKILLS? ............................74 HOW DOES FIRST OUT-OF-SCHOOL JOB AFFECT SKILLS DEVELOPMENT? ...............................................................................76 3 TO WHAT EXTENT ARE THERE OCCUPATIONAL MISMATCHES IN THE LABOR MARKET?............................................................ 80 HIGHLIGHTS OF FINDINGS ............................................................................................................................................. 81 8. CONCLUSIONS AND WAY FORWARD ...................................................................................................................83 REFERENCES .............................................................................................................................................................. 86 TECHNICAL ANNEXES ................................................................................................................................................92 ANNEX 1: THE STEP SKILLS MEASUREMENT PROGRAM: THE HOUSEHOLD SURVEY .............................................................92 ANNEX 2: SKILLS MEASURED IN THE STEP SKILLS SURVEY .............................................................................................. 96 ANNEX 3. SELECTED STATISTICAL RESULTS ................................................................................................................... 104 LIST OF FIGURES FIGURE 1.1. SKILLS TOWARD EMPLOYMENT AND PRODUCTIVITY (STEP) FRAMEWORK ............................................................. 17 FIGURE 2.1. GROWING DEMAND FOR MANUAL SKILLS AND LIMITED DEMAND FOR PROFESSIONAL SKILLS .....................................26 FIGURE 2.2. EMPLOYERS FIND IT DIFFICULT TO HIRE SKILLED WORKERS ...................................................................................26 FIGURE 2.3. EMPLOYERS’ VIEWS OF THE ADEQUACY OF TECHNICAL AND GENERAL EDUCATION ................................................... 27 FIGURE 2.4. EMPLOYERS’ RANKING OF THE IMPORTANCE OF SKILL GROUPS, BY TYPE OF WORKER ..............................................28 FIGURE 2.5. EMPLOYERS’ RANKING OF THE JOB-RELATED CHARACTERISTICS, BY TYPE OF WORKER..............................................29 FIGURE 3.1. PARTICIPATION IN ECE BY GENDER AND AGE COHORT ......................................................................................... 31 FIGURE 3.2. PARTICIPATION IN ECE BY HOUSEHOLD SOCIOECONOMIC STATUS AT AGE 15 .......................................................... 32 FIGURE 3.3. PROBABILITY OF GRIT AND OPENNESS SCORES BY ECE PARTICIPATION ..................................................................34 FIGURE 3.4. PARTICIPATION IN ECE AND USE OF READING SKILLS ..........................................................................................34 FIGURE 3.5. PROBABILITY OF USING COMPUTER SKILLS AT WORK BY ECE PARTICIPATION ......................................................... 35 FIGURE 4.1. EDUCATIONAL ATTAINMENT BY GENDER AND AGE COHORT ..................................................................................38 FIGURE 4.2. PROPORTION OF ADULTS USING READING, NUMERACY AND COMPUTER SKILLS BY INTENSITY LEVEL AND EDUCATIONAL ATTAINMENT ..........................................................................................................................................................39 FIGURE 4.3. PROPORTION OF ADULTS USING READING, COMPUTER, AND PROBLEM SOLVING AND LEARNING SKILLS BY INTENSITY LEVEL .................................................................................................................................................................... 41 FIGURE 4.4. DISTRIBUTION OF READING LITERACY PROFICIENCY BY LEVEL FOR OECD COUNTRIES (NATIONAL) AND ARMENIA (URBAN) ............................................................................................................................................................... 42 FIGURE 4.5. AVERAGE READING LITERACY SCORE IN OECD COUNTRIES (NATIONAL) AND ARMENIA (URBAN) BY GENDER AND EDUCATION ATTAINMENT .........................................................................................................................................43 FIGURE 4.6: DISTRIBUTION OF READING LITERACY PROFICIENCY BY LEVEL, GENDER, AGE, AND EDUCATIONAL ATTAINMENT .......... 44 FIGURE 4.7. PREDICTED SCORE IN CONSCIENTIOUSNESS (SCALE OF 1-4) BY READING LEVEL ...................................................... 44 FIGURE 4.8. PREDICTED SCORE IN GRIT AND DECISION MAKING BY EDUCATIONAL ATTAINMENT..................................................45 FIGURE 4.9. DISTRIBUTION OF READING, COMPUTER AND SOLVING AND LEARNING SKILLS BY PAST HOUSEHOLD SOCIOECONOMIC STATUS.................................................................................................................................................................. 47 FIGURE 4.10. AVERAGE SCORES IN CONSCIENTIOUSNESS AND OPENNESS BY PAST SOCIOECONOMIC HOUSEHOLD STATUS ............ 48 FIGURE 4.11. DISTRIBUTION OF RESPONDENTS BY HOUSEHOLD SES AT AGE 15 AND HOUSEHOLD WEALTH INDEX NOW ................. 49 FIGURE 4.12. DISTRIBUTION OF RESPONDENTS BY HOUSEHOLD SES AT AGE 15 AND HOUSEHOLD WEALTH INDEX NOW, BY EDUCATIONAL ATTAINMENT .................................................................................................................................... 49 FIGURE 4.13. DISTRIBUTION OF USE OF PROBLEM-SOLVING AND LEARNING SKILLS BY HOUSEHOLD SES AT AGE 15 AND HOUSEHOLD WEALTH INDEX, NOW.............................................................................................................................................. 50 FIGURE 4.14. DISTRIBUTION OF READING EFFICIENCY BY HOUSEHOLD SES AT AGE 15 AND HOUSEHOLD WEALTH INDEX NOW, BY READING PROFICIENCY LEVEL ................................................................................................................................... 50 4 FIGURE 4.15. PROBABILITY OF SCORING AT HIGHER LEVELS IN READING PROFICIENCY BY MOTHER’S EDUCATIONAL ATTAINMENT .... 51 FIGURE 4.16. ADULTS’ PROBABILITY OF USING SOLVING AND LEARNING SKILLS BY MOTHER’S EDUCATIONAL ATTAINMENT ............ 52 FIGURE 4.17. ADULTS’ PROBABILITY OF USING READING SKILLS BY HOUSEHOLD VULNERABILITY AT AGE 15 (MEASURED IN NUMBER OF SHOCKS EXPERIENCED) ........................................................................................................................................ 52 FIGURE 5.1. LABOR FORCE PARTICIPATION AND EMPLOYMENT DISTRIBUTION BY EDUCATIONAL ATTAINMENT .............................. 55 FIGURE 5.2. LABOR FORCE PARTICIPATION AND EMPLOYMENT DISTRIBUTION BY READING PROFICIENCY LEVELS .......................... 56 FIGURE 5.3. LABOR FORCE PARTICIPATION AND EMPLOYMENT SHARE BY AGE GROUP AND READING PROFICIENCY LEVELS ............ 56 FIGURE 5.4. CHANGES IN THE PROBABILITY OF PARTICIPATING IN THE LABOR FORCE BY TYPE OF SKILLS ...................................... 57 FIGURE 5.5. GENDER DIFFERENCES: PROBABILITY OF BEING EMPLOYED BY DIFFERENT CHARACTERISTICS, SKILL GROUP, AND GENDER ........................................................................................................................................................................... 58 FIGURE 5.6. RETURNS TO YEARS OF EDUCATION BY DIFFERENT CHARACTERISTICS AND SKILL GROUPS ........................................ 59 FIGURE 5.7. RETURNS TO UNIT INCREASE OF DIFFERENT SKILLS ON HOURLY EARNINGS ............................................................ 60 FIGURE 5.8. ON-THE-JOB TRAINING, PROFESSIONAL CERTIFICATE AND APPRENTICESHIP PARTICIPATION BY AGE COHORT...............62 FIGURE 5.9. DIFFERENCES IN USE AND INTENSITY OF JOB- RELEVANT SKILLS BY ON-THE-JOB TRAINING, PROFESSIONAL CERTIFICATE, AND APPRENTICESHIP ..............................................................................................................................................63 FIGURE 6.1. DISTRIBUTION OF ENTREPRENEURS BY GENDER AND AGE COHORT ....................................................................... 66 FIGURE 6.2. EDUCATION LEVELS FOR THE SELF-EMPLOYED AND WAGE WORKERS .....................................................................67 FIGURE 6.3. DISTRIBUTION OF OCCUPATIONS AMONG ENTREPRENEURS AND WAGE WORKERS...................................................67 FIGURE 6.4. READING SKILLS FOR ENTREPRENEURS AND WAGE WORKERS ............................................................................. 68 FIGURE 6.5. INTENSITY OF COMPUTER SKILL USE AT WORK, ENTREPRENEURS AND WAGE WORKERS .......................................... 69 FIGURE 6.6. EDUCATION LEVELS OF INDIVIDUALS WORKING IN HIGH- AND LOW-INNOVATION SECTORS .......................................70 FIGURE 6.7. INTENSITY OF USE OF READING SKILLS AMONG HIGH- AND LOW-INNOVATION WORKERS .......................................... 71 FIGURE 6.8. INTENSITY OF COMPUTER SKILLS AMONG HIGH- AND LOW-INNOVATION WORKERS ................................................. 72 FIGURE 7.1. SCHOOL-TO-WORK TRANSITION BY EDUCATIONAL ATTAINMENT ............................................................................ 75 FIGURE 7.2. USE AND INTENSITY OF READING, WRITING, AND NUMERACY SKILLS BY SCHOOL-TO-WORK TRANSITION TIME .............76 FIGURE 7.3. GENDER COMPOSITION OF FIRST OUT-OF-SCHOOL OCCUPATION DISTRIBUTION....................................................... 77 FIGURE 7.4. OPENNESS BY OCCUPATION OF FIRST OUT-OF-SCHOOL JOB AND EDUCATIONAL ATTAINMENT ................................... 77 FIGURE 7.5. PROBLEM-SOLVING AND LEARNING SKILLS’ USE AND INTENSITY, BY OCCUPATION OF FIRST OUT-OF-SCHOOL JOB.........78 FIGURE 7.6. USE OF READING, WRITING, NUMERACY, AND COMPUTER SKILLS FOR INACTIVES AND EMPLOYED .............................79 FIGURE 7.7. GENDER, EDUCATIONAL ATTAINMENT, DROPOUT RATES, AND FIRST JOB RATES FOR INACTIVE AND EMPLOYED............79 FIGURE 7.8. OCCUPATIONAL MISMATCH BY GENDER ........................................................................................................... 80 FIGURE 7.9. COMPUTER USE SCORE, BY JOB QUALIFICATION ................................................................................................. 81 FIGURE 7.10. SCORE OF CONTACT WITH PEOPLE OUTSIDE OF WORK, BY JOB QUALIFICATION ...................................................... 81 LIST OF BOXES BOX 3.1. INVESTING IN EARLY CHILDHOOD EDUCATION AND DEVELOPMENT IN ARMENIA ........................................................... 35 BOX 4.1. THE EDUCATION SYSTEM IN ARMENIA .................................................................................................................. 40 BOX 4.2. RESULTS FROM THE 2012 TRENDS IN INTERNATIONAL MATHEMATICS AND SCIENCE STUDY, ARMENIA ........................ 46 BOX 5.1. FINDINGS OF THE SABER-WORKFORCE DEVELOPMENT STUDY, ARMENIA................................................................. 61 LIST OF TABLES TABLE 1.1. SKILLS MEASURED ............................................................................................................................................ 21 TABLE 2.1. LABOR FORCE PARTICIPATION (% TOTAL) OF ARMENIA, AZERBAIJAN, AND GEORGIA ................................................ 24 TABLE 2.2. UNEMPLOYMENT RATE (TOTAL) OF ARMENIA, AZERBAIJAN, AND GEORGIA ............................................................ 24 5 LIST OF ANNEX FIGURES ANNEX FIGURE 1: STRUCTURE OF HOUSEHOLD SURVEY ........................................................................................................ 94 ANNEX FIGURE 2: DIRECT ASSESSMENT OF READING LITERACY – READING ASSESSMENT FLOW CHART ........................................ 98 LIST OF ANNEX TABLES ANNEX TABLE 1. MAIN CHARACTERISTICS ...........................................................................................................................92 ANNEX TABLE 2: STEP AND ALTERNATIVE ESTIMATES OF URBAN POPULATION (15-64 YEARS) BY GENDER & AGE GROUP ..............93 ANNEX TABLE 3: SAMPLE SIZES AND RESPONSE RATES, BY COUNTRY ..................................................................................... 95 ANNEX TABLE 4: STEP SKILLS MEASURED ......................................................................................................................... 96 ANNEX TABLE 5: DIRECT MEASUREMENT OF READING PROFICIENCY | KEY INDICATORS ............................................................ 98 ANNEX TABLE 6: INDIRECT MEASUREMENT OF COGNITIVE SKILLS | KEY INDICATORS ............................................................... 100 ANNEX TABLE 7: SOCIO-EMOTIONAL SKILLS | ITEMS ............................................................................................................ 101 ANNEX TABLE 8: JOB-RELEVANT SKILLS ............................................................................................................................ 102 ANNEX TABLE 9: AGGREGATION OF VARIABLES.................................................................................................................. 103 ANNEX TABLE 10: PROBABILITY OF USING COMPUTER SKILLS AT WORK BY ECE PARTICIPATION ............................................... 104 ANNEX TABLE 11: PROBABILITY OF EXTRAVERSION, CONSCIENTIOUSNESS, AND GRIT BY ECE PARTICIPATION BY ECE PARTICIPATION .......................................................................................................................................................................... 106 ANNEX TABLE 12: PREDICTED SCORE IN CONSCIENTIOUSNESS BY READING LEVEL ................................................................... 108 ANNEX TABLE 13: PREDICTED SCORE IN GRIT AND DECISION MAKING BY EDUCATIONAL ATTAINMENT ......................................... 110 ANNEX TABLE 14: PROBABILITY OF SCORING AT HIGHER LEVELS IN READING PROFICIENCY BY MOTHER’S EDUCATIONAL ATTAINMENT ........................................................................................................................................................................... 112 ANNEX TABLE 15: ADULTS’ PROBABILITY OF USING SOLVING AND LEARNING SKILLS BY MOTHER’S EDUCATIONAL ........................ 114 ANNEX TABLE 16: ADULTS’ PROBABILITY OF USING READING SKILLS BY HOUSEHOLD VULNERABILITY AT AGE 15 (MEASURED IN NUMBER OF SHOCKS EXPERIENCED).......................................................................................................................... 116 ANNEX TABLE 17: PROBABILITY MODELS FOR LABOR FORCE PARTICIPATION AND EMPLOYMENT .............................................. 118 ANNEX TABLE 18: RETURNS ON HOURLY EARNINGS BY AGE GROUPS. .................................................................................. 120 6 Executive Summary Context Governments around the world assign top priority to job creation and productivity growth. Developing the right skills among potential and actual workers not only makes capital and labor more productive, it also makes the adoption and invention of new technologies possible. Recent research also indicates that skill acquisition has a long-lasting impact on the trajectory of a person’s life and that inequality in skills is associated with inequality in income. Low-productivity employment is a distinctive challenge for the labor market in Armenia. Many existing jobs do not pay enough to lift people out of poverty. Nearly 40 percent of workers are employed in agriculture, while the sector accounts for only 17 percent of GDP (Rutkowski, 2012). This discrepancy between input and output implies that most agricultural jobs are of low productivity. The share of the non-agricultural sector has increased by less than one percentage point in the last five years. The modern, high-productivity sector is small and it is barely expanding. Moreover, the proportion of non-agricultural low-earning jobs is high in Armenia. Low-earning jobs are defined as those that earn less than two-thirds of the median wage. By this criterion, one in four jobs in Armenia falls into this category, which represents a significantly higher share than that in most European countries, where the incidence of low pay is within the 15-20 percent range. The significant presence of informality in non-agricultural sectors is another factor that contributes to the low-productivity and low-earnings employment. Creating modern and high-earning jobs in the formal sectors can only benefit the economy if these jobs can be filled quickly by capable Armenian workers. It is therefore important for Armenia’s workforce to be equipped with the right skills to prepare for new and modern jobs as well as with skills that can foster entrepreneurship and innovation. In terms of educational attainment, Armenia stands out in the region with a high tertiary education enrollment— 46 percent in 2012. An important question then is whether this high educational attainment is translating into skills relevant for the labor market, and not just producing diplomas (Sondergaard and Murthi, 2012). In fact, despite the high availability of labor and these high educational levels, Armenia’s employers are struggling to find the right workers, which seems to point to a problem of skills in the labor force. 7 To better understand skill shortages in Armenia, this report looks into the current demand for skills from the labor market, together with the landscape of skills formation and utilization in the country, using the newly available data from the World Bank’s Skills Toward Employment and Productivity (STEP) household and employer surveys, which were undertaken in the country between 2012 and 2013. These extensive surveys sampled Armenia’s urban population and firms. Based on these surveys, this report aims to provide a key diagnosis of skills demand and supply issues in Armenia, highlighting a few initial steps that need to be taken to build a highly productive Armenian labor force, one that can contribute to as well as benefit from the accelerated economic growth. Key findings The findings in this report are organized to follow the five steps that make up the STEP conceptual framework, which follows a life-cycle approach to skill acquisition. The STEP conceptual framework builds on five steps: (i) getting children off to the right start; (ii) ensuring that all students learn; (iii) building job-relevant skills that employers demand; (iv) encouraging entrepreneurship and innovation; and (v) matching the supply of skills with employers’ demands. Following are the broad, key findings from the study, arranged under these same five steps. It should also be noted that the findings are representative only of Armenia’s urban areas. Getting children off to the right start. Participation in early childhood education (ECE) programs is positively correlated with the development of cognitive and socio-emotional skills, and these skills are associated with better labor market outcomes. Quality early childhood education has been found by numerous studies to help build the cognitive and socio-emotional skills that are later valuable to employability, worker productivity, and flexibility in the work environment. In this area, participation in ECE programs in Armenia has expanded over time, although the trend may be reversing as younger cohorts report slightly lower participation rates compared to older adults. Moreover, the analysis shows that early childhood programs are positively associated with having higher socio- emotional skills and using reading and computer work-skills (both related to better labor market outcomes). Expanding and improving the quality of ECE programs shows a clear benefit in potentially increasing cognitive and socio-emotional skills. Given that skills formation is cumulative, the sooner the foundations are laid for children, the easier and less costly it will be to build on them at later stages in the life cycle. 8 Ensuring that all students learn. Although education attainment is high, the relevance and quality of the education system needs to be strengthened. Building strong educational systems with clear learning standards, effective teachers, adequate resources, and an appropriate regulatory environment is key to better workforce development. For more than a generation, Armenia has had a good track record of high educational attainment—more than half the adult urban population holds a tertiary degree and more than 85 percent have at least an upper-secondary education. However, average reading proficiency scores in Armenia are below international standards. Particularly worrisome are the relatively low reading proficiency scores of tertiary education graduates, which are not substantially different from those of upper-secondary graduates. Tertiary education graduates, however, do show higher scores than other adults in the socio-emotional skills, such as grit and decision-making, two skills that are important to success in entrepreneurship and for earnings as employees. Overall, there is scope to increase the relevance and quality of education in order to improve the acquisition of foundational and job-relevant skills for the future workforce. Moreover, there is also a need to assess the curricula and teaching practices to determine the extent to which these may be improved to promote socio-emotional skills and, by extension, promote the skills development that makes individuals more employable. Building job-relevant skills that employers demand. A stronger emphasis needs to be placed on the quality of learning, since both education and skills are key drivers of labor market outcomes. Given both the large quantity of secondary-school graduates in Armenia and the poor match-up of their skills with employers’ needs, it is vital to know what skills formal education is still failing to prepare graduates with. This study finds that some of the skills individuals acquire are associated with labor market outcomes in addition to formal schooling. This may be due to the large share of tertiary educated workers, so that employers and workers are using skills as a way to differentiate among workers and job candidates. Moreover, there is a gender gap in labor outcomes that will need to be addressed with specific efforts to improve women’s skills set. Finally, Armenia’s low rates of participation in on-the-job training and other forms of formal professional accreditation may be resulting in a loss of skill acquisition and missed opportunities for updating skills. Encouraging entrepreneurship and innovation. Although entrepreneurial activities are currently uncommon in Armenia, there is a window of opportunity to promote job creation and productivity enhancement in an equitable manner. Entrepreneurial activity may be a valuable option in markets with low demand for workers and high unemployment, like Armenia today. In addition, 9 entrepreneurship can help bring new products to the market and foster innovation. Measured as a percentage of self-employed out of total employment, the rate of entrepreneurship in the country today is low. Currently, entrepreneurs are most likely to be middle-aged men whose socio-emotional skill levels are similar to those of wage earners, although they have slightly less education and lower job-related and cognitive skills than wage workers. This strongly suggests that much of the self- employment may be driven by the inability to find a job. That being said, it is also clear that entrepreneurship literally pays off, since entrepreneurs, defined as those individuals who are self- employed or own a business of any size, have higher average incomes than wage earners. Regarding innovation, one of the most innovative sectors—high technology—displays an unexpected feature: the overwhelming majority (70 percent) of employees in this sector are women. Overall, it seems that the economy may benefit from a future increase in the share of entrepreneurs due to the benefits entrepreneurship brings to innovation, the country’s large unemployed and inactive population, and the higher earnings of entrepreneurs when compared with wage workers. Matching the supply of skills with employers’ demands. The school-to-work transition in Armenia is rather quick, and most workers seem to have adequate qualifications for the jobs they hold. Nonetheless, inactivity is high, particularly for women. In most countries, longer school-to-work transitions are associated with poorer labor market outcomes, and Armenia is not an exception. Armenia’s labor market seems to absorb new workers rapidly, but it may do so in low-skilled jobs. For instance, among individuals between ages 25 and 40, 92 percent of those who never passed beyond a lower-secondary education found a job within 6 months of leaving school, whereas for those whose education ended with upper secondary, only 62 percent found work that quickly. Moreover, the analysis found that the first out-of-school job that a worker obtains matters for their future skill development. Another feature of the labor market is the high level of inactivity—that is, those neither studying nor looking for work nor being employed— which is most common among women ages 25 to 40. These women also make up a disproportionate number of those who did not find a job after searching for more than six months. Finally, the analysis found that about 70 percent of workers seem to have adequate qualifications for the jobs they have. Overall, in addition to improvements needed in cognitive, job-related and other skills, as implied by the preceding details, Armenia will need to make specific efforts to improve women’s situation in order to close the gender gap in labor force participation. 10 Future policy and research agenda Policy options can also be explored following the STEP framework. To further strengthen Armenia’s early childhood education system, priority should be given to three areas: (i) Access to early education should be improved for socioeconomically disadvantaged households, for whom the system is currently out of reach. (This equity concern also justifies further increasing public investment in service provision). (ii) In addition, the apparent declining trend in recent participation in early education needs to be further looked into, to discern its patterns and likely causes. Policy interventions may be needed to reverse this trend. (iii) Although the service quality issue is not covered by the data, global experience shows that expanding public services on a large scale sometimes risks lowering the bar for service quality. Ensuring service quality is therefore critical for the effectiveness of expanded early education programs. Expansion without quality will only lead to a waste of resources. Moving forward to improve its education and training system, Armenia will need to set three priorities: (i) Improve learning outcomes and skills formation. Worldwide experience shows that this is not an easy task, and results may take years to show. Lessons from other countries also show that having a high-quality, effective teaching force is the foremost necessary condition for high- quality education. Having a strong quality assurance and accountability system for learning results is another important prerequisite. (ii) Strengthen tertiary education. This will be necessary because over half of Armenia’s labor force comes from tertiary graduates. Both systemic and institutional reforms are needed, giving emphasis to building a close link with the labor market need for a skilled and innovative labor force – not only as employees, but also as entrepreneurs. (iii) Narrow the socioeconomic gaps in learning outcomes and skills. Measures need to be put in place targeting those who lag behind in the school system. An effective school system should serve as an important channel to narrow the gaps associated with the different family backgrounds that students come from. On top of the foundational skills, it will be necessary to build technical and job-specific skills, based on a re-thinking of the two important stages of skill formation: (i) Vocational education and training 11 systems need to be assessed and strengthened. The first challenge is how to improve the effectiveness of the current preliminary and middle-level vocational schools in their provision of skills needed. In the longer term, as the demand for high-level skills rises, whether to delay tracking after 12 years of foundational learning can be evaluated and considered. (ii) On-the-job training remains the most effective way to acquire job-specific skills. To increase the opportunities of on-the-job training, incentives can be designed to encourage firms’ increased provision of on -the job training to their employees. The current apprentice programs do not seem to work in equipping trainees with job-relevant skills, so the programs’ designs should be closely examined before considering extending their future. To encourage productive entrepreneurship and innovation, future policy assessment and consideration can aim at: (i) creating an enabling business environment and improving business supporting services and risk-management policies and instruments to encourage entrepreneurship that targets innovative sectors; (ii) re-aligning some of the education and training courses, particularly at the tertiary level, with the objective of fostering entrepreneurship by equipping trainees with essential knowledge, skills, and attitudes. To fully understand labor market mobility and job match, two important questions need to be examined in future research: (i) What is the job turnover rate in Armenia, both in the present and expected in the future, and how should the system of adult learning and skills re-training or updating respond to the need? (ii) What is the status and performance of any public employment services that exist—if indeed they exist—and how could they function more effectively to foster labor market mobility? How could they provide real-time labor market information and bridge employers with job- seekers? 12 1. About the Report Skills are at the core of improving individuals’ employment outcomes and increasing countries’ productivity and growth. A labor force with adequate skills is quintessential for enhancing labor and total factor productivity (TFP), as it enables firms and workers to adapt to rapid technological change and innovation. This is particularly relevant as today’s developing and emerging countries seek higher sustained growth rates. In recent decades, scientific development has sped up and new technologies have become increasingly available and accessible worldwide. In the meantime, the markets for goods, services, capital, and labor have become increasingly global, creating opportunities as well as challenges for countries and individuals. These global trends have given rise to structural changes in the labor market. The demand for and returns to high-level worker skills have been rising. Cross-country data show a strong positive correlation between the level of economic development and the intensity of cognitive skills in the economy (Aedo et al., 2013). In the meantime, there are increasing gaps between the skills of the labor force and the skills needed by firms. Surveys of businesses often indicate that the skill set of workers is one of the main bottlenecks. More and more employers demand a sophisticated set of skills, including behavioral and higher-order cognitive skills, but education and training systems have yet to be adapted in order to impart these skills. Earlier boom years in the Europe and Central Asia (ECA) region already exposed significant bottlenecks to growth with respect to the skills of the labor force. A shortage of worker skills has emerged as one of the most important constraints to firm expansion. As shown in Mitra, Selowsky, and Zalduendo (2010), over the latter half of the first decade in the 2000s there was a substantial increase in the share of firms reporting that finding workers with adequate skills was a major or very severe constraint to business growth, and this increase occurred in virtually all ECA countries. Because the countries of the ECA region are recovering from the recent global recession, they are facing post-crisis conditions that are also very different from those of preceding years. Financial resources are more limited and more expensive, and export growth is restrained by potentially slower growth in destination countries. Restoring and sustaining growth in this context requires deep reforms that can boost competitiveness and increase labor productivity. 13 Context and objective In Armenia, skills development is a priority for a number of reasons: first, for governments and citizens alike, employment has taken center stage as a high priority in the political and economic discourse. Second, there is considerable interest in enhancing skills for enhanced productivity and competitiveness. Not unrelated to this is strong interest in skills to unleash entrepreneurship. The education systems in Armenia have important advantages, including higher average enrollments than most countries at comparable levels of development, as well as higher attainment levels. The country embarked on ambitious reform programs which have yielded some positive results; however, the educational systems have been slow to adjust to the changing needs of the economy: The earlier vocational education systems, which had been adequate for providing skills that would lead directly to employment opportunities in the centrally planned economy, collapsed at the time of the breakup of the Soviet Union; few vocational and technical education programs have been able to provide the required skills for the modernizing economies or to ensure the levels of employment needed. The scenario in Armenia is no exception to what was analyzed in the recent World Bank ECA regional report, Skills Not Just Diplomas (Sondergaard and Murthi, 2012). That report uses a range of different data sources to argue that the skills problem in the ECA region is more closely related to the quality and relevance of the education provided in ECA countries than to problems of access. A central argument of the report is that policymakers are constrained in a number of ways from effectively managing their education and training sectors. The lack of systematic data on key skills-related issues is one of the impediments to improving quality and relevance. While there are international assessments that provide information on student competencies up to the age of 15 (usually the end of lower-secondary education), this is not an age when most people in the region are entering the job market. Beyond these early-stage assessments, no comprehensive, fully reliable information exists on individual competencies. What is more, the information on competencies that is available relates only to fundamental cognitive skills, not the behavioral skills—involving such issues as work ethic and teamwork—that are emphasized by employers. For policymakers to better understand the causes of the emerging skills bottleneck and how best to address it, this informational gap needs to be closed. In another ECA regional report, Back to Work: Growing with Jobs in Europe and Central Asia (Arias and 14 Sanchez-Paramo, 2014), the authors use the job lens to examine the extent to which workers in the region are prepared to take on new job opportunities; they look into the skills gaps that constrain employment in ECA. They argue that skills gaps hinder the labor performance of both youth and older workers, to varying degrees across countries. As in more modern economies, jobs in the ECA region are becoming increasingly intensive in higher-order (new economy) skills—especially in countries more integrated into external markets and with a more skilled workforce—and economic developments post-transition may have rendered obsolete the skills of many older workers. The response from education and training systems has been uneven in the region. Tertiary schooling has expanded fast among youth—delivering diplomas, though with varying quality and relevance. Meanwhile, the legacy of early tracking into technical and vocational education and training (TVET) and labor training with loose market links limits the needed skills upgrading among adults and older workers. As a result, youth and older workers are affected by skills gaps in distinctive ways. Youth do better in acquiring skills, but many often acquire the wrong set (both generic and technical). Many older workers educated for centrally planned economies are at risk of skills obsolescence, which hinders their capacity to tap into new employment opportunities. And in some fast-aging countries, the emigration of better educated youth compounds skills constraints. In a nutshell, throughout the ECA region reforms and policies aiming at strengthening the system of building skills for the workplace should prioritize the development of a strong foundation of generic skills, ensure quality and relevance in expanding tertiary education systems, and make the training system market responsive and age sensitive to enable lifelong skills acquisition. This new report on skills development in Armenia builds on these two recent World Bank regional reports, adding country-specific analysis enriched by the latest employer and household surveys. The objective of this report is to build a country-specific knowledge base and provide a solid platform for Armenia to explore policy reforms in skills development. With a focus on skills formation, the new information from the survey data sets enables this report to answer some important questions on how to build relevant skills in today’s Armenia. The report presents findings on the overall trends on educational attainment, skills use, and labor market outcomes. While labor force surveys monitor standard indicators of educational attainment, employment and unemployment, occupation and industry, earnings, and hours worked to track short- and long-term labor market conditions, the new survey data sets underlining this report fill some important information gaps. These include: what types of skills are actually required by 15 current or future jobs; why a particular level of education might be needed for certain kinds of jobs; how education and skills are relate to the technological content of certain jobs; and even what level of education and skills are needed for these jobs. The new survey data also measure mismatches between skill stocks and job skill requirements or the degree to which the skills of workers and jobs are meeting national goals. In addition, the information collected covers both cognitive and socio-emotional skills and personal characteristics. With a growing body of research that has been documenting the importance of these skills in determining labor market performance (Cawley, Heckman and Vytlacil, 2001; Jacob, 2002; Behrman et al., 2006; Heckman, Stixrud, and Urzua, 2006; Borghans, Meijers, and Ter Weel, 2008; Hanushek and Woessmann, 2008), introducing the multi-dimensional measurement of skills has become critical to understanding the complexity of skills formation and utilization. The information presented in this report is not meant to be exhaustive or definitive. Most of the findings reflect suggestive correlations, but not causalities. Nonetheless, these exploratory findings identify areas of high policy relevance and pave the ground for further policy research on skills, employment, productivity, and earnings. Conceptual framework and research questions Skill formation can be considered as a continuum over the life cycle of an individual, with key stages for the development of both cognitive and socio-emotional skills. Cunha et al., (2006) find evidence that there are sensitive and critical periods for inputs in the acquisition of cognitive skills and non- cognitive skills. For example, their findings suggest that parental inputs are particularly critical for cognitive skill formation during a child’s early years (preschool and early primary grades).1 Non- cognitive skills appear to be more malleable throughout the life cycle; however, early interventions appear to have large positive effects. For example the Perry Preschool program in the United States, which focused on early childhood development, was found to have large positive effects on non- cognitive skills (Heckman et al., 2010). These findings have important implications for the design of effective policies and the delivery of time-sensitive programs. 1 The evidence from several studies surveyed in Cunha et al., (2006) shows that IQ is relatively stable after age eight, which implies that early childhood is the key period for the development of cognitive skills. 16 Figure 1.1. Skills Toward Employment and Productivity (STEP) framework Source: World Bank, Stepping up Skills for More Jobs and Higher Productivity (2010b). This report follows the five steps included in the STEP conceptual framework (World Bank, 2010b), which follows a life cycle approach. The STEP framework brings together research-based evidence and practical experience from diverse areas—from research on the determinants of early childhood development and learning outcomes to policy experience in the reforming of vocational and technical education systems and labor markets. It also provides a set of powerful messages for policymakers, researchers, and practitioners. The five steps from the framework may be summarized as follows (Figure 1.1). Step 1. Getting children off to the right start—by developing the technical, cognitive, and behavioral skills conducive to high productivity and flexibility in the work environment through early childhood education (ECE), with an emphasis on nutrition, stimulation, and basic cognitive skills. Research shows that handicaps developed early in life are difficult if not impossible to remedy later and that effective ECE programs can have a very high payoff. The key questions to consider in this step include these:  What is the status of ECE in terms of access and quality?  Are the current ECE programs effective in early skills development?  Are there life-long benefits of ECE?  What can be done to further improve ECE for effective and equitable skills development? 17 Step 2. Ensuring that all students learn—by building stronger education systems with clear learning standards, good teachers, adequate resources, and a proper regulatory environment. Lessons from research and on-the-ground experience indicate that the key decisions about education systems are how much autonomy to allow and to whom, how much accountability to expect from whom and for what, and how to assess performance and results. The key questions to consider in this step include these:  Is the current education system delivering high-quality and job-relevant skills?  How do family socioeconomic factors affect the educational attainment and skills development of school-age children?  How can the current education system be strengthened to deliver better and more equitable learning outcomes? Step 3. Building job-relevant skills that employers’ demand—by developing the right incentive framework for both pre-employment and on-the-job training programs and institutions (including higher education). A growing body of experience is showing how public and private efforts can be combined to achieve more relevant and responsive training systems. The key questions to consider in this step include these:  To what extent do skills shortages or mismatches explain unemployment?  Is the current pre-employment TVET system effective?  How can pre-employment TVET and on-the-job training be more effective in delivering job- relevant skills? Step 4. Encouraging entrepreneurship and innovation—by creating an environment that encourages investment in knowledge and creativity. Emerging evidence shows that this requires innovation- specific skills (which can be developed starting early in life) and investments to connect people with ideas (such as through collaborations between universities and private companies) as well as risk- management tools that facilitate innovation. The key questions to consider in this step include these:  What are the skill sets possessed by the current entrepreneurs?  How do skill sets differ between entrepreneurs and wage earners?  What are the skills gaps for innovation and knowledge powered growth? Step 5. Matching the supply of skills with employers’ demands—by moving toward more flexible, efficient, and secure labor markets. Avoiding rigid job protection regulations while strengthening 18 income protection systems, supplemented by efforts to provide information and intermediation services to workers and firms, make up the final complementary step that enables skills to be transformed into actual employment and productivity. The key questions to consider in this step include these:  Are there clear market signals of demand for skills?  What are the symptoms of supply-demand mismatch?  What are the key regulatory barriers for job market entry and mobility?  How can labor market information and intermediation services function better? Data, type and definition of skills measured, and methodology Data The report draws its evidence mainly from the STEP surveys, which include both a household-based survey and an employer survey. The household survey has three unique modules that cover the skills mentioned above: (i) a direct assessment of reading proficiency and related competencies scored on the same scale as the OECD’s PIAAC (Programme for International Assessment of Adult Competencies) assessment; (ii) a battery of self-reported information on personality, behavior, and preferences (e.g., Big Five, GRIT, decision-making, and hostility bias); and (iii) a series of questions on the task-specific skills that the respondent possesses or uses in his or her job. The employer survey gathers information on job skill requirements using questions parallel to those in the household survey, a feature that facilitates analysis of skill gaps and mismatches. The employer survey also has information on practices relating to (i) hiring and compensation, (ii) training, and (iii) enterprise productivity. Additionally, the following sources of information were included to situate and contextualize findings: (i) the World Bank’s Systems Approach for Better Education Results (SABER) country reports on Workforce Development (WfD) and Early Childhood Development (ECD) systems in Armenia; (ii) Armenia’s Integrated Living Conditions surveys; (iii) UNESCO’s World Data on Education; (iv) OECD’s Programme for International Assessment of Adult Competencies (PIAAC); and (v) the Trends in International Mathematics and Science Study (TIMSS). Type and definition of skills measured 19 1.1 The STEP survey measures three broad categories of skills: cognitive skills, socio-emotional job-relevant skills (Pierre et al., 2014). Within each of these categories are more specific skills that can each be measured, either by direct measurement or by self-reporting. All the skills are listed in 20 Table 1.1 below. Cognitive skills are defined as the “ability to understand complex ideas, to adapt effectively to the environment, to learn from experience, to engage in various forms of reasoning, to overcome obstacles by taking thought.”(Neisser et al. 2006). Literacy, numeracy, and the ability to solve abstract problems are all cognitive skills. The STEP survey provides a direct measurement of reading proficiency and an indirect measurement of reading, writing, and mathematics skills. The direct measurement of cognitive skills entails the assessment of reading proficiency through the reading literacy assessment designed by the Educational Testing Services (ETS). Cognitive reading assessment results are scored on the same scale as the test in the OECD’s PIAAC. The reading literacy assessment has three parts. The first part (Section A) evaluates foundational reading skills, including word meaning, sentence processing and passage comprehension. The second part (Section B) consists of a core literacy assessment that is intended as a filter to sort the least literate adults from those with higher reading skill levels. The core has a total of eight items. Respondents with three or more correct responses are regarded as having met a minimum reading literacy threshold. The third part (Exercise booklets) is only administered to respondents having passed the core assessment. The booklets use a variety of materials, focusing on non-school-based items that are encountered in daily life and involve different types of tasks, including tasks that require respondents to access and identify information (in both text-based and non-prose materials such as tables, graphs and forms), to integrate and interpret information, and to evaluate information by assessing the relevance, credibility, or appropriateness of the material for a particular task. Items present varying levels of difficulty, with tasks ranging from locating a single piece of information in a very short advertisement to summarizing reasons for using generic drugs as presented in a newspaper article. Overall reading proficiency scores are reported on a scale ranging from 0 to 500, which is divided into 5 levels, with Level 1 characterized by the least demanding tasks and Level 5 by the most demanding. For each respondent, 10 plausible values were generated. 21 Table 1.1. Skills measured Cognitive skills Direct measurement of reading literacy based on  Reading proficiency the Survey of Adult Skills instruments Indirect assessment (self-reported) on individuals’  Reading use of foundational skills--at work or in daily life  Writing  Numeracy Socio- Personality traits  Openness emotional  Conscientiousness skills  Extraversion  Agreeableness  Neuroticism  Grit Behavior  Hostile attribution bias  Decision making Risk and time preference Job-relevant Qualifications required for the job and job learning times skills Indirect assessment of skills used at work  Computer use  Contact with clients  Solving and learning  Autonomy and repetitiveness  Physical tasks The indirect measures of cognitive skills include self-reported information on respondents’ use of cognitive skills in daily life and at work, namely if they read, write, or use mathematics, as well as the intensity of use, which in most instances is a proxy for complexity. For each of these cognitive skills, a score ranging from 0 to 3 is computed. When a respondent reports not using a given skill, the score is set at 0. For respondents who do use a given skill, intensity or complexity of use is defined (1 for low, 2 for medium, and 3 for high). Socio-emotional skills, sometimes referred to in the literature as non-cognitive skills or behavior skills, relate to traits covering multiple domains (such as social, emotional, personality, behavioral, and attitudinal). The survey builds on the “Big Five” personality traits: openness, conscientiousness, extraversion, agreeableness, and neuroticism (or its opposite, emotional stability). Measures of grit, which has been shown to have an impact on life outcomes, and hostile attribution bias are also included, as well as questions pertaining to how individuals make important decisions. Response categories range from 1 (“almost never”) to 4 (“almost always”). 22 Job-relevant skills are task-related and build on a combination of cognitive and socio-emotional skills. The STEP survey asks respondents about their use of such skills on the job, including, among others, computer use, repair and maintenance of electronic equipment, operation of heavy machinery, client contact, solving and learning, and supervision. For each skill, a score ranging from 0 to 3 is computed. When a respondent reports not using a given skill, the score is set at 0. For respondents who do use a given skill, intensity or complexity of use is defined (1 for low, 2 for medium, and 3 for high). The survey also includes self-reported information about the educational attainment required to do one’s job and the time required to learn how to do one’s job; these responses contribute to our understanding of what the education system is currently producing, what the current positions seem to demand, and how well they match. This report uses several other concepts which it is important to clarify here. First, it uses the term ECE to refer to participation in any form of pre-school education; participation does not necessarily mean actual enrollment rates, nor is anything implied about the quality of such education. Second, the socioeconomic status at age 15 is a self-reported variable, for which the survey asks each individual to rank his or her household’s economic well-being when he or she was age 15, using a score from 1 to 10. Third, the current wealth index is estimated using factor analysis on several non-income related assets and dwelling characteristics. Fourth, entrepreneurs are defined as all those who reported being self-employed, with or without employees; hence the report uses the terms entrepreneur and self-employed interchangeably. Lastly, in categorizing commercial sectors this report includes medium-high-innovation manufacturing and knowledge-intensive services under high innovation sectors and medium-low-innovation manufacturing and less knowledge-intensive services under low innovation sectors. The agricultural and construction sectors are classified as low innovation sectors. Methodology and limitations This report relies on a combination of three type of analysis: descriptive analyses, simple probability models, and earnings regressions. All of these are used throughout the report with different levels of intensity, and all the estimates reported are statistically significant unless noted to the contrary. The estimation is carried out with sample weighting, and the results are representative for the urban population in Armenia. The descriptive analyses present variable distributions and a range of cross-tabulations. For example, they include distributions of use and intensity of skills, and the scores of reading proficiency, which 23 are further cross-tabulated by gender, age cohort, and socioeconomic status. This type of analysis also includes illustrations of an individual’s skill profiles by: (i) socioeconomic status at age 15 and current household wealth index, and (ii) current education and education required for the job. The probability models are used to estimate the effects of participation in education and socioeconomic status on the acquisition of education and skills, and to model the effects of skills and education on employment participation. They control for different characteristics such as age, gender, mother’s and father’s education, own education, socioeconomic status at age 15, number of household shocks experienced by age 12, and in some cases for cognitive, socio-emotional and job relevant skills. The earnings regressions follow Bowles, Gintis and Osborne (2001) and Weinberger (2013) in order to estimate the returns to earnings and analyze how different skill sets contribute in conjunction with education. The objective of following these authors is to shed light on skills that are rewarded in the labor market but are not captured by the usual measures of educational attainment and to motivate the discussion of complementarity between cognitive and social skills. We acknowledge the limitations that this estimation procedure may have in terms of multi-collinearities, biases, and/or weak instruments. However, this is a first step towards exploring these issues in greater depth. 24 2. Labor Market and Demand for Skills in Armenia Unemployment and low productivity employment Although economic growth has started to improve in Armenia, a critical issue is that key labor market indicators still need to catch up. At 63 percent, Armenia has a labor force participation rate comparable to those of its neighbors—Georgia and Azerbaijan. Nonetheless, it still lags behind on some key indicators, such as unemployment. The unemployment rate in Armenia today is still quite high, even though it has come down from the recent peak of nearly 30 percent in 2008 (Table 2.1 and Table 2.2). Table 2.1. Labor force participation (% total) of Armenia, Azerbaijan, and Georgia 2005 2006 2007 2008 2009 2010 2011 2012 Armenia 60 60 59 59 59 62 62 63 Azerbaijan 64 64 64 64 65 65 65 66 Georgia 64 64 63 64 64 64 64 65 Source: World Bank, World Development Indicators 1990-2013 (2014b). Table 2.2. Unemployment rate (total) of Armenia, Azerbaijan, and Georgia 2005 2006 2007 2008 2009 2010 2011 2012 Armenia 28 25 28 29 19 19 18 19 Azerbaijan 6 7 7 6 6 6 5 5 Georgia 14 14 13 17 17 16 15 15 Source: World Bank, World Development Indicators 1990-2013 (2014b). Low-productivity employment is a distinctive challenge for the labor market in Armenia. Many existing jobs do not pay enough to lift people out of poverty. Nearly 40 percent of workers are employed in agriculture, while agriculture accounts for only 17 percent of GDP (Rutkowski 2012). This discrepancy between input and output implies that most agricultural jobs are of low productivity. The share of the non-agricultural sector in overall GDP has increased by less than one percentage point in the last five years. In the meantime, the modern, high-productivity sector is small, and it is only barely expanding. Informality is another correlate of low-productivity employment. According to official data from Armstat (the National Statistical Service of the republic of Armenia), informal employment accounted for nearly 20 percent of total non-agricultural employment (ADB and NSS 2011). However, 25 the estimates of the size of the informal sector are sensitive to the criteria used to identify informal employment. The criterion used by Armstat is the lack of a written employment contract. If one instead uses eligibility for statutory annual and sick leave as the criterion, informal employment represents as much as 43 percent of total non-agricultural employment in Armenia. On average, informal jobs pay less than formal jobs. The difference is small in the case of wage earners (4 percent), but it is large in the case of the self-employed: those with registered businesses earn over 90 percent more than those with unregistered ones. Moreover, non-agricultural low-earning jobs make up a high proportion of all jobs. Low-earning jobs are defined as those that earn less than two-thirds of the median wage. By this criterion, one in four jobs in Armenia is low-paid, which is significantly more than in most European countries, where the incidence of low pay is within the 15-20 percent range. Productivity gains have the biggest impact on poverty when achieved through the reallocation of labor from less to more productive uses. Such reallocation, associated closely with industrial restructuring, requires—first and foremost—a favorable investment climate. Creating modern and high-earning jobs in formal sectors can only benefit the economy if these jobs can be filled quickly by capable Armenian workers. It is therefore important for Armenia’s workforce to be equipped with the right skills to prepare for new and modern jobs as well as with skills that can foster entrepreneurship and innovation. In terms of educational attainment, Armenia stands out in the region with a high tertiary education enrollment of 46 percent in 2012. An important question then is whether this high educational attainment is translating into skills relevant for the labor market, as opposed to just diplomas (Sondergaard and Murthi, 2012). In fact, despite the high availability of labor and these high educational levels, Armenia’s employers are struggli ng to find the right workers, which seems to point to a problem of skills in the labor force. Skills in demand The skills gap is indeed significant in Armenia today. The STEP Employer Survey examines a range of issues in the demand for skills in Armenia’s labor market. The survey looks particularly closely into the various types of skills—cognitive skills, job-related skills, as well as socio-emotional skills—and their relevance to the labor market. Employers among firms in the survey are demanding a wide range of types of workers with varying 26 levels of skills. Over 40 percent of new hires are highly skilled white collar workers, followed by crafts and related workers (blue collar), who make up over 30 percent of the distribution. Employers are also planning to increase their employment mainly in blue collar occupations. Over 12 percent of the firms answered that they were planning to hire operators and craft workers over the next 12 months, while only less than 4 percent were planning to hire highly skilled white-collar workers (Figure 2.1). Figure 2.1. Growing demand for manual skills and limited demand for professional skills Planned employment growth over next 12 months Operators Craft workers Technicians Elementary Clerks Sales workers Service workers Professionals Managers 0 5 10 15 percent Source: Rutkowski, 2013. However, employers are finding difficulty filling job vacancies, and they point to the lack of required skills as the predominant reason. The evidence suggests that around 90 percent of the firms have problems when hiring either highly skilled white-collar workers or low-skilled white-collar or craft and related workers (Figure 2.2). Employers also pointed out that it is not only the lack of technical skills that is hampering filling their vacancies, but also a lack of socio-emotional skills. Figure 2.2. Employers find it difficult to hire skilled workers 27 Percentage of firms that had problems hiring Craftsmen Technicians Service workers Sales workers Professionals Elementary Operators Managers Clerks 0 10 20 30 40 50 60 percent Source: Rutkowski, 2013. Not surprisingly, Armenian employers are critical of the quality of the skills possessed by the current workforce. Modern employers (firms that have international business contacts or have introduced new technology) are more likely than traditional employers to see the education of the labor force as a major problem. Generally, all employers are critical of the quality of education. More particularly, firms are even more critical of vocational education than of general education. Roughly two out of three employers stated that technical and vocational education does not meet their skill needs. These responses show that there is a disconnect between education and the skills needed at the workplace (Figure 2.3). Figure 2.3. Employers’ views of the adequacy of technical and general education Percentage of employers that claim that education... Meets skill needs Doesn't produce practical skills Doesn't produce updated knowledge Doesn't produce LEVEL of skills needed Doesn't produce KINDS of skills needed Doesn't produce good attitude 0 10 20 30 40 50 60 70 80 90 percent Technical/vocational education General education Source: Rutkowski, 2013. 28 Most firms look for job-related skills, particularly technical skills for both white-collar and blue-collar workers. Employers were asked what type of skills are the most important ones when deciding to retain an employee. Among employers, the most important skills cited are cognitive and job-related skills. Socio-emotional skills are important too, but much less so than job-related skills. The least important factor, from the employers’ viewpoint, is personal characteristics (such as gender, age, appearance, etc.), although those factors can influence hiring decisions in some situations (Figure 2.4). This view applied equally among employers regarding both white- and blue-collar workers and for small, medium, and large firms. Employers also reported that job-specific skills are somewhat more important for type A workers (highly skilled/white-collar workers) to have than type B workers (less skilled/blue-collar workers), whereas socio-emotional skills and personal characteristics are more important for type B workers who interact directly with customers than for type A ones. It appears from this that white-collar workers might be hired specifically for their technical skills, while blue-collar workers are expected to use their socio-emotional skills much more. Socio-emotional skills and personal characteristics cannot be taught in a training session the way job-specific skills can. Figure 2.4. Employers’ ranking of the importance of skill groups, by type of worker Importance of skill groups (Index 0 - 2) 2 1.8 1.6 1.4 1.2 1 Type A 0.8 0.6 Type B 0.4 0.2 0 Cognitive and job related Socio-emontional skills Personal characteristics skills Source: Rutkowski, 2013. Note: Index = 2.0 if all employers point to a given skill as the most important, and the index = 0 if no employer points to the skill as important. Among job-related skills, cognitive and technical skills are the most valued skills for white- and blue- collar workers. Employers highly value numeracy, literacy, teamwork skills, and problem-solving skills among white-collar workers; while for blue-collar positions they are more concerned with skills in communication, teamwork, ability to work independently, and time management (Figure 2.5). This 29 seems to suggest that workers in Armenia need both cognitive and socio-emotional skills, but that highly skilled jobs require workers with adequate cognitive skills. Among socio-emotional skills, conscientiousness is the skill employers’ value most for both white- and blue-collar workers. Conscientiousness includes elements of responsibility, self-discipline, carefulness, and motivation. This skill has come up as a desirable skill among many employers in countries including Macedonia, Poland, Russia, and the United Kingdom (World Bank, 2013a, p. 63). This suggests that workers who are hard-working and do a thorough job are more likely to be hired than workers who do not. For white-collar workers the second most desirable personality trait is openness to experience, followed by emotional stability, while for blue-collar workers the second most desirable characteristic after conscientiousness is emotional stability, followed by agreeableness and extraversion. Figure 2.5. Employers’ ranking of the job-related characteristics, by type of worker Importance of job-related characteristics (Index 0 - 5) 3.5 3 2.5 2 1.5 1 Type A 0.5 0 Type B Source: Rutkowski, 2013. Note: Index = 5.0 if all employers point to a given skill as the most important, and the index = 0 if no employer points to the skill as important. Overall, there is clear indication from the feedback of Armenian employers that the demand for highly-skilled workers is already significant for both white- and blue-collar occupations. However, the graduates today do not seem to be equipped with adequate skills. The highly desired skills include cognitive and job-related skills, followed by socio-emotional skills or character and behavioral skills. Addressing the skills challenge and narrowing the skills gap will require a fundamental understanding of the existing systems of skills formation and utilization, together with returns to skills and related market mechanisms and signals for the acquisition of the right type of skills. The 30 next few chapters will provide insights in these key aspects of the subject. 31 3. Step 1 | Getting Children Off to the Right Start in Armenia Skills developed in early childhood—from birth to entry into primary school—form the basis of future learning and labor market successes. This section reviews trends in participation in early childhood education (ECE) programs across the age cohorts of the adult Armenian population. It analyzes the relationships between participation in ECE and use of foundational and job-relevant skills, including computer and solving and learning skills. This section focuses on the following two questions: (i) Has participation in ECE2 programs gained ground overtime? and (ii) Is participation in ECE paying off? To answer these questions, this section uses a combination of descriptive analyses and simple probability models to estimate the effect of participation in ECE.3 The results presented here are statistically significant, after controlling for gender, educational attainment, mother’s education, father’s education, number of household shocks experienced by age 12, household socioeconomic status at age 15, and indicator variables for age groups. It is important to stress that the survey provides a cross-sectional picture of the population at a given point in time. As such, it is not possible to observe age cohort trajectories over time or to move beyond establishing correlations. Has participation in early childhood education programs gained ground over time? More adults have participated in ECE programs over time in Armenia. Almost 60 percent of adults ages 15 to 64 reported having participated in some form of organized early childhood education program (e.g., kindergarten, crèche, daycare, nursery school, etc.) before entering first grade. Although there are no significant differences in participation between men and women, there are significant differences across age cohorts. Participation peaked at 78 percent for the 25-to-34 age cohort compared to about 35 percent for the 55-to-65 age cohort. It is of concern, however, that the upward trend has reversed recently, reaching only 65 percent of the 15 to 24 age cohort (Figure 3.1). Figure 3.1. Participation in ECE by gender and age cohort 2 Participation in ECE is measured broadly (e.g., it includes kindergarten, crèche, daycare, nursery school before entering first grade); hence it is different from gross or net enrollment in preschool education. 3 The questionnaire does not capture the length or intensity of early education services, making it difficult to estimate quality. 32 Armenia adult urban population 100% % Adults participating in ECE 80% 60% program 40% 20% 0% Male Female 15-24 25-34 35-44 45-54 55-65 Total Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. Participation in ECE programs is correlated with household socio- economic status4. About 60 percent of adults who lived in high socioeconomic status households at age 15 participated in early education compared to 50 percent of adults living in low socioeconomic status households (Figure 3.2). Figure 3.2. Participation in ECE by household socioeconomic status at age 15 Armenia adult urban population 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Low SES Middle SES High SES Did not attend ECE Attended ECE Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. 4 For constructing past socioeconomic status of the household, the STEP surveys asked individuals to rank their household’s socioeconomic status when they were 15 years old from 1 to 10. Similarly to construct a wealth index for the household at the time of the survey was administered, the survey included information from different types of assets and dwelling characteristics which was compiled into an index using factor analysis methods. Robustness checks rendered the variables reliable. 33 Is participation in early childhood education paying off? Adults who participated in ECE programs are more likely than other adults to score higher in socio- emotional skills. Participation in early education programs is associated with a higher probability of scoring higher on extraversion, conscientiousness and grit. For example, adults who participate in ECE are more likely to get a higher grit score (3.16) compared to adults who did not participate and whose grit scores are likely to be lower (3.11) (Figure 3.3). The differences are statistically significant after controlling for gender, educational attainment, mother’s education, father’s education, number of household shocks experienced by age 12, household socioeconomic status at age 15, and indicator variables for age groups. This finding in Armenia is consistent with the international literature. These traits—in addition to others—are associated with “character skills” that matter for a range of outcomes in life, including employability and labor market outcomes (Heckman and Kautz, 2013). The finding is also consistent with research from Duckworth et al. (2007) which indicates that grit—broadly defined as perseverance and passion for long-term goals—is a trait capable of predicting long-term success. According to Duckworth’s research, individuals who score high on grit tend to work tirelessly through challenges, failures, and adversity to achieve goals, making them more likely to achieve their goals when compared to individuals who lack similar stamina. Similarly, conscientiousness is defined as being thorough, careful, efficient and organized. Conscientious people have a tendency to show self-discipline, act dutifully, and aim for achievement; and they display planned rather than spontaneous behavior (Costa and McCrae, 1992; Goldberg, 1992). Prevo and ter Weel (2003) applied measures of conscientiousness at age 16 to explain adult wages and other outcomes, including crime, health and savings behavior. Their estimates suggest a significant and sizeable correlation between early conscientiousness and adult outcomes. Goodstein and Lanyon’s (1999) systematic review of the literature indicates that the Big Five factors of personality can predict job performance and may be used for staffing selection. Barrick and Mount’s (1991) analyses show conscientiousness as having a consistent relationship with all the job performance criteria they evaluated (job proficiency, training proficiency, and personnel data) for all occupational groups studied (professionals, police, managers, sales, and skilled/semi-skilled). 34 Figure 3.3. Probability of grit and openness scores by ECE participation Armenia adult urban population 3.3 3.25 3.2 Predicted Score 3.15 3.1 3.05 3 2.95 2.9 2.85 Extraversion Conscientiousness grit Did not attend ECE Attended ECE Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. Adults who participated in ECE programs are also more likely to use their reading skills regularly. Adults who participated in ECE programs tend to use their reading skills more often and with higher intensity (i.e., read longer documents) compared to those who did not participate in early education (Figure 3.4). Figure 3.4. Participation in ECE and use of reading skills Armenia adult urban population 50% 40% 30% 20% 10% 0% Reading skills NOT Low intensity Medium intensity High intensity used Did not attend ECE Attended ECE Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. ECE participants also have a higher probability of using computers at work. Adults who participated in ECE programs have a higher probability of using their computer skills at work and of using them with higher intensity (every day). Fifty-four percent of adults who participated in early education programs reported using computer skills at high intensity, compared to 50 percent of adults who did not participate (Figure 3.5). Although the differences are small in magnitude, they are statistically significant. 35 Figure 3.5. Probability of using computer skills at work by ECE participation Armenia adult urban population 60% 50% 40% 30% 20% 10% 0% Not using computer skills High intensity Did not attend ECE Attended ECE Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. These findings reaffirm the potential payoffs of expanding ECE programs in Armenia, something that has already been identified by the Government of Armenia and the World Bank as one of the top priorities in the education sector. The recent Education Improvement project (EIP) (Box 3.1)—partly financed by the World Bank—provides increased funding support to the ECE programs in the country and aims to improve school readiness in young children, including the development of cognitive and socio-emotional skills. Box 3.1. Investing in early childhood education and development in Armenia The newly approved Education Improvement Project (World Bank, 2014c) supports government efforts to promote children’s school readiness. The project finances interventions to increase access, enrollment, and quality of preschool education services in Armenia. Approximately 2,400 five- and six-year-old children from vulnerable households are expected to benefit from the project every year. Ultimately, the project seeks to increase enrollment from the current 75 percent to 90 percent by 2017. The project is in line with research by economist James Heckman, a Nobel laureate. Heckman has praised early investment in disadvantaged children as “a rare public policy with no equity-efficiency tradeoff” because it promotes social justice and fairness by reducing inequality and at the same time raising society’s productivity. Both medical and economics literatures agree that gaps in cognitive and non-cognitive skills emerge early in life and persist over the life cycle. Policies to mitigate these gaps later in life have proven to be less cost-effective. Source: World Bank, “Armenia Education Improvement Project (EIP),” Project Appraisal Document, February 2014 (p. 58). 36 Highlights of findings  Early childhood education programs in Armenia have expanded over time, although the trend may have begun reversing recently.  Participation in early childhood education programs in Armenia is positively associated with higher socio-emotional scores and with higher reading and computer skills at work—and both skill sets are linked to better labor market outcomes.  Early childhood education programs in Armenia are more readily available to individuals living in households with a high socioeconomic status compared to individuals from low-income households. Narrowing the gap will be essential to ensuring equitable opportunities for skills development. 37 4. Step 2 ǀ Ensuring that All Students Learn in Armenia Schools are expected to teach basic competencies that enable students to acquire the skills they will need to make informed life choices and that will later be valued by employers and useful for self- employment. These competencies include cognitive skills such as reading, writing, and numeracy; socio-emotional skills, including the Big Five personality traits—openness, emotional stability, conscientiousness, agreeableness and neuroticism; and job-relevant skills, such as techniques in information technology, communications, problem-solving, and teamwork. This section provides insights on trends in educational attainment, the use of foundational skills (reading, writing, and numeracy), and the current level of reading proficiency in the Armenian adult population using a reading assessment module developed to capture reading ability. It also explores the roles played by past socioeconomic status, current household wealth status, and parental background in influencing the use of foundational and job-relevant skills. The section addresses the following questions: (i) Is educational attainment translating into high- quality and job-relevant skills? (ii) What is the role of past socioeconomic status on education attainment and skill acquisition? (iii) Do early skills gaps translate into current skills gaps? and (iv) How do parental education and household resilience to shocks affect skills outcomes? To answer these questions, this section uses descriptive analyses and estimates simple probability models to determine the effect of socio-emotional skills on educational attainment, use of job- relevant skills, and gaps in skills acquisition. These analyses further take into consideration past household socioeconomic status and current household wealth index. The results presented here are statistically significant, after controlling for gender, mother’s education, father’s education, number of household shocks experienced by age 12, and indicator variables for age groups. Is educational attainment translating into high quality and job-relevant skills? Armenia has a highly educated population. More than half of all adults hold a tertiary education - degree. Over 85 percent of adults have completed at least senior upper-secondary education, and about 60 percent hold a tertiary education degree. The high level of educational attainment had already been achieved over a generation ago and has remained stable since—as illustrated by the 38 relatively high percentage of tertiary graduates in several consecutive age cohorts. Few differences exist between women and men, although women make up a slightly higher proportion of individuals with tertiary education degrees (Figure 4.1). Figure 4.1. Educational attainment by gender and age cohort Armenia adult urban population 100% 80% 60% 40% 20% 0% Men Women 25-34 35-44 45-54 55-65 Primary or Less Lower Secondary Upper Secondary Tertiary Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. Educational attainment is correlated with the use of foundational and job-relevant skills and with the use of these skills at a higher intensity level. Adults who have completed tertiary education are more likely to use reading, computer, and problem-solving and learning skills regularly than adults who have only completed upper secondary education. While the use of numeracy is nearly universal across educational attainment, it is important to highlight that a sizable proportion of adults— ranging from 31 to 45 percent—do not use reading, computer, or problem-solving and learning skills regularly. Tertiary education graduates are more likely to use reading and computer skills at medium or high intensity, that is, every day (Figure 4.2). 39 Figure 4.2. Proportion of adults using reading, numeracy and computer skills by intensity level and educational attainment Armenia adult urban population Problem solving Readin Computer and learning Not using skills 100% 80% low 60% 40% medium 20% 0% high Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. 40 Box 4.1. The education system in Armenia The formal education system in Armenia follows a 4-8-4 structure. Basic education comprises four years of primary schooling. Eight years of secondary education follows, consisting of five years of lower secondary and three years of either upper secondary education (high school) or primary vocational (craftsmanship) education. Post-secondary education for a bachelor’s degree is normally four years. (See figure below.) Education is free and compulsory until the age of 16. After the country’s independence from the Soviet Union, the main language of instruction became Armenian, and the curriculum changed as well. Literacy rates in Armenia have remained high since independence —nearly 99 percent. Enrollment rates at primary and secondary levels of education have generally increased but are far from universal. The adjusted net enrollment rate, which includes all students who are enrolled in primary and secondary school, was 92.9 percent in 2007, which is an increase from 81.6 percent in 2004. The net secondary enrollment rate was 83 percent in 2012, which is a drop from its peak during 2000 –12 at 90 percent in 2007 (UIS, 2014). The gross enrollment ratio for tertiary increased from 35 percent in 2000 to 51 percent in 2011, although the ratio dropped drastically to 46 percent in 2012 (World Bank EdStats, 2014). The transition period may have harmed the education system, partly due to a reduction of public spending in education. National government investments in education are low. In 2012, public expenditure on education was 3.3 percent of GDP, lower than the OECD-recommended 5 percent of GDP, and it made up 14 percent of total government expenditure (UIS, 2014). 41 Young adults are the persons most likely to use reading and computer skills, and they tend to use them more intensively. The use of these two skills is positively associated with decreasing age— young adults (in the 15 to 24 age cohort) report the highest use of skills. For other job-related skills such as problem solving and learning at work, adults between age 25 and 34 report the highest use. This finding is likely related to the fact that older adults are more likely to be employed and tap into their accumulated experience to solve problems. Over half of adults between age 15 and 24 use reading skills at the high-intensity level, compared to one-third of adults age 25 or above. The pattern is the same when it comes to computer skills, with the youngest cohorts using computer skills twice as intensively as the oldest cohort (Figure 4.3). Figure 4.3. Proportion of adults using reading, computer, and problem solving and learning skills by intensity level Armenia adult urban population Problem solving Reading Computer and learning 100% 80% 60% 40% 20% 0% 15-24 25-34 45-54 15-24 25-34 45-54 15-24 25-34 45-54 Not using skills low medium high Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. The STEP household survey administered a reading assessment instrument to evaluate adults ’ reading proficiency in depth. The assessment instrument uses a variety of materials, focusing on non- school- based items encountered in daily life. Reading proficiency is grouped into five levels: levels 0/1 to level 5. This five-level reading competency scale is consistent with international benchmarks of reading proficiency (OECD, 2013). An advantage of the five-level scale is that it also provides an assessment of what an individual at each level is capable of in terms of reading proficiency (see Technical Annex 2). For example, adults scoring below level 1 perform at a very basic level and are not required to understand the structure of sentences or paragraphs or make use of other text features. In contrast, adults scoring at level 5 are able to search and integrate information across multiple, dense texts, construct syntheses of 42 similar and contrasting ideas, or evaluate evidence-based arguments. Adults in urban areas in Armenia score lower on average than adults in OECD countries as a whole. Looking at the distribution of reading proficiency, almost 65 percent of adults in urban Armenia are clustered in levels 2 or below, compared to 38 percent in high-performing countries such as Finland (Figure 4.5). Figure 4.4. Distribution of reading literacy proficiency by level for OECD countries (national) and Armenia (urban) Georgia** Armenia** Republic of Korea* Poland* Finland* OECD Average* 0% 20% 40% 60% Total Levels 4 and 5 Total Level 3 Total Level 2 Total Level 0/1 Source: World Bank, Armenia Step Skills Measurement Survey, 2013; and OECD, Programme for the International Assessment of Adult Competencies (PIAAC), 2013. * PIAAC levels, representative of national populations of adults ages 15-65. ** STEP levels, representative of urban areas for adults ages 15-64. The widest gap between urban Armenia and the national OECD average is by educational attainment, especially at the upper secondary level (a −14 point difference in test scores) and tertiary level (a −31 point difference). The only exception is for adults with primary or less than primary education in Armenia, whose score is 29 percentage points higher than the OECD national average for adults with the same level of education (Figure 4.5). 43 Figure 4.5. Average reading literacy score in OECD countries (national) and Armenia (urban) by gender and education attainment 325 300 275 OECD 250 Average 225 (national) 200 175 Armenia 150 (urban) Tertiary Men Women Upper Secondary All Lower Secondary Primary or Less Source: World Bank, Armenia STEP Skills Measurement Survey, 2013; and OECD, Programme for the International Assessment of Adult Competencies (PIAAC), 2013. Note: PIAAC scores for OECD countries are representative of national populations of adults aged 15-65. Adults with more education and from younger cohorts score higher in reading proficiency. In Armenia, about 12 percent of adults score at level 1 or below, 53 percent score at level 2 and 33 percent score at level 3. Only 2 percent of adults score beyond level 3. While tertiary graduates outperform all other groups, their average scores are still clustered in levels 2 and 3. Only 3 percent score at levels 4 and above compared to 24 percent for the OECD national averages. It should be noted the STEP data provides a snapshot in a particular point in time. As such, differences in reading proficiency across age cohorts may be linked to a range of factors, including decline in skills, less schooling or differences in content of schooling across older groups. This issue may be explored in greater depth by looking at cohort analyses. 44 Figure 4.6: Distribution of reading literacy proficiency by level, gender, age, and educational attainment Armenia adult urban population 100% 80% 60% 40% 20% 0% Total Men Women Primary or Lower Upper Tertiary Less Secondary Secondary Level 4 and 5 Level 3 Level 2 Level 1 or below Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. Socio-emotional skills such as conscientiousness are positively associated with reading proficiency. Adults who score high in conscientiousness have a higher probability than other adults of scoring at the highest levels of reading proficiency—levels 3 to 5 (Figure 4.7). The results are statistically significant when controlled for education, gender, current household wealth, mother education, number of household shocks experienced by age 12, socioeconomic status at age 15, other socio- emotional skills, type of school, and school proximity. Figure 4.7. Predicted score in conscientiousness (scale of 1-4) by reading level Armenia adult urban population 60% 50% 40% 30% 20% 10% 0% Level 0/1 Level 2 Level 3 Level 4/5 1 2 3 4 Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. Educational attainment is associated with higher scores in socio-emotional skills such as grit and decision-making. The probability of scoring higher in grit and decision-making increases with educational attainment, the contrast being starkest between adults with primary education or less 45 and those with completed tertiary education. For example, the predicted grit score for adults with primary education or less is 2.8, as compared with 3.2 for adults with tertiary education (Figure 4.8). The predicting model controls for education, gender, current household wealth, mother’s education, number of household shocks experienced by age 12, household socioeconomic status at age 15, type of school, and school proximity. Figure 4.8. Predicted score in grit and decision making by educational attainment Armenia adult urban population 3.4 3.3 3.2 3.1 3 2.9 2.8 2.7 2.6 Primary or less Lower Secondary Upper Secondary Tertiary Grit Decision making Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. Openness is positively associated with the use of solving and learning skills at work. The higher the openness score—ranging from 1 to 4—the greater the likelihood of using problem solving and learning skills at work and of using them with high intensity (that is, every day). The results are statistically significant and they control for education, gender, current household wealth, mother’s education, number of household shocks experienced by age 12, socioeconomic status at age 15, other socio-emotional skills, type of school, and school proximity. 46 Box 4.2. Results from the 2012 Trends in International Mathematics and Science Study, Armenia The Trends in International Mathematics and Science Study (TIMSS) is an international exam that assesses, every four years, how well students are performing in math and science. Armenia fares much better in math than in science TIMSS scores, but some recent trends are worrisome. Overall, Armenian students’ results in math (mean score of 467) were better than in science (mean score of 437) for 8th grade in the 2011 exam. However, Armenia has a higher share of students scoring at the “below low” proficiency level in math and science than OECD and EU averages. What is even more worrisome is that the percentage of students in Armenia in the “below low” category in math increased from 18 to 24 percent between 2003 and 2011, while the proportion of students in the “advanced” category increased only by 1 percentage point (see Figure B.1). Figure B.1. TIMSS Scores in Math and Science, 2003 and 2011 500 478 467 461 456 452 TIMSS points 450 437 437 416 2003 400 2011 350 4th grade 8th grade 4th grade 8th grade Math Science Source: TIMSS, 2011. Note: TIMSS has a mean of 500 points and a standard deviation of 100 points. Analysis suggests that students from less well-off families may have been the most affected by the drop in test scores. Disaggregating trends in achievement suggest that test scores dropped for a number of groups. Test scores dropped for students in different household wealth quintiles, using the number of books at home as a proxy variable. However, the highest drop in test scores (44 points) between 2003 and 2011 can be seen for students whose parents have less than lower-secondary completed; this is in stark contrast to students whose parents have higher levels of education, where the drop was only 3 points. The drop in test scores affected students in medium-size cities (50k - 100k people) the most, although it seems to have been gender neutral. 47 What is the role of past socioeconomic status in skill acquisition? In determining patterns in skills acquisition and reading proficiency outcomes by household socioeconomic status, this section uses a variable scaling the socioeconomic status of the household when the respondent was 15 years old (individuals were asked to rank their household’s socioeconomic status from 1 to 10), as well as a household wealth index constructed using information on current household dwelling characteristics and assets. Robustness checks were carried out to ensure the viability and precision of the indicators. Using these variables, we explore the extent to which the education and skills profiles of respondents who were living in low- socioeconomic-status households at age 15 but are living in high-wealth-index households at present are significantly different from the profiles of respondents who were living in low-socioeconomic- status households at age 15 and remain in low-wealth-index households at present. Past household socioeconomic status is associated with the use of reading and computer skills and with using problem-solving and learning skills at work. Adults who lived in high-socioeconomic- status households at age 15 are more likely to use computer and solving at learning skills at work and to use those skills more intensively compared to adults who lived in low-socioeconomic-status households at the age of 15 (Figure 4.9). All results are statistically significant controlled for education, gender, current household wealth, mother’s education, number of household shocks experienced by age 12, socioeconomic status at age 15, socio-emotional skills, type of school, and school proximity. Figure 4.9. Distribution of reading, computer and solving and learning skills by past household socioeconomic status Armenia adult urban population Reading Computer Problem solving & learning 100% 80% 60% 40% 20% 0% Low SES High SES Low SES High SES Low SES High SES skill not used low medium high 48 Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. Adults who once lived in high-socioeconomic-status households at the age of 15 score higher on socio-emotional skills such as conscientiousness and openness. Scores for these socio-emotional skills are higher for adults who grew up in high-socioeconomic-status households. For example, scores in openness averaged 3.27 for adults from high-socioeconomic-households, compared to 3.15 for adults in low-socioeconomic-status households (Figure 4.10). Figure 4.10. Average scores in conscientiousness and openness by past socioeconomic household status Armenia adult urban population Conscientiousness Openness 3.3 3.25 3.2 3.15 3.1 Low SES Middle High SES Total Low SES Middle High SES Total SES SES Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. Do early skills gaps translate into current skills gaps? As noted above, skills acquisition and reading proficiency outcomes are positively correlated with the household’s socioeconomic status in the past. The question that remains to be explored here is whether adults who did not transition from low socioeconomic status have a different skill profile than those adults who managed to transition into middle or high socioeconomic status. Figure 4.11 shows the distribution of respondents by socioeconomic status at age 15 and by household wealth index at present. The graph shows an increase in the proportion of respondents falling into the low-socioeconomic-status category. The trend may be explained by a range of factors, including the economic shifts that followed the political, social, and economic transitions in the region in the 1990s, as well as the most recent global financial crisis. Recent estimates show that the poverty rate increased from 28 percent in 2008 to 35 percent in 2011. Although it increased most acutely in rural areas, it also affected urban centers, and inequality (as measured by the Gini coefficient) has also increased—from 0.24 to 0.27—over the same period (World Bank, 2014). 49 Figure 4.11. Distribution of respondents by household SES at age 15 and household wealth index now Armenia adult urban population 100% 80% 60% 40% 20% 0% HH socio-economic status at age 15 HH wealth index now Low Medium High Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. Adults who remained in low-socioeconomic-status households have generally low educational attainment and use problem solving and learning skills less than others do. About 42 percent of adults who transitioned out of a low-socioeconomic-status household have tertiary education, compared to 31 percent for those who remained behind. Adults who transitioned into high-wealth- index households used solving and learning skills more often and more intensively compared to adults who did not transition out of low socioeconomic status (Figure 4.12 and Figure 4.13). Figure 4.12. Distribution of respondents by household SES at age 15 and household wealth index now, by educational attainment Armenia adult urban population LOW SES HH age 15 ● HIGH wealth index HH now LOW SES HH age 15 ● MEDIUM wealth index HH now LOW SES HH age 15 ● LOW wealth index HH now 0% 20% 40% 60% 80% 100% Tertiary Upper Secondary Lower Secondary Primary or Less Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. 50 Figure 4.13. Distribution of use of problem-solving and learning skills by household SES at age 15 and household wealth index, now Armenia adult urban population LOW SES HH age 15 ● HIGH wealth index HH now LOW SES HH age 15 ● MEDIUM wealth index HH now LOW SES HH age 15 ● LOW wealth index HH now 0% 20% 40% 60% 80% 100% High use Medium use Low use Not using skill Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. However, there are few differences in reading proficiency outcomes between adults who remained in low socioeconomic status and those whose households have made an upward transition. The distribution of reading proficiency scores is similar across these groups of households of varied social mobility, with most respondents clustering in levels 1/0 or 2 (Figure 4.14). Figure 4.14. Distribution of reading efficiency by household SES at age 15 and household wealth index now, by reading proficiency level Armenia adult urban population LOW SES HH age 15 ● HIGH wealth index HH now LOW SES HH age 15 ● MEDIUM wealth index HH now LOW SES HH age 15 ● LOW wealth index HH now 0% 20% 40% 60% 80% 100% Level 4/5 Level 3 Level 2 Level 1/0 Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. 51 How do parental education and household resilience to shocks affect skills outcomes? Parental educational attainment is associated with reading proficiency. Adults whose mothers attained only lower-secondary or less education score lower on reading proficiency (at levels 0 through 2) compared to adults whose mothers attained tertiary education (Figure 4.15). The results are statistically significant, controlling for education, gender, current household wealth, mother education, the number of shocks at age 12, socioeconomic status at age 15, other socio-emotional skills, type of school, and school proximity. Figure 4.15. Probability of scoring at higher levels in reading proficiency by mother’s educational attainment Armenia adult urban population 60% 50% 40% 30% 20% 10% 0% Level 0/1 Level 2 Level 3 Level 4/5 Secondary Tertiary Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. Higher parental education attainment is also associated with a higher probability of using problem- solving and learning skills at work. Adults whose mothers completed secondary or tertiary education have a higher probability of using solving and learning skills at work compared to adults whose mothers completed less than primary education (Figure 4.16). The model controls for education, gender, current household wealth, mother’s education, number of household shocks experienced by age 12, socioeconomic status at age 15, other socio-emotional skills, type of school, and school proximity. 52 Figure 4.16. Adults’ probability of using solving and learning skills by mother’s educational attainment 50% Armenia adult urban population 40% 30% 20% 10% 0% Not using solving Low Medium High & learning skills Less than primary Secondary Tertiary Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. Finally, household vulnerability at age 15 is associated with a lower probability of using reading skills. Adults who grew up in households that experienced severe shocks (such as death of a member, family break up, employment loss, natural disaster, violence or displacement) had a lower probability of using reading skills and of using them with higher intensity compared to adults from households that experienced one or no shocks (Figure 4.17). The results for more than one shock are statistically significant, controlling for education, gender, current household wealth, mother’s education, number of household shocks experienced by age 12, socioeconomic status at age 15, other socio- emotional skills, type of school, and school proximity. Figure 4.17. Adults’ probability of using reading skills by household vulnerability at age 15 (measured in number of shocks experienced) 50% Armenia adult urban population 40% More than one HH shock 30% 20% One HH shock 10% 0% Not using Low Medium High No HH shocks Reading skills experienced Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. 53 Highlights of findings  Overall, educational attainment is high among Armenia’s urban population. Nonetheless, the average reading proficiency remains relatively low by international standards. Most adults in Armenia score at level 2 or below, as compared to levels 2 to 3, which is the average for OECD countries in the PIAAC assessment.  The score gap between urban Armenia and OECD averages is widest when comparing the difference among adults between those with upper-secondary and those with tertiary education.  There is a positive association between educational attainment and o use of foundational and job-relevant skills; o scores in socio-emotional skills such as grit and decision-making; o scores in reading proficiency.  The socioeconomic status of the parental household is associated with adults’ use of reading and job-relevant skills, and some of the early skills gaps remain over time.  Parental education and household protection from shocks are positively associated with foundational, socio-emotional, and job-relevant skills. The higher the educational attainment of the parents and the fewer the shocks experienced in the parental household, the more use there is (as an adult) of foundational and job-relevant skills and the higher the scores in socio-emotional skills. 54 5. Step 3 | Building Job-Relevant Skills in Armenia The education system and the household play a pivotal role in improving the labor market performance of individuals, as shown in the previous section. Both the educational system and the household provide solid foundational skills as well as job-relevant skills that enhance the individual’s employability and productivity. Notwithstanding the fundamental role that education plays in equipping people with appropriate skills, the process of skills acquisition can continue over a lifetime. As such, continuing education and training programs as part of the workforce development system are important to ensure that workers maintain or upgrade their skills to keep up with changing business and production processes and, more broadly, with technological changes altering the workplace environment. This section discusses how skills matter and shows that they matter at least as much as education for the labor market. It lays out evidence regarding how (and which) skills pay off in Armenia in terms of labor force participation, employment and earnings. It also examines to what extent training or other forms of accreditation, such as certification or apprenticeships, can help in skills acquisition. In more detail, this answers the following three questions: (i) To what extent can skills help explain the labor force participation and employment? (ii) How are different skills sets, in conjunction with education, contributing to workers’ earnings? and (iii) Is training providing value-added to workers in terms of skills? The analysis carried out in this section involves descriptive statistics and two econometric models. The first one is a probability model for two outcome variables: labor force participation and employment. The model controls for cognitive skills (mainly, literacy proficiency scores), socio- emotional skills, computer use, household related variables, dropout, age, mother education, father’s education, socioeconomic status at age 15, and the number of household shocks at age 12. The second model follows Bowles, Gintis, and Osborne (2001) and Weinberger (2013) in order to estimate the returns to earnings and to analyze how different skills sets contribute in conjunction with education. The model controls for experience, gender, years of education, literacy score, socio- emotional skills, computer use, contact with clients, solving and learning new things, autonomy, economic sector, occupation, and (using an indicator variable) whether one is a wage worker. All the results presented here are statistically significant unless noted otherwise. We acknowledge the possible limitations of the estimation procedure, in terms of multi-collinearities and measurement 55 and/or omitted variable biases. However, we expect this to be a first step towards exploring these issues in greater depth. To what extent can skills help explain the labor force participation and employment? In the absence of a direct measurement of skills, there is vast evidence on the relationship between educational attainment and labor force participation and employment in Armenia. For instance, the probability of actively participating in the labor force increases with educational attainment, rising from 39 percent for those with primary or less education to 65 percent for those with tertiary education. Education also increases the probability of being employed—which rises from around 55 percent for those with upper-secondary education to 70 percent for those with tertiary education (Figure 5.1). Figure 5.1. Labor force participation and employment distribution by educational attainment Armenia adult urban population 100% 80% 60% 40% 20% 0% Primary or Lower Upper Tertiary Primary or Lower Upper Tertiary Less Secondary Secondary Less Secondary Secondary Labor Force Participation Employment Active Employed Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. Additionally, direct measures of skills such as reading proficiency suggest that they have a stronger association with labor force participation and employment than educational attainment alone. The chances of actively participating in the labor force are higher across the reading proficiency levels when compared to educational attainment, except for tertiary education. Probability of active participation is around 53 percent for reading level 1 or below, 58 percent for levels 4 and 5. Similarly, the probabilities of being employed also increase with reading proficiency and are consistently higher than those based on educational attainment alone, rising from 63 percent for reading level 1 56 or below to 76 percent for levels 4 and 5 (Figure 5.2). Figure 5.2. Labor force participation and employment distribution by reading proficiency levels Armenia adult urban population 100% 80% 60% 40% 20% 0% Level 1 Level 2 Level 3 Level 4/5 Level 1 Level 2 Level 3 Level 4/5 and and below below Labor Force Participation Employment Active Employed Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. Labor force participation and employment share vary across age groups and proficiency levels. There are no large differences in labor force participation between reading proficient levels 0 and 3 across the age range of 25 to 65 years, although the employment share of those participating dips for those between ages 35 and 44. However, for the high performers (level 4/5), both labor force participation and employment share increase with age. (Figure 5.3) Figure 5.3. Labor Force participation and employment share by age group and reading proficiency levels Armenia adult urban population 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 15-24 25-34 35-44 45-65 15-24 25-34 35-44 45-65 15-24 25-34 35-44 45-65 15-24 25-34 35-44 45-65 Level 1 and below Level 2 Level 3 Level 4 and 5 Employment Share LF participation Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. More precisely, skill sets can explain up to half of the effect attributed to education on the probability of being active in the labor market. An additional year of education alone seems to 57 increase the chances of being active in the labor market by around 2 percentage points. But accounting for skill sets can reduce this probability by as much as half a percentage point. For instance, when accounting for cognitive skills, socio-emotional skills, and computer skills, an additional year of education increases the probability of being in the labor force by 1.5 percentage points. This suggests that investing in education still matters, since additional years of education still increase the probability of being in the labor force. However, investing in promoting better skills may yield better results for labor force participation. Socio-emotional skills are associated with labor force participation and employment even after controlling for years of education. Some socio-emotional skills also improve the probability of participation, such as conscientiousness (by 10 percent) and decision making (by 5 percent). Other personality traits, such as risk aversion, which are not necessarily skills but can be learned through training, reduce the probability by 4 percent. Those who reported that they use their overall computer skills are 9 to 10 percent more likely to be participating in the labor force than those who reported that they do not use these skills at all. All the results are statistically significant (Figure 5.4). Figure 5.4. Changes in the Probability of participating in the labor force by type of skills Armenia adult urban population Employm Conscientiousness ent Grit Conscientiousness participation Computer use (high) Labor force Computer use (low) Decision making Risk aversion -5% 0% 5% 10% 15% Percentage point Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. It is worth noting that women still have low employment chances, even after accounting for different characteristics and skills. Among Armenia’s urban population, women’s participation in the workforce is consistently lower than men’s. Household-related variables (marital status, number of children, labor dependents, parents’ education, and household vulnerability) alone could only explain around half of this lower participation. Figure 5.5 compares the employment probability of 58 Armenian men and women by skill group and household characteristics. The evidence suggests that women’s participation rate is between 14 and 17 percentage points lower than men’s. Figure 5.5. Gender differences: Probability of being employed by different characteristics, skill group, and gender Armenia adult urban population 60% 50% 40% 30% 20% 10% 0% No Skills HH Related Cognitive Socio Computer* All Skills* Emotional* Men Women Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. Furthermore, the explanatory power of skills surpasses that of education concerning the probability of employment conditional on being active in the labor market. The results of this analysis are presented in the last column in Annex Table 17. An additional year of education does not seem to increase the probability of being employed—at least not statistically significantly—whereas grit and conscientiousness add between 5 and 12 percentage points, respectively, to that probability. Also, the gender differences seem to narrow even further and they are not statistically significant. How are different skills sets, in conjunction with education, contributing to higher earnings? Skills seem to matter at least as much as education in terms of earnings. There is evidence supporting a payoff for workers who invest in acquiring education and skills, albeit with caveats present in both theory and estimation (for instance, Griliches, 1977; Heckman, Stixrud, and Urzua, 2006; Hanushek and Woessmann, 2008). Following Bowles, Gintis, and Osborne (2001) and Weinberger (2013), a standard statistical analysis suggests that for full-time workers in Armenia, an additional year of education would increase hourly earnings by around 2.4 percentage points. This rate remains rather constant when one accounts for different skill groups, even after taking into account type of occupation and economic sector (it only reduces about half of a percentage point). 59 This result suggests that investing in education still matters, but individuals can benefit greatly from complementary skills. This is the case because the labor market rewards skills, such as socio- emotional skills and job-related skills that are an indicator of technical skill and not only of credentials (Figure 5.6). Figure 5.6. Returns to years of education by different characteristics and skill groups Armenia adult urban population All Socio-Emotional and Job Related Cognitive and Job-Related Job Related Skills Cognitive and Socio-Emotional Socio-Emotional Skills Cognitive Skills Basic 0.0% 1.0% 2.0% 3.0% Returns to Years of Education Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. Skills affect earnings in different ways depending on the industry. This may reflect correlations between certain skills and the average job compensation for different industries, as well as the complementarities among types of skills. For instance, grit, risk aversion, and hostility bias seem to positively correlate with higher hourly earnings. So do certain job-related activities, such as high computer use at work and high solving and learnings activity (the latter is defined as frequently undertaking tasks requiring 30 minutes or more of thinking and often requiring learning new things). On the other hand, agreeableness, more careful decision making and more contact with clients are associated with lower hourly earnings. This last part aligns with the idea that service-related jobs (those where workers are more prone to have more contact with clients and use this second group of skills) tend to pay less on average in the labor market. One potential risk with this estimation appears when one tries to disentangle the individual effect of each socio-emotional skill or job- related task, as they rarely occur in isolation. However, joint significance tests suggest that the impact of both the socio-emotional skill set and job-related tasks are statistically significantly different from zero. This would reinforce the point that these skills complement education as they also correlate with earnings (Figure 5.7). 60 Figure 5.7. Returns to unit increase of different skills on hourly earnings Armenia adult urban population Solving and learning (High) Computer at work (High) Grit Hostility Bias Risk Aversion Agreeableness Decision Making Contact with Clients (High) -15 -10 -5 0 5 10 15 20 25 Years of Education Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. Returns to education and skills are also differentiated by age cohorts. For instance, there are positive and significant returns to an additional year of education for those between ages 25 and 44. However, this is not the case for the oldest age cohort (45 to 65 years old). In terms of skills, the results are more diverse across age groups. For example, no single socio-emotional skill seems to matter across all the age groups. In the case of job-related skills, it is worth noting that when they do matter, their effects are larger in magnitude than those of education and socio-emotional skills. The data also shows that that using problem solving and learning skills for at least 30 minutes at the work place is highly rewarded for the 25 – 34 age group, whereas autonomy in decision making is more rewarded among the older cohort, ages 35 to 44. This may reflect the progression on the complexity and responsibility in a job, and associated different skill requirements. 61 Box 5.1. Findings of the SABER-Workforce Development Study, Armenia A recent World Bank study assessed the current status of workforce development policy and institutions in Armenia. That study, known as SABER-WfD, was based on an analytical framework that identifies and ranks a country in three dimensions: (i) Strategic framework, which represents the advocacy, championship, and coordination of the objective of workforce development with national priorities; (ii) System oversight, which refers to provisions for funding, quality, and learning pathways that affect individuals, employers, and training providers; and (iii) Service delivery, which encompasses the diversity, organization, and management of training provision. The SABER diagnostic’s findings on the state of Armenia’s workforce development system and what is needed to improve it may be summarized as follows. The state of Armenia’s workforce development system  There is currently a process for connecting employers, workers, and the educational system, which was started years ago; it includes the National Center for VET Development and the National Council for VET Development, which coordinate the workforce development strategies.  The workforce development system relies heavily on government resources, and there are no incentives to use them efficiently or to base them on performance. The quality assurance system is not enforced uniformly across the workforce development policies. Even though the system is being updated and policies are being implemented to diversify the pathway for skill acquisition, the VET system is still seen as one limited to vertical movement toward tertiary education.  The training providers are regulated, but they have limited accountability in meeting quality standards, since there is almost no monitoring and evaluation. This results in a lack of evidence for and poor information on labor market training needs. As a result, training is still developed without industry’s input; where some linkages do exist, they are sporadic and non-institutionalized. Policy recommendations  Improve the strategic framework of the workforce development system by increasing cooperation among stakeholders through the identification of an apex-level body to oversee key priorities with the private sector and coordinate policy implementation. The involvement of the private sector will be necessary to overcome the mismatch of training with needs, to introduce incentives, and to identify best practices for replication in training.  Improve system oversight by introducing performance-based funding mechanisms to provide incentives and accountability for increased efficiency and effectiveness, build institutional capacity to respond more flexibly to local market needs, and foster innovation through competition and best practices in training delivery. This can be achieved by allocating resources (financial and legal, among others) to the quality assurance system.  Improve the service delivery by enhancing the monitoring and evaluation system linked to accreditation and certification, introducing and enforcing measures for training providers through performance indicators, and building a strong labor market information system. Source: World Bank, 2014a. 62 Is training providing value-added to workers in terms of skills? Workers need to keep their skills relevant and updated to meet the job market need. They can achieve this by participating in vocational education or through various forms of training. Either way, it is important that such service provision be adequate and relevant and that it generate enough incentives for participation. A workforce development system in Armenia has been established, but it lacks adequate institutional structure to link it to the productive sector, to oversee training quality, and to generate incentives to participation (Box 5.1). This section provides a quick overview of the participation rates in different types of training and the cost in terms of skill gaps. Skills matter as long as they are relevant and updated, but participation in on-the-job training, professional certification, and apprenticeships is not widespread in Armenia. The overall percentage of 15-to-64-year-olds participating in training is about 6 percent, while for certification it is 7 percent and for apprenticeships it is 10.2 percent. Of those who have participated in any form of skill updating, the younger age groups underwent training, certification or apprenticeship programs more frequently than others. The peak percentages for all three age groups are among 25-to-34-year- olds, and they diminish for the older cohorts (Figure 5.8). Figure 5.8. On-the-job training, professional certificate and apprenticeship participation by age cohort Armenia adult urban population 40% 30% 20% 10% 0% Total 15-24 25-34 35-44 45-54 55-65 On-the-Job Training Professional Certificate Apprenticeship Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. Training and certification seem to help in acquiring job relevant skills, whereas apprenticeships do not. Individuals who attended training were more likely than others to report using cognitive skills and more likely to use them intensively. Similarly, those who had attended a certification program were more likely to report using a computer at work and more likely to use it intensively. However, 63 those who attended apprenticeship programs were not as likely as others to report using their skills of autonomy (freedom to decide on how to do your work) and repetitiveness (frequency of performing short and repetitive tasks), although they were more likely to use those skills intensively when they do (Figure 5.9). Figure 5.9. Differences in use and intensity of job- relevant skills by on-the-job training, professional certificate, and apprenticeship Armenia adult urban population 100% 80% 60% 40% 20% 0% No Yes No Yes No Yes On-the-Job Training: Professional Certificate: Apprenticeship: Autonomy Solving and Learning Computer Use and Repet. low medium high skill not used Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. 64 Highlights of findings  Skills, especially socio-emotional ones, are positively associated with labor force participation and employment opportunities in Armenia. Skills are paying off in the labor market in terms of hourly earnings, and this payoff is large and statistically significant.  Disparities in labor market outcomes across age groups and gender exist today. The returns to education are larger for the younger cohorts than for the older groups, but skills affect the hourly earnings across age cohorts differently. In terms of the gender disparity, it is consistently large for men and women with comparable skill profiles.  Skills that are attributable to the learning process contribute the most to increased earnings. This puts an emphasis on the quality of the skills that workers need to acquire.  Armenia’s low rates of participation in on-the-job training, professional certification, and apprenticeship programs may represent missed opportunities for workforce development, as these programs do seem to help in acquiring job relevant skills. 65 6. Step 4 | Encouraging Entrepreneurship and Innovation in Armenia Skills are not only essential for employable and productive workers, they are also prerequisites for successful entrepreneurs. Entrepreneurial activity is a valuable option in markets with low demand for workers and high unemployment, like Armenia, and it can help bring to the market new products and foster innovation. When their number is sizable, entrepreneurs contribute to economic development through large enterprises that create jobs and through enterprises that serve in case of necessity for vulnerable populations. These factors are key to boosting shared prosperity. Research also shows that improving regulations, reducing business environment constraints to doing business, and increasing access to finance and credit all encourage entrepreneurship and innovation (Valerio, Parton, and Robb, 2014). Some clarification of terminology may be helpful. An entrepreneur is often defined as someone who is willing to take the risk of starting a business in order to make a profit (Chell and Ozkan, 2010). In this section, the definition of entrepreneur includes self-employed individuals and all those who own a business of any size. Due to data limitations, the definition does not distinguish between formal and informal arrangements or sectors of businesses. The term innovation is used in this section to signify the use of knowledge to create new marketable products. Innovation is also considered an important component in a country’s economic success. To look at innovation, this report analyzes the skills in high-innovation manufacturing and knowledge-intensive services in comparison to those sectors that use technology and knowledge less intensively, including agriculture and construction. This section uses descriptive statistics to answer the following questions: (i) Who are the entrepreneurs in Armenia? (ii) How do entrepreneurs and wage workers differ in their use of skills? (iii) Who is working in the high innovation sectors in Armenia? and (iv) What skills are used in the sectors most related to innovation? Who are the entrepreneurs? There are relatively few entrepreneurs in Armenia, and they tend to be middle-aged men with slightly less education than wage workers. The share of entrepreneurs in Armenia is small compared to 66 those who have a wage-earning job, and entrepreneurs also tend to differ in gender and age characteristics. Approximately 89 percent of employed individuals have a wage-earning job; only 11 percent identify as entrepreneurs as defined. Being an entrepreneur does not seem to be associated with current household wealth. Men are more likely to be entrepreneurs than women; with over 92 percent of working women having a wage-earning job. There are differences across age cohorts, too. Entrepreneurship is highest, at 31 percent, for the 45 to 54 age cohort, compared to about 13 percent reported by the 15 to 24 age cohort (Figure 6.1). Figure 6.1. Distribution of entrepreneurs by gender and age cohort Armenia adult urban population 60% 50% 40% 30% 20% 10% 0% Male Female 15-24 25-34 35-44 45-54 55-65 Total Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. The education profile of entrepreneurs is different from that of wage-earning workers. Like most of Armenia’s adults, the majority of entrepreneurs and wage-earners have completed a tertiary degree (51 percent of entrepreneurs and 69 percent of wage workers). However, the share of the population that is engaged as wage-earning workers increases with higher levels of educational attainment, whereas the share of entrepreneurs increases with lower levels of education. This suggests that self-employment may be driven by the inability to find a job, particularly among individuals with lower education (Figure 6.2). 67 Figure 6.2. Education levels for the self-employed and wage workers Armenia adult urban population 100% 80% 60% 40% 20% 0% Self-employed Wage workers Tertiary Upper Secondary Lower Secondary Primary or Less Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. The profile of entrepreneurs varies by economic sector and also by occupation. Most entrepreneurs and wage-earning workers are in the services sector. Except for trade, where 52 percent of all workers are self-employed, there are far more wage-earning workers than entrepreneurs across all economic sectors. The majority of workers across the various occupations are wage-earners too, except for those employed in skilled agricultural work; in this last field, 43 percent of workers reported that they were self-employed. Finally, a majority of wage-earners are employed in high- skilled white-collar jobs, while among the self-employed the majority are in low-skilled white collar jobs (Figure 6.3). Figure 6.3. Distribution of occupations among entrepreneurs and wage workers Armenia adult urban population Skilled agriculture work 100% Elementary occupations 80% Crafts and related trades 60% workers; Plant and machine operator and assemblers 40% Low skilled white collar 20% Highly skilled white collar - Managers/Professionals/Tech 0% nicians Self-employed Wage workers Military personnel Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. 68 How do entrepreneurs and wage-earning workers differ in their use of skills? Overall, entrepreneurs reported less use of cognitive skills than wage-earning workers. Entrepreneurship is associated with less use of reading skills than the work of wage earners. For instance, 43 percent of wage earners reported high use of reading skills, compared with 28 percent of the self-employed. Entrepreneurs are also less likely to use computers at work. For instance, almost 40 percent of wage earners reported high use of computer skills at work, compared with 28 percent of entrepreneurs. This adds to the evidence presented before that self-employment in Armenia is related to occupations that generally use lower levels of skills, since individuals with higher levels of education are more likely to find employment (Figure 6.4). Figure 6.4. Reading skills for entrepreneurs and wage workers Armenia adult urban population 50% 40% 30% 20% 10% 0% skill not used low medium high Self-employed Wage-workers Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. Entrepreneurs have scores similar to wage earners’ in most socio-emotional skills. For instance, the mean score in extraversion for entrepreneurs is 3.0, while the average is 2.9 for wage earners. Results are similar for entrepreneurs and wage earners for skills such as conscientiousness, grit, agreeableness, and openness. In fact, the only skills where there is a difference between the two groups are emotional stability and risk taking, entrepreneurs reporting less stability and more risk taking than wage earners. Being an entrepreneur pays off, as entrepreneurs are likely to have higher incomes than wage earners. An analysis of the determinants of earnings shows that entrepreneurs or self-employed individuals in Armenia are much more likely (up to 30 percent more likely) to earn higher income than wage earners. However, it is possible that this premium comes from business profit. This can be 69 particularly the case for those with relatively formal businesses and those that have employees. The premium may also reflect returns to risk-taking. In the meantime, there are differences between wage earners’ and entrepreneurs’ job-related skills that may account for this difference as well. This is aligned with recent evidence research on entrepreneurship (Levine and Rubinstein, 2013). While risk-taking may not be considered a skill, but rather an attitude or a behavior, it still can be acquired through training. The most used job-relevant skills vary between entrepreneurs and wage workers. Wage earners are more likely to use reading, writing, and computer skills at work, while entrepreneurs are more likely to have contact with people outside of work. For instance, 20 percent of wage workers use reading skills at work with high frequency, compared with only 6 percent of entrepreneurs. Additionally, 38 percent of wage workers use computers with high frequency, compared with 28 percent of entrepreneurs. This suggests that entrepreneurship is much more related to activities that have to do with direct people-to-people contact and that wage workers use cognitive and job-related skills more frequently (Figure 6.5). Figure 6.5. Intensity of computer skill use at work, entrepreneurs and wage workers Armenia adult urban population 100% 28% 38% 80% 60% 40% 65% 55% 20% 0% Self-employed Wage workers high medium low skill not used Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. Overall, entrepreneurs are greater risk takers and are in more contact with people than wage- earning workers. Young people who wish to become entrepreneurs may need to be more open than other people to risk-taking activities and to a certain amount of lack of security on the job front, as well as more open to using their people skills than their cognitive or job-specific skills. Entrepreneurship may also be an important option in times of high unemployment, particularly for youth. 70 Who is working in the high-innovation sectors? A division usually made by researchers to explore the innovation topic is that between high- innovation and low-innovation manufacturing. This section expands on that approach, following the World Bank’s recent report on jobs in Europe and Central Asia, by dividing workers between those in high tech, that is, in high-innovation industries, and those in medium and low tech, that is, in low- innovation sectors (including construction). For instance, we include medium-high manufacturing and knowledge-intensive services in the high innovation category and medium-low manufacturing and less knowledge-intensive services in the low innovation category. The agriculture and construction sectors are classified in low innovation. Highly educated women with some experience are the most likely to work in high-innovation sectors in Armenia. There are slightly more workers in high-innovation sectors than in low-innovation ones (56 percent and 44 percent, respectively). The proportions of individuals working in low- and high- innovation sectors are very similar. Around 25 percent of individuals in both sectors are between 45 and 54 years old. Overwhelmingly, the majority of individuals working in high-innovation sectors are women (70 percent), compared with 30 percent of men. Not surprisingly, over 80 percent of individuals working in these sectors have completed their tertiary education (Figure 6.6). Figure 6.6. Education levels of individuals working in high- and low-innovation sectors Armenia adult urban population 100% 80% 60% 40% 20% 0% Primary or Less Lower Secondary Upper Secondary Tertiary High tech Low tech Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. 71 What skills are used in sectors most related to innovation? Cognitive skill use is much higher among individuals working in high-innovation sectors. The frequency and intensity of use of reading and writing skills are much higher among individuals working in high-innovation sectors than among those in low-innovation sectors. For instance, about 50 percent of workers in high-innovation use reading skills with high frequency, compared with 30 percent of those in low-innovation. Surprisingly, the use of numeracy skills is quite similar in both groups of individuals, although slightly higher in those individuals who work in high-innovation technology occupations (Figure 6.7). Figure 6.7. Intensity of use of reading skills among high- and low-innovation workers Armenia adult urban population 60% 50% 40% 30% 20% 10% 0% skill not used low medium high High tech Low tech Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. While the job-relevant characteristics of individuals working in high- versus low-innovation occupations are different, their socio-emotional characteristics are fairly similar. Individuals working in high-innovation sectors not only use reading, writing and computer skills on the job much more often than their counterparts in low-innovation occupations, they also report having more contact with people outside work and use thinking and learning skills more frequently. For instance, 47 percent of individuals working in high tech reported a high use of computer skills, compared with 23 percent of low-innovation workers. Overall, individuals in high tech sectors demonstrate greater use of higher-order skills on the job than workers in low tech sectors (Figure 6.8). 72 Figure 6.8. Intensity of Computer skills among high- and low-innovation workers Armenia adult urban population 80% 60% 40% 20% 0% skill not used low medium high High tech Low tech Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. Highlights of findings  Entrepreneurship, measured as a percentage of self-employed among total urban employment, is low in Armenia.  Overall, entrepreneurs have different characteristics and skills when compared with wage- earning workers. They tend to be male, between 35 and 44 years of age, and to have lower levels of education than wage workers, yet on average they earn higher incomes.  Concerning their skills, overall entrepreneurs have lower scores in job-relevant, socio- emotional, and cognitive skills than wage earners. Entrepreneurs report less use of their cognitive and job-related skills than wage earners—except for risk taking, a skill they report using more. The use of socio-emotional skills is fairly similar between entrepreneurs and wage earners.  Among wage earners, the profiles differ between those working in high-innovation and those working in low-innovation sectors. A substantial share of all wage earners in Armenia work in high-innovation sectors (56 percent), and women are more likely to be employed in these sectors than men. Individuals working in the high-innovation sectors use their cognitive skills more often than individuals in low-innovation sectors.  It seems that the economy may benefit from an increase in the share of entrepreneurs due to 73 the benefits that entrepreneurship brings to innovation, the country’s large unemployed and inactive population, and the higher earnings of entrepreneurs when compared with wage earners. 74 7. Step 5 ǀ Facilitating Labor market Mobility and Job Matching in Armenia The previous sections have shown that the educational system and households in Armenia can promote learning for all and can help build the right skills for jobs, and that such skills matter for employability. Another key factor for employability is the quality of labor mobility. While multiple measures of labor mobility exist, the STEP survey first focuses on individuals’ school-to-work transitions. The STEP survey results suggest that the longer an individual takes to find a first job after completing his or her highest level of education, the less promising his or her employment prospects are, and the lower the rewards to his or her skills. Further, labor mobility facilitates job matching, which increases economic efficiency, and enables individuals to be productive and to use their skills efficiently. This section starts by exploring the school-to-work transition in Armenia and its implications for the further use of skills. It then explores the occupational mismatches between the qualifications of the workers and the qualifications required by the job. In more detail, this section focuses on answering the following three questions: (i) How does the school-to-work transition affect current labor market outcomes and skills? (ii) How does first out-of-school job affect skill development? and (iii) What are the skill profiles of those who are neither studying nor actively looking for employment (NEETs)? The analysis is mainly carried out using descriptive statistics on adults between 25 and 40 years old in urban Armenia. By using this age group of population, we aim to capture the transitions of those individuals who are most likely to have just completed their highest education level and with some labor market experience. How does the school-to-work transition affect current labor market outcomes and skills? There is a rather quick school-to-work transition in Armenia, but it seems to vary widely by educational attainment. For instance, a large proportion of individuals between ages 25 and 40 found a job within the first six months after completing their highest level of education. However, this proportion ranges as low as 62 percent for upper-secondary graduates to as high as 92 percent for lower-secondary graduates. It is possible that individuals with lower levels of education may be 75 more willing to take lower-level occupations, since they cannot afford to be unemployed. The proportion of tertiary education graduates that found a job in less than six months is 72 percent. It is worth noting that half of the tertiary education graduates found a job within one month of leaving school, suggesting a rapid transition to employment (Figure 7.1). Figure 7.1. School-to-work transition by educational attainment Armenia adult urban population (25-40 years old) 100% 80% 60% 40% 20% 0% Lower Secondary Upper Secondary Tertiary 0 months 1 to 6 months 7 or more months Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. The quick transition is positive news, because longer school-to-work transitions are associated with poorer labor market outcomes. The percentage of employed individuals between ages 25 and 40 declines as the duration of their first job search lengthens. However, individuals within this age group seem to continue actively looking for jobs even after long school-to-work transitions. This is evidenced by the percentage of people in the “inactive” category remaining fairly constant across school-to-work transitions (25.8 percent). The effects of longer school-to-work transitions are also visible in the status of people’s current skills. For instance, the longer school-to-work transitions last, the less likely people are to report using cognitive (reading, writing, and numeracy) skills. Individuals with longer transitions also report using all three skills less intensively (Figure 7.2). 76 Figure 7.2. Use and intensity of reading, writing, and numeracy skills by school-to-work transition time Armenia adult urban population (25-40 years old) 100% 80% 60% 40% 20% 0% 0 1 to 6 7 or 0 1 to 6 7 or 0 1 to 6 7 or months months more months months more months months more months months months Read Write Numeracy low medium high skill not used Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. development? How does first out-of-school job affect skills development? The field of work for a person’s first out-of-school job affects his or her current employability. Overall, among employed workers, first out-of-school jobs were concentrated in white-collar occupations, with marked differences in gender. The white-collar occupations represent about 80 percent of all first-job positions; consisting of 48 percent in high-skilled white collar (HSWC) and 32 percent in low-skilled white collar (LSWC). The gender composition is highly skewed toward women, who make up 79 percent and 76 percent of these first-job positions in HSWC and LSWC, respectively. The gender composition is reversed in crafts and related occupations, where men account for 65 percent of these positions, while in the elementary occupations the proportion is almost even (51 percent men and 49 percent women) (Figure 7.3). 77 Figure 7.3. Gender composition of first out-of-school occupation distribution Armenia adult urban population (25-40 years old) 100% 80% 60% 40% 20% 0% HSWC LSWC Crafts and Related Elementary occupations Male Female Total Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. The occupation of the first out-of-school job also influences whether an individual will further develop socio-emotional skills. For instance, some occupations seem to help the acquisition of socio- emotional skills such as openness. In Armenia, HSWC and crafts-related occupations are associated with increases in openness, quite apart from workers’ educational attainment, whereas openness remains rather constant among workers in LSWC and elementary occupations (Figure 7.4). Figure 7.4. Openness by occupation of first out-of-school job and educational attainment Armenia adult urban population (25-40 years old) 4 3.5 3 2.5 2 HSWC LSWC Crafts and Related Elementary occupations Lower Secondary Upper Secondary Tertiary Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. Job-related skills are also associated with differences in occupation of first out-of-school job and educational attainment. For instance, individuals in crafts and related occupations and elementary occupations were less likely to report using solving and learning skills than were those in HSWC and 78 LSWC occupations. Individuals in HSWC occupations reported using these skills more intensively (Figure 7.5). Figure 7.5. Problem-solving and learning skills’ use and intensity, by occupation of first out-of-school job Urban population: 25-40 years old 100% 80% 60% 40% 20% 0% HSWC LSWC Crafts and Related Elementary occupations Low Medium High Skill not used Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. What about the skills of those inactive between ages 25 to 40? Among individuals ages 25 to 40 in urban Armenia, around 41 percent reported that they were inactive, that is, neither studying nor actively looking for employment. These individuals require special attention both because they signify a loss in human capital and because they are at risk of becoming vulnerable, either through loss of current income or through poverty at old age. The lower skill levels of these inactive individuals limit their possibilities of finding quality employment. For instance, inactive individuals (abbreviated as NEET in the accompanying figures) were consistently less likely than other individuals to report that they use cognitive skills (reading, writing, and numeracy) as well as job-related skills like computer skills. Inactives are also less likely to use the skills intensively. The lesser use (and lower intensity of use) of these skills, in combination with a difficult school-to-work transition, may pose limits to their employment possibilities (Figure 7.6). 79 Figure 7.6. Use of reading, writing, numeracy, and computer skills for inactives and employed Armenia adult urban population (25-40 years old) 100% 80% 60% 40% 20% 0% Inactive Employed Inactive Employed Inactive Employed Inactive Employed Read Write Numeracy Computer low medium high skill not used Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. Inactivity is more common among women and those who did not have a smooth school-to-work transition. Inactive individuals are predominantly women in the 25-to-40 age group. These women also make up a disproportionate number of those whose job search lasted more than six months, in spite of the fact that these women are highly educated (Figure 7.7). Figure 7.7. Gender, educational attainment, dropout rates, and first job rates for inactive and employed Armenia adult urban population (25-40 years old) 100% 80% 60% 40% 20% 0% Male Female Primary or Lower Upper Tertiary No Yes No Yes Less Secondary Secondary Gender Education Dropouts First Job Inactive Employed Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. The rest of this section explores aspects of job matching. Workers have to both learn and use their skills in the workplace in order to be productive. However, mismatches can occur between job requirements and a worker’s qualifications and skills due to imperfectly competitive labor markets, 80 incomplete information about an applicant’s qualifications, or limited geographic mobility. Mismatch is defined as a poor fit between the individual’s qualifications and job-required qualifications (Quintini, 2011). Over-qualification is defined as a situation where a worker’s qualification is higher than what is required for the job, while under-qualification is the reverse. In this section, we use realized occupational matches following Clog and Shockey (1984) and Verdugo and Verdugo (1989). This method roughly assumes that the years of schooling reflect the abilities or skills of the individual. The section then explores how the skills of those who experience mismatch compare with the skills of those who do not experience mismatch. To what extent are there occupational mismatches in the labor market? Most of Armenia’s workers have adequate qualifications for the jobs they hold. Overall, 71 percent of Armenia’s workers have an adequate number of years of education, while 15 percent of them are overqualified and 14 percent are underqualified. This difference in occupational mismatch is gender- neutral, and it is highest for those in the 60 – 64 age cohort (Figure 7.8). Figure 7.8. Occupational mismatch by gender Armenia adult urban population 80 70 Percentage points 60 50 40 72 70 30 20 10 13 15 16 15 0 Underqualified Adequate Overqualified Male Female Source: World Bank, Armenia STEP Skills Measurement Survey, 2013. As expected, cognitive skills vary for those individuals who are overqualified compared with those who are underqualified or adequate for the job. For instance, overqualified workers use reading and writing skills much more than workers who are adequate or underqualified. Individuals who are overqualified also have much better numeracy skills than those who are adequate or underqualified. In contrast, socio-emotional skills tend to be similar among the three types of workers (by 81 qualification match) while some job-relevant skills are more likely to be used by overqualified workers. The difference in socio-emotional skills is not large between the three types of workers. In contrast, overqualified individuals are much more likely to report a higher use of computer skills and to be in contact with people outside work than adequate or underqualified individuals (Figure 7.9 and Figure 7.10). Figure 7.9. Computer use score, by job qualification Armenia adult urban population 2.25 2.00 1.95 1.75 1.82 1.69 1.50 1.25 1.00 0.75 0.50 Underqualified Adequate Overqualified Source: World Bank, Armenia Step Skills Measurement Survey, 2013 Note: “Computer skills” is measured by an index that runs from 0 to 3, 0 signifying almost no use of computer skills, and 3 signifying high use of computer skills. Figure 7.10. Score of contact with people outside of work, by job qualification Armenia adult urban population 2.00 1.75 1.50 1.46 1.25 1.18 1.00 0.94 0.75 0.50 0.25 0.00 Underqualified Adequate Overqualified Source: World Bank, Armenia Step Skills Measurement Survey, 2013. Note: Contact with people is measured by an index that goes from 0 to 3, 0 signifying a low contact with people (i.e., routine contact with colleagues in office), and 3 signifying intense and high contact with people (i.e., doctors). Highlights of findings  There is a relatively quick school-to-work transition in Armenia. However, this transition varies widely by educational attainment, and even those with a long school-to-work 82 transition continue actively looking for a job instead of becoming discouraged  Quick school-to-work transition often leads to better employment prospects, together with better and more job-relevant skills and social-emotional skills.  The inactive population in Armenia is comprised primarily of women and individuals who did not have a first out-of-school job after their highest educational attainment (among individuals between 25 and 40 years old), which takes a significant toll on their human capital prospective.  About 70 percent of workers seem to have adequate qualifications for the jobs they have. Underqualified, adequately qualified, and overqualified individuals have different cognitive and job-relevant skills, while socio-emotional skills tend to be similar among these three categories. 83 8. Conclusions and Way Forward Given the challenges faced by Armenia in employment and growth, it is essential for Armenia’s current and future workforce to be equipped with the right skills for the newest and most modern jobs as well as with skills that can foster entrepreneurship and innovation. Under the STEP framework and enriched with the newly available survey data sets, the findings presented in this report point to several key areas that need to be given due attention as Armenia moves way forward in building a highly skilled workforce. Step 1 ǀ Getting children off to the right start in Armenia. Compared with neighboring countries, Armenia has better coverage of early childhood education programs in terms of enrollment rate. The expansion of the early childhood education programs over time has a demonstrated pay-off in the rising cognitive and socio-emotional skills of the individuals who participated in the programs. Those individuals are also more likely to use certain job-related skills frequently associated with better earnings. To further strengthen Armenia’s early childhood education system, priorities should be given to three areas: (i) Expanding access to the socioeconomically disadvantaged households that are out of the current reach of the system. Concern for equity also justifies further increasing public investment in this service provision. (ii) In addition, the seeming recent trend of declining early education participation needs to be further looked into – its patterns and likely causes need to be identified. Policy interventions may be needed to reverse this trend. (iii) Although the service quality issue is not covered by the data, global experience shows that expanding public services on a large scale sometimes risks lowering the bar for service quality. Ensuring service quality is therefore critical for the effectiveness of expanded early education programs. Expansion without quality will only lead to the waste of resources. Step 2 ǀ Ensuring that all students learn in Armenia. Armenia has a well-educated population overall, with high average educational attainment as measured by years of schooling. The report shows that in Armenia, higher levels of educational attainment are associated with higher reading proficiency; higher scores in socio-emotional skills; and more frequent use of foundational and job-relevant skills. However, compared with OECD countries, Armenian adults’ average reading proficiency is low. Most importantly, the gains in skills achieved by completing higher education levels appear to be much smaller than the gains achieved in OECD countries. In other words, Armenia’s education system does 84 not add as much value as is added elsewhere. In addition, the education and skills gap is prominent across the socioeconomic strata of the households. Moving forward, Armenia will need to prioritize by doing the following: (i) Armenia needs to improve the learning outcomes and skills formation in its education system. Worldwide experience shows that this is not an easy task, and results may take years to show. Lessons from other countries also show that having a high quality and effective teaching force is the foremost necessary condition for high quality education. Having a strong quality assurance and accountability system for learning results is another important pre-requisite. (ii) Strengthening tertiary education will be necessary, as over half of Armenia’s labor force comes from tertiary graduates. The systemic as well as institutional reforms should give emphasis to building a close link with the labor market need for a skilled and innovative labor force – not only as employees, but also as entrepreneurs. (ii) To narrow the learning outcomes and skills gaps, Armenia needs to put in place measures targeting those who lag behind in the school system. An effective school system should serve as an important channel to narrow the gaps associated with the different family backgrounds from which the students come. Step 3 ǀ Building job-relevant skills in Armenia. In Armenia, higher levels of educational attainment as well as reading proficiency are distinctively associated with higher labor force participation and employment rates. In addition, socio-emotional and job related skills seem to matter at least as much as education in terms of labor force participation, employment and earnings. This is a finding that is consistent with the ranking of importance of skills by employers in the employers ’ survey. On top of the foundations skills, building technical and job-specific skills will particularly require re- thinking the two important stages of the formation of these skills: (i) Vocational education and training systems need to be assessed and strengthened. How to improve the effectiveness of the current preliminary and middle-level vocational schools in providing the skills needed is the first challenge. In the longer term, as the demand for high-level skills rises, whether to delay tracking after 12 years of foundation learning can be evaluated and considered. (ii) On-the-job training remains the most effective way to acquire job-specific skills. To increase the opportunities for on-the- job training, incentives can be designed to encourage firms to increase their provision of such training to their employees. The current apprenticeship programs do not seem to work in equipping trainees with job-relevant skills, so the program designs should be closely examined before considering the future of such programs. Step 4 ǀ Encouraging Entrepreneurship and Innovation in Armenia. The prevalence of 85 entrepreneurship in Armenia, defined as self-employment in this report, is low as measured by the percentage of total employment. The findings indicate that Armenian entrepreneurs tend to have lower levels of education attainment and lower scores in job-relevant, socio-emotional, and cognitive skills than wage earners, but on average they earn higher incomes. This is likely to be linked to the payoffs of the risks they take, which is consistent with their reported higher frequencies of using “risk-taking” skills. While it is unclear to what extent the entrepreneurship as defined is linked to innovation and higher productivity activities, there are signs of high potential as a substantial share of all wage workers in Armenia work in high-innovation sectors (56 percent), particularly for women. Future policy assessment and consideration can aim at: (i) creating an enabling business environment and improving business supporting services and risk management policies and instrument to encourage entrepreneurship targeting innovative sectors; and (ii) aligning some of the education and training courses, particularly at the tertiary level, with the objective of fostering entrepreneurship by equipping trainees with essential knowledge, skills, and attitudes. Step 5 ǀ Matching the supply of skills to what employers demand in Armenia. In Armenia, the transition period between finishing school and starting the first job is generally short. This report has found that smooth school-to-work transition often leads to better and more job-relevant skills in the future and further increases employability and earnings prospects. In addition, a majority of workers seem to have adequate qualifications for the jobs they have, even though “qualifications” may not be fully aligned with “skills”. To fully understand labor market mobility and job match, two important questions need further research: (i) What are the job turnover rates in Armenia, both in the present and expected in the future, and how should the system of adult learning and skills re- training or updating respond to the need? (ii) What public employment services exist, what is their performance record, and how could they function most effectively to foster labor market mobility by providing real-time labor market information and bridging need and demand between employers and job-seekers? 86 References Aedo, C. and I. Walker (2012), Skills for the 21st Century in Latin America and the Caribbean, World Bank, Washington, DC. http://dx.doi.org/10.1596/978-0-8213-8971-3 Arias, O., and C. Sanchez-Paramo. 2014. Back to Work. Growing with Jobs in Europe and Central Asia. Washington, DC: World Bank. Armstat (National Statistical Service of the Republic of Armenia). 2012a. Labour Market in the Republic of Armenia 2004 - 2008. Yerevan. Retrieved April 24, 2014, from www.armstat.am. ________. 2012b. Statistical Yearbook of Armenia. Retrieved April 24, 2014 from www.armstat.am. Atkinson, Rita L., R. C. Atkinson, E. E. Smith, D. J. Bem, and S. Nolen-Hoeksema, eds. 2000. Hilgard's Introduction to Psychology (13th ed.). Orlando, Fla.: Harcourt College Publishers. Barrick, M. R., and M. K. Mount. 1991. “The Big Five personality Dimensions and Job Perfomance: A Meta-analysis.” Personnel Psychology 44 (1): 1–26. DOI: 10.1111/j.1744-6570.1991.tb00688.x Behrman, J., J. Hoddinott, J. Maluccio, E. Soler-Hampejsek, E. Behrman, R. Martorell, M. Ramirez-Zea, and A. Stein. 2006. “What Determines Adult Cognitive Skills? Impacts of Pre-Schooling, Schooling and Post-Schooling Experiences in Guatemala.” Working Paper, Middlebury College, Department of Economics. Borghans, G., H. Meijers, and B. Ter Weel. 2008. “The Role of Noncognitive Skills in Explaining Cognitive Test Scores.” Economic Inquiry 46 (1): 2–12. Bowles, Samuel, Herbert Gintis, and Melissa Osborne. 2001. “The Determinants of Earnigns: A Behavioral Approach.” Journal of Economic Literature 39 (4): 1137–76. Cawley, J., J. Heckman, and E. Vytlacil. 2001. “Three Observations on Wages and Measured Cognitive Ability.” Labour Economics 8 (4): 419–42. Chell, E., and K. M. Ozkan. 2010. Nascent Entrepreneurship and Learning. Northampton, Mass.: Edward Elgar. Clog, C., and W. J. Shockey. 1984. “Mismatch between Occupation and Schooling: A Prevalence Measure, Recent Trends and Demographic Analysis.” Demography 21 (2): 235–57. Cunha, F., J. Heckman, L. Lochner, and D. Masterov. 2006. “Interpreting the Evidence on Life Cycle 87 Skill Formation.” In Handbook of the Economics of Education, edited by E. A. Hanushek and F. Welch. Amsterdam: North Holland. Cunha, F., J. Heckman, and S. Schennach. 2010. “Estimating the Technology of Cognitive and Noncognitive Skill Formation.” Econometrica 78 (3): 883–931. Costa, P. T., and R. R. McCrae. 1992. NEO Personality Inventory -- Professional Manual. Odessa, Fla.: Psychological Assessment Resources. Duckworth, Angela L., Christopher Peterson, Michael D. Matthews, and Dennis R. Kelly. 2007. “Grit: Perseverance and Passion for Long-Term Goals." Journal of Personality and Social Psychology 92 (6): 1087. Economy and Values Research Center, EV Consulting (EV Research Center). 2012. “2011-2012 National Competitiveness Report of Armenia: Agenda for Upgrading Management Practices.” Retrieved April 10, 2014, from http://ev.am/sites/default/files/attachments/pdf/ACR%202011-2012_eng.pdf European Commission. 2013. Higher Education in Armenia. Retrieved April 8th, 2014 from http://eacea.ec.europa.eu/tempus/participating_countries/overview/armenia_tempus_country_fi che_final.pdf. European Training Foundation (ETF). 2013. “Results of the 2011/12 Migration Survey on the Relationship Between Skills, Migration, and Development.” Working Paper. Retrieved April 8th, 2014 from http://www.etf.europa.eu/webatt.nsf/0/9D759BA5148D99FBC1257B730046F443 /$file/Migration&skills_Armenia.pdf Goldberg, L. R. 1992. “The Development of Markers for the Big-Five Factor Structure.” Psychological Assessment 4: 26–42. Goldin, C., and L. Katz. 2008. The Race Between Education and Technology. Cambridge, Mass.: The Belknap Press of Harvard University Press. Goodstein, L. D., and R. I. Lanyon. 1999. “Applications of Personality Assessment to the Workplace: A Review.” Journal of Business and Psychology 13 (3): 291-322. Griliches, Z. 1977. “Estimating the Returns to Schooling: Some Econometric Problems.” Econometrica 45: 1–22. 88 Hanushek, E. A., G. Schwerdt, S. Wiederhold, and L. Woessmann. 2013. “Returns to Skills Around the World: Evidence from PIAAC.” Working Paper no. 19762. Cambridge, Mass.: National Bureau of Economic Research. Hanushek, E., and L. Woessmann, L. 2008. "The Role of Cognitive Skills in Economic Development." Journal of Economic Literature 46 (3): 607–68. Heckman, J., J. Humphries, S. Urzua, and G. Veramendi. 2010. "The Effects of Educational Choices on Labor Market, Health, and Social Outcomes.” Working Paper no. 2011-002, Human Capital and Economic Opportunity Working Group, University of Chicago. Retrieved from http://humcap.uchicago.edu/RePEc/hka/wpaper/HHUV_2010_effect-edu-choice.pdf. Heckman, J. and T. Kautz. 2013. “Fostering and Measuring Skills: Interventions that Improve Character and Cognition.” Working Paper no. 19656. Cambridge, Mass.: National Bureau of Economic Research. Heckman, J., L. J. Lochner, and P. E. Todd. 2006. “Earnings Functions, Rates of Return and Treatment Effects: The Mincer Equation and Beyond.” In Handbook of the Economics of Education, Volume 1, edited by E. Hanushek and F. Welch. Elsevier. Heckman, J., J. Stixrud, and S. Urzua. 2006. “The Effects of Cognitive and Noncognitive Abilities on Labor Market Outcomes and Social Behavior.” Journal of Labor Economics 24: 411–82. Helpman, E. 2004. The Mystery of Economic Growth. Cambridge, Mass.: Harvard University Press. Higgins, D. M., J. B. Peterson, A. Lee, and R. O. Pihl. 2007. “Prefrontal Cognitive Ability, Intelligence, Big Five Personality and the Prediction of Advanced Academic and Workplace Performance.” Journal of Personality and Social Psychology 93: 298–319. doi:10.1037/0022-3514.93.2.298 International Bureau of Education (IBE). 2010. “Armenia.” World Data on Education. 7th Edition. Paris: UNESCO. http://www.ibe.unesco.org/fileadmin/user_upload/Publications/ WDE/2010/pdf- versions/Armenia.pdf [accessed November 15, 2013]. International Labour Organization (ILO). 2013. “Labor Force Survey, Armenia.” Available from ILO Bureau of Library and Information Services. Retrieved from http://www.ilo.org/dyn/lfsurvey/lfsurvey.list?p_lang=en Ishikawa, M., and D. Ryan. 2002. “Schooling, Basic Skills and Economic Outcomes.” Economics of Education Review [serial online] 21 (3): 231–43. 89 Jacob, B. 2002. “Where the Boys Aren’t: Non-Cognitive Skills, Returns to School and the Gender Gap in Higher Education.” Working Paper no. w8964. Cambridge, Mass.: National Bureau of Economic Research. Komarraju, M., S. Karau, R. Schmeck, and A. Avdic. 2011. “The Big Five Personality Traits, Learning Styles, and Academic Achievement.” Personality and Individual Differences 51 (4): 472–77. Levine, R., and Y. Rubinstein. 2013. Smart and Illicit: Who Becomes an Entrepreneur and Does it Pay? Working Paper no. 19276. Cambridge, Mass.: National Bureau of Economic Research. Retrieved April 25, 2014 from http://www.nber.org/papers/w19276 Matthews, G., I. Deary, and M. Whiteman. 2009. Personality Traits, 3rd edition. Cambridge, UK: Cambridge University Press. McCrae, Robert R., and Paul T. Costa. 1987. “Validation of the Five-Factor Model of Personality Across Instruments and Observers.” Journal of Personality and Social Psychology 52 (1): 81-90. doi: 10.1037/0022-3514.52.1.81 Ministry of Education and Science. 2006. National Curriculum for General Education. Yerevan. Mitra, Pradeep; Selowsky, Marcelo; Zalduendo, Juan. 2010. Turmoil at Twenty: Recession, Recovery, and Reform in Central and Eastern Europe and the Former Soviet Union. World Bank. © World Bank. https://openknowledge.worldbank.org/handle/10986/2682 License: CC BY 3.0 IGO Neisser, Ulric, Gwyneth Boodoo, Thomas Jr Bouchard, Wade A. Boykin, Nathan Brody, Stephen J. Ceci, Diane F. Halpern, John C. Loehlin, R. Perloff, Robert J. Sternberg, and S. Urbina. 1996. “Intelligence: Knowns and unknowns.” American Psychologist 51, 77-101. Organisation for Economic Co-operation and Development (OECD). 2013. Skilled for Life? Key Findings from the Survey of Adult Skills. Paris. Pierre, G., M. L. Sanchez-Puerta, A. Valerio, and T. Rajadel. 2014. “STEP Skills Measurement Surveys: Innovative Tools for Assessing Skills.” Washington, DC: World Bank. Prevo, T., and B. ter Weel. 2003. “The Importance of Early Conscientiousness for Socio-Economic Outcomes: Evidence from the British Cohort Study.” IZA Discussion Paper no. 7537. Quintini, G. 2011. “Overqualified or Under-skilled: A Review of Existing Literature.” OECD Social, Employment, and Migration Working Paper No. 121. Paris: Organisation for Economic Co- operation and Development. Retrieved March 28, 2014 from 90 http://www.oecd.org/els/48650026.pdf. Rutkowski, Jan. 2012. “Promoting Productive Employment in Armenia.” Washington, DC: World Bank. __________. 2013. “Skills Employers Seek: Results of the Armenia STEP Employer Survey.” Washington, DC: World Bank. Sondergaard, Lars; Murthi, Mamta; Abu-Ghaida, Dina; Bodewig, Christian; Rutkowski, Jan. 2012. Skills, Not Just Diplomas: Managing Education for Results in Eastern Europe and Central Asia. World Bank. © World Bank. https://openknowledge.worldbank.org/handle/10986/2368 License: CC BY 3.0 IGO UNESCO Institute for Statistics (UIS). 2014a. “Education 2000-2015.” [statistics]. Available from UNESCO Institute for Statistics Education database. http://data.uis.unesco.org/Index.aspx?DataSetCode=EDULIT_DS&popupcustomise=true&lang=e n ________. 2014b. “Education 2000-2013.” Available from UIS Education database http://stats.uis.unesco.org/unesco/tableviewer/document.aspx?ReportId=143 [accessed April 10, 2014] UNICEF. n.d. “Education in Armenia.” Country Profile. New York. http://www.unicef.org/ceecis/Armenia.pdf [accessed November 15, 2013]. Valerio, A., B. Parton, A. Robb. 2014. Entrepreneurship Education and Training Programs Around the World: Dimensions for Success. Washington, DC: World Bank. Verdugo, R. R., and N. T. Verdugo. 1989. “The Impact of Surplus Schooling on Earnings: Some Additional Findings.” Journal of Human Resources 24 (4): 629–43. Weinberger, Catherine. 2013. “The Increasing Complementarity Between Cognitive and Social Skills.” Review of Economics and Statistics, Forthcoming. World Bank. 2010a. “Policy Forum: Armenia After the Crisis: Challenges and Opportunities Concept Note.” Washington, DC. http://imagebank.worldbank.org/servlet/WDSContentServer/ IW3P/IB/2012/06/03/000356161_20120603231849/Rendered /PDF/ l692280ESW0whit0evised0August026.10.pdf World Bank. 2010b. “Stepping Up Skills for More Jobs and Higher Productivity.” Washington, DC. 91 ________. 2011. “Strengthening Skills and Employability in Peru: Final Report.” Report no. 61699-PE. Washington, DC: The World Bank. http://documents.worldbank.org/curated/en/2011/05/14963817/strengthening-skills-employability- peru-final-report ________. 2012a. “Armenia: An Overview.” ECA PEG Brief Series. Washington, DC. http://imagebank.worldbank.org/servlet/WDSContentServer/IW3P/IB/2013/08/12/000445729_201d 30812161034/Rendered/PDF/801530BRI0PEG000Box0379800B00OUO.90.pdf ________. 2012b. “Armenia: Promoting Productive Employment.” Washington DC: World Bank, Human Development Sector Unit. Retrieved April 8th, 2014 from http://documents.worldbank.org/curated/en/2012/09/17368635/armenia-promoting-productive- employment. ________. 2012c. “SABER: Early Childhood Development, Armenia 2012.” Washington, DC. ________. 2013a. “Skilling Up Vietnam: Preparing Vietnam’s Workforce for a Modern Market Economy.” Vietnam Development Report. Washington, DC: World Bank. ________. 2013b. World Development Indicators 2013 [statistics]. Washington, DC. Retrieved from http://databank.worldbank.org/data/download/WDI-2013-ebook.pdf ________. 2014a. “SABER: Workforce Development, Armenia 2013.” Washington, DC. ________. 2014b. World Development Indicators 1990-2013 [statistics]. Available from World Bank World Development Indicators database. Washington, DC. http://data.worldbank.org/data- catalog/world-development-indicators. ________. 2014c. “Armenia Education Improvement Project (EIP).” Project Appraisal Document, February. Washington, DC. World Bank ,2014d. EdStats. Washington, D.C. World Bank and IFC. 2014. “Armenia.” Country Program Snapshot. Washington, DC. http://www.worldbank.org/content/dam/Worldbank/document/Armenia-Snapshot.pdf. 92 Technical Annexes Annex 1: The STEP Skills Measurement Program: The Household Survey 1. Geographic coverage The STEP Skills Measurement household survey program covers a large sample of low and middle- income countries, including Armenia, Bolivia, Colombia, Georgia, Ghana, Kenya, Lao PDR, Macedonia, Sri Lanka, Ukraine, Vietnam, and the Yunnan Province in China. Surveys were carried out between 2011 and 2013. In Armenia, the household survey was implemented in 2013. The field work was carried out from April to June 2013. 2. Target population The STEP target population is the urban population ages 15 to 64. The sampling strategy was designed to ensure that the target population represents at least 95 percent of the urban working- age population (aged 15 to 64) in each country. The specifics for the Armenia data are included below: Annex Table 1. Main characteristics Target population Non-institutionalized persons 15 to 64 years of age (inclusive) living in private dwellings in urban areas of Armenia at the time of data collection Initial sample size 6,814 activated households, of them 5,052 eligible households Response rate 50.3% Final sample size 2,992 participating selected persons with a final person weight Field work April –June 2013 Coverage 36 urban cities representing ten provinces (marzes), and the country's capital Yerevan Source: WB, 2013, STEP Survey Weighting Procedures Summary: Armenia, Draft 2, November 14 It should be noted the STEP samples in Armenia, both unweighted and weighted, are significantly more biased to the female population. The unweighted STEP sample is also biased to older age groups but this bias has been almost corrected in the weighted sample. 93 Annex Table 2: STEP and alternative estimates of urban population (15-64 years) by gender & age group STEP STEP Admin. statistics based Data on labor (unweighted) (weighted) on Census-2011, resources based on beginning of 2013 LFS-2012 (average annual) Total (persons) 2,992 1,185,359 1,355,817 1,328,100 Female (%) 71.7 70.3 53.7 n.a. Age group (%) 15-19 7.3 9.1 9.1 10.1 20-24 11.3 13.1 12.6 12.5 25-29 11.0 11.4 12.9 11.8 30-34 12.3 11.5 11.4 10.5 35-39 9.6 8.8 9.4 9.1 40-44 8.4 8.2 8.3 7.6 45-49 9.6 9.7 8.7 9.1 50-54 11.1 10.7 10.9 10.9 55-59 9.9 9.8 9.4 9.7 60-64 9.6 7.7 7.3 8.8 Source: STEP Armenia; National Statistical Service of RA, http://www.armstat.am/en/?nid=420 (demographic statistics based on Census), Statistical Yearbook of Armenia 2013 (LFS data). 3. Background questionnaire The STEP survey collects comprehensive information not typically captured by traditional household surveys. It includes two distinct instruments: a background questionnaire and a reading literacy assessment. The background questionnaire is organized in three household background modules and seven thematic modules. Module 1 starts with a standard household roster and a section on dwelling characteristics. It concludes with the random selection of a household member aged 15 to 64 to whom the remainder of the survey is administered. The random selection is based on strict guidelines set by the STEP technical standards. Modules 2 to 11 are applied to the randomly selected respondent. Module 2 collects in-depth information on education, training, and the person’s first job. Module 3 asks health-related questions. Module 4 gathers extensive information on the respondent’s current occupation(s). Modules 5 and 6 include some of STEP’s most innovative features. Module 5 asks detailed questions on the respondent’s use of reading, writing, and numeracy skills in daily life and at work, as well as job-relevant skills used at work. Module 6 provides information on personality traits, behavior, and preferences. It is important to note the survey gathers skills information from the entire sampled population, regardless of their labor force status (employed, unemployed or inactive) and type of employment. Job-relevant skills, however, are captured only for respondents who are currently working or have worked at some time in the 12 months prior to the survey. Finally, Module 7 focuses on family background. 4. Reading literacy assessment The second part of the STEP survey consists of a reading literacy assessment, which was specifically developed for the STEP survey by Educational Testing Services (ETS). This assessment provides a direct measure of respondents’ reading proficiency. It is organized in three parts. The first part focuses on foundational reading skills, including word meaning, sentence processing and passage comprehension. The second part consists of a core literacy assessment, which is used as a screener intended to sort the least literate from those with higher reading skill levels. The third part is only administered to respondents having passed the core literacy assessment. It provides a finer evaluation of reading skills for the most literate individuals in the sample. The STEP Survey in the 94 Yunnan province of China, Lao PDR, and Sri Lanka included only parts one and two of the reading literacy assessment. In Sri Lanka the test was administered in Tamil and Sinhala. Annex Figure 1: Structure of household survey Household Information Application random Household Roster Dwelling characteristics selection criteria to select respondent Randomly selected respondent ages 15-64 Background Skills Educatio Employm Health n& ent Family Cognitive Job- Socio-emotional Training History relevant Indirect Direct Person- Assess- Assess- ality Behavior Preferences ment ment traits Use and intensity of use Reading Assessment of: Reading components Short Reading Reading Core Literacy Assessment Writing Numeracy Literacy booklets Extended Reading Assessment 5. Technical standards The STEP Skills Measurement household survey was specifically designed to ensure data comparability. Coordination and supervision were centralized so survey instruments were administered in a standardized way across all participating countries, including Armenia. All survey firms benefited from the STEP team’s technical assistance throughout the implementation process and complied with the STEP technical standards. Each survey firm’s implementation plan was summarized in a National Survey Design Planning Report. The sampling strategy and data weighting were carried out by a single survey methodologist to ensure consistency across methodologies (see STEP Skills Measurement Survey Methodology Note, 2014). 95 6. Overall sample size and response rates Sample sizes vary from 1,196 observations in Sri Lanka to 3,405 observations in Vietnam. Response rates range from 60 percent in Sri Lanka to 98 percent in the Yunnan Province. In Bolivia and Colombia however, response rates were markedly lower (respectively 43 percent and 46 percent). Annex Table 3: Sample sizes and response rates, by country Yunnan Armenia Bolivia Colombia Georgia Ghana Lao PDR Sri Lanka Vietnam Province Sample Size 2,992 2,435 2,617 2,996 2,987 2,032 1,196 3,405 2,017 Response 50% 43% 48% 63% 83% 94% 60% 62% 98% rate Note: All samples are for urban areas only. 96 Annex 2: Skills Measured in the STEP Skills Survey The STEP surveys measure the skills shown below in Table A4. Annex Table 4: STEP skills measured Direct measurement of reading literacy based on  Reading proficiency the Survey of Adult Skills instruments Cognitive Skills Indirect assessment (self-reported) on  Writing individuals’ use of foundational skills--at work or  Numeracy in daily life  Openness  Conscientiousness  Extraversion Personality traits  Agreeableness Socio-emotional  Neuroticism skills  Grit  Hostile attribution bias Behavior  Decision making Risk and time preference Qualifications required for the job and job learning times  Computer use Job-relevant  Contact with clients skills Indirect assessment of skills used at work  Solving and learning  Autonomy and repetitiveness  Physical tasks Cognitive skills are defined as the “ability to understand complex ideas, to adapt effectively to the environment, to learn from experience, to engage in various forms of reasoning, to overcome obstacles by taking thought.” Literacy, numeracy, and the ability to solve abstract problems are all cognitive skills. The STEP survey provides a direct measurement of reading proficiency and an indirect measurement of reading, writing, and mathematics skills. 1. Direct measurement The survey includes a direct measure of reading proficiency through the reading literacy assessment designed by the Educational Testing Services (ETS), as outlined in Figure A2. This assessment has three parts: (i) The first part of the assessment (Section A) evaluates foundational reading skills, including word meaning, sentence processing and passage comprehension. Word meaning exercises ask respondents to match written words to pictures of everyday objects. Sentence processing exercises ask individuals to identify whether or not a given sentence makes sense. Finally, passage comprehension exercises require respondents to complete sentences embedded in a paragraph, by selecting one of two words that best fit the overall meaning of the section. As these exercises were timed, the analysis may also include a time dimension. The data provided by part one of the assessment can translate into multiple variables (e.g., score, pace). (ii) The second part (Section B) consists of a core literacy assessment, which sorts the least literate from those with higher reading skill levels. This section includes eight items. 97 Respondents with three or more correct responses are regarded as having met a minimum reading literacy threshold. The present document uses an indicator variable to identify respondents having passed the core reading assessment. (iii) The third part (Exercise booklets) is only administered to respondents having passed the core assessment. It evaluates reading proficiency in more depth. The assessment uses a variety of materials, focusing on non-school-based items encountered in daily life. It also involves different types of tasks, including tasks that require respondents to access and identify information (in both text-based and non-prose materials such as tables, graphs and forms), to integrate and interpret information, and to evaluate information by assessing the relevance, credibility, or appropriateness of the material for a particular task. Items present varying levels of difficulty, with tasks ranging from locating a single piece of information in a very short advertisement to summarizing reasons for using generic drugs as presented in a newspaper article. Overall reading proficiency scores are reported on a scale ranging from 0 to 500, which is divided into 5 levels, with Level 1 characterized by the least demanding tasks and Level 5 the most demanding. (Table A5 below explains the levels and scoring in detail.) For each respondent, 10 plausible values were generated. Findings presented in this report are based on using all 10 plausible values. 98 Annex Figure 2: Direct assessment of reading literacy – reading assessment flow chart Direct Assessment Basic Reading Assessment Extended Reading Assessment Annex Table 5: Direct measurement of reading proficiency | Key indicators Core Literacy Assessment Did not pass Answered fewer than 3 Respondent’s reading proficiency is below a minimum reading literacy correct responses out of 8 threshold items Passed Answered 3 or more correct Respondent has met a minimum reading literacy threshold responses out of 8 items Reading Proficiency Levels and Score Literacy Below Level 1 The tasks at this level require the respondent to read brief texts on 0 to 175 familiar topics to locate a single piece of specific information. Only basic vocabulary knowledge is required, and the reader is not required to understand the structure of sentences or paragraphs or make use of other text features. There is seldom any competing information in the text and the requested information is identical in form to information in the question or directive. While the texts can be continuous, the information can be located as if the text were noncontinuous. Tasks below Level 1 do not make use of any features specific to digital texts. 99 Literacy Level 1 Most of the tasks at this level require the respondent to read relatively 176 to 225 short digital or print continuous, non-continuous or mixed texts to locate a single piece of information which is identical to or synonymous with the information given in the question or directive. Some tasks may require the respondent to enter personal information into a document, in the case of some noncontinuous texts. Little, if any, competing information is present. Some tasks may require simple cycling through more than one piece of information. Knowledge and skill in recognizing basic vocabulary, evaluating the meaning of sentences, and reading of paragraph text is expected. Literacy Level 2 At this level, the complexity of text increases. The medium of texts may be 226 to 275 digital or printed, and texts may comprise continuous, noncontinuous or mixed types. Tasks in this level require respondents to make matches between the text and information, and may require paraphrase or low- level inferences. Some competing pieces of information may be present. Some tasks require the respondent to cycle through or integrate two or more pieces of information based on criteria, compare and contrast or reason about information requested in the question, or navigate within digital texts to access and identify information from various parts of a document. Literacy Level 3 Texts at this level are often dense or lengthy, including continuous, 276 to 325 noncontinuous, mixed or multiple pages. Understanding text and rhetorical structures become more central to successfully completing tasks, especially in navigation of complex digital texts. Tasks require the respondent to identify, interpret or evaluate one or more pieces of information and often require varying levels of inference. Many tasks require the respondent to construct meaning across larger chunks of text or perform multistep operations in order to identify and formulate responses. Often tasks also demand that the respondent disregard irrelevant or inappropriate text content to answer accurately. Competing information is often present, but it is not more prominent than the correct information. Literacy Level 4 Tasks at this level often require respondents to perform multiple-step 326 to 375 operations to integrate, interpret, or synthesize information from complex or lengthy continuous, noncontinuous, mixed, or multiple type texts. Complex inferences and application of background knowledge may be needed to perform successfully. Many tasks require identifying and understanding one or more specific, non-central ideas in the text in order to interpret or evaluate subtle evidence claims or persuasive discourse relationships. Conditional information is frequently present in tasks at this level and must be taken into consideration by the respondent. Competing information is present and sometimes seemingly as prominent as correct information. 100 Literacy Level 5 At this level, tasks may require the respondent to search for and integrate 376 to 500 information across multiple, dense texts; construct syntheses of similar and contrasting ideas or points of view; or evaluate evidence-based arguments. Application and evaluation of logical and conceptual models of ideas may be required to accomplish tasks. Evaluating reliability of evidentiary sources and selecting key information is frequently a key requirement. Tasks often require respondents to be aware of subtle, rhetorical cues and to make high-level inferences or use specialized background knowledge. 2. Indirect measurement The STEP survey also asks respondents to report on their use of cognitive skills in daily life and at work, namely if they read, write, or use mathematics. For each skill, a score ranging from 0 to 3 was computed. When a respondent reports not using a given skill, the score is set at 0. For respondents who do use a given skill, intensity or complexity of use was defined (1 for low, 2 for medium, and 3 for high). The aggregation process and reliability testing are described in the STEP Skills Measurement Survey Methodology Note (2014). Table below lists the key indicators for cognitive skills. Annex Table 6: Indirect measurement of cognitive skills | Key indicators Use of reading and writing skills Intensity of use Level Does not read/write = Does not use 0 Read/write documents of 5 pages or less = Low 1 Read/write documents of 6 to 25 pages = Medium 2 Read/write documents of more than 25 pages = High 3 Use of numeracy skills Complexity of use Level Does no math = Does not use 0 Measures or estimates sizes, weights, distances; calculates = Low 1 prices or costs; performs any other multiplication or division Uses or calculates fractions, decimals or percentages = Medium 2 Uses more advanced math such as algebra, geometry, = High 3 trigonometry Socio-emotional skills, sometimes referred to in the literature as non-cognitive skills or soft skills, relate to traits covering multiple domains (social, emotional, personality, behaviors, attitudes, etc.). The survey builds on the “Big Five” personality traits: openness, conscientiousness, extraversion, agreeableness, and neuroticism (or its opposite, emotional stability). Measures of grit, which has been shown to have an impact in life outcomes and hostile attribution bias, were also included, as well as questions pertaining to how individuals make important decisions. Response categories range from 1, “almost never”, to 4, “almost always”. The aggregation process and reliability testing are described in the STEP Skills Measurement Survey Methodology Note (2014). Table A7 presents the questionnaire items used for each socio-emotional skill. 101 Annex Table 7: Socio-emotional skills | Items Socio-emotional skill Items Do you come up with ideas other people haven't thought of before? Openness Are you very interested in learning new things? Do you enjoy beautiful things, like nature, art and music? When doing a task, are you very careful? Conscientiousness Do you prefer relaxation more than hard work? Do you work very well and quickly? Are you talkative? Do you like to keep your opinions to yourself? Extraversion Do you prefer to keep quiet when you have an opinion? Are you outgoing and sociable, for example, do you make friends very easily? Do you forgive other people easily? Agreeableness Are you very polite to other people? Are you generous to other people with your time or money? Are you relaxed during stressful situations? Emotional Stability Do you tend to worry? (Neuroticism) Do you get nervous easily? Do you finish whatever you begin? Do you work very hard? For example, do you keep working when others Grit stop to take a break? Do you enjoy working on things that take a very long time (at least several months) to complete? Do people take advantage of you? Hostile Bias Are people mean/not nice to you? Do you think about how the things you do will affect you in the future? Decision-making Do you think carefully before you make an important decision? Do you ask for help when you don’t understand something? Job-relevant skills are task-related and build on a combination of cognitive and socio-emotional skills. The STEP survey asks respondents about their use of such skills on the job, including among others computer use, repair and maintenance of electronic equipment, operation of heavy machinery, client contact, solving and learning, supervision, etc. For each skill, a score ranging from 0 to 3 was computed. When a respondent reports not using a given skill, the score is set at 0. For respondents who do use a given skill, intensity or complexity of use was defined (1 for low, 2 for medium, and 3 for high). The STEP Skills Measurement Survey Methodology Note (2014) provides more information on the selection of this particular set of skills and on the way these skills are assessed in the STEP survey. See Tables A8 and A9, below, for lists of skills and associated scales. 102 Annex Table 8: Job-relevant skills Computer use Intensity of use Level “As a part of your work do you use a computer?” “As a part of your life [outside work] have you used a computer in the past 3 months?” Does not use a computer/use a computer almost never = Does not use 0 Uses computer less than three times per week = Low 1 Uses computer three times or more per week = Medium 2 Uses computer every day = High 3 Contact with clients Intensity of use Level “As part of this work, do you have any contact with people other than co-workers, for example customers, clients, students, or the public?” * Does not have any contacts with clients = Does not use 0 Involvement scale ranges from 1 to 4 = Low 1 Involvement scale ranges from 5 to 7 = Medium 2 Involvement scale ranges from 8 to 10 = High 3 * Scale ranges from 1 to 10, where 1 is little involvement and 10 means much of the work involves meeting or interacting with people other than co-workers. Solving and learning at work Intensity of use Level Item 1. “Some tasks are pretty easy and can be done right away or after getting a little help from others. Other tasks require more thinking to figure out how they should be done. As part of this work, how often do you have to undertake tasks that require at least 30 minutes of thinking?” Never = Does not use 0 Less than once per month = Low 1 Less than once a week but at least once a month OR at least = Medium 2 once a week but not every day Average of 2 items Every day = High 3 Item 2. “How often does (did) this work involve learning new things?” Rarely = Does not use 0 At least 2-3 months or at least once a month = Low 1 At least once a week = Medium 2 Every day = High 3 103 Autonomy and repetitiveness Intensity of use Level Item 1. “Still thinking of your work, how much freedom do you have to decide how to do your work in your own way, rather than following a fixed procedure or a supervisor's instructions? Use any number from 1 to 10 where 1 is no freedom and 10 is complete freedom.” Decision freedom scale from 1 to 2 = Close to none 0 Decision freedom scale from 3 to 6 = Low 1 Decision freedom scale from 7 to 9 = Medium 2 Decision freedom scale 10 = High 3 Average of 2 items Item 2. “How often does (did) this work involve carrying out short, repetitive tasks?” Almost all the time = Close to none 3 More than half the time = Low 2 Less than half the time = Medium 1 Almost never = High 0 Annex Table 9: Aggregation of variables Score Use of External Thinking Learning Autonomy Repetitive Physical tasks computer (at inter- at work of new things at work tasks at at work work and personal at least 30 at work work outside it) skills at work minutes 0 Does not use Does not Never Rarely Decision Almost Not at all (not a computer/ have any freedom all the physically used/clos almost never contacts scale from time demanding e to none) uses a with clients 1 to 2 computer 1 Uses Involve- Less than At least 2- Decision More Physical (low) computer ment scale once per 3 months freedom than half demand scale less than ranges from month or at least scale from the time ranges from 2 three times 1 to 4 once a 3 to 6 to 4 per week month 2 Uses Involvement Less than At least Decision Less Physical (medium) computer scale ranges once a once a freedom than half demand scale three times from 5 to 7 week but week scale from the time ranges from 5 or more per at least 7 to 9 to 6 week once a month OR at least once a week but not every day 3 Uses Involve- Every day Every day Decision Almost Physical (high) computer ment scale freedom never demand scale every day ranges from scale 10 ranges from 7 8 to 10 to 10 104 Annex 3. Selected Statistical Results Annex Table 10: Probability of using computer skills at work by ECE participation Marginal Effects on Probability of Computer Use Armenia VARIABLES None Low Medium High (1) (2) (3) (4) 1.gender 0.027* 0.002 0.002 -0.030* (0.016) (0.001) (0.001) (0.018) years_educ -0.039*** -0.003*** -0.002*** 0.043*** (0.003) (0.000) (0.000) (0.003) 1.ece -0.019 -0.001 -0.001 0.021 (0.016) (0.001) (0.001) (0.018) 1.mother_educ 2.mother_educ -0.052 -0.003 -0.001 0.055 (0.049) (0.002) (0.001) (0.050) 3.mother_educ -0.107** -0.007*** -0.004*** 0.117** (0.050) (0.002) (0.001) (0.052) 4.mother_educ 0.018 0.001 -0.000 -0.019 (0.067) (0.002) (0.001) (0.068) 1.father_educ 2.father_educ -0.035 -0.003 -0.002 0.039 (0.056) (0.004) (0.003) (0.062) 3.father_educ -0.139** -0.015*** -0.017*** 0.170*** (0.054) (0.005) (0.006) (0.062) 4.father_educ -0.010 -0.001 -0.000 0.011 (0.054) (0.003) (0.002) (0.059) 1.shocks2 0.054** 0.003** 0.002*** -0.059** (0.024) (0.001) (0.001) (0.026) 2.shocks2 0.028 0.002 0.001 -0.031 (0.031) (0.002) (0.001) (0.034) 2.ses -0.078** -0.004*** -0.001 0.083*** (0.031) (0.001) (0.001) (0.031) 3.ses -0.103*** -0.006*** -0.003** 0.111*** (0.031) (0.002) (0.001) (0.032) 2.age_group 0.132*** 0.024*** 0.037*** -0.193*** (0.019) (0.004) (0.006) (0.027) 3.age_group 0.196*** 0.031*** 0.043*** -0.271*** (0.023) (0.005) (0.006) (0.031) 4.age_group 0.360*** 0.038*** 0.040*** -0.438*** (0.021) (0.005) (0.006) (0.027) Observations 2,935 2,935 2,935 2,935 Did Not Participate in ECE 0.262 0.0769 0.154 0.507 Participated in ECE 0.240 0.0740 0.151 0.535 Participated in ECE: g1524 0.0655 0.0333 0.0874 0.814 g2534 0.190 0.0658 0.142 0.603 g3544 0.257 0.0763 0.153 0.513 g4565 0.441 0.0882 0.148 0.295 Did Not Participate in ECE: g1524 0.0747 0.0365 0.0938 0.795 g2534 0.209 0.0692 0.146 0.576 g3544 0.280 0.0790 0.155 0.486 g4565 0.468 0.0882 0.151 0.319 Standard errors in parentheses 105 *** p<0.01, ** p<0.05, * p<0.1 106 Annex Table 11: Probability of extraversion, conscientiousness, and grit by ECE participation by ECE participation Armenia VARIABLES Extraversion Conscientiousness grit (1) (2) (6) 1.gender 0.053** 0.015 0.030 (0.025) (0.022) (0.027) 1.ece 0.050** 0.051* 0.047* (0.025) (0.027) (0.025) years_educ 0.008* 0.013*** 0.007* (0.004) (0.004) (0.004) 1.mother_educ 2.mother_educ -0.088 -0.142** -0.113* (0.064) (0.056) (0.061) 3.mother_educ -0.087 -0.113** -0.101 (0.066) (0.057) (0.064) 4.mother_educ -0.079 -0.220*** -0.151* (0.094) (0.074) (0.085) 1.father_educ 2.father_educ 0.074 0.092 0.054 (0.096) (0.068) (0.092) 3.father_educ 0.100 0.036 0.029 (0.090) (0.068) (0.083) 4.father_educ 0.149* 0.066 0.038 (0.084) (0.062) (0.078) 1.shocks2 -0.072* 0.024 -0.004 (0.039) (0.031) (0.040) 2.shocks2 0.010 0.032 -0.081 (0.054) (0.043) (0.058) 2.ses 0.013 0.008 0.002 (0.044) (0.042) (0.042) 3.ses -0.013 0.030 -0.016 (0.049) (0.049) (0.051) 2.age_group -0.034 0.068** 0.015 (0.042) (0.032) (0.038) 3.age_group -0.112*** 0.112*** 0.092** (0.037) (0.036) (0.043) 4.age_group -0.158*** 0.136*** 0.171*** (0.036) (0.034) (0.038) Constant 2.917*** 2.975*** 2.994*** 107 (0.111) (0.098) (0.111) Observations 2,934 2,934 2,933 R-squared 0.019 0.030 0.024 Did Not Participate in ECE 3.010 3.205 3.114 Participated in ECE 3.060 3.256 3.161 Participated in ECE: g1524 3.146 3.119 3.030 g2534 3.113 3.187 3.092 g3544 3.034 3.282 3.169 g4565 2.988 3.255 3.201 Did Not Participate in ECE: g1524 3.096 3.170 3.077 g2534 3.063 3.238 3.046 g3544 2.984 3.231 3.122 g4565 2.938 3.306 3.247 Standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 108 Annex Table 12: Predicted score in conscientiousness by reading level Marginal Effects on Probability of Reading Proficiency Armenia VARIABLES Level 0/1 Level 2 Level 3 Level 4/5 (1) (2) (3) (4) 1.gender -0.002 -0.000 0.002 0.000 (0.016) (0.006) (0.019) (0.003) 2.asset_p 0.024 0.010 -0.029 -0.005 (0.022) (0.011) (0.028) (0.005) 3.asset_p 0.055** 0.017 -0.063** -0.009* (0.027) (0.011) (0.031) (0.005) 1.mother_educ 2.mother_educ -0.081 0.001 0.073* 0.007* (0.052) (0.012) (0.041) (0.004) 3.mother_educ -0.120** -0.015 0.121*** 0.014*** (0.052) (0.014) (0.041) (0.005) 4.mother_educ -0.030 0.004 0.024 0.002 (0.069) (0.012) (0.055) (0.005) 1.shocks2 -0.007 -0.003 0.009 0.001 (0.023) (0.010) (0.029) (0.004) 2.shocks2 0.019 0.005 -0.021 -0.003 (0.029) (0.007) (0.032) (0.004) 2.ses -0.040 -0.011 0.044 0.006 (0.032) (0.008) (0.034) (0.004) 3.ses -0.023 -0.004 0.024 0.003 (0.032) (0.006) (0.033) (0.004) 2.age_group 0.047** 0.026** -0.062** -0.010** (0.019) (0.012) (0.026) (0.005) 3.age_group 0.030 0.019 -0.042 -0.007 (0.019) (0.013) (0.027) (0.005) 4.age_group 0.069*** 0.031** -0.087*** -0.014** (0.021) (0.013) (0.027) (0.006) 2.edu1 0.049 0.021 -0.061 -0.009 (0.092) (0.044) (0.109) (0.018) 3.edu1 0.042 0.021 -0.054 -0.009 (0.087) (0.043) (0.105) (0.018) 4.edu1 -0.027 -0.004 0.028 0.003 (0.085) (0.044) (0.104) (0.018) extraversion -0.009 -0.003 0.010 0.002 (0.011) (0.004) (0.013) (0.002) 109 conscientiousness -0.032** -0.012* 0.038** 0.006* (0.015) (0.006) (0.018) (0.003) openness -0.018 -0.006 0.021 0.003 (0.015) (0.006) (0.018) (0.003) stability 0.001 0.000 -0.001 -0.000 (0.011) (0.004) (0.013) (0.002) agreeableness 0.002 0.001 -0.002 -0.000 (0.017) (0.006) (0.020) (0.003) grit 0.011 0.004 -0.014 -0.002 (0.014) (0.006) (0.017) (0.003) decision -0.009 -0.003 0.011 0.002 (0.015) (0.006) (0.018) (0.003) hostile 0.018 0.007 -0.022 -0.003 (0.012) (0.004) (0.014) (0.002) 2.school_type 0.082 0.007 -0.080 -0.009* (0.066) (0.011) (0.054) (0.005) 3.school_type -0.030 -0.021 0.042 0.008 (0.049) (0.037) (0.069) (0.015) 2.school_prox 0.012 0.004 -0.014 -0.002 (0.028) (0.009) (0.032) (0.005) 3.school_prox 0.103* 0.003 -0.095** -0.011** (0.062) (0.019) (0.044) (0.005) Observations 2,908 2,908 2,908 2,908 Conscientiousness (1) 0.245 0.562 0.185 0.008 (0.046) (0.018) (0.036) (0.004) Conscientiousness (2) 0.206 0.563 0.219 0.011 (0.026) (0.016) (0.025) (0.004) Conscientiousness (3) 0.172 0.557 0.256 0.016 (0.014) (0.017) (0.018) (0.004) Conscientiousness (4) 0.141 0.543 0.294 0.022 110 Annex Table 13: Predicted score in grit and decision making by educational attainment Armenia VARIABLES grit decision (6) (7) 1.gender 0.033 0.013 (0.028) (0.026) 2.asset_p 0.050 0.001 (0.039) (0.037) 3.asset_p 0.113** 0.029 (0.049) (0.040) 1.mother_educ 2.mother_educ -0.126** -0.023 (0.060) (0.067) 3.mother_educ -0.121* -0.043 (0.064) (0.065) 4.mother_educ -0.167** -0.078 (0.084) (0.098) 1.shocks2 0.001 -0.017 (0.038) (0.035) 2.shocks2 -0.094 -0.033 (0.057) (0.044) 2.ses -0.000 0.053 (0.042) (0.040) 3.ses -0.028 0.003 (0.048) (0.048) 2.age_group 0.024 -0.088** (0.036) (0.036) 3.age_group 0.088** -0.081** (0.040) (0.038) 4.age_group 0.157*** -0.094*** (0.033) (0.034) 2.edu1 0.215* 0.286* (0.117) (0.161) 3.edu1 0.211* 0.308* (0.108) (0.160) 4.edu1 0.266** 0.454*** (0.107) (0.159) 2.school_type 0.062 -0.088 (0.072) (0.077) 3.school_type -0.006 0.136* 111 (0.104) (0.081) 2.school_prox -0.099** -0.154*** (0.047) (0.048) 3.school_prox 0.113* -0.077 (0.059) (0.057) Constant 2.903*** 2.912*** (0.120) (0.180) Observations 2,971 2,972 R-squared 0.035 0.037 Predicted Scores 1bn.edu1 2.905 2.833 (0.106) (0.158) 2.edu1 3.120 3.119 (0.049) (0.046) 3.edu1 3.117 3.141 (0.026) (0.025) 4.edu1 3.171 3.286 (0.025) (0.023) 112 Annex Table 14: Probability of scoring at higher levels in reading proficiency by mother’s educational attainment Marginal Effects on Probability of Reading Proficiency Armenia VARIABLES Level 0/1 Level 2 Level 3 Level 4/5 (1) (2) (3) (4) 1.gender -0.002 -0.000 0.002 0.000 (0.016) (0.006) (0.019) (0.003) 2.asset_p medium 0.024 0.010 -0.029 -0.005 (0.022) (0.011) (0.028) (0.005) 3.asset_p high 0.055** 0.017 -0.063** -0.009* (0.027) (0.011) (0.031) (0.005) 1.mother_educ primary 2.mother_educ secondary -0.081 0.001 0.073* 0.007* (0.052) (0.012) (0.041) (0.004) 3.mother_educ tertiary -0.120** -0.015 0.121*** 0.014*** (0.052) (0.014) (0.041) (0.005) 4.mother_educ don't know -0.030 0.004 0.024 0.002 (0.069) (0.012) (0.055) (0.005) 1.shocks2 1 shock -0.007 -0.003 0.009 0.001 (0.023) (0.010) (0.029) (0.004) 2.shocks2 > 1 shock 0.019 0.005 -0.021 -0.003 (0.029) (0.007) (0.032) (0.004) 2.ses med -0.040 -0.011 0.044 0.006 (0.032) (0.008) (0.034) (0.004) 3.ses high -0.023 -0.004 0.024 0.003 (0.032) (0.006) (0.033) (0.004) 2.age_group 0.047** 0.026** -0.062** -0.010** (0.019) (0.012) (0.026) (0.005) 3.age_group 0.030 0.019 -0.042 -0.007 (0.019) (0.013) (0.027) (0.005) 4.age_group 0.069*** 0.031** -0.087*** -0.014** (0.021) (0.013) (0.027) (0.006) 2.edu1 0.049 0.021 -0.061 -0.009 (0.092) (0.044) (0.109) (0.018) 3.edu1 0.042 0.021 -0.054 -0.009 (0.087) (0.043) (0.105) (0.018) 4.edu1 -0.027 -0.004 0.028 0.003 (0.085) (0.044) (0.104) (0.018) extraversion -0.009 -0.003 0.010 0.002 (0.011) (0.004) (0.013) (0.002) 113 conscientiousness -0.032** -0.012* 0.038** 0.006* (0.015) (0.006) (0.018) (0.003) openness -0.018 -0.006 0.021 0.003 (0.015) (0.006) (0.018) (0.003) stability 0.001 0.000 -0.001 -0.000 (0.011) (0.004) (0.013) (0.002) agreeableness 0.002 0.001 -0.002 -0.000 (0.017) (0.006) (0.020) (0.003) grit 0.011 0.004 -0.014 -0.002 (0.014) (0.006) (0.017) (0.003) decision -0.009 -0.003 0.011 0.002 (0.015) (0.006) (0.018) (0.003) hostile 0.018 0.007 -0.022 -0.003 (0.012) (0.004) (0.014) (0.002) 2.school_type 0.082 0.007 -0.080 -0.009* (0.066) (0.011) (0.054) (0.005) 3.school_type -0.030 -0.021 0.042 0.008 (0.049) (0.037) (0.069) (0.015) 2.school_prox 0.012 0.004 -0.014 -0.002 (0.028) (0.009) (0.032) (0.005) 3.school_prox 0.103* 0.003 -0.095** -0.011** (0.062) (0.019) (0.044) (0.005) Observations 2,908 2,908 2,908 2,908 114 Annex Table 15: Adults’ probability of using solving and learning skills by mother’s educational Marginal Effects on Probability of Cognitive Challenge Intensity (solving and learning skills at work) Armenia VARIABLES None Low Medium High (1) (2) (3) (4) 1.gender 0.044*** 0.021*** -0.005* -0.061*** (0.016) (0.008) (0.003) (0.023) 2.asset_p -0.049** -0.023** 0.007 0.065** (0.020) (0.009) (0.004) (0.026) 3.asset_p -0.036 -0.016 0.006 0.046 (0.027) (0.013) (0.005) (0.035) 1.mother_educ 2.mother_educ -0.185*** -0.041*** 0.081** 0.146*** (0.057) (0.009) (0.031) (0.034) 3.mother_educ -0.223*** -0.061*** 0.085*** 0.199*** (0.058) (0.010) (0.031) (0.034) 4.mother_educ -0.093 -0.011 0.048 0.057 (0.085) (0.015) (0.042) (0.057) 1.shocks2 0.005 0.002 -0.001 -0.007 (0.021) (0.010) (0.003) (0.028) 2.shocks2 0.009 0.004 -0.001 -0.012 (0.032) (0.015) (0.005) (0.041) 2.ses -0.088** -0.033*** 0.024 0.096*** (0.038) (0.012) (0.015) (0.035) 3.ses -0.091** -0.034*** 0.024 0.100*** (0.040) (0.013) (0.016) (0.037) 2.age_group -0.048* -0.022* 0.007 0.063* (0.029) (0.013) (0.007) (0.036) 3.age_group -0.027 -0.012 0.005 0.034 (0.030) (0.012) (0.007) (0.036) 4.age_group -0.006 -0.002 0.001 0.007 (0.033) (0.013) (0.008) (0.038) 5.age_group -0.051 -0.024* 0.007 0.068* (0.031) (0.015) (0.007) (0.041) 2.edu1 -0.019 -0.004 0.010 0.013 (0.164) (0.030) (0.084) (0.110) 3.edu1 -0.044 -0.010 0.021 0.033 (0.163) (0.029) (0.083) (0.109) 4.edu1 -0.164 -0.065** 0.050 0.180 (0.162) (0.030) (0.083) (0.109) extraversion -0.003 -0.001 0.000 0.004 115 (0.015) (0.007) (0.002) (0.020) conscientiousness 0.000 0.000 -0.000 -0.000 (0.021) (0.010) (0.003) (0.028) openness -0.079*** -0.038*** 0.010* 0.107*** (0.020) (0.009) (0.005) (0.025) stability -0.027** -0.013** 0.003 0.037** (0.013) (0.006) (0.002) (0.017) agreeableness 0.018 0.008 -0.002 -0.024 (0.016) (0.008) (0.002) (0.021) grit -0.026* -0.012* 0.003 0.035* (0.015) (0.007) (0.003) (0.020) decision -0.033** -0.016** 0.004 0.044** (0.017) (0.008) (0.003) (0.022) hostile -0.023* -0.011* 0.003 0.031* (0.014) (0.006) (0.002) (0.018) 2.school_type -0.005 -0.002 0.001 0.006 (0.089) (0.044) (0.009) (0.124) 3.school_type 0.049 0.020 -0.011 -0.057 (0.108) (0.036) (0.034) (0.111) 2.school_prox -0.012 -0.006 0.001 0.017 (0.030) (0.015) (0.002) (0.043) 3.school_prox -0.014 -0.007 0.001 0.019 (0.025) (0.013) (0.002) (0.036) Observations 1,119 1,119 1,119 1,119 116 Annex Table 16: Adults’ probability of using reading skills by household vulnerability at age 15 (measured in number of shocks experienced) Marginal Effects on Probability of Reading Intensity Level Armenia VARIABLES None Low Medium High (1) (2) (3) (4) 1.ece -0.015 -0.003 0.001 0.017 (0.015) (0.003) (0.001) (0.018) 1.gender -0.030* -0.006** 0.002 0.034* (0.016) (0.003) (0.001) (0.018) 2.asset_p -0.014 -0.003 0.001 0.016 (0.021) (0.004) (0.002) (0.024) 3.asset_p -.075*** -.019*** 0.000 0.093*** (0.024) (0.006) (0.002) (0.030) 1.mother_educ 2.mother_educ 0.015 0.003 -0.001 -0.017 (0.042) (0.008) (0.003) (0.046) 3.mother_educ -0.051 -0.012 0.001 0.061 (0.043) (0.009) (0.003) (0.049) 4.mother_educ 0.031 0.005 -0.003 -0.033 (0.053) (0.009) (0.006) (0.056) 1.shocks2 -0.009 -0.002 0.000 0.010 (0.020) (0.004) (0.001) (0.024) 2.shocks2 -0.051** -0.013* -0.000 0.064* (0.025) (0.007) (0.002) (0.034) 2.ses 0.008 0.002 -0.000 -0.009 (0.029) (0.006) (0.002) (0.034) 3.ses -0.018 -0.004 0.001 0.022 (0.031) (0.007) (0.001) (0.036) 2.age_group 0.183*** 0.058*** 0.015** -0.256*** (0.019) (0.008) (0.007) (0.026) 3.age_group 0.172*** 0.056*** 0.017*** -.245*** (0.020) (0.008) (0.006) (0.028) 4.age_group 0.197*** 0.059*** 0.013** -.270*** (0.023) (0.008) (0.007) (0.031) 5.age_group 0.163*** 0.055*** 0.018*** -.235*** (0.026) (0.009) (0.006) (0.035) 2.edu1 0.260*** 0.016 0.072*** 0.173*** (0.079) (0.015) (0.026) (0.045) 3.edu1 -.273*** 0.014 0.074*** 0.185*** (0.074) (0.015) (0.025) (0.037) 117 4.edu1 -.404*** -0.014 0.080*** 0.338*** (0.075) (0.016) (0.026) (0.038) extraversion -0.003 -0.001 0.000 0.003 (0.012) (0.002) (0.000) (0.014) conscientiousness -.055*** -.012*** 0.002 0.064*** (0.016) (0.004) (0.002) (0.019) openness -.103*** -.022*** 0.004 0.121*** (0.017) (0.004) (0.003) (0.020) stability -0.020* -0.004* 0.001 0.024* (0.011) (0.002) (0.001) (0.013) agreeableness -0.020 -0.004 0.001 0.023 (0.015) (0.003) (0.001) (0.018) grit 0.007 0.002 -0.000 -0.009 (0.014) (0.003) (0.001) (0.016) decision 0.010 0.002 -0.000 -0.012 (0.016) (0.003) (0.001) (0.018) hostile 0.027** 0.006** -0.001 -0.031** (0.013) (0.003) (0.001) (0.015) 2.school_type 0.064 0.010 -0.006 -0.069 (0.090) (0.011) (0.013) (0.088) 3.school_type 0.006 0.001 -0.000 -0.007 (0.057) (0.012) (0.003) (0.066) 2.school_prox -0.001 -0.000 0.000 0.002 (0.027) (0.006) (0.001) (0.031) 3.school_prox -.192*** -.083*** -0.056** 0.331*** (0.024) (0.017) (0.022) (0.060) Observations 2,921 2,921 2,921 2,921 0bn.shocks2 0.232 0.178 0.244 0.346 (0.014) (0.011) (0.013) (0.018) 1.shocks2 0.222 0.175 0.245 0.358 (0.022) (0.012) (0.013) (0.028) 2.shocks2 0.179 0.160 0.244 0.417 (0.026) (0.012) (0.014) (0.038) 118 Annex Table 17: Probability models for Labor Force Participation and Employment Labor Force Participation Employment Variables 1 2 3 4 5 6 7 8 9 10 Women -0.3140*** -0.1419*** -0.1524*** -0.1348*** -0.1471*** -0.0941*** -0.0784 -0.0830 -0.0736 -0.0831 (0.0260) (0.0431) (0.0421) (0.0413) (0.0428) (0.0296) (0.0546) (0.0528) (0.0537) (0.0539) Years of Education 0.0369*** 0.0209*** 0.0185*** 0.0168*** 0.0161*** 0.0263*** 0.0028 0.0033 -0.0009 -0.0012 (0.0063 (0.0040) (0.0050) (0.0049) (0.0050) (0.0052) (0.0049) ) (0.0063) (0.0064) (0.0065) Reading Score Level 2 -0.0091 -0.0132 0.0412 0.0477 (0.0411) (0.0429) (0.0583) (0.0575) Reading Score Level 3 -0.0185 -0.0355 0.0716 0.0629 (0.0461) (0.0463) (0.0720) (0.0721) Reading Score Level 4 and 5 0.0295 -0.0050 0.1811 0.1857 (0.1211) (0.1227) (0.1277) (0.1290) Extraversion 0.0131 0.0118 0.0089 0.0049 (0.0221) (0.0228) (0.0245) (0.0252) Conscientiousness 0.1042*** 0.1021*** 0.1236*** 0.1206*** (0.0250) (0.0261) (0.0382) (0.0390) Openness 0.0417 0.0325 -0.0313 -0.0297 (0.0296) (0.0298) (0.0389) (0.0400) Emotional Stability 0.0050 0.0063 -0.0231 -0.0232 (0.0205) (0.0209) (0.0270) (0.0270) Agreeableness 0.0008 0.0024 -0.0272 -0.0281 (0.0252) (0.0257) (0.0310) (0.0319) Grit 0.0121 0.0131 0.0482 0.0543* (0.0231) (0.0236) (0.0311) (0.0316) Hostile Bias 0.0303 0.0285 -0.0379 -0.0361 (0.0195) (0.0199) (0.0254) (0.0257) Decision Making 0.0512* 0.0509* -0.0225 -0.0243 (0.0272) (0.0278) (0.0332) (0.0337) Risk Aversion -0.0345*** -0.0363*** 0.0249 0.0252 (0.0116) (0.0119) (0.0158) (0.0158) Computer Use (Low) 0.1051** 0.0935* -0.1779** -0.2227** (0.0477) (0.0500) (0.0884) (0.0893) Computer Use (Medium) 0.0505 0.0447 -0.0156 -0.0241 (0.0402) (0.0425) (0.0600) (0.0604) Computer Use (High) 0.0992*** 0.0891*** 0.0669 0.0526 (0.0330) (0.0344) (0.0430) (0.0433) Household Related No Yes Yes Yes Yes No Yes Yes Yes Yes Observations 2,946 2,856 2,929 2,936 2,849 1,580 1,535 1,569 1,573 1,531 Mean probability 0.537 0.557 0.556 0.556 0.557 0.648 0.735 0.741 0.735 0.744 Standard errors in parentheses. Marginal effects estimated after a Probit model. Omitted categories are Reading Proficiency Level 1 and below and computer use (not use). The household related variables include: whether individual has a spouse, and its interaction with gender; whether there are children living in the household, whether there are children under 6 years old, and their interaction; whether individual has labor dependents; mother and father education level, socio economic status at age 15; and 119 whether individual had economic shocks at age 12. *** p<0.01, ** p<0.05, * p<0.1 120 Annex Table 18: Returns on Hourly Earnings by Age groups. Age group 15-65 15-24 25-34 35-44 45+ (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) Women -0.3448*** -0.3439*** -0.3418*** -0.3414*** -0.3303*** -0.3291*** -0.3313*** -0.3307*** -0.1475 -0.5031*** -0.3779*** -0.2953*** (0.0509) (0.0509) (0.0495) (0.0493) (0.0525) (0.0524) (0.0515) (0.0513) (0.1286) (0.0966) (0.1020) (0.0738) Years of Education 0.0244*** 0.0246*** 0.0233*** 0.0234*** 0.0189** 0.0192** 0.0178** 0.0180** 0.0234 0.0605*** 0.0477** -0.0135 (0.0080) (0.0083) (0.0082) (0.0086) (0.0080) (0.0083) (0.0082) (0.0085) (0.0300) (0.0199) (0.0213) (0.0172) Reading Proficiency Score -0.0002 -0.0001 -0.0002 -0.0001 0.0026 -0.0011 -0.0008 -0.0004 (0.0008) (0.0008) (0.0008) (0.0008) (0.0019) (0.0020) (0.0018) (0.0011) Extraversion 0.0088 0.0085 0.0155 0.0150 -0.3438*** 0.0859 0.1425** -0.0453 (0.0315) (0.0316) (0.0316) (0.0318) (0.1092) (0.0646) (0.0717) (0.0495) Conscientiousness 0.0380 0.0384 0.0422 0.0427 0.0930 0.1434* 0.0228 0.0348 (0.0434) (0.0439) (0.0445) (0.0448) (0.1561) (0.0811) (0.1148) (0.0742) Openness 0.0657 0.0656 0.0582 0.0579 0.1343 -0.0343 0.0204 0.0895 (0.0465) (0.0466) (0.0475) (0.0476) (0.1169) (0.1204) (0.0816) (0.0684) Emotional Stability -0.0020 -0.0019 -0.0041 -0.0041 -0.1955*** -0.0494 0.0294 -0.0383 (0.0345) (0.0344) (0.0342) (0.0341) (0.0736) (0.0845) (0.0575) (0.0517) Agreeableness -0.0872** -0.0870** -0.0759** -0.0756** 0.0639 -0.2069** -0.0164 -0.0964* (0.0368) (0.0368) (0.0367) (0.0367) (0.1159) (0.0940) (0.0758) (0.0541) Grit 0.0861** 0.0858** 0.0654* 0.0648* 0.0797 0.1474* -0.0402 0.1481** (0.0382) (0.0383) (0.0383) (0.0382) (0.1071) (0.0848) (0.0855) (0.0622) Hostile Bias 0.0672** 0.0672** 0.0523* 0.0522* -0.1306 -0.0013 0.1069** 0.0627 (0.0312) (0.0312) (0.0308) (0.0308) (0.1072) (0.0783) (0.0527) (0.0482) Decision Making -0.0846** -0.0845** -0.0901** -0.0900** -0.1298 -0.0941 -0.1088* -0.1020* (0.0407) (0.0408) (0.0412) (0.0412) (0.1772) (0.1124) (0.0605) (0.0582) Risk Aversion 0.0448** 0.0448** 0.0408** 0.0408** -0.0055 0.0240 0.1043*** 0.0088 (0.0182) (0.0182) (0.0179) (0.0179) (0.0614) (0.0380) (0.0345) (0.0273) Computer Use (Low) 0.1042 0.1045 0.0734 0.0735 0.7988** 0.3596* -0.0356 0.2406 (0.1126) (0.1130) (0.1217) (0.1220) (0.3486) (0.1906) (0.1593) (0.2695) Computer Use (Medium) 0.0515 0.0561 0.0220 0.0240 0.1293 -0.0588 0.1146 0.1590 (0.0988) (0.1020) (0.0862) (0.0888) (0.2975) (0.1737) (0.2083) (0.1616) Computer Use (High) 0.1162** 0.1178** 0.1182** 0.1189** -0.1398 0.0588 0.1949* 0.1346* (0.0509) (0.0510) (0.0496) (0.0495) (0.1113) (0.1243) (0.1121) (0.0798) Interpersonal Contact (Low) -0.1666* -0.1648* -0.1363 -0.1355 -0.3796 0.0467 -0.2020 -0.2023 (0.0874) (0.0874) (0.0851) (0.0850) (0.3068) (0.2418) (0.1323) (0.1256) 121 Interpersonal Contact (Medium) -0.1053* -0.1043* -0.0806 -0.0801 0.1373 -0.2068** -0.1115 -0.0437 (0.0568) (0.0569) (0.0584) (0.0585) (0.1006) (0.0982) (0.1142) (0.0936) Interpersonal Contact (High) -0.1103** -0.1079** -0.0956* -0.0944* -0.0680 -0.2314** -0.1331 -0.0308 (0.0505) (0.0509) (0.0497) (0.0503) (0.1271) (0.1092) (0.1066) (0.0823) Problem Solving and Learning (Low) 0.0953 0.0955 0.0818 0.0822 0.1907 0.3872** -0.2903** 0.1025 (0.0677) (0.0678) (0.0675) (0.0676) (0.1751) (0.1616) (0.1410) (0.1116) Problem Solving and Learning (Medium) 0.1793*** 0.1807*** 0.1471** 0.1482** 0.3929** 0.2632** -0.0771 0.1520 (0.0672) (0.0672) (0.0652) (0.0653) (0.1602) (0.1186) (0.1358) (0.1304) Problem Solving and Learning (High) 0.1810** 0.1818** 0.1717** 0.1724** 0.3941** 0.4159*** -0.0267 0.1771 (0.0835) (0.0834) (0.0796) (0.0796) (0.2006) (0.1307) (0.1547) (0.1382) Autonomy (Low) -0.0171 -0.0173 -0.0080 -0.0081 -0.1650 0.1203 0.1967* -0.0846 (0.0549) (0.0550) (0.0547) (0.0548) (0.1574) (0.1083) (0.1141) (0.0799) Autonomy (Medium) 0.0813 0.0804 0.0702 0.0699 -0.0491 0.1219 0.3983*** 0.0038 (0.0616) (0.0620) (0.0638) (0.0642) (0.1252) (0.1351) (0.1136) (0.0894) Autonomy (High) 0.1201 0.1201 0.0955 0.0955 0.2413 0.3179 0.3399** -0.1058 (0.0856) (0.0854) (0.0859) (0.0859) (0.2363) (0.2165) (0.1366) (0.1164) Constant -0.5568** -0.5158* -0.8186*** -0.8050** -0.6448*** -0.5941** -0.8130** -0.7865** -0.9325 -1.7140** -1.1735 0.8481 (0.2323) (0.2710) (0.3126) (0.3304) (0.2395) (0.2775) (0.3221) (0.3414) (1.0666) (0.7977) (1.0411) (0.9765) Observations 744 744 743 743 727 727 727 727 75 177 159 316 R-squared 0.2981 0.2984 0.3252 0.3254 0.3328 0.3332 0.3536 0.3538 0.7169 0.4445 0.6123 0.4231 Standard errors in parentheses. Regression model with log hourly earnings as a dependent variables. The model includes potential experience and potential experience squared, mother's education, and indicator variables for wage workers, occupations and economic sector. The omitted categories for computer use, interpersonal contact, problem solving and learning, autonomy is task not used. 122