WPS8155 Policy Research Working Paper 8155 Can Business Input Improve the Effectiveness of Worker Training? Evidence from Brazil’s Pronatec-MDIC Stephen D. O’Connell Lucas Ferreira Mation João Bevilaqua T. Basto Mark A. Dutz Trade and Competitiveness Global Practice Group July 2017 Policy Research Working Paper 8155 Abstract This study evaluates the employment effects of a public- distinguishable from those of a broader and institutional- ly-run national technical vocational education training ly-similar publicly-administered skills training program run program in Brazil that explicitly takes input from firms in at the same time that did not take input from firms. The determining the location, scale, and skill content of courses study finds that the demand-driven program better aligned offered. Using exogenous course capacity restrictions, the skill training with future aggregate occupational employ- study finds that those completing the course following ment growth—suggesting the input from firms captured receipt of a course offer have an 8.6 percent increase in meaningful information about growth in skill demand. employment over the year following course completion. Courses offered in occupations that grew more over the These effects come from previously unemployed train- year following requests exhibited larger employment effects, ees who find employment at non-requesting firms. The explaining the effectiveness of the demand-driven model. demand-driven program’s effects are larger and statistically This paper is a product of the Trade and Competitiveness Global Practice Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at mdutz@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team Can Business Input Improve the Effectiveness of Worker Training? Evidence from Brazil’s Pronatec-MDIC Stephen D. O’Connell, Lucas Ferreira Mation, ao Bevilaqua T. Basto, Mark A. Dutz Jo˜ July 31, 2017 Keywords: Vocational education and technical training; Training programs; Business de- velopment services; Labor demand; Unemployment; Brazil. (JEL J24, J23, J31, J68, J62, M53) Author affiliations and contact: O’Connell: Corresponding author; Department of Economics, Massachusetts Institute of Technology (MIT) and the IZA Institute of Labor omica Aplicada Economics, soconnel@mit.edu; Ferreira Mation: Instituto de Pesquisa Econˆ (IPEA), lucas.mation@ipea.gov.br; Bevilaqua: World Bank, jbevilaqua@worldbank.org; Dutz: World Bank, mdutz@worldbank.org. Acknowledgments: The authors are grateful for invaluable research assistance by Nicolas Soares Pinto and Pui Shen Yoong. The authors thank several colleagues for helpful input and discussions, including Onur Altında˘g, John Giles, Aguinaldo Nogueira Maciente, David McKenzie, Pedro Olinto, Truman Packard, and Romero Rocha, as well as participants in seminars at The World Bank, Pontificia Universidad Javeriana, the 2016 Jobs for Development Conference, Secretaria Especial do Assuntos Estrat´ encia da egicos - Presidˆ Rep´ublica, and the Minist´ ustria, Com´ erio da Ind´ cos. Staff at MDIC and MEC were ercio Exterior e Servi¸ instrumental in providing data and details of program administration. The study was conducted in partner- ship with the Instituto de Pesquisa Econˆomica Aplicada (IPEA) and funded as part of Brazil’s Productivity Programmatic Approach (P152871) at the World Bank. Views expressed are those of the authors and not of any institution with which they are associated. 1 Overview A primary objective of active labor market policy is to shape a workforce that matches and continuously upgrades the skill distribution to the evolving labor demand of enterprises. Vocational training programs remain among the most common incarnation of active labor market policies despite a sizable historical literature providing only mixed evidence as to their effectiveness (Card et al. (2015), Kluve (2010), and Heinrich et al. (2013), among others). Betcherman et al. (2004) provide an early synthesis of research of over 150 evaluations of active labor market programs, suggesting a cautious approach in what such policies can “realistically achieve.” Whether the design of publicly-funded skills training can increase its effectiveness is par- ticularly important for policymakers choosing whether to start, adapt, or abandon such programs. In this paper, we investigate the employment returns to a large-scale, publicly administered, technical skills training program in Brazil that explicitly takes input from firms in determining course offerings. To our knowledge, ours is the first study to evaluate a “demand-driven” training program in which firms provide direct input to the government as to the type, volume, and location of skills in demand by private-sector businesses. Be- cause of the unique institutional setting, we then compare program effects to those of an otherwise-similar national skills training program run in parallel within the same institu- tional and logistical context. This allows us to more meaningfully gauge the potential for a demand-driven design to improve program effectiveness. Using comprehensive monthly administrative panel data on employment, estimates con- trolling for individual-level unobservables suggest the program increased the employment rate of trainees by approximately two to three percentage points in the year following the course. To address the endogeneity of course completion, we use exogenous restrictions in course space availability across individuals to implement a difference-in-differences IV 1 strategy estimating the causal effects of training. Some registrants were able to attend and complete the course while others who registered for the course were not allowed to attend due to class oversubscription. The IV estimates suggest those completing the course fol- lowing receipt of a course offer had an eight-percentage-point higher employment rate than those who did not receive a course offer. This effect is statistically distinguishable from the concurrent technical skills training program without demand-driven features, which had a negligible effect on employment. The demand-driven program is more effective in nearly all subgroups based on course and trainees characteristics. We then explore differences in the composition of courses offered in the two programs, finding that business input improves the matching of course supply to future growth in private sector skill demand. The current study builds on the literature in a number of ways. First, and most impor- tantly, we evaluate a skills training program that explicitly elicited demands from businesses for the skills and volume of traineeships required in a given area to serve their needs. Because of this, we refer to our focal program, Pronatec-MDIC, as being demand-driven. Second, the institutional setting allows us to compare the program’s effects across types of trainees within Pronatec-MDIC courses, and to similar courses provided outside the demand-driven segment. That is, we investigate differential employment effects for firm-supplied trainees compared to broader sets of registrants concurrently attending the same business-requested courses, including unemployment beneficiaries and those who self-registered. Third, we com- pare employment effects of the business-requested courses to those from a national training program in which businesses did not provide input into the choice of program course offer- ings. We provide credible evidence in making these comparisons by using the same empirical strategy that addresses endogenous course completion in both programs. We also eschew common concerns over post-program sample attrition through the use of comprehensive ad- ministrative social security data containing monthly employment and earnings information available for all formal sector workers in Brazil. 2 The following section places the current study in the recent literature evaluating similar programs. We then provide further details on the Pronatec and Pronatec-MDIC programs and describe the program records we used to link the course requests, courses held, and stu- dent records to the administrative data. We then discuss our empirical strategy and present results. We discuss why the program likely exhibited greater effectiveness in generating em- ployment among trainees, and provide concluding observations on the program and thoughts for future work. 2 Background: Technical vocational education and train- ing around the world Evidence of the effectiveness of skill training programs remains quite mixed. In a meta- analysis of nearly 100 studies, Card et al. (2010) conclude that skills training programs can effect employment gains in the medium- and long-term, although many programs are ultimately ineffective in reducing unemployment. A host of studies have found widely varying degrees of effectiveness of publicly-provided skill training. Positive employment effects have opo et al., 2007; D´ been found among programs in Peru (N˜ ıaz and Rosas Shady, 2016), Colombia (Attanasio et al., 2011; Kugler et al., 2015), Liberia (Adoho et al., 2014), Nepal (Chakravarty et al., 2015), Malawi (Cho et al., 2013), and Kenya (Honorati, 2015). Other a and Brassiolo, programs have shown negligible impacts, including those in Argentina (Alzu´ 2006), Germany (Caliendo et al., 2011), Dominican Republic Card et al. (2011), Kenya (Hicks et al., 2013), Jordan (Groh et al., 2016), and in an RCT in Turkey (Hirshleifer et al., 2016).1 Specifically in Brazil, Reis (2015) provides a comprehensive overview of the history of training programs, which began as early as 1942. He analyzes the impact of PLANFOR, 1 For earlier reviews, see Betcherman et al. (2004) and Kluve (2010). 3 a worker training program that began in the 1990s, finding short-term effects of the train- ing program of approximately one percentage point increase in employment and seven to eight percent increases in monthly wage rates. Related to the current study, Corseuil et al. (2012) evaluate an apprenticeship-based youth employment program in Brazil, finding that apprentices have a higher probability of getting a formal job in the years after the program. Causal evaluations of vocational training are available for a newer generation of programs that focus on disadvantaged youth, offer training through private providers, and combine coursework with an internship or work experience at a private firm. Despite remarkably similar in program design, the rigorous experimental evaluations of such programs that exist for Colombia and the Dominican Republic find markedly different results. Attanasio et al. (2011) show that this training in Colombia improved women’s, but not men’s, employment and wages. While their original study was based on a surveyed sample, a more recent study that uses administrative records to track the full applicant pool finds positive effects for both men and women in the short and long run (Attanasio et al., 2017). Results for the Dominican Republic (DR) indicate no effect on employment and only very modest effects on earnings Card et al. (2011). A follow-up study indicates positive results in terms of tested skill acquisition and short term employment for women, a negative effect on employment and no skill acquisition among men, and no long term effects except on women’s expectations and self-esteem (Acevedo et al., 2017). A strand of the recent literature has argued for the effectiveness of new approaches to reduce unemployment, including cash or capital injections to small businesses or providing business training to potential entrepreneurs (see Blattman and Ralston (2015) for an in-depth review of this literature). Another dimension of this debate focuses on whether existing programs can be reformulated to more closely involve the private sector in determining which skills to provide (World Bank, 2013). Although there are many means by which skills training can be aligned more closely with the needs of the private sector, few of the proposed 4 formulations have been rigorously evaluated in a context where effectiveness across program types can be compared. Our paper contributes to the above studies in several ways. First, the credible studies cited above argue that the work experience component obliges course providers to offer courses for which there is actual demand. However, this feature is a singular design aspect of these programs. Therefore, the importance of the demand-driven design cannot be compared to traditional programs in the same context. In our context, we can compare the effectiveness of a demand-driven and a non-demand driven program, both of which are provided by quasi-public institutions. Second, we evaluate a national program at scale, serving several hundred thousand trainees in the demand-driven segment and more than one million trainees in the main program segment – which are orders of magnitude larger than existing studies. This aspect allows us to maintain statistical power even when disaggregating by course and student characteristics. We also evaluate a program that is not explicitly focused on youth, which, in addition to being a national program, allows us to make broader claims of external validity. In the following section, we describe the institutional context and design of the program that is the focus of this study. 3 Brazil’s Pronatec and Pronatec-MDIC We study a publicly-provided vocational technical skills training program that was among the first to directly incorporate private sector demand into its design and provision of skills training, and compare its performance with a concurrent, traditionally-designed vocational technical skills training program. In 2011, the federal government of Brazil created the National Program for Access to Technical Education and Employment (Pronatec ). The program was launched at a time of falling unemployment, with the goal of raising the earnings and employability of lower-income segments of the workforce through participation in the formal labor market. Through late-2015 the program had enrolled more than eight million 5 people in “initial and continuing training courses” that carry a moderate in-class training component (less than 500 hours) and train in skills relevant to a particular occupation. The structure of Pronatec is such that several federal ministries administer and contribute to course and trainee selection. The focus of this paper is Pronatec-MDIC – the segment of Pronatec administered by Brazil’s Ministry of Industry, Foreign Trade and Services (hereafter MDIC). The unique feature of Pronatec-MDIC is its “demand-driven” nature: MDIC receives explicit requests for specific skills training courses from individual businesses. This program began with only a limited number of training courses in 2013, greatly expanded in 2014, and drastically reduced its scale in 2015 due to federal budget constraints. In 2014, more than 2,000 firms applied for more than 16,000 skills training courses to be given to current or prospective employees across a wide range of industries and occupations. Ultimately more than 300,000 prospective trainees were registered for Pronatec-MDIC -requested courses in 2014 or 2015, with approximately 40 percent of these registrants completing their training course. Below, we discuss the characteristics of the data sources used to analyze effects of the program. 3.1 Program administrative records Figure 1 summarizes the process from application to course provision in the Pronatec-MDIC program. MDIC receives course requests through a standardized process in which firms indicate the skills, occupation, or course they are looking to have taught, and the number of “seats” (individuals trained) they would like to be provisioned in the given municipality for which they are making the request. These requests have no pecuniary cost for the requesting firms. The requests go through an initial screening by MDIC staff in terms of their viability and appropriateness.2 Approximately one half of firms’ requests were denied at this stage. 2 In correspondence with MDIC staff involved in administering Pronatec-MDIC, we were informed that the review process was made to ensure a reasonable volume of seats requested relative to the firm’s scale and 6 Some firms chose to reapply with a different (lower) number of seats requested, while others did not. The process between a course request and the first day of a course can vary, and is discussed in further detail below. MDIC provided us a comprehensive listing of the course requests in 2014, which contains a measure of the demand for courses/trainees by requesting firms. Each of these requests includes the number of people the requesting organization would like trained (seats), as well as the name of the company or organization that submitted the request, the company’s tax ID, the course requested, the occupation code related to the course, and the municipality in which the course is requested. Table 1 shows that there were 16,782 records for course applications to MDIC in 2014. Of these, approximately half (8,340) were approved by MDIC. The average number of seats requested in a given course was approximately 38; the average number of seats in approved courses was only slightly higher at 43.6. Not all requestors were firms, however: 17% of course requests were made either by industry or workers’ associations. Among the firm requestors, we are able to match 97% to administrative employment records (described below) based on either the tax ID or the combination of firm name and municipality. Requests approved by MDIC were then forwarded to the Ministry of Education, which is the body responsible for the overall administration of Pronatec. The Ministry of Educa- tion aggregates course demands across ministries, and this aggregation is further screened according to course viability (i.e., having the minimum number of seats to hold a course in a municipality requested across ministries), technical criteria, and budget availability. The Ministry of Education is also responsible for selecting training providers and for setting the criteria for the registration of students. There are a number of providers that offer Pronatec courses, although the majority are offered by Brazil’s Sistema S, which in principle ensures projected needs. If found excessive, course requests would be denied (as opposed to adjusted) to discourage firms from seeking training that significantly exceeded their projected needs. 7 a certain homogeneity in the training provided.3 The training providers all receive identical reimbursement from the Ministry of Education at a rate of around 10 Reais (approx. 4 USD in 2014) per student-class hour. The essential difference between the MDIC program and the rest of Pronatec is that the genesis of the requests that MDIC forwards to the Ministry of Education are responsive to direct input from firms. Course requests from other ministries do not come from an explicit application process but rather from varied processes originating within the requesting min- istries without formal consultation with prospective employers. This includes taking input from municipal bodies, social assistance centers, and unemployment benefits (UB) programs. Table 2 presents summary statistics on the firm-requested courses. Appendix Figure 1 shows how firm-requested courses were in more skill-intensive occupations (as measured by the me- dian wage rate in 2012 for the occupation corresponding to the course).4 More than 25% of courses approved by MDIC were not approved by the Ministry of Education. The average course size (conditional on being approved) had 13 seats.5 That is, based on the average number of seats per firm request several classes would be held to fill a single course demand. The average class ran for 200 course hours, met for approximately eight hours per week, and lasted between five and six months in duration. In Table 3 we compare characteristics of students in firm-requested courses to students in the rest of the Pronatec program. The student file provides rich information on students’ backgrounds, including employment status, gender, age, and the completion status of the student in the class, as well as additional class-level fields including meeting dates and times 3 Sistema S is an amalgamation of quasi-governmental organizations in Brazil that administer low-cost or free professional training courses at schools and learning centers throughout the country. 4 We find that firm-requested classes were concentrated in more specific skill-intensive sectors, including those covering industrial processes and production, infrastructure, and ICT. Course topics relatively under- requested through MDIC were more likely to be in service-oriented fields (education/social development, tourism and hospitality, and environment and health sectors). χ2 tests across MDIC and non-MDIC courses strongly reject equal distributions. 5 Note that there are more classes offered than requested, as requests were typically 2-3 times larger than class sizes available. 8 and the institution providing the training. Compared to students in non-MDIC classes, students in MDIC courses are slightly older, more likely to be male, and more likely to have been unemployed at the time of registration, and exhibit higher course completion rates. Because all records contain unique national identification numbers, we are able to link the students in Pronatec-MDIC courses with national administrative data from the Ministry of Labor containing information on all formal sector employment. In the following section we describe the employment data and how we linked them to trainees in firm-requested courses. 3.2 Monthly employment records cao Anual de Informa¸ Our employment data come from the Rela¸ c˜oes Sociais (hereafter RAIS). RAIS is an annual administrative dataset containing employment and earnings informa- tion collected primarily for the purpose of administering social welfare programs such as unemployment and retirement benefits. Our data contain full details on the monthly em- ployment and wage earnings of all formally employed workers in Brazil from calendar years 2013 to 2015. RAIS contains employer-reported records of a worker’s hours, earnings, hir- ing/dismissal dates and reasons (if applicable), as well as the firm’s industrial classification, the worker’s occupational classification, and the worker’s sex, education level, and age. The data importantly contain unique identifiers (i.e., tax IDs) for both workers and employing firms, which are the standard identification provided to and used by firms and workers for their various dimensions of interaction with the government. These fields are used to link the student information to their employment records as well as to identify employment at requesting firms versus other businesses. We deflate earnings using a standard monthly consumer price index (IBGE, 2016).6 6 For workers who have multiple records within a given month, or who worked only part of the month (based on precise hiring and firing dates), we add all deflated earnings across jobs, and construct a monthly wage rate based on the share of the month worked. 9 Table 4 contains summary statistics on the worker-level panel dataset used for analysis. There are 335,300 unique individuals in the administrative data that registered for any of the firm-requested courses, including registrants supplied by requesting firms, unemploy- ment beneficiaries, and others. 60 percent of the sample is male, and 40 percent completed the training course for which they registered. Although only eight percent of the sample was denied a seat for exogenous reasons (which we term “administrative constraints”), 60 percent of all students were in a class that had at least one registrant not complete the course due to these reasons. Nine percent of the students were registered through MDIC by requesting firms, and while the average employment rate was 60 percent, only four per- cent were employed in requesting firms. The average employment rate across person-months was 57 percent, and the mean real monthly earnings (excluding months unemployed) was approximately 1,250 (in 2012 Reais). Since courses are typically not exclusive to a particular type of registrant, the firm- requested courses are comprised of a mix of trainees who registered through different chan- nels. Appendix Tables 1, 2, and 3 contain summary statistics on the three focal subsamples of registrants in the firm-requested courses: those who were pre-registered by requesting firms, unemployment beneficiaries, and those who were registered through neither channel.7 As we expected, and these tables show, these groups of registrants have very different labor market situations: the firm-supplied registrants have higher employment rates (75%, compared to 60% for UB registrants and 53% for others), were far more likely to be employed in request- ing firms (23.7% vs. <2.5% in both other groups), and earned higher wages (R$1,492/month vs. R$1,135/R$1,129). In the analyses below, we estimate effects both for all registrants in firm-requested courses as well for as these three focal subsamples. 7 Unemployment beneficiaries are required to register for a Pronatec training course as a condition to receive unemployment benefits in certain cases (for those seeking unemployment benefits for the second time within 10 years). 10 4 Empirical Strategy Firm-requested courses were offered throughout all months in 2014 and 2015. We exploit differential course timings and the detailed information on the reasons why students did or did not complete the course for which they registered to estimate effects of course completion. We first estimate course effects on employment in a standard difference-in-differences ap- proach. One concern with this approach is the construction of the counterfactual group, as it is well-documented that the timing of the start of skills training is related to time-variant un- observables – in particular, recent unemployment spells (i.e., the “Ashenfelter Dip”). In our context, however, the counterfactual group is constructed of individuals who registered for training courses with specific start dates, but did not attend or complete them. This allows us to estimate a difference in the relative employment across these two groups controlling for common employment patterns relative to the start of the training course. Because course attendance may have different effects on trainees during the course versus after its conclusion, we segment the analysis window into three time periods: pre-course (before the course begins), during the course, and post-course. The difference-in-differences estimator is then: Yict = β0 + β1 ∗ courseict + β2 ∗ postcourseict + β3 ∗ courseict ∗ Completeri + β4 ∗ postcourseict ∗ Completeri + λi + γt + uict (1) In equation 1, i indexes individuals registered for class c whose employment is being observed in month t. β1 and β2 capture aggregate level differences in employment in the course and post-course period (relative to the pre-course period), β3 captures the “during course” effect of course completion on employment, and β4 gives the focal difference-in-differences estimator of the course completion on the outcome. The vector of individual fixed effects in λi absorbs 11 individual-level unobservables (as well as location and classroom effects, and the “main effect” of completion) and γt controls for common (monthly) shocks to the labor market. We then correct for within-class correlations in the error term (i.e., across students taking the same course in the same place for the same period; >15,000 classes/clusters) and for potential aggregate correlations by month t (36 clusters). We limit the sample to 18 months before and after the start of the course, and estimate equation 1 via OLS. As a complement to the above approach, we then estimate program effects in an event study framework to better gauge the evolution of effects over varying horizons relative to training. We choose this to accommodate the amount of time it may take for effects to mate- rialize from vocational/technical training (Card et al., 2015), as it allows for an unrestricted examination of the differences in labor market outcomes in the months before and after a worker begins the training course. From the event study analysis, we are then better able to characterize the likely pattern of effects we expect to see during a slightly longer time horizon. In the event study specification, each month relative to the start of the course is given its own differential effect for completers and is estimated by the following equation: Yict = α0 + αk ∗ [M onths to Startct = k ] k + βk ∗ [M onths to Startct = k ] ∗ Completeri + λi + γt + uict (2) k In equation 2, the vector β measures differences in outcomes for completers relative to non- completers at each period relative to course start, capturing the effect of program completion. Similar to the above, individual fixed effects in λ and month fixed effects in γ control for individual selection on time-invariant unobservables and common aggregate shocks. In the estimation, the course start month (indexed to zero) is the omitted group. 12 5 Results 5.1 Completers vs non-completers with individual fixed effects We first discuss the main difference-in-differences results, with coefficients from the estima- tion of equation 1 on the firm-requested-course registrants presented in Table 5. We find a 0.023 percentage-point (or 0.046 standard deviation) increase in the probability of employ- ment for course completers, on average, in the year following course completion. This effect can then be decomposed, across panels B and C, into effects emanating from employment at requesting firms versus employment outside requesting firms, respectively. We find that a small fraction of this effect comes from employment at requesting firms, while the ma- jority comes from higher employment probability at non-requesting firms. In column 2, we condition the estimation just on the sample of firm-supplied registrants, and find that their overall employment effect (0.031) comes largely through increased employment at requesting firms (0.020).8 We estimate sizable negative effects of course completion on employment among UB registrants (column 3), and small but significant positive effects among all other registrants who find employment at requesting firms (Panel B, column 4). Event study estimates from the estimation of equation 2 are presented in Appendix Tables 4 to 5, with results plotted graphically in Figures 2 to 10, respectively. In the graphs, series for completers and non-completers correspond to equation 2 vectors α and (α + β ), respec- tively. The event study analysis provides some immediately apparent patterns regarding employment and participation in worker training in this context that would otherwise not be apparent from the main analysis. Figure 2 shows how the employment rate of program participants in the month in which they start their course is, for the overall sample, at its lowest point in the preceding months, confirming that a substantial share – more than 40% 8 It is important to note that many of the registrants already worked for requesting firms prior to the training course, so this estimate is necessarily net of pre-course employment at these firms. 13 – of individuals selecting into the program experienced a job loss in the year prior to the start of their course (commonly known as “Ashenfelter’s dip”). We then see non-completers outpace course completers in their employment rate for the months following the start of their course. There may be a number of reasons for this, the most likely being that job offers received prior to or in the early stages of the course period induce dropout. Nevertheless, the event study provides visual evidence (or not) of the parallel trends assumption in the difference-in-differences estimator. Across the three groups, the employment patterns in the pre-course period are markedly different. Among MDIC-supplied registrants, we find no stark pre-course dip in employment rates (Figure 4). This is likely attributable to the selection of individuals who are registered through the MDIC process who have higher pre-course employment rates than the average trainee and have not been registered for a training course due to recent unemployment. Figure 4 highlights two additionally important points. First, completers are significantly different from non-completers in pre-course employment levels. Second, while employment rates for completers are approximately maintained in the post-course period, those for non- completers exhibit a flat trend after the course, while completers continue trending upward. The gap in employment rates leads to a nearly ten-percentage-point difference for the year following the end of the course. Among unemployment beneficiaries who also attended firm- requested courses, we find a sharp pre-course reduction in employment rates (Figure 5), reflecting precisely the institutional situation described above. The approximately equal differentials across completers and non-completers before and after the course validate the DD estimates showing that the program had minimal employment effect on these registrants. Table 4 contains a subset of focal coefficients in the β vector capturing the difference in the outcome for the completers relative to non-completers in months before the training begins, in the month of completion, and three, six, nine, and 12 months after the end of the course. Employment effects are statistically distinguishable by the end of the course period (for the overall sample, and MDIC registrants) or within a few months after the completion of 14 the course (for the “all other” registrants), although these point estimates do not account for pre-course differences. This analysis additionally allows us to gauge the likely trend in program effects over our study horizon. Effects appear to be steadily increasing over time (for all but UB registrants), suggesting that estimates in the current analysis may be smaller than those that use a longer horizon. 5.2 Identification challenges The identifying assumption in the analysis above is that completers and non-completers experienced common shocks in the months prior to the start of the course and would have exhibited similar employment patterns in the absence of treatment. The approach assumes that the endogeneity between course completion and later employment is entirely due to time-invariant individual-level unobservables. This is a strong assumption, precisely because employment gained during the course period might affect whether an individual completes the course. That is, some individuals may receive acceptable employment offers during the course period that cause them to drop the course. Since these offers are not observable, increase employment, and are negatively correlated with course completion, we can then reason that this approach above likely gives a downward-biased estimate of the effect of training on employment. In the next section, we address this concern by using exogenously determined variation in space availability in courses to identify the causal effect of training on employment. 5.3 IV estimates using course offer as instrument Since we remain concerned about the endogeneity of course completion, we make use of the detailed information on the reasons why students did not complete the courses they registered for to identify students who were kept from attending a course due to exogenous 15 reasons. In general, the reasons for which an individual would register for a course but not be able to complete it could be either personal (e.g., quits and no-shows) or what we refer to as “administrative reasons” – those outside the control of the individual registrant. These typically include a lack of seats due to class oversubscription or space limitations.9 In the case of course oversubscription, which is the cause of the vast majority of course offer restrictions, the segment of unemployment beneficiary registrants was used to make space for various other groups registered through other channels. This was done on a first-come, first-served basis, although the administrative program data did not retain sufficient detail to observe the precise cutoff for course offer receipt. We use this information on administrative restrictions to create an instrument for whether a student received a “course offer” – taking the value of one for those who had registered for a class and were not restricted administratively from attending the class. Because the analysis is split into three periods relative to course start and end, we then interact the instrument separately with indicators for the during-course and post-course periods. The structural equation remains as in equation 1, where the two endogenous variables, courseict ∗ Completeri and postcourseict ∗ Completeri are instrumented with courseict ∗ Of f eri and postcourseict ∗ Of f eri . (Note that the main effects of Of f eri are absorbed in the individual fixed effects.) 5.4 Test of parallel trends One of the central assumptions underlying this approach is whether the two groups satisfy the “parallel trends” assumption – that is, that trends across offer recipients and non-recipients were parallel in the pre-course period and would have continued to be so in the absence of the course. We test the first part of this assumption (whether there were parallel trends prior to the course) by limiting the sample to all pre-course observations and estimating a 9 We confirmed with the Ministry of Education that the record codes used to identify administrative non-completers always corresponded to reasons for non-completion that were outside the control of trainees themselves; e.g., force majeure cancellations or seat reductions, class oversubscription, etc. 16 slope coefficient on the number of months relative to the course and a slope differential for offer recipients, via: Yict = α0 + α1 ∗ months relative to courseict + α2 ∗ months relative to courseict ∗ of f eri + λi + γt + eict (3) In equation 3, α2 gives the slope differential for offer recipients’ pre-course employment trends. In this case, a positive coefficient would raise concerns about upward bias in the resulting estimates (and vice versa for a negative coefficient). The coefficients from this esti- mation across the pooled and separated samples are in Table 6. In column 1, the estimated slope differential is small and statistically insignificant at conventional levels. Because of the precision afforded by the sample, we can reject slope coefficients as small as 0.001 – a magnitude still not large enough to present meaningful concern for our estimates below. Even among subsamples there is no statistically significant differential trend among offer recipients, lending support for the parallel trends assumption as applied to the pre-course period. 5.5 IV estimates: First- and second stages First stage coefficients from the estimation of the focal endogenous variable, postcourseict ∗ Completeri , are in Table 7. The coefficient effectively reflects the completion rate among those offered a seat in the course (i.e., compliance), which, as expected, is slightly higher than the overall course completion rate. Second stage coefficient estimates are in Table 8. We find average employment effect (Panel A, column 1) at 8.6 percentage points to be substantially larger than the naive difference-in-difference estimates. This implies the reduced form intent- 17 to-treat effect of course offer receipt is just under four percentage points. We find that the UB registrants (column 3), to whom the instrument was particularly relevant and from whom the IV estimates are largely identified, exhibit large (14.6 percentage-point) employment effects of course completion induced from receiving a course offer. Firm-supplied registrants (column 2) do not appear to exhibit employment effects from this variation, and all other registrants (column 4) exhibit small but significant increases in employment. The majority of the large effects for UB registants (Panel A) are attributable to employment at non- requesting firms (Panel C). The strong effects among the unemployed finding employment in non-requesting firms may be partly explained by the fact that “lead” firms would submit requests on behalf of smaller, local suppliers – providing further evidence that the courses requested were indicative of general skill shortages experienced by several firms. We find these effects to be relatively large, especially in view of the mixed results found in previous literature. However, the identifying variation comes from students in courses that had sufficient demand to be oversubscribed; if student demand is at all an indicator of the quality of courses or the employment prospects they generate, we can reason that these estimates will be larger than those for the entire course distribution. Among the types of students taking courses in our context, the UB registrants have the lowest pre-course employment rates and are thus more likely to exhibit effects on the extensive margin of employment compared to, for example, firm-supplied registrants of whom many are employed at the start of the course. 5.6 Robustness: Common support sample We address further concerns of unobservable differences between offer recipients and those not receiving a course offer by constructing a within-class, common support matched sample. In this process, we take the set of registrants who do not receive a course offer and match them 18 with their nearest neighbor (without replacement) based on offer propensity predicted by a discrete choice model using individual characteristics and detailed employment histories to estimate the offer indicator. This effectively balances the sample to have 50% offer recipients and leaves a final sample comprised of individuals from the set of classes that experienced any capacity constraints. The results from this matched analysis (available up on request) mirror closely the magnitudes for the UB sample in Table 8 of approximately a 15 percentage-point effect on employment. 5.7 Estimation of effects from the broader Pronatec program A major benefit of the Brazilian context is our ability to perform the exact same analysis on an otherwise-similar national skills training program that did not take input from firms in allocating skills training. This program, which was implemented by the same ministries and course providers, existed at the same time as our focal program and essentially ran in parallel to the demand-driven program. Over the study period, the main Pronatec program was approximately six times larger than the demand-driven segment, serving over 1.6 million trainees for whom we can similarly observe complete employment and wage histories in the administrative data. We undertake the same analysis as above on all Pronatec trainees outside the firm-requested courses who registered for a class scheduled to be held in 2014 or 2015 (corresponding to the same period during which the business-requested courses were held). We estimate the IV specification for individuals who registered for a training course out- side of the demand-driven segment. The estimates for the broader Pronatec program are presented in Table 9, and show a markedly different pattern from those of the demand-driven segment: IV estimates of the program effect are negligible overall, and precisely estimated. This masks substantial heterogeneity across UB recipients and the majority of other reg- 19 istrants: UB registrants saw average increases of eight percent in employment probability following the course (compared to approximately 15 percent in the MDIC program, Table 8, Panel A, column 3), while other registrants saw a nearly ten percent lower employment rate – combining into an overall minimal and statistically insignificant effect on employment. 5.8 Effect heterogeneity by skill group, region, and trainees’ for- mal education To better understand effect heterogeneity within and across the programs, we estimate the IV specification across subgroups in four dimensions. In Table 10 we present the IV coeffi- cients for both program segments estimated separately by each of seven major occupational subgroups, five skill levels (based on the 2013 median wage in the occupation trained), five regions of the country, and trainee education level. We find that even within occupational groups, the demand-driven segment exhibited either similar or meaningfully larger effects in courses in the four occupational groups (Industrial Processes and Production; Management and Business; Information and Communication; and Environment, Health and Education) that comprise approximately three-quarters of individuals trained in either the demand-driven program or Pronatec overall. We then seg- ment courses into quintiles based on the median wage of the occupation for which they were training. The demand-driven program had substantially larger effects than the main program in the lowest-skill courses, as well as middle- and upper-skill trainings (the latter of which exhibited negative effects in the main program). Again, the overall effect is not driven by a compositional effect but by differential effectiveness within skill levels. Finally, the demand-driven program was more effective in four regions of the country (all except the North, where performance was similar). The demand-driven program has a slightly larger share of trainings in the Northeast, where it was almost three times as effective as the main 20 program, compared to a larger proportion of trainings in the Southeast by the main program, where employment effects were negative. By trainee skill level, we find that the demand- driven program was more effective at all levels – and was particularly more effective among primary- and middle-school educated trainees. 5.9 Can effectiveness be explained by selection on students? A final concern is that the differential effectiveness of the two programs may be explained by selection of UB trainees who join the firm-requested courses versus the overall program. We believe this is unlikely for institutional reasons (firm-requested courses were never outwardly advertised or known to applicants or providers to have their provenance from this channel). We show below empirically that this was not the case. To test for differential selection of UB trainees into the firm-requested courses versus the main program, we take the pooled sample of UB registrants in both programs and estimate an indicator for registering for a firm-requested course in a multivariate regression containing the registrants’ pre-course employment history at different periods relative to the course start. Despite the empirical strategy fully accounting for individual-level unobservables, this test allows us to speak to whether the UB registrants in either program were systematically different from each other in their pre-course employability. We estimate the indicator for being in a demand-driven course with individual employment between one and six months prior to the course start. Results from this estimation are found in Appendix Table 6, and show that, if anything, UB registrants in the demand-driven segment had lower employment rates prior to the course than those in the main program segment. 21 5.10 Alternative cross-sectional IV Finally, we present below an alternative IV strategy used to validate the main results. Since the assignment mechanism is ultimately cross-sectional, we estimate employment rates at each month after the start of the individual’s course, starting from the month of course completion and up to 18 months after the start of the course. In this approach, we are also able to explicitly control for differential pre-course employment and education levels as well as saturate the specification with pre-course employment indicators. As before, the first stage estimation reveals the rate at which those with a course offer complete the course, conditional on the included fixed effects and covariates. The cross- sectional first-stage equation is given by: Completedic = δ0 + δ1 ∗ of f eri + λc + uic (4) where i indexes individuals across courses c. Table 11 presents estimates from the first stage equation. Column 1 contains the first stage coefficient including only a vector of month fixed effects. Column 2 presents estimates of the first stage relationship with a full set of class fixed effects, and Column 3 adds the education level of the individual registrant. We find that the first stage coefficient is stable around .40 across these columns, indicating the course completion rate of approximately 40 percent as shown in Table 4. The first-stage coefficient remains unchanged with the inclusion of a larger vector of class fixed effects, the registrant’s education level, as well as pre-period employment indicators. The second stage equation is then given by: ˆ Employedic = β0 + β1 ∗ Completedi + γc + eic (5) Table 12 presents estimates of the second-stage coefficient on course completion estimated 22 across varying specifications and months relative to the start of the registrant’s course. As above, we estimate separate specifications including month fixed effects, class fixed effects, and including education and pre-period employment rates as additional controls. The final specification, with a full set of controls for pre-course employment at one, three six, and nine months prior to the start of the course, has slightly larger estimates than the main IV estimates above – although they are sharply decreasing over the study period. Nonetheless, the employment effects are still sizable and, on average, of a comparable magnitude to the main estimates above, and we confirm that these magnitudes are largely driven by the UB registrant sample (Table 13). 6 Explaining program effectiveness: Matching skill train- ing to growth in skill demand Why did the demand-driven program exhibit such substantially larger employment effects than the larger, non-demand-driven segment? Our primary hypothesis is that the input from firms contained meaningful information that allowed the provision of skill trainings to be better aligned with aggregate growth in skill demand. To assess whether this was the case, we compare the occupational and geographic distribution of trainings in the demand- driven and main program to two “reference” distributions: the cross-sectional distribution of aggregate employment, and growth in employment over the 2013 - 2015 period. We compute the sum of squared deviations (SSD) of the focal training program to the two reference distributions to understand which training program was better matched to the cross-section of employment and which was better matched to aggregate employment growth. In Table 14 we present the values of the SSD for the main and demand-driven programs compared to the focal reference distributions for three categorical variables: three-digit occu- 23 pation, state, and state-by-occupation. (Note that higher values indicate greater mismatch.) The conclusions from this analysis are clear. Comparing Columns 1 and 2, the main program was better aligned with the cross-section of employment both geographically and occupa- tionally. Comparing Columns 3 and 4, the demand-driven program was better matched to the growth in skill demand by occupation. (Note also that the programs are, in general, better matched to the cross-section of employment than its growth, given by the lower mag- nitudes in Columns 1-2 compared to Columns 3-4). We note that this difference between the two programs is particularly pronounced when we focus our reference distribution just on the period of end-2014 to end-2015 (not presented), which comprises a period that oc- curred following the timing of requests made by firms (rather than spanning the period of requests) – suggesting the requests indicated expected future demand rather than recent trends. This provides further evidence that business input into training programs can be useful in identifying economy-wide skill shortages and future growth in skill demand. Whether better matching of courses to future employment growth explains differential effectiveness across programs hinges on the existence of differential effectiveness of courses offered in faster-growing occupations. To test whether this is indeed the case, we estimate heterogeneous effects of the program by interacting the instrument and focal endogenous variable with the 2013 to 2015 share of aggregate employment growth corresponding to the occupation of the course attended. We transform this measure to have mean zero and unit standard deviation, and estimates for both program segments are found in Table 15. We find stronger employment effects are among faster-growing occupations in both segments. That is, courses offered in fast-growing occupations exhibited larger employment effects, which ex- plains the effectiveness of the demand-driven model. We also find this is particularly relevant for employment among the vast majority of self-supplied registrants, and that differential effect is larger among the non-demand-driven segment – possibly due to diminishing margins in the degree of program alignment to growth in skill demand. 24 7 Conclusion The fast-evolving skill demand of employers in both developed and developing countries has the potential to leave those less skilled, less connected, and already economically marginalized even further behind. Occupational training programs are a common active labor market policy with the potential to address skill shortages and mismatch, although their effectiveness across contexts varies greatly. In this paper, we evaluate the effects of a recent large-scale technical skill training program in Brazil that took informational input from firms into consideration for the provision of publicly administered vocational technical training. Our empirical strategy and use of comprehensive monthly labor market data allow us to avoid several of the common challenges in the evaluation of occupational training programs. We find that the program has a substantial causal effect on employment appearing shortly after course completion and persisting for at least one year following the course. Program effects are substantially larger than those in a similarly-run occupational training program that does not take input from firms in deciding the content and scale of skills training. The informational input from firms allows the allocation of skill trainings to better match future growth in skill demand rather than the static distribution of employment, which we show likely explains increased program effectiveness. There may be an important efficiency-equity tradeoff in demand-driven programs. This is particularly germane in contexts where economic inequality is sought to be attenuated through active labor market policies. Such designs have the potential to exclude segments of firms and workers from accessing program benefits. In particular, we find that the dis- tribution of courses provided under the demand-driven program is concentrated in more skill-intensive occupations that are more likely to find employment in industries offering relatively higher-wage formal employment. Participating businesses may also comprise a se- lected segment of the distribution of firms. For workers, if initial skill levels play a meaningful role in access or ability to complete these courses, this dimension may serve to exacerbate 25 inequality between program trainees and those not able to access the program’s benefits. Similar effects may occur if firms’ access to the demand-driven program is limited by their size or the scale of their requests. Nevertheless, our results confirm the potential for partnership between the public and pri- vate sectors in investing in skills. We find strong evidence of employment effects among regis- trants not previously connected to requesting firms who gain employment at non-requesting firms. This suggests two things: the firms requesting skills training are indicating local skills shortages applicable to more than just their own firm, and the trainings requested under the program are likely providing general rather than firm-specific skills. When this is the case, trainees (as opposed to firms) are the residual claimants to the skills provided – suggesting the training program is effective in garnering information about broader local skills shortages and that the ultimate beneficiaries of the training are trainees, rather than partnering firms. This allows for a cautious optimism regarding the potential to expand demand-driven train- ing programs in other contexts in ways that can have pro-poor impacts. Future work in this and other contexts should investigate how demand-driven training programs affect request- ing businesses’ labor demand and output expansion, as well as affect workers’ productivity and occupational mobility within and across firms. 26 References Acevedo, P., G. Cruces, P. Gertler, and S. Martinez (2017): “Living Up to Expectations: How Job Training Made Women Better Off and Men Worse Off,” NBER Working Paper No. 23264. Adoho, F., S. Chakravarty, K. Jr, D. T., M. Lundberg, and A. Tasneem (2014): “The Impact of an Adolescent Girls Employment Program: The EPAG Project in Liberia,” World Bank Policy Research Working Paper, no. 6832. ´ , M. L. and P. Brassiolo (2006): “The impact of training policies in Argentina: Alzua an evaluation of Proyecto Joven,” Inter-American Development Bank Working Paper. ın, C. Medina, and C. Meghir (2017): “Vocational Training Attanasio, O., A. Guar´ for Disadvantaged Youth in Colombia: A Long-Term Follow-Up,” American Economic Journal: Applied Economics, 9, 131–143. Attanasio, O., A. Kugler, and C. Meghir (2011): “Subsidizing Vocational Training for Disadvantaged Youth in Developing Countries: Evidence from a Randomized Trial,” American Economic Journal: Applied Economics, 3, 188–220. Betcherman, G., K. Olivas, and A. Dar (2004): “Impacts of Active Labor Market Programs: New Evidence from Evaluations with Particular Attention to Developing and Transition Countries,” Social Protection Discussion Paper, World Bank, Washington, DC. Blattman, C. and L. Ralston (2015): “Generating employment in poor and fragile states: Evidence from labor market and entrepreneurship programs,” Columbia University, Mimeo. ¨ nn, and R. Schmidl (2011): “Fighting Youth Unemployment: Caliendo, M., S. Ku The Effects of Active Labor Market Policies. IZA DP No. 6222,” Institute for the Study of Labor. 27 ´ n, F. Regalia, D. Rosas-Shady, and Y. Soares (2011): Card, D., P. Ibarrara “The Labor Market Impacts of Youth Training in the Dominican Republic,” Journal of Labor Economics, 29, 267–300. Card, D., K. J., and A. Weber (2015): “What Works? A Meta Analysis of Recent Active Labor Market Program Evaluations,” NBER Working Paper No. 21431. Card, D., J. Kluve, and A. Weber (2010): “Active labour market policy evaluations: A metaanalysis,” The Economic Journal, 120, 452–477. Chakravarty, S., M. Lundberg, P. N. Danchev, and J. Zenker (2015): “The role of training programs for youth employment in Nepal: impact evaluation report on the employment fund.” The World Bank, Washington, DC. Cho, Y., D. Kalomba, A. M. Mobarak, and V. Orozco (2013): “Gender Differences in the Effects of Vocational Training: Constraints on Women and Drop-Out Behavior,” IZA DP No. 7408. Corseuil, C. H., M. Foguel, G. Gonzaga, and E. P. Ribeiro (2012): “The Effect of an Apprenticeship Program on Labor Market Outcomes of Youth in Brazil,” Presented in the 7th IZA/World Bank Conference: Employment and Development, New Delhi. ıaz, J. J. and D. Rosas Shady (2016): “Impact Evaluation of the Job Youth Training D´ Program Projoven,” Inter-American Development Bank Working Paper. Groh, M., N. Krishnan, D. McKenzie, and T. Vishwanath (2016): “The Impact of Soft Skills Training on Female Youth Employment: Evidence from a Randomized Exper- iment in Jordan,” IZA Journal of Labor and Development, 5, 9. Heinrich, C. J., P. R. Mueser, K. R. Troske, K.-S. Jeon, and D. C. Kahvecioglu (2013): “Do Public Employment and Training Programs Work?” IZA Journal of Labor Economics, 2, 1–23. 28 Hicks, J. H., M. Kremer, I. Mbiti, and E. Miguel (2013): “Vocational education in Kenya: Evidence from a randomized evaluation among youth,” Mimeo. Hirshleifer, S., D. McKenzie, R. Almeida, and C. Ridao-Cano (2016): “The Impact of Vocational Training for the Unemployed: Experimental Evidence from Turkey,” Economic Journal, 126, 2115–2146. Honorati, M. (2015): “The Impact of Private Sector Internship and Training on Urban Youth in Kenya,” World Bank Policy Research Working Paper No. 7404. IBGE (2016): “´ cos ao Consumidor Amplo (IPCA),” Instituto Indice Nacional de Pre¸ ıstica, Sistema Nacional de ` Brasileiro de Geografia e Estat´ cos ao Consumidor Indices de Pre¸ (IBGE/SNIPC). Kluve, J. (2010): “The Effectiveness of European Active Labor Market Programs,” Labour Economics, 17, 904–918. Kugler, A., M. Kugler, J. Saavedra, and L. O. H. Prada (2015): “Long-term direct and spillover effects of job training: Experimental evidence from Colombia (No. w21607),” National Bureau of Economic Research Working Paper Series. ˜ po, H., M. Robles, and J. Saavedra (2007): “Occupational Training to Reduce Gen- No der Segregation: The Impacts of ProJoven,” Inter-American Development Bank Working Paper. Reis, M. (2015): “Vocational Training and Labor Market Outcomes in Brazil,” B.E. Journal of Economic Analysis and Policy, 15, 377–405. World Bank (2013): “World Development Report 2013: Jobs,” World Bank Group. Wash- ington, DC. 29 Table 1: Summary Statistics, MDIC course demand applications Variable Mean Std. Dev. Obs Whether course was approved [0/1] 0.50 0.50 16,782 Number of seats requested 37.8 178.1 16,782 Number of seats requested — approved 43.6 82.3 8,340 Requestor is worker/industry association [0/1] 0.17 0.37 16,782 Whether requestor found in RAIS — firm 0.97 0.17 13,969 Source: Authors’ calculations using MDIC course demand data. Table 2: Summary Statistics, MDIC-requested courses Variable Mean Std. Dev. Obs Whether course was approved [0/1] 0.74 0.44 35,834 Number of seats | approved 16.0 36.5 22,198 Course hours | approved 198.4 44.6 26,666 Source: Authors’ calculations using MEC course data. Note: Sample comprised of records matching course-municipality of requests to MDIC. Less than 1% of approved requests were among courses with greater than 150 seats. 30 Table 3: Comparison of MDIC-course students vs. non-MDIC-course students Variable MDIC non-MDIC difference s.e. t p-value N Completed course [0/1] 0.399 0.322 0.077 0.001 111.1 <0.001 4,235,525 Age of student (years) 29.272 27.629 1.644 0.012 108.7 <0.001 4,219,858 Male [0/1] 0.544 0.384 0.160 0.001 223.2 <0.001 4,235,525 Unemployed at registration [0/1] 0.273 0.182 0.090 0.000 156.2 <0.001 4,235,525 Source: Authors’ calculations using MEC course data. Note: MDIC course students comprised of records matching course and municipality of requests to MDIC. 31 Table 4: Summary statistics, all registrants in MDIC-requested courses Variable Mean Std. Dev. Min. Max. N Worker number of observations 30.55 4.183 18 35 335300 Male[0/1] 0.6 0.49 0 1 335300 Course Offer 0.931 0.253 0 1 335300 Unemployment benefits registrant 0.185 0.388 0 1 335300 MDIC registrant 0.087 0.282 0 1 335300 Completed course[0/1] 0.397 0.489 0 1 335300 Course had admin. noncompleters [0/1] 0.597 0.49 0 1 335300 Employed [0/1] 0.565 0.496 0 1 10243541 Employed in an MDIC-requesting firm 0.041 0.199 0 1 10243541 Gross deflated monthly earnings(R$) 1186.507 771.306 1 6139.095 5458730 ln( gross deflated monthly earnings(R$)) 6.897 0.643 0 8.722 5458730 Gross deflated monthly wage rates 1251.957 787.062 1.009 6139.095 5458902 ln(gross deflated monthly wage rates) 6.982 0.547 0.009 8.722 5458902 Notes: Authors’ calculations using RAIS. 32 Table 5: Program employment effects, difference-in-differences estimates Outcome: Employed [0/1] Sample All registrants MDIC registrants Unemp. Benefic. All others (1) (2) (3) (4) Panel A: Employment at any firm Post-course * completed 0.023*** 0.031*** -0.042*** 0.005 (0.003) (0.006) (0.006) (0.003) Mean of outcome 0.565 0.748 0.604 0.533 St. dev. of outcome 0.496 0.434 0.409 0.499 R2 0.40 0.45 0.39 0.43 Panel B: Employment at requesting firms Post-course * completed 0.004*** 0.020*** -0.003** 0.004*** (0.000) (0.004) (0.001) (0.000) Mean of outcome 0.041 0.237 0.009 0.025 St. dev. of outcome 0.198 0.425 0.094 0.156 R2 0.76 0.82 0.42 0.69 Panel C: Employment at nonrequesting firms Post-course * completed 0.018*** 0.010 -0.038*** 0.001 (0.003) (0.007) (0.006) (0.003) Mean of outcome 0.524 0.511 0.595 0.508 St. dev. of outcome 0.499 0.5 0.491 0.5 2 R 0.42 0.60 0.39 0.44 N (all panels) 10,243,541 912,053 1,872,837 7,458,651 Notes: Table presents present difference-in-differences IV estimates of course comple- tion. Heteroskedasticity-consistent robust standard errors two-way clustered by class and month*year reported in parentheses. All specifications include an unreported constant term and vectors of individual and month*year fixed effects. Significance indicated by: ∗ p < .1, ∗∗ p < .05, ∗ ∗ ∗ p < .01. 33 Table 6: Testing for differential pre-course trends Outcome: Employed [0/1] Sample All registrants MDIC registrants Unemp. Benefic. All others (1) (2) (3) (4) Months relative to course 0.000928 -0.001781 -0.001771 -0.000028 * course offer (0.001117) (0.001428) (0.001851) (0.000532) R2 0.61 0.60 0.32 0.62 N 4,794,972 485,327 816,802 3,492,843 Notes: Table presents estimates from the estimation of equation 3 in the text, adjusted to test parallel trends in pre-course employment differentially across those receiving the course offer and those not. Sample comprised of all periods prior to the start of registrants’ training course. The coefficient for [Months relative to course*offer] gives the differential slope term for those offered a course seat in the pre-course period. Heteroskedasticity-consistent robust standard errors two-way clustered by individual and month*year reported in parentheses. All specifications include an unreported constant term, a primary slope coefficient for months relative to the course, and vectors of individual and month*year fixed effects. Significance indicated by: ∗ p < .1, ∗∗ p < .05, ∗ ∗ ∗ p < .01. Table 7: First stage estimates, difference-in-differences IV Outcome: Post-course period [0/1] * Completed course Sample All registrants MDIC registrants Unemp. Benefic. All others (1) (2) (3) (4) Post-course period [0/1] 0.435062*** 0.519792*** 0.367063*** 0.442523*** * course offer (0.003203) (0.006737) (0.007700) (0.002292) R2 0.57 0.61 0.54 0.57 N 10,243,541 912,053 1,872,837 7,458,651 Notes: Table presents first-stage coefficients from the estimation of the endogenous vari- able [Post-course * Completed] in equation 1 as predicted by an indicator for being in the post-course period and having received a course offer. Heteroskedasticity-consistent robust standard errors two-way clustered by class and month*year reported in parentheses. All specifications include an unreported constant term and vectors of individual and month*year fixed effects. Significance indicated by: ∗ p < .1, ∗∗ p < .05, ∗ ∗ ∗ p < .01. 34 Table 8: Program employment effects, difference-in-differences IV estimates Outcome: Employed [0/1] Sample All registrants MDIC registrants Unemp. Benefic. All others (1) (2) (3) (4) Panel A: Employment at any firm Post-course * completed 0.086*** -0.029 0.146*** 0.023* (0.014) (0.032) (0.032) (0.013) Mean of outcome 0.565 0.748 0.604 0.533 St. dev. of outcome 0.496 0.434 0.409 0.499 R2 0.41 0.45 0.37 0.43 Panel B: Employment at requesting firms Post-course * completed 0.009*** 0.032 0.005 0.009*** (0.002) (0.022) (0.003) (0.002) Mean of outcome 0.041 0.237 0.009 0.025 St. dev. of outcome 0.198 0.425 0.094 0.156 R2 0.76 0.82 0.42 0.69 Panel C: Employment at nonrequesting firms Post-course * completed 0.077*** -0.062* 0.140*** 0.014 (0.014) (0.036) (0.032) (0.013) Mean of outcome 0.524 0.511 0.595 0.508 St. dev. of outcome 0.499 0.5 0.491 0.5 R2 0.42 0.60 0.37 0.44 N 10,243,541 912,053 1,872,837 7,458,651 Notes: Table presents second-stage coefficients capturing effects of course completion induced by receiving a course offer. Heteroskedasticity-consistent robust standard errors two-way clustered by class and month*year reported in parentheses. All specifications include an unreported constant term and vectors of individual and month*year fixed effects. Significance indicated by: ∗ p < .1, ∗∗ p < .05, ∗ ∗ ∗ p < .01. 35 Table 9: Pronatec program employment effects, difference-in-differences IV estimates Outcome: Employed [0/1] Sample All registrants Unemp. Benefic. All others (1) (2) (3) Post-course * completed 0.005 0.088*** -0.112*** (0.016) (0.021) (0.013) Mean of outcome 0.561 0.602 0.558 St. dev. of outcome 0.496 0.489 0.497 R2 0.45 0.40 0.46 N 51,082,933 3,505,993 47,576,940 Notes: Table presents second-stage coefficients capturing effects of course completion induced by receiving a course offer, estimated on the sample of all Pronatec course registrants exclud- ing MDIC-requested courses. Heteroskedasticity-consistent robust standard errors two-way clustered by class and month*year reported in parentheses. All specifications include an un- reported constant term and vectors of individual and month*year fixed effects. Significance indicated by: ∗ p < .1, ∗∗ p < .05, ∗ ∗ ∗ p < .01. 36 Table 10: Comparing demand-driven and overall program effects by subgroup Estimated coefficients Demand-driven program Pronatec (all) (1) (2) All sectors 0.086*** 0.005 Panel A: by occupational group Industrial Processes and Production 0.106** 0.001 Management and Business 0.093*** 0.079* Infrastructure 0.085*** 0.064*** Information and Communication 0.153*** 0.074 Environment, Health and Education 0.088** 0.117*** Food Production 0.185* 0.171*** Natural Resources, Culture and Tourism 0.118** 0.056* Panel B: by occupational skill/wage level 1 (Low) 0.080** 0.046*** 2 0.090*** 0.057*** 3 0.086*** -0.043 4 0.127*** -0.328*** 5 (High) -0.198* -0.771*** Panel C: by region North 0.105*** 0.017 Northeast 0.078*** 0.016 Southeast 0.090*** 0.014 South 0.043* -0.024 Midwest 0.045** 0.027 Panel D: by worker education level Primary Incomplete 0.056 -0.043 Primary 0.236*** 0.107*** Middle 0.094*** 0.026 Secondary 0.074*** 0.052*** Post-secondary 0.036 0.033 Notes: Columns 1 and 2 present difference-in-differences IV estimates of course comple- tion, estimated separately by course or trainee characteristics in firm-requested and broader Pronatec courses. Significance for coefficients presented in Columns 1 and 2 indicated by: ∗ p < .1, ∗∗ p < .05, ∗ ∗ ∗ p < .01. 37 Table 11: Alternative crosssectional IV: First-stage estimates of completion based on course offer Outcome: Completed course [0/1] Class FE Class FE Specification: Month FE Class FE + educ + educ + Et−1,3,6,9 (1) (2) (2) (3) Course offer 0.427 0.398 0.397 0.398 (0.000)*** (0.002)*** (0.002)*** (0.002)*** Worker education (years) 0.009 0.009 (0.001)*** (0.001)*** Employment 6 mos. 0.015 before course start (0.001)*** Month fixed effects Y N N N Class fixed effects N Y Y Y N 335,300 334,954 334,954 334,249 R2 0.049 0.229 0.231 0.231 Notes: Table presents coefficients from the crosssectional first-stage regression estimating completion with course offer status across specifications indicated in the column headings. Heteroskedasticity-consistent robust standard errors clustered by course reported in paren- theses. All specifications include an unreported constant term. Significance indicated by: ∗ p < .1, ∗∗ p < .05, ∗ ∗ ∗ p < .01. 38 Table 12: Alternative crosssectional IV: Second-stage coefficient estimates by time period relative to course start – All MDIC registrants Outcome: Employed [0/1] Class FE Class FE Specification: Month FE Class FE + educ + educ + Et−1,3,6,9 (1) (2) (3) (4) t+6 (course ends) 0.081*** 0.070*** 0.067*** 0.110*** t+7 0.071*** 0.060*** 0.058*** 0.098*** t+8 0.073*** 0.069*** 0.066*** 0.104*** t+9 0.068*** 0.059*** 0.057*** 0.092*** t+10 0.069*** 0.059*** 0.056*** 0.090*** t+11 0.058*** 0.051*** 0.048*** 0.078*** t+12 0.058*** 0.051*** 0.047*** 0.077*** t+13 0.053*** 0.047*** 0.043*** 0.072*** t+14 0.048*** 0.041*** 0.038*** 0.065*** t+15 0.052*** 0.050*** 0.046*** 0.070*** t+16 0.055*** 0.057*** 0.053*** 0.075*** t+17 0.050*** 0.046*** 0.043*** 0.064*** Notes: Table presents second-stage coefficients from the estimation of equation 5 in the text. Each row comes from a separate cross-sectional estimation of employment at the indicated month relative to course start. Specification variants are indicated in the column headings. Heteroskedasticity-consistent robust standard errors two-way clustered by course and month*year reported in parentheses. All specifications include an unreported constant term. Significance indicated by: ∗ p < .1, ∗∗ p < .05, ∗ ∗ ∗ p < .01. 39 Table 13: Alternative crosssectional IV: Second-stage coefficient estimates by time period relative to course start – UB registrants only Outcome: Employed [0/1] Class FE Class FE Specification: Month FE Class FE + educ + educ + Et−1,3,6,9 (1) (2) (3) (4) t+6 (course ends) 0.087*** 0.093*** 0.092*** 0.094*** t+7 0.085*** 0.094*** 0.092*** 0.093*** t+8 0.097*** 0.113*** 0.111*** 0.112*** t+9 0.097*** 0.108*** 0.107*** 0.105*** t+10 0.110*** 0.114*** 0.113*** 0.113*** t+11 0.089*** 0.094*** 0.092*** 0.089*** t+12 0.085*** 0.084*** 0.082*** 0.081*** t+13 0.075*** 0.077*** 0.074*** 0.072*** t+14 0.069*** 0.061*** 0.059** 0.058** t+15 0.068*** 0.070*** 0.068*** 0.067*** t+16 0.071*** 0.075*** 0.072*** 0.071*** t+17 0.061*** 0.061** 0.058** 0.057** Notes: Table presents second-stage coefficients from the estimation of equation 5 in the text for the sample of unemployment beneficiary registrants. Each row comes from a separate cross-sectional estimation of employment at the indicated month relative to course start. Specification variants are indicated in the column headings. Heteroskedasticity-consistent robust standard errors two-way clustered by course and month*year reported in parentheses. All specifications include an unreported constant term. Significance indicated by: ∗ p < .1, ∗∗ p < .05, ∗ ∗ ∗ p < .01. 40 Table 14: Did the demand-driven program match training to the stock or flow of aggregate employment? Reference distribution: 2013 empt. crosssection Aggregate empt. growth, 2013-2015 Focal program: Demand-driven program Main program Demand-driven program Main program Distribution across: (1) (2) (3) (4) 3-digit occupation 0.050 0.033 0.219 0.289 Dtate 0.057 0.035 0.194 0.138 State-occupation 0.006 0.004 0.057 0.061 Notes: Table presents the sum of squared deviations (SSD) across program training and aggregate reference distributions (indicated in column headings) for the categorical variables indicated in row headings. 41 Table 15: Differential effects by post-program employment growth, difference-in-differences IV estimates Outcome: Employed [0/1] Sample All registrants MDIC registrants Unemp. Benefic. All others (1) (2) (3) (4) Panel A: Demand-driven segment Main effect 0.075*** -0.040 0.155*** 0.011 (0.015) (0.034) (0.034) (0.014) Main * standardized employment growth share 0.005*** 0.007* -0.000 0.005*** (0.001) (0.004) (0.002) (0.001) Mean of outcome 0.565 0.748 0.604 0.533 St. dev. of outcome 0.496 0.434 0.409 0.499 R2 0.40 0.44 0.37 0.42 N 8,212,030 817,655 1,409,271 5,985,104 42 Panel B: All other courses Main effect -0.005 0.105*** -0.136*** (0.018) (0.023) (0.016) Main * standardized employment growth share 0.011*** -0.000 0.012*** (0.001) (0.001) (0.001) Mean of outcome 0.041 0.009 0.025 St. dev. of outcome 0.198 0.094 0.156 R2 0.45 0.39 0.46 N 45,919,339 2,752,395 42,920,272 Notes: Table presents present difference-in-differences IV estimates of course completion. Heteroskedasticity-consistent robust standard errors two-way clustered by class and month*year reported in parentheses. All specifications include an unreported constant term and vectors of individual and month*year fixed effects. Significance indicated by: ∗ p < .1, ∗∗ p < .05, ∗ ∗ ∗ p < .01. Figure 1: Overview of course request process Course demand Class creation Enrollment Stage aggregation MDIC MEC application application rejections rejections (50%) (26%) Courses cut Reasons: N •  Oversubscription Evaluation of •  Course cancellation Budget and Approved •  No-shows creation of courses •  Quit MEC classes Was Training Course requests MDIC approved completed MDIC Individuals from companies requests ? review employed in the MDIC (demand) (50%) company Y MDS demand for similar courses Individuals on Registrants Social assistance (student file) MDS 43 Ministry-specific processes MDS demand for other courses MTE demand for Individuals on similar courses unemployment MTE benefits MTE demand for other courses Notes: Figure presents a stylized depiction of the process by which MDIC receives course requests/applications and the various stages of screening and approval required until prospective trainees register and a course is held. Figure 2: Change in employment relative to course start: completers and non-completers All registrants -.04 .01 .06 .11 .16 .21 .26 .31 .36 .41 Parameter estimate with 95% CI Course -18 -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 Months relative to start of employment training Non-completers Completers Notes: Figure depicts the set of coefficients for course completers and non- completers corresponding to (αk +βk ) and (αk ), respectively, from equation 2 estimated for employment [0/1]. 95% confidence intervals constructed from heteroskedasticity-consistent standard errors clustered by individual and month. 44 Figure 3: Change in ln(real monthly wage rate) relative to course start: completers and non-completers All registrants Course .06 Parameter estimate with 95% CI .01 -.04 -18 -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 Months relative to start of employment training Non-completers Completers Notes: Figure depicts the set of coefficients for course completers and non-completers corresponding to (αk + βk ) and (αk ), respectively, from equation 2 estimated for ln(real monthly earnings). Sample implicitly comprised of person-months with positive wage earnings. 95% confidence intervals constructed from heteroskedasticity-consistent standard errors clustered by individual and month. 45 Figure 4: Change in employment relative to course start: completers and non-completers, MDIC registrants MDIC registrants .11 .06 Course Parameter estimate with 95% CI .01 -.04 -.09 -.14 -18 -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 Months relative to start of employment training Non-completers Completers Notes: Figure depicts the set of coefficients for course completers and non- completers corresponding to (αk +βk ) and (αk ), respectively, from equation 2 estimated for employment [0/1]. 95% confidence intervals constructed from heteroskedasticity-consistent standard errors clustered by individual and month. 46 Figure 5: Change in employment relative to course start: completers and non-completers, UB registrants UB registrants -.1 0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1 1.11.2 Course Parameter estimate with 95% CI -18 -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 Months relative to start of employment training Non-completers Completers Notes: Figure depicts the set of coefficients for course completers and non- completers corresponding to (αk +βk ) and (αk ), respectively, from equation 2 estimated for employment [0/1]. 95% confidence intervals constructed from heteroskedasticity-consistent standard errors clustered by individual and month. 47 Figure 6: Change in employment relative to course start: completers and non-completers, all other registrants All other registrants .37 .32 .27 Course Parameter estimate with 95% CI .22 .17 .12 .07 -.03 .02 -18 -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 Months relative to start of employment training Non-completers Completers Notes: Figure depicts the set of coefficients for course completers and non- completers corresponding to (αk +βk ) and (αk ), respectively, from equation 2 estimated for employment [0/1]. 95% confidence intervals constructed from heteroskedasticity-consistent standard errors clustered by individual and month. 48 Figure 7: Change in employment relative to course start: completers and non-completers, MDIC registrants at MDIC firms MDIC reg. at requesting firms .05 0 Course Parameter estimate with 95% CI -.05 -.1 -.15 -.2 -18 -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 Months relative to start of employment training Non-completers Completers Notes: Figure depicts the set of coefficients for course completers and non- completers corresponding to (αk +βk ) and (αk ), respectively, from equation 2 estimated for employment [0/1]. 95% confidence intervals constructed from heteroskedasticity-consistent standard errors clustered by individual and month. 49 Figure 8: Change in employment relative to course start: completers and non-completers, UB registrants at non-MDIC firms UB reg. at nonrequesting firms -.1 0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1 1.11.2 Course Parameter estimate with 95% CI -18 -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 Months relative to start of employment training Non-completers Completers Notes: Figure depicts the set of coefficients for course completers and non- completers corresponding to (αk +βk ) and (αk ), respectively, from equation 2 estimated for employment [0/1]. 95% confidence intervals constructed from heteroskedasticity-consistent standard errors clustered by individual and month. 50 Figure 9: Change in employment relative to course start: completers and non-completers, all other registrants at MDIC firms All other reg. at requesting firms .01 Course Parameter estimate with 95% CI 0 -.01 -18 -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 Months relative to start of employment training Non-completers Completers Notes: Figure depicts the set of coefficients for course completers and non- completers corresponding to (αk +βk ) and (αk ), respectively, from equation 2 estimated for employment [0/1]. 95% confidence intervals constructed from heteroskedasticity-consistent standard errors clustered by individual and month. 51 Figure 10: Change in employment relative to course start: completers and non-completers, all other registrants at non-MDIC firms All other reg. at nonrequesting firms .37 .32 .27 Course Parameter estimate with 95% CI .22 .17 .12 .07 -.03 .02 -18 -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 Months relative to start of employment training Non-completers Completers Notes: Figure depicts the set of coefficients for course completers and non- completers corresponding to (αk +βk ) and (αk ), respectively, from equation 2 estimated for employment [0/1]. 95% confidence intervals constructed from heteroskedasticity-consistent standard errors clustered by individual and month. 52 Appendix: For Web Publication Only A1 Cost benefit calculations Using parameters from the data and estimated employment effects, we calculate the net present value (NPV) of the training course based on increased employment earnings. We describe the funding/cost structure of the program below, and then present calculations for the representative course and trainee. It is important to note that these calculations do not include administrative costs nor impacts on training markets, including possible crowding out of alternate training providers. Furthermore, the calculations rely on several assumptions regarding interest and depreciation rates, as well as the rate at which formal employment displaces informal earnings. Pronatec courses were financed by the federal government on a student-hour basis. For 2014, cost per student-hour was R$10.00 for in-person courses. Course providers were then reimbursed for the student-hours of “confirmed students.” These were students who: (a) attended at least 20% of the course hours or (b) attended 25% of the course hours in the first four months of the course.10 Course providers were reimbursed according to the full number of hours of the course for all students who reached this minimum attendance threshold, independent of whether the student completed those hours or dropped out before the end of the course. The average course in the demand-driven program consists of 200 hours of classroom time. Across all courses, this results in an average cost of R$2,000 per trainee (approxi- mately 1,780 in real 2012 Reais). However, because some registrants did not incur costs as described above, the per-registrant cost is slightly smaller than the amount above (80 percent of course registrants are counted in the calculation of course costs) – resulting in a per registrant cost figure of approximately R$1,470. In this calculation, we assume that the benefits of the program are captured entirely in earnings from increased employment 10 See article 64 of Portaria No. 168/2013, available at http://pronatec.mec.gov.br/images/stories/ pdf/port_168_070313.pdf. 53 Appendix: For Web Publication Only rather than increased wage rates. The estimates of employment effects used in conjunc- tion with the median monthly wage rate of approximately R$1,380 to estimate the nominal monthly earnings benefit of the program of approximately R$119 per month assuming an 8.6 percentage-point employment effect. However, since these benefits are realized only by course completers, the per-registrant benefit is around 40% of this figure, or approximately R$46 per month. This figure, however, does not account for the displacement effect of increased formal employment, which will reduce the net benefit depending on the degree to which trainees would have otherwise been informally employed. We first assume that the formal net wage premium is 20 percent, and then present calculations based on two scenarios. The first, more conservative scenario assumes that all those who gained formal employment due to the program would have been otherwise informally employed, so displaced earnings are 80% of the gained employment earnings. The second scenario assumes that the program displaces informal employment among 75% of trainees, the remaining 25% of whom would have been unemployed – resulting in a net earnings displacement of 60%. Assuming a five percent annual interest rate and also five percent annual real depreciation of skills, we find that under 60% earnings displacement the per-registrant NPV of the program is R$2,189, which after course fiscal costs yields a net benefit of R$638. Assuming displaced informal earnings would have been 80% of formal earnings, the program yields a NPV of R$1,094 per registrant, resulting in a net benefit of -R$456 after course costs. 54 Appendix: For Web Publication Only A2 Appendix Tables and Figures Appendix Table 1: Summary statistics, MDIC registrants Variable Mean Std. Dev. Min. Max. N Worker number of observations 31.131 4.283 18 35 29297 Male[0/1] 0.781 0.414 0 1 29297 Course Offer 0.96 0.195 0 1 29297 Unemployment benefits registrant 0 0 0 0 29297 MDIC registrant 1 0 1 1 29297 Completed course[0/1] 0.493 0.5 0 1 29297 Course had admin. noncompleters [0/1] 0.341 0.474 0 1 29297 Employed [0/1] 0.748 0.434 0 1 912053 Employed in an MDIC-requesting firm 0.237 0.425 0 1 912053 Gross deflated monthly earnings(R$) 1491.79 879.04 1.038 6138.825 659514 ln(gross deflated monthly earnings(R$)) 7.148 0.593 0.038 8.722 659514 Gross deflated monthly wage rates 1530.628 882.11 1.218 6138.825 659520 ln(gross deflated monthly wage rates) 7.191 0.537 0.197 8.722 659520 Notes: Authors’ calculations using RAIS. Appendix Table 2: Summary statistics, Unemployment benefits registrants Variable Mean Std. Dev. Min. Max. N Worker number of observations 30.209 3.76 18 35 61995 Male[0/1] 0.61 0.488 0 1 61995 Course Offer 0.832 0.374 0 1 61995 Unemployment benefits registrant 1 0 1 1 61995 MDIC registrant 0 0 0 0 61995 Completed course[0/1] 0.3 0.458 0 1 61995 Course had admin. noncompleters [0/1] 0.822 0.382 0 1 61995 Employed [0/1] 0.604 0.489 0 1 1872837 Employed in an MDIC-requesting firm 0.009 0.096 0 1 1872837 Gross deflated monthly earnings(R$) 1134.875 666.16 1.019 6139.095 1091600 ln( gross deflated monthly earnings(R$)) 6.881 0.601 0.019 8.722 1091600 Gross deflated monthly wage rates 1213.404 694.817 1.01 6139.095 1091624 ln(gross deflated monthly wage rates) 6.982 0.481 0.01 8.722 1091624 Notes: Authors’ calculations using RAIS. 55 Appendix: For Web Publication Only Appendix Table 3: Summary statistics, All other registrants Variable Mean Std. Dev. Min. Max. N Worker number of observations 30.567 4.264 18 35 244008 Male[0/1] 0.576 0.494 0 1 244008 Course Offer 0.953 0.212 0 1 244008 Unemployment benefits registrant 0 0 0 0 244008 MDIC registrant 0 0 0 0 244008 Completed course[0/1] 0.411 0.492 0 1 244008 Course had admin. noncompleters [0/1] 0.571 0.495 0 1 244008 Employed [0/1] 0.533 0.499 0 1 7458651 Employed in an MDIC-requesting firm 0.025 0.157 0 1 7458651 Gross deflated monthly earnings(R$) 1129.573 714.946 1 5209.774 3690130 ln(gross deflated monthly earnings(R$)) 6.849 0.645 0 8.558 3690130 Gross deflated monthly wage rates 1192.811 724.908 1.009 5209.774 3690272 ln(gross deflated monthly wage rates) 6.937 0.547 0.009 8.558 3690272 Notes: Authors’ calculations using RAIS. 56 Appendix Table 4: Employment effects of worker training: selected β coefficient estimates Estimation of employment across course completers and non-completers Outcome: Employed [0/1] Sample All registrants MDIC registrants Unemp. Benefic. All others (1) (2) (3) (4) 18 months prior to course start (t=-18) 0.014 0.041 0.135 0.043 (0.006)** (0.010)*** (0.014)*** (0.004)*** 12 months prior to course start (t=-12) 0.008 0.04 0.119 0.042 (0.006) (0.010)*** (0.009)*** (0.004)*** 9 months prior to course start (t=-9) 0.002 0.039 0.105 0.046 (0.006) (0.010)*** (0.010)*** (0.004)*** 6 months prior to course start (t=-6) 0.001 0.028 0.105 0.048 (0.007) (0.010)*** (0.011)*** (0.004)*** 1 month prior to course start (t=-1) 0.03 0.012 0.094 0.038 (0.006)*** (0.009) (0.018)*** (0.003)*** At end of course (t=+6) -0.014 0.045 0.011 -0.002 57 (0.006)** (0.012)*** (0.011) (0.003) 3 months after course (t=+9) 0.018 0.059 0.062 0.035 (0.006)*** (0.011)*** (0.011)*** (0.004)*** 6 months after course (t=+12) 0.037 0.068 0.083 0.056 (0.005)*** (0.012)*** (0.010)*** (0.004)*** 9 months after course (t=+15) 0.052 0.073 0.094 0.073 (0.005)*** (0.013)*** (0.010)*** (0.005)*** Appendix: For Web Publication Only 12 months after course (t=+18) 0.059 0.074 0.098 0.079 (0.005)*** (0.013)*** (0.010)*** (0.004)*** Individual fixed effects Y Y Y Y Year * Month fixed effects Y Y Y Y Mean of outcome 0.565 0.748 0.604 0.533 St. dev. of outcome 0.496 0.434 0.409 0.499 R2 0.41 0.45 0.44 0.43 N 10,243,541 912,053 1,872,837 7,458,651 Notes: Table presents selected coefficients measuring the difference in outcomes between course completers versus non-completers at different time periods relative to the start of course completion (β vector in equation 2 in the text) for the sample group indi- cated in the column headings. Heteroskedasticity-consistent robust standard errors two-way clustered by class and month*year reported in parentheses. All specifications include an unreported constant term and vectors of individual and month*year fixed effects. Significance indicated by: ∗ p < .1, ∗∗ p < .05, ∗ ∗ ∗ p < .01. Appendix Table 5: Employment effects due to MDIC-requestors and outside employers: selected β coefficient estimates Estimation of employment across course completers and non-completers, by employer requesting status Outcome: Employed in requesting firm [0/1] Employed in other firm [0/1] Sample MDIC UB All others MDIC Unemp. Benefic. All others (1) (2) (3) (4) (5) (6) 18 months prior to course start (t=-18) 0.005 0.003 -0.003 0.036 0.132 0.047 (0.008) (0.001)** (0.000)*** (0.010)*** (0.014)*** (0.004)*** 12 months prior to course start (t=-12) 0.005 0.005 -0.001 0.036 0.114 0.043 (0.007) (0.001)*** (0.000) (0.008)*** (0.009)*** (0.004)*** 9 months prior to course start (t=-9) 0.006 0.005 <0.001 0.033 0.1 0.047 (0.005) (0.001)*** (0.000) (0.008)*** (0.011)*** (0.004)*** 6 months prior to course start (t=-6) 0.004 0.005 <0.001 0.024 0.1 0.048 (0.005) (0.001)*** (0.000) (0.008)*** (0.012)*** (0.004)*** 1 month prior to course start (t=-1) <0.001 0.002 0.001 0.012 0.092 0.038 (0.006) (0.000)** (0.000)** (0.005)** (0.018)*** (0.003)*** At end of course (t=+6) 0.023 <0.001 0.001 0.022 0.011 -0.003 (0.005)*** (0.000) (0.000)** (0.009)** (0.011) (0.003) 3 months after course (t=+9) 0.023 0.002 0.004 0.036 0.060 0.031 58 (0.006)*** (0.000) (0.000)*** (0.010)*** (0.011)*** (0.004)*** 6 months after course (t=+12) 0.026 0.001 0.004 0.042 0.082 0.052 (0.007)*** (0.001) (0.000)*** (0.010)*** (0.010)*** (0.004)*** 9 months after course (t=+15) 0.027 0.001 0.004 0.046 0.093 0.069 (0.009)*** (0.001) (0.000)*** (0.012)*** (0.010)*** (0.005)*** 12 months after course (t=+18) 0.026 0.001 0.004 0.048 0.097 0.075 (0.010)** (0.001) (0.000)*** (0.010)*** (0.010)*** (0.004)*** Individual fixed effects Y Individual fixed effects Y Y Y Y Appendix: For Web Publication Only Year * Month fixed effects Y Year * Month fixed effects Y Y Y Y Mean of outcome 0.237 0.009 0.025 0.511 0.595 0.508 St. dev. of outcome 0.425 0.096 0.157 0.5 0.491 0.5 R2 0.82 0.42 0.69 0.60 0.44 0.44 N 912,053 1,872,837 7,458,651 912,053 1,872,837 7,458,651 Notes: Table presents selected coefficients measuring the difference in outcomes between course completers versus non-completers at different time periods relative to the start of course completion (β vector in equation 2) for the sample group indicated in the column headings. Heteroskedasticity-consistent robust standard errors two-way clustered by class and month*year reported in parentheses. All specifications include an unreported constant term and vectors of individual and month*year fixed effects. Significance indicated by: ∗ p < .1, ∗∗ p < .05, ∗ ∗ ∗ p < .01. Appendix: For Web Publication Only Appendix Table 6: Testing selection of UB registrants on observables across programs Outcome: UB registrant took firm-requested course[0/1] Specification: Base Incl. wages (1) (2) Employed 1 month before course [0/1] -0.025*** -0.022** (0.005) (0.007) Employed 3 months before course [0/1] 0.012* 0.025** (0.007) (0.009) Employed 6 months before course [0/1] -0.047*** -0.029 (0.012) (0.015) ln(wage) 1 month before course -0.006* (0.003) ln(wage) 3 months before course 0.004 (0.003) ln(wage) 6 months before course 0.011* (0.005) Education (years) 0.012*** (0.001) Class fixed effects Y Y N 167,706 167,706 R2 0.006 0.0065 Notes: Table presents coefficients from the estimation of an indicator for being in a firm-requested course among all UB registrants of Pronatec and Pronatec-MDIC. Heteroskedasticity-consistent robust standard errors clustered by class reported in paren- theses. All specifications include an unreported constant term. Significance indicated by: ∗ p < .1, ∗∗ p < .05, ∗ ∗ ∗ p < .01. 59 Appendix Figure 1: MDIC course skill intensity Density plot of Pronatec courses by skill level, 2014 5 MDIC courses nonMDIC courses 4 Difference (MDIC−nonMDIC) 3 Density 2 1 60 0 −1 6.5 7.0 7.5 8.0 Appendix: For Web Publication Only median ln(wage) for occupation corresponding to course Note: Figure depicts difference in skill distribution of courses requested under Pronatec-MDIC compared to all other Pronatec courses offered in 2014. Skill level is measured by the median 2012 monthly wage in the occupation served by the course. Difference in densities (MDIC - non-MDIC) shown in black. Bandwidth chosen at 0.03.