/ ZVI/S THE WORLD BANK Internal Discussion Paper EUROPE AND CENTRAL ASIA REGION Report No. IDP-156 Evaluations of Retraining Programs in OECD Countries: Implications for Economies in Transition by Amit Dar and Indermit S. Gill October, 1995 Office of the Vice President Europe and Central Asia Region Discussion Papers are not formal publications of the World Bank. They present preliminarv and unpolished results of country analysis or research that is circulated to encourage discussion and comment; citation and the use of such a paper should take account of its provisional character. The findings, interpretations, and conclusions expressed in this paper are entirely those of the author(s) and should not be attributed in any manner to the World Bank, to its affiliated organizations, or to members of its Board of Executive Directors or the countries they represent. ACKNOWLEDGMENT This paper was prepared under the general guidance of Kathie Krunnm, Principal Economist, ECAVP, and the immediate supervision of Jane Armitage, Manager, Labor Markets and Social Insurance Group, PSP. The authors are grateful to Ana Revenga, EC4C1, and Michal Rutkowski, EC4HR for helpful suggestions. TITLE: EVALUATIONS OF RETRAINING PROGRAMS IN OECD COUNTRIES: IMPLICATIONS FOR ECONOMIES IN TRANSITION AUTHORS: Amit Dar and Indermit S. Gill ABSTRACT This paper presents evaluative information on retraining schemes for unemployed adults in OECD countries. These evaluations fall into three categories: experimental evaluations that compare the effects of retraining programs for participants and a control group chosen prior to the scheme's initiation; quasi-experimental evaluations that choose the control group after the program is completed; and non-scientific evaluations that use no control group. We discuss the correlates - the specific labor market problem addressed, general market conditions, and intervention design - of success and failure of these schemes, separately for programs for workers laid-off en masse and the long-term unemployed. Finally, we draw the implications of these findings for countries in the former Soviet Union and Eastern Europe. TABLE OF CONTENTS Section Title Page Abstract Summary ii I. Introduction 1 II. Evaluation Techniques 2 1. Different techniques 2 2. The importance of considering costs 2 III. Evaluations of Retraining Programs in OECD Countries 3 1. Plant closures and mass layoffs 3 2. Long-term unemployed 4 IV. Conclusions 8 V. References 10 Annex: Detailed Table of Evaluations 13 EVALUATIONS OF RETRAINING PROGRAMS IN OECD COUNTRIES: IMPLICATIONS FOR ECONOMIES IN TRANSITION SUMMIARY Scope. This survey re-examines more than twenty retraining programs that have been evaluated in OECD countries since 1980. These evaluations use a variety of techniques ranging from randomized experiments to informal or non-scientific estimations of program effectiveness. We distinguish here between retraining schemes for workers who have been displaced due to plant closures or restructuring, and retraining programs for the long-term unemployed. These groups diffei in that the long-term unemployed are a relatively heterogenous group of individuals, are generally more dispersed geographically, and - almost by definition - have been unemployed longer than those displaced en masse. Retraining for displaced workers. For those displaced en masse. we find that retraining programs in OECD countries have the following main features: First, these programs address manufacturing sector layoffs and plant closures, generally during deteriorating conditions in industry or aggregate employment. Second, the programs were mostly classroom-based, and accompanied by job search assistance (JSA). With only one exception, on-the-job training was not provided. Third, evaluations indicate poor results for retraining programs. Some programs resulted in modest gains in re-employment probabilities, but wage changes are generally negative. Fourth, quasi-experimental evaluations indicate that retraining programs are generally no more effective than JSA in increasing either re-employment probabilities or post-intervention earnings. Finally, retraining programs for displaced workers appear to be between two and four times more expensive than JSA. Combined with the previous finding, this implies that JSA may be more cost-effective than retraining in assisting displaced workers get jobs. Retraining for the long-tern unemployed. Retraining programs for the long-term unemployed (LTU) have the following main features: First, they are generally found during improving conditions in industry or aggregate employment, in contrast to retraining programs for workers displaced due to mass layoffs and plant closures. Second, while retraining programs for workers displaced en masse are mostly classroom-based, on-the-job training is somewhat more important for the LTU. Third, as with retraining programs for workers displaced en masse, evaluations indicate disappointing results. Few programs result in gains in either re-employment probabilities or wages. Some evaluations indicate that these programs are more beneficial for some groups such as women. Where gains in re-employment are observed, longitudinal studies generally indicate dissipation of the effects of retraining within a couple of years after the program. Fourth, evaluations also indicate that retraining programs are generally no more effective than JSA in increasing either re-employment probabilities or post-intervention earnings. Finally, retraining programs for the LTU appear to be twice as costly as JSA. Combined' with the previous finding, this implies that JSA mnay be more somewhat more cost-effective than relt airaing in assisting the long-term unemployed get jobs. Implications for economies in transition. The preliminary lessons that transition and other ii Retraining Programs in OECD Countries Implications for economies in transition. The preliminary lessons that transition and other restructuring economies can draw from this somewhat sketchy summary of OECD experience are: First, it is necessary to evaluate retraining (and other) public interventions using sound techniques. It is instructive that while the non-scientific evaluations of retraining programs present a rosy picture based on placement rates and other informal evidence, scientific evaluations are quite discouraging. Relying on non-scientific evaluations may lead countries to incorrect policy conclusions. Second, rigorous evaluations, while not necessarily allowing a complete social cost- benefit analysis, can be useful guides for policymakers in solving public expenditure issues related to labor programs. Reviews of rigorous evaluations like this study find, for example, that job search assistance measures - which cost less than retraining but appear equally effective - may be a relatively cost-effective device in assisting displaced workers. The absence of such evaluations may result in inefficient allocation of government funds. Third, OECD experience of retraining programs for workers displaced en masse may be useful in designing assistance programs in countries that soon expect labor-shedding from large enterprises in the manufacturing and mining sectors, such as the economies of the former Soviet Union: principally, this involves recognizing that retraining should not be the main form of assistance. Fourth, East and Central European economies which are beginning to experience long- term unemployment can learn from OECD experience with retraining programs for the long-term unemployed, which indicates that retraining programs are more beneficial for some groups than for others within this relatively heterogeneous group of job-seekers. However, it is difficult to predict a priori who will benefit most from retraining: principally, these results call for using modest pilot programs, evaluating them rigorously, and then tightly targeting retraining to those for whom it is found most cost- effective. iii EVALUATIONS OF RETRAINING PROGRAMS IN OECD COUNTRIES: IMPLICATIONS FOR ECONOMIES IN TRANSITION I. INTRODUCTION AND OBJECTIVES Industrialized countries often spend a sizeable fraction of their budgets on active labor programs. In 1992, OECD countries spent between 0.1 percent (Japan) to 2.6 percent (Sweden) of GDP on active labor programs. In several countries, tTaining for the unemployed is "the largest category of active programs, and is often perceived as the principal alternative to regular unemployment benefits' (OECD, 1994). Countries in Eastern Europe - where the role of active labor programs is a topic of current debate - also spent between 0.2 and 3 percent of GDP on these programs (OECD, 1994a). In spite of this - possibly because such assistance is viewed almost as a fundamental right of workers in Western Europe - these programs are often not carefully evaluated outside the US. Nevertheless, evaluations of retraining programs do exist for some OECD countries. This paper surveys this evidence. This paper examines only the evaluations of retraining programs, so the focus is largely on adults with previous work experience, rather than on unemployed school-leavers. For surveys of training programs in OECD and other countries, see OECD (1993) and World Bank (1995). We distinguish here between retraining schemes for workers who have been displaced due to plant closures or restructuring, and retraining programs for the long-term unemployed. These groups differ in important ways. First, the long-term unemployed are a relatively heterogenous group of individuals as compared to those laid off en masse from a single plant or firm. Second, though some programs for the long-term unemployed are targeted at specific regions, the long-term unemployed are generally more dispersed geographically. Third, the duration of unemployment is, almost by definition, greater for the long-term unemployed. Finally, as it turns out, retraining programs for the long-term unemployed are generally a mix of classroom and in-plant training, while for displaced workers they usually are only the former. In this paper, we present evaluative information on retraining schemes for unemployed adults gleaned from documents that describe these schemes. These evaluations fall into three categories: First, experimental evaluations that compare the effects of retraining programs for participants and a control group chosen prior to the scheme's initiation; second, quasi-experimental evaluations that choose the control group after the program is completed; and finally, non-scientific evaluations that use no control group. We discuss the correlates - the specific labor market problem, general market conditions, and intervention design - of success and failure of these schemes separately for workers laid-off en masse and the long-term unemployed. Finally, we draw the implications of these findings for countries in the former Soviet Union and Eastern Europe; given the sketchy nature of OECD evidence, these should be regarded as suggestive rather than conclusive. Retraining Programs in OECD Countries 2 II. EVALUATION TECHNIQUES 1. DIFFERENT TECHNIQUES Techniques for evaluating the effectiveness of retraining labor programs can be broadly classified into scientific and non-scientific evaluations. Scientific evaluations are of two types: experimental and quasi-experimental. Experimental evaluations require selection of both the "control" and "treatment" groups - those who receive the assistance and those who do not, respectively - prior to the intervention. If large numbers of individuals are randomly assigned to treatment and control groups, average characteristics of the two groups should not differ significantly so that any difference in outcomes can be attributed to program participation. Program impacts are then computed simply as the difference in means of the outcomes of interest between program participants and the control group. In quasi-experimental studies, treatment and control groups are selected after the intervention. To compute program effectiveness, statistical techniques are used to correct for differences in characteristics between the two groups. Non-scientific techniques do not use control groups in evaluating interventions. They generally rely on statistics compiled by program administrators. These evaluations are of little use: without a control group, it is difficult to attribute success or failure of participants to the intervention, since these effects are contaminated by other factors such as worker-specific attributes or economy-wide changes. 2. THE IMPORTANCE OF CONSIDERING COSTS For the purposes of informing policy decisions, an evaluation is not complete until costs of both the retraining program and its alternatives are considered. Thus, for example, if retraining is shown to be twice as costly as job search assistance to the unemployed, but only as effective as JSA in facilitating access to jobs and wage gains, then JSA is twice as cost-effective as retraining even though the two are equally effective. At least at the margin, this would constitute a case for reallocating resources from retraining to JSA. Unfortunately, costs appear to be the least analyzed aspect of these programs in OECD countries. However, even the most careful evaluations of retraining programs cannot be used for social cost- benefit analysis. The main reason is that retraining programs may simply result in displacement of previously employed workers by the retrainees, so that aggregate unemployment rates remain unaffected by the intervention.' However, when done correctly, evaluations are good guides forprivate cost-benefit analysis, which can be used by policymakers to institute cost-recovery in public programs and in l The use of flow analysis of administrative macro data to establish the overall effect of a measure on outflows from unemployment may also be indicative of the effectiveness if the program is administered on a large scale. The measure can be considered effective if there is a statistically significant positive correlation between the measure and outflows from unemployment. If a matching function is built properly with hiring of the unemployed into regular jobs regressed on the stock of vacancies and unemployment, and the stock of program participants, the methodology can bring about credible evaluation results. Retraining Programs in OECD Countries 3 promoting private provision. Evaluations may also help in deciding whether retraining programs contribute to reduced budgetary expenditures by moving people off unemployment benefits into productive employment, or whether they are a net drain inspite of being effective in doing so. III. EVALUATION RESULTS In this section, we analyze the results of retraining program evaluations. To do so, we classify the specific labor market problem that the program is designed to address, list other relevant labor-related indicators, describe the intervention design, classify the evaluation type, and discuss the main results. (a) Labor market problem. The immediate problems that retraining programs have been used to address in OECD countries can be categorized as assisting workers laid off en masse, those who have lost their jobs due to plant closures, and the long-term unemployed. As discussed above, workers displaced en masse and due to plant closures share crucial attributes and can be distinguished from the long-term unemployed. Evaluations of retraining programs for these two groups are discussed separately. (b) Relevant indicators. It is likely that the success of retraining programs depends on aggregate or regional labor market conditions such as unemployment rates and the state of the major industry. These indicators are reported to provide some understanding of the state of labor demand. (c) Intervention design. We examine what type of retraining is provided, e.g., whether the training is in classrooms or on-the-job, and whether retraining is accompanied by or in lieu of other measures such as job search assistance (JSA). (d) Evaluation type. Evaluations are classified into experimental, quasi-experimental, and non- scientific. While the results of non-scientific evaluations are reported, we do not draw any inferences on the success/failure of retraining programs from these evaluations. (e) Main results. The effects of the program on re-employment probability and wage gains are summarized, both for sub-groups of retrainees and intervention type. (f) Special comments. These comments mostly concern the costs of the program, when they are reported. 1. MASS LAYOFFS AND PLANT CLOSURES Table 1 summarizes the results of studies examining the effectiveness of retraining programs for workers displaced through mass layoffs and plant closures. The annex provides more information on the same cases in the same format as Table 1. The main findings are: (a) Retraining programs address manufacturing sector layoffs andplant closures. The evaluations examined here are for retraining programs to assist workers in the automobile, shipbuilding, mining, steel and pulp industries. The number of workers laid off vary from about 500 to 3000 across cases. Retraining Programs in OECD Countries 4 (b) These programs are generally during deteriorating conditions in industry or aggregate employment. Generally, these retraining programs were instituted during periods of high or rising aggregate unemployment, or contraction of the manufacturing subsector. The rationale appears to be to assist the affected workers in any way possible. (c) The programs generally provided classroom or workshop training andjob search assistance. The retraining programs were mostly classroom-based, and accompanied by JSA. With only one exception, on-the-job training (OJT) was not provided or facilitated. (d) There are no experimental studies and many non-scientific evaluations. Quasi-experimental and non-scientific techniques were used for evaluating retraining programs for these workers. None of these studies were longitudinal, providing at best a snapshot of the labor market benefits of the retraining program. (e) Evaluations indicate poor results for retraining programs. Some programs resulted in modest gains in re-employment probabilities, but wage changes are often negative. Quasi-experimental evaluations (we do not discuss the results of nonscientific evaluations) indicate that retraining programs are generally no more effective than JSA in increasing either re-employment probabilities or post-intervention earnings. (f) Retraining programs for displaced workers are costlier than JSA. Retraining programs appear to be between two and four times more expensive than JSA. Combined with the previous finding, this implies that JSA may be more cost-effective than retraining in assisting displaced workers get jobs. (g) Costs, when known, vary between $3,500 and $25,000 per person. Evaluations seldom report the full costs of retraining or JSA programs, so it is difficult to determine the absolute cost- effectiveness of these programs. 2. LONG-TERM UNEMPLOYED Table 2 summarizes the results of studies examining the effectiveness of retraining programs for the long-term unemployed. The table in the annex provides more information in the same format as Table 2. The main findings are: (a) The clientele for retraining programs for the long-term unemployed (LTU) is relatively heterogeneous. Since the LTU consist of individuals displaced from various sectors, they are likely to be more heterogeneous in terms of skills, age, and education. (b) Generally improving conditions in industry or aggregate employment. In contrast to retraining programs for workers displaced due to mass layoffs and plant closures, these retraining programs are generally instituted during periods of low or falling aggregate unemployment. The rationale appears to be that these programs, if appropriately designed, can enable the LTU to obtain some of the jobs that are being created. (c) The programs provide a mix of classroom or workshop training, OJT, and JSA. While Retraining Programs in OECD Countries 5 retraining programs for workers displaced en masse are mostly classroom-based, on-the-job training (OJT) is somewhat more important for the LTU. (d) Both experimental and longitudinal studies exist for the US. In the case of the LTU, experimental evaluations have been conducted in the US. Quasi-experimental and non-scientific techniques have also been used for evaluating retraining programs for these workers. Some of these studies were longitudinal, providing an indication of the long-term labor market benefits of the retraining program. (e) Evaluations indicate disappointing results. Again, we do not discuss the results of nonscientific evaluations. Few programs result in gains in either re-employment probabilities or wages. Some evaluations indicate that these programs are more beneficial for women. Where gains in re-employment are observed, longitudinal studies generally indicate dissipation of the effects of retraining within a couple of years after the program. Evaluations also indicate that retraining programs are generally no more effective than JSA in increasing either re-employment probabilities or post-intervention earnings. (f) Retraining programs for the LTU are costlier than JSA. Retraining programs appear to be twice as costly as JSA. Combined with the previous finding, this implies that JSA may be more somewhat more cost-effective than retraining in assisting the long-term unemployed get jobs. (g) Costs, when known, vary between $900 and $12,000 per person. Again, as in the case of retraining for those laid off en masse, evaluations seldom report the full costs of retraining or JSA programs for the long-term unemployed, so it is difficult to determine the absolute cost- effectiveness of these programs. Table 1: Overview of Evaluation Studies: Plant Closures & Mass Layoffs Labor Market Problem Relevant Indicators Intervention Design Result Comments Evaluate Type Ford Plant Closure in San lose, 2400 A 25 percent decline in auto Basic skills trainitig as well as targeted Non- High success rate in placement. Training cost about $ 6000/ workers to lose their jobs (I1982!. production between 1978 and 1980. vocational training in marketable skills. scientific worker Shiipyard closure in Storstrom, High unemployment rates in Program in traininig women entrepreneurs Non- 51 businesses were set up by (mostly Low success rate and quite Denmark (1986). 20(X) lose jobs. regions, especially among women. to help them start their own business. scientific part-time). Low additional costly. employment. Steel/Coal plant closure in Creusot, Contracting steel sector, with over Workeis promised reemploymenit in Non- High success rate in placement. Cost-effectiveness and long France (1984). 1230 people lost jobs. 6000 job losses over the previous region after retraining; all provided JSA; scientific term employment impact few montis. firimis given financial incentives to hIire unclear. Closure of Pulp plant in Kramfors in Unemp. in Sweden rose from 1.6% NA Quasi- Lower wages relative to control No benefits. Negative Northem Sweden (1977). in to 2.2% during that period. Exper. group. Eamings drop is significant in retums after factoring in the first year. Low long-term gains. costs. Mass Layoffs (3000) due to auto A 25 percent decline in auto Workers provided a mixture of job search Quasi- Training did not iniprove Training cost twice as much plattt closures in Michigan (1980-83). production between 1978 and 1980. assistance and classrooni training. Exper. reemployment rate. Eamings as JSA. ambiguous. Mass Layoffs due to auto and steel High unemployment rates in Workers provided with a mixture of job Quasi- No incremental effect above that of Costs of CT and OJT plant closures in Buftalo (1982-83). 1981/82 nationally (9.5%). search assistance and either classroom or Exper. JSA for either classroom training or programs ($3300) are four OJT. OJT. times those of JSA ($850) Plant closures at 13 steel factories NA Job search assistance and training Quasi- Slight increase in reemployment Positive impact. No cost- and mines in Canada. provided to these workers. Exper. probability. benefit analysis. 445 workers laid off due to High unemp. rates in Australia at Classroom training. Main distinction Quasi- Except driver training, other courses Self-selection problem as automobile plant closure in Australia 9% 1984. Fell to 8% in 1985. between courses relates to provision of Exper. lead to a decline in the probability of control consists of people (1984). driver training. employment. declining participation. Around 2000 workers laid off at the Volvo decided to establish an auto Courses offered in welding, engineering Non- By late 1987, over 90% of the Reason for success - Uddevalla shipyard in Sweden in plant at Uddevalla at the same time. and control engineering. Scientific trainees had found jobs or become economy and LM was 1985. self-employed. buoyant. The Volvo plant at Goteborg, Unemp. rates were rising sharply - Training to help the existing workforce to None Costs are to run to around Sweden planned to lay off 1000 from 3.2 % in 1991 1o 5.9% in manage the change without job loss. $25 million ($25000 per workers in 1992. 1992. person). Sweden: 1980s and 1990s NA Participants have greater difficulty in Cost-effectiveness of finding jobs than openly unemployed training programs has declined: effectiveness fell and costs increased. 6 Table 2: Overview of Evaluation Studies: Long-term Unemployed Labor Market Problem Relevant Indicators Intervention Design Evaluate Results Comments Type Provide better access to jobs for Unemployment rates have begun Four main skill areas are: metal training, Non- In 1991, 82 individuals were enrolled Training quite expensive as long-term unemployed in Tilburg falling in Holland since the mid wood training, installation techniques and scientific in the course. 52 completed and 41 annual funding of the (since early 1980). 1980's. clothing. went onto a job or further education. program is about $7.5m ($10,000/ trainee). Tackle high regional unemployment In 23 regions in Germany Retraining of these labor force Non- Training reduced unemployment High cost of training levels in Germany (1979). unemployment levels were above 6 participants in fimis. scientific somewhat. In 2 yrs 40 percent had left ($500m for training and percent. their jobs again. other interventions) Train workers eligible for displaced A decline in petrochemical Workers provided with job search Experi- No additional eaming gain accrued Classroom training twice as worker program in Houston (1983- industry. In U.S. unemp. rates assistance and classroom training. mental from classroom training as compared costly as JSA ($2,900 versus 85). declined. to the JSA. $1,400 respectively). Train workers eligible for displaced In the U.S., unemployment rates Workers provided with job search Experi- After I year, no effect on men's and Earnings exceeded program worker program in El Paso (83-85) declined. assistance and classroom training. mental increase in women's eamings. costs for women not men. Retrain long term unemployed in Unemployment rates in the U.S. A mixture of JSA, JSA followed by OJT, Experi- Trg. significantly increases eamings Self-selection problem. New Jersey (1986-87). fell. JSA followed by CT. mental and employment over 2.5 years. Costs exceed benefits. Increase the employability of long Unemp. rates declined from 11.2 Formal and on-the-job training. Wages Quasi- Employability of women goes up Program cost = term unemployed in Canada (1985-). percent in the mid 1980s to 7.5 and direct cost of classroom training are Exper. while it declines for males. Female $9300/person. Training is percent in 1989. subsidized. eamings constant, and fall for males. not cost-effective. Retrain individuals at risk of Vocational training. Non- 54% got jobs. Re-emp. rate lower No data on costs available becoming unemp. in Germany. scientific for older individuals. 30% of men and women unemployed after 2 years. Same as above in Germany (1987- Unemployment steady between Same as above. Quasi- No impact on the flows out of or into No information on wages or 88). 1986 and 1988. Fell by 1990. exper. of short- or long-term unemployment. costs. Labor flow study. Equip displaced workers with skills Unemp. rates fell from 7.1 % in Develop skills in occupations different Quasi- Little effect on wages and Cost = $12000/trainee. to enter a new occupations, U.S., 1985 to 6.1% by 1987. from occupations in pre-displacement Exper. employment. Training is not cost- 1986. jobs. effective. Help displaced workers gain High unemployment rates (near Voc. Trg. provided by local authorities. Non- Little impact on flows from long-term Cost-effectiveness is likely employment in England (1980's). 11% I) persisted through the 1980s. Low private sector involvement. scientific unemployment. to be poor. Help displaced workers gain better Vocational training courses. Quasi- Even after two years employment No data is available on employment opportunities. Exper. situation for trainees not significantly wages and costs. better. 7 Retraining Programs in OECD Countries 8 IV. CONCLUSIONS In OECD countries, retraining is often an important component of the total package of unemployment benefits for those laid-off due to plant closure or restructuring, as well as programs to assist the long-term unemployed. However, there exist few thorough evaluations of the effectiveness of retraining programs in increasing employability and/or wages, and almost none of their cost- effectiveness which is pertinent for policy makers concerned with financing issues. Where retraining programs have been evaluated, studies vary in their complexity and sophistication, even apart from the fundamental choices of methods or focus. In the U.S., which has led the push for impact evaluation, research sometimes involves experiments where participants and control groups are randomly assigned in advance. In other OECD countries, randomized experiments are almost non-existent. Participants are at best compared with benchmark groups of persons selected after the program has been in place for a while - these evaluations are called "quasi-experimental" if they use statistical techniques to correct for differences in characteristics. Out of the 22 evaluations reviewed here, 10 were non-scientific, some of the others (experimental and quasi-experimental) were of dubious quality, and none contained a rigorous cost-benefit analysis of the retraining program it evaluated. The paucity of rigorous evidence on the costs and effectiveness of retraining programs do not allow us to conclude definitively on the economic justification of these interventions. The scattered evidence that we do analyze, however, does not appear to justify expansion of retraining programs to cover more of the unemployed. These conclusions are consistent with the findings in OECD (1993), which are summarized as follows: "For the broadly targeted sub-group of programs, the overall impression is most troubling. Available evidence does not permit strong conclusions, but it gives remarkably meagre support of a hypothesis that such programs are effective." OECD Employment Outlook (1993), page 58. We find, however, that alternatives to retraining such as job search assistance are more promising avenues. JSA is cheaper than public retraining programs, and seems to be as effective in facilitating access to jobs. It is worth noting that this result is found in both improving and deteriorating general labor market conditions, and for the recently laid off as well as the long-term unemployed. Again, what we find is consistent with OECD (1993) which states: "A review of progranmme-level evaluation studies shows that several major labor market programmes have helped to improve job opportunities and future earnings for participants. Many successful programmes share the characteristic of being targeted on specific client categories, or are designed to address specific labor market problems. This applies in particular to training programmes. Less favourable results have been reported for certain broadly targeted programs, such as training programs offered to all the unemployed. Some of the most consistently positive results have been reported for intensified placement and counselling efforts aimed at encouraging effective job search by the unemployed. Such measures have proved to be very cost-effective, especially when targeted on particular groups, such as workers affected by plant closures or the long-term unemployed." OECD Employment Outlook, 1993, page x. Retraining Programs in OECD Countries 9 What lessons can transition and other restructuring economies draw from this summary of OECD experience? The evidence is rather sketchy, so firm conclusions are difficult. Nevertheless, the following conclusions reflect a reasonable interpretation of the findings: First, it is useful to evaluate retraining (and other) public interventions using sound techniques. It is instructive that while the non-scientific evaluations of retraining programs present a rosy picture based on placement rates and other informal evidence, scientific evaluations are quite discouraging. Relying on non-scientific evaluations may lead countries to incorrect policy conclusions. Second, rigorous evaluations, while not necessarily allowing a complete social cost-benefit analysis, can be useful guides for policymakers in solving public expenditure issues related to labor programs. Reviews of rigorous evaluations like this study find, for example, that job search assistance measures - which cost less than retraining but appear equally effective - may be a relatively cost-effective device in assisting displaced workers. The absence of such evaluations may result in inefficient allocation of government funds. Third, OECD experience of retraining programs for workers displaced en masse may be useful in designing assistance programs in countries with an impending labor shedding from large enterprises in the manufacturing and mining sectors, such as the economies of the former Soviet Union: principally, this involves recognizing that retraining should not be the main form of assistance. The absence of experimental evaluations of retraining programs for such workers in OECD countries also has implications for evaluation design in other countries facing similar problems, since it points to quasi-experimental evaluations as being more feasible. Finally, East and Central European economies which are beginning to experience long-term unemployment can learn from OECD experience with retraining programs for the long-term unemployed, which indicates that retraining programs are more beneficial for some groups than for others within this relatively heterogeneous group of job-seekers. However, it is difficult to predict a priori who will benefit most from retraining: principally, these results call for using modest pilot programs, evaluating them rigorously, and then tightly targeting retraining to those for whom it is found most cost-effective. Retraining Programs in OECD Countries 10 V. REFERENCES Addison, J. T. and W. S. Siebert, 1994. "Vocational Training and the European Community", Oxford Economic Papers (Vol. 46, No. 4). Addison, J.T. and M. Blackburn, 1994. "Policy Watch: The Worker Adjustment and Retraining Notification Act", The Journal of Economics Perspectives (Vol. 8 No. 1). Alfthan, T. and B. Janzon, 1994. "Retraining Adult Workers in Sweden", Training Policy Study No. 3, ILO. Bjorklund, A., R. Haveman, R. Hollister and B. Holmlund, 1991. "Labor Market Policy and Unemployment Insurance", Clarendon Press, Oxford. Bosch, G., 1992. "Retraining - not Redundancy", Intemational Institute for Labor Studies, Geneva. Boesel, D. and L. McFarland, 1994. "National Assessment of Vocational Education: Summary and Recommendationis (Vol. 1)", Office of Education Research and Improvement, U.S. department of Education. Chalmers, N., 1994. "Retraining under Conditions of Restructuring: Japan", Training Policy Study No. 6, ILO. Clements, N., J. Heckman and J. Smith, 1994. "Making the Most Out of Social Experiments: Reducing the Intrinsic Uncertainty in Evidence From Randomized Trials With an Application to the National JTPA Experiment", NBER Technical Working Paper, No. 149. Corson, W., P. Decker, P. Gleason and W. Nicholson, 1993. "International Trade and Worker Dislocation: Evaluation of the Trade Adjustment Assistance Program", Mathematica Policy Research, Inc.. Couch, K., 1992. "New Evidence on the Long-Term Effects of Employment Training Programs", Journal of Labor Economics (Vol. 10, No. 4). Creticos, P. and R. Sheets, 1992. "Evaluating State-Financed, Workplace Based Retraining Programs: Case Studies of Retraining Projects", National Commision for Employment Policy. Doolittle, F. et al., 1993. "A Summary of the Design and Implemenitation of thte Natienal JTPA Study", Manpower Demonstration research Corporation. Fong, P. and L. Kiat, 1994. "Industrial Restructuring and Retraining if Singapore", Training Policy Study No. 2, ILO. Hecknian, J., 1992. "Randoniization and Social Policy Evaluation", in C. Manski and I. Garfinkel, eds.. "Evaluating Welfare and Training Programs". llarvard University Press. Retraining Programs in OECD Countries 11 Heckman, J., Roselius, R., and J. Smith, 1994. "U.S. Education and Training Policy: A Reevaluation of the Underlying Assumptions Behind the "New Consensus", Draft. Hechman, J., Smith, J. and C. Taber, 1994. "Accounting for Dropouts in Evaluations of Social Experiments", Draft. Hechman, J. and R. Roselius, 1994. "Evaluating the Impact of Training on the Earnings and Labor Force Status of Young Women: Better Data Help A Lot", Draft. ILO, 1992. "Training in the Iron and Steel Industry", Iron and Steel Comrnmittee, Twelfth Session. ILO, 1992. "Skill Requirements, Training and Retraining in the Building, Civil Engineering and Public Works Industries", Building. Civil Engineering and Public Works Committee, Tewlfth Session. ILO, 1994. "Consequences of Structural Adjustment for Employment, Training, Further Training and Retraining in the Metal Trades", Metal Trades Committee. Thirteenth Session. Johanson, R., 1994. "Retraining Adults in Gerrmany", Training Policy Study No. 4, ILO. Johanson, R., 1994. "Ireland: Adult Training and Retraining". Training Policy Study No. 5, ILO. Johanson, R., 1995. "Retraining Workers Displaced by Structural Adjustment", Draft for World Labor Report. LaLonde, R.J., 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data", American Economic Review (Vol. 76, No. 4). Leigh, D., 1992. "Retraining Displaced Workers: What can Developing Nations Learn From OECD Nations?", WPS 946, The World Bank. Leigh, D., 1994. "Retraining Displaced Workers: The U.S. Experience", Training Policy Study No. 1, ILO. Middleton, J., A. Ziderman and A.V. Adams, 1991. "Vocational and Technical Education and Training", World Bank. Nord, S. and Ting, Y., 1991. "The Impact of Advance Notice of Plant Closings on Earnings nad the Probability of Unemploymenit", Industrial and Labor Relations Review (Vol. 44, No. 4). OECD, 1991. "Evaluating Labor Market and Social Programs: The State of a Complex Art". OECD, 1993. "Partnerships: The Key to Job Creation. Experiences from OECD Countries". OECD, 1993. "Industry Training in Australia, Sweden and the United States. OECD, 1993. Employment Outlook. Chapter 2. Retraining Programs in OECD Countries 12 OECD, 1994. "Vocational Training in the Netherlands: Reform and Innovation". OECD, 1994. "Vocational Training in Germany: Modernization and Responsiveness". OECD, 1994. The OECD Jobs Study: Evidence and Explanations. Paris. Oliveira, J. and Lau, A., 1992. "Retraining the European Workforce: How Technologies Can Help", Training Discussion Paper No. 95, Training Policy Branch, ILO. Orr, L. et al., 1994. "The National JTPA Study: Impacts, Benefits, and Costs of Title II-A", Abt Associates, Maryland. Penn, R., M. Rose and J. Rubery (eds.), 1994. "Skill and Occupational Change", Oxford University Press. Rainbird, H., 1992. "The Effectiveness of Training Boards: A Case Study of the United Kingdom", Training Discussion Paper No. 99, Training Policy Branch, ILO. Ruhim, C., 1994. "Advance Notice, Job Search and Postdisplacement Earnings", Journal of Labor Economics (Vol. 12, No. 1). Rutkowski, M., 1995. "Workers in Transition Economies", Background Paper, World Development Report 1995. Steinberg, D. and F. Monforte, 1987. "Estimating the Effects of Job Search assistance and Training Programs on the Unemployment Durations of Displaced Workers" in K. Lang and J. Leonard (eds.) "Unemployment and the Structure of Labor Markets", Basil Blackwell. World Bank, 1995. World Development Report: Workers in an Integrating World. Vocational Training: European Journal, 1994. "Training and the Labor Market", Vol. 2. Zweimuller, J. and R. Winter-Ebrner, 1991. "Manpower Training Programs and Employment Stability", Working Paper, No. 9105, Johannes Kepler University Linz. Annex: Overview of Evaluation Studies Labor NMarket Problem Relevant Indicators Intervention Design Type of Result Comments Evaluation Displaced Workers Ford Plant Closure in San Jose, A 25 percent decline in auto Basic skills training as well as targeted Non-scienififc High success rate in placement. No scientific evaluation U S. (1982). 2400 workers to production between 1978 and 1980. vocational training in marketable skills. done. Causes for perceived Iose their jobs. Unemployment rates tn the U.S. success were - adequate rose from 7.5 percent tn 1981 to resource base ($6000 9.5 percent in 1982 and 198X. grantlworker); high degree Simultaneously inanufacturitig of coorditation atid employment dechiied by S percent assistance provided by Ford, during ile time pcliod. UAW and goveritment. Shipyard closure in Storstrom Higlh unemployment rates in Program in tratinig woomen .tnirepreneurs Non-scientific 51 busiiiesses were set up by 1989. No scientific evaliation county, D1eniark (1986). 2001) rcgions, especially among womiienl t1986-89) to help themit start their own Less than a third ot participants done. This program does not people lose jobs. General National unemploymenit aries for bustiess. A total of 200 hours of opened up a fuli-tnitie business and seem to have met winh much econiolitic deciline. meii in 1986 and 1987 were 6.1 introductory and specitic business few hired any entployees, dius success and may have been atid 6.4 percent respectively while oriented training were provided. generating low additional quite costly (no cost the corresponding nunibers for etuployilsent. information available). women were 10.0 and 9.6 percent. Employment in manufacturing remained stagnant sitce the mid 1980s. Steel and Coal plant closure in Contractitig steel sector, with over Workers to receive 70 percent of their Non scientific High success rate in placement. No scientific evaluation done Creusot-Loire, Franice (1984). 6000job losses over the previous torimer salaries for (l months dunng ott cost-effectiveness and 1230 people lost jobs almnst few months. In France which they participate in retraimnig and long term employment immediately. unemployment rates rose trom 8.1 job-search activities after which they impact of retraining. percent in 1982 to 10.2 percent by were promised reemployment in different 1985. In mining industry firnis in region. Training provided in employment has fallen steadily engineering courses, plastic molding, throughout the 1980s - by close to refrigeration etc.. Also short courses on 40 percenit between 1980 and 1990. work skills, production methods in small Manufacturing employment fell by firms, job search and career counselling. about 3 percent between 1983 and Financial incentives were provided to 1984 and a further 3 percent in the firmis to hire these workers. next year. 13 Closure of Pulp plant in | Unemployment rates in Sweden Comparison of Participation in training gives lower No information provided on Kiamfors in Northern Sweden rose front 1.6 percent in 1976 to retrained weekly wages than those niot reemployment rates or costs. (1977). 2.2 percent by 1978. Employment workers with receiving training upon being Benefits from retraining in manufacturing declined by 4 employed and employed. Drop in eamings are program were insignificant. percent in 1977 attd a further 3 unemployed especially significattt in the first year After factoring in costs, percetnt the following year. workers. with no appreciable long-tenn gains. training may yield negative retums. Mass l ayoffs (around 3000) A 25 percent declitte in auto Downriver Project. Workers provided a Quasi- Classroom training (CT) did not Eamings estimates vary &Je o1, auto and aoto parts plant produictioni between 1978 and 1980. mixture of job search assistance and experimental significatitly improve program (ranging frotm negative to closures in Michigan (1980-83). Iligh unemployment rates in classroom training. Services were panicipants post-program significantly positive). The general profile of those 1981/82 nation ally (9.5 percent). provided promptly after plant closures. reemploynment rate. Training does tot seem to laid ofl was experien cd blue They h.ad fallen to 7.5 percent by Retraining was provided in occupations have been very effective, collar male workers who eamed 1984. Manufacttiritig employment in which there was perceived to be especially in light of fact that high wages. rose by 5 percent between 1983 and growing demand. Training curricula training cost twice as much 1984. provided instructions in blue collar as JSA. trades. Mlass L.ayoffs due to auto and( High uneniploynient rates in Buffalo Project. Displaced workers were Quasi- ISA only services are found to have Classrooni training atid OJT stcel plant closures in Buiffalo 1981/82 rationally (9.5 percent). provided with a itiixture of job search experimental a fairly large impact oii earnings are ineffective. Cost of CT 11982-83). Thc gen)eral protle IThey had lallen to 7.5 percent by assistance and either classroomil or on the measured over the first six posL- and OJTr programs are of thise laid off was 1984. Manufacturing employment job traitiing. Program services were program months. llowever there is around four times as tmuch as experienced blue collar ntale rose by 5 percent betwseen 1983 atid provided after a fairly lengthy period of no evidence that there is any JSA. implying ihat JSA is workers who eamred high 1)84. post-layoff unemployment. increiental effect above that of ISA potentially the otily cost wages. for either classroom training or OJT. effective program. No evidence is provided about employability of programn participants. Plant closures at 13 steel Job search assistance and trainitig Quasi- ILikelihood of worker at supported Training seems to have a factories aiiti miues in Canada provided to these workers. Experimental sites having a job was seven percent greater impact than the job higher than comparably displaced counselling offer, but no workers not in the progr-amn. This informationi on costs. Job impact was attributed to training counselling has linle impact which was used by 28 percent of in Canada as targeted job workers. However at two mining counselling is provided by sites, the program hiad no impact. Canada's Public Employment Service as a standard service to all unemployed. 14 445 workers laid off doe to Unenplonyment rates rose steadily Labor Adjustment Tratinng Arrangement. Quasi- Over a itine month period, driver No indication of cost- autoniobile plant closure tn in Australia reachiing a Iiigii of 9 Provision of classroom training (average experimental training increases the probability of effectiveness of different type Australia (1984). percent in 1984 before faling to 8 length 19 weeks) to meet retraining needs reemployment. However, other of training courses. Self- percent for dlie nexi few years. of workers. Main distinction between training courses resulted it a decline selection problens as Between 1980 and 1984 courses was whether they provided driver in reemployment probabilities. individuals who chose not to employnietit in manufacturing training or not. participate are in the control shrunLk by 4 percent. group. ltnpact of longer training courses is biased downwards (as there is less time to look for job). .Aruind 21)0O workers laid off j Local economy was giveti a boost A sigiiificant number of workers joined Non Scientific By November 1987, over 90 percent No scientific evaluationi of at the Ud.levalla shipyaFd In by decision ot Volvo to establish a retraining programs several mottths prior of the workers who had completed retraining. Two major Swedlen rin 1985. car mainufacturing plant at to beiisg laid-off. Courses of varying trainitig had found jobs or become factors accounted for the Uddevalla Unemployment rates duration were offered in weldiiig, self-eniplnyed - most of them in success of the training were decltning in ihis period while etigineering and contol eiigiiieering. occupations they had trained for. prograim - econiomy and nianufacturing employment was These retraining programs were provided labor market conditions were tairly steady. by the state owned training board, buoyani in the region municipal education institutions and other throughout the phase out adult education instirsitions. period; shipyard management, employment offices and rraining agencies worked in close cooperation. [Ihe Volvo plant at Goteborg, Volvo planned to recruit 800 Retraining program was proposed which None Program costs are expected Swedcn planned to lay o,ff 1)00) workers to prepare for the would help the existing woikfoce to to be about $25 niillion woi kers is 1992 associatcd with production of the itew line of cars. manage the change without job loss. The ($25000 per person the phising out ol an older car Unemploymiient rates were rising conipaity accepted the proposal under the i-etrained). The government model withi a new model shaprly - frons 3.2 perceiit In 1991 stipulatioti that die cost be shaied by the contribution is about $8 to 5.9 percent in 1992. government. The program is a broad million. In judging the cost- Manufacmnung employment dropped competency raising program \which effectiveness of this program, by 9 percet in 1992. Total includes specific traitting to prepare die it should be compared with employmetit also fell by four participanits for die production of the new the expected unetiiploynient percent. automobiles. benefits of $6.5 million that the governmient would have to pay. Sweden: 1980s and 1990s: Uttetnployment rising steadily over Various types NA Retraining programs have beconie Cost-effectiveness of training general evaluation of public the period of study more ineffective over time - programs has declined retraining programs especially since the economny has because effectiveness has begun deteriorating. Participants fallen and costs have have greater difficulty in finding jobs increased. than the openly unemployed. 15 Lonig-Term UnIemployed Improve skills and thus provide Unemployment rates have begun Training provides hands on experience Non-scientific In 1991, 82 individuals were enrolled No scientific evaluation better access to jobs for long- falling in Holland since the nsid through a simulated workshop. in the course. 52 completed and 41 done. Training quite term unemployed in Tilburg, 1980's and were aroung 7 percent Emphasis is placed on technical skill went onto a job or further education. expensive as annual funding Netherlands (Since early in 1992. development as well as well as instilling Metal works program is the most of the program is about 1980's). good work habits. Four main skill areas successful in placing students. $7500000 ($10000 per are: metal training, wood training, trainee). installation techniques and clothing. Courses run froti 4-10 months. Tackle high regional In 23 of the 142 regions in Among other interventions, training and Non- Training reduced unemployment In light of the extremely high unemploynient levels in Germany unemployment levels retraining of these labor force experimental somewhat (oiis result is significant at cost of training (around $50)( Germany (1979). were above 6 percent. Some firns participants in firms. The firms who the 10 percent level). Ilowever it is millioit for training and oilier in this region were also facing employed these workers received a wage cstimated that by 1981, over 40 interventions), results are serious problems in adjusting to subsidy of 90 percent of wages for 24 percent of these haid-to place very disappointing. No data economic changes. months. individuals had already left their oni wages. jobs. TIrain workers eligible for A declitie in petrochemical industry Tex:as WAD project. Displaced workers Experimental By enid of first year after Despite high costs of JPTA Title Ill program led to layoff of workers. In the provided with Job seatch assistatice or a participation, no additional eamtting classroom training (twice as (displaced worker) in Houstan. U.S., unemploynsent rates declined mixture of job search assisiance with gain accrued from classroomi training much as 1SA) no additional Texas (1983-85). Individuals front 9.5 percent in 1983 to 7.4 classrooiti training as conipared to the JSA only. gains accrued front this type eligible were Ul recipiemiis (or percent in 1985 (and maintained of training. exhaustees) who had a low this trend till die late 1980s). probabiliry of rentiming to their Emoploynient in this industry previous occupation or increased slightly betweeni 1983 and industry. 1985. Train workers elivible for PIA In the lJ.S., unemployment rates Texas WAD project. Displaced workers Fxperimental By end of first year after Increase in eanmings for Title Ill program (displaced declined front 9.5 perceint in 1983 provided with Job search assistance or a participation, while this program had womeit exceeded program worker) in El Paso, Texas to 7.4 percent in 1985 (and mixnure of job seatrch assistancc with no effect ott tiiale eantings it had a costs slightly. However nto (1983-85). (Criteria satise as maintained this trend till the late classroom traimming. pernnament effect on increasing beneficial effects for males. above). Workers laid off from 1980s). Employment in eamings for wotiieit. light manufacruring plants. manufacturing fell by close to 2 percent berween 1982 and 1983 but rose by about 5 percent till 1985. 16 Retraining Programs in OECD Coutities 17 Job se--ch assistance and Unemployment rates in the U.S. New Jersey Ul Reemployment Experimental Focussing only on those who While results of training are iraining provided for the long fell from 7.1 percent in 1985 to 6.1 Demonstration project. Displaced undertook training, over 10 quarters positive the following caveats term unemployed in New percent by 1987. While workers provided with a mixture of ISA, after the program, both CT and OJT apply - claimants receiving Jersey (1986-87). In general, employment in manufacturing JSA followed by on-the-job training significantly increase eamings training are self-selected the unemployed were laid off remained fairly steady between (OJT), JSA followed by classroom relative to JSA recipients only. (these results may not apply from nanufacturing, trade and 1985 and 1988, trade employment training (CT) or JSA followed by These individuals are also employed for a random group of services increased by 7 percent and that in reemployment bonus. for greater amounts of time/quarter claimants); only 15 percent services by about 5 percent over as compared to the JSA-only group. of those offered training this time period. accepted it; training seems to mainly benefit those who already possessed marketable skills. Finally, both in the short and long run, cost- benefit analysis shows that costs exceed benefits. Increase the employability and As compared to the U.S. Job Development Program. Provision of Quasi- Employability of women goes up In view of high program cost eamings of the long term unemployment rates have been formal and on-the-job trainiing to experimental significantly while that of males does (around $9300/participant) unemployed in Canada by fairly high in Canada. However unemployed. Wages and direct cost of not (in fact it declines). Weekly training is not cost effective, providing them with training they declined from 11.2 percent in classroom training are subsidized. The eamings for females is insignificant especially for men. (1985-). the mid 1980s to 7.5 percent in wage subsidy helps employers cover the relative to control group while it is 1989. Spurred by a growth of cost of on the job training. negative for males. commerce and services over employment grew by about 3 percent annually over the time period. Reirain individuals who are Training individuals to be certifiable in Non-scientific Results measured in terms of No scientific analysis. No either at tbe risk of becoming one of the 375 apprenticeable trades. retention rates (those who complete data on cost available. unemployed or those who are This takes upto two years. Contents and the course); pass rate (those who Dropout from employment unemployed in Germany. specification of retraining correspond to pass exam); and employment rate was fairly high - two years those of initial vocational training. (those who find jobs in occupations after completing training for which they are retrained). These only 60 percent of men and rates are 70%, 85% and 90% 66 percent of women were respectively. Thus the overall still employed. success rate is 54 percent. Employmeiit rate varied by age - for those over 45 unemployed over a year it was below 50 percent while for those 25-35 it was 86 percent. 17 Same as above in Germany Unemployment rates were steady in Same as above. This is an evaluation Of Econometric No type of training has any No information on wages or (1987-88), Gennany between 1986 and 1988. four programs, two of which are training study of lahor significant impact on the tlows out of costs of training is available. They fell somewhat by 1990. (one offering turdier training for flows short-term or long-term Bellmann and Lehmann Eniploymsent grew at slightly over I employed and uneinployed individuals unemployment nor on the flows into (1990). percent during this period. while the other offered retraining for the uneniployment. untemployed). Retrain displaced workers in Unetiiploynient rates in the U.S. Trade Adjustment Assistance Program. Quasi- Individuals who received training Longer-term ittvestments in the U.S. to equip them witls fell fromi 7.1 percent it 1985 to 6.1 Training program intended to develop experitmcntal begin earning significantly more (as training may be effective in skills to enter a new occupation pcrcent by 1987. While skills in occuipationis differenit fromt compared to those who received increasing eamings. or industry (1988). NMost employment in manufacturing occupations in pre-displacememt jobs. extended income-maintenance However the training is workers have been laid off due remained fairly steady between Most of these skills were supplied by a benefits) by tde 6th quarter and this costly (each trainee was to a plant closing. 1985 and 1988, trade employmenit vocational college or local community difference continues to increase till given a $12000 training increased by 7 percent anid tiat in college in courses tiat were over a year the 12th (last) quarter reaching a voucher). This analysis has services by about 5 percent over long. level of $500. only been done for the this time period. manufacturing ittdustry. HIelp long term unemployed High unemrploynment rates (around Coninmunity Progratm. Targeted Little inipact on flows frotm long- Cost-effectiveness of these and displaced workers gain 10-II percent) persisted through vocational training to be provided by term unensploymetnt. (Similar programs is likely to be enmployment in England much of the 1980s. Berween 1983 local authorities in conjunction with local programs in germany have also been negative. (1980's). and 1990 employnient has risen by colleges, central govermment and largely unsuccessful). aroutnd 1.5 percent. voluntary organizations. Low private sector involvement. Help adult unemployed and Ceiiters for Adult Vocational Training. Quasi- Unemployed persons who did not While no data is available on displaced workers gain better Experimental undergo training found jobs as wages and costs, these employment opportunities quickly as those who did - if training programs are not likely to be through access to training in duration was counted as search time. cost-effective. (OECD, Holland. However, even two years down the 1993). line the employment situation for the two groups was not significantly different. 18