SP DISCUSSION PAPER NO.9901 20116 Active Labor Market Programs: A Review of the Evidence from Evaluations Amit Dar Zafiris Tzannatos January 1999 , x/ Protnton LABOR MARKETS, PENSIONS, SOCIAL ASSISTANE E T H E W O R L D B A N K ACTIVE LABOR MARKET PROGRAMS: A REVIEW OF THE EVIDENCE FROM EVALUATIONS Amit Dar Zafiris Tzannatos* January 1999 * Social Protection Department, Humnan Development Network, The World Bank. We would like to thank Alan Abrahart, Arvil Van Adams, Peter Auer, Gordon Betchernan, Duncan Campbell, David Fretwell, Inderrnit Gill, Robert Holzmann, Arvo Kuddo, Chris O'Leary, Funko Park, Nicholas Prescott, Makoto Ogawa and Gin Wong for their commnents and suggestions. Earlier versions of this paper were presented at and received useful comments from seminar participants at the "International Conference on Active Labor Programs Design and Impact" in Antalya, Turkey in June 1998, and at the "International Workshop on Employment Policies for Economic Adjustment: International Experience and Korean Prospects" in Seoul, South Korea in July 1998. The views expressed here are solely those of the authors and should not be attributed to the World Bank or the countries they represent. ACTIVE LABOR MARKET PROGRAMS: A REVIEW OF THE EVIDENCE FROM EVALUATIONS TABLE OF CONTENTS PAGE Foreword Summary ii CHAPTER I Introduction 2 What are Active Programs 3 3 Expenditures on Active Labor Market Programs 8 4 The Importance of Good Evaluations 11 A. Impact Evaluation Techniques 12 B. The Importance of Considering Costs 15 5 An Interpretation of Evaluation Results 17 A. Public works Programs 17 B. Job Search Assistance 19 C. Training Programs 21 D. Micro-Enterprise Development (Self-Employment) Schemes 25 E. Wages/Employment subsidies 28 6 Conclusions 32 7 References 35 LIST OF TABLES AND BOXES Table 2.1: Active and Passive Labor Market Programs: Some Key Features 5 Table 3.1: Public Expenditures on Active Labor Market Programs in OECD 8 Countries (As % of GDP) Table 3.2: Expenditures on Various Active Programs as a % of Total Active 9 Expenditure (Selected OECD Countries 1995/96) Table 3.3: Distribution of Active Labor Market Components by Region (FY92-FY96) 10 Box 4.1: Main Types of Evaluations 11 Box 4.2: Some Commonly Used Terms in the Impact Evaluation Literature 13 Box 4.3: Statistically Significant an Economically Significant?: An Example 16 Table 5.1: Overall Impact of Public Service/Community Employment 18 Programs in Transition Economies Table 5.2: Annual Cost of Job Creation in Public Works 18 Box 5.1: What Makes a Public Works in a Low-Income Country? 19 Box 5.2: The U.S. Job Training Partnership Act Evaluation 23 Table 5.3: Failure Rates of MEDA Businesses 27 Box 5.3: Quick Reemployment Bonuses: An Alternative to WES? 29 Table 5.4: Effectiveness of Wage Subsidy Programs 30 Table 6.1: Evaluations of Active Labor Market Programs 32 Table 6.2: Overview of Active Labor Programs 33 LIST OF APPENDICES Appendix 5.1: Public Work Schemes 40 Appendix 5.2: Job Search Assistance 43 Appendix 5.3: Training for the Long-Term Unemployed 48 Appendix 5.4: Retraining in Cases of Plant Closures & Mass Layoffs 56 Appendix 5.5: Training Programs Geared Towards Youth 60 Appendix 5.6: Micro-Enterprise Development 62 Appendix 5.7: Wage/Employment Subsidies 66 Foreword Active labor market programs have been widely implemented in OECD countries for over two decades and over the past few years the Bank has become increasingly involved in providing assistance to client countries to design such programs. Inspite of their widespread use, there are polar positions both within the Bank and among policymakers and researchers on the effectiveness of active labor market programs - on the one hand, proponents of these programs argue that active labor market programs are both necessary and useful, short only of a panacea for reducing unemployment and protecting workers, while opponents contend that these programs as a waste of public money with high opportunity costs. Part of this divergence of views arises from the fact that these programs have rarely been rigorously evaluated, and, until recently, the evidence from evaluations had not been studied in a systematic manner. Building up on studies does by the OECD and the ILO over the past few years, this paper contributes to the discussion by synthesizing the findings of these evaluations - over 100 studies - and attempting to draw some best practice lessons. This paper justifies the importance of doing a rigorous evaluation to examine the impact and cost- effectiveness of active labor market programs. Based on a thorough evaluation of evidence, it also shows that while some active labor market programs can be useful to some workers, their effectiveness depends not only on their design but also on the overall macro and labor market framework within which they operate. Robert Holzmann Director, HDNSP Summary Many interventions in the formal labor market are clustered under the title "active labor market programs" (ALMP). Such programs may lead to direct job creation (through additional jobs offered by a new public works scheme), help the unemployed fill existing vacancies (through re-training to meet the new job requirements), or improve the functioning of the labor market (through employment information and labor offices). The analytics of these programs vary considerably; for example, public works is very much a demand side intervention, training a supply side one, while labor market intermediation can be seen as an attempt to bridge these two sides of the labor market. Active programs vary in their aims. Some programs emphasize efficiency; for example, more information leads to better job matching. Others concern distributional aspects. For example, public works can be targeted to specific areas particularly hit by poverty. Yet others can be introduced or maintained based on political considerations; for example, retraining is offered to some groups of dismissed workers, while the already unemployed could have filled these jobs. These programs rest on the assumption that, for one reason or another, some market failure exists in the labor market or in other markets (for instance, existence of monopolies in product markets). Some also rest on the premise that certain market outcomes are socially unacceptable (as is the case with high unemployment leading to social unrest). Some people would argue, however, that the term "labor market program;' is a contradiction in itself: if the market works, no program should be required. The policy emphasis instead should instead be on making markets work. The theoretical debate on the need for active and passive programs is bound to continue, depending on the values and assumptions adopted by economists. However, given that many countries do implement these programs, a more pragmatic approach is not whether to have them, but whether the intended objective ("benefit") is met, and at what cost. Empirical evidence from evaluations of active programs is, in this respect, indispensable. In this paper, we survey evidence based on the evaluation of active labor market programs. We have examined about 100 evaluations. Many of these studies have already been summarized by others (such as OECD and ILO), but we have also included a significant number of individual studies. And though most studies apply to OECD countries - mainly the U.S., Canada, U.K., Sweden and Germany - we have added inforrnation on developing and transition economies such as Hungary, Poland, the Czech Republic, Turkey and Mexico. While it can be argued that the lessons from developed countries on the effectiveness of these programs may not be directly applicable to developing countries, it is unlikely that these programs will be more successful in developing countries given the scarcity of administrative capacity to implement these programs and the paucity of monitoring and evaluation experience to study their effectiveness. The studies we summarize are recent, and have taken advantage of the advances made in model development and econometric analysis. However, a number of issues affecting the reliability of the findings of these studies for guiding public policy remain open. First, there are unresolved technical issues, such as handling selection bias and assessing deadweight and displacement/substitution effects. Second, there are a variety of data problems in the specific surveys. These include benchmarking pre- intervention profiles (employment history, human capital attributes, etc.) and the tracking of participants and non-participants for no more than one or at most two years while, in many cases, the full impact of policies is unlikely to plav out in this short period of time (such as in the case of training and self- ii employment). Third, administrative data which may be called upon to provide supplementary information tend to be surprisingly poor, so that the nature or the intensity of the intervention received by the participant is often uncertain. While these remarks indicate that a definitive conclusion on which and under what conditions ALMP can be justified economically, the evidence points to some generalizations about active labor programs. We summarize them programmatically: * Public works can help the more disadvantaged groups (older workers, the long-term unemployed, those in distressed regions) as a poverty/safety net program. They are ineffective instruments as an escape route from permanent unemployment. Program participants are less likely to be employed in an unsubsidized job, and they earn less than individuals in the control group. * Job search assistance has positive impact and is usually cost-effective relative to other ALMPs. Programs that have yielded positive results have generally been implemented under favorable macro- economic conditions. However, job search assistance does not seem to significantly improve either the employment prospects or wages of youth. * Training for the long-term unemployed can help when the economy is improving. Small-scale, tightly targeted on-the-job training programs, often aimed at women and older groups, offer the best returns. However, the cost-effectiveness of these programs is generally disappointing. The real rate of return is rarely positive, and they are no more successful than job search assistance programs in terms of post-program placement and wages. A caveat here is that job search assistance may not be a direct substitute for training as it may cater to different groups of the unemployed. - Retraining for those laid off en masse usually has little positive impact and, as in case for the long-term unemployed, it is more expensive and no more effective than job-search assistance. Again, job search assistance may not be a direct substitute for retraining, as the target groups may be somewhat different. X Training for youth generally has no positive impact on employment prospects or post-training earnin,s - it clearly cannot make up for the failures of the education system. Taking costs into account, the real rate of return of these programs in both the short- and long-run is usually negative. - Micro-enterprise development programs are usually taken up by only a small fraction of the unemployed and are associated with high deadweight and displacement effects. The failure rate of these businesses is quite high. As in the case of training for the long-term unemployed, assistance targeted at particular groups - in this case, women and older individuals - seems to have a greater likelihood of success. * Wage subsidy programs are unlikely to have a positive impact. They have substantial deadweight and substitution effects, and the wage and employment outcomes of participants are also generally negative as compared to a control group. Careful targeting can reduce, but not eliminate, substitution and deadweight effects, and further controls may be necessaryv to ensure that firms do not misuse this program as a permanent subsidy program. There are polar positions on the effectiveness of active labor market programs. On one hand, proponents of these programs argue that active labor market programs are both necessary and useful, short only of a panacea for reducing unemployment and protecting workers. Opponents of the programs tend to summarily dismiss these programs as a waste of public money with high opportunity costs to other social programs and labor market efficiency as a whole. Based on a thorough evaluation of evidence, this paper shows that some programs can be useful to some workers in some cases. There are also good design features for each program, but external (to the programs) conditions need to be taken into account (a good program in one country can prove to be a bad one for another; a program found to iii be useful in the past may no longer be the case). This calls for realism in setting the objectives of ALMP and also setting standards against which active labor market programs should be evaluated. However, due to lack of evaluative evidence, the conditions under which programs will succeed have not been fully identified. A very broad generalization on the effectiveness of these programs leads to the conclusions that: * Some of these programs - such as wage subsidies or training for youth - are unlikely to be cost- effective instruments in reducing unemployment. * Some programs - such as job search assistance - are likely to have positive impacts on the probability of finding employment if they are well-designed and implemented. * However, the impact and cost-effectiveness of most of the active labor market programs depends not only on their design, but also on the overall macro and labor market framework in which they are designed. This assessment dictates the following policy approach: * If a country is going to institute labor market programs, a good practice is to start with modest programs. * Sound impact evaluation techniques should be used to evaluate the instituted programs. Relying only on non-scientific evaluations may lead to incorrect policy conclusions. A good micro evaluation will involve comparing labor market outcomes for individuals who have gone through a particular program with those of a control group of their peers and will also utilize data on program costs. These will help to answer the important questions: (a) what is the impact of the program?; (b) are the impacts large and costs low enough to yield net social gains?; and (c) is this the best outcome that could have been achieved for the money spent? * Based on these evaluations, the programs should be tightly targeted at those for whom they are found to be the most cost-effective, or, if the evaluations point towards these programs being ineffective, they should be amended or discarded. iv ACTIVE LABOR MARKET PROGRAMS: A REVIEW OF THE EVIDENCE FROM EVALUATIONS I. INTRODUCTION This paper provides a review of evaluations of active labor market programs (ALMPs). The empirical literature on evaluations of ALMPs is vast and often provides contradictory conclusions depending on country, time period and specific program characteristics under consideration. The aim of this review is to provide a summary of these evaluations and attempt to draw some best practice lessons. Most of the evaluations come from industrialized countries, though ALMPs are also found in developing ones. International organizations, specialized research institutes and individual researchers have undertaken the evaluations. Most are in-depth studies of one policy in a particular country (especially the US, Canada and the UK) at a time, though some compare programs in a cross-country context. Data problems, and the inability to track participants over a reasonably long time period after they graduate from the program, account for the variety of the results of ALMP evaluations in developing countries. In addition, administrative data which may be called upon to provide supplementary information are often surprisingly poor, so the evaluator is not sure about the nature or the intensity of the intervention the participant has actually received. ALMPs are effectively an intervention: they are usually funded by public funds and payroll taxes and aim to increase the allocative function of the labor market (such as in the case of job search assistance), direct the distributional outcomes of labor markets (as in the case of targeting the poor in public works programs), or increase the political acceptability of reforms (for example, retraining may be provided to retrenched workers to enable them to compete for other jobs). Alternatively, ALMPs can be thought of as an instrument to correct some market failure (for example, to address a credit market failure in the case of micro-enterprises) or, even when such a failure does not exist, to divert an "economically efficient" outcome to a "socially desirable" one. In the latter case, political economy considerations are important. For example, if unemployment is deemed "unacceptably" high, then an intervention to reduce it in one way or another may be considered appropriate, even though it may create some other inefficiencies in the labor market or in other markets. Often these social and political considerations dictate which programs are to be offered. Evaluating ALMPs is, therefore, a complicated task, as they often address multiple objectives, for many of which there are significant trade-offs. This can be the case between efficiency and distribution objectives, or between those two objectives and political economy considerations. Therefore, an evaluation of an ALMP needs to take into account the context and the objectives of the program. Additional complications arise because of measurement problems. Though the direct financial cost of a program is usually known, its indirect (opportunity) costs are typically unknown. For example, the opportunity cost of re-training unemployed workers includes the loss of opportunities for offering pre- employment training to first time job seekers. Similarly, what appears as a benefit to a program participant may be offset by a corresponding loss among non-participants. For example, a wage subsidy can increase the employability of a dismissed worker. However, he may take up a job currently filled by a non-subsidized worker. Notwithstanding difficulties in the conceptualization and measurement of the effects of ALMPs, there have been many studies evaluating some or all aspects of these programs, such as their costs, effectiveness, efficacy, net impact, efficiency, sustainability as well as issues of program implementation. These aspects can be highlighted as follows: a job created through a public works program, say, costs $5,000 for one year and creates 3,000 jobs, all of which were taken up by newly unemployed. In terms of efficacy (the relationship between project objective and outcome), the project may have "failed," in the sense that it originally aimed primarily at the long-term unemployed poor. The net impact might have been small or large depending on how many of the 3,000 newly unemployed would have found a job had the program not been in place - if all 3,000 would have found a job of similar duration, earnings and so forth, the net impact would be zero (alternatively stated - the program has a 100% deadweight loss). To measure efficiency, one needs to know the net impact and compare it to total (direct and indirect) costs of the program. One can also add here the notion of "relative efficiency": even if this public works program is found to have the desired effect, it may not be worth pursuing if there are alternative interventions that have higher social returns. The preceding discussion shows that ALMPs are not "something for nothing": they can be useful but they also have costs. In a period of scarce resources for social services (broadly defined to include health, education, pensions and assistance to the poor in addition to unemployed workers), it is crucial to rigorously assess whether these programs have been successful in generating employment and gains in earnings for those who participate in them in an efficient way. Hence, evaluating them can provide guidance about when to introduce them, and how to design and implement them. At the same time, evaluations can also suggest when ALMPs should be adjusted or be reduced in scale or, for some of them, even be abandoned. The paper is arranged as follows. Section II defines active and passive policies. Section III examines the patterns of expenditures on these programs. Section IV discusses the importance of evaluating these programs and the techniques to perform the evaluations. Section V discusses the results of evaluations for the various active labor programs, and Section VI concludes. 2 II. WHAT ARE ACTIVE PROGRAMS? The use of policies aiming to facilitate employment growth and re-employment started increasing with the rise in unemployment in OECD economies after 1973. These policies, later collectively termed 'Active Labor Market Policies" (OECD, 1993) included three main elements: * Mobilizing labor supply with job creation schemes, job subsidies and so on; * Developing employment-related skills with measures such as retraining; * Promoting efficient labor markets with employment services, job-matching and counseling. Within this framework, the more common ALMPs are (Table 2.1 describes these programs): ' I. direct job creation (public works schemes/public service employment)2; 2. public employment services/job search assistance agencies; 3. training/retraining for unemployed adults and those at risk of unemployment; 4. support to unemployed persons in starting up small businesses (micro-enterprise development); 5. wage/employment subsidies to firms to hire unemployed individuals. In theory, ALMPs policies can lower structural unemployment two ways: by promoting more efficient matching between job-seekers and vacancies, and bringing discouraged and socially excluded workers (such as the long-term unemployed or women workers who dropped out of the labor force) back into the labor market. Targeted retraining and employment services would enhance their ability to re-enter the labor force and compete effectively for jobs. However, to the extent that unemployment is structural, one might argue that there are bound to be some adjustments in demand and supply, with corresponding changes in prices and incentives. Thus, the use of ALMPs begs the question as to whether one should speed up the pace of adjustment, or leave it to market forces while spending funds in other social areas. As time has passed, the need to answer this question has become urgent. Most ALMPs in OECD countries were initially introduced to counter what was expected to be a short-term increase in unemployment. However, this did not materialize ("ratchet effect")3, and doubts were therefore raised about the justification of ALMPs. For example, if a crisis is expected to last for a few months, then it may be wise to preserve employment, even through wage subsidies, to avoid what appear to be high private costs (such as costs of workers' dislocation and fixed costs of firing/hiring for firms). However, if the crisis lasts longer, it does not requires much persuasion to conclude that deficit-financed preservation of employment can be unsustainable at the macro level and can damage labor supply incentives at the micro level. 1 There are several smaller programs that we have not mentioned in this overview. In former Soviet Union countries. for example, vacancy fairs are regularly organized. In Russia in 1996, more than 2,500 vacancy fairs were organized in which 11.000 enterprises and 647.000 people participated. Close to 100,000 people were placed in jobs as a result of these fairs. 2 While we use the terms public works and public service employment interchangably, public works usuallv refer to physical/civil works to develop infrastructure usually carried out in developing countries. In OECD countries, similar programs are denoted as public service employment and may be used to employ individuals as. for examples, nurse aids and teachers aids. 3 Average unemployment levels reach new highs in successive recessions and stay above their previous level in periods of boom. 3 As ALMPs became more permanent features of OECD labor markets, the need for evaluation became more pressing. In the next section, we examine evaluation evidence after we present some statistics on expenditures on ALMPs 4 Table 2.1 Active and Passive Labor Market Programs: Some Key Features Program Description and Objective Examples of Bank Possible Pros Possible Cons Some Key Issues Projects I. Public P'ublicly funded low-wage Road construction - I. May assist disadvantaged 1. Crowding out of private 1. How long should job last? Works arid employment to address Rwanda, Albania, groups to regain labor sector jobs, especially if 2. What should be the wages Public poverty arid nutr ition Colombia, market contact. targeting is ineffective. on the job? Service objectives and create Morocco; 2. Leads to production of 2. Stigma attached to them may 3. What should the proportion Employriienit temporary employmtietit - Sewage/waste public goods and develops not increase individual's of wages in the total program these programils are mainily collection - infrastructure. employability. cost be? income-generating schemes Senegal, Lithuania, 3. Self-targeting, if wages 3. Jobs may be of low 4. Ilow should the benefits of r ather than autonomilous Morocco; are set effectively. productivity. such a job be maximized employment generation ConstrLuction of 4. Program imay be used to despite their low marginal schools and health get people off social productivity? care facilities - assistance - people refusing 5. Are these programs Madagascar, jobs are not provided effective in generating Hlonduras, unemnploymenit insiurance. employnient and higher wages Indonesia, Yemen. for program participants, and are they cost-effective? 6. Should private or public contractors be hired to implement the projects? 2. Job 'lihe main objective of Provision of 1. Helps reduce the length 1. Crowding out of private 1. What is the role of private search employment services is counseling services of unemploynment. services. job search agencies? assistance/ brokerage - matching jobs to displaced workers 2. Reasonably inexpensive. 1. Deadweight loss - individuals 2. What is the effectiveness of Employment with job seekers. Job-search -Turkey; Assisting 3. Used to pre-screen who get jobs through counseling services? Services assistance comprises many redundant workers participants who may get employmnent services are 3. What training/material do different types of services; for in their job search, assistance from other generally the most qualified and staff in employment offices example, initial inteiviews at including ALMPs. would have gotten the jobs in the need to perform their functions employment offices, in-depth preparation of absence of these services. more effectively? couliselinig durinig the resumes and 2. Benefits only a fraction of unemploymilent spell, job providing job-seekers. clubs etc. information about employmenit opportunities - Brazil, Jamaica. 5 Table 2.1 (continued) Active and Passive Labor Market Programs: Some Key Features IProgram Description and Examples of Bank Possible Pros Possible Cons Some Key Issues Objective Projects 3. TIr aining Ains at helping new Establish a training 1. Increase in productivity 1. Usually programs are 1. How can these programs and entranits to the labor force program to help of unemployed individuals. poorly targeted, resultinig in be better targeted? Retraining and redeployed workers retrenched workers - 2. When well-targeted may deadweight loss - those who 2. Is unemployment arising (either the long-terimi Jamaica; Improve the benefit some groups (e.g. benefit from the training are due to skill shortages? uLnemployed or those laid effectiveness of training the disadvantaged, women). those who would have gotten 3. What is the durationi of off en masse) to programs by making jobs anyway. these programs and how accumulate skills that will them demand-driven - 2. Job search intensity may be effective are they in isolation enable them to compete for Cape Verde; Provide reduced during the course of or in coordination with other jobs. adult training for training. labor market programs? unlemployed workers to 3. Do poorly when the 4. What is the role of the reorient their skills - economy is not growing (i.e. government and the private Romania; Finance wheni there arc few jobs) and sector in the provision and apprenticesliip programs this is typically the situation financing of traininig? to promote employmenit during which they are 5. How cost-effective are for youths -- Argentinia. instituted the most. these programs: are people 4. One of the most costly who go through these ALMPs instituted the most, programs better off in terms of wages and re-employment probabilities? 6. Should training providers sign contracts with prenegotiated rates of placement and financial incentives/disincentives? 6 Table 2.1 (continiued) Active and Passive Labor Market Programs: Some Key Features Program Description and Examples of Bank Possible Pros Possible Cons Some Key Issues Objective Projects 4. Micro- Creating and Support private micro 1. Assist in creating 1. Individuals who form 1. What kind of support - e.g. enterprise promoting small- earning generating entrepreneurial spirit. micro-enterprises would finanicial/techinical - should be provided Development scale new activities through 2. An alternative to have formed them anyway to those starting up an enterprise? businesses and establishlmlent of a unemployment without this assistance 2. What is the duration of this support? providing self- Funld - Eritrea; insurance. (deadweight loss). 3. What instruments should be devised to employment Promotion of small 2. Small businesses who most effectively screen potential through technical businesses through the do not get this assistance participants? assistance, credit development of small may be displaced. 4. What is the impact of these programs - and other support. business incubators - what is the proportion of unemployed that They are often Albania; Credit to take advantage of these benefits, the aimed at countering women to help in proportion of businiesses that survive and "market failures," starting up small scale how many additional jobs are created? not in the labor enterprises in rural market but in the areas - Morocco; Small capital or land business incubators- markets. Poland. 5. Wage Generally aimed at Provision of wage I. May lead to I. Deadweight loss - I. WhIo should receive the subsidy - the Subsidies the long-term subsidies to firms to permanent employment those that are provided employer or the employee? unemployed and hiire unlemployed - by helping individuals subsidies would be hired 2. What is the duration of the subsidy youthl, these are Argentina; Wage develop some work- anyway. and mode of payment (lump sum or designed to subsidies to firms - related skills. 2. A worker taken by a spread out over time)? subsidize Poland and Flungary. 2. Helps individuals firm in a subsidized job is 3. At what point in the unemployment employer's cost of maintain contact with substituted for an spell should the subsidy be offered? hiring unemployed thc labor market. unsubsidized worker who 4. Is the subsidy more effective when individuals. The 3. May be somewhat would have otherwise combined with any other intervention governmenit pays equitable - providing the been hired. The het (e.g. training)? part of the salary for long-term unemployed employment effect can be 5. What is the effectiveness of these a period of timie with jobs even if it zero. programs: how maniy individuals are following whichi happens at the expense 3. Employers may view retained in their jobs/find new jobs at the firms may decide to of the short-term workers as cheap labor end of the apprenticeslhip and how many hire the individual unemployed, and lay them off once the rejoin the rolls of the uLnemployed? and pay the entire subsidy period ends. salary. 7 III. EXPENDITURES ON ACTIVE LABOR MARKET PROGRAMS In 1985/86, industrialized OECD countries spent about 0.75 percent of GDP on average on these programs4 There was significant variation across countries - while the U.S. spent 0.3 percent of GDP on ALMPs, Sweden spent over 2.1 percent of GDP on these programs. By 1992/93, average expenditures on these programs had risen somewhat to about 1.1 percent of GDP but the average has remained roughly constant since then (Table 3.1). Since the late 1980s, transition economies have also instituted these programs. Expenditures on ALMPs in transition economies included in the Table are lower on average than in industrialized countries and have declined since the beginning of the decade. Table 3.1 Public Expenditures on Active Labor Market Programs in OECD Countries (as % of GDP) Country 1985/86 1992/93 1995/96 Australia 0.42 0.76 0.84 Austria 0.28 0.36 0.39 Belgium 1.23 1.21 1.41 Canada 0.63 0.67 0.56 Denmark 1.09 1.97 2.26 Finland 0.91 1.68 1.57 France 0.67 1.06 1.30 Germany 0.81 1.62 1.43 Greece 0.21 0.31 0.27 Ireland 1.58 1.31 1.75 Italy 0.45 1.88 1.08 Japan na 0.09 0.10 Netherlands 1.09 1.40 1.37 New Zealand 0.84 0.80 0.71 Norway 0.66 1.34 1.16 Portugal 0.41 0.87 0.83 Spain 0.34 0.59 0.67 Sweden 2.11 3.07 2.25 U.K. 0.75 0.59 0.46 U.S.A. 0.28 0.21 0.19 Unweighted Average 0.77 (0.77) 1.08 (1.14) 1.03 (1.08) Eastern Europe Czech Republic na 0.18 0.14 Hungary na 0.61 0.43 Poland na 0.38 0.32 Aote: Averages in parentheses exclude Japan. Source: OECD (1997. 4 These numbers pertain to national government expenditures. In some countries, state and provincial governments also finance such initiatives separately. 8 In almost all OECD countries, training for the unemployed is "the largest category of active programs (Table 3.2), and is often perceived as the principal alternative to regular unemployment benefits" (OECD, 1994a). In many countries, in fact, training - for those laid off en masse, for the long-term unemployed, and for youth - accounts for over 50 percent of the expenditure on active labor market programs. This is followed by expenditures on employment services and public works programs. Countries generally spend less than 10 percent of expenditures on active programs on micro-enterprise development or wage subsidies, a notable exception being Poland, where over 30 percent of public expenditures on active programs go into these two programs. Table 3.2 Distribution of Expenditures on ALMPs (% of Total Active Expenditure on ALMP) (Selected OECD Countries 1995/96) Country Training Public Micro- Job Subsidies Emp. Total as % of Works Enterprises Services GDP Australia 33.7 26.5 3.6 7.2 28.9 0.84 Belgium 35.7 40.7 0.0 7.9 15.7 1.41 Canada 48.2 5.4 7.1 3.6 35.7 0.56 Denmark 77.0 12.8 3.5 1.3 5.3 2.26 France 55.8 17.1 3.1 12.4 11.6 1.30 Germany 55.2 21.0 2.1 4.9 16.8 1.43 Ireland 32.0 38.3 1.1 14.3 14.3 1.75 Netherlands 54.7 9.5 0.0 9.5 26.3 1.37 Sweden 59.1 19.1 3.1 7.6 11.1 2.25 U.K. 53.2 2.1 2.1 0.0 42.6 0.46 U.S.A. 57.9 5.3 0.0 0.0 36.8 0.19 Unweighted Average 51.1 18.0 2.2 6.2 22.3 1.3 Eastern Europe Czech Republic 14.3 7.1 0.0 7.1 71.4 0.14 Hungary 30.2 25.6 0.0 14.0 30.2 0.43 Poland 40.6 21.9 6.3 25.0 6.3 0.32 Note: Training includes measures foryouth and the disabled, some of which may be non-training related Source: OECD (1997) While active and passive labor market programs are common in OECD countries, many developing countries have also instituted these programs. Over the past few years, the World Bank has also been involved in supporting active labor market interventions - with the exception of job subsidies - in developing countries. Between FY92 and FY96, the Bank has funded projects which have supported over 270 such components across all regions (Table 3.3). 9 Table 3.3 Distribution of World Bank Project Components on ALMPs by Region (FY92-FY96) Region Public Micro Training Emp. Adjustment Works Enterprises Service Services Africa 43 22 23 2 5 Europe and Central Asia 20 10 13 13 6 East Asia and the Pacific 17 5 10 8 2 South Asia 8 1 2 0 0 Latin America and the Caribbean 21 14 10 5 2 Middle East and North Africa I I 5 2 1 All 120 55 63 30 14 Note: Adjustment Services include a variety of active programs, often tied directly to large-scale privatization programs (e.g. in Romania, Turkey and Macedonia). Source: Dar and Tzannatos (1998) Lending for public works programs forms the bulk of the Bank's support for ALMPs, but significant resources have also gone into funding micro-enterprises (especially in Africa, Latin America and ECA), training activities (especially in Africa and East Europe and Central Asia), employment services (especially in East Europe, Central Asia, and East Asia) and adjustment services (especially East Europe and Central Asia). 10 IV. THE IMPORTANCE OF GOOD EVALUATIONS In spite of the large public expenditures on these programs, rigorous evaluations of ALMPs have been relatively uncommon. In the continuing effort to improve the targeting and efficiency of social programs, policymakers are now increasingly realizing the importance of rigorous evaluations. They want to know what programs accomplish, what they cost, and how they should be designed to achieve maximum efficiency and cost-effectiveness. These questions determine which of the types of evaluation are chosen (Box 4.1). Box 4.1: Main Types of Evaluations * Performance Monitoring: These evaluations provide information on how a system or program is operating, and on the extent to which specified program goals are being achieved. Examples of performance indicators in the case of active labor market programs are: proportion of program participants employed, monthly earnings of program participants, cost of program etc.. Results are used by policymakers to assess program performance and accomplishments. * Impact Evaluations: These evaluations focus on the question of causality. Did the program have its intended effects? What was the magnitude of the effect? Did the program have any unintended consequences: positive or negative? These evaluations, when done well, can be used to compare program outcomes with some measure of what would have happened without the programs. * Cost and Cost-Benefit Evaluations: These evaluations address how much the program costs, preferably in relation to alternative uses of the same resources and to the benefits being produced by the program. * Process Evaluations: These answer questions about how the program operates, and document the procedures and activities undertaken in service delivery. Such evaluations help identify problems faced in delivering services and strategies for overcoming these problems. Source: Harrell et. al. (1996) A comprehensive evaluation will include all the activities mentioned in the Box above. However, in reality, there are few such evaluations. The questions raised, the target audience for findings, and resource constraints limit the evaluation focus to one or two of these activities. As we shall see in the next section, the evaluations we have reviewed are almost exclusively impact evaluations. In some of cases, where cost data are available, cost evaluations have also been done. Below we discuss some of the evaluation techniques used in performing an impact evaluation. 11 A. IMPACT EVALUATION TECHNIQUES5 Techniques for evaluating the effectiveness of labor programs can be either scientific and non-scientific. The scientific evaluations are of two types: experimental and quasi-experimental. Experimental or classically designed evaluations require selection of both the "control" and "treatment" groups- those who receive the assistance and those who do not - prior to the intervention. If large numbers of individuals are randomly assigned to treatment and control groups, average characteristics of the two groups should not differ significantly; thus, any difference in outcomes can be attributed to program participation. Quasi- experimental studies, select treatment and control groups after the intervention. To compute program effectiveness, statistical techniques correct for differences in characteristics between the two groups. Non-scientific techniques do not use control groups in evaluating the impact of interventions, relying instead on statistics compiled by program administrators. These evaluations are of little use in determining whether program participants are doing better: without a control group, it is difficult to attribute success or failure of participants to the intervention, since these effects are contaminated by other factors, such as worker-specific attributes. However, in some cases, these evaluations (e.g. interviews with employers and employees) can provide some information on deadweight loss, as well as substitution and displacement effects. Nonetheless, it is difficult to judge how robust the results are, as this will depend on how the sample of firns was chosen and how respondents were interviewed. (Box 4.2 lists some of the commonly used terms in the impact evaluation literature.) (i) Classically Designed (Randomized) Experiments This technique was originally developed to test drug-effectiveness, with program participants and those excluded from treatment randomly selected prior to the intervention. If large samples are randomly assigned to treatment and control groups, observable and unobservable characteristics of the two groups should not differ on average, and so any difference in outcomes can be attributed to program participation. The main appeal here lies in the simplicity of interpreting results - the program impact is the simple difference between the means of the samples of program participants and control group members on the outcome of interest. The pitfalls are failure to assign randomly (because of nepotism, or excluding high risk groups to achieve better results), changed behavior upon learning of assignment to either group (enrolling in private programs, intensifying job search), high costs due to large sample sizes, and ethical questions about excluding some people from the intervention. While randomization is thought to ensure the absence of selection bias among participants, proponents of randomized experimentation make a controversial assumption: that randomization does not alter the program behavior being studied. This may not be the case, and in fact the bias induced by randomization may be quite strong (Heckman, 1992). For example, individuals who might have enrolled in a nonrandomized regime may make plans anticipating enrollment in training. With randomization they may alter their decision to apply or undertake activities complementary to training (the Hawthorne effect). Thus risk-averse persons will tend to be eliminated from the program. 5 This section draws from Dar and Gill (I 998) 12 Box 4.2: Some Commonly Used Terms in the Impact Evaluation Literature Some of the commonly used terms in the evaluation literature are defined below. * Deadweight Loss: Program outcomes are not different from what would have happened in the absence of the program. For exarnple, wage subsidies place a worker in a firm which would have hired the worker in the absence of the subsidy. * Substitution Effect: A worker hired in a subsidized job is substituted for an unsubsidized worker who would otherwise have been hired. The net employment effect is thus zero. * Displacement Effect: This usually refers to displacement in the product market. A firm with subsidized workers increases output, but displaces/reduces output among firms who do not have subsidized workers. This can also occur in helping individuals start up enterprises. * Additionality: This is the net increase in jobs created. It is the total number of subsidized jobs less deadweight, substitution and displacement effects. * Treatment and Control Group: Program beneficiaries are the "treatment" group. In a scientific evaluation, their outcomes are compared with a "control" group of individuals who did not participate in this program. The treatment and control groups could be assigned at random ex-ante (before the program) or chosen ex-post. This will be discussed in detail in Section III. * Selection Bias: Program outcomes are influenced by unobservable factors not controlled for in an evaluation (e.g. individual ability, willingness to work). Such factors can also arise as a by-product of the selection process into programs where individuals "most likely to succeed" are selected into programs ("creaming"). * Randomization Bias: This refers to bias in random-assignment experiments. In essence, this says that the behavior of individuals in an experiment will be different because of the experiment itself and not because of the goal of the experiment. Individuals in an experiment know that they are part of a treatment group and may act differently, and the same could hold true of individuals in the control group. The potential change in behavior is referred to as the Hawthorne effect. (ii) Quasi-Experimental Techniques In these techniques, the treatment and control groups are selected afierthe intervention. To get the effect of the program, econometric techniques correct for the differences in characteristics between the two groups. The main appeal lies in relatively low costs, and that interventions can be done at any time. The main drawback is that these techniques - if done properly - are statistically complex. Attributes of individuals in treatment and control groups are different; techniques for adjusting for differences in observable attributes (e.g., sex, education, age, region) are relatively straightforward but subject to specification errors; correcting 13 for unobservable characteristics (e.g., motivation, family connections) requires a convoluted procedure that can yield wildly different results depending upon specification. A quasi-experimental evaluations are of three types - regression-adjusted for observables, selectivity-corrected (regression-adjusted for both observables and unobservables), and matched pairs: Regression-adjustedfor observables. This technique assesses the impact of participation in a program when the observable characteristics (e.g. sex, age, education) of the participant and comparison groups differ. This method is appropriate for computing program impact estimates when the difference between the participant and comparison samples can be explained by observable characteristics. Regression-adjustedfor observed and unobservable variables (selectivity-corrected) When selection into programs is not random, and participation in a program is due to both observable and unobservable characteristics, program impacts computed using the technique in (i) above are likely to be biased. The concern is that even if participants and non-participants have similar observable characteristics, there are some unobservable characteristics (e.g. innate ability) which would cause non-participants to have different responses to the program if they had participated. This technique uses the Heckman selectivity method to try to control for these unobservables. Matched pairs. As observable characteristics of the individuals chosen in the control and treatment groups are bound to be different, these groups are likely to have different success rates in finding employment, even in the absence of active labor market programs. To control for these spurious differences, synthetic control groups are constructed using a matched pairs approach. The synthetic control group, a subset of the entire control group, is composed of individuals whose observable characteristics most closely match those of the treatment group. (iii) Relative Strengths Of Techniques Estimating the effect of trainee earnings on an employment program using randomized and quasi- experimental techniques, Lalonde (1986) has shown that randomized experimentation yields significantly different results from quasi-experimental techniques. Policymakers should be aware that available non- experimental evaluations of training programs may contain large biases. However, while randomized experimentation is theoretically the best technique to estimate the effects of interventions, quasi-experimental techniques may be superior in practice. The main weakness of randomized experiments is that among those who participate in the evaluation, it is difficult - if not impossible - to ensure that individuals in the control group do not alter their behavior in a way that contaminates the experiment. For example, people who were denied participation in public training may enroll in private programs, which would bias the results of any evaluation of public programs.6 The second weakness of randomized experiments is that it may be difficult to assure that assignment is truly random. For example, applicants may be selected into the program due to nepotism, or administrators of the program may intentionally exclude high-risk applicants to achieve better results. The third problem concems ethical questions about treating humans as experiment subjects. Finally, experimental evaluations are only possible for future programs, since the control and treatment groups have to be selectedex ante. 6 Heckman (1992) documents other limitations of this technique when applied to social experiments that arise due to selectivity biases. as randomization may differentially affect the decision of people to participate in such prograrns. For example. relatively risk-averse persons may decide not to enroll for randomized experiments, preferring instead to enroll in private training. A4 Using the dual criteria of rigor and feasibility, then, randomized experiments are not necessarily superior to quasi-experimental techniques. Factoring in the high costs of setting up such experiments, and the fact that the labor market programs which need to be evaluated are often already in place, randomized evaluations should perhaps be the last alternative. Within quasi-experimental techniques, selectivity-correction may not add much, especially when information is available for a considerable number of observable individual and labor market characteristics (education, age, sex, household wealth, and region of residence). Besides being cumbersome and somewhat unintuitive, this method often gives arbitrary results depending on the selectivity-correction specification used. This leaves the matched pairs and the regression-adjusted techniques. Between the two, the matched pairs technique is preferred for the following reasons: First, the procedure is less arbitrary because, since the observable differences between the treatment and comparison group are minimized, functional form assumptions become less important. Second, because program impact measures are a simple difference of means of the variables of interest (re-employment probabilities and wages) between the control and treatment groups, they are easier to interpret by non-statisticians. A weakness shared by both the experimental as well as non-experimental evaluations is that they do not take into account displacement effects of the retraining program. Thus, for example, in countries where demand for labor is constrained, retrainees may simply "bump" or displace previously employed workers so that aggregate unemployment may not change despite the size of the program. In general, displacement implies that the social benefits - from higher re-employment probabilities due to the retraining program - are lower than indicated by the evaluation, however well done. B. THE IMPORTANCE OF CONSIDERING COSTS For the purposes of informing policy decisions, an evaluation is not complete until one considers the costs of both the ALMP and its alternatives. As Box 4.3 illustrates, a program may be effective in a statistical sense, but when costs are taken into account, it may not be effective economically. Similarly, if training is shown to be twice as costly as job search assistance to the unemployed, but only as effective as job search assistance in facilitating access to jobs and wage gains, then job search assistance is twice as cost-effective as retraining even though the two are equally effective. At least at the margin, this would constitute a case for reallocating resources from training to job search assistance. Unfortunately, costs appear to be the least analyzed aspect of these programs in OECD countries. Even the most careful impact evaluations of ALMP's cannot be used for social cost-benefit analysis. The main reason is that retraining programs may simply result in displacement of previously employed workers by the retrainees, so that aggregate unemployment rates remain unaffected by the intervention. But when done correctly, evaluations are good guides forprivate cost-benefit analysis, which policymakers can use to institute cost-recovery in public programs and to promote private provision. Evaluations may also help in deciding whether ALMPs programs contribute to reduced budgetary expenditures by moving people off unemployment benefits into productive employment, or whether are a net drain inspite of being effective in doing so.7 7 Evaluators in Russia have proposed another simple. though less precise. way to compute the cost-effectiveness of their ALMPs - comparisons of costs of employment programs with potential tax revenues from actual placements. For example. the cost of Russia's Job Vacancy Fairs was Rb. 3.1 billion as compared to porential tax revenues of Rb. 160 billion from those who were employed as a result of their fairs. 15 Box 4.3: Statistically Significant And Economically Insignificant?: An Example Let us assume that a country with an annual per capita income of $20,000 institutes a retraining program that costs $10,000 per individual trained. The true impact of the program is to reduce job search/unemployment duration by three days for men and two days for women and increase post- unemployment earnings by $5/month for men and $1 for women. An effect can be statistically insignificant in a small sample... If this retraining program was initially evaluated using small samples (for example, 50 men and women), the indicated gender impact may be large or small (and can be wrongly found to negative) but most likely will be statistically insignificant: the standard errors are large because of the smallness of the sample. but can become significant as sample size increases ... As the sample size is expanded (to 500 men and 500 women for example), the impact of training becomes increasingly statistically significant, and the results show that indeed the program reduces job search/unemployment duration by three days for men and two days for women and increase post- unemployment earnings by $5/month for men and $1/month for women. still, this does not mean that the effect is economically significant..... This will depend on the duration of unemployment and average wages. If the average duration of unemployment for both women and men were 10 days, the impact of the program on men is significantly greater than that on women (by 50%). However, there will be no practical difference if the duration of unemployment was six months. Similarly, the impact on wages will depend on whether the beneficiaries of the retraining program were low-paid workers (e.g. a monthly income of $250) or higher paid workers (e.g. with a monthly income of $1000). In the former case the increase in earnings is two percent for men and 0.4 percent for women while in the latter case the impact is 0.5 percent for men and 0.1 percent to women. or efficient..... Now compare the group of individuals who have been trained with a control group who were not trained. Assume that the training has a statistically significant impact on the probability of employment of 10 percent. Thus, if 60 percent of the control group is employed, 70 percent of the treatment group manage to find jobs after completion of their training. This implies that 100 individuals need to be trained to place 10 extra trainees in jobs. The cost of training 1 00 people will be $1,000,000, and the cost per placement will then be $100,000. Given that this is five times the per capita income of the country, the program is unlikely to be economically desirable even though it results in a positive impact on employment. 16 V. AN INTERPRETATION OF EVALUATION RESULTS The remainder of this paper presents our reading of the evaluative evidence (mainly from OECD countriest of active labor market programs. In various appendices we have included summaries of the studies upon which our synthesis is based. The reader is encouraged to refer to them for a more objective presentation of individual studies. A. PUBLIC WORKS PROGRAMS/PUBLIC SERVICE EMPLOYMENT Public works programs are one of the most heavily funded ALMPs in OECD countries. Expenditures on public works programs range from insignificant (close to zero percent of GDP in U.K. and the U.S.) to about 0.5 percent of GDP (Belgium and Ireland) in OECD countries. On average, OECD countries spend about a quarter of all expenditure on ALMPs on public works programs. These programs have generally been instituted in times of economic slowdown, and participants are provided jobs in various sectors of the economy, e.g. construction, health, welfare and municipal government. Public works often serve a dual objective; they provide temporary employment and act as a short-term safety net. Although most countries target the displaced and the long-term unemployed ("the hardest to place groups"). youths in some countries also participate as a way to introduce them to the world of work. For example, in Sweden, they have been used as a safety net for the young unemployed (Forslund and Krueger. 1994). In other cases, such as Benelux countries, public works have come to be used as an almost permanent means of employment for those who would not find jobs elsewhere (OECD, 1993). We summarize the results of thirteen evaluations of public works programs, nine of which are quasi- experimental and four of which are non-scientific (see Appendix 5. 1). Among industrialized countries, the evaluations point to some general conclusions. First, non-experimental evaluations show some desirable short-run effects in the form of employment increase/unemployment decline. Second, some scientific evaluations suggest displacement effects which can reach 100 percent (as they did in Sweden; Skedinger, 1995). Third, participants have a smaller probability of being employed in a non-assisted job after participation in the program, and are likely to eam less than their counterparts in the control group.9 Finally, these programs do not seem to have a significant impact on reducing long-term unemployment in the economy. Among transition economies, Fretwell, Benus and O'Leary (1998) provide the following summary picture of the impact of public works (called "public service employment programs") in the Czech Republic, Hungary and Poland (Table 5.110). In Hungary, these evaluations (O' Learv, 1998(a)) show that those who participated in the program are over 20 percent less likely to be employed and earn about $15/month less 8 The World Bank is in the process of completing a four-country study (Hungary. The Czech Republic. Poland and Turkey) which uses state of the art quasi-experimental techniques to evaluate active labor market programs. Some of the preliminary results of this evaluation (for Hungary and Poland) are included in this paper. See Fretwell. Benus and O'Leary (1998). 9 In many countries in the former Soviet Union, public works programs cater mainly to those individuals who want to participate temporarily in the labor force (e.g. women with young children) and hence the issue of post-program comparison with a control group is meaningless. 10 In this table initial employment and earnings refer to the probability that an individual in the treatment group is employment relative to someone in the control group - just after completing the program and at a later survey date respectively. Similarly, initial earnings and current earnings refer to the relative wage difference between treatment group and control group participants - just after completing the program and at a later survey date respectively. The unemployment compensation refers to the difference in unemployment compensation paid to individuals the control group relative to individuals in the treatment group. 17 than individuals in the control group after program completion. While results for Poland (O' Learv, 1998(b)) are less negative, they also point to the lack of general effectiveness of these programs. While data on program costs are available, it is unnecessary to undertake a cost-benefit analysis: the ineffectiveness of the intervention suggests that it is unlikely to be cost-effective. Table 5.1 Overall Impact of Public Service/Community Employment Programs in Transition Economies Indicator Czech Republic Hungary Poland Initial employment No impact Negative Negative Current employment Negative Negative Negative Initial earnings n.a. Positive n.a. Current earnings No impact negative Non impact Unemployment compensation Negative No impact Positive Memo items 1. Cost per participant (US$) 625 1200 800 2. Cost per participant (PPP$) 1578 1867 1543 3. Per capita GDP ($US) 4740 4340 3230 Note: 1) Costs are per participant, not per year. For example, in the Czech Republic, the duration of participation is 6 months which implies that annualized program costs were twice those reported 2) The Purchasing Power Parity (PPP) conversion factor is defined as the number of units of a country's currency required to buy the same amount of goods and services in the domestic market as one dollar would buy in the United States. Source: Constructedfrom Fretwell, Benus and O'Leary (1998) (WDR 1997, 214-5) These findings suggest that there can be some short-lived decline in unemployment as a result of individuals joining the program, but there are no longer term "multiplier" effects in the sense of helping reduce the rate of unemployment through re-integrating or increasing labor force attachment of the unemployed. This conclusion is to a large extent predictable: unlike other ALMPs, such as training, retraining and support program for self-employment, public works provide mainly current benefits (temporary safety net) and are only escape routes from unemployment. Irrespective of the merits of public works, an economist's first impression is that public works can be generally expensive (Table 5.2) and are not an effective instrument if the objective is to get people into long-term gainful employment. Table 5.2 Annual Cost of Job Creation in Public Works Egypt Hionduras Nicaragua Madagascar Bolivia Senegal Ghana l. Cost/job (US$) 1401 2120 2580 786 2700 5445 2122 2 .Cost/job (PPP) 7212 9759 14302 3620 9388 12100 10610 3. Per capita GDP (US$) 790 600 380 230 800 600 390 4. Ratio (1/3) 1.77 3.53 6.79 3.42 3.38 9.08 5.44 Source: Adaptedfrom Subbarao (1997,; WDR 199 18 However, public works can be a short-run anti-poverty intervention (Box 5.1). That's why some developing countries have used them extensively in periods of hardship, such as in Botswana, where 21 percent of the labor force was employed in public works in 1985-6, in Chile (13 percent in 1983) and Honduras (5 percent between 1990-3). In other cases, such as the Maharashtra Employment Guarantee Scheme in India, year-round employment is guaranteed. However, the wage rates have been set very low, so that only the poorest are targeted.11 In Korea public works were extensively used, then were gradually phased out as the country industrialized, and were introduced again following the 1997 financial crisis. Box 5.1: What Makes a Public Works Program in a Low-Income Country? Good design features include: * The wage rate should be set at a level no higher than the prevailing market wage for unskilled manual labor in the setting in which the scheme is introduced. * Restrictions on eligibility should be avoided; the fact that one wants work at this wage rate should ideally be the only requirement for eligibility. * If rationing is required (because demand for work exceeds the budget available at the wage set) then the program should be targeted to poor areas, as indicated by a credible "poverty map"S. However, flexibility should be allowed in future budget allocations across areas, to reflect differences in demand for the scheme. * The labor intensity (share of wage bill in total cost) should be higher than normal for similar projects in the same setting. How much higher will depend on the relative importance attached to immediate income gains versus (income and other) gains to the poor from the assets created. This will vary from setting to setting. * The projects should be targeted to poor areas, and should try to assure that the assets created are of maximum value to poor people in those areas. Any exceptions -- in which the assets largely benefit the non-poor -- should require co-financing from the beneficiaries, and this money should go back into the budget of the scheme. * Performance in reducing poverty should be monitored using careful evaluations. Source: Ravallion (1998) B. JOB SEARCH ASSISTANCE/EMPLOYMENT SERVICES Expenditures on these programs range anyvwhere from five percent of active labor market program budgets (in Denmark) to over 70 percent (in the Czech Republic). On average, OECD countries spend about a quarter of their active labor market program budgets on these programs. It should be noted here that this expenditure is often not used solely to finance job search assistance programs, but also to finance the administration of the unemployment benefit system, as well as the administration and costs of the other active labor market programs. I IRecent evaluations of this scheme have showed that, due to a significant increase in the wages, the self-targeting nature of the program has been somewhat compromised. 19 Job search assistance is often justified on the grounds that are especially benefit the disadvantaged - while the private sector provides core labor exchange services successfully to some segments of the labor force, for example, the employed, skilled and the white collar workers, public employment services are beneficial for the poor, semi-skilled and long-term unemployed workers (Fretwell and Goldberg, 1994). In the evaluations examined, these programs aim at assisting a varied group of unemployed and job seekers - including the long-term unemployed in the U.K., women on welfare in the U.S., individuals laid off as a result of industrial restructuring in Canada, and the youth in Holland. Economic conditions in countries that have instituted these programs also vary significantly - a steep decline in unemployment rates in the U.K. in the late 1980s as compared to a sharp rise in unemployment in Canada in 1989-1991 (Fay, 1996). Job seekers are usually provided a wide range of services. In Hungary and Poland in the mid 1990s, for example, they were provided with job referrals, job counseling, skills assessment, job search training, resume preparation and job clubs (O' Leary, 1998(a), 1998(b)). In New Zealand they were assisted through a job screening interview, workshops, follow-up interviews and personal case management (NZ DOL, 1995). In Australia they were helped in resume writing and interviewing techniques (Fay, 1996). Of the 18 evaluations we examined, all except one are scientific. Six of the scientific evaluations are experimental and 11 are quasi-experimental (Appendix 5.2). The evaluations suggest that job-search assistance is in some serise one of the most successful active labor market programs: in the general case, it costs little to provide and the program is not any less effective than alternative ALMPs. It is, therefore, the low cost that drives this conclusion. However, as argued below, much depends on whether the economy is growing or in a recession (as far as effectiveness is concerned) and on the availability of public funds (which can be scarce during a recession). More specifically, while some evaluations yield "negative" results, most studies indicate positive results, and in a few cases large ones.12 Less successful programs are generally associated with periods of recessions and rising unemployment rates. For example, job search assistance to those laid offen masse in Canada in the late 1980s did not raise their probability of employment or earnings (as compared to a control group) at a time when unemployment was rising. In fact, participants who had been laid offen masse spent a significantly greater amount of time searching for jobs than their counterparts who did not use this service (Fay, 1996). On the other hand, the effectiveness of job search assistance seems to increase when economic conditions improve; that is, when new jobs are generated. During the decline in unemployment rates in the Netherlands in the mid-to-late 1980s, program participants were more likely to be employed than those in the control group (OECD, 1993). Similarly, as the U.S. economy was growing in the mid-to-late 1980s, program participants (usually female AFDC recipients) were four percent more likely to get jobs and earn 20 percent more than the control group (Gueron, 1990). Evaluations in Hungary and Poland also show that while the overall effectiveness of this intervention is not significant in terms of increasing probability of employment, if the economy is improving, some subgroups of program participants - particularly women - fare significantly better than non-participants (O' Leary, 1998(a), 1998(b)). However, several of the evaluations show that job search assistance had no beneficial impact on youth irrespective of general economic conditions (Fey, 1996). Job search assistance is also supposed to be one of the most-cost effective of the active interventions. Leigh (1995) summarizes his findings of selected programs: job search assistance measures cost less than 12 Deadweight loss is rarelv measured in these evaluations. It may happen that those who benefit from these services are those who would have got jobs anyway because they were the most motivated in looking for jobs. 20 training and retraining (two to four times), but appear equally effective. This of course does not mean that job-search assistance is a substitute for training - it is possible, for example, that those who use job search assistance are more "employment-ready" than individuals who get training. Hence, substitution of one program for the other will only be the case if job search assistance and training programs cater to roughly the same clientele. Other evidence also seems to point in the direction of greater cost-effectiveness of job search assistance programs. One of the most extensive evaluations of these programs was an experimental demonstration carried out in the U.S. at five test-sites in Charleston, New Jersey, Washington, Nevada and Wisconsin during the 1970s and the 1980s. These evaluations showed a significant decline in the number of weeks of unemployment insurance claimed by program participants. They also showed that post-program participants' earnings were likely to be higher than those of individuals in the control group. A cost- benefit analysis of these experiments showed that these employment effects (post-program reduction of duration of search among participants from 0.5 to 4 weeks) were associated with a reduction in unemployment benefits paid out to participants (Meyer, 1995).13 Overall, the evidence suggests that job search assistance can have some positive effects and is usually cost-effective (relative to other ALMPs). There does seem to be a positive correlation between the likelihood of success of a program and local labor market conditions. Programs that have not yielded positive results are generally associated with rising unemployment rates, while economic conditions have been generally favorable in the case where programs have succeeded. However, job search assistance - like other interventions - does not seem to improve either the employment prospects or wages of youth significantly as compared to a control group of their peers. C. TRAINING PROGRAMS Training (and re-training) generally accounts for the largest share of expenditures on ALMPs ranging between 40% to 60% and reaching 77 percent (Denmark). Spending on training can reach many different groups, and in our summary of the review below, we concentrate on (i) retraining programs geared towards those laid off en masse; (ii) training and re-training programs for the long-term unemployed; and (iii) training programs geared towards youth. We discuss the results of evaluations of these programs in turn. (i) Training Programs for the Long-TernI Unemployed We reviewed 23 studies - six experimental, 13 quasi-experimental and four non-scientific (see Appendix 5.3). A few of these studies are longitudinal, so it is also possible to study the long term impact of the programs. Though there is no generally agreed definition of what constitutes long-term unemployment, the term usually refers to those who have been unemployed for more than 12 months (OECD, 1993). This group tends to consist of "older" workers who used to be employed in what turned out to be a declining industry or region. But significant variation exists among them in terms of age, skills, and education, especially if unemployment is the result of regional depression. For example, New Jersey's retraining program in 1986- 13 In a study by White and Lakey (1992) the duration of search of non-participants was on average 8 weeks longer than those of program beneficiaries. 21 87 included mainly workers in their 50s (Anderson, Corson and Decker, 1991), while half of the long-term unemployed workers in Gerrnany's who participated in retraining programs were less than 35 years old (Johanson, 1994). In similar programs in Hungary and Poland, trainees ranged mid-20s to 50 years old (O' Leary, 1998(a), 1998(b)). Compared to scientific evaluations, nonscientific evaluations usually provide an inflated picture of the effects of training and retraining programs on the long-terrn unemployed..*4 Scientific evaluations suggest that the programs for the long-term unemployed can have a positive impact but this is not always the case, and the impact is usually small. Evaluative evidence of some retraining programs for the long-term unemployed in the U.S. shows that these programs are also more expensive than job search assistance programs (approximately by a factor of two) (Leigh, 1995). The success of programs for the long-term unemployed tends to be heavily dependent on the business cycle: programs have performed better when they were instituted at times when the economy was expanding. A good example of this is Hungary, where training outcomes seem to have improved over time as the economy started to grow (O' Leary (1995, 1998(a)). In general, programs seem to be more effective for women (Friedlander et. al., 1997; Goss, Gilroy et. al., 1989). Longitudinal studies indicate mixed results - while in some cases the positive effects dissipated within a year or two after program completion, in a couple of cases the impacts persisted. For example, in Sweden, labor market training provided to the unemployed raised their earnings in the short-term but the long-term impact (over two years) was somewhat negative (Meager and Evans, 1998). Conversely, long-term unemployed provided training as part of the New Jersey Reemployment Demonstration project in the mid 1980's were earning more than the control group more than two and a half years after program completion (Anderson, Corson and Decker, 1991). In most cases, retraining programs are generally no more effective than job search assistance in increasing either re-employment probabilities or post-intervention earnings. For example, evaluations of the Texas Worker Adjustment Demonstration (WAD) program indicate that participants were likely to be employed more quickly than non-participants. However, by the end of the first year after program completion, employment opportunities for male participants were no better than those for non-participants, or for those who only got job search assistance (Bloom, 1990). Costs, when known, vary between $900 and $12,000 per person. In most cases the costs are found to be so high compared to the benefits of the program that, even if the effects persisted for 10 years, the social return of the program could remain negative (especially for males) (Friedlander et. al., 1997)j5 The U.S. JTPA program appears to be a rare exception - both male and female participants were doing significantly better than the control group and the training program was relatively inexpensive (Box 5.2). 14 In many former Soviet countries, for exarnple, participants in public training programs are still guaranteed ajob after completion. With placement rates reaching close to 1 00 percent in these cases. non-scientific evaluations tend to show that these programs are highly successful. 15 The social return is based on a comparison of measurable economic costs and benefits and does not take into account possible extemalities associated with the reintegration of the ]ong-term unemployed into the labor force or of reducing high levels of unemployment in specific regions. 22 Box 5.2: The U.S. Job Training Partnership Act Evaluation The Job Training Partnership Act (JTPA) provided for programs which are national in scope and is targeted at disadvantaged adults (aged 22 and older) and youth (16 to 21 years old). Various activities are undertaken by local service delivery areas (SDAs). SDAs provide specific employment and training services either directly through their own staffs or by contracting with other local service providers (e.g. community colleges or community based organizations). The specific training services offered generally fall into one of two categories - classroom training and on-the-job training (other services provided were: job search assistance, basic education and work experience). Classroom training includes both occupational skills training in fields like clerical, food service, electronics, or home health care, as well as training in basic academic skills. On-the-job training consists of subsidized training that usually takes place as part of a paying job, generally in the private sector. The program pays half of the wages for up to six months, but the job is supposed to be permanent. At the time the evaluation was done, the net cost of training per participant averaged approximately $1200 for adults and over $2000 for youth - making it one of the least expensive training programs in the U.S.. The evaluation of the training program was experimental in nature, and the research team undertook extensive sensitivity analysis to examine possible underreporting bias and survey non-response bias in the follow up surveys - these two being threats to the internal validity of experimental research. The experiment was carried out at 16 sites. Evaluation results show that the JTPA's effects on earnings are significant for men and women (an increase of over 10 percent), with short-term classroom training having the least beneficial effects while on-the-job training was more beneficial. While evaluations of some other programs have also shown a positive effect of training on women, the findings for men represented a break from the results of other evaluations which have generally reported insignificant effects for men. Extended follow up data show that effects for adults continued over five years of follow up, although the later year effects are smaller and statistically insignificant. Like most other evaluations, the JTPA did not have a significant effect on increasing the employment or earnings prospects of youth. Cost-benefit analysis suggests that the societal rate of return for JTPA is over 50 percent for adults (the rate of return is negative for youth), making it one of the few cost-effective adult training programs instituted in the U.S. However, in spite of the positive results, the evaluators caution that the aggregate effects of JTPA are likely to be modest, both on the target population and on the labor force as a whole. This is because the effects diminish over time, and there is bound to be some substitution effect (whose magnitude was not captured in the evaluation). Further, it is unclear whether the high rates of return would still be observed if the scale of participation was substantially increased. Unfortunately, little evidence exists on this issue. One of the major implications emerging from the analysis is that training should not be seen as a panacea for reducing unemployment - it usually does not work, especially if the economy is not growing. The evidence shows that programs aimed at the long-term unemployed, usually, established in periods when the economy is doing better, are somewhat more effective (at least in the short-run). Evaluations also show that small-scale tightly targeted on-the-job training programs, usually aimed at women and other 23 disadvantaged groups among the long-term unemployed, often offer the best returns. While rigorous cost-benefit analyses of these programs are seldom carried out, where done, the evidence shows that the real rate of return of these programs is rarely positive and these programs are not cost-effective - they are no more successful than job search assistance programs in terms of post-program placement and wages while they cost anywhere between 2-4 times as much. It needs to be stressed here again that this does not mean that job-search assistance programs are substitutes for training programs - these programs may be serving different groups of the unemployed and hence cannot be directly compared on the basis of cost- benefit analysis. (ii) Retraining Programsfor Those Laid off En-Massel6 We summarize the results of eleven studies, five quasi-experimental, five non-scientific, and one which used a variety of techniques (Appendix 5.4). The studies relate to retraining programs for workers displaced through mass layoffs as a result of significant enterprise restructuring or plant closures. Unfortunately, none of the evaluations was longitudinal, so they do not really provide insights into the dynamics of the labor market and longer-term benefits of retraining programs. The retraining programs under consideration were offered mainly to workers in the automobile, shipbuilding, mining, steel and pulp industries. The number of workers who lost their jobs vary from about 500 to 3,000 per plant. Generally, these losses occurred during periods of high or rising unemployment economy-wide, or contraction of the manufacturing. Most of the training provided was classroom-based, and was generally accompanied by job search assistance. Evaluations of three retraining programs for US auto workers show the contrast between scientific and nonscientific techniques: in San Jose, a nonscientific evaluation indicated high placement rates, while in Buffalo and Michigan - during the same period - scientific evaluations showed that these programs were ineffective. More generally, nonscientific evaluations indicate that these programs are very effective with high placement rates of participant workers in jobs following completion of the program (Alfthan and Janzon, 1994). T'he more reliable quasi-experimental evaluations challenge this optimism: though they find that some retraining programs may result in some modest increase in re-employment probabilities, this result is often statistically insignificant (Corson, Long and Maynard, 1985). The results for post-program earnings are more discouraging: wage effects on participants (compared to control group workers) are rarely positive and in most cases negative (OECD, 1991). In all, we conclude that, from an economic perspective and based on the evidence of employment/wage impacts, retraining must have high deadweight cost. These results compare unfavorably to the effects of programs for the long-term unemployed where there is some evidence of positive impacts. Evaluations seldom report the full costs of retraining but, when known, direct costs - usually measured as total recurrent program costs - vary between $3,500 and $25,000 per participant (Dar and Gill, 1998). This makes retraining programs between two and four times more expensive than, say, job search assistance (JSA). For example, in Buffalo the cost of JSA per participant was $850, while that of retraining was $3,300. Combined with the finding that retraining and JSA have roughly similar impact, this implies that JSA can be more cost-effective in assisting displaced workers than retraining. Some researchers have gone as far as claiming that there is no evidence of any incremental effect above that of job search assistance for 16 This section is taken from Dar and Gill (1998). 24 retraining programs (Corson, Long and Maynard, 1985). As before, it needs to be stressed here that these programs may be serving different groups of the unemployed and hence may not be direct substitutes for one another. OECD experience of retraining programs for workers displaced en masse may be useful in designing assistance programs in transition countries and other economies that expect large scale labor redundancies. The evidence on the lack of effectiveness and cost-effectiveness of these programs suggest that they should not be the principal source of support to assist individuals to move to gainful employment. If these programs are to be used, they should be small scale and targeted towards those subgroups who can benefit the most from them. (iii) Training Programsfor Youth of The most rigorous evaluations of training programs geared towards youth have been conducted in North America and Scandinavia. We have examined seven evaluations: five experimental and two quasi- experimental (see Appendix 5.5). These training programs usually aim at assisting school dropouts (often paying special attention to youngsters who drop out prior to completing upper secondary education) or youth who come from severely disadvantaged families. The assisted individuals are typically below the age of 20. The evaluations examined different aspects of training assistance for the youth, such as classroom training, on-the-job training and, in one case, job search assistance. Evaluations results for youth training are the most discouraging of all (that is, compared to training programs for the long-term unemployed and those laid-off en masse), even though the programs examined here were often introduced in periods of relatively stable or declining youth unemployment They show that training rarely has an effect on earnings or employment probabilities of program beneficiaries compared to their counterparts in some control groups. In almost all cases, participants did no better than the control group either in enhancing their post-training employment probability or their earnings (Fay, 1996; DOL, 1995). A more mixed, and more promising, picture arises from the evaluation of the Canadian Job Entry Program. Though a quasi-experimental evaluation of the program showed that youth who only undertook classroom training did no better than their comparators, those who undertook enterprise training did significantly better than the control group (OECD, 1993). This positive effect was attributed to youngsters staying-on with the training firms. Still, enterprise training seem to be more effective for young men, with little effects on young women. Cost-benefit analysis of several of the youth training programs suggests that the social rates of return to these programs are typically negative both in the short- as well as the long-run (Friedlander et. al., 1997). The evidence suggests that it is very difficult to correct what appears to be a failure of the education system during the previous 5-10 years of the youth's life with some kind of training which is usually short in duration and takes place relatively late in life. D. MICRO-ENTERPRISE DEVELOPMENT (SELF-EMPLOYMENT) SCHEMES We have summarized the results of thirteen evaluations of programs aimed at helping unemployed individuals start up their own businesses: two of the evaluations are experimental, five are quasi- experimental and the rest are non-experimental (see Appendix 5.6). These programs come under a variety 25 of names such as micro-enterprise schemes or self-employment schemes but below we generically refer to them as "micro-enterprise development assistance" (MEIDA). MEDA attracts limited funds compared to other active labor market policies: no OECD country spends more than 10 percent of expenditures on ALMPs on such programs. In general, these programs are not targeted at any particular group. For example, such assistance has been given to the newly unemployed (such as in Massachusetts, U.S., in the early 1990s), to those who have been unemployed for at least five months (such as in Denmark in the 1980's) and those who were laid-off in Hungary and Poland in the 1990's. Similarly, they are available under varied economic conditions. In Ireland, for example, these programs were instituted in the mid 1980s at a time when unemployment rates were high (around 15 percent) and rising. In Australia, on the other hand, they became more common in the late 1980s when unemployment rates fell from eight to six percent. Program conditions also vary, for example; participants may receive assistance to set up their businesses as a lump-sum payment or periodic allowances. Often there is "screening", that is, potential beneficiaries undergo a rigorous assessment which evaluates their likelihood of success (for example, in Germany) but in other countries, such as the US, screening is more cursory (Wilson and Adams, 1994). In most cases participants may also receive post-startup business advisory services and business counseling. Though theoretically the idea that someone who cannot get ajob as an employee will be inclined to become self-employed makes sense, very few among the unemployed are tempted to take up opportunities for self- employment. Generally, the take-up rate among the unemployed hovers at no more than five percent in OECD countries (Wilson and Adams, 1994). One explanation for this may be that individuals are generally risk averse, and given a choice between getting unemployment benefits or money to start up a - possibly unsuccessful - venture, prefer to get the unemployment benefits. As usual, non-scientific evaluations provide more encouraging results than scientific evaluations. Still, there is general agreement that MEDA programs have high deadweight loss. Estimates of deadweight losses vary from about 30 percent in the self-employment experiments in Massachusetts and Washington State in the late 1980's and early 1990's (Fay, 1996) to over 50 percent in Canada's self-employment assistance program in 1992-93 (Graves and Gauthier, 1995) and Denmark's enterprise startup grant in the late 1980's (Balakrishnan, 1998). In Ireland, the estimated 50 percent deadweight loss seems to have been accompanied by a 30 percent displacement effect - this led to a small net employment impact of 20 percent (OECD, 1993). Evaluations show that businesses are short-lived - typically one-third to half of MEDA created businesses close down in the first year of their operation.17 For example, in Denmark, the failure rate of business is 60 percent in the first year and in Australia it is more than 70 percent in the first two years. Results are more encouraging in Canada where a crucial feature of this scheme is that individuals contribute up to 25 percent of the start up costs of the business: there, the first year's failure rate is only 20 percent. In France 17 We do not have data on failure rate of small businesses who did not take part in MEDA programs. However, anecdotal evidence seems to suggest that usually these rates are similar to those of that of MEDA businesses. Thus, while MEDA created businesses do no worse in terms of survival rates than businesses which did not benefit from the program, they do not appear to do any better. 26 the failure rate is 50 percent but over a period of five years (Table 5.3). Evidence seems to show that businesses assisted through mentoring and business counseling are more likely to succeed. Table 5.3 Failure Rates of MEDA businesses Program Failure Rate Australia in the late 1980's (New Enterprise 58% of businesses failed within first year and 71 % Initiative) within two years. Canada in the early 1990's (Self-Employment 20% of businesses failed within first year Assistance Program) Denmark in the mid to late 1980's (Enterprise 60% of businesses failed within first 12 months Allowance Schemes) France in the early 1980's (Micro-Enterprise 50% of businesses failed within 4.5 years. Development) Hungary in the mid 1990's (MEDA) 20% of businesses failed within first 15 months. Netherlands in the early 1990's 50% of businesses failed within four vears. Poland in the mid 1990's (MEDA) 15% of businesses failed within first two years. U.S. in Washington in 1990 (Self-Employment 37% of businesses failed within the first 15 months Experiment) Source: Summarizedfrom Appendix 5.6 While there are high deadweight losses and high rates of business failures, MNEDA participants seem to fare reasonably well in terms of employment outcomes as compared to a control group. Scientific evaluations show that participants are more likely to be employed than individuals in the control group in almost all the programs for which such evaluations exist. However, this does not necessarily translate into higher earnings. For example, in the Washington self-employment experiment, while participants are more likely to be employed than the control group, they earn significantly less (Fay, 1996). In Hungary, participants were as likely to be employed as individuals in the control group, but earned $30/month less (O'Leary, 1998(a)). In the case of Poland, on the other hand, participants were 25 percent more likely to be employed than the control group and eamed significantly more (O' Leary, 1998(b)). In the Hungarian and Polish programs, women and older workers generally had better outcomes than individuals in other sub-groups. Even in cases of businesses that survive, there is only a small multiplier effect. Most surviving businesses create, on average, half an additional job. In Hungary, each surviving enterprise created 0.3 additional jobs, in France, 0.5, and in Australia, during a period of declining unemployment, 0.7 (OECD, 1993; Wilson and Adams, 1994). While there is some scattered data on costs, the cost-benefit issue has rarely been addressed. Where available, data indicate that the cost of starting up a small business vary from $4500 (in France) to $13,000- $14,000 (in Canada and Denmark). The Canadian evaluation states that the long-term cost-effectiveness of these programs is uncertain, while preliminary analysis from Poland and Hungary indicate a loss to the unemployment insurance system with both the average duration and level of unemployment benefits paid to participants being greater than those paid to individuals in the control group. However, it would be premature to draw any conclusions on the cost-effectiveness of these programs on the basis of so little evidence. 27 In all, evaluations suggest that these programs work for only a small subset of the unemployed population (generally below five percent) and are associated with high deadweight and displacement effects rendering the "net" effects of these programs to be quite low. The failure rate of these businesses are quite high in most cases (though businesses assisted through mentoring and business counseling are more likely to succeed). As in the case of training, assistance targeted at particular groups - in this case women and older individuals - seems to have a greater likelihood of success. E. WAGES/EMPLOYMIENT SUBSIDIES We examine eighteen interventions - one of which is experimental, 11 quasi-experimental and six non- scientific (see Appendix 5.7). Along with micro-enterprise development, wage/employment subsidies (WES) are among the least funded active labor market programs in OECD countries. In most OECD countries, they attract less than 10 percent of expenditure on active labor market programs - in fact, in both the U.S. and the U.K., funding for this program is negligible. Eastern European countries generally put greater emphasis on this intervention. In Hungary, expenditures on these subsidies account for 14 percent of spending on ALMPs, while Poland expends 25 percent of its ALMP budget on these programs. Wage subsidy programs have been used for the long-term unemployed, those coming from severely disadvantaged areas (e.g. areas with high unemployment), and youth. They aim to reduce social exclusion, that is, to help these individuals regain contact with the world of work. They are instituted under varying economic conditions - in Australia during periods of falling unemployment in the mid 1980s (Mangan, 1988) as compared to Scotland where unemployment rates rose from six to 10 percent while the program was underway between 1989-1992 (NERA, 1995).18 WES typically is a payment to firms as a proportion of the wage in order to induce them to hire program participants, but the level and duration of these subsidies varies significantly between programs and countries. Thus, for example, under the U.S. Targeted Job Tax credit, firms are paid 50 percent of the individual's wages for a period of up to two years, while the U.K. job subsidy program provides up to 100 percent of the wages (as well as paying for all training costs) for a period of six months. Both non-scientific and scientific evaluations tend to agree that WES have high deadweight loss and substitution effects. In the extreme case of Ireland's wage subsidy program, the deadweight and substitution losses combined totaled over 95 percent - alternatively, the net incrementality of the program was a meager four percent (OECD, 1993). Belgium's program did not lag far behind, with the net employment impact of only 10 percent (Fay, 1996). Evaluations of similar programs in Australia, Holland and the U.K. also indicate high deadweight and/or substitution effects (Table 5.4) Equally disappointing are evaluations which compared wages and employment outcomes of participants with those of a control group. For example, the U.S. Targeted Job Tax Credit program, which is a longitudinal study, saw earnings of participants significantly higher than those of individuals in the 18 In Japan, Employment Adjustment Grants are a form of wage subsidies which aim to lower unemployment by extending grants to employers who have offered temporary leave, provided education and training, or temporarily transferred employees to other companies due to unavoidable reductions in business activity attributable to business fluctuations and changes in the industrial structure. These grants cover part of the costs for temporary leave allowances, ordinary wages and wages for transferred employees. There grants are financed from the employment security fund which is accumulated from employer premiums. However, the effectiveness of these programs has rarely been evaluated. 28 control group in the first year. This effect declined in the second year and disappeared after that - not surprisingly coinciding with the end of the subsidy when the employer could fire the worker and hire another subsidized worker - thus maintaining a permanent subsidy (OECD, 1993). Similarly, evaluation results for Hungary show that participants were significantly less likely to be employed and earned less (though not significantly so) than those in the control group (O' Leary, 1998(a)). Box 53: Quick Reemployment Bonuses: An Alternative to WES? An interesting variation of the wage subsidy intervention was tried in the U.S. in experiments in four states (Illinois, New Jersey, Pennsylvania and Washington) in the 1980s where reemployment bonuses were provided to unemployed workers if they could find employment within a specified time period (Meyer, 1995). For example, in Illinois, an unemployed individual would receive a bonus of $500 (four times the average weekly unemployment insurance (UTI) benefit) if they found employment within 11 weeks of filing for U1 and kept the job for at least four months. New Jersey's bonus offer also had a four month reemployment requirement, but had a 12 week qualification period and a bonus amount which decreased as duration of unemployment increased. Pennsylvania and Washington each conducted separate experiments (11 in all). Evaluations showed that unemployment spells fell in all four experiments, and there were no significant declines in reemployment earnings. However, interestingly, in one treatment group in Illinois where the bonus was provided to employers (in effect a wage subsidy), the impact on earnings was insignificant as compared to the control group. Due to the high costs of the programs, societal benefits varied. They were positive in Illinois and Pennsylvania (slightly) but negative in New Jersey and Washington. In 1994 and 1995, new mechanisms for early identification of UTI beneficiaries were implemented in order to provide aprofile for identifying UI beneficiaries most likely to exhaust their entitlements. Simulations suggest that targeting reemployment bonuses with profiling models - targeting reemployment bonuses to claimants with high exhaustion probabilities - may appreciably increase the cost-effectiveness of the bonus. While the experiments yielded somewhat positive results, evaluations caution against the permanent adoption of such bonuses as they are usually not cost-effective and end up generating losses for the UI system. However, some recent evidence suggests that profiling may improve the cost-effectiveness of the bonuses. A reemployment bonus also makes filing for UT much more valuable, as claimants become eligible for a large payment if they file and then find a job within a short period. These effects may be significant. Source: Meyer (1995), OL'eary (1998(c)) One exception to these results seems to be the U.S. Job Training Partnership act program, where employers were provided subsidies to hire workers in conjunction with on-the-job training. Evaluations of this program show that single mothers who are AFDC recipients benefit significantly from these subsidies, and males also benefited to some extent (Bloom, 1994). 29 While very few studies carried out a direct cost-benefit analysis, the high dead-weight losses and substitution effects associated with this program strongly suggest that WES are unlikely to have positive social returns in the way measured by economists, though they may contribute to some reduction in social exclusion among older workers and single mothers. Table 5.4 Effectiveness of Wage Subsidy Programs Country Deadweight and Additionality (%) Substitution Effects (%) Australia in mid 1980s (Jobstart Deadweight=65% 35% Program) Belgium in the early 1990s Deadweight=53% 11% (Recruitment Subsidy) Substitution=36% England 1986-1990 (Training and Deadweight=69% 31 % Employment Grant) England late 1980s (Workstart I) Deadweight=45% 25% Substitution=30% England mid 1970s (Small Firms Deadweight=70% 30% Employment Subsidy) England early 1980s Deadweight=63% 27% Substitution=10% Germany in mid 1970s (Wage Deadweight-75% 25% Subsidy Scheme) Ireland in the 1980's (Employment Deadweight= 70% 4% Incentive Scheme) Substitution=2 1 % Displacement=4% Netherlands during early 1980s Deadweight=25% 25% (Vermeend-Moor Act) Substitution=50% Netherlands during the late 1980s Substitution =80%. 20% (JOB scheme) Scotland 1989-1992 (Employment Deadweight=20%. 25% Subsidy) Substitution and Displacement=55% U.S. in mid 1980s (Targeted Job Tax Deadweight-'70% 20% Credit) Substitution= 10% Note: Additionality is the net employment effect after accountingfor deadweight, displacement and substitution effects. Source: Summarizedfrom Appendix 5.7 In summary, evidence indicates that these programs are unlikely to be effective; substantial deadweight and substitution effects are associated with them. Wage and employment outcomes of participants are also generally negative as compared to a control group. Careful targeting may be necessary to reduce 30 substitution and deadweight effects - e.g. unemployed individuals could be targeted to work in industries/occupations where there is excess demand. However, individuals may not have requisite skills that make them attractive to employers; otherwise, they would have been hired in the absence of the subsidy, too. Further, controls are necessary in wage subsidy programs to ensure that firms do not misuse this program. Employers may refuse to hire the unemployed unless they receive a large subsidy and, in effect, use the program as a permnanent subsidy to the workforce. Monitoring of employer behavior is thus necessary. 31 VI. CONCLUSIONS In this study we have examined close to 100 impact evaluations (some of which are summaries of a group of evaluations) - both non-scientific as well as scientific - of active labor market programs (training, public works, microenterprise development, job subsidies and job search assistance schemes) in an attempt to draw some lessons about the efficiency and cost-effectiveness of these schemes (Table 6.1). While the evidence covered is probably not exhaustive, this study has managed to look at most of the evidence available on different issues for these programs. Table 6.1 Evaluations of Active Labor Market Programs Type of Intervention No. of Non-scientific Experimental Quasi- Evaluations experimental Public Works 13 4 0 9 Job Search Assistance 18 1 6 11 Training Long-term Unemployed 23 4 6 13 Laid offen masse 11 5 0 6 Youth 7 0 5 2 MEDA 13 2 6 5 WES 1 5 8 4 3 Source: Appendices 5.1-5.7 We should mention a few caveats that apply to the evaluations examined: * First, even in the most thorough evaluations, issues relating to program design and implementation, staffing and other related institutional matters (i.e. process evaluations) have not been evaluated. These are important issues and will have an impact on the likelihood of the success of a program. * Second, due to time and cost considerations, few evaluations are longitudinal in nature. They therefore miss a crucial aspect of the program - its long-term impact. Thus, for example, questions about the persistence of effects for programs that are successful in generating employment and raising earnings in the short-run are rarely answered. * Third, cost-benefit evaluations usually fail to take into accountsavings from placements of those who obtain jobs (e.g., direct savings from reduced expenditures on unemployment benefits and social assistance schemes as well as indirect savings through societal benefits such as reduced crime, improved health outcomes etc.). While the paucity of rigorous evidence on the costs and effectiveness of active labor market programs does not allow a definitive conclusion as to whether such interventions can be justified economically, the general conclusion is that large scale application of these programs should be avoided without knowledge of their effects. We draw the following general lessons from the experience in OECD countries (lessons for each program are summarized in Table 6.2). Active labor market programs should take into account the general state of the economy and set the right objectives and expectations. One needs different programs if the cause of distress in the labor market is a cyclical downturn or a shock or a systemic transition: in most cases when great concern for labor is present, the cause is demand deficiency rather than seasonal, cyclical, frictional or even structural (say, lack of skills) factors (Sapsford and Tzannatos, 1993). And expectations should be modest: rarely can a 32 program meet the two strict criteria of economic policy (efficiency and distribution) and also political considerations at the same time. Sound impact evaluation techniques should be used to evaluate active labor market programs. Relying only on non-scientific evaluations may lead to incorrect policy conclusions. A good micro evaluation compares labor market outcomes for individuals who have gone through a particular program with those of a control group of their peers, and also utilizes data on program costs to attempt to answer the following questions: (a) what are the impact estimates of the program on the individual; (b) are the impacts large enough to yield net social gains; and (c) is this the best outcome that could have been achieved for the money spent. Table 6.2 Overview of Active Labor Programs Program Appear to Help Comments I Public Works Severely disadvantaged Long-term employment prospects not helped: program Programs/Public groups in providing participants are less likely to be employed in a normal job and Service temporary employment earn less than do individuals in the control group. Not a cost- Employment (13 and a safety net. effective instrument if objective is to get people into gainful evaluations) employment after program completion. 2 Job-search Adult unemployed Relatively more cost-effective than other labor market assistance/ generally when economic interventions (e.g. training) - mainly due to the lower cost, youth Employment conditions are improving; do not benefit usually. Difficulty lies in deciding who needs help Services (18 women may benefit more. in order to minimize deadweight loss. evaluations) 3. Training of long- Women and other These programs are no more effective than job-search assistance tern unemployed disadvantaged groups in increasing re-employment probabilities and post-intervention (23 evaluations) generally when economy earnings and are 2-4 times more costly. However, job search is improving, assistance may not be a direct substitute as it may cater to a different groups of the unemployed. 4. Retraining in the Little positive impact - These programs are no more effective than job-search assistance case of mass layoffs mainly when economy is and significantly more expensive. Rate of return on these (11 evaluations) doing better. programs usually negative. However, job search assistance may not be a direct substitute as it may cater to a different groups of the unemployed. 5. Training for No positive impact. Employment/eamings prospects not improved as a result of going youth (7 through the training. Taking costs into account - the real rate of evaluations) return of these programs both in the short as well as the long run is negative. 6. Micro-enterprise Relativelv older groups, Very low take-up rate among unemployed. Significant failure Development the more educated. rate of small businesses. High deadweight and displacement Programs (13 effects. High costs - cost-benefit analysis rarely conducted but evaluations) sometime show overall costs to UI budget higher than that of the control group. 7. Employment/ Long-term unemployed in Extremely high deadweight and substitution effects. Impact Wage subsidies (15 providing an entry into the analysis shows treatment group does not do well as compared to evaluations) labor force. However, no control. Sometimes used by firms as a permanent subsidy long-term impact. program. Source: Appendix 5.1-5.7 33 Evaluations show that it is extremely difficult to address problems of large scale unemployment through active labor market programs - these programs may work for specific groups under specific circumstances - hence, they have to be carefully targeted on the basis of evaluative evidence. It is clear that programs that work for one group in one country may not work for the same group in another country. It follows that these programs should not be examined in isolation, but in the context of the overall macro-framework as well as the institutional labor market framework. Evaluations suggest that programs are generally most effective when the economy is growing and the programs are well-targeted. This is not surprising, as labor demand is bound to be a crucial determinant of the success of any ALMP. However, even under the best-case scenarios, payoffs are usually modest and cost-benefit analysis suggest that social rates of return to effective programs are also sometimes negative. Some evidence seems to suggest that programs delivered by private providers are more effective than programs delivered by public providers or through NGOs Finally, if a country is going to institute active labor market programs, then evidence tells us that they should start off with modest programs, evaluate these programs rigorously using sound, scientific evaluation techniques, and then tightly target these programs to those for whom they are found most cost- effective. 34 VII. REFERENCES Addison, J. T. and W. S. Siebert (1994). Vocational Training and the European Community. Oxford Economic Papers, Vol. 46(4). Atkinson, A. and J. Micklewright (1991). Unemployment Compensation and Labor Market Transitions: A Critical Review. Journal of Economic Literature Vol. 29 (4). Alfthan, T. and B. Janzon (1994). Retraining Adult Workers in Sweden. Training Policy Study No. 3, ILO. Anderson, P. and B.D. Meyer (1994). Unemployment Insurance Benefits and Takeup Rates. NBER Working Paper No. 4787. Anderson, P., W. Corson and P. Decker (1991). The New Jersey Unemployment Insurance Re-employment Demonstration Project Follow-up Report. Unemployment Insurance Occasional Paper 91-I, U.S. Department of Labor. Bloom, H. (1994). The National JTPA Study: Overview of Impacts, Benefits and Costs of Title II-A. An Abt Associates Study. Bloom, H. (1984). Estimating the Effect of Job-Training Programs, Using Longitudinal Data: Ashenfelter 's Findings Reconsidered. Journal of Human Resources, Vol. 19(4). Bloom, H. (1990). Back to Work: Testing Re-employment Services for Displaced Workers Working Paper, W.E. Upjohn Institute for Employment Research. Card, D. and W. C. Riddell (1992). A Comparative Analysis of Unemployment In Canada and the United States. Working Paper No. 297, Industrial Relations Section, Princeton University. Christofides, L. and C. McKenna (1996). Unemployment Insurance and Job Duration in Canada. Journal of Labor Economics, Vol. 14(2). Christofides, L. and C. McKenna (1995). Employment Patterns and Unemployment Insurance. Human Resources Development Canada Working Paper. Clark, K. and L. Summers (1982). Unemployment Insurance and Labor Market Transitions in "Workers, Jobs and Inflation" ed. N. Bailey, The Brookings Institute. Classen, K. (1979). Unenmployment Insurance and Job Search in "Studies in the Theory of Search" ed. S. Lippman and J. McCall, North Holland Press. Corson, W., S. Long and R. Maynard (1985). An Impact Evaluation of the Buffalo Dislocated Worker Demonstration Program. Working Paper, Mathematica Policy Research. Corson, W., S. Dunstan, P. Decker and A. Gordon (1989). New Jersey Unemployment Insurance Re- employment Demonstration Project. Unemployment Insurance Occasional Paper 89-3. U.S. Department of Labor. 35 Cremieux. P., P. Fortin, P. Storer and M. Audenrode (1995). Unemployment Insurance and Job Search Productivity. Human Resources Development Canada Working Paper. Dar, A. and Z. Tzannatos (1998). World Bank Lendingfor Labor Markets: 1991-1996. Social Protection Discussion Paper Series No. 9801. The World Bank. Dar, A. and I. Gill (1998). Evaluating Retraining Programs in OECD Countries: Lessons Learned. The World Bank Research Observer, Vol. 13(1). Fay, R. (1996). Enhancing the Effectiveness ofActive Labour Market Policies: Evidence From Program Evaluations in OECD Countries. OECD Labor Market and Social Policy Occasional Paper No. 18. Forslund, Anders and Alan Krueger (1994). An Evaluation of the Swedish Active Labor Market Policy: New and Received Wisdom. National Bureau of Economic Research Working Paper No. 4802. Fretwell, D. and S. Goldberg (1994). Developing Effective Employment Services in "Equity, Efficiency and Adjustment in Labor Markets", World Bank. Discussion Paper No. 208 Fretwell, D., J. Benus and C. J. O'Leary (1998). Evaluating the Impact ofActive Labor Programs: Results of Cross Country Studies in Europe and Central Asia (forthcoming). Friedlander, D., D. Greenberg and P. Robins (1997). Evaluating Government Training Programs for the Economically Disadvantaged Joumal of Economic Literature, Vol. 35(4). Gill, I. and A. Dar (1995). Costs and Effectiveness of Retraining in Hungary. Internal Discussion Paper IDP- 155, Europe and Central Asia Region, The World Bank. Goss, Gilroy and Associates (1989). Evaluation of the Job Development Program. Final Report Report prepared for the Program Evaluation Branch, Employment and Immigration Office (Canada). Graves, F. and B. Gauthier (1995). Evaluation of the Self-Employment Assistance Program. An HRDC Working Paper. Gruber, J. (1997). The Consumption Smoothing Effects of Unemployment Insurance. The American Economic Review, Vol. 87(1). Gritz, R.M. and T. MaCurdy (1997). Measuring the Influence of 7Unemployment Insurance on Unemployment Experiences. Journal of Business Economics and Statistics, Vol. 15(2). Green, D. and T. Sargent (1995). Unemployment Insurance and Employment Durations: Seasonal and Von-Seasonal Jobs. Human Resources Development Canada Working Paper. Gueron, J. (1990). Work and Welfare: Lessons on Employment Programs. Journal of Economic Perspectives, Vol. 4(1). Harrell, A., M. Burt, H. Hatry, S. Rossman, J. Roth and W. Sabol (1996). Evaluation Strategies for Human Services Programs: A Guide for Policymakers and Providers. The Urban Institute, Washington D.C.. 36 Heckman, J. (1994). Active Labor Market Policies to Expand Employment Opportunities in "Reducing Unemployment: Current Issues and Policy Options: A Symposium sponsored by the Federal Reserve Bank of Kansas City" Heckman, J. (1992). Randomization and Social Policy Evaluation in C. Manski and I. Garfinkel, eds., "Evaluating Welfare and Training Programs", Harvard University Press. Hunt, J. (1995). The Effect of Unemployment Compensation on Unemployment Duration in Germany. Journal of Labor Economics, Vol. 13(1). ILO (various years). ILO Yearbook of Labor Statistics. Jackman, R. (1995). What Can Active Labour Market Policy Do? Center for Economic Performance Discussion Paper No. 226. Jackman, R. and H. Lehmann (1990). Labor Market Policies in the OECD Countries. London School of Economics Working Paper No. 13. Johanson, R. (1994). RetrainingAdults in Germany. Training Policy Study No. 4, ILO. Jones, S. (1995). Effects of Benefit Rate Reduction and Changes in Entitlement on Unemployment, Job Search Behavior and New Job Quality. Human Resources Development Canada Working Paper. LaLonde. R.J., (1986). Evaluating the Econometric Evaluations of Training Programs with Experimental Data. American Economic Review Vol. 76(4). Leigh, D. (1995). Assisting Workers Displaced by Structural Change: An International Perspective W.E. Upjohn Institute for Employment Research. Leigh, D. (1994). Retraining Displaced Workers: The U S. Experience. Training Policy Study No. 1, ILO. Leigh, D. (1992). Retraining Displaced Workers: What can Developing Nations Learn From OECD Nations? WPS 946, The World Bank. Lehman, Hartmut (1995). Active Labor Market Policies in the OECD and in Selected Transition Economies. Background Paper for the 1995 World Development Report. The World Bank. Mangan. J. (1988). Wage Subsidies for the Disabled: A Discussion of Their Impact in Australia. International Journal of Manpower, Vol. 11 (1). Meager, N. and C. Evans (1998). The Evaluation of Active Labor Market Measures for the Long-Term Unemployed. Employment and Training Papers No. 16, ILO, Geneva. Meyer, B.D. (1995). Lessons from US. Unemployment Insurance Experiments. Journal of Economic Literature, Vol. 33. Meyer, B.D. (1989). A Quasi-Experimental Approach to the Effects of UJnemployment Insurance. NBER Working Paper No. 3159. 37 Moffitt, R. (1985). Unemployment Insurance and the Distribution of Employment Spells. Journal of Econometrics, Vol. 28. NERA (1995). OECD Wage Subsidy Evaluations: Lessons for Workstart, Report for the Department of Education and Employment by NERA. Working Paper. New Zealand Department of Labor (1995). Evaluation of Job Action. New Zealand Employment Service Report. OECD (1997). OECD Employment Outlook OECD (1994a). The OECD Jobs Study: Evidence and Explanations. OECD (1994b). Vocational Training in Germany: Modernization and Responsiveness. OECD (1994c). Vocational Training in the Netherlands: Reform and Innovation. OECD (1993). Employment Outlook. OECD (1991). Evaluating Labor Market and Social Programs: The State of a Complex Art O'Leary, Christopher (1995). An Impact Analysis of Employment Programs in Hungary. W.E. Upjohn Institute for Employment Research Working Paper 95-30. O'Learv, Christopher (1998(a)). Evaluating the Effectiveness of Active Labor Programs in Hungary. W.E. Upjohn Institute for Employment Research Draft Working Paper. O'Leary, Christopher (1998(b)). Evaluating the Effectiveness of Active Labor Programs in Poland. W.E. Upjohn Institute for Employment Research Draft Working Paper. O'Leary, Christopher (1998(c)). Profiling for Reemployment Bonus Offers. Employment Research. W.E. Upjohn Institute for Employment Research. Orr, L., H. Bloom, S. Bell, F. Doolittle, W. Lin and G. Cave (1996). Does Training for the Disadvantaged Work? Evidence From the N,ational JTPA Study. The Urban Institute. Sapsford, D. and Z. Tzannatos (1993). The Economics of the Labor Market. The Macmillan Press, U.K.. Schmid. G., C. Buchtemann, J. O'Reilly and K. Schomann (1996). International Handbook of Labor Market Policy and Evaluation. Edward Elgar Press. Skedinger, P. (1995). Employment Policies and Displacement in the Youth Labor Market. Swedish Economic Policy Review, Vol. 2(1). Subbarao, K. (1997) Public Works as an Anti-Poverty Program: An overview of Cross-Country Experience. American Journal of Agricultural Economics. 38 Tzannatos, Z. (1995). Labor Policies and Regulatory Regimes in "Regulatory Policies and Reform: A Comparative Perspective" (ed.) Claudio Frischtak. A World Bank Publication. Wilson, S. and A.V. Adams (1994). Self-Employment for the Unemployed: Experience in OECD and Transitional Economies. World Bank Discussion Paper No. 263. Winter-Ebmer, R. (1996). Benefit Duration and Unemployment Entry: Quasi-experimental Evidence for Austria. Center for Economic Policy Research Discussion Paper No. 1521. 39 Appendix 5.1 Public Works Schemes Labor Market Problem Relevant Indicators Intervention Design Type of Result Comments Evaluation Ongoinig schemes in Changing over time due to Individuals placed in short-term Quasi- Displacement effect in Long-term evaluation Sweden to provide shor-t- large tiinc period studied. employment in the construction, experimental construction sector is using data from 1976 terim1 employmiienit and healtlh and welfare sectors. 69%. Displacement to 1990. At best, safety net to the effect in healthi and programs create no uniemployed (Forslund and welfare is insignificant. displacement. No data Kriueger, 1994) on costs. No evidence on value of assets created. Providinlg short-term Unemploymiient rate was Individuals placed in short-term Quasi- P ositive impact on employmiienit to the fairly steady between 3- employment. experimental subsequent employment unlemployed in Austria in 4%. and income levels the early 1990s (Meager compared with control and Evans, 1998) group. Short-terimi employment Denmark spent about Individuals provided with shiort- Non-scientific Likelihood of leaving Participants finding provided to the 0.2% of GDP on these term employment as well as unemployment peaks subsequent jobs kept disadvantaged in Denmar k programs. Unemployment subsidized jobs. after participation in them longer than non- in the late 1980s (Meager rate around 7% temporary jobs. participants. and Evans, 1998) I-lowever such effects are not strong enough to compensate for reduced employment impact during period of scheme. Unemployed individuals Unemilploylmienit ratcs over Various tempor-ary employmenit Quasi- After controlling for If these differences provided short-term i 5 percent. Expenditurle schemes provided. experimenltal unobser ved differences, were not controlled employmilenit in Ireland in on these programs rose short-tenn and long- for, the short-term and the mid 1980s (Breen, fiom 0.09% of GDP in the term impacts on long-term employment 1991) mid 1980's to 0.25% by employment are not impacts are positive. 1990. significait. 40 Appendix 5.1 (continued) Public Works Schemes Labor Market Problem Relevant Indicators Intervention Design Type of Result Comments Evaluation Unemployed individuals Unemploymilent rates rose Employment Action programs. Quasi- Reduced probability of getting provided temporary from 7% in 1990 to over 10% experimental a job after one year (by one employment in the U.K. in in 1993. percent), increased probability the early 1 990s after three years (by 4%) Individuals provided I'ime period not known Public works relief program Quasi- Participants more likely to Some selection bias temporary employment in experimental remain in unemployment as likely. Sweden (Meager and compared to non-participants Evans, 1998) Ongoing schemes in Changing over time due to Youths placed in short-term Quasi- Displacement effect is 100%. - Long-term evaluation Sweden to provide short- large time period studied. etnployment in different sectors of experimental a one percent increase through using data fiom 1976 to term employment and the economy. job creation schemes leads to a 1990. No data on costs. safety net to the young one percent decline in regular No evidence on value of unenmployed (Skedinger, youth employment. assets created. 1995) Provide employment for Unemployment rates at Adults unemployed over a year and Non- Positive short-term effect but Participants became long-term unemployed around 5% and rising. Finland youth unemployed over three scientific increased re-inflow into unemployed once again adults and short-tenn spends about 0.4 percent of its months placed in public work unlemployment in the longer after the program was unemployed youth in GDP on public works schemes. term. completed. Finland in 1987 (OECD, schemes - more than on any 1993) othel- active labor programs. Aid unemployed gain Unemployment rates were NA Non- Significant increases in flow Participants became short-ternm employment in steady in Germany at around scientific out of short-term unemployed once again Germany during 1987 and 7%. unemployment. However no after the program was 1988 (OECD, 1993). significant impact on long-term completed. unemployment. 41 Appendix 5.1 (continued) Public Works Schemes Labor Market Problem Relevant Indicators Intervention Design Type of Result Comments Evaluation Provide employmilent for Unemploymiienit rates fell from Long-term unemployed Non- Deadweight effect=15% and the long-term unlemployed about 13 percent in the early provided permanent jobs in the scientific substitution effect=15% in Ilolland in the late 1980s to below 10 percent by the municipal governments. 1980's (OECD, 1993) endl of thie decade. I olland spent less than 0.02 percetnt of GDP on these programs. Aid displaced and long- Employmcnt had fallen by over Individuials employed in public Quasi- Matchied pairs estimates show Evaluation done in three ternm unemployed workers 20% dur ing 1989-92. service employmenit i.ncluding experimental that, as compared to a control counties. Using cost to gain temporary UJnemploymenit rate r ose from 0.5 maintenance of public group, participants are 50% less data along with employment in Hunigary in in 1989 to 8% by 1992. facilities, assistance to social likely to be in a normal job after effectiveness data shows 1992-93 (O'Leary, 1995) welfare agencies. participating in these programs. that these programs are Earnings are not significantly extremely cost- different though. ineffective. Assist displaced and long- Sharp rise in unemployment: from Individuals employed in public Quasi- After participating in the Evaluation done in 10 term unemployed workers below one percent at the turn of service employment including experimental program, individuals are 29% counties. Using cost data to gain temporary the decade to 12 percent by 1994. maintenance of public less likely to be employed in a along with effectiveness employment in H-lungary in After falling continuotisly for five facilities, assistance to social nonlal job and earn $15/month data shows that these the mid 1990s (O'Leary, years, real GDP started growing welfare agencies. less than those in the control programs are extremely 1998(a)) slowly. Of spending on active group. cost-ineffective. programs - 15 percent spent on public works. Aid displaced and long- GDP started growing slowly from Individuals employed on Quasi- After participating in the Evaluations done in 8 term unernployed workers 1994 but untemilployment rate rose projects organized by experimental program, individuals are 8% Polish voivods. Using to gain temporary fiom 0% in 1989 to 16% by 1994, government agencies including less likely to be employed in a cost data along with employment in Ploland(l in thoughi it has (leclined since. In minicipal governments. nornal job. No significant effectiveness data shows the mid 1990s (O'Leary, 1994, 35% of expenditures on difference in earnings. that these programs are 1998(b)) active labor programs were cost-ineffective. expended on public works. 42 Appendix 5.2 Job Search Assistance Labor Market Problem Relevant Indicators Intervention Design Type of Result Comments Evaluation Help place job seekers Unemployment remained Canada's National Employment Non-scientific No measurable impact of 72 districts suiveyed. Job through counseling high but fell steadily from Service assists individuals (not agency on employment in the search services considered to assistance in Canada 12% in 1983 to 10% in 1987. necessarily unemployed) in labor market at large, nor on have improved equity as the during mnid 1980s (OECD, finding jobs. job-finding for persons whose most disadvantaged benefited 1993) general situation was more. relatively good. However, a relatively positive impact in communities that were the hardest hit. Help place job seekers Tine period not known NA Quasi- Positive impact on subsequent Follow-Up conducted up to throughl counselinig in experimental employment and income two years after program Austria (Meager and levels as compared to participation. Evans, 1998) controls. Job seekers provided job Time period not knowin Public employment services Experimenital Participanits 1 1% more likely search assistance in provided intensified job search to be employed as compared Sweden (Meager and assistance through counseling to control group and also Evans, 1998) and interviews. earning higher wages. Help unemploymenit Proportion of long-term U.K. Restart Program. Benefit Experimental Different evaluationis point to Net impact not clear as many benefit claimanits get untemployed in total claimants interviewed assistance positive short run impacts exits were not into information on unemployed declined from given every six months to (treatment group found unsubsidized jobs but into employmenlt or migrate to about 45% to 30%. provide positive hielp and also employment 55 days quicker other government subsidized other programs in the U.K. Unemnployment rates fell act as a gateway to other on average). Long-tern programs. Likelihood of in late 1980s (Meager and from 12% in 1986 to 6% in government schemes. effects are weaker withi ranldomizationi bias. Evans, 1998) 1990. 0.1 % of GDP spent on Attendance compulsory, with participants no more likcly to these measLires. sanctions imposed on get stable jobs (as compared individuals not deemed to be to control). actively seeking work. 43 Appendix 5.2 (continued) Job Search Assistance Labor Market Problem Relevant Indicators Intervention Design Type of Result Comments Evaluation Unemploymenlt insurance Individuals provided job Quasi- Farnings gains lasted for up to claimants, particular ly search assistance with the experimenital three to four years, but single motlhets provided threat of compulsory uLipaid paiticipants did not enjoy job searchi assistance in the work expericnce. greater job security than U.S. in the late 1980's contr-ol group. (Balakrishlian, 1998) Long term unetimployed Propor-tion of long-term Individuals unemiiployed and Experimental Likelihood of findinig work Tiue effects may be provided job counlselilng il unemployed in total claiming unlemploymenlt improved but may come at understated as 25% of the U.K. in the late 1 980s ullemilployed declined from benefits for at least six moniths the expense of those who are control group were also (OECD, 1993; Jackman about 45% to 30%. provided job counseling. the shiort-term unemployed. provided some job and l.ehmann, 1990) Unemployment rates fell counseling. from 12% in 1986 to 6% in 1990. 0.1%ofGDP spent on these measures. IProvide job counseling and Unemployment rates Clients of AFDC (Assistance Experimental Significant impact on Program is less usefild for searcih assistance to women remained between 6-7% in for Families with Dependent employment and earnings. those with recent work on welfare to get jobs in the U.S.. 0.08% of GDP Children) - mostly lone Participants 4% more likely to experience as well as those the U.S. in the mid 1980s spent on these programs. mothers - were provided job be employed and earn over who had been on welfare for (Gueron, 1990) counseling and job search 20% more. a long time. workshops and (in some cases) some work experience Individuals in Australia Unemployment rates in Individuals are assisted in Quasi- Participants were II percent Evaluation done I year after assisted in their job searchl Australia were risinig to over rcsuimiie wr itinig anid experimnenital morc likely to be employed. individuals had entered job thiougig job clibs in 1992 1 )% over this perio(l. interviewinig techliniquies as well I'his service was more club. Those who had gone (Fey, 1996) Austr alia spent about 0.09% as practical matters - provision beneficial for less educated throuigh other labor market of GDP on these measur-es in of stationery, postage etc. males, intervenitionis prior to joininig 1992. in the job club did not benefit. 44 Appenidix 5.2 (continiued) Job Search Assistance Labor Market Problem Relevant Indicators Intervention Design Type of Result Comments Evaluation Assisting individuals laid After ycars of decline, a The Industrial Adjustment Quasi- Those who availed of the Program hindered clients in off as a result of industrial sharp r ise in unemployment Scrvice. Job counseling experimental service spent more time their job search withi no restructur-inIg in Canada durinig this period firom 8% provided to those laid off en searclinig for jobs (12 weeks) positive benefits. 1989-1991 (Fay, 1996) to 11%. By 1990-91, Canada imiasse. Other services e.g. and lost income ($7200). was spendinlg about 0.2% of retraining were also provided. GDP of employmilenit services and their administration. Assisting youth in After years of decline, a Job search assistance among a Quasi- No impact on raising Program did have a positive increasing their sharp rise in unemilployment variety of services provided to experimental probability of employment or impact in raising education employability in Canada during this period from 8% help disadvantaged students in earnings or participants as level of participants (0.3 1989-1992 (Fey, 1996). to I 1%. Youth their school to work transition. compared to control group. years). unemployment rose from 12% to 16% during this period. Assisting long-term Dutch Unemployment rates falling Persons who had been Quasi- Small improvement in No information available on ullemployed through job steadily from 8% to 5% interviewed for three years or experimental probability of employment (2- earnings gain. counseling services in the during this period. more were given job 3%). mid to late 1980s (OECD, reorientation 1993) interviews/couniseling. Scheme to help youth find Unemployment had fallen Provision of counseling and Quasi- No impact on the probability Program did not have any employment in Holland in from 9% in the mid 1980s to job search assistance for school experimental of findinig ajob or increasing impact on likelihood of the early 1 990s (Fey, 1996) 6% by 1991 but rose again to leavers upto 20 years old. earnings for the treatment participants enteritng 9% by 1994. Youth group as compared to the education or training either. unemployment remained control group. fairly steady at around 11% during this period. 45 Appendix 5.2 (contitiued) Job Search Assistance Labor Market Problem Relevant Indicators Intervention Design Type of Result Comments Evaluation Schcme to help youth find Unemploymiienit had fallen Provision of counseling and Quasi- Counseling offered to youtil No inforimiatiotn on earnings eimploymilent in I Holland in from 9% in the mid 1980s to job search assistance for school experimental had little impact on job gain for participants. thc early 1990s (Fey, 1996) 6% by 1991 buti ose again to leavers upto 20 years old chances. Temporaryjob 9% by 1994. Youth followed by subsidized placement (subsidized by unemployment remained temporary work. government)appeared to lead fairly steady at around 11% to regular employment gaiis duLrig this period. of 20%. Assisting the very long Over 30% of tile For individuals employed for Quasi- A moderate increase in full- Evaluationi done five montihs ter-im unem)ployed gain unlemployed (10% of labor at least two years. Consisted experimenital time work (5%). after individuals joined employmtieint in New force) are long-term of four elements: a job program. No informationi on Zealand in 1994 (NZ DOL, unemployed. screening interview, a one wage gain of participants 1995) U]nemilploymilenit rates were week workshiop, a follow up relative to control. fairly steady. Expenditures interview and case on thcse programs were managemeit. about 0.12% of GDP. Assisting AFDC Unemployment rates Participants - mnainly women Experimental Participants earn 4% more Cost of training and job- (Assistance to Families remained between 6-7% in with young children - provided than the control group. search assistance combined is with Dependent Childreni) the U.S.. 0.08% of GDP trainiing in various skills. They $16,250 per recipient (most of participants gain spent on these programs. also attend job searcil clubs for the cost is for training), employable skills in the two weeks and then search for making the program highly U.S. il the 1980s jobs. cost-ineffective. (Heckman, 1994) Get people off Unemliploymilenit rates Vayinig lengthi job searcih Experimental Significant decline in number Experiments lasted from 6-12 unecimip loymcnt benefits rein-ei a icd between 6-7% in seminars. The sem ioiar-s anid of weeks of unemilploytimenit motithis after whichi evaluatio and into employment in the l.S.. 0.08% of (Dl)P personial visits with job searchi bonis claimed (between one was conducted. Cost-benefit Charleston, New Jersey, spent on these programs. counselors exceed that of the and four weeks). Slight analysis tend to show that the Washington, Nevada and control group. increase in re-emiiploymenit Ul system bemiefited fiom Wisconisini in the 1970s and earninlgs. these intervenitionis. tile 1980s (Meyer, 1995) 46 Appendix 5.2 (continued ) Job Search Assistance Labor Market Problem Relevant Iiidicators Intervenitioni Design Type of Result Comments Evaluation Job search assistance Sharp rise in unemployment: UJnemployed provided a full Quasi- Participants seven percent Evaliation done in 10 provided to the fiom below one percent at r ange of placement services experimental more likely to be employed. counties. No significant uniemlployed in Hungary in the tiril of the decade to 12 including job referrals, However, no impact on difference in duration or the mid 1990s (O'Leary, percent by 1994. After counseling, skills assessment, earnings. Women benefit amount of unemployinent 1998(a)) falling continuously for five job search training, resume more thanl men. benefits received. years, real (iDP started preparation and job clubs. growinlg slowly. 1-lungar-y spent 0.15% of GDP on these programs. Job search assistance GDP started growinig slowly Unemployed provided a full Quasi- Participants are 15% more Evaluation done in 8 counties. provided to the from 1994 but range of placement services experimental likely to be employed than Treatment group was unlemiiployed in Poland in unemployment r ate rose including job referrals, control group. They earn unemployed for almost 0.5 the mid 1990s (O'Leary, from 0% in 1989 to 16% by counseling, skills assessment, about $5 month more months more and drew $24 1998(a)) 1994, though it has declined job search training, resume (significant). Women benefit more in unemployment since. Poland spent 0.01% preparation and job clubs. more than men. compensation than control of GDP on these programs. group 47 Appendix 5.3 Training for the Long-term Unemployed Labor Market Problem Relevant Indicators Intervention Design Type of Result Comments Evaluation Inprovinig skills and thus Unemploymenit rates fell in Training provided hands on Non-scientific In 1991, 82 individuals were Training was quite expensive provide better access to Ilolland since thc mid 1980's experience through a enrolled in the course. 52 as annual funding of the jobs for tlhe long-terimi and were arounid 7 percenit in simulated workshop. completed and 41 went onto program was about uLlemilployed in 'I'ilburg, 1992. Emphasis was placed on a job or further education. $7,500,000 ($ 10,000 per Nethcllanids since the technical skill developincnt Metal works program is the trainee). carly 1980's (OECD, as well as well as instilling most successful in placing 1993a). good work habits. Four students. main skill areas were: metal training, wood trainiing, installationi techniques and clothing. Courses ran from 4-10 months. I ligh regional In 23 of the 142 regions in Among other interventions, Non-scientific Training reduced In light of the extremely higi unlemploylmlelnt levels in Germany unlemployment training and retraining of unemployment somewhat. cost of training (around $500 Germtiany in the late 1970s levels were above 6 percenit. unemployed in firms were However it is estimated that million for training and other (OECD, 1991), Some firimis in this regioln provided. The firms wlho by 1981, over 40 percent of interventionis), results are were also facing serious employed these trainees the hard-to place individuals very disappointing. No data problems in ad justing to received a wage subsidy of had already left their jobs. on wages. economic chaniges. 90 percenit of wages for 24 months. Provisioni of skills to the Australia's unemployment rate Individuals provided with Quasi- Individuals 12 percent more There is a possible selection unemployed in Australia was declining from 8% in classroom and on-the-job Experimental likely to be employed than bias in this study. Training in the late 1980s and early 1986 to 6% in 1988 but then training control group. However, is less effective than other 1990s through the Jobtrain rose sharply again to around impact dissipated rapidly - interventions - e.g. job clubs. programn (McIagcr and 10% by 1991. after five months No evidence on program Evans, 1998) unemploycd participants costs. were as likely to find job as individuals in control group. 48 Appendix 5.3 (continued) Training for the Long-term Unemployed Labor Market Problem Relevant Indicators Intervention Design Type of Result Comments Evaluation Training provided to the Ireland had the highest Various short-term Quasi- Participants are more likely If unobservable unemployed in Ireland in unrenployment rate among trainin1g measures experimental to be employed in short- characteristic differences the mid 1980s (Breeni the OECD countries during provided to the term (16%). Aftcr one year, are controlled for, 1991) this period (averaginig about unlemployed. participants only 7% more participants no more likely 15%) likely to be employed. to be employed. Traininig provided to the Time period not known. Labor market training Quasi- Short-term positive impact Gains dissipate over time. ullemployed to enhance provided to the experimenital on earnings - 3% hiigher their skills in Sweden oniemployed. than control group after six (Meager and Evans, 1998) months; hiowever, long-terin impact is somewhat negative (after two years). Unemployed individuals Time period not known. Employment Training on- Quasi- Significanitly increased provided with traininlg to the-job experimental probability of getting a job enhance skills in the U.K. by 3% after a year and 22% (Meager and Evans, 1998) after 3 years. Provision of skills to long A decline in petrochemical Texas WAD project. Experimental By end of first year after Despite high costs of terml unemployed workers industry led to layoff of Displaced workers were participation, no additional classroom training (twice as in Houston, Texas (1983- workers. In the U.S., provided with JSA or a earning gain accrued from much as JSA) no additional 85). Individuals eligible unemploymenit rates declined mixture of JSA with classroom training as gains accrued from this type were unlemploymenit from 9.5 percent in 1983 to classroom training. compared to the JSA only of training. henefit recipients who had 7.4 percent in 1985 (and sample. a low probability of maintained this trend till the returning to their previous late 1980s). Employment in occupation or industry this industry increased (Bloom, 1990). slightly between 1983 and 1985 49 Appendix 5.3 (continiued) Training for the Long-term Unemployed Labor Market Relevant Indicators Intervention Design Type of Result Comments Problem Evaluation Provision of skills to In the U.S., unlemiploymlient Texas WAD project. Experimental By end of first year after Increase in earnings for long term-i unemployed rates declined from 9.5 percenit Displaced workers were participation, while this women slightly exceeded workers in El Paso, in 1983 to 7.4 percent in 1985 provided with JSA or a program had no effect on program costs. However no TIexas (1983-85). (and maintained this trend till mixture of JSA with male earnings it had a positive beneficial effects for males. (Criteria same as above). the late 1980s). Employment in classroom training. effect on increasing earnings Workers laid off from manufacturing fell by close to 2 for women. light m alnufacturing percent between 1982 and 1983 plauts (Bloom, 1990). but rose by about 5 percenit till 1985. I lelp the long term Unemployment rates in the New Jersey Ul Re- Experimental Focusinig only on those who Tl he tollowing caveats apply unenmployed in New U.S. fell from 7.1 percent in employment Demonstrationi undertook training, over 10 - these results may not J.ersey find cmployment 1985 to 6.1 percent by 1987. project. Displaced workers quarters after the program, apply for a random group of (1986-87). In general, While employment in were provided with a both CT and OJT significanitly claimants; only 15 percent the unemployed were manufacturing remnained fairly mixture of JSA, JSA increased earnings relative to of those offered trainintg laid off from steady between 1 985 and 1988, followed by on-the-job JSA recipients only. These accepted it; training manufacturing, trade trade employment increased by training (OJT), JSA followed individuals were also benefited those who already and services (Corson et. 7 percent and that in services by classroom training (CT) employed for greater amounts possessed marketable skills. al., 1989; Andersoni, by about 5 percent over this or JSA followed by re- of time/quarter as compared Cost-benefit analysis show Corson and Decker, tine period. employment bonus. to the JSA-only group. that costs exceed expected 1991). benefits. Increase thc As compared to the U.S., The job development Quasi- Employability of women goes In view of high program employability and unemployment rates have been program has been in experimetntal up while that of males cost (around earninigs of the long fairly highi in Canada. I-lowever operation since 1985. declines. Weekly earnings for $9300/participant) training term unemployed in they declinied from I i .2 percent Formal and on-the-job females are insignificant is not cost effective, Canada. (Goss, Gilroy in the mid 1980s to 7.5 percent training are provided to tlhe relative to control group while especially for men. et. al., 1989). in 1989. Spuired by a growth of unemployed. Wages and they are lower for males. commercc and services, direct cost of classroom employmenit grew by about 3 training are subsidized. The percent annually over the time wage subsidy helps period. employers cover the cost of on the job training. 50 Appendix 5.3 (contitnued) Training for the Long-term Unemployed Labor Market Problem Relevant Indicators Intervention Design Type of Result Comments Evaluation Assist individuals either Unemploymenit rates Training individuals to be Non-scientific Results measured in terms of No data on cost available. at the risk of becoming r emained fairly steady at certifiable in one of the retention rates (those who Dropout froni employment unemployed or those who just over 10 percent in the 375 apprenticeable trades. complete the course); pass was fairly high - two years were unemployed in early 1980s. The program took up to rate (those who pass exam); after completing training Germany in the early two years. Contents and and employment rate (those only 60 percent of men and 1980s (Johanson, 1994). specification of retraining who find jobs in occupations 66 percent of women were corresponded to those of for which they are retrained). still employed. initial vocational training. T hese rates were 70%, 85% and 90% respectively. Thus the overall success rate was 54 percent. Employment rates varied by age - for those over 45 unemployed over a year it was below 50 percent while for those 25-35 it was 86 percent. Individuals at the r isk of Uneinploymenit rates were This is an evaluation of Quasi- No type of training has any No information on wages or being unemployed in steady in Germany between four programs, two of experimental significant impact on the costs of training is available. 1987-1988 in Germany 1986 and 1988. They fell whichi are training (one flows out of short-terim or (Johanson, 1994b) somewhat by 1990. offering further training long-term unemployment nor Employment grew at for employed and on the flows into slightly over I percent unemployed individuals unemployment. during this period. while the othier offered retraining for the unemployed). 51 Appendix 5.3 (continued) Training for the Long-tcrm Uneniployed Labor Market Problem Relevant Indicators Intervention Design Type of Result Clomments Evaluation Aid long-teriml untemnployed Unemiiploymiletit rates in The Trade Adjustnent Quasi- Individuals who received Analysis was only done for workers in the U.S. to find the U.S. fell from 7.1 Assistance Program was experimental training began earning the manufacturing industry. jobs (1988). Most workers percent in 1985 to 6.1 intetided to develop skills in significantly more (as While investment in were been laid off due to a percent by 1987. While occupations different from compared to those who training may have been plant closing (Leigh, 1992). employment in occupations in pre- received extended incomne- effective in increasiig manlfacturing remained displacement jobs. Most of maintenanice benefits) by the earnings, the training was fairly steady between these skills were supplied by 6th quarter and this difference costly (each trainiee was 1985 and 1988, trade a vocational college or local continued to increase till the given a $12000 training employment increased community college in 12th (last) quarter reaching voucher). by 7 percent and that in courses that were over a year $500 per qtiarter. services by about 5 long. percent over this time period. Assist long term unemployed Unemnploymnent rates fell Long-term AFDC recipients Experimental On average, participants Cost of program over and welfare recipients gain sharply in the U.S. and ex-offenders were earned $420 more/annum than $13,400 per participant. employment in the U.S. between 1975 and 1978. provided training as well as control group a year after Hence, real rate of return between 1975-78 government-subsidized work participating in the program. even if mean effect lasted (Friedlanidet, 1997) experience in enterprises in 10 years was negative. this demonstration project. Between the early 1960s and 'I'ime period too long. P'articipants provided with Quasi- On average, male participants Evaluation done two years thei mid 1970s, classroom training and on- experimental earned $150 more per annum after participation. Cost of disadvantaged adults the-job training. as compared to control group. training per participant was provided with training to over $6000. If mean effect enhance employability in the lasts 10 years, real rate of U.S. (Bloom, 1984) ieturnl will still be negativc. 52 Appendix 5.3 (continued) Training for the Long-term Unemployed Labor Market Problem Relevant Indicators Interventioin Design Type of Result Comments Evaluation I lelp disadvantaged and After falling in the late 1970s, Participants provided with Quasi- Males earned significantly Evaluation done two years long-terlm unemiiployed unemployment rates rose to on the job training, experimental less (over $500/annum) after participation. Cost of adults gain employment 10% by 1982. classroom traiing, compared to control group. training about between 1974 and 1983 government subsidized work However, women earned $9000/participant. If in U.S. under the experience as well as public $1800/annum more. effects persist for 10 years, Comprehensive service employment (public rates of return will remain Employnment and works programs). negative for males. Training Act (CETA) However positive rate of (Friedlander et. al., 1997) return (15%) for women. I lelp disadvanitaged Time period too long. National in scope. Experimental Both men and women Evaluation done two years adults gain employable Participants provided with increased earnings after participation. Cost of skills in the U.S. 1983- classroom traininlg and on- significantly (close to training was around $1200. present unider the Job the-job training. $1 000/ann1umi). Very higlh rate of returm for T raining Placement Act both men and women (over (JTPA) (Friedlander, et. 50%) if effect persists for al, f 997) over 10 years. On the job training was significantly more cost-effective than classroom training. I lelp miniority women There was a shar p decline in Demonstrationi program Experimental Participating women earned Evaluation done two years gain skills under the the unemployment rate from under which participants (all over $700/annum more on after participation. Training Minority Female Single 9.5% to just over 5% durinig womnen) were provided average than non-participating was fairly expensive Parenit (MFSP) program this time period. classroom training as well as women. However, this effect ($6,000/participant) and between 1982-1988 in the on-the-job training. is not significant. hence in the short-run real U.S. (Friedlander et. al., rate of returni was negative. 1997). However, slightly positive rates of return if effect persists at least 10 years. 53 Appendix 5.3 (contitiued) Training for the Long-term Unemployed Labor Market Relevant Indicators Intervention Design Type of Result Comments Problem Evaluation I lelp long term iHigh uLnemployment rates Comiiimunity program. Non-scientific Little impact on flows fiom Cost-effectiveness of these uelmiiiployed and (around 10-1 1 percent) 'I'argeted vocational training long-term unemployment. programs was likely to be displaced workers gain persisted thiougil muchi of the was provided by local negative. cmploymcnt in Einglandl 1980s. Between 1983 and authorities in conjunction in 1980's (Addison and 1990 employmilenit has r isen by with local colleges, central Siebert, 1994). around 1.5 percent. government and voluntary organizations. There was low private sector involvement. I lelp adult uLnemllployed tJenemploymenlt rates fell from Centers for adult vocational Quasi- Unemployed persons who No data is available on and displaced workers about 13 percent in the early training provided training to Experimental did not undergo training wages and costs. gain better employmiienit 1980s to below 10 percent by these workers. found jobs as quickly as However, in terms of opportunities thlroughl the end of thc decade. those who did. Even two employability, training access to training in years down the line the seems to have been Holland (OECD, employment situation for ineffective. 1994c). the two groups was not significantly differenit. Assist unlemploycd and at Employmenit had fallen by over Largely classroom based- Quasi- Training raises the Training costs are risk workers gain skills to 20% during 1989-92. instruction experimental probability of findinig $900/participant. It would become more Uncmploymilenit rate rose fiom employment by six percent. take over 30 years to employable in Hungar y 0.5 in 1989 to 8% by 1992. Earninigs of participants no recover costs of program in the early 1 990s hiigher thani those of the based on reasonable (O'Leary, 1995; Gill and control group. assumptions about the Dar, 1995). durability of program effects. 54 Appendix 5.3 (continued) Training for the Long-term Unemployed Labor Market Problem Relevant Indicators Intervention Design Typevof Result Comments Evaluation Assist displaced and long- Sharp rise in unemployment: On the job skills training. Quasi- Trainees 10% more likely to be Evaluation done in 10 term unemployed workers from below one percent at the Participants receive a stipend experimental employed than the comparison counties. These results to gain skills in Hungary turn of the decade to 12 percent that has a 10% premium over group and earn significantly pertaini to individual in 1996 (O'Leary, by 1994. After falling the unemployment . more. Women perform retraining and not group 1998(a)) continuously for five years, r eal compensation benefit. significantly better than men. retraining. No cost-benefit GDP started growing slowly. analysis. Aid displaced and long- GDP started growing slowly On the job skills training. Quasi- Trainees are 10% more likely Evaluations done in 8 Polish term unemployed workers from 1994 but unemployment Participants receive a stipend experimental to be employed and earn more voivods. From a societal to gain skills in Poland in rate rose fiom 0% in 1989 to that has a 15% premium over than the comparison group. perspective, training is not the mid 1 990s (O'Leary, 16% by 1994, thoughi it has the unemployment Women benefit more than cost effective (net benefit to 1998(b)) declined since. compensation benefit. men. I'rainces have reccived society is -140 dollars). close to $40 more in unemployment compensation than the control group. 55 Appendix 5.4 Retraininig in Cases of Plant Closures & Mass Layoffs Labor Market Relevant Indicators litervcntion Design Type of Result Coniments Problem Evaluation Ford Plant ClosuL-e A 25 percent declinie in auto Basic skills training as well as Non-scientific I-ligh success rate in Causes for perceived success in San Jose, U.S. produIctionl between 1978 andt targeted vocational traininig in placement. were - adequate resource base (1982). 2400 1980. Unemployment rates in marketable skills. ($6000 grant/worker); high workers lost their the U.S. rose from 7.5 percent in degree of coordination and jobs (OECD, 1993a) 1981 to 9.5 percent in 1982 and assistance provided by Ford 1983 andc manufacturing and government. employment declinied by 5 percent duirinig 1981-83. Shipyard closure in High unlemilploymenlt rates in Program in training women Non-scienitific 51 businesses were set This progratn did not seem to Storstrom county, regiois, especially among entrepreneurs ( 1986-89) to help up by 1989. Less than have met with muchl success Dienmark (1986). women. National them start their own business. A a third of participants and was apparently quite 2000 people lost uiieniployment rates for men in total of 200 houirs of introductory opened uip a full-time costly (precise cost estimates their jobs (OLCD, 1986 and 1987 were 6.1 and 6.4 and specific business oriented business and few hired unavailable). 1 993a). per cent respectively while the training were provided. any employees, thus corresponding numbers for generating low women were 10.0 and 9.6 additional percent. F.mployment in employment. maniufacturing remained stagnanit since the mid 1980s. Steel and Coal plant Contracting steel sector. In Workers received 70 percent of their Non-scientific High success rate in No evidence of long term closure in Creusot- mininig industry, cmployment forner salaries for 10 months during placement. employment impact of Loire, France has fallcn steadily throughout which they participated in retraining program participation. (1984). 1230 people the 1980s - by close to 40 and job-search activities after which becamne uLniciiployed percent between 1980 and 1990. thiey wcre promised re-employmncit almost immediately UJinemploymcint rates rose tiom in differenit firimis in thic regioni. (OECD, 1 993a). 8.1 perccit in 1982 to 10.2 Traininig was provided in percent by 1985. enginieering courses, plastic molding, ManulfactUring employment fell refrigeration etc., Financial by about 6 percent between incentives were provided to firms to 1983 and 1985. hirc these workers. 56 Appendix 5.4 (continued) Retraining in Cases of Plant Closures & Mass Layoffs Labor Market Relevant liidicators Intervention Design Type of Result Comments Problem_ Evaluation Closure of Pulp planit I Jnemiploymenit rates in Participants provided with Quasi- Upon completion, participants No information provided itn Krainfors in Sweden rose from 1.6 percent classroom training. experimental received lower weekly wages on re-employment rates or Northelrn Sweden in in 1976 to 2.2 percent by 1978. than those not receiving costs. Benefits from 1977 (OECD, 1991). Employment in manufacturing training. Drop in earnings are retraining program were declined by 8 percent between especially significant in the first insignificant. 1977 and 1979. year with no appreciable long- term gains. Over 3000 workers A 25 percent decline in auto Workers were provided a Quasi- Classroom training (CT) did not Earnings estimates varied laid off due to auto and production between 1978 atid mixture ofjob search assistance experimental significantly improve program (ranging from negative to auto parts plant 1980. Iligh unemilployment (JSA) and classroom training. participants post-program re- significantly positive). closures in Michiiganl rates in 1981/82 nationally (9.5 These services were provided employment rate. Trainees did Training did not seem to (1980-83). The percent) which had fallen to 7.5 promptly after plant closures. no better than those receiving have been very effective, general profile of those percent by 1984. Retraining was provided in JSA. especially in light of fact laid off was Manufacturing employmilenit occupations in whichi there was that trainilig cost twice as experienced blue collar rose by 5 percent between 1983 perceived to be growing much as JSA. male workers who and 1984. demand. Training curricula earned hiighi wages provided instructions in blue (Leigh, 1994). collar trades. Mass Layoffs due to liigh unemployment rates in Displaced workers were Quasi- JSA only services were found to Classroom training and auto arid steel plant 1981/82 nationally (9.5 provided withi a mixture of JSA experimenital have a fairly large impact on OJT were ineffective. Cost closunres in Buffalo percent) wlhich had fallen to 7.5 and either classroom or on the earnings measured over the first of CT and OJT programs (1982-83). The percent by 1984. job training (OJT). Program six post-program months. were around four times as generall profile of those Manufacturing employment services were provided after a However there was no evidence much as JSA, implying that laid off was rose by 5 percent between 1983 fairly lengthiy period of post- of any incremental effect above JSA was potentially the experienced blue collar and 1984. layoff unemploymcnt. that of JSA for either classroom only cost effective male workers who training or OJT. program. No evidence was earned high wages provided about (Cotrson, Long and employability of program Maynard, 1985). participants. 57 Appendix 5.4 (continued) Retraining in Cases of Plant Closures & Mass Layoffs Labor Market Relevant Indicators Intervention Design Type of Result Comments Problem Evaluation Plant closuLes at 13 Uncenploymenit rates rose JSA and traininig was provided to Quasi- Likelihood of worket havitig 'I'raining seems to have had a steel factories and between 1980 and 1983. these workers. Experimental ajob was seven percent greater impact thani JSA. minies in Canada in thle Employmnenit in noni- higher than comparably However, no information on early i980s (Leigh, agricultural activities fell displaced workers not in the costs. Job couniselinig had little 1992). slightly over thie tine period. program. '[his impact was impact in Canada as targeted attributed to training which job counseling is provided by was used by 28 percent of Canada's lPublic E'mployment workers. However at two Service as a standard service to mining sites, the program had all unemployed. no impact. 445 workers were laid Unlemploymenit rates rose Labor Adjustment 'I'raining Quasi- Over a nine month period, No indication of cost- off due to automobile steadily in Australia reaching a Arrangement. Provision of experimental driver training increased the effectiveness of different type plant closure in high of 9 percent in 1984 classroom training (average probability of re-employment. of training courses. Self- Australia in 1984 before falliig to 8 percent for length 19 weeks) to meet Ilowever, other traininig selection problem arose as (Leigh, 1992). the next few years. Between retraining needs of workers. courses resulted in a decline individuals who chose not to 1980 and 1984 employment in Main distinction between courses in re-employment participate were included in the maniufactur ing shrunik by 4 was whether they provided driver probabilities. control group. Impact of percent. training or not. longer traininig courses was negative. Around 2000 workers Local economy was given a A significant number of workers Non -scientific By November 1987, over 90 Two major factors accounted laid off at the boost by decision of Volvo to joined retraining programs percent of the workers who for thie success of the training Uddevalla shipyard in establish a car manufacturing several months prior to being had completed training had program - economy and labor Sweden in 1985 plant at Uddevalla. laid-off. Courses of varying found jobs or become self- market conditions were (Alfthani and Janzoni, Unemployment rates were duration were offered in welding, employed - mnost of them in buoyant in the region 1994). declininig in this period while engineer ing and control occupations they had trainied througlhout the phase out manufacturing employment enginieering. 'I'lTese retraining for. period; shipyard management, was fairly steady. programs were provided by the employment offices and state owned training board, training agencies worked in municipal education institutions close cooperation. and other adult education institutions. 58 Appendix 5.4 (continued) Retraining in Cases of Plant Closures & Mass Layoffs Labor Market Relevant Indicators Intervention Design Type of Result Comments Problem Evaluation TIhe Volvo plant at Volvo planned to recruit 800 Retraining program was Non-scientific Not available as program too Program costs are expected Goteborg, Sweden workers to prepare for the proposed which would help the recent to be evaluated. to be about $25 million planined to lay off 1000 productioni of the new line of existing workforce to manage ($25000 per person) close workers in 1992 cars. Unemployment rates the change without job loss. The to half of which will be associated with the were risinig sharply - fiom 3.2 company accepted the proposal paid by the government. plhasinig out of an older percent in 1991 to 5.9 percent under the stipulation that the cost In judging the cost- car model with a new in 1992. Manufacturing be shared by the government. effectiveness of this model (Alftihan and employment dropped by 9 The program is a broad program, it should be Janzon, 1994). percent in 1992. Total competency raising program compared with the employment also fell by four which includes specific traininig expected unemployment percent. to piepare the participants for the benefits of $6.5 million the production of the new govemment would have to automobiles. pay. Sweden: 1980s and Unemployment rose steadily Various types Various types Retraining programs have Cost-effectiveness of I 990s: general over the period of study. become more ineffective over training programs hias evaliation of public time - especially since the declined both because retraining programs for economy has begun effectiveness has fallen and those laid off en masse deteriorating. Participants have costs have increased. (OECD, 1991) greater difficulty in finding jobs than the openly unemployed. 59 Appendix 5.5 Training Programs Geared Towards Youth Labor Market IProblem Relevant Indicators Intervention Design Type of Result Comments E:valuation Provide high school Th1lere was a sharp decline in IProviding classroom training to Experimental No significanit impact on One site (San Jose) out of 13 dropouits with skills to uLicniployment rates for disadvantaged youth. employment or earnings had positive earning gains. make themil more youth from about 17% in the over a four year follow up Part of the reasons may have employable in the U.S. mid 1980's to about 11% by period of program been because the training dulring the 1 990s (Fay, 1990. TIhis had riseni slightly participants. was closely linked to labor 1996) over the next few years but market needs. held steady at arounld 12% since 1993. IProvidinig training (as well TIime period not knowin. Youths provided withi classroom Experimnental No statistically significant No improvemenlt in other as othel programs) to out- training, on the job training and positive impact on out-of- indicators: youtIl crime or of-school youth in AFDC job search assistance. schiool youth for any welfare receipts by youth. families in the U.S. strategy. (IDOL, 1995) Providinig training to There was a shiarp decline in The U.S. Job Training Experimental No significant impact on Costs=$2000 per trainee. disadvantaged youtil unemployment rates for Partnership Act. Youths earnings. Taking into account costs, nationially in the U.S. fiom youthi firom about 17% in the provided with classroom real rate of return if expected 1983 onwards (Orr, et. al.. mid 1980's to about 1I% by traininig and on-the job training itnpact lasts 3 years or 10 1996; Friedlander, et. al., 1990. This had risen slightly years is negative for both. 1997) over the next few year's but held steady at arounid 12% since 1993. Iligli school dropouts There was a sharp decline in Jobstart demonstration program. Experimental Insignificant impact of Expensive program (over provided with skills in the unemilployineit rates for Youths provided with classroom program on youths $6000/traince). Hence real U.S. betweren 1985-1988 youtil fiom about 17% in the training. earnings. rate of return negative both (Friedlander et. al., 1997) mlid 1980's to about 1 1% by in the short-r un as well as in 1990. Tllis had risen slightly the long-run. over the next few years but lheld steady at around 12% since 1993. 60 Appendix 5.5 (continued) Training Programs Geared Towards Youth Labor Market Probleni Relevanit Indicators Intervention Design Type of Result Comments Evaluation Training high-school Canadian youth Providing some individuals with Quasi- Classroom traininlg had no impact dropouts with employable unemilployment rates fell classroom training and others experimental on earnings or employment skills in Canada during the from 22% to 14% during with classroom as well as probability. Entcrprise training mid to late 1980s (OECD, this period. enterprise training. was relatively effective for young 1993a) men - with significant increases in employment and earnings. No impact on younig women. Provision of skills to There was a sharp decline Both classroom and workplace Experimental Participants did no better than Some improvement in severely disadvantaged in unemiiploymilent rates for based training provided. control group both in terms of social results - reduction youtil in the U.S. in the late youth firom about 17% in earnilgs as well as employment. of criminial activity. 1980's (OECD, 1993a) the mid 1980's to about 11 % by 1990. This had risen slightly over the next few years but held steady at around 12% since 1993. Providing out-of-school Youth unemployment Vocational training courses of Quasi- Those enrolled in longer courses Selection bias problem. youth with skills for the job rates had reached double varying duration provided to experimental were less likely to be employed market in Norway 1990-91 digits and were slowly school dropouts. . than control group. Those (Fay, 1996) rising during this period, enirolled in shiort courses were (lmore likely to be employed. 61 Appendlix 5.6 Micro-Enterprise Development Labor Market Probleni Relevant Indicators Intervention Design Type of Result Coinmelits Evaluation Aid provided to newly Unemployment rates Participants are eligible to receive Experimental Less than five percent of eligible Evaluation performed 15 uliemployed to lhelp in remained between 6-7% a lump-sum financial payment to population were interested in months after participants starting up small businiess in the U.S.. Expendituires start up small business, or they taking up program. Treatment left program. Overall in Washiingtoni state in late on these programs in the can choose to receive a periodic group more likely to be employed costs to IJI budget higher 1980s (Fay, 1994). U.S. was less thai 0.01% allowance for up to 26 weeks, than control group. No significant than that of the control of GDP. Participants provided optional gains in total earnings, group. Net cost-benefit counselinig /advisory service for a Deadweight effect - 27 percent. effect indeterimiinate. short period afler starting up business. Aid provided to newly Unemployment rates were Participants receive a periodic Experimenital Less thani five percent of eligible Evaluation perfoi-med 13 unliciiployed to help in about 7% in the early paymenit (for 24 weeks) to start populationi were interested in months after participants starting up small business 1990s. Expenditur es on up their businiess and are required taking up program. Treatment left program. Overall in Massachtisetts in early self-employment programs to attend an intenisive business group 14 percent more likely to be costs to Ul budget higher 1990s (Wilson and Adams, were negligible, advisory service after start up. employed than control group and than that of the control 1994). earning significantly more. group. Net cost-benefit Deadweight effect = 30 percent. effect not computed. Assistance to individuals in Time period not knowil Unemiiployed individuals given Non-scientific Suivival rate of business=50% No estimates of startinig up small businiesses grants to start up their- own after four years. Better qualified deadweiglht or in the Netherlanids (Meager enterprises, and sliglitly older have higher displacement. and Evans, 1998) suivival iates. Assist individutals in Time period too long The l.K. Enterprise allowance Non-scientific Deadweight=50-70%, starting up businesses in schemile - unemilployed individuials displacement is also high (about the U.K. fi-om the 1970's given allowances to start-up 50% dependinig on sector). Just onwards (Meager and enterprises. 0.2 additional jobs created per E vans, 1998) husiness started. 62 Appenidix 5.6 (continued) Micro-Enterprise Developmeint Labor Market Problem Relevant Indicators Intervention Design Type of Result Comments Evaluation Contracts awarded to six Unemployment rates fell Individuals with business plans Non-scientific Significant positive results Evaluation performed community development to 6.5% from 7.5% in were provided financial recorded in job generation. almost two years after start organizationis in tthe U.S. to 1992. Expenditures on assistance and counseling on However average earnings of up of business. help unemilployed start a self-employment programs starting lip businesses. participants lower than in sites Displacement/substitution/ businiess between 1991-93 were negligible. which did not receive this dead weight loss not (Fay, 1996) assistance, estimated. Beginninig in 1985, Highlest uLIemploymenlt No screening used to approve Quasi- Two percent of unemployed Low screening and finanicial assistance rates among the Nordic assistance to participants. experimental participate in program. 60 percent attractive program benefits provided to those who have countries - around 9% Periodic allowances provided for of businesses failed within the first lower probability of been unemployed for five which rose to over 11% by up to 3.5 years. However, 12 months. Deadweight success and increase montils within the last eight the early 1990s. Denmark palticipants are provided effect=56%. deadweight loss. months to start up their spent 0.1 1% of its GDI' on counseling on starting up a Cost-$14,000 per own cnterpriscs in self-employment business. participant. Denmark (Wilson and promotion in the early Adams, 1994) 1990s. Beginning in 1979, Average unemployment Minimal screening used to Quasi- Below two percent of Evaluation done in early providing aid to rate was close to 9.5% approve assistance to experimental unemployed participated in 1991 of 1986 entrants. unemployed to stalt up throughi the 1980's. It rose participants. A lump sum . program. Survival rates of Cost=$4400 per their own enterprise in from about 8% in the early payment given to entrepreneurs. businesses is 50 percent - lower participant. France (Wilson and 1980's to about 10% later Participants have to participate in than that of start-up businesses that Adams, 1994) on in the decade. post-entry training and business did not receive this assistance. counseling. Half an additional job created for each survivinig enterprise. Deadweight loss=60% 63 Appendix 5.6 (contilnued) Micro-Enterprise Development Labor Market Problem Relevant Indicators Intervention Design Type of Result Comments Evaluation Assist uniemployed Amonig the highest Not available Non- scientific Survival rate after two years was Including additional employees individuals start up their unemployment rates in 60%. Deadweight loss=50% and hired by these enterprises, "net own enterprises in Ireland EuLope - averaging over displacement=30%. job creation was arouLid 34%. durinig the miid 1980s 15% in the 1980s. (OECD, 1993) Irelanid spent about 0.02% of its GDP on these measures in the late 1980s. Assist unlemiiployed to start Unemploymiienit rates of Entrepreneurial grants Non-scientific 50% businesses suivived after Deadweighit losses and own enterpr ises in Norway arotiuid 5% in Norway provided to unemilployed four years. Younig (below 25) as displacemiienit effects not in 1989 and 1990 (Fay, during this time. wiling to start up their own well as less educated estimated. 1996). business, entrepreneurs fared poorly as did those who were the long-term unemployed. I lelp uncmployed start up Australia's Australia's New Enterprise Non-scientific Survival rate after one year was Evaluation done in 1990-91. For their own cnterprises in unemilployment rate was Incenitive. Participants 42% and after two years was each surviving enterprise about Austtalia during 1987 and dcclinlilng fiom 8% in provided a subsidy to start up 29%. 0.7 additional jobs created. 1988 (OECD, 1993) 1986 to 6% in 1988 but enterprise as well as training thieni rose sharply again and business counseling. to around 10O% by 1991. UJnemploynmenit instiurance Canadian Canada's Self-Employment Quasi- Over 80 percent of businesses Evaluation done within two years and social assistance uniemilployment rates had Assistance Program. experimental surviving. Deadweight loss over after program. Participants draw claimianits given finanicial risenl fiom 8% in 1990 Participants receive 52 weeks 50%. Programii participants carn $2000 less thatn comparisoni assistance to star-t up their to 12% by 1993. By of incomiie sUpport and sigihificantty more than groUp from IJI but this does not own businesses in Canada 1995, they had fallen contribute 25% out of their comparison group (about take into consideration the in 1 992-93 (G(raves and again to arouLid 10%. own pocket (up to $4000) in $200/month) but work additional $13000 that was spent Gauthier, 1995) cash or in-kind to start a full- significantly longer hlouirs on program. Long-tern impact time business venture. (14/week) so no impact on hotirly of program uncer tain. eamnings. 64 Appendix 5.6 (continued) Micro-Enterprise Development Labor Market Problem Relevant Indicators Intervention Design Type of Result Comments Evaluation Aid displaced and long- Sharp rise in Self-employment assistance Quasi- 80 percent of enterprises survived Evaluation done in 10 term unemployed workers untemployment: from provided to individuals eligible experimental 15 mnonths. Participants 13 percent counties. No measure to stalt up their own below one percent at the for unemployment benefits. more likely to be employed than of displacement or business in Hlungary in the turn of the decade to 12 Assistance is provided in comparison group, but earning deadweiglht. Each mid 1990s (O'Lcary, percent by 1994. After monthly payments equal to significantly less. Older surviving enterprise 1998a) falling continuously for unemployment compensation for participants and women do better. created 0.3 additional five years, real GDP a period of Lip to 18 months. jobs. started growing slowly. Business counseling and training Of spending on activc are also provided. programiis lcss thani 5 percent spenit on these schemes. Aid displaced and long- GDP started growing Self-employment assistance, up Quasi- 85 percent of enterprises had Evaluation done in 8 term unemployed workers slowly from 1994 but to 20 times the minimum wage, experimental survived 24 months. Participants counties. Program to start up their own unemployment rate rose is provided to unemployed 30 percent more likely to be seems very successful. business in Poland in the fromn 0% in 1989 to 16% individuals. Loans' principal is employed and earn significantly However, no measure of carly and mid 1990s by 1994, though it has reduced by 50 percent if business more than control group. Older displacement or (O'Leary, 1998b) declined since. In 1994, survives two years, participants and women do better. deadweight. Cost- below 0.5% of benefit analysis points expenditures on active towards a societal "loss" labor programs was of over $1300. expended on self- employment schemes 65 Appendix 5.7 Wage/Employment Subsidies Labor Market Problem Relevant In(licators Interveiition Design Type of Result Comments Evaluation P'rovicding subsidies to hire Among the highest Fmployers paid between 30-60 Non- Deadweight loss-70%, 400 firmis who hired long-terill unelmclployed in unlemploymenit rates in ELurope Irish potiunds/week for up to 24 scientific substitution=21%, subsidized individuals Ircland in the 1980's - averaging over 15% in the weeks, contingent on worker displacement=4%. 1-lence "net" interviewed. No (OECD, 1993) 1980S. remaining with firm. 'Ihis is about effect=5% information on cost 30-50% of gross wages. available. Subsidies almost completely ineffective. Emiiployers given subsidies Unemploymenit rates falling Employers paid up to Non- Deadweight over 65 percent. 1000 workers who were to hire long teriml fiom 9% to 7% during this AU$200/week for up to 16 weeks scientific subsidized, and firms in unemilployed in Australia in period. Australia spent about (approximately 50% of gross which they worked, were mid 1980s (Mangatn, 0.03% of its GDP on these wages) to hire the long-tern interviewed. 1988). measures. uncmployed Increasinig employability of Unemilploymiienit r ates falling IThe JOB schemne. Subsidies given Non- H-ligh deadweight loss about 22- No control group, hence loIng-term un????employed fiom 9% to 7% durinig this to employers to hire youth who scientific 30%. Very little evidence of no measure of net youth by providinig wage period. Austialia spent about have been unemployed for two net increase in total impact. Deadweight subsidies in lHollanid duLinlg 0.03% of its GDP on these years or more. employment. Intervention does could be as high as 80% the late 1980s (Meager and measures. increase probability of as another 50% of EvIns, 1998) employmnent of long-term individuals may have unemployed youth, but by less found a job anyway. thanl 10%. Increase the employability Dutchi unemilploymenit rates Subsidies given to firms to hire Quasi- Substitution effect=80%. Less thani 20% of new of the long-tenn were falling steadily fiom 8% individtuals who have been experimental jobs were additional to unemployed, providing to 5% dtrinig thiis period. unemployed for two years (or what would have been wage subsidies to Speniciilig less than 0.03%/O of ethniic minorities who have been accomplished withotit employers to iluc thelin (Ditl' on wage subsidies. unemilployed a year). the prograimi. HIolland durinig the late 1980s (OECI), 1993) 66 Appendix 5.7 (continued) Wage/Employment Subsidies Labor Market Problem Relevant Indicators Intervention Design Type of Result Comments Evaluation P'rogram to promote Unemployment rates 'I'he U.S. Targeted Job Tax Credit. Experimental Significant impact on job Some stidies have also employment of remained between 6-7% in Employers paid up to 50% of earnings in first year relative suggested a negative disadvantaged urban the U.S.. Negligible eligible wages for two years. to control; however, impact impact on employment population in the 1J.S. in expenditur es on these Employees either got vouchers that decreases in second year. No of targeted persons, the m id 1980s (OECD, programs. they took to firms, or employers longer term impact once perhaps resulting firom a 1993) got the money directly after subsidy is removed. "stigmatizing" effect. submitting applications for workers they decided to hiire. Increase employability of 'Iime period not knowin A reduction in payroll taxes of Experimental No positive impact of subsidy unemiiployed in Sweden employers who hire long-terimi on employment (Meager and Evans, 1998) unemployed. Programil to promote Timc period too long Under the UJ.S. Job Training Experimental Significant impact on earnings Single mothers who are employmlienit of Placemenit Act (JTPA) employers for women. However, impact on Assistance to disadvantaged workers in were provided subsidies to hire on men only partly significant. Families with Dependent the U.S. (Bloom, 1994) workers mainly to provide on-the- Children (AFDC) benefit job traininig. most. Program targeted at highi U.K. uniemployment rates rose Wage grant offered for trainees Non scientific Deadweight loss=20%. Displacement effect not uliemployment areas to from 6% in 1989 to 10% by varying, from 50% to 100% of the Additional jobs created over estimated. Low provide wage subsidies to 1992. However, expenditures trainees wages for a period of 6 25% deadweight loss may be the unemployed and those on these programs remained motiths, and entire traiting costs due to employers being at r isk in Scotland 1989- negligible. also covered. Employers had to forced to declare at the 1992 (NERA, 1995) employ individual in a full time job outset that the job would lasting at least one year. not have been offered in the absence of a subsidy. 67 Appendix 5.7 (continued) Wage/Employment Subsi(lies Labor Market Problem Relevaiit Indicators Iintervention Design Type of Result Comnelits Evaluation Wage subsidy provided to Proportionl of long-term Subsidy given to employees. To Quasi- Deadweight loss=69 percent. Evaluation performed the long-term unlemilployed unlemilployed in total get subsidy, participants had to experimental No significant eam-ings or three months after to get themii back into tile unlemilployed declined fiom earn below 90 pounids per week employment impact. subsidy ended. 30 labor force in Englanid about 45% to 30%. and work at least 35 hours/week. percent of individuals between 1986 and 1990 Unemilploymilenit rates fell fi-om The job had to be at least thiree had left/been let go by (NlERA, 1995) 12% in 1986 to 6% in 1990. monthls in duration. firms. Deadweighlt loss was inversely proportional to the subsidy. Wage subsidies provided in Proportioni of long-teriml The Traininig and Employmenit Non- Deadweight loss=47% and ????? the U.K. in the late 1980s unlemiiployed in total Grant Scheme. Employers got scientific substitution=23%. (NlE'RA, 1995). T his was uLnemilployed declined firom between 50-60% of gross wages Additionality=27% mainly targeted at the about 45% to 30%. for up to six months and had to youth and long-term UJnemilploymilenit r ates fell firom pl ovide 200 hours of structured unlemployed in high 12% in 1986 to 6% in 1990. training. unemployment areas. Wage subsidies targeted at Tine period not known The Youn1g Workers Schenme, Non- Deadweight=63% and Targeting done on the youth in the U.K. Eligible employers paid 15 scientific Substitution= 10%. basis of choice of (Balakrishlinan, 1998) pounds/week for up to a year to Additionality=27% eligible firms. hire younig unemployed (under 18). Wage subsidies given to tJlnemlploylmlelnt rates at around Any additional employees hired by Non- Deadweight=60-75%. Substitutioni is zero by small finns to hire 5%. small manufacturing firms in scientific Additionality=25-40% definition, as there was unlelmiployed workers in the special developmenit areas were no untargeted group of U.K. in the early 1970s. subsidized Up to 20 pounds/week workers who could be for up to 26 weeks. substituted against. 68 Appensdix 5.7 (continlued) Wage/Employment Subsidies Labor Market Problem Relevant Indicators Intervention Design Type of Result Comments Evaluation T o enhanlce employment Higli and rising rates of Employers given a wage subsidy Non- Deadweight effect=53% and Negligible net prospects of long-term unemiploymenit - 9% in 1991 for a period of up to six months. scientific substifttion=36% employment effect. unemployed, employers to 12% in 1993. ILong-term provided subsidies to hire unlemiploymiienit over 60% of tlhem in Belgiumti in the total unemployment. 0.05% early 1990s (Fay, 1996) of GDP spent on these measures. I lave re-employment Unemployment rates generally Uployloyed workers provided Experimental Weeks of benefit fell in all Due to the higih costs of bonuses worked in getting stable in the U.S. during the bonuses if they could find work four experiments. Effect was the progreams, societal people of Ul be?efits and periods of the experihmenits. withinl a specified period. E.g. in strongest in Illinois whaere it benefits varied. They into better jobs? An Illinois, an unemployed individual fell by a week (average weeks were positive in Illinois experimcntal study of the would receive a bonis of $500 of benefit received ranged and Pennsylvania employment bonus (four times the average weekly Ul from 15-20). Experiments also (slightly) but negative in experiments in Illinois benefit) if they found employment show no significant declines in New Jersey and (1984), New Jersey (1986- withlin I I weeks of filing for Ul re-employment earnings. In Washington on the 87), Pennsylvania (1988- and kept the job for at least four fact, in Illinois, the treatment whole. Similar results 89) and Washington months. group had slightly higher found for Illinois by (1988) (Meyer, 1995). earnings than the control Woodbuiy and group. Spiegelman (1987). Only in one case was bonus provided to employers - in that case effect was insignificant. Emiiployers provided Sharp rise in unemploymelit: Payment of up to 50% of wage Quasi- Participants are 10 percent less Evaluation done over subsidies to hire the long- from below one percent at the made to employers to hire experimental. likely to be employed, and earn year after program term unemployed in turn of tlhe decade to 12 individuals who were previously significantly less, than control completion, in 10 Ilungary rnid-1990s pcrccnt by 1994. After falliig unemployed for at least six months. group in current job. counties. (O'Leary, 1998a) continuously for five years, If workers are not retained by firm real (iDP started growing after subsidy ends for a period as slowly. Of spendinig on active long as the subsidy was paid, prograrns, over 12 percent employer must repay the subsidy. spent on these schemes. 69 Social Protection Discussion Paper Series No. Title 9917 Taking Stock of Pension Reforms Around the World 9916 Child Labor and Schooling in Africa: A Comparative Study 9915 Evaluating the Impact of Active Labor Programs: Results of Cross Country Studies in Europe and Central Asia 9914 Safety Nets in Transition Economies: Toward a Reform Strategy 9913 Public Service Employment: A Review of Programs in Selected OECD Countries and Transition Economies 9912 The Role of NPOs in Policies to Combat Social Exclusion 9911 Unemployment and Unemployment Protection in Three Groups of Countries 9910 The Tax Treatment of Funded Pensions 9909 Russia's Social Protection Malaise: Key Reform Priorities as a Response to the Present Crisis 9908 Causalities Between Social Capital and Social Funds 9907 Collecting and Transferring Pension Contributions 9906 Optimal Unemployment Insurance: A Guide to the Literature 9905 The Effects of Legislative Change on Female Labour Supply: Marriage and Divorce, Child and Spousal Support, Property Division and Pension Splitting 9904 Social Protection as Social Risk Management: Conceptual Underpinnings for the Social Protection Sector Strategy Paper 9903 A Bundle of Joy or an Expensive Luxury: A Comparative Analysis of the Economic Environment for Family Formation in Western Europe 9902 World Bank Lending for Labor Markets: 1991 to 1998 9901 Active Labor Market Programs: A Review of the Evidence from Evaluations 9818 Child Labor and School Enrollment in Thailand in the 1990s 9817 Supervising Mandatory Funded Pension Systems: Issues and Challenges 9816 Getting an Earful: A Review of Beneficiary Assessments of Social Funds 9815 The Quest for Pension Reform: Poland's Security through Diversity Social Protection Discussion Paper Series continued No. Title 9814 Family Allowances 9813 Unemployment Benefits 9812 The Role of Choice in the Transition to a Funded Pension System 9811 An Alternative Technical Education System: A Case Study of Mexico 9810 Pension Reform in Britain 9809 Financing the Transition to Multipillar 9808 Women and Labor Market Changes in the Global Economy: Growth Helps, Inequalities Hurt and Public Policy Matters 9807 A World Bank Perspective on Pension Reform 9806 Government Guarantees on Pension Fund Returns 9805 The Hungarian Pension System in Transition 9804 Risks in Pensions and Annuities: Efficient Designs 9803 Building an Environrment for Pension Reform in Developing Countries 9802 Export Processing Zones: A Review in Need of Update 9801 World Bank Lending for Labor Markets: 1991 to 1996 Summary Findings In this paper, we survey evidence based on the evaluation of active labor market programs. We have examined about 1 00 evaluations. Many of these studies have already been summarized by others (such as OECD and ILO), hut we have also included a significant number of individual studies. And though most studies apply to OECD countries - mainly the U.S., Canada, U.K., Sweden and Germany -we have added information on developing and transition economies such as Hungary, Poland, the Czech Republic, Turkey and Mexico. While it can be argued that the lessons from developed countries on the effectiveness of these programs may not be directly applicable to developing countries, it is unlikely that these programs will be more successful in developing countries given the scarcity of administrative capacity to implement these programs and the paucity of monitoring and evaluation experience to study their effectiveness. i ~~~HUMAN DEVELOPMENT NETWORK About this series... Papers in this series are not formal publications of the World Bank. They present preliminary and unpolished results of analysis that are circulated to encourage discussion and comment; citation and the use of such a paper should take account of its provisional character. The findings, interpretations, and conclusions expressed in this paper are entirely those of the author(s) and should not be attributed in any manner to the World Bank, to its affiliated organizations or to members of its Board of Executive Directors or the countries they represent. For free copies of this paper, please contact the Social Protection Advisory Service, The World Bank, 1818 H Street, N.W., Room G8-138, Washington, D.C. 20433-0001. Telephone: (202) 458-5267, Fax: (202) 614-0471, E-mail: socialprotection@worldbank.org or visit the Social Protection website at www.worldbank.org/sp.