Document of The World Bank Report No: ICR00002860 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-73710) ON A LOAN IN THE AMOUNT OF US$25 MILLION TO THE DOMINICAN REPUBLIC FOR A YOUTH DEVELOPMENT PROJECT December 20, 2013 Human Development Sector Management Unit Caribbean Country Management Unit Latin American and the Caribbean Regional Unit CURRENCY EQUIVALENTS (Exchange Rate Effective December 9, 2013) Currency Unit = RD$ RD$1.00 = US$ 0.024 US$ 1.00 = RD$ 42.45 FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS CAS Country Assistance Strategy DR Dominican Republic DYAE Department of Youth and Adult Education EBA Adult Basic Education (Educación Básica de Adultos) FM Financial Management GDR Government of Dominican Republic IADB Inter-American Development Bank IBRD International Bank for Reconstruction and Development ICR Implementation Completion and Results Report ISR Implementation Status Report IMF International Monetary Fund INFOTEP National Training Institute (Instituto de Formación Técnica y Profesional) IERR Internal Economic Rate of Return ISR Implementation Status Report KOI Key Outcome Indicator MIS Management Information System MoE Ministry of Education MoL Ministry of Labor M&E Monitoring and Evaluation NPV Net Present Value PAD Project Appraisal Document PCU Project Coordination Unit PDO Project Development Objective PJyE Youth Employment Program (Programa Juventud y Empleo) PREPARA Adult Secondary Education (Educación Secundaria para Adultos) PRF Project Result Framework PSR Project Status Report QAG Quality Assurance Group QEA Quality at Entry Assessment QSA Quality of Supervision Assessment SET Secretary of Labor SGCE School Management System (Sistema de Gestión de Centros Educativos) SIL Standard Investment Loan SMES Small and medium-size enterprises SNE National Employment Service (Servicio Nacional de Empleo) TEP Temporary Employment Program TTL Task Team Leader Vice President: Hasan A. Tuluy Country Director: Sophie Sirtaine Sector Director: Mansoora Rashid (acting) Sector Manager: Mansoora Rashid Project Team Leader: Juan Martín Moreno ICR Team Leader: Juan Martín Moreno DOMINICAN REPUBLIC Youth Development Project CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 4 3. Assessment of Outcomes ............................................................................................ 9 4. Assessment of Risk to Development Outcome......................................................... 18 5. Assessment of Bank and Borrower Performance ..................................................... 19 6. Lessons Learned ....................................................................................................... 21 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 22 Annex 1. Project Costs and Financing .......................................................................... 23 Annex 2. Outputs by Component ................................................................................. 24 Annex 3. Economic and Financial Analysis ................................................................. 28 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 34 Annex 5. Beneficiary Survey Results ........................................................................... 36 Annex 6. Stakeholder Workshop Report and Results................................................... 37 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 38 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 42 Annex 9. Summary of Randomized Impact Evaluation Studies ................................. 44 Annex 10. List of Supporting Documents .................................................................... 48 MAP A. Basic Information Dominican Republic Country: Dominican Republic Project Name: Youth Development Project Project ID: P096605 L/C/TF Number(s): IBRD-73710 ICR Date: 12/30/2013 ICR Type: Core ICR DOMINICAN Lending Instrument: SIL Borrower: REPUBLIC Original Total USD 25.00M Disbursed Amount: USD 22.82M Commitment: Revised Amount: USD 25.00M Environmental Category: B Implementing Agencies: Secretary of Education Secretary of Labor Cofinanciers and Other External Partners: IDB - Inter-American Development Bank B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 10/06/2005 Effectiveness: 09/20/2007 09/20/2007 09/15/2010 Appraisal: 12/12/2005 Restructuring(s): 06/06/2012 Approval: 03/23/2006 Mid-term Review: 10/05/2009 10/05/2009 Closing: 06/30/2011 06/30/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Low or Negligible Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory Implementing Quality of Supervision: Satisfactory Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Moderately Satisfactory Performance: Performance: i C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry Yes None at any time (Yes/No): (QEA): Problem Project at any Quality of Yes None time (Yes/No): Supervision (QSA): DO rating before Moderately Closing/Inactive status: Satisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Adult literacy/non-formal education 21 21 Central government administration 13 13 Vocational training 66 66 Theme Code (as % of total Bank financing) Education for all 25 25 Improving labor markets 50 55 Vulnerability assessment and monitoring 25 20 E. Bank Staff Positions At ICR At Approval Vice President: Hasan A. Tuluy Pamela Cox Country Director: Sophie Sirtaine Caroline D. Anstey Sector Manager: Mansoora Rashid Evangeline Javier Project Team Leader: Juan Martin Moreno Andrea Vermehren ICR Team Leader: Juan Martin Moreno ICR Primary Author: Maria Cecilia Zanetta F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The project's development objective was to support the Government of the Dominican Republic (GDR) in improving the employability of poor at-risk youth by: i) building their work experience and life skills, and ii) expanding second-chance education programs to complete their formal education. ii Revised Project Development Objectives (as approved by original approving authority) The revised PDO was to support the Borrower in improving the employability of: (a) poor at-risk youth by building their work experience and life skills and expanding second-chance education programs to complete their formal education, and (b) poor, low- skilled, long-term unemployed people between the ages of 18 and 65. (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years At least 65% of the Youth Employment Program (PJyE) graduates are employed Indicator 1 : or self-employed six months after program completion. Value quantitative or n.a. 65% 71.8% Qualitative) Date achieved 02/01/2006 06/30/2011 06/30/2013 Comments Exceeded. Employment rate reported in 2008-2009 to PJyE graduates, part of the (incl. % impact evaluation. Positive impact on job formality, increasing the probability of achievement) holding a job with benefits among those that received training. Indicator 2 : PJyE graduates have earnings 10% higher than control group. No significant impact on income for PJyE graduates Value as a whole quantitative or 0% 10% Qualitative) 5% to 7% higher income among some population subsets. Date achieved 02/01/2006 06/30/2011 06/30/2013 Partially Achieved: Results from randomized impact evaluations show no Comments significant impact on income among PJyE graduates as a whole. Although not (incl. % significant, reported results show slightly positive impact for women and achievement) negative for men. At least 80% of additional students of Adult Basic Education (EBA) and Adult Indicator 3 : Secondary Education (PREPARA) Programs obtain their basic or secondary degrees, respectively. Value 70% for EBA and 80% for EBA and 72.7% for EBA and quantitative or PREPARA PREPARA PREPARA Qualitative) Date achieved 02/01/2006 06/30/2011 06/30/2013 Comments Partially Achieved. Graduation rates have not increased as anticipated, with a (incl. % 72.7% average aggregate graduation for EBA (66.3%) and PREPARA (84.2%) achievement) programs. Between 2,500 and 4,000 beneficiaries participate in the Temporary Employment Indicator 4 : Program (TEP) iii Value quantitative or 0 n.a. 2,500 to 4,000 3,920 Qualitative) Date achieved 02/01/2006 06/30/2011 06/30/2012 06/30/2013 Achieved. This indicator was added in 2010 as part of the Level 1 Project Comments restructuring. A total of 3,920 low-income, unskilled unemployed completed (incl. % training and their temporary employment, at the upper bound of the original achievement) 2,500-4,000 target. Decrease in risky behaviors (substance abuse, cigarette use, alcohol use, Indicator 5 : unprotected sex, gang participation, etc.), and increase in self-esteem. Value Indirect effects No specific target quantitative or n.a. have been was defined Qualitative) identified Date achieved 02/01/2006 06/30/2011 06/30/2013 Comments Achieved: Results from the IE suggest several indirect effects, including a 5 pp. (incl. % reduction in teenage pregnancy. This effect is consistent with an overall increase achievement) in youth's positive expectations about the future. (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years (Int. Objective 1) Expanded coverage: 32,000 beneficiaries attending courses Indicator 1 : offered through the ICAPs. Value (quantitative n.a. 32,000 38,008 or Qualitative) Date achieved 02/01/2006 06/30/2011 06/30/2013 Comments Exceeded. The PJyE has effectively extended its coverage, with the number of (incl. % beneficiaries exceeding the original target by 19%. achievement) (Int. Objective 1) Improved poverty-focused targeting: At least 60% of PJyE Indicator 2 : beneficiaries are from the poorest areas as identified in the Dominican Republic's Poverty Map. Value (quantitative n.a. 60% 94.9% or Qualitative) Date achieved 02/01/2006 06/30/2011 06/30/2013 Comments Exceeded. Through its geographical based targeting, the PJyE has effectively (incl. % reached youth living in the poorest areas—i.e., 95% of its beneficiaries come achievement) from these targeted areas compared to the 60% original target. (Int. Objective 1) Improved gender-focused targeting: At least 45% of Indicator 3 : beneficiaries are women. Value (quantitative 40% 45% 57.8% or Qualitative) iv Date achieved 02/01/2006 06/30/2011 06/30/2013 Comments Exceeded. The PJyE has effectively expanded its targeting of women, with (incl. % almost 60% of its beneficiaries being female compared to the 45% original achievement) target. (Int. Objective 1) Improved monitoring and evaluation: Secretary of Labor's Indicator 4 : (SET) system links payments with results. Impact evaluation covers medium- term impact and expands indicators to include risky behaviors. Results-based Results-based payments to payments to providers in place providers Value (quantitative n.a. Impact evaluation Impact evaluation or Qualitative) was conducted, which includes which included indicators on risky indicators on risky behaviors behaviors Date achieved 02/01/2006 06/30/2011 06/30/2013 Achieved. The PJyE was supported by a robust MIS that permitted the Comments implementation of a payment for results mechanism. In addition, the PJyE was (incl. % subjected to rigorous IEs, thus contributing to the body of knowledge on training achievement) interventions. (Int. Objective 1) Improved linkages with education: At least 20% of PJyE Indicator 5 : beneficiaries are registered in EBA or PREPARA one year after program completion'. Value (quantitative n.a. 20% 27% or Qualitative) Date achieved 02/01/2006 06/30/2011 06/30/2013 Comments Exceeded.: By creating incentives for PJyE graduates to enroll in "second- (incl. % chance" education programs (e.g., fee waivers), links are being built between achievement) training and education targeting vulnerable young adults. (Int. Objective 1) Expanded employer participation. Increase in the diversity of Indicator 6 : private firms involved in PJyE (by offering internships). Value (quantitative n.a. 2,000 3,002 or Qualitative) Date achieved 02/01/2006 06/30/2011 06/30/2013 Exceeded. The PJyE has successfully expanded employer participation, with Comments over 3,000 private firms offering internships to PJyE graduates, amply exceeding (incl. % the 2,000 target. New firms were smaller in size (measured by number of achievement) employees). (Int. Objective 1) Expanded course offerings. Increase in the number of new Indicator 7 : courses offered by the ICAPs Value (quantitative n.a. 40 24 or Qualitative) Date achieved 02/01/2006 06/30/2011 06/30/2013 Comments Not Achieved. The expansion in the scope of the training fell below expectations. (incl. % A more effective mechanism to capture employers' demand for labor skills is v achievement) needed as well as mechanisms to facilitate the development of new courses. (Int. Objective 2) Expanded Coverage: INFOTEP successfully co-finances 400 Indicator 8 : training courses for youth at risk. Value (quantitative 0 400 400 or Qualitative) Date achieved 02/01/2006 06/30/2011 06/30/2013 Comments Achieved. The Dominican Republic’s national training entity, INFOTEP, (incl. % provided key institutional support during the PJyE implementation, including achievement) financing 400 courses. (Int. Objective 3) Expanding coverage: EBA and PREPARA increase their Indicator 9 : coverage by 12 and14% respectively. 12% increase (i.e., 4.6% increase (i.e., 10,440 additional 3,965 additional EBA participants) EBA participants) Value 87,000 for EBA (quantitative 14% increase (i.e., 61% increase (i.e., or Qualitative) 48,000 for PREPARA 6,720 additional 29,288 additional PREPARA PREPARA participants) participants) Date achieved 12/31/2006 06/30/2011 12/31/2012 Comments Partially Achieved. Coverage for second-chance education programs has (incl. % expanded substantially at the secondary level (PREPARA) but has experienced achievement) only a modest expansion for basic education (EBA). (Int. Objective 3) Improved poverty-focused targeting: At least 70% of EBA and Indicator 10 : PREPARA beneficiaries from poorest 40% of households. Value 60% for EBA 70% for EBA and 76.2% for EBA and (quantitative PREPARA PREPARA or Qualitative) 55% for PREPARA Date achieved 12/31/2006 06/30/2011 12/31/2011 Comments Exceeded. Over 76% of EBA and PREPARA beneficiaries live in the poorest (incl. % areas identified as priority in the Dominican Republic's Poverty Map, exceeding achievement) the 70% target. (Int. Objective 3) Improved gender-focused targeting: At least 45% of EBA and Indicator 11 : PREPARA program beneficiaries are women. Value 44% for EBA 45% for EBA and 51.2% for EBA and (quantitative PREPARA PREPARA or Qualitative) 60% for PREPARA Date achieved 12/31/2006 06/30/2011 12/31/2012 Comments Exceeded. Women represent over half of EBA and PREPARA beneficiaries, (incl. % exceeding the 45% target. achievement) (Int. Objective 3) Improved quality: 4,500 teachers trained in Adult Education Indicator 12 : methodologies. 4,000 for EBA 4,247 for EBA Value 7,000 for EBA and (quantitative 0 PREPARA 500 None or Qualitative) for PREPARA for PREPARA vi Date achieved 02/01/2006 06/30/2011 06/30/2012 06/30/2013 Comments Partially Achieved. The MoE decided to concentrate its efforts on enhancing (incl. % second-chance education at the basic-education level, overhauling its curricula achievement) adopting a flexible, modular approach, and training a total of 4,247 teachers. (Int. Objective 3) Effectiveness: Program drop-out rate decreases from 23% to Indicator 13 : 18% Value 23% for EBA and 18% for EBA and 18% for EBA and (quantitative PREPARA PREPARA PREPARA or Qualitative) Date achieved 12/31/2006 06/30/2011 06/30/2013 Comments Achieved. Substantial differences are reported in the drop-out rates of the two (incl. % programs—24.2% and 7% for the EBA and PREPARA respectively. The latter achievement) exhibits a better performance in terms of enrollment, and graduation rates. Indicator 14 : (Int. Objective 3) Books for EBA and PREPARA are distributed. Educational materials reflecting the newly developed flexible curricula for EBA Value have been (quantitative n.a. n.a developed for seven or Qualitative) of the 12 modules, of which three have been published and are ready for distribution to students. Date achieved 12/31/2006 06/30/2011 06/30/2013 Comments Partially Achieved. The review, publishing and distribution of learning and (incl. % teaching materials were delayed due to the MoE's decision to adapt the EBA achievement) curricula to a flexible, competency-based modality. (Int. Objective 3) Improved monitoring and evaluation: From no M&E system to Indicator 15 : an integrated MIS which tracks enrollments, attendance and pass rates, and provides regular tracer studies to assess longer term impact. Value SGCE operational (quantitative none none none in both EBA and or Qualitative) PREPARA Date achieved 02/01/2006 06/30/2011 06/30/2013 06/30/2013 Comments Achieved. The MoE's School Management System (Sistema de Gestión de (incl. % Centros Educativos – SGCE) now includes the EBA and PREPARA Programs, achievement) providing data at the student level. G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 06/01/2006 Moderately Satisfactory Moderately Satisfactory 0.00 vii 2 10/29/2006 Moderately Satisfactory Moderately Satisfactory 0.00 Moderately 3 05/02/2007 Moderately Satisfactory 0.00 Unsatisfactory Moderately 4 09/13/2007 Moderately Satisfactory 0.00 Unsatisfactory Moderately Moderately 5 04/15/2008 0.06 Unsatisfactory Unsatisfactory 6 08/16/2008 Moderately Satisfactory Moderately Satisfactory 1.06 Moderately 7 04/07/2009 Moderately Satisfactory 1.93 Unsatisfactory 8 11/24/2009 Moderately Satisfactory Moderately Satisfactory 6.72 9 06/15/2010 Moderately Satisfactory Moderately Satisfactory 10.17 10 01/01/2011 Moderately Satisfactory Moderately Satisfactory 13.13 11 08/03/2011 Moderately Satisfactory Moderately Satisfactory 15.74 12 03/25/2012 Moderately Satisfactory Moderately Satisfactory 18.35 13 09/29/2012 Moderately Satisfactory Moderately Satisfactory 19.90 14 04/13/2013 Moderately Satisfactory Moderately Satisfactory 20.95 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions Level One restructuring to, among other changes: i) modify PDO to reflect the introduction of a temporary employment 09/15/2010 Y MS MS 11.20 program; ii) modify the Results Framework; and iii) extend closing date from June 30, 2011 to June 30, 2012. Level Two restructuring to: i) reallocate funds between 06/06/2012 MS MS 19.17 components; and ii) extend closing date from June 30, 2012 to June 30, 2013. If PDO and/or Key Outcome Targets were formally revised (approved by the original approving body) enter ratings below: Outcome Ratings Against Original PDO/Targets Moderately Satisfactory Against Formally Revised PDO/Targets Moderately Satisfactory Overall (weighted) rating Moderately Satisfactory viii I. Disbursement Profile ix 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. Dominican youth, particularly the poor, face structural barriers to enter labor markets, including insufficient education, life skills, and job skills to make the transition from school to work. Exacerbating this already precarious situation, a banking crisis unleashed in 2003 had decreased the chances for youth to enter the labor market, particularly those who were poor and unemployed. During the crisis, the youth unemployment rate increased by roughly a third from the 2003 level to reach 31 percent for ages 15-24 by 2005, as compared to 17 percent for the population at large (World Bank 2005). 2. The operation responded to the urgent need to provide more opportunities to disadvantaged youth from poor families and with little schooling. The project formed part of a larger effort on the part of the Government of the Dominican Republic (GDR) to strengthen the country’s social protection system and was complementary to other operations, including the Bank-financed Social Protection Investment Project (P90010; for US$1.4 million; approved in November 2005), and the Inter-American Development Bank- financed (IADB) Labor Markets and Social Transfers Project (DR-L1006; for US$10 million; approved in November 2005). 1.2 Original Project Development Objectives (PDO) and Key Indicators 3. The Project's Development Objective was to support the GDR in improving the employability of poor, at-risk youth by building their work experience and life skills, and expanding second-chance education programs to complete their formal education. 1 The corresponding Key Outcome Indicators (KOIs) were:  At least 65 percent of the Programa Juventud y Empleo (PJyE) graduates are employed or self-employed six months after program completion.  PJyE graduates have earnings 10 percent higher than the control group.  At least 80 percent of additional students from the Adult Education basic and secondary equivalency programs (Adult Basic Education (EBA), and Adult Secondary Education (PREPARA), respectively) obtain their basic and secondary degrees.  Decrease in risky behaviors (e.g., substance abuse, cigarette use, alcohol use, unprotected sex, gang participation) and increase in self-esteem. 2 1 Original Project Appraisal Document (p. 14); Original Loan Agreement, Schedule 1 (p. 5). 2 There are some variations between the KOIs as defined in the PAD’s main text (p. 14) and the Project Results Framework (p. 43-44). This ICR adopts the ones in the Results Framework, as they do not represent significant differences from those in the main text and are the ones that had been used during project supervision by both the Bank team and the Borrower. Moreover, these are the ones reflected in the Project Manual, to which the Loan Agreement explicitly identifies as the ones to be used to monitor and evaluate the Program (paragraph 2, Schedule 2). 1 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 4. The PDO was expanded as part of a Level One restructuring that took place in August 2010 to reflect the inclusion of a Temporary Employment Program (TEP) in the Santiago province—Santiago Trabaja—under the restructured Project. The revised PDO was to support the Borrower in improving the employability of: (a) poor at-risk youth by building their work experience and life skills and expanding second-chance education programs to complete their formal education, and (b) poor, low-skilled, long-term unemployed people between the ages of 18 and 65. 5. An additional KOI was included in the original Project Results Framework (PRF) to reflect the change in the PDO:  Between 2,500 and 4,000 beneficiaries participate in the TEP. 1.4 Main Beneficiaries 6. As originally designed, the Project aimed at benefiting primarily poor, at-risk youth. In the case of PJyE, the enrollment of beneficiaries was made by training providers and the eligibility criteria were verified by the Project Coordination Unit (PCU) on the basis of self- reported information. As a result of the 2010 restructuring, the Project’s beneficiaries also included poor, low-skilled, long-term unemployed people between the ages of 18 and 65. The selection of TEP beneficiaries was based on a profiling strategy and the application of an employability index to prioritize beneficiaries from the pool of enrolled unemployed. 1.5 Original Components (as approved) 7. Component 1: Consolidate and expand the Youth and Employment Program (Programa Juventud y Empleo - PJyE) to provide increased access to a first work experience for youth at risk (US$22 million, including US$18 million in Bank financing): As requested by the GDR, this component aimed at broadening the PJyE’s coverage to focus more intensively on poor and disadvantaged youth and expanding its content by supporting pilot training courses and internships designed to promote entrepreneurship and self-employment, in addition to the Program’s previous focus on private and public sector employers. 8. Component 2: Build capacity in the then Secretary of Labor (SET) 3 and the National Training Institute (INFOTEP) to more effectively focus on at-risk youth (US$2 million total, all of which was to be financed by Bank): The objective of this component was to improve the institutional capacity to reach disadvantaged out-of school youth through strengthened program coordination, implementation, and evaluation, including the gradual transition of the PJyE to the National Training Institute (INFOTEP) in order to foster the sustainability of the program’s outcomes. 3 The 2010 constitutional reform in the Dominican Republic converted the then Secretaries into Ministries. This report uses the term Ministry throughout the text, except in the cases in which the original text is being quoted. 2 9. Component 3: Strengthen and expand the ‘Second Chance’ education programs EBA and PREPARA (US$9.8 million total, including US$5 million in Bank financing): This component aimed at strengthening and expanding primary and secondary equivalency programs (EBA and PREPARA, respectively) focusing on accelerated courses with improved materials, modules in life skills, as well as strengthening the institutional capacity of the Department for Youth and Adult Education (DYAE) to implement the expansion of these programs and monitor their results and impact over time. 1.6 Revised Components 10. Component 4: Support a Pilot Temporary Employment Program (US$2 million total, all of which was to be financed by Bank): This component, which was added under the 2010 restructuring, provided support to pilot a TEP to reach between 2,500 to 4,000 beneficiaries. The overall duration of the TEP was 18 months, with individual beneficiaries participating for four months. The TEP was targeted to selected municipalities in the province of Santiago, in which employment was negatively affected as a result of the crisis. 1.7 Other significant changes 11. In addition to the change in the Project’s PDO to reflect the change in the target population and the incorporation of a new component to support a pilot TEP, the following changes were introduced under the 2010 Level One Project restructuring: i) the environmental safeguard category of the Project changed from C to B as a result of physical works under the new TEP activity, triggering a new safeguard (i.e., OP 4.01 Environmental Assessment); ii) Project funds were reallocated to finance the new component, creating a new category of eligible expenditures and allowing financing of the new TEP component costs from May 1, 2010 onwards; iii) the Closing Date was extended by twelve months from June 30, 2011 to June 30, 2012; iv) the definition of “operational costs” was modified and definitions for “remuneration for temporary workers” and “life skills learning courses “were introduced in the Loan Agreement to reflect the new TEP component; and v) the target value of an intermediate outcome indicator under Component 3 (i.e., number of teachers trained and certified in adult education methodology) was reduced from 7,000 to 4,500. 12. A Level Two Restructuring took place in 2012, which introduced the following changes: i) given its sluggish implementation, US$1 million from Component 3 (implemented by the Ministry of Education - MoE) was reallocated to Component 1 (implemented by the Ministry of Labor - MoL); and ii) the Closing Date was extended for a second time from June 30, 2012 to June 30, 2013, to allow the MoL to provide additional training courses to at-risk youth, as well as to complete pending activities, such as the final survey of the impact evaluation, which was carried out at the household level. 4 4 The former Secretary of Education and Secretary of Labor became the Ministry of Education and the Ministry of Labor in 2010 3 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry Strengths: 13. The Project was part of the GDR’s larger efforts to strengthen the social protection system. It was also fully aligned with the GDR’s strategy toward youth development. The Dominican Republic is one of the few countries in the world with a congressionally approved General Youth Law, passed in 2000, which established both a Ministry of Youth as well as a Youth Policy. The Youth Policy called for the development of educational policies to foster a holistic approach to youth development, with corresponding strategies for other sector policies, programs and community development initiatives. It also guaranteed the active participation of youth themselves in the formal and non-formal educational process. Youth representatives participated actively in the Project’s preparation and their involvement continued during implementation. 14. The Project was fully consistent with the focus of the World Bank’s Country Assistance Strategy (CAS; Report No. 31627-DO; 2005), which emphasized improving competitiveness, and achieving greater social equity through investing in human development and enhancing the access to and quality of basic services. 15. The operation built upon an already successful Youth Employment Program that had been originally financed with IADB support, which enhanced the Project’s probability of success, reduced Project start-up times, and capitalized on the existing institutional capacity of the MoL’s PCU. 16. The Project reflected state-of-the-art knowledge on active labor market interventions, building on the lessons learned from the previous version of the PJyE, and other youth skills development programs in the region. While the core of the PJyE remained the same—i.e., two-stage training followed by an internship—some innovations were incorporated, including: i) a closer relationship between the training providers and the firms providing the internships to develop training courses that reflected the firms’ specific needs; and ii) the life-skills section of the training, which had been identified by employers as the most valuable component of the training, was revamped and strengthened. 17. The PJyE also embedded key principles of an integrated workforce development system, including linkages between training and adult education programs, competition among private providers, inputs from employers in training content, and quality control mechanisms such as accreditation standards. 18. Although logistically challenging, the Project’s design brought together key institutional actors in workforce development—i.e., the MoL, INFOTEP, and MoE—and relied on them to play their natural roles within the context of the Project. This strategy eventually proved very successful, as it allowed capitalizing on the institutional capacity of the various actors and contributed toward the sustainability of outcomes. 19. The Project‘s overall design effectively balanced the role of public and private actors. Specifically, the Project provided support to the MoL, INFOTEP and the MoE to enhance their financing, quality assurance, and regulatory roles. It also relied on existing private sector providers for the provision of training. The Project was grounded in a competitive, demand-driven design that introduced a set of incentives to induce the sound 4 performance of both, private training providers and trainees, including output-based payment schemes. In addition, training providers played a key role in helping align training content with the skills needs of private firms by seeking their input and securing internships for trainees. 20. The Bank’s and the IADB’s projects were in full alignment. They both provided support to the PJyE and the randomized impact evaluation and, individually, supported complementary actions. A randomized experimental design was embedded in the Project’s design, which set the foundations for a robust evaluation of the PJyE’s impact on both labor market and non-labor market outcomes. Weaknesses 21. In hindsight, it is now apparent that several key weaknesses in the curricula of the EBA and PREPARA programs supported under Component 3 that were overlooked at appraisal. Specifically, as per findings during mid-2009, the curricula supporting the two programs were excessively long and lacked flexibility, neither of them was based on competencies, and PREPARA did not have a curricula specifically designed for adults, but instead shared the same curricula as the regular secondary education. In the absence of a drastic overhaul of their curricula, the activities envisioned under the Project (e.g., revising the textbooks) were not relevant. 5 In this context, the Bank endorsed the MoE’s decision to proceed with a review of the EBA curricula and, once that task was completed, reviewed the textbooks to support the resulting curricular changes. 22. The Project Result Framework exhibited some weaknesses, which are discussed under Section 2.3. 2.2 Implementation Factors outside the control of the Government and the implementing agency 23. The 2008 financial crisis and the ensuing global recession had a negative impact on the country’s economic activity and overall employment. The International Monetary Fund (IMF) approved a US$1.7 billion Stand-By Agreement for the GDR in November 2009 to support the country’s strategy to cope with the negative global environment by implementing countercyclical policies and reduce vulnerabilities. The 2010 Project restructuring was consistent with the GDR’s coping strategy and IMF support, as it allowed providing temporary employment opportunities to the Santiago province, which was negatively affected by the global economic crisis. However, limited budget allocations for counterpart funds and disbursement ceilings throughout the Project’s lifetime in a context of fiscal austerity slowed down its implementation. 5 See Aid Memoires from July and October 2009, when these weaknesses were detected and discussed in depth. 5 Factors Generally Subject to Government Control 24. The project got started off slowly with an 18 month effectiveness delay due to the need for congressional approval of the Loan. 25. The Project benefited from strong support from top authorities within the MoL. This support, in turn, was reflected in the high performance of the PCU that was responsible for overall project coordination and M&E, as well as the smooth implementation the PJyE. Even though the Project received continuous support from top MoE authorities throughout its lifetime, recurrent changes within the DYAE hindered the implementation of activities focused on the EBA and PREPARA programs under Component 3 (i.e., there were six different DYAE Directors during the Project’s lifetime, four Directors of the EBA and PREPARA programs and three different technical teams). Factors Generally Subject to Implementing Agency Control 26. The commitment of the MoL’s PCU toward fostering participation and collaboration was a key determinant on the rich relationships forged under the Project among the various institutional actors, including the MoE, INFOTEP, MoL, and the Ministry of Youth. Although intangible, the relationships built under the Project are highly valuable, as they have served to build trust and coordination among the various institutional actors, including incentives to foster the integration between training and adult education programs. 27. As mentioned above, implementation of Component 3 was sluggish as a result of several factors, including high rotation of DYAE authorities and technical teams, the decision to transform the EBA curricula into a modular, competency-based one, and delays in the procurement of key Project activities, such as the publishing of learning materials. As a result, the disbursement of funds represented only 72 percent of the funds originally assigned to this component. 6 The remaining funds were reallocated to Component 1 under the 2012 Project restructuring. 28. The demand-driven mechanisms envisioned under the PJyE did not fully work as planned. Although employers’ willingness to pay a stipend to PJyE interns signals that they valued the trainees’ skills, the following weaknesses were identified:  Although training providers were required to include written commitments from at least one local firm to offer internships to trainees as part of their proposals, a survey among employers that participated in the PJyE revealed that only 39 percent of them were consulted about their specific demands for skills. Thus, training offerings have been fairly general in terms of their content and not tailored to specific employers.  There have been, however, some exceptions in which new courses were developed to respond to the specific needs of employers, who, in turn, guaranteed hiring a certain number of permanent positions (i.e., Empleo Seguro). However, as denoted by the lower than expected new courses that were developed under the Program (i.e., IO 7), the incentives (i.e., greater payments to training providers, and customized course 6 The original allocation for Component 3 was US$5.0 million. Later on, this amount was reduced to US$4.0 million under the second Project restructuring, of which US3.7 million were actually disbursed. 6 offerings for employers in exchange for guaranteed employment of interns) were not effective in strengthening the alignment between training contents and employers’ demand.  Due to the time lag between the presentation of bids by training providers, the awarding of contracts, and the completion of training, some employers were no longer able to offer the internships initially promised. Thus, many trainees were placed in internships offered by employers different from those originally contacted by the training providers and, in some cases; the internships did not fully match the training they had received. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 29. M&E Design: The PJyE’s implementation was supported by a robust management information system (MIS) that incorporated all key users (i.e., the PCU, INFOTEP, and training providers). In addition, the Project placed a strong emphasis on the implementation of the MoE’s School Management System (Sistema de Gestión de Centros Educativos – SGCE) in the orbit of the EBA and PREPARA adult and youth education programs. 30. As noted earlier, one of the Project’s great strengths was the incorporation of a randomized experimental evaluation in its design that included follow-up phone surveys to PJyE graduates and an exhaustive household survey to be conducted 18 to 24 months after graduation. 7 The evaluation’s rigorous methodological design was carried out by an interdisciplinary team from the MoL, the IADB and the World Bank. The core experimental evaluation was complemented by a series of studies that explored specific aspects of the program, such as perceptions of training providers and employers. 31. Although robust, the Project Result Framework (PRF) exhibited some weaknesses, some of which reflect the challenges inherent to capturing the impact of skill development projects aimed at enhancing employability of their beneficiaries. Specifically:  Adopting an explicit definition of employability at appraisal would have been desirable to ensure the alignment of the PRF with the PDO.  KOI 1 focused on the absolute employment rate of PJyE graduates (i.e., At least 65 percent of the PJyE graduates are employed or self-employed six months after program completion), which is greatly affected by factors, such as economic activity and the structure of labor markets, which go beyond the skills training provided under the Project. In this regard, the PRF could have built upon the built-in experimental design and focus on the differences in employment outcomes between PJyE graduates and the control group.  The target value for the earnings-based indicator (i.e., KOI 2) was set too high (i.e., PJyE graduates have earnings 10 percent higher than control group.) considering that an 7 The household survey included 15 modules that collected data on household composition and socioeconomic characteristics, labor force participation, labor history, assets, time use, courses and the internship, consumption, health status, risk aversion, future expectations, pregnancy history, dwelling materials and basic skills, including non-cognitive skills and self-esteem. 7 impact evaluation of previous round of the PJyE had shown an impact effect of 7 to 10 percent conditional on employment (Card et al, 2011).  KOI 3, which was added during restructuring to measure the impact of the newly added Temporary Employment Program (i.e., Between 2,500 and 4,000 beneficiaries participate in the TEP), was limited, as it only captured the employment outputs rather than the employability of the target population. 32. M&E Implementation: The PJyE’s MIS and MoE’s SGCE were implemented as anticipated—the former one early on during the Project’s lifecycle and the latter in 2009. The Bank and the IADB coordinated efforts with the MoL to carry out the PJyE’s experimental evaluation, applying the same evaluation methodology to different cohorts. In addition, the Bank carried out a complementary evaluation based on another cohort of trainees to analyze the impact of providing only life skills versus traditional technical training. 33. M&E Utilization: The PJyE’s MIS effectively supported all stages of implementation as well as the operationalization of a results-based payment scheme to training providers. It also permitted close monitoring of progress toward the achievement of outputs and intermediate outcomes. PJyE program managers have also utilized the MIS data to evaluate its implementation and to improve subsequent phases. As a result of the SGSC implementation, the DYAE now can effectively monitor enrollment, drop-out and completion rates at the student level. 34. The PJyE’s experimental evaluation has permitted the rigorous assessment of its overall impact on employment, wages and non-cognitive skills, as well as, the drawing out of valuable lessons that can be useful in the design of similar programs in the region and beyond. While no impact evaluation was put in place for TEP, data from the Project’s M&E system was crossed checked with data from the Social Security Administration to monitor formal employment rates of TEP beneficiaries after the program’s completion. 2.4 Safeguard and Fiduciary Compliance 35. Environmental and Social Safeguards: Performance in relation to environmental and social safeguards was satisfactory. The Project’s environmental safeguard category changed from C to B as a result of physical works under the new TEP activity, under the 2010 restructuring, triggering a new safeguard (i.e., OP 4.01 Environmental Assessment). The Project’s compliance with the Environmental Management Framework was satisfactory and the minor weaknesses that were identified during implementation were subsequently corrected. 36. Financial Management – FM: FM performance was moderately satisfactory, mainly as a result of weak FM performance under Component 3, including the closing of the MoE’s Special Account in 2008 due to inactivity. Both the MoL’s PCU and the MoE utilized the FM country systems—i.e., the sub-system within the GoRD’s integrated financial administration system for projects with external financing (Sistema de Unidades Ejecutoras de Préstamos Externos - UEPEX). Financial audits for the last fiscal year were conducted on both implementing entities, and did not identify significant shortcomings, while the last external audit for the Project has already been completed. 8 37. Procurement: Performance in terms of procurement was moderately satisfactory, mostly as a result of weaknesses in the application of the procurement guidelines both in the MoL and MoE, as well as a failed competitive process for the development and publishing of textbooks under Component 3. 8 These weaknesses were due to: i) lack of a dedicated, full time procurement specialist in the MoL’s PCU; and ii) low procurement capacity and high staff turnover at the MoE. 38. Compliance with DCA Covenants: At closing all covenants had been met. 2.5 Post-completion Operation/Next Phase 39. Two new operations continue to support the GDR’s efforts to foster the employability of unskilled vulnerable population groups, particularly unskilled youth, as well as the development of labor market institutions. In addition to its continuous support for the PJyE, IADB’s Program to Support the National Employment System Project (DR- L1036, US$20.00 million, approved on July 20, 2011) focuses on improving labor intermediation, including reducing the duration of job and hiring searches. The Bank’s Flexible Employment System Project (P131109; US$20.00 million; under preparation) is aimed at strengthening the National Employment Service to expand the Temporary Employment Program. It will also support the consolidation of the PJyE into more permanent and sustainable policies as part of INFOTEP’s regular functions. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Rating: Substantial 40. The Project’s objective of enhancing the employability of poor, at-risk population groups, particularly youth, continues to be highly relevant. Unemployment has stubbornly remained at around 14 percent despite strong growth in 2009 and 2010. Likewise, unemployment is disproportionally concentrated among the youth (24 percent) and women (20 percent), with almost one third of all young women in the labor force being unemployed in 2011. As mentioned above, given the positive experience of the PJyE, the MOL has decided to continue this Program and provide sustainable financing through INFOTEP, increasing the intensity of life-skills courses and extending the duration of on- the-job training. This is clearly in line with the priorities of the new administration, which aims at creating 400,000 ‘living’ jobs (empleos dignos) as part of the Great Social Pact. The Project’s objectives are also consistent with the National Development Plan 2010-2030 (Estrategia Nacional de Desarrollo – END), which articulates a long term vision to transform the Dominican Republic, with an emphasis on achieving a society with equal 8 The design and printing of Adult Education Modules required extensive discussions with technical teams in MINERD that were constantly changing, which resulted in additional delays. Two competitive procurement processes were launched with no success. Competition was low and prices were not acceptable for the Ministry, this required a shift in the strategy. One of the possible reasons that could have caused the failure of these processes was that the size of the contract, which did not seem attractive enough for qualified editing firms to participate and offer feasible prices. 9 opportunities, including education, the gradual reduction of poverty, and the generation of decent employment. Particularly notable is the high priority that the new administration has given to the eradication of illiteracy, which has been declared a national interest. Established in 2013, the new presidential flagship program “Quisqueya Aprende Contigo” seeks to teach nearly 700,000 illiterate youth and adults to read by 2014. This initiative is conceived as a feeder for the second-chance education programs supported under the Project, the EBA program in particular. The DYAE is one of the key members of the newly constituted National Literacy Board that is responsible for implementing this initiative, with an allocated budget of RD$1,900 million. 9 In addition, the Project’s development objectives are fully aligned with the Bank’s current Dominican Republic Country Partnership Strategy 2010-2013 (Report No. 49620-DO, August 12, 2009), which emphasizes protecting the poor while enhancing competitiveness and strengthening public institutions for performance accountability. 41. The relevance of the Project’s design is considered to be Substantial. Component 1 in particular, which focused on the PJyE program, continues to be a central component of the new Bank-financed and IABD-financed operations. The Project’s embedded experimental design also contributed to its relevance, as it contributed to the development of a rigorous, evidence-based knowledge base to enhance both future lending operations and active labor market policies. Despite oversight in component 3 to recognize the need to revise the curricula of second-chance programs as a prerequisite for the other activities that were supported under the Project, the overall design remained relevant. 42. The relevance of the Project’s Implementation is deemed Substantial, as the two restructurings introduced the necessary adjustments to ensure the Project’s continuous relevance in view of changing macroeconomic conditions and implementation experience. 3.2 Achievement of Project Development Objectives Rating: Substantial 43. Overall PDO achievement is deemed Substantial for both the pre- and post- restructuring periods. Specifically: • Pre-restructuring period: As shown in Table 1, the Project exhibited substantial achievements in terms of enhancing employability of poor at-risk youth by building their work experience and life skills through the PJyE, but only negligible achievement by expanding second-chance programs (i.e., EBA and PREPARA). Given that PJyE represented the bulk of the Project’s financing (i.e., US$16.6 million for Components 1 and 2, focused on PJyE compared to only US$3.6 for Component 3, focused on the EBA and PREPARA programs), the overall achievement for PDO 1 is considered to be substantial for the pre-restructuring period. • Post-restructuring period: Achievement of PDO 1 continues to be substantial, as no modifications were introduced during the restructuring. Achievement of PDO 2, which was introduced under the 2010 restructuring, is also considered substantial as, with the 9 Plan de Gobierno (p. 18). 10 Project’ support, the MoL was able to successfully launch the first Temporary Employment Program in the Dominican Republic as envisioned. Table 1. PDO Achievement PDO Not Achieved Partially Achieved Achieved Exceeded Achievement PDO 1 - To improve the employability of poor at-risk youth by: PDO 1.a - Building their work experience and life skills (PJyE) Level of Achievement: Substantial KOIs 1 1 IOIs 1 1 6 PDO 1.b - Expanding second-chance education programs to complete their formal education. Level of Achievement: Negligible KOIs 1 IOIs 3 2 2 PDO 2 - To improve the employability of poor low-skilled long-term unemployed people between the ages of 18 and 66 Level of Achievement: Substantial KOIs 1 IOIs n.a. 44. The Project’s achievement of individual PDOs is described below and in Annex 2: PDO 1 - To improve the employability of poor at-risk youth by: a) building their work experience and life skills; and b) expanding second-chance education programs to complete their formal education. Overall, the achievement of PDO 1 is considered to be Substantial. The Project has improved the employability of participating poor youth by building their work experience and life skills, mainly through the PJyE, which was the Project’s core intervention. The impact evaluations conducted by the Bank and the IADB have identified positive impacts in terms of employment, earnings, and quality of employment for some subsets of the PJyE beneficiaries compared to the control group. Likewise, the results from the impact evaluation point to non-labor market outcomes, such as higher self-esteem and improved cognitive skills among PJyE compared to the control group. By helping the MoL and INFOTEP develop the institutional capacity required to implement skills development programs aimed at poor at-risk youth, the Project’s impact is not limited to only those trainees that benefited directly from the PJyE under the Project, but future cohorts as well. While the Project’s impact in terms of expanding second-chance programs was negligible, it still had some significant achievements, including ensuring adequate electricity supply in 350 schools that offer EBA and PREPARA education mainly in the evenings. The adaptation of the EBA curricula into a flexible, competency-based modality that includes soft skills training is expected to have a positive impact once it is implemented. As described below, the development of the new EBA curricula has already had a substantial—albeit unanticipated—impact in the design and implementation of the literacy flagship program of the new administration. Overall, activities focused on second-chance programs accounted for 16 %of overall IBRD financing. The specific achievement under PDO 1 can be summarized as follows: PDO 1.a - Impact on employability of poor at-risk youth by building their work experience and life skills - PJyE Achievement of intermediate objectives 11  Intermediate Objective 1 – The Project fully achieved Intermediate Objective 1, which called for the consolidation and expansion of the PJyE to provide increased access to a first work experience for youth at risk. Specifically, the PJyE effectively extended its coverage, with over 38,000 beneficiaries under the Project, exceeding the end-project target of 32,000 by 19%. In addition, it reached its target population, as the overwhelming majority of PJyE beneficiaries lived in the poorest areas and more than half were women. The PJyE also successfully expanded employer participation, with over 3,000 private firms offering internships to PJyE graduates, exceeding the end-project target by 50%. In addition, stronger linkages between the PJyE and second-chance education programs have been developed, with 27% of PJyE beneficiaries being registered in EBA or PREPARA one year after program completion, exceeding the end-project target by 35%. Thus, all the targets under Intermediate Indicator 1 were exceeded, with the exception of one—i.e., the expansion in the number of new course offerings, which was significantly smaller than anticipated—24 compared to the end-project target of 40 courses—which reflects the generalized nature of the training that was offered by training providers under the program as opposed to training offerings customized to the needs of specific employers as was hoped for.  Intermediate Objective 2 – The Project fully achieved Intermediate Objective 2, which called for strengthening the institutional capacity to focus on at-risk youth in both the MoL and INFOTEP with a special emphasis on program coordination, implementation, and evaluation functions. Specifically, the implementation of the PJyE was supported by a robust MIS that permitted the implementation of a well-developed set of incentives that aimed to link payments to training providers to results (e.g., the final payment being tied to the completion of an internship). In addition, the PJyE was subject to a rigorous impact evaluation on the part of both the Bank and IADB, thus contributing further to advancing the limited body of knowledge on the outcomes of training interventions. It also supported the gradual transition of the PJyE to INFOTEP, which provided key institutional support during the PJyE implementation, including financing 400 courses. Taking full advantage of INFOTEP’s institutional capacity to support the implementation of PJyE was critical not only to ensure the Project’s successful implementation but also to help expand the agency’s traditional focus to include unemployed, at- risk youth. Impact on employability of PJyE Program The Project’s impact, on enhancing employability of the target group by building their work experience and life skills, is substantial. The full achievement of the Intermediate Objectives 1 and 2 is reflected in the substantial achievement of the three Key Outcome Indicators related to the PJyE.  KOI 1 - At least 65% of PJyE graduates are employed or self-employed six months after program completion.  Employment rate reported in the follow-up household survey by PJyE beneficiaries from the 2008-2009 cohorts was 71.8% (i.e., 110 % level of achievement). 10  KOI 2 - PJyE graduates have earnings 10%higher than control group.  While there was no significant impact on earning for PJyE graduates as a whole, there is a positive income effect equivalent to roughly 7% of monthly earnings among employed PJyE 10 Vezza et al., 2013. 12 graduates, including women in general and men in Santo Domingo, as well as 5% for women in general regardless of employment status. 11 Overall, these two groups represent 67.5% of PJyE beneficiaries.  KOI 4 - Decrease in risky behaviors (substance abuse, cigarette use, alcohol use, unprotected sex, gang participation, etc.), and increase in self-esteem.  A reduction in teenage pregnancy resulting from the reproductive health components of the life-skills component. 12  The Project seems to have had a positive impact on self-esteem, as PJyE graduates, particularly females and younger individuals, report higher perceptions about their current situation and their expectations about the future. 13  Other impacts  In terms of quality of employment, results from the impact evaluations point to increased formal employment among PJyE graduates, particularly women that received training, who had a 12% increase in the probability of having retirement benefits, health insurance or a contract. 14  PJyE graduates that are working appear to have greater job satisfaction than those in the control group, as signaled by a 3% reduction in the desire to seek a different job. 15  The PJyE had a positive impact on non-cognitive skills as measured by three different scales. Scores improve between 0.08 and 0.16 standard deviations with the Program, which, as suggested by the literature, might increase employability and quality of employment. 16  A positive impact of training was also found on alternative measures of life-skills, particularly leadership skills, conflict resolution, self-organization and persistency of effort. 17  PJyE graduates are more likely to participate in political or religious associations than their peers in the control group. 18 PDO 1.b - Impact on employability of poor at-risk youth by expanding second-chance education programs to complete their formal education - EBA and PREPARA Achievement of Intermediate Objective  Intermediate Objective 3 - There was limited progress toward the achievement of this Intermediate Objective, which called for the strengthening and expansion of second chance education programs EBA and PREPARA. One of the Project’s main contributions in this area was the financing of 600 power generators that were installed in 350 schools offering EBA and PREPARA programs. Given that these generators provide electricity and thus, lighting, to schools that operate mainly in the evenings and that otherwise would have to rely on candles or gas lamps, the impact was significant in terms of providing a safer physical environment more conducive to learning. Another contribution under the Project was the overhaul of the EBA curricula, shifting from a traditional one to a flexible competency-based one that incorporates life skills training. Learning materials have been developed for seven of the 12 modules corresponding to basic education, four of which have already been published. Over 4,000 EBA teachers have been trained in the new curricula, which will be put in practice during this 11 I barrarán et al., 2012; Cruces (2013). 12 Ibarraran et al., 2012 and Vezza et al., 2013 13 Ibarraran et al., 2012. 14 Cruces (2013). 15 Vezza et al., 2013 16 Ibarraran et al., 2012. 17 Ibarraran et al., 2012. 18 Vezza et al., 2013 13 academic year for the first three modules. Since the curricular change has not yet been implemented, its impact is yet to be realized. However, the work done to develop EBA’s new curricula and the preparation of supporting learning and teaching materials permitted the DYAE to respond swiftly to the preparation and launching of one of the flagships programs established under the new administration—i.e., the Contigo Aprendo Literacy Program. Impact on employability of second-chance education programs The Project’s impact on enhancing employability of the target group by expanding second-chance education programs to complete their formal education is negligible, as the new curricula is yet to be implemented. This, in turn, is illustrated by the disappointing performance in relation to the corresponding Key Outcome Indicator:  KOI 3 - At least 80% of additional students of EBA and PREPARA obtain their basic or secondary degrees, respectively.  Graduation rates have not improved as anticipated, with a 72.7% average aggregate graduation for EBA and PREPARA compared to 70% at the beginning of the Project and 80% end-target. Graduation rates for PREPARA, however, exceeded the target (84.2%), while those of EBA are lagging behind (66.3%). PDO 2 – Improving the employability of poor low-skilled long-term unemployed people between the ages of 18 and 65. Achievement of Intermediate Objective • Intermediate Outcome 4 - To address the intense negative shock on employment in Santiago in 2010, the MoL effectively piloted a Temporary Employment Program. The Program provided the poor and less employable with an income to help them support their families, which is particularly important in times of crisis, but also provides them with a temporary job with benefits and health coverage. Based on the experience from the PJyE, Santiago Trabaja included a life-skills training program to motivate program graduates to search for a more permanent job. Impact on employability of the TEP The TEP’s impact on the employability of its beneficiaries beyond the temporary employment provided under the Project is considered to be Substantial. Life skills training and additional work experience that they received under the program, as well as potential gains in self-esteem, can be expected to have a positive effect on participants' future labor market outcomes. The project’s targets in terms of the number of TEP beneficiaries were met. In addition, after having successfully implemented the country’s first TEP, the MoL now has the institutional capacity needed to implement and scale up temporary employment programs.  KOI 5 - Between 2,500 and 4,000 beneficiaries participate in the TEP  A total of 3,920 low-income, unskilled long-term unemployed adults completed training and their temporary employment, which is at the upper bound of the original 2,000-4,000 targets.  TEP beneficiaries gained work experience by working in selected projects managed by non- profit organizations and municipalities, including, among others, maintenance of physical infrastructure such as street and drainage maintenance, and cleaning of public parks and watersheds.  In addition, TEP beneficiaries can be expected to have enhanced life skills as a result of the 14 training they received under the program.  Results obtained from cross checking TEP M&E data with administrative records from the Social Security Administration indicate that the formal employment rate of TEP beneficiaries increased with respect to non-beneficiaries after the program was completed. 19 3.3 Efficiency Rating: Modest 45. The Project’s overall economic efficiency is considered to be Modest, given the economic efficiency for each of the individual components, which can be summarized as follows: 46. Youth Employment Program (PJyE): The estimated economic efficiency of the PJyE is considered to be Modest, with an estimated Internal Economic Rate of Return (IERR) of at least 5.6 percent. This lower-bound estimated could result in a higher rate if other more difficult to quantify effects had been included. While the project had direct and indirect benefits, only a few were quantifiable which could explain the modest estimate of economic efficiency. The quantifiable economic benefits associated to the PJyE include: i) the stream of benefits derived from a 7 percent increase in earnings among employed men living in St. Domingo and 5 percent for all PJyE female graduates over a 20 year life span; and ii) the cost savings from averted teenage pregnancies considering a 5 percent reduction among the 5,813 females age 15-19 that participated in the PJyE, based on annual savings of US$1,318 per pregnancy over a 20 year time horizon. The economic efficiency of the project, or its IRR, could be higher if other economic benefits, such as those derived from having health insurance, reduced incarceration and crimes, and improved life skills would have been possible to quantify. The economic costs included the cost of training under the PJyE, equivalent to US$15 million. 47. Second-chance education programs: The economic efficiency of the investment made to enhance second-chance education programs, EBA in particular, can be expected to be High. This is as a result of this operation’s contributions to some of the expected labor market outcomes resulting from the increased earning capacity from graduates of the literacy program that was designed based on the new EBA curricula. Specifically, the economic benefits include the productivity gains that can be attributed to the improved EBA curriculum (as opposed to the presumably less effective curricula that would have been utilized otherwise). Although there is no data to perform a formal economic analysis, it can be expected to be considerable given the Program’s extensive coverage (i.e., 700,000 beneficiaries in 2013 alone). For example, considering only a 0.1 percent differential from the enhanced curriculum in the productivity gains among those enrolled during 2013 alone and assuming participants have the same characteristics as PJyE participants in terms of gender and average income, the investments made under Component 3 would yield an 19 World Bank (2013), Dominican Republic: Flexible Employment System Project (P131109), Project Appraisal Document, “based on data from the Santiago Trabaja Program and the Social Security Administration. 15 IERR of roughly 85 percent. Indirect benefits with a more effective curriculum for the literacy program would also include improved competitiveness and sustained economic growth and accelerated poverty reduction and enhanced equity. 48. Temporary Employment Program “Santiago Trabaja:” The economic efficiency of this Program is expected to be Modest. Temporary workers were assigned to work in projects proposed by non-profit organizations and municipalities. While no formal economic analysis was performed, these projects can be reasonable expected to have economic benefits greater than, or at least equal to the wages paid to temporary workers. Indirect benefits that can also be expected in terms of enhanced life skills of beneficiaries as a result of the training. These skills and added work experience, as well as the potential gains in self-esteem resulting from their participation in the TEP can be expected to have a positive effect on participants' future labor market outcomes, as suggested by the results obtained from crosschecking TEP data with that of the Social Security Administration. Given the lack of data on the type of investments under the TEP project, no economic analysis was performed for this intervention. 3.4 Justification of Overall Outcome Rating Rating: Satisfactory 49. Overall relevance is considered to be Satisfactory. Overall efficacy is considered to be Substantial, particularly taking into consideration the relative weight of the PJyE in relation to the other two interventions (i.e., second-chance education programs and TEP). Finally, the efficiency rating is considered to be Modest for both the pre- and post- restructuring periods. The combination of these partial ratings yields an overall Satisfactory rating for both the pre- and post-restructuring periods, respectively (Table 2). In turn, the overall ratings for the two periods are weighted by the disbursement percentages prior to and following the 2010 restructuring, yielding an overall weighted rating of 5.00, equivalent to an overall Satisfactory rating. Table 2. Weighted Project Rating Share of Total IBRD Pre-restructuring Post-restructuring Overall Financing Pre- and Post-restucturing outcome rating Relevance Substantial Substantial i. Employability of poor, at-risk youth a. Work experience and life skills US$16.6 million Substantial Substantial Efficacy b. Second-chance education programs US$3.6 million Negl ibigle Negligible ii. Employability of adults US$1.8 million Substantial Overall Substantial Substantial Efficiency Modest Modest Rating Satisfactory Satisfactory Overall outcome rating Rating value 5 5 Weight 43% 57% Weighted value 2,15 2,85 5,00 Final rating Satisfactory 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 50. Improved poverty-based targeting. Through its geographically based targeting, the PJyE has effectively reached youth living in the poorest areas—equivalent to 95 percent of its beneficiaries compared to the 60 percent original target. Poverty-based targeting improved for second-chance education programs EBA and PREPARA, with 76.2 percent of participants living in the poorest areas according to the DR Poverty Map. 16 51. Improved gender-focused targeting. The PJyE has effectively expanded its targeting of women, with almost 60 percent of its beneficiaries being female compared to the 45 percent original target and 40 percent at the beginning of the Project. Female participation also increased in second-chance education programs EBA and PREPARA, in which women represented 51.2 percent of all participants compared to the 45 percent original target. Approximately 20 percent of them were single mothers 52. Disproportionate impact on women: The PJyE seems to have had a greater impact on female participants in terms of income. In addition, the impact evaluation identified a reduction in teenage pregnancy by five percentage points in the treatment group (i.e., about 45 percent). (b) Institutional Change/Strengthening 53. The Project has played an active role in the creation of dynamic linkages between the programs under their orbit, including granting a tuition exception to those graduates of the PJyE that choose to continue their secondary studies under the PREPARA secondary equivalence program and actively disseminating the PJyE among those enrolled in the EBA and PREPARA programs. Likewise, by requiring that PJyE training providers work closely with the private sector identifying training needs and securing internships, the Project has helped take a first step toward enhancing the pertinence of training. Finally, the Project also had a substantial institutional impact on INFOTEP, which has expanded its regular course offerings by including courses that were taught under the PJyE. Likewise, the number of accredited training providers increased from 170 to 240 approximately, as a result of the PJyE requirement that made their accreditation mandatory. Finally, after having implemented the first TEP in the country, the MoL now has the institutional capacity to duplicate and scale-up this type of program. (c) Other Unintended Outcomes and Impacts (positive or negative) 54. As an indirect effect from the PJyE’s implementation, there has been substantial progress toward the development of institutions aimed at enhancing the functioning of the countries’ labor market, including the creation of the National Employment Service (Servicio Nacional de Empleo - SNE) and a Labor Market Observatory. All PJyE participants were automatically enrolled in the SNE. 55. As noted above, the DYAE played a key role in the swift preparation and implementation of the new Presidential Literacy Program, utilizing the structure of the flexible curricula and learning materials that had been developed for the EBA program under the Project. A total of 700,000 illiterate youth and adults have already registered in the program for 2013 alone. Although this outcome was unintended, it constitutes a significant Project achievement, particularly in view of the ambitious targets and robust budget of the Literacy Program. 56. All PJyE participants have received their national ID card and have opened a bank account, which are important steps toward their social and financial integration into society. 57. The PJyE and the lessons learned by the MoL have been widely disseminated among other countries in the region, being awarded, among other recognitions, Best 17 Practices in Youth Policies and Programs in Latin America and the Caribbean by the United Nations Development Programme (UNDP) and the Bank. 20 4. Assessment of Risk to Development Outcome Rating: Negligible to Low 58. With respect to the PJyE, the MoL will continue to transfer operational responsibility for training program targeting poor, at-risk youth to the INFOTEP. Such transfer of responsibility is a natural next step, as INFOTEP has played a central role in the PJyE’s implementation. The integration of all training programs—i.e., INFOTEP’s traditional training targeted toward formal workers and training programs targeted toward poor unemployed youth—under the agency’s regular operations will not only result in efficiency gains but will also help ensure their sustainability over the medium and long terms. The PJyE will continue to receive financial support under the new Bank-financed and IADB-financed operations. These operations will also continue to provide financial and technical support to the MoL to continue its efforts to enhance labor intermediation mechanisms. Professional staff within the PCU is in the process of being absorbed within the MoL’s permanent structure. 59. With respect to the MoE’s second-chance education programs, the new presidential initiative on combating illiteracy will serve to ensure the implementation of the newly developed EBA curricula. EBA directors are already reporting increased enrollment due to added demand from graduates from the literacy program. The DYAE has made the decision to implement the new flexible curricula in phases that are in sync with the demand of the first cohort of graduates from the literacy program. Regarding the sustainability of the equipment that was financed under the Project, it is important to underscore the importance to assign resources to program directors to purchase batteries for the generators, most of which have run out, as well as cartridges for printers. While these are relatively small purchases, they seem to surpass the financial capacity of individual schools. Likewise, large-scale purchases made by the MoE would be more price efficient than leaving it to individual schools 60. The successful implementation of the TEP ”Santiago Trabaja” provided the opportunity to the MoL to develop the institutional capacity needed to design and implement this type of program. While the GDR has not yet decided whether it will implement other TEPs, it now has the capacity needed to do so should macroeconomic conditions warrant it. 20 Puerto Vallarta, Jalisco, Mexico, Nov. 4-6, 2009. 18 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory 61. The Bank’s preparation team worked in close partnership with the Borrower. The Project was well prepared and its design was robust and innovative, integrating state-of-the- art knowledge in both the Project’s substantive area as well as its evaluation methodology. It also included a sound system of incentives to encourage the various institutional and private players to deliver and make good use of market-relevant training. As noted earlier, the incorporation of a randomized experimental evaluation into the Project’s design constituted a notable strength. The failure to identify the weaknesses in EBA and PREPARA at appraisal and some weaknesses in the PRF were the only shortcomings in an operation with an otherwise strong quality at entry. (b) Quality of Supervision Rating: Satisfactory 62. The Project was well supervised, with regular field missions and in-depth supervision reports. Sector Managers followed project implementation closely, as reflected by their detailed comments in ISRs. Missions regularly included field visits to training providers and employers participating in the PJyE and the TEP, as well as EBA and PREPARA education centers. A wide array of specialists was included in these missions, contributing valuable expertise on adult education, technical training, as well as procurement and financial management. The Bank’s team also made it possible for the PCU team to have first-hand exposure to other relevant international experiences and disseminate its own experiences through study tours and other cross-fertilization activities. The Bank exhibited flexibility and the disposition to respond to the Borrower’s needs by restructuring the project twice. The 2010 restructuring responded to the Borrower’s request to support a TEP in response to rising unemployment in the aftermath of the 2008 global financial crisis. Likewise, the 2012 restructuring allowed reallocating Loan proceeds to make effective use of Project resources and to extend the closing date to complete the final stage of the impact evaluation. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 63. Overall Bank performance is rated Moderately Satisfactory given that quality at entry and supervision are rated moderately satisfactory and satisfactory, respectively. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory 64. The Project had the full support of the MoL, INFOTEP, and MoE authorities. However, high staff rotation within DYAE and the EBA and PREPARA programs hindered 19 performance under Component 3. The pace of implementation was also hindered by recurrent budgetary problems, including lack of adequate counterpart funds and budget limits that were below the recourses required for the pace of execution programmed at appraisal. (b) Implementing Agency or Agencies Performance Rating: Satisfactory 65. Operating within the MoL, the PCU was intensively involved in the design of the Project as well as its implementation. Given the Project’s logistical complexity, sound project management was a necessary condition for its successful implementation. Throughout the Project’s lifetime, the PCU exhibited strong ownership of the Project, willingness to learn by doing, and the ability and will to reflect upon the implementation experience and make the necessary adjustments. The PCU also effectively coordinated and provided support to multiple implementing agencies, including the MoL, INFOTEP and the MoE, as well as the unit responsible for the implementation of the TEP in the city of Santiago. During the PJyE’s impact evaluation, it was also effective in coordinating the actions of the IBRD and IADB research teams to ensure internal consistency. Supported by a robust information system, the PCU was also able to effectively monitor both implementation progress as well as results. Stability within the PCU’s management brought much needed continuity in the overall’s Project management. It generously shared its experience in the implementation of the PJyE with other countries in the LAC and other regions. Overall, the performance of the MoL’s PCU is deemed Highly Satisfactory. 66. Also noteworthy was INFOTEP’s performance, which played an active role in the implementation the PJyE. The agency exhibited a collaborative and flexible attitude throughout the Project’s lifetime, contributing its know-how in content development, preparation of bidding documents, evaluation of proposals, supervision, and accreditation mechanisms of training providers. INFOTEP’s already established institutional scaffolding was a critical determinant to the PJyE’s successful implementation, including regional units and its network of accredited training centers and instructors. 67. The performance of the implementation unit responsible for the Santiago Trabaja TEP is also deemed satisfactory. It succeeded in setting in motion and effectively implementing a logistically complex program in a relatively short time, widely disseminating it among both potential beneficiaries and employers. Together with the SEN, it deployed effective recruitment campaigns on the field and provided strong technical support to potential employers, including municipalities, to ensure that their proposals were in line with the Program’s requirements. 68. As already noted, the lack of stability within the DYAE resulted in a difficult environment for Component 3, which, in turn, translated into very slow pace of implementation, leading to the closure of the MoE’s Project account due to inactivity in 2008. These implementation delays, in turn, negatively affected the Project’s outcomes, as they resulted in the delivery of fewer outputs than anticipated. Performance within the MoE, however, improved considerably from 2011onward. It is also noteworthy that DYAE has been instrumental in transferring the know-how and EBA-related outputs developed under the Project to the swift preparation and launching of the new presidential 20 administration’s flagship literacy program. Considering the performance of both implementing agencies the overall agencies’ performance is Satisfactory (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 69. Overall Borrower performance is rated Moderately Satisfactory given that the Government’s and Implementing Agencies’ Performance are rated moderately satisfactory and satisfactory, respectively. 6. Lessons Learned 70. Results from the PJyE impact evaluation point to positive albeit modest overall labor market outcomes. A large heterogeneity is observed, with more significant results reported for women in general, men in Santo Domingo, etc. While the specific design features of individual programs can play a role in determining their efficacy and economic efficiency, exogenous factors such as the level of economic activity and the structure of labor markets are also likely to play an important role. Ultimately, this type of program can help better align the supply and demand of labor skills within the envelope determined by exogenous factors, such as the structure of labor markets and macroeconomic conditions. 71. There are trade-offs between equity and efficiency objectives in the context of skills development programs such as PJyE. Targeting the most vulnerable sectors of the population, particularly poor at-risk youth who tend to have structurally less access to labor markets than their peers from other socio-economic groups, is a clear priority from an equity perspective. However, the impact on labor outcomes among this group can be expected to be limited given the short nature of the training provided and the often dramatic deficiencies in skills and education among this population group. 72. A sound system of incentives is crucial to the success of skills development programs. The PJyE was conceived as a demand-driven, results-oriented program; thus, it effectively served to induce positive behaviors on the part of individual actors, including trainees, employers and training providers. Some salient characteristics included employers’ requirement to pay for trainees’ stipends, and payment schemes to training providers that emphasize the trainee’s completion of the entire program. These characteristics were crucial determinants of the program’s success and served to keep supervision at a minimum. 73. Focusing on soft skills can improve effectiveness of training programs. Both employers and trainees appear to value them more than technical skills. In addition, results from the impact evaluation suggest that training in soft skills might be more effective in terms of employability than technical training. The experience of Santiago Trabaja also illustrates that soft skills can be valuable for older beneficiaries as well. 74. Women appear to be particularly susceptible to the potential positive impact from skills building programs targeted to poor, unskilled youth, such as the PJyE. This is particularly important, since poor young women tend to bear a disproportionate share of unemployment. Indirect effects, other than labor market outcomes, such as teenage pregnancy or reproductive health, seem to have been particularly effective among women, especially younger cohorts. 21 75. The experience of Santiago Trabaja illustrates that TEPs can be swiftly implemented in localities particularly hard hit by hikes in unemployment, providing a social safety net to individuals affected by an economic shock or natural disaster. This type of Program also benefits employers, such as municipalities and other public agencies that, under severe economic conditions, are also forced to reduce public works and other activities. The Santiago Trabaja also illustrates that these programs can be scaled down when economic conditions improve. 76. The high level of complementary exhibited by operations financed by IADB and the Bank focusing on labor markets in the Dominican Republic, as well as the coordination between, both supervision and evaluation teams of the two institutions should be considered a best practice. The Borrower’s strong ownership of the Project—the PJyE in particular—was a critical condition in ensuring the close alignment of the two banks. 77. As the Bank’s and the IADB’s evaluations utilized the same research methodology and instrument, applying to different cohorts of the same population of PJyE trainees, it would be useful to explore the possibility of combining the data and analyzing it as a whole, which would result in roughly twice the sample size as the one in each of the two evaluations, which we conducted separately. 78. Conducting a curricula assessment should be a pre-condition of any Project activities focusing on the development, publishing and distribution of learning and teaching materials. This is an important lesson that emerged from Component 3, which focused on the second-chance education programs. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies The Government of the Dominican Republic, through the Ministry of Labor provided comments, reactions and suggestions to the final draft of this document. A summary of these inputs is presented in detail in Annex 7. (b) Co-financiers See Annex 8 for comments from IADB. (b) Other partners and stakeholders N.A. 22 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Actual/Latest Appraisal Estimate Percentage of Components Estimate (USD (USD millions) Appraisal millions) 1. Consolidate and Expand the PJyE 17.0 15.0 88% 2. Building capacity in SET, SEJ, and 3.0 1.6 53% INFOTEP 3. Second-chance education programs 5.0 3.6 72% 4. Temporary Employment Program n.a. 1.8 n.a. 21 Total Project Costs 25.0 22.0 88% (b) Financing Appraisal Actual/Latest Type of Co- Percentage of Source of Funds Estimate Estimate financing Appraisal (USD millions) (USD millions) Budgetary Borrower 8.91 n.a .00 resources Inter-American Development Bank Loan 10.00 n.a 100% International Bank for Reconstruction and Loan 25.00 23.00 88% Development Bilateral Agencies (unidentified) 6.00 n.a .00 Total 49.91 n.a. n.a. 21 Note that this component was not included at appraisal stage, hence it is impossible to estimate this indicator. Also note that the resources to finance this component were reallocated, which in turn modified the disbursement rates of the other components. 23 Annex 2. Outputs by Component 22 1. Component 1 – Youth employment Program (Programa Juventud y Empleo - PJyE) The Project successfully consolidated and expanded the PJyE to provide increased access to a first work experience for youth at risk (see Box 1 for a PJyE’s description). Specifically, it effectively expanded its coverage both among that targeted population (at-risk youth), and private sector firms offering internships. It also contributed toward expanding the universe of private sector training providers. Building upon the Program’s strong evaluation tradition, the Project also supported knowledge building through a rigorous randomized evaluation. The specific outcomes under Component 1 include:  Expanded coverage Exceeded (118 percent level of achievement): The PJyE has effectively extended its coverage, with the number of beneficiaries exceeding the original target by 18 percent.  Expanded employer participation 23 Exceeded (150 percent level of achievement): The PJyE has successfully expanded employer’s participation, with over 3,000 private firms offering internships to PJyE graduates, amply exceeding the 2,000 target.  Expanded course offerings 24  Not Achieved (60 percent level of achievement): The expansion in the scope of the training felt below expectations, as the number of new course offerings was significantly smaller than anticipated—24 compared to the target of 40 new courses. While the PJyE’s demand-driven nature is clearly a factor determining course offering, it might also be pointing to the need to develop more effective mechanisms to capture employers demand for labor skills.  Improved poverty-based targeting Exceeded (158 percent level of achievement): Through its territorially based targeting, the JEP has effectively reached youth living in the poorest areas—equivalent to 95 percent of its beneficiaries compared to the 60 percent original target.  Improved gender-focused targeting Exceeded (132 percent level of achievement): The PJyE has effectively expanded its targeting of women, with almost 60 percent of its beneficiaries being female compared to the 45 percent original target and 40 percent at the beginning of the Project. 22 All Intermediate Outcome Indicator have been included in this section, including those listed under the “Results Framework” (Annex 3, p. 43) as well as the “Arrangements for Results Monitoring” tables (Annex 3, p. 46), as there was not a full overlap between the two sections. 23 While this indicator is not included under the “Results Framework” Table (Annex 3, p. 43), but it is included as one of the Intermediate Outcome Indicators under “Arrangements for Results Monitoring” (Annex 3, p. 46). 24 Same as above. 24 Box 2.1: Dominican Republic’s Youth Employment Program (Programa Juventud y Empleo - PJyE) Program Overview Program Description: In operation since 2001, the Youth and employment Program (Programa Juventud y Empleo – PJyE) is an active labor market program aimed at improving employability of youth between 16 and 29 years of age who did not complete high school.1 Content of training: The program offers a wide range of job training courses (e.g., administrative assistant, data entry, hair stylist, beautician, clerk, auto mechanic, bartender, etc.). The training, which has a total of 225 hours, is divided into two parts: i) 75 hours of basic or life skills training aimed at strengthening trainees’ self- esteem and work habits; and ii) 150 hours of technical or vocational training aimed at addressing the technical training needs of local employers. Training is followed by a two-month internship in a private sector firm. Eligibility criteria: The eligibility criteria are that participants should be 16 to 29 years old, living in poor neighborhoods; not attending school; with incomplete high school or less; unemployed, underemployed or inactive. Training providers: The Ministry of Labor outsources the provision of training services to private training institutions (Instituciones de Capacitación - ICAPs) that are registered and approved by the national training entity (Instituto Nacional de Formación Técnico Profesional - INFOTEP). The ICAPs are selected through a competitive bidding process. Demand-driven mechanisms: ICAPs play a pivotal role in: i) tailoring course offerings to match the needs of employers; and ii) matching trainees with employers. To ensure that the course offerings respond to the needs of employers, ICAPs are required to contact potential employers and to develop training programs tailored to their demand for skills. In their proposals, ICAPs are also required to include written commitments from at least one local firm to offer two-month internships to trainees graduating from the provider’s program. Young people are identified by the ICAPs according to their preferred vocation and the availability of the desired course. They are recruited in a variety of ways, including word of mouth, posters and other advertisement, and house visits. Remuneration for trainees: While trainees were not paid during the classroom component of the training, they received partial reimbursement for their transportation costs and meals, equivalent to roughly US$40 per month). The program also provided trainees with insurance against workplace accidents. Innovations: Two modifications were introduced with respect to the PJyE’s previous version: i) ICAPs were supposed to work closer to the firms that provide the internship in order to develop tailored courses to train people for real vacancies; and ii) the life-skills section of the training was subject to a drastic overhaul, as firms argued that what they valued most from training were the general job-readiness/life-skills rather than the technical training. 1 Active labor market policies (ALMPs) are government programs that intervene in the labor market to help the unemployed find work. They can be grouped in three main categories: i) public employment services; ii) training schemes, such as the PJyE; and iii) employment subsidies.  Improved monitoring and evaluation 25 Achieved (100 percent level of achievement). The implementation of the PJyE was supported by a robust management information system that permitted to link payments to training providers to results, with the final payment being tied to the completion of internship. In addition, the PJyE was subject to a rigorous impact evaluation on the part 25 This indicator is included under the “Results Framework” Table (Annex 3, p. 43), but it is not included as one of the Intermediate Outcome Indicators under “Arrangements for Results Monitoring” (Annex 3, p. 46). Thus, it has not been monitored under the Project’s ISRs. 25 of both the Bank and IADB, thus contributing further to advancing the limited body of knowledge on the outcomes of training interventions.  Improved linkages with education Exceeded (135 percent level of achievement): A total of 27 percent of PJyE beneficiaries were registered in EBA or PREPARA one year after program completion, exceeding the 20 percent original target. This was done by offering incentives for PJyE graduates to enroll in “second-chance” education programs (i.e., fee waiver) and providing them information, the Project helped build links between training and education programs targeting vulnerable young adults, which are vital components toward the development of an integrated lifelong learning strategy. 2. Component 2: Build capacity in the then Secretary of Labor (SET) and the National Training Institute (INFOTEP) to more effectively focus on at-risk youth After the joint implementation of the PJyE, the MoL and INFOTEC now have an enhanced capacity to target disadvantaged out-of school youth through soft skills and technical training programs. The implementation experience has also served to strengthen program coordination between the two entities, which is expected to continue during the gradual transition of the PJyE to INFOTEP to ensure its permanent insertion within the government bureaucracy. Specific outcomes under Component 2 include:  Expanded Coverage Achieved (100 percent level of achievement): The Dominican Republic’s national training entity, INFOTEP, provided key institutional support during the PJyE implementation, including financing 400 courses as part of the PJyE. 3. Component 3: Strengthen and expand the ‘Second Chance’ education programs EBA and PREPARA This component aimed at strengthening and expanding primary and secondary equivalency programs (EBA and PREPARA, respectively) with emphasis on accelerated courses with improved materials, modules in life skills, as well strengthening the institutional capacity of the Department for Adult Education to implement the expansion of the programs and monitor their results and impact over time. Specific outcomes under Component 3 include:  Expanding coverage - EBA and PREPARA increase their coverage by 12 and 14 percent respectively 26 Partially Achieved (38 percent and 432 percent level of achievement for EBA and PREPARA respectively): Coverage for second-chance education programs has expanded substantially at the secondary level (PREPARA with 61 percent increase in coverage) but has experienced a more modest expansion for basic education (EBA, with 4.6 percent increase in coverage).  Improved poverty-focused targeting - At least 70 percent of beneficiaries from poorest 40 percent of households 26 The target values are not consistent between those shown on the Results Framework Table (Annex 3, p. 43) and the “Results Monitoring Arrangements” (Annex 3, p. 47). The highest target values have been adopted, which is consistent with those that had been monitored by the MoE. 26 Exceeded (109 percent level of achievement): Over 76 percent of EBA and PREPARA beneficiaries live in the poorest areas identified as priority areas I and II in the Dominican Republic’s Poverty Map, exceeding the 70 percent target.  Improved gender-focused targeting - At least 45 percent of program beneficiaries are women. Exceeded (107 percent level of achievement): Women represent over half of EBA and PREPARA beneficiaries, exceeding the 45 percent target.  Improved quality - 4,500 teachers trained in Adult Education methodologies. Partially Achieved (106 percent and 0 percent level of achievement for EBA and PREPARA, respectively). The MoE decided to concentrate its efforts on enhancing second-change education at the basic-education level (i.e., EBA), drastically overhauling its curricula adopting a flexible, modular approach, and training a total of 4,247 teachers in the curricula. The new flexible EBA curricula incorporate life skills modules. No action was taken at the secondary level (i.e., PREPARA).  Effectiveness - Program drop-out rate decreases from 23 percent to 18 percent Achieved (100 percent level of achievement): The aggregate drop-out rate for the EBA and PREPARA programs was reduced from 23 percent to 18 percent, meeting the anticipated target. However, it should be noted that there are substantial differences in the drop-out rates of the two programs—24.2 percent and 7 percent for the EBA and PREPARA respectively.  Effectiveness -Books for EBA and PREPARA are distributed. Partially Achieved (25 percent level of achievement): Educational materials reflecting the newly developed flexible curricula for EBA have been developed for seven of the 12 modules, of which three have been published and are ready to be distributed to students.  Improved monitoring and evaluation Achieved (100 percent level of achievement): From having no M&E system, EBA and PREPARA now have an integrated MIS that tracks enrollments, attendance and pass rates, and provides regular tracer studies to assess longer term impact (i.e., SGCE). 4. Component 4: Support a Pilot Temporary Employment Program This component, which was added under the 2010 restructuring, provided support to a pilot TEP to reach between 2,500 to 4,000 beneficiaries. The overall duration of the TEP was 18 months, with individual beneficiaries participating for four months. The TEP was targeted to selected municipalities in the province of Santiago, in which employment was negatively affected as a result of the crisis. Specific outcomes under Component 4 include:  Expanding coverage –Between 2,500 and 4,000 low-skilled, unemployed adults participate in the temporary employment Santiago Trabaja. Achieved (100 percent level of achievement): A total of 4,395 beneficiaries were contracted under the TEP to work in 104 subprojects, with 3,988 of them receiving at least one payment. A total of 3,920 completed both the training and the temporary employment (i.e., 98.2 percent of all those that received at least one payment and 89.2 percent of all those that were contracted). . 27 Annex 3. Economic and Financial Analysis 1. The Project’s overall economic efficiency is considered to be Moderate, given the economic efficiency for each of the individual components, which can be summarized as follows: 2. Youth Employment Program (PJyE): The economic benefits from the training delivered under the PJyE can be distinguished as follows: 3. Labor market outcomes: Although the Project largely achieved the Intermediate Outcomes and KOIs specified in the PRF for Component 1, it had only a modest impact on employment and income when comparing labor market outcomes of PJyE and the control group. These results are consistent with the 2013 World Development Report ‘Jobs,’ which states that these types of programs have positive yet modest results. Specifically, results from the impact evaluations conducted under the Project have identified the following labor-market outcomes: 27  No significant impact on the probability of being employed between PJyE beneficiaries and their peers in the control group;  A modest impact on earnings affecting roughly 68 percent of PJyE graduates—i.e., roughly 7 percent increase in earnings among employed men living in Santo Domingo, and a 5 percent increase for women in general;  A 12 percent increase in the probability of having a formal job with health insurance and other benefits between employed PJyE female beneficiaries and their peers in the control group. 4. Non-labor outcomes: The impact evaluations have also identified other types of impact that, although not related to employment per se, still have an economic impact. These include the following ones:  A reduction in teenage pregnancy by five percentage points in the treatment group (about 45 percent), which is consistent with an overall increase in youth expectations about the future compared to about 11.1 percent of females aged 15-19 years old in the control group already having children or being pregnant.  Women that received training under PJyE show a 4 percent reduction in the probability of committing a crime or being incarcerated; and  A positive impact of training on alternative measures of life skills, particularly leadership skills, conflict resolution, self-organization and persistency of effort. These skills, in particular persistency of effort, have been shown that they improve labor market outcomes in developed countries (Heckman et al., 2006). 5. The economic efficiency of the PJyE is considered to be Moderate, with an estimated Internal Economic Rate of Return (IERR) of 5.6 percent. The economic benefits associated to the PJyE include: i) the stream of benefits derived from a 7 percent increase in earnings among employed men living in St. Domingo and 5 percent for all PJyE female graduates over a 20 year life span; and ii) the cost savings from averted teenage pregnancies 27 See Annex 9 for a summary of the results from the IADB’s and World Bank’s impact evaluations. 28 considering a 5 percent reduction among the 5,813 females age 15-19 that participated in the PJyE, based on annual savings of US$1,318 per pregnancy over a 20 year time horizon. 28 The other economic benefits, such as those derived from having health insurance, reduced incarceration and crimes, and improved life skills have not been quantified. The economic costs included the cost of training under the PJyE, equivalent to US$15 million (see Tables 3.1, 3.2 and 3.3). Table 3.1 Parameters Utilized in the Economic Analysis of the PJyE PJyE graduates 38,008 Males 40% Female 60% Employed (%) Males 76% Female 52% Geographic location Males SD 25% Women all 100% Monthly earnings (RD$) Males 4,558 Female 1,932 Monthly earnings (US$) Males 107.2 Female 45.5 US$ 1.00 = RD$ 42.5 Increase in earnings Employed men in SD 7.0% Women in general 5.0% Teenage pregnancy rate PJyE 6.1% Control group 11.1% Annual societal cost averted by delaying pregnancy to age 20 per pregnancy 30 1,318.5 Source: Ibarrarán et al. (2012) 28 Cost savings estimates are based on those reported by Maynard RA. (1997), Kids having kids: economic costs and social consequences of teen pregnancy, Washington DC: Urban Institute Press. They have been adjusted downwards using a 9.16 ratio, which is equal to that of the GNP per capita (Atlas Method, current US$) for the United States and the Dominican Republic as reported in the World Development Indicators Database (World Bank, 2013). 29 Table 3.2 Efficiency Estimates PJyE PJyE Graduates Males 15,203 Females 22,805 Additional annual earnings Employed graduates Males 11,524 Females 11,767 By geographic location Males (in Santo Domingo) 3,801 Females (all) 22,805 Beneficiaries with positive income effect Men (employed and in Santo Domingo) 2,858 Women in general 22,805 Annual earnings (US$) Males 1,287 Females 546 Annual increase in earnings (US$) Males 90 Females 27 Additional annual earnings under the PJyE Males 257,488 Females 622,008 Total 879,496 Teenage pregnancy Teenage pregnancies Women age 16-19 5,813 Number of averted teenage pregnancies 291 Annual societal cost savings 383,208 Program costs Cost PJyE (Component 1) US$15 million PJyE Economic Efficiency IERR 5.6% 6. Second-chance education programs: The economic efficiency of the investment made to enhance second-chance education programs, EBA in particular, can be expected to be High, as some of the expected labor market outcomes resulting from the increased earning capacity from graduates of the literacy program that was designed based on the new EBA curricula should also be attributed to this operation. Specifically, the economic benefits include the productivity gains that can be attributed to the improved EBA curriculum (as opposed to the presumably less effective curricula that would have been utilized otherwise). Although there is not sufficient data to perform a formal economic analysis, its economic impact can be expected to be considerable given the program’s extensive coverage (i.e., 700,000 beneficiaries in 2013 alone). For example, considering only a 0.1 percent differential from the enhanced curriculum in the productivity gains among those enrolled during 2013 and assuming they have the same characteristics as PJyE participants in terms of gender and average income, the investments made under Component 3 would yield an IERR of roughly 85 percent. Indirect benefits that can also be expected to arise from the Project’s implementation include improved competitiveness and 30 sustained economic growth, reduced costs of crime when idle youth are back in school or at work, and accelerated poverty reduction and enhanced equity. 7. Temporary Employment Program “Santiago Trabaja”: The economic efficiency of this program is expected to be Modest. Temporary workers were assigned to work in projects proposed by non-profit organizations and municipalities. While no formal economic analysis was performed, these projects can be reasonable expected to have economic benefits greater than, or at least equal to the wages paid to temporary workers. These projects included, among others, maintenance of physical infrastructure, like street, and drainage maintenance, and cleaning of public parks and watersheds. Indirect benefits that can also be expected in terms of enhanced life skills of beneficiaries as a result of the training. These skills, as well as potential gains in self-esteem associated with the life skills training they received under the program, can be expected to have a positive effect on participants' future labor market outcomes. Given the lack of data on the type of project, no economic analysis was performed for this program. 8. Weighted IERR: The IERR for the overall project can be expected to be around 21 percent when weighting the IERRs for the PJyE (i.e., 5.6 percent) and the Literacy Program (i.e., 84.9 percent) by their respective financing under the Project (i.e., US$15 million and US$3.6 million). The contrast between the economic efficiency of the PJyE and the Literary Program is mainly the result of the number of beneficiaries under each intervention. An important lesson learned from the implementation experience of this operation points to importance of focusing technical and financial resources on programs with large coverage in order to maximize economic efficiency. 31 Table 3.3 Economic Costs and Benefits of the Individual Programs Supported under the Operation Estimated Annual Net Type of Impact Description Ec onomic benefits Benefit (US$ million) Youth Employment Program Economic Be nefits Labor marke t outcomes 1. Increased earnings Ad ditional earnings that can be attributed to the PJyE Results from the impact evaluations point to a re considered to be an economic benefit, as they increased earnings among PJyE graduates with can be expected to re flect the productivity gains 879,494 resp ect to the control group (i.e., approx. 7% higher re sulted from th e training. earnings among 2,858 employed men living in Santo Domingo and 22,805 female PJyE graduates). 2. Quality of employment Havin g access to health care can be expected to There is an increased on formal emp loyment among h ave a positive effe ct on the health of beneficia ries, female PJyE graduates, with a 12% increase in the a s they are more likely to see med ical care earlier probability of having retirement benefits, health a nd, thus, avert po ntentially seriou s consequences of insurance or a con tract. Conversely, men that u ntreated medical condition s. Not quantified rece ived training appear less likely to have formal emp loyment than tho se that did not receive training, with a 6% decrease in the probability of having retire ment benefits, health insurance or a contra ct. Non-labor market 1. Averted teenage pregnancies It is estimated that a total of 291 pregnancies were Children born to teenage mothers are at high risk of averted as a result of PJyE participation . p reterm birth, low birth weig ht, child abuse, poverty, a nd premature death. They are also mo re likely to h ave educational underachieveme nt and mental h ealth problems. Although they exhibit higher p roductivity than the ir peers in the short turn, 383,208 teenage mothers are at high risk for dropping out of h igh school and for having low self-esteem and symptoms of dep ression. They are also more likely to re ceive public a ssistance. 2. Risk of committing a crime and Wome n that received training under PJyE show a 4% The e conomic gains include the averted economic inca rceration reduction in the pro bability of committing a crime or l oses associated to the averte d crime and the direct being incarcerated . a nd indirect costs (both indivi dual and to so ciety as a whole) resulting from incarceration. Not quantified 3. Improved life skills Training under PJyE had a p ositive impact of training These skills, in particular persistency of effort, have on a lternative measures of life skills, particul arly b een shown that they improve labor marke t leadership skill s, conflict resolution, self-organization o utcomes in developed countri es (Heckman an d Not quantified and persistency of effort. Urzua, 2006). Economic costs Program costs Costs for Component 1 (i.e., US$15 mill ion to provide train ing to over 38,000 trainees). Cost for trainees No co st is taken into considera tion, as trainees rece ived an stipend whil e participating in the program. Given the high unemployment an d low earning capacity of P JyE beneficiaries, the stipend is deemed sufficien t to offset other opportunity costs. Internal Economic Rate of Return over a 20-year timeframe 5.6% 32 Estimated Annual Net Type of Impact Description Ec onomic benefits Benefit (US$ million) "Second Chance" Programs Economic Be nefits Labor marke t outcomes 1. Increased earnings Direct labor market outcomes resulting from the The e conomic benefits in terms of earnings increase in earnin g capacity from graduates of the a ssociated to this operation include the productivity lite racy program that was d esigned base d on the g ains that can be attributed to the improved EBA new EBA curricula should also be attribu ted to this curriculum (as opposed to the presumably less operation. Although there is not sufficient data to e ffective curricula that would have utilize d otherwise). perfo rm an economi c analysis, its economic i mpact 4,695,320 can be expected to be considerable g iven the program’s extensive coverage (i.e., 700,000 beneficiaries in 2013 alone). Non-labor market 1. Indirect benefits Indirect benefi ts that can also be expected to arise from the Project’ s implementa tion include improved competitiveness and sustain ed economic growth , re duced costs of crime when idle yo uth are back in Not quantified school or at work, and accelerated poverty reduction a nd enhanced equity. Economic costs Program costs Costs for Component 3 (i.e., US$3.6 million). Cost for trainees No co sts are taken into conside ration, as beneficiaries did not spend an y a dditional time (perhaps they even gained time and effort) as a resu lt of the improved curricula. Internal Economic Rate of Return over a 20-year timeframe 84.9% Temporary Employment Program "Santiago Traba ja" Economic Be nefits Economic benefits associate d with ph 1. Increased earnings Temp orary workers were asigned to work in projects p roposed by non-profit organizations and municipaliti es. While no formal economic analysis was performed, these projects can be reasonable Not quantified e xpected to have econo mic benefits equal to or g reater than the wages paid to tempo rary worke rs. They include, among others, mainte nace of physical i nfrastructure l ike street and drainage main tenance Non-labor market 1. Indirect benefits Indirect benefi ts that can also be expected in terms o f enhanced life skills of beneficiaries as a result o f the training. These skills, as well as potential gains in self-esteem associated with the program, can b e Not quantified e xpected to have a positive effect on p articipants'future labor market outcomes. Economic costs Program costs Costs for Component 4 (i.e., US$1.8 million). Mainly th e wag es paid during temporary employment Cost for trainees No opportunity costs are taken into consideration, as f 3 920 l d d lt beneficiaries were compensated for the time they worked. Internal Economic Rate of Return over a 20-year timeframe n.a. 33 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Zoila Catherine Abreu Rojas Procurement Specialist LCSPT Fabiola Altimari Montiel Senior Counsel LEGLE LCSHS- Ligia Margarita Borrero Zea Consultant DPT Samuel C. Carlson Consultant LCSHE Peter Anthony Holland Senior Education Specialist LCSHE Patricia De la Fuente Hoyes Senior Finance Officer CTRLN HDNCY - Linda McGinnis Lead Economist His Maria Lourdes Noel Senior Program Assistant LCSEN Oswaldo Patino Consultant LCSTR Supervision/ICR Zoila Catherine Abreu Rojas Procurement Specialist LCSPT Paloma Acevedo Alameda Monitoring & Evaluation Specialist HDNCE Fabiola Altimari Montiel Senior Counsel LEGLE Carlos Andres Asenjo Ruiz Consultant MNSHD Miguel Eduardo Ceara Asuad Consultant WBIGA Christian Alberto Contin LCSHS- E T Consultant Steinemann DPT Mary A. Dowling Language Program Assistant LCSHE Peter Anthony Holland Senior Education Specialist LCSHE Patricia De la Fuente Hoyes Senior Finance Officer CTRLN Sebastian Martinez Senior Economist HDNCE HDNCY - Linda McGinnis Lead Economist His Mariana Margarita Montiel Senior Counsel LEGLE LCSHS- Juan Martin Moreno Social Protection Economist DPT Rodrigo Munoz Consultant PRMPR Maria Lourdes Noel Senior Program Assistant LCSEN Tomoko Ono Consultant LCSHH Jenniffer Perdomo Suero Consultant LCCDO Juan Prawda Consultant EASHD Gustavo Castro F. Raposo Finance Analyst CTRLS Maritza A. Rodriguez De Sr Financial Management Specia LCSFM Pichardo LCSHS- Maria Claudia Vasquez Alvarez Consultant DPT Evelyn Villatoro Senior Procurement Specialist EASR1 Luisa Sigrid Vivo Guzman Consultant HDNVP Amy L Walter Consultant LCSHE 34 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY06 47.90 221.70 FY07 0.21 0.85 Total: 48.11 222.55 Supervision/ICR FY07 10.40 46.48 FY08 14.54 67.61 FY09 17.61 91.15 FY10 39.29 126.27 FY11 30.93 95.86 FY12 28.25 119.58 FY13 32.62 86.37 Total: 173.64 633.33 35 Annex 5. Beneficiary Survey Results N.A. 36 Annex 6. Stakeholder Workshop Report and Results N.A. 37 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR The Government of the Dominican Republic through the Ministry of Labor provided inputs to the final version of this ICR. Specific comments, inputs and recommendations are classified in five categories: a) aspects that could not be implemented as originally planned; b) aspects of Project implementation that are worth repeating; c) aspects that deserve to be re-considered; d) remarks regarding the sustainability of the Project; and e) Bank-borrower coordination. Aspects that could not be implemented as originally planned a) During the internship phase, the design of the program included a 30-hour course that was expected to be delivered at the COS. Although optional, including these short modules was considered relevant for participants to share with their peers the experiences at the workplace. In future editions of the PJyE, it would be advisable to ensure that these short courses are being implemented by COS whenever necessary. b) Few COS requested the provision of a bonus (cash incentive) originally thought to increase the likelihood that firms hires participants beyond the internship phase. c) The employment rates resulted in much lower figures than expected. Only 32 percent of the internship participants received a job offer from the firm, and just half of those accepted the offer and started the job. d) In the original design, 30 percent of the beneficiaries were expected to return and complete formal education (either EBA or PREPARA). The current figures reflect that this target was almost achieved with a 27 percent rate of participants returning to formal education. e) In the absence of progress on the national youth survey, the resources from component II, originally assigned to the Secretary of Youth, had to be reallocated for other purposes within the same component. The survey could not be carried out as planned. f) A communication media campaign, originally planned to be financed out of component 2 could not be carried out due to difficulties following the World Bank procurement guidelines. g) The improvements to the information system to select PJyE beneficiaries could not be carried out by INFOTEP. It was originally thought to be financed out of resources from component 2. h) The process to register beneficiaries for the Temporary Employment Program (component 4) needed to be modified with respect to what was originally envisioned. Mobile units of the Employment Offices were used to register beneficiaries in the areas where the program was deployed. This alternative strategy proved to be more effective than enrolling beneficiaries at the downtown locations of the Employment offices in Santiago. i) The original Operations Manual of the TEP Santiago Trabaja had to be modified to avoid unintended consequences in the health coverage of certain beneficiaries. The MoL paid contributions to Social Security for beneficiaries participating in the TEP Santiago Trabaja. TEP participants that failed to report being covered by the non- 38 contributory health system automatically lost the insurance. The operations manual was modified and private risk insurance introduced in the cases in which beneficiaries were detected as covered. j) Project implementation suffered from the lack of availability of local resources to finance the expenses considered to be non-eligible in the loan agreement. Aspects of Project implementation that are worth repeating a) A very positive aspect of this Project is that from the design, it allowed the articulation of different institutions working with an overlapping target population. Including INFOTEP as an executing agency is extremely important to ensure the Project sustainability. Moreover, the Ministry of Education and the second chance education programs benefited from the development of a life-skills training curricula of the youth training programs b) Most of program participants are drop-outs aged 15 to 25 years old that are neither in education or employment. The life-skills training curricula was developed as a stand-alone course that allows young people to create their own employment search strategy, work life plan on the basis of an effective timely guidance. Efficiency gains are obtained as a consequence of the higher results achieved and the lower cost incurred in the current set up of the PJyE c) The rigorous impact evaluation included in the initial design of the PJyE, allowed feedback into the design of the successive versions of the PJyE. Strengthening the life-skills training courses is clearly the outstanding design feature modified as part of this feedback process d) The TEP Santiago Trabaja resulted in an efficient and effective program that achieved its targets in spite of the reduced number of staff responsible for its implementation. Including municipalities as potential sub-project executing entities allowed the program to finance more inclusive and varied number of community initiatives. Three design aspects have worked as expected and are worth highlighting to be repeated in the future: a) payments are made through debit cards, increasing program transparency; b) contributions to social security were paid for program participants; and c) the enrollment of participants was performed through the intermediation services at the MoL. Aspects that deserve to be re-considered a) The PJyE courses need to be updated to better reflect the demands from the private sector. b) The life skills training courses need to be adjusted upwards to include 150 hours of training. Courses on technical skills need to be adjusted accordingly to hold the overall number of training hours within budget and original designed (250 hours in total) c) Given the positive experiences found in the life-skills training courses, it would be advisable to institutional strengthen by creating a life-long learning system in which the knowledge can be transferred across cohorts of trainers. d) Monitoring course delivery and beneficiary performance can be improved by implementing a new management information system. Just in time smart reports are needed to be able to take timely decision while courses are being delivered. 39 e) The management information system should move from decentralized databases (at COS level) to a unify registry of participants to improve program delivery and management. f) Increase the amount of the incentive bonus offered to COS to promote participant labor insertion. g) The beneficiary enrollment process for the TEP needs to be improved, to avoid potential dependence between beneficiaries and NGOs or sub-project executing agencies. h) It would be advisable that life-skills teachers get paid directly. COS’ intermediation adds an extra step in the procedures without noticeable advantages. i) The MIS module for the TEP needs to be improved to ease the use of the employability index used to prioritize applicants. j) Additional resources to train administrative and PIU staff to improve their motivation and productivity. Remarks regarding the sustainability of the Project a) A Youth Entrepreneurship Unit was created within the Ministry of Labor. The former is funded through resources from the national budget with a yearly cost of US$ 83.000. b) A large part of the team of consultants that have been working in the PIU since 2001 will be no longer employed as consultants. Since 2013, the majority of the persons working at the PIU will be incorporated into the civil service of the Dominican Republic c) INFOTEP has created a special unit within the PJyE to evaluate, monitor and supervise the program’s course delivery. Bank-Borrower relationship a) The relationship between the MoL through PIU and the World Bank officials leading the program has been fluid and fruitful. Despite having had three TTLs during the lifetime of the Project, the transition from one to another was performed in a smoothly way. b) The Bank has provided great support to the various terms at the Mo, including a restructuring of the Project to launch the TEP "Santiago Trabaja". This innovative work was timely carried out as per MoL’s request to adapt the existing tools to provide response to a critical employment crisis in the city of Santiago. c) A randomized impact evaluation has been considered in original design of the program and carried out through the lifetime of the Project. The World Bank team provided technical and non-reimbursable financial assistance and successfully completed the task towards the end of the Project’s cycle. A greater knowledge transfer process is still highlighted as a challenge for future operations. d) The largest majority of the resources involved in this loan have been transferred to either beneficiaries or to training providers as part of the payment for the training courses. The Project experienced a slow start given discrepancies and interpretations generated at the operations manual and the World Bank procurement 40 guidelines. Once clarified, the Project has been able to compensate the lost time and recover the disbursement rate. e) The disbursement department based in Brazil, provided excellent and valuable support and technical assistance to the fiduciary teams at the MoL’s PIU. f) The World Bank would benefit from a deeper integration of its units, to be able to perform better and faster Project design process. Acknowledgements On behalf of the Government of the Dominican Republic, the Ministry of Labor would like to thank the World Bank staff that accompanied the implementation of the PJyE, special thanks to Juan Martin Moreno, who joined the process as part of the restructuring team in 2009 and performed the TTL role over the last two years of the Project. Cornelia Tesliuc, TTL who started the implementation of the program and Andrea Vermehren, who led the Project preparation. To the impact evaluation team: Sebastián Martinez, Paloma Acevedo y Carlos Asenjo. To the disbursement team in Brasilia: Joseph Formoso y Mónica Tambucho. To the local team in the DR, in particular to Maritza Rodriguez, FM specialist and Catherine Abreu, Procurement specialist. To the World Bank Country Managers in the DR: Christina Malmberg, Roby Senderowitsh and McDonald Benjamin. 41 Annex 8. Comments of Co-financiers and Other Partners/Stakeholders Contribution from the Inter-American Development Bank (IADB) 29 1. In 2005, the Dominican Republic was emerging from a crisis that left almost 20 percent of the population unemployed. The crisis affected particularly new labor market entrants, most of them youth and women. At the same time, there was a limited capacity to respond to this situation due to the low institutional capacity of the labor authorities and the absence of training and intermediation mechanisms, which could help the youth and unemployed to find jobs. Previous to the existence of the IDB’s Labor Markets and Social Transfers Phase I Project (DR-L1006), there was little investment in job-search services and on training for youth at risk. The existing investment in training for youth at risk lacked the appropriate approach, it did not include training on life skills, and it did not connect the trainees to labor market opportunities in the productive sector. 2. To help the DR to tackle this situation, the Project called “Labor Markets and Social Transfers Phase I” (DR-L1006), approved in 2005, funded the creation of a labor intermediation system, which would help workers to search for jobs and employers to recruit workers, as well as a youth training program called “Youth and Employment”, to improve the labor market insertion of youngsters 16-29 years old who did not complete high school. Regarding the intermediation system, the Project helped establishing an online job search database and financed the creation of 12 regional employment offices and the realization of job fairs. Regarding the program Youth and Employment, training services were outsourced to private training institutions, which were registered and approved by the National Training Institution (INFOTEP). Training activities were split in three parts: First a life skills training modulus (75 hours); a technical/vocational training modulus (150 hours); and an on-the-job training module in a private sector firm for up to three months. In order to be able to initiate stages one and two of the training activities, training institutions had to demonstrate that they had identified a company which would accept a trainee for stage three. Finally, the Project also supported a number of activities directed to strengthen the response capacity of labor authorities, such as financing labor statistics and a labor markets observatory, which would improve the quality of the information and analysis, and strengthen labor market policy. 3. The targeted population of the Project was low-income youth, workers searching for jobs, and companies seeking to hire workers. With financing from the two multilateral agencies, 57,000 low-income youth received labor training. Participants were secondary school drop-outs of which, 22.6 percent had not completed primary education. More than 50 percent of the beneficiaries were women, and 20 percent of them were single mothers. On its part, during 2008-10, the intermediation service received 101,463 unemployed people and registered 31,302 vacancies, from 4,069 companies. 29 Comments provided by Laura Ripani, Senior Economist at the IADB’s Labor Markets and Social Security Unit and TTL of the Labor Markets and Social Transfers - Phase I, and Support to the National Employment System Projects. 42 4. According to the randomized evaluation implemented in 2011, the Youth and Employment program increased the probability of formal employment by 17 percent and also had a positive impact on earnings. The evaluation also shows that the program as unexpected impacts; for example, it lowers the probability of teenage pregnancy. Based on this evaluation, the IADB has approved in 2011 another Project called Support to the National Employment System (DR-L1036) which consolidates and expands on these gains. The new program will expand training in life skills, as recent studies show them to be the most effective part of the training. It will also offer different modalities of services depending on the degree of vulnerability of beneficiaries. The National Employment Service will be consolidated and strengthened. 5. The Ministry of Labor of the Dominican Republic has used the results from this Project to promote higher investments for youth at risk in the Dominican Republic. The experimental evaluation has served as a credibility instrument for a results-based approach and help building a credibility of the Ministry in their employments services, not only the Youth and Employment programs but others as well. 6. Starting with the “Labor Markets and Social Transfers Phase I” (DR-L1006) Project, the Ministry of Labor was able to build a stronger relationship with the Institute of Vocational and Technical Training (INFOTEP). Before the Project, the relationship was almost non-existent. The Project increased the collaboration of the two institutions on training programs and the new phase of support from the IDB (Support to the National Employment System, DR-L1036) will have the two institutions as co-executors, revealing the importance and richness of that cooperation. One of the interesting spillovers of the Program has been that, understanding the importance of the provision of training on socio- emotional/basic skills, especially for the most vulnerable groups of the population, INFOTEP has introduced these modules in its regular programs for those groups. This is gain is incredible high, because it is an effort that has been institutionalized and therefore, is sustainable. 7. The Youth and Employment Program was financed by the IADB, as well as by the World Bank. The two institutions have been partners in the execution of labor markets programs in the Dominican Republic since 2005. The collaboration between the two institutions has been fruitful because there is a coherent vision of where to go in terms of the interventions necessary to overcome the challenges faced by the population. Given that shared vision, the two institutions have made the necessary arrangements to execute their programs using the same executing structure within the Ministry of Labor. In that sense, the same PCU has been in charge of executing both IDB and World Bank Projects. For example, the PJyE has been, at different points in time, financed by both agencies. 8. In terms of research collaboration, the impact evaluation teams from both agencies have been working together in the design of the randomized impact evaluation of the PJyE. This collaboration will allow coherence in the data extracted for individual evaluations and will allow putting together the two data sets in order to understand specific issues that can only be analyzed with a larger dataset. 43 Annex 9. Summary of Randomized Impact Evaluations of the Dominican Republic’s Youth Employment Program (PJyE). First Impact Evaluation by the Inter-American Development Bank (IADB) Source: Card, D., Ibarrarán, P., Regalia, F., Rosas-Shady, D. and Soares, Y. (2011), “The Labor Market Impacts of Youth Training in the Dominican Republic,” Journal of Labor Economics 29 (2), pp. 267- 300. Methodology A sample of applicants of the second cohort of the JyE program who applied to receive training in early 2004. Baseline data were collected from applicants prior to random assignment through registration forms completed at the COS. A follow-up survey was administered from May to July 2005; some 10 to 14 months after most trainees had finished their initial coursework. Main Findings  The estimated impacts on employment are all fairly close to zero, and there are no significant differences by gender, age, education, or geographic location (p. 285)  There is evidence of a modest (7% to 10% ) impact on earnings per month (conditional on employment), although the estimated effect is only marginally significant (p. 287).  The estimated impacts on monthly earnings are fairly similar for men and women, and for younger and older workers, but they show interesting patterns by education and geographic location. If one compares better-educated applicants in Santo Domingo to all others the results are striking: this subgroup accounts for virtually all of the observed positive impact on monthly earnings (p. 285-7).  This article also contributed to the literature by providing an operational definition for “employability,” based on transition probabilities between employment and non-employment status. Building on this definition, a logistic model with state dependence and unobserved heterogeneity was fitted for the observed employment transitions of the treatment and control groups. The results of the model suggest that the JE program had no significant impact on trainee employability, although a similar model shows a modest impact on job quality, as measured by the probability of holding of a job that offers health insurance. Main Conclusion Findings of this evaluation suggest that the first round of PJyE training had (at best) relatively modest effects on participants’ labor market outcomes, which is consistent with the results from evaluations in many developed countries. Although it may be possible to improve the effectiveness of this and similar programs in other Latin American and Caribbean countries, it is unlikely that programs of this nature, operating under similar financial and operational constraints, can fully address the many barriers and problems faced by disadvantaged youth in the region. Second Impact Evaluation by the IADB Source: Ibarraran, P., Ripani, L., Taboada, B., Villa, J. M. and García, B. (2012), Life Skills, Employability and Training for Disadvantaged Youth: Evidence from a Randomized Evaluation Design, Institute for the Study of Labor (IZA), Research Paper Series, IZA Discussion Paper No. 6617 (http://ssrn.com/abstract=2085204) Methodology The analysis was based on a sample of applicants for the cohort of trainees that participated in a version of the JE program that was modified as a result of the first impact evaluation. The cohort under study applied to receive training during 2008. After the completion of the courses, a follow-up household survey was carried out between November 2010 and February 2011 (18 to 24 months after graduation) on a random sample of 5000 out of the 10,309 young people who had initially registered. This sample has 3,250 individuals from the treatment group and 1,750 from the control group. (The sample size was set at 5,000 to detect an 8% increase in income with a power of 0.8 and an attrition of 30% of the sample with the sampsi Stata command.) The questionnaire for the follow-up household survey was put together by an interdisciplinary team from the Ministry of Labor of the Dominican Republic, the IADB and the World Bank. It included 15 modules that collect data on household composition and socioeconomic characteristics, labor force participation, labor history, assets, time use, courses and the internship, consumption, health status, risk aversion, future expectations, pregnancy history, dwelling materials and basic skills, including non-cognitive skills and self- esteem. 44 Main Findings In terms of labor market outcomes, the Program was found to have the following effects:  Simple comparisons between trainees in the follow-up survey and members of the control group show little impact on employment. The estimated impacts on employment are all fairly close to zero, and there are no significant differences by gender, age, education, or geographic location.  For men it had significant impacts on formality, measured as employer provided health insurance or as having a written contract. Males assigned to the treatment group are four percentage points more likely to get a job in the formal sector than males assigned to the control group. This represents an impact of 17% over the control group average. This is consistent with the results from the first evaluation of the program, and with evaluations of similar programs that report impacts on the probability of getting formal employment (Attanasio et al (2011), Ibarraran and Rosas (2009)). The results are particularly strong for males in Santo Domingo.  Estimates for monthly earnings conditional on being employed show that the program does have a positive statistically significant impact. Among those employed, individuals assigned to the treatment group have monthly earnings seven percent higher than individuals assigned to the control group. This holds for women in general, and for men in Santo Domingo.  There also seems to be a relation with the duration of unemployment, which is longer for males assigned to the treatment group than for males assigned to the control group. This could reflect the fact that males assigned to the treatment group are searching for better quality jobs even if these are harder to find, or that they have higher reservation wages. In terms of non-labor market outcomes, the program was found the following effects:  A reduction in teenage pregnancy by five percentage points in the treatment group (about 45%), which is consistent with an overall increase in youth expectations about the future. In the country, about 17%of females aged 15-19 years old already have children (ONE, 2008).  A positive impacts in terms of perceptions and expectations about the future, in particular for young women who simultaneously reduce their pregnancy rates significantly.  The Project has a positive impact in improving youth perceptions about their current situation and about their expectations regarding the future. On average, individuals assigned to the treatment group are more likely to consider having a very good health than those in the control group, and are more likely to have higher expectations about their future in topics related to their education level, neighborhood, owning a business, having the desired job, completing professional aspirations, having a better life in 20 years, offering their children a better life, and having a wealthier position and more social recognition in 10 years. These impacts are stronger for females and younger individuals.  A positive impact of training on alternative measures of life-skills, particularly leadership skills, conflict resolution, self-organization and persistency of effort. These skills, in particular persistency of effort, have been analyzed and the findings show that they improve labor market outcomes in developed countries. The impact of those soft skills on labor market performance in developing countries is a rich area for future research.  A positive impact on non-cognitive skills as measured by three different scales, with scores improving between 0.08 and 0.16 standard deviations with the program. 30 On average, the total CPS score of individuals assigned to the treatment group is 0.11 standard deviations higher than the score of individuals assigned to the control group. Impacts of similar magnitudes hold for CPS scores on leadership, behavior in situations of conflict, self-esteem, and order and self- organization. There are no impacts on the CPS scores on abilities to relate with others and empathy and communication skills. For most of the indicators these results hold for both males and females, but the impact on males tends to be larger. By groups, the impact concentrates on the youngest individuals, on the better educated--incomplete high school is the highest education level to be eligible for the program--and in Santo Domingo. However, it is hard to interpret the practical significance of these results as evidence on this topic is very limited. Further analyses carried out in this paper show no correlation between some of these measures and employment. 30 The Social and Personal Competencies Scale, which was developed in July 2010 by a team of experts to evaluate the life skills component of the program, as well as the Rosenberg and Grit scales, which are standard and proven methodologies in psychology to measure personality traits and socio-emotional competence. 45 First Impact Evaluation by the World Bank Source: Cruces, G. (2013, Resultados de la evaluación de impacto del Programa Juventud y Empleo - A partir de la encuesta final, Centro de Estudios Distributivos, Laborales y Sociales (CEDLAS), Manuscript, June 2013. Vezza, E., Cruces, G. and Amendolaggine, J. (2013), Evaluación de Impacto Programa Juventud y Empleo - República Dominicana, Manuscript, August 5, 2013. Methodology The IBRD impact evaluation applied the same methodology and questionnaire as the second IADB’s evaluation, as the two were developed jointly in collaboration with the Borrower. The household survey conducted in 2012 among 3,841 respondents from 2008 and 2009 PJyE cohorts. The only difference between the two evaluations was that the IBRD evaluation included a complementary evaluation focusing on the impact of providing only life-skills versus the traditional training. Main Findings In terms of labor market outcomes, the Program was found to have the following effects:  In terms of employment, there was no significant impact for the overall group of participants, as both groups show the same probability of being employed.  There are no significant impacts in terms of income, except for women who received technical and life skills training, who have an overall income that is 5%higher than that of its peers in the control group.  In term of quality of employment, there is an increased on formal employment among women that received training, with a 12% increase in the probability of having retirement benefits, health insurance or a contract. Conversely, men that received training appear less likely to have formal employment than those that did not receive training, with a 6%decrease in the probability of having retirement benefits, health insurance or a contract.  No significant impact on inactivity or actively seeking work, except for employed men that received training who are more actively seeking another job than their peers.  An interest finding is that participating women are more likely to have rejected a job offer in the past two years (a 4% increase in the probability). Concomitantly, the expected salary is higher (also by roughly 4%).  Participants were less likely to look for a job requiring skills different from those in which they were trained under PJyE.  Other statistically significant differences with respect the control group include a 4%higher probability among male participants that he or a family member having his/her own business.  Preliminary findings suggest that there is no valued added of the technical training when soft skills training is being provided, pointing to the importance of the latter. In terms of non-labor market outcomes, the Program was found to have the following effects:  There is a 4%higher probability to have children among those receiving training compared to the control group; and, among women, a 9% reduction in the number of children.  There is a 2%higher probability of participation in an organization.  Women that received training perceive themselves as being better off than their peers in terms of wealth right now (6%).  Men perceive themselves as being better off than their peers in the future (9%) and the likelihood of having their own business (10%).  Both men and women that received training have better expectations than their peers in terms of obtaining a desired job (5%) or achieving their professional objectives (5%).  Women that received training show a 4% reduction in the probability of committing a crime or being incarcerated. 46 Table 9.1 Summary of Labor Market Outcomes from Impact Evaluation Studies of the PJyE Phase I (2001-2005) Phase II (2006-2013) IBD's First Impact Evaluation IBD's Second Impact Evaluatio IBRD's Impact Evaluation Card et al. (2011) Ibarraran et al. (2012) Cruces (2013) Labor Market Outcomes Employment Overall sample No significant impact No significant impact No significant impact Population subsets Thre are no significant Thre are no significant differences by gender, age, differences by gender, age, education, or geographic education, or geographic location. location. Earnings Overall sample No significant impact No significant impact No significant impact Population subsets Among those employed, there Among those employed, Women who received technical is a 7% to 10% impact on individuals assigned to the and life skills training have an monthly earnings but with only treatment group have monthly overall income that is 5% higher marginal statistically earnings 7% higher than than that of its peers in the significance individuals assigned to the control group. control group. This holds for women in general, and for men in Santo Domingo. Better-educated participants in Santo Domingo account for virtually all of the observed positive impact on monthly earnings. Quality of employment Overall sample Modest impact on job quality, as measured by the probability of holding a job that offers health insurance. Population subsets Males assigned to the There is an increased on formal treatment group are four employment among women percentage points more likely to that received training, with a get a job in the formal sector 12% increase in the probability than males assigned to the of having retirement benefits, control group. This represents health insurance or a contract. an impact of 17 percent over Conversely, men that received the control group average. The training appear less likely to results are particularly strong have formal employment than for males in Santo Domingo. those that did not receive training, with a 6% decrease in the probability of having retirement benefits, health insurance or a contract. Other labor market outcomes Overall sample Preliminary findings suggest that there is no valued added of the technical training when soft skills training is being provided, pointing to the importance of the latter. Population subsets Duration of unemployment, 4% higher probability among which is longer for males male participants that he or a assigned to the treatment group family member having his/her than for males assigned to the own business. control group. This could reflect the fact that males assigned to the treatment group are searching for better quality jobs even if these are harder to find, or that they have higher reservation wages. 47 Annex 10. List of Supporting Documents Attanasio, O. P., Kugler, A. D. and Meghir, C. (2009). Subsidized vocational training for disadvantaged youth in developing countries: Evidence from a randomized trial, Discussion Paper no. 4251, IZA (Institute for the Study of Labor), Bonn. Card, D., Ibarrarán, P., Regalia, F., Rosas-Shady, D. and Soares, Y. (2011),“The Labor Market Impacts of Youth Training in the Dominican Republic, ”Journal of Labor Economics 29 (2), pp. 267-300. Cruces, G. (2013), Resultados de la evaluación de impacto del Programa Juventud y Empleo - A partir de la encuesta final, Centro de Estudios Distributivos, Laborales y Sociales (CEDLAS), Manuscript, June 2013. Fazio, M. V. (2011), Análisis de la percepción de los empleadores acerca de las pasantías y las perspectivas de inserción laboral en esas empresas, Nota Técnica IDB-TN-240, Inter-American Development Bank, Washington, D.C. González-Velosa, C. Ripani, L. and Rosas-Shady, D. (2012) ¿Cómo mejorar las oportunidades de inserción laboral de los jóvenes en América Latina?, Unidad de Mercados Laborales y Seguridad Social, Nota Técnica IDB-TN-305, February 2012, Inter-American Development Bank, Washington, D.C. Heckman, J., Stixrud, J. and Urzua, S. (2006), “The Effects of Cognitive and Noncognitive Abilities on Labor Market Outcomes and Social Behavior,” Journal of Labor Economics, 24 (3), pp. 411-482. Ibarraran, P., Ripani, L., Taboada, B., Villa, J. M. and García, B. (2012), Life Skills, Employability and Training for Disadvantaged Youth: Evidence from a Randomized Evaluation Design, Institute for the Study of Labor (IZA), Research Paper Series, IZA Discussion Paper No. 6617 (http://ssrn.com/abstract=2085204) Martínez, S. (2011), Impact Evaluation of Juventud y Empleo, Manuscript, World Bank, Washington, D.C. Vezza, E., Cruces, G. and Amendolaggine, J. (2013), Evaluación de Impacto Programa Juventud y Empleo - República Dominicana, Manuscript, August 5, 2013, Buenos Aires. World Bank (2012), Dominican Republic - Youth Development Project: Restructuring Project Paper, Report N. 6891, June 6, 2012, Washington D.C. (http://documents.worldbank.org/curated/en/2012/06/16387964/dominican-republic-youth- development-project-restructuring-vol-1-2-main-report) World Bank (2010), Dominican Republic - Youth Development Project, Amendment to the Loan Agreement, Sept. 20, 2010, Washington, D.C. (http://documents.worldbank.org/curated/en/2010/09/13153586/amendment-loan-agreement-loan- 7371-conformed) World Bank (2010), Dominican Republic - Youth Development Project, Restructuring Project Paper, Report No. 53981, August 17, 2010,Washington D.C. (http://documents.worldbank.org/curated/en/2010/08/12882091/dominican-republic-youth- development-project-restructuring) World Bank (2006), Loan Agreement L7371-DO Conformed, July 6, 2006, Washington, D.C. (http://documents.worldbank.org/curated/en/2006/07/6914041/loan-agreement-l7371-do-conformed) World Bank (2006), Dominican Republic - Youth Development Project, Project Appraisal Document, Report No. 34235, February 23, 2006, Washington D.C. (http://documents.worldbank.org/curated/en/2006/02/6637223/dominican-republic-youth- development-project). World Bank (2005), Dominican Republic Poverty Assessment: Achieving More Pro-PoorGrowth, Washington, D.C. World Bank (2005), Dominican Republic - Country Assistance Strategy, Report No. 31627, Washington D.C. 48 (http://documents.worldbank.org/curated/en/2005/05/5779425/dominican-republic-country- assistance-strategy). Meetings Conducted during ICR Mission INFOTEP Susana Sierra, Manager of Regulation, Supervision and Support to Centers Division Isabel Taveras Colón, Link between INFOTEP and PCU Ministry of Education Rosa Mery Kasse, Director, Office of International Cooperation Miriam Camilo Recio, General Director, Youth and Adult Education Jesus Manuel Mejia Constanzo, Office of International Cooperation Staff of Youth and Adult Education Office Directors of Centers of Adult Education, Regional Technical Staff Participants of Youth Employment Program Training Centers Firms that provided internship Patricia Sena, Ferretería Americana Serve Modesto Valerio, Servis Travel Chamelis Matos, Global Group Jorge Dueña, Salón Hermanos Dueña Henry Acosta. Henry Muebles Beneficiaries Temporary Employment Program (Santiago Trabaja) Beneficiaries Employers Project Coordination Unit José Luis Polanco, PCU Coordinator Douglas Hasbun, PCU Brigida García, PCU Martha Gonzalez, PCU Adi Ventura, PCU Ramon Valdez, Coordinator, Santiago Trabaja Coordinating Unit Carlos Peterson, Santiago Trabaja Coordinating Unit Wanda Pichardo, National Employment Services World Bank Juan Martín Moreno, TTL 49 72º W 71º W 70º W DOM INICAN AT LA NT IC O CEA N REP UBL IC 20º N SELECTED CITIES AND TOWNS 20º N PROVINCE CAPITALS Luperón Monte Cristi NATIONAL CAPITAL Puerto Plata Pepillo Salcedo MONTE Villa Vásquez C ord Imbert Sousúa ESPAILLAT RIVERS CRISTI Yaq ille PUERTO PLATA MAIN ROADS or t e r a ue el N Cabrera d Sep Rio San To tent Fort Liberté Fort-Liberté Dajabón Mao 3 riona l Gaspar Hernández Juan RAILROADS MARÍA PROVINCE BOUNDARIES Santigo TRINIDAD 1 Sabaneta Moca 4 SÁNCHEZ INTERNATIONAL BOUNDARIES 2 Jánico San Francisco Salcedo de Macorís Nagua Restauración San Jose This map was produced by the Map Design Unit of The World Bank. de las Matas HAITI La Vega SAMANÁ The boundaries, colors, denominations and any other information shown on this map do not imply, on the part of The World Bank AC SANTIAGO Pimental Sánchez Group, any judgment on the legal status of any territory, or any Camu DUARTE endorsement or acceptance of such boundaries. Ñ Samaná PI ord ille Rincón Fantino r a Pico Duarte Cotuí Sabana de la Mar AS (3175 m) Monte Mijo Ce LA VEGA Yu na Sabana Grande 19º N nt 5 HATO Miches 19º N ELÍ (2266 m) Bonao de Boya ra l MAYOR SAN JUAN Constanza 6 EL SEIBO To Lascahobas Elías Piña San Juan Monte Tina MONTE PLATA Cordillera Orient (2830 m) Yamasa Monte Bayaguana al El Macao Hato El Cercado Plata El Seibo Mayor O 7 za Villa Higüey ma Cordillera Altagracia Neiba AZUA San Jose 9 LA de Ocoa 8 10 BAORUCO r ALTAGRACIA u lS To Lago Azua de Jimaní Enriquillo Neiba e San Cristóbal Port au Prince Port-au-Prince IN Ya qu SANTO Boca San Pedro D EPE Duverge Vicente Nigua DOMINGO Chica de Macorís La Romana Boca de Yuma ND ENCIA Noble Bahía DISTRITO NACIONAL Bah de Ocoa Baní LA ROMANA í Nei a de To Cabral ba PERAVIA Punta Palenque Barahona Si Grand-Gosier Polo Isla P ED er 1. DAJABÓN Saona ra de Ba BARAHONA 2. SANTIAGO RODRÍGUEZ ER 18º N Pedernales or 18º N uc o 3. VALVERDE NA S 4. HERMANAS MIRABAL LE DOMINICAN 5. SÁNCHEZ RAMÍREZ REPUBLIC Car i bbe an S e a Oviedo 6. MONSEÑOR NOUEL 7. SAN JOSÉ DE OCOA 0 20 40 Kilometers 8. SAN CRISTÓBAL IBRD 33398R Isla Beata 9. SANTO DOMINGO MAY 2009 0 20 40 Miles 10. SAN PEDRO DE MACORÍS 72º W 71º W 70º W 69º W