Document of The World Bank Report No: ICR2905 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-75610) ON A LOAN IN THE AMOUNT OF US$6.29 MILLION TO THE HASHEMITE KINGDOM OF JORDAN FOR AN EMPLOYER-DRIVEN SKILLS DEVELOPMENT PROJECT March 21, 2014 Human Development Sector MNCO2 Middle East and North Africa Region CURRENCY EQUIVALENTS (Exchange Rate Effective September 30, 2013) Currency Unit = Jordanian Dinar (JD) JD 1.00 = US$1.40 US$1.00 = JD 0.7 FISCAL YEAR January 1 – 31 December ABBREVIATIONS AND ACRONYMS CAQA Center for Accreditation and Quality Assurance CAS Country Assistance Strategy CIDA Canadian International Development Agency DCU Development Coordination Unit DSS Decision Support System EDSDP Employer-Driven Skill Development Project ETF European Training Foundation E-TVET Employment, Technical and Vocational Education and Training ICR Implementation Completion and Results Report IEG Independent Evaluation Group ISR Implementation Status and Results Report JD Jordanian Dinar M&E Monitoring and Evaluation MOE Ministry of Education MOF Ministry of Finance MOHE Ministry of Higher Education MOL Ministry of Labor MOPIC Ministry of Planning and International Cooperation NET National Employment Training Company PAD Project Appraisal Document PDO Project Development Objective TA Technical Assistance USAID United States Agency for International Development VTC Vocational Training Corporation VTI Vocational Training Institute Vice President: Inger Andersen Country Director: Ferid Belhaj Acting Sector Manager: Hana Brixi Project Team Leader: Juan Manuel Moreno Olmedilla ICR Team Leader: Juan Manuel Moreno Olmedilla ii THE HASHEMITE KINGDOM OF JORDAN THE JORDAN EMPLOYER DRIVEN SKILLS DEVELOPMENT PROJECT Contents Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. PROJECT CONTEXT, DEVELOPMENT OBJECTIVES AND DESIGN ........................................ 1 2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES ...................................... 6 3. ASSESSMENT OF OUTCOMES ..........................................................................................................14 4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME...........................................................20 5. ASSESSMENT OF BANK AND BORROWER PERFORMANCE ...................................................20 6. LESSONS LEARNED.............................................................................................................................23 7. COMMENTS ON ISSUES RAISED BY BORROWER/IMPLEMENTING AGENCIES/PARTNERS............................................................................................................................25 ANNEX 1. PROJECT COSTS AND FINANCING ..............................................................................................26 ANNEX 2. OUTPUTS BY COMPONENT .........................................................................................................28 ANNEX 3. ECONOMIC AND FINANCIAL ANALYSIS .....................................................................................30 ANNEX 4. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESSES ............................31 ANNEX 5. BENEFICIARY SURVEY RESULTS ...............................................................................................33 ANNEX 6. STAKEHOLDER WORKSHOP REPORT AND RESULTS ...................................................................34 ANNEX 7. SUMMARY OF BORROWER'S ICR AND/OR COMMENTS ON DRAFT ICR ......................................35 ANNEX 8. COMMENTS OF COFINANCIERS AND OTHER PARTNERS/STAKEHOLDERS ..................................49 ANNEX 9. LIST OF SUPPORTING DOCUMENTS ............................................................................................50 MAP ..............................................................................................................................................................51 iii A. Basic Information Jordan Employer Country: Jordan Project Name: Driven Skills Development Project Project ID: P100534 L/C/TF Number(s): IBRD-75610 ICR Date: 03/24/2014 ICR Type: Core ICR HASHEMITE Lending Instrument: SIL Borrower: KINGDOM OF JORDAN Original Total USD 7.50M Disbursed Amount: USD 2.37M Commitment: Revised Amount: USD 6.29M Environmental Category: C Implementing Agencies: Ministry of Labor Cofinanciers and Other External Partners: CIDA B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 04/05/2007 Effectiveness: 07/23/2008 07/23/2008 Appraisal: 12/03/2007 Restructuring(s): 03/04/2013 Approval: 06/03/2008 Mid-term Review: 03/30/2011 05/08/2011 Closing: 09/30/2013 09/30/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Development Outcome: High Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Unsatisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Moderately Quality at Entry: Moderately Satisfactory Government: Unsatisfactory Implementing Quality of Supervision: Moderately Satisfactory Moderately Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Moderately Satisfactory Performance: Performance: Unsatisfactory iv C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any Quality of Yes None time (Yes/No): Supervision (QSA): DO rating before Moderately Closing/Inactive status: Unsatisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 31 31 Vocational training 69 69 Theme Code (as % of total Bank financing) Education for the knowledge economy 50 50 Improving labor markets 50 50 E. Bank Staff Positions At ICR At Approval Vice President: Inger Andersen Daniela Gressani Country Director: Ferid Belhaj Joseph P. Saba Sector Manager: Mourad Ezzine Michal J. Rutkowski Project Team Leader: Juan Manuel Moreno Olmedilla Soren Nellemann ICR Team Leader: Juan Manuel Moreno Olmedilla ICR Primary Author: Michael T. Mertaugh F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) Realign policy formulation with Employment, Technical and Vocational Education and Training (E-TVET) operational mechanisms through the development of employer community participation in (i) sector policy formulation, (ii) institutional development and reform, and (iii) skills development program design and delivery. Revised Project Development Objectives (as approved by original approving authority) Realign the E-TVET sector with the National Employment Strategy by enhancing the enabling conditions for employer participation in: (i) TVET institutional development; and (ii) skills development program design and delivery by the Vocational Training Corporation. v (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Number of employer-driven occupational profiles (OPs) developed and endorsed Indicator 1 : by the E-TVET Council Value quantitative or 0 NA 6 6 Qualitative) Date achieved 08/01/2008 09/30/2013 09/30/2013 09/30/2013 Comments Target was completely achieved by Year 3 of the Project and progressively being (incl. % adopted by CAQA achievement) Organizational audit, staffing plan and organizational structure introduced in Indicator 2 : VTC (a) Organizational Structure accepted in April 2013 (b) Organizational Audit accepted in Organizational May 2013 audit, staffing (c) Staffing Plan Value plan and accepted in June quantitative or Not Available not applicable organizational 2013 Qualitative) structure (d) VTC revised introduced in by-laws (Finance, VTC Supplies, HR, Administrative) submitted to Cabinet for endorsement in July 2013 Date achieved 08/01/2008 08/01/2008 09/30/2013 09/30/2013 Comments Partially achieved. The new organizational structure would be introduced to VTC (incl. % once the By-laws are approved achievement) Number of employer-driven skills development programs designed and Indicator 3 : implemented Value quantitative or 0 not applicable 3 0 Qualitative) Date achieved 08/01/2008 09/30/2013 09/30/2013 09/30/2013 Comments Three training programs (Carpentry, Air-Conditioning, and Metal Fabrication) (incl. % developed but not delivered. achievement) vi (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Development and introduction of a TVET Sector Performance Assessment Indicator 1 : System (PAS) E-TVET Sector Annual Assessment Report generated E-TVET Revised Second Sector Annual assessment report Assessment submitted in Value Report September 2013; (quantitative none not applicable discussed & and or Qualitative) endorsed by TVET Providers the E-TVET reports were not Council developed TVET providers are generating PAS-based reports Date achieved 08/01/2008 09/30/2013 09/30/2013 09/30/2013 Comments Partially achieved as the TVET providers have not generated PAS-generated (incl. % reports achievement) Indicator 2 : Increasing public awareness for E-TVET sector Value (quantitative 3.4 not applicable 3.4-3.6 not applicable or Qualitative) Date achieved 09/30/2010 09/30/2013 09/30/2013 09/30/2013 Comments (incl. % Due to MOF budget cap, the follow-up survey was not carried out. achievement) Piloting of employer-driven Business, Training and Governance Models in three Indicator 3 : VTC centers New Business Model Piloted Preliminary steps Value towards piloting New Training (quantitative not applicable not applicable conducted but not Model Piloted or Qualitative) full piloting of the models New Governance vii Model Piloted New Organizational Structure Piloted Date achieved 08/01/2008 09/30/2013 09/30/2013 09/30/2013 Comments Partially achieved. It would be difficult to quantify the preliminary steps taken (incl. % towards the piloting of the new models, considering the high risk facing the achievement) project outcomes. Indicator 4 : Increased enrolment rate at VTC 2% increase in 12% increase from Baseline value for regular regular baseline in 2012 for Value courses is 7603 courses regular courses (quantitative not applicable or Qualitative) Baseline value for short- 5 % increase 11% decrease from term courses is 5194 in short-term baseline in 2012 for courses short-term courses Date achieved 09/30/2012 09/30/2013 09/30/2013 09/30/2013 Comments (incl. % Project attribution may not be possible due to cumulative delays in Component 2. achievement) Indicator 5 : Increased female enrolment rate at VTC 2% increase in 36% increase from Baseline value for regular baseline in 2012 for Value regular courses is 1938 courses regular courses (quantitative not applicable or Qualitative) Baseline value for short- 4% increase in 42% decrease from term courses is 2031 short-term baseline in 2012 for courses short-term courses Date achieved 09/30/2012 09/30/2013 09/30/2013 09/30/2013 Comments (incl. % Project attribution may not be possible due to cumulative delays in Component 2. achievement) Indicator 6 : Completion Rate of VTC graduates Value (quantitative 65 percent not applicable 70 percent 73 percent or Qualitative) Date achieved 08/01/2008 09/30/2013 09/30/2013 09/30/2013 Comments (incl. % Project attribution may not be possible due to cumulative delays in Component 2. achievement) viii G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 11/10/2008 Satisfactory Satisfactory 0.00 2 06/18/2009 Satisfactory Satisfactory 0.50 3 12/14/2009 Satisfactory Satisfactory 0.50 4 04/21/2010 Moderately Satisfactory Moderately Satisfactory 0.59 Moderately 5 12/04/2010 Moderately Satisfactory 0.64 Unsatisfactory Moderately 6 07/17/2011 Moderately Satisfactory 0.73 Unsatisfactory Moderately 7 11/04/2011 Moderately Satisfactory 0.73 Unsatisfactory Moderately 8 05/21/2012 Moderately Satisfactory 1.08 Unsatisfactory Moderately Moderately 9 07/06/2012 1.08 Unsatisfactory Unsatisfactory Moderately Moderately 10 01/06/2013 1.71 Unsatisfactory Unsatisfactory Moderately Moderately 11 02/25/2013 1.86 Unsatisfactory Unsatisfactory Moderately Moderately 12 06/18/2013 1.86 Unsatisfactory Unsatisfactory Moderately Moderately 13 09/29/2013 2.10 Unsatisfactory Unsatisfactory Moderately Moderately 14 11/25/2013 2.35 Unsatisfactory Unsatisfactory H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions (i) revise the Project Development Objective (PDO); (ii) revise Project Components; (iii) cancel the Component 3 (related to the E-TVET Fund); 03/04/2013 Y MU MU 1.86 (iv) revise the project’s results framework; (v) reallocate funds within expenditure categories; and (vi) cancel US$1.21 million from the Loan ix If PDO and/or Key Outcome Targets were formally revised (approved by the original approving body) enter ratings below: Outcome Ratings Against Original PDO/Targets Moderately Unsatisfactory Against Formally Revised PDO/Targets Moderately Unsatisfactory Overall (weighted) rating Moderately Unsatisfactory I. Disbursement Profile x 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. Country and Sector Background. At the time of appraisal of the Employer-Driven Skills Development Project (EDSDP) in November 2007, Jordan’s economy was growing at 6 percent per year and the share of population in poverty had declined to a low level of 13 percent. Unemployment was strikingly high for a setting of such rapid growth, averaging 14 percent. Most of the unemployed were less than 25 years old. At the time of appraisal, 21.8 percent of the population was between 15 and 25 years old, 1 with 60,000 citizens entering the labor force each year. 2 An unusual feature of the employment situation in Jordan was that many unemployed Jordanians were reportedly willing to work, but only at higher-than-prevailing wage rates. 3 Foreign workers willingly filled the gap, accounting for two-thirds of new employment. 4 2. The 2006-2015 National Agenda established the overall framework for design of the EDSDP. 5 This document was developed under a participatory process involving representatives of the Government, Parliament, civil society, the private sector, media, and political parties. First-phase implementation of the National Agenda was proposed to focus on actions designed to improve employment and measures to improve the availability of entry-level workers with the skills needed in the labor market, as well as training programs to upgrade the skills of those already employed. 3. Various technical and vocational skill training programs are available in Jordan – particularly through the secondary vocational secondary schools managed by the Ministry of Education (MOE) and the skill training programs of varying duration managed by the Vocational Training Corporation (VTC). At the time of appraisal, the persistence of unemployment signaled problems of relevance and quality of the training offered in these programs. As the National Agenda noted: “The vocational training sector has been yielding poorly trained and uncommitted workers. Moreover, the lack of private sector collaboration at all stages of training has widened the gap between private sector expectations and the skillset taught to vocational trainees. Consecutive governments have taken these challenges seriously yet the efforts have yielded modest results, primarily due to fragmented programs adopted and the lack of a holistic approach in addressing this issue.” 6 4. The E-TVET Fund 7 – financed primarily by a 1 percent training levy on enterprise profits to finance skill training relevant to enterprises’ needs – was not working as intended. As the EDSDP appraisal report pointed out: “The Fund has collected about JD 40 million through a 1 percent levy on enterprise profits. However, it has been ineffective in linking the funding with training provision, outreach activities and intermediary services and has largely functioned as an effective commercial bank.” 8 1 United Nations Economic and Social Commission for West Asia (ESCWA), http://www.escwa.un.org/popin/members/Jordan.pdf 2 Page 3, Project Appraisal Document on, World Bank Report No. 41218-JO, May 2, 2008. 3 World Bank, Resolving Jordan’s Labor Market Paradox of Concurrent Economic Growth and High Unemployment, Report No. 39201-JO, March 31, 2007. 4 Ibid 5 National Agenda Steering Committee, 2006-2015 National Agenda: The Jordan We Strive For, 2006. 6 Ibid, page 25. 7 During EDSDP implementation, the TVET Fund was renamed the E-TVET Fund. 8 Pages 4-7, EDSDP PAD, World Bank, 2008, op. cit. 5. The EDSDP project was designed to address these problems. The appraisal report identifies five key issues that the project was designed to address: (i) a lack of long-term strategic planning of TVET needs and delivery; (ii) supply-driven and low quality vocational training; (iii) the absence of employer participation in decision making related to most aspects of workforce development; (iv) diverted training funds under the E-TVET Fund, and (v) undeveloped private- sector training initiatives. 9 6. In early 2007, nine priority areas were identified from the established 96 goals and agreed among stakeholders and included: (i) Establish policy making bodies; (ii) Stakeholder participation in E-TVET strategy; (iii) Capacity building and institutional development; (iv) Strengthen public-private partnerships; (v) Improve capacity of training providers; (vi) Enhance information systems; (vii) Review financing of public and private training provision, including E- TVET Fund; (viii) Initiate the development of the design of a Jordan National Qualification Framework; and (ix) Promotion/ awareness of TVET. According to the Project Appraisal Document (PAD), “through consultations and discussions with key stakeholders, it was agreed that the Bank should focus its support on the overall capacity building and system development, the restructuring of the Vocational Training Corporation (VTC)VTC, and support to Training Employment Fund(TEF)to fulfill its mandate. The project’s design is also consistent with the findings and recommendations of preparatory studies carried out under a Japanese PHRD grant,10 as well as two reports that appeared shortly before project appraisal: a 2006 sector report by the European Training Foundation (ETF) 11 and a 2007 report by the World Bank on the reasons for high unemployment under rapid economic growth. 12 The 2006 ETF report suggested that “Development partners could be instrumental in helping to design and support initial implementation of the National Agenda in technical and vocational education and training.”13As described below (Section 1.5), the EDSDP supported activities in all these areas. 7. The EDSDP’s objectives and activities were also consistent with three of the key recommendations of the 2007 World Bank report: (i) involvement of employers in the operational level of sector policy formulation and institutional development; (ii) improved efficiency and responsiveness of VTC’s training programs; and (iii) rationalized financing of vocational education and training under the E-TVET Fund. 8. An alternative approach for involving employers in vocational training was piloted under an earlier Bank-supported operation in Jordan, the Training and Employment Support Project (TESP). 14 Despite its promising results in terms of employment outcomes, 15 the project was de facto cancelled prior to completion “because the MOL no longer supported the TESP approach.” 16 Though the EDSDP PAD does not mention the TESP experience with the MOL pertaining to lessons learned, as part of project preparation, a consultant was contracted to review TESP, as well as all other background documents and interventions to be incorporated in the 9 Pages 5-7, EDSDP Appraisal Report, World Bank, 2008, op. cit. 10 Policy and Human Resources Development 11 European Training Foundation, Technical and Vocational Education in Jordan: Areas for Development Cooperation, Turin, 2006. 12 World Bank, Resolving Jordan’s Labor Market Paradox of Concurrent Economic Growth and High Unemployment, Report No. 39201-JO, March 31, 2007. 13 Page 10, European Training Foundation, 2006. 14 World Bank, Project Appraisal Document for a Proposed Loan in the Amount of US$5.0 million to the Hashemite Kingdom of Jordan for a Training and Employment Support Project, World Bank Report no. 17151-JO, April 21, 1998. 15 85% of participants were employed twelve months after completion of training. 16 Page 9, Implementation Completion Report on a Loan in the Amount of US$5 Million to the Hashemite Kingdom of Jordan for a Training and Employment Support Project, World Bank Report No. 25576, March 14, 2003. 2 project design. All lessons learned that were found relevant were included in EDSDP: Supporting employers’ participation at the institutional level, and putting great emphasis on restructuring the Vocational Training Corporation (VTC) into a demand-driven training provider. 9. Rationale for Bank Involvement. The EDSDP’s objectivities and activities were consistent with the Bank’s 2006 Country Assistance Strategy (CAS), which generally endorsed the National Agenda and gave a high priority to strengthening skills. The first of four pillars in the Bank’s 2006 strategy is “productive job creation through strengthening the investment environment and human resources for a skill-intensive and knowledge-based economy.” 17 The PAD emphasized the Bank’s experience with TVET reform. The Bank had supported manpower development in Jordan since 1972, and had acquired significant global experience in reform of vocational education and training to make it more responsive to employers’ needs. Annex 9 of the PAD described another element of the project rationale: the market failures that justified public investment in TVET. More so, the rationale for Bank involvement arose from the expected public-good benefits of the interventions in terms of improved employment and productivity, and reduced public expenditures for safety-net programs for unemployed job seekers. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 10. Original Project Development Objective. As described on page 2 and in Annex 3 of the PAD, the original project development objective was “to align policy formulation with Employment, Technical and Vocational Education and Training (E-TVET) operational mechanisms through the development of employers’ participation in (i) sector policy formulation, (ii) institutional development and reform, and (iii) skill development program design and delivery.” The presentation of the PDO in the Loan Agreement was worded differently: “The objective of the Project is to assist the Borrower in improving its TVET sector through the harmonization of policies pertaining to the development of said sector and the realignment of such policies with the operational functions of E-TVET, including the development of employer participation in the formulation of said policies and in carrying out institutional development reforms as well as in the design and delivery of a skills development program.” 18 As described below, the PDO was revised under the 2013 project restructuring and the discrepancy between the PAD and the Legal Agreement addressed. 11. Key Performance Indicators. The PAD presents four key performance indicators for monitoring and evaluating achievement of the PDO: • a functional employment, technical and vocational education and training (E-TVET) system with a functional E-TVET Council and functional Secretariat, • an autonomous, restructured VTC (new business structure and training delivery model, increased linking of financing with training centers’ fulfillment of performance targets), • improved VTC internal efficiency through increased trainee/instructor ratio (6.4 in 2006 to 10 in 2013) in restructured VTC centers or re-oriented training programs, • increased number of firms receiving training through the E-TVET Fund. 17 Page 17, International Bank For Reconstruction And Development and International Finance Corporation, Country Assistance Strategy for The Hashemite Kingdom Of Jordan for the Period FY 2006-FY 2010, World Bank Report No. 35665-JO, April 6, 2006. 18 Page 4, Loan Agreement (Employer Driven Skills Development Project) between the Hashemite Kingdom of Jordan and the International Bank for Reconstruction and Development, 8 July, 2008. 3 In addition, Annex 3 of the PAD lists fifteen intermediate outcome indicators related to the various components and subcomponents of the project. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 12. The project was restructured in March 2013 to: (i) revise the Project Development Objective (PDO); (ii) revise project components; (iii) cancel Component 3 (related to the E- TVET Fund); (iv) revise the project’s results framework; (v) reallocate funds within expenditure categories; and (vi) cancel US$1.21 million from the Loan amount. 13. The PDO was revised to read: “to realign the E-TVET sector with the National Employment Strategy by enhancing the enabling conditions for employers’ participation in (i) TVET institutional development, and (ii) skills development program design and delivery by the Vocational Training Corporation.”19 14. The PDO indicators were revised to be: • Number of employer-driven occupational profiles developed and endorsed by the E-TVET Council; • Organizational audit, staffing plan and organizational structure introduced in VTC; • Number of employer-driven skills development programs designed and implemented. 15. The project restructuring aimed to ensure and enhance the project’s relevance to Jordan’s National Employment Strategy (NES), which was formally launched in June 2012. It removed activities that were deemed unachievable within the remaining life of the project. It was envisaged that the Project would focus on activities that are relevant and realistic in the current country and sector context. By doing so, it scaled down the project’s objective from achieving employers’ participation to “enhancing the enabling conditions for employer participation”. As will be detailed in section 2.1, the key outcome of the May 2011 Mid-Term Review was the need to restructure the project, including revising the results framework. At that point, it was not possible to anticipate that the restructuring process would last for more than a year, a delay that was caused mostly by MOPIC’s doubts about the project and the restructuring itself. Yet, both the Bank and the GOJ agreed on moving ahead with the restructuring as it was understood that it was possible to accomplish the revised PDO in the remaining implementation time; a Project extension was also a possibility, and the request did in fact come from MOPIC, though just a few weeks before closing date and when it was clear that the implementation constraints did not make it likely for the Project to achieve its revised objectives. 1.4 Main Beneficiaries 16. The PAD described expected project benefits and beneficiaries as follows: “The private sector, SMEs, businesses, Chambers, Federations and Associations will benefit from increased support, the opportunity to participate, and funding which will lead to higher productivity and profitability. The employees and unemployed will benefit from better training and support and better skill levels, leading to increased employability and therefore mobility. Ultimately, the Government and the country will benefit from better labor market outcomes and an improved 19 Page 5, Restructuring Paper on a Proposed Project Restructuring of Jordan Employer Driven Skills Development Project Loan June 3, 2008 to the Hashemite Kingdom of Jordan, January 30, 2013, World Bank Report No. 4 linkage between macroeconomic policies, productivity and economic growth, and active and passive labor market policies. Specific groups like youth and women will be eligible for support under the piloting of the active labor market programs.” 20 17. The expected benefits were based on two implicit assumptions: (i) that an improved process for planning and delivering TVET with active employer involvement will lead to more relevant and higher quality training, and (ii) that these changes will in turn lead to higher levels of employment and higher productivity and earnings for TVET graduates. In effect, the building blocks developed under the project, including the new business model for VTC to cater for a more demand-driven training would enhance the relevance of training programs developed and delivered, and eventually also graduates’ employability and workers’ productivity. 1.5 Original Components (as approved) 18. The project as appraised comprised four components: Component 1 – E-TVET system and Council Development with employer participation to support institutional development of the E-TVET Council and its Secretariat and implementation of key aspects of employment-related planning, development and training with TVET stakeholders through the following subcomponents: (i) E-TVET Council orientation and capacity development; (ii) E-TVET sector policy, planning, and organizational development with active employers’ participation; (iii) E-TVET system performance assessment with active employers’ participation; and (iv) E-TVET sector promotion and awareness among TVET stakeholders. Component 2 - Restructuring of the Vocational Training Corporation to increase completion rates in training courses and to provide incentives for improved job-relevance and quality of training, through three subcomponents: (i) Establishment of the mandate, governance structure, and regulatory framework for the new Skills Development Agency; (ii) Organizational restructuring and staff development; and (iii) Reorientation of the training delivery model to reflect employer- defined competencies. Component 3 - Strengthening the E-TVET Fund to make it more responsive to labor-market skill needs, and Component 4 - Project Management. 1.6 Revised Components 19. In addition to revising the PDO and results framework (Section 1.3), the 2013 project restructuring eliminated component 3, made several changes in components 1 and 2 to remove activities that were no longer feasible or relevant, and added a new technical assistance activity to support implementation of the National Employment Strategy. Specifically, the restructuring cancelled further project support and capacity building activities for the E-TVET Council and its Secretariat, reflecting the less-than-satisfactory outcome of the Council in providing employer- driven policy guidance to its stakeholders. It also ended project support for performance based- planning, development of a medium-term expenditure framework, and realignment of budget allocations in accordance with established objectives and results framework. These changes and 20 Page 18, PAD. 5 those in the related performance targets are further described in the 2013 Restructuring Paper. 21 1.7 Other significant changes 20. Partial Loan Cancellation. The changes in project activities under the restructuring affected estimated costs where project costs declined under the restructuring from US$8.8 million to US$7.59 million; estimated loan costs declined from US$7.5 million to US$6.29 million. It was agreed to cancel the US$1.21 million of loan cost savings (driven by component 1 and component 3) under the project restructuring, resulting in a revised Loan amount of US$6.29 million. 21. Reallocation of Proceeds. The allocation of loan proceeds was changed as follows to reflect the changes introduced under the restructuring: Category of Allocation % of Financing Expenditure Current Revised 1. Goods 1,800,000 1,800,000 100% of Foreign Expenditures; 100% of Local Expenditures (ex- factory cost), and 90% of Local Expenditures for other items procured locally. 2. Consultants’ 4,881,000 4,471,250 90% for firms within territory of services, the Borrower; 95% for services of including audit, individual consultants within and Training territory of the Borrower; and 100% of foreign expenditures. 3. Unallocated 800,250 0 Front End Fee 18,750 18,750 Total 7,500,000 6,290,000 Cancelled as of October 4, 2012 1,210,000 22. A minor change in implementation responsibilities was also introduced under the restructuring, mostly to reflect a de facto situation: The E-TVET Council Steering Committee was deleted from the project because the Development Coordination Unit (DCU) and the E- TVET Council Secretariat were in fact performing the role foreseen at appraisal for such Steering Committee. 23. A subsequent request for closing date extension (dated July 21, 2013) was rejected by the World Bank because the proposed implementation schedule from September 2013-March 2014 and the level of progress witnessed did not offer a solid ground to guarantee that the revised project development objective and the end of project targets could be met within the proposed time period. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 21 Restructuring Paper on a Proposed Project Restructuring of Jordan Employer Driven Skills Development Project, Approved on June, 2008 to the Hashemite Kingdom of Jordan, World Bank Report No. RES9540, January 30, 2013. 6 24. Project Risks. The November 2007 Quality at Entry Review expressed concern about the risks related to the Government’s future commitment to TVET reform in Jordan, but agreed with the project team that the conditions at that time appeared auspicious for attempting the reforms which the project aimed to support: “Previous projects attempting to support the development of policy in the TVET sub-sector have had disappointing results, in part because of weak buy-in from the government. Recent years have seen greater evidence of political will to reform, expressed in the National Agenda, government policy documents and in government and royal backing for reform. These indications suggest that there may now be an opportunity to support meaningful reforms of the sub-sector. The pressure for reform will come also from a combination of economic factors, such as: economic growth, evolving labor market demand and competition from qualified immigrants. However, there remains a significant implementation risk, and the team has candidly recognized that the policy environment is subject to change, and that while there is high-level support for the objectives of the project, the practical implications may not be as well supported. This risk should be adequately reflected in the project risk assessment. The panel suggests that some work can be done to mitigate the implementation risk by (i) explicit planning at the appraisal stage, (ii) working closely with the other donors to ensure consistent policy messages, (iii) de-emphasizing the retrenchment of staff and centers, and (iv) providing positive institutional incentives for reform through “quick wins”.22 25. This advice was reflected in project design and implementation, 23 except for the final point: it was difficult to provide quick wins in a project focused on institutional reforms. The “Critical Risks” section of the PAD acknowledged the high risk of waning political ownership. The Team attempted to minimize this risk by situating the project firmly within the framework of the Government-endorsed National Agenda and by developing institutional capacity and broader ownership among stakeholders. The fact that the project complemented other ongoing and planned assistance from other donors also helped reduce the risk of waning government ownership. But it remained a calculated risk: As the PAD acknowledged project success would inevitably depend upon sustained government support, including support for changes in legal statues for the E-TVET Council and the VTC. 26. Project’s Objective. At the time of project appraisal, the project’s objective of aligning policy formulation with operational mechanisms; involving employers in sector policy formulation, institutional development and design and delivery of vocational training, appeared feasible given that (i) the National Agenda was developed with effective stakeholders participation and that it was adopted by the Royal Court as the “roadmap” for Jordan’s development; (ii) TVET was at the core of the National Agenda; and (iii) the Minister of Labor as well as other related representatives in the E-TVET Council showed strong ownership and commitment during project preparation. The project design also benefitted from TESP experience and lessons learnt (a consultant was hired just to identify those during project preparation). The implementation experience revealed that these expectations may have been somewhat ambitious. Yet, in the summer of 2007, none of the changes that followed could have been foreseen. This includes the global financial crisis of 2010 that impacted Jordan’s fiscal situation, resulting in lower budget ceilings for line ministries and cuts in capital expenditures (including IBRD). Still, this was adequately reflected in the Bank Team’s assessment, the Project was rated as “High” risk and was approved as such based on its relevance and potential high rewards. It was understood that there were many potential benefits in involving and including employers’ in the sector in a 22 November 6, 2007 memorandum from QER panel to the task team leader, “Quality Enhancement Review Panel Report, Jordan Employer Driven Skills Development Project, P100534” 23 This included assurances that VTC reforms to improve efficiency would not entail staff layoffs, despite redundant VTC training staffing in some specializations. 7 systematic and progressive manner and that such potential benefits were worth taking the risk. The project’s objective was to start a paradigm shift that required sustained government commitment, decision and support. It was understood that operationalizing this paradigm shift would be a long-term process, involving changes in the enabling environment and participation incentives as the first steps in an evolution towards more employer-driven training. 27. The 2013 project restructuring recognized the ambitiousness of the project objective by scaling down the PDO from “...the development of employers’ participation in sector policy formulation, institutional development and reform, and skill development program design and delivery” to “... enhancing the enabling conditions for employers’ participation in TVET institutional development and skills development program delivery by the Vocational Training Corporation.” As indicated earlier, the need to revise the PDO was agreed in May 2012; albeit the Level 1 restructuring could only be approved in May 2013. 28. In hindsight, the description of the project’s main beneficiaries may seem to be rather broad and untargeted. Yet, it does seem relevant considering the institutional nature of the PDO which aimed to create and set up “building blocks” towards institutional development, demand- driven training provision, and more generally employers’ participation in the TVET sector, all of which are not just limited to one group or the other but rather to potentially serve all the socio- economic stakeholders in the country. 29. Selectivity. In a vocational training arena that was characterized by many challenges and by the complexity of multiple, stakeholders, providers, and donors,24 the EDSDP was deliberately designed as a small operation with a selective and focused institutional development objective and design. 25 The design process involved consultation with stakeholders and donors, and built upon the findings of the preparatory work financed under a Japanese-financed PHRD 26 Grant. It also reflected the policy orientation of the National Agenda and the findings of recent research and analysis. 27 30. The EDSDP implementation experience raises two questions about whether the project objective and design should have been even more selective. The first concerns the appropriateness of EDSDP support for technical assistance to help strengthen the institutional capacity of the Training and Employment Fund. The US$60,000 of support that the EDSDP provided for this activity was minuscule in comparison to the US$6 million that the Fund was receiving from the Canadian International Development Agency (CIDA), to the extent that the choice of this component had just a symbolic value for the sake of donor harmonization. However, it was obvious from the outset that the success or failure of this activity depended upon that of the larger CIDA assistance. There was no evidence provided by the Fund to suggest an interest in seeking the Bank’s support after the closing of CIDA’s intervention prior to the agreed time, coupled with low GOJ commitment towards the Fund, did not allow a substantive enough engagement to deal effectively with the situation. This led to the eventual deletion of this component from the project. 31. A second design question concerns the role of the Ministry of Education (MOE) and 24 The PAD documents other donors activities to improve TVET effectiveness. 25 The “Alternatives Considered and Reasons for Rejection” section of the PAD discusses rejected alternative project designs involving broader scope. 26 Policy and Human Resources Development 27 Notably, World Bank, Resolving Jordan’s Labor Market Paradox of Concurrent Economic Growth and High Unemployment, Report No. 39201-JO, March 31, 2007 8 Ministry of Higher Education (MOHE) in the project. Although the Ministry of Labor was the lead implementation agency and the focus of the project was on VTC programs, the MOE and MOHE were duly represented in the E-TVET Council; the original project objective included the expectation of improved employer participation in sector policy formulation. During implementation, the MOE and MOHE were not fully engaged in the activities of the Council and the improvements sought under the project. With the benefit of hindsight, it appears that the E- TVET’s Council composition, which certainly included relevant stakeholders, was not enough to effectively address sector fragmentation and to issue enforceable sectoral policy directives. It is debated that the E-TVET Council Law has fallen short of an appropriate legal provision that would effectively engage MOE, MOHE and Balqa University representatives in a sectoral policy dialogues beyond MOL, E-TVET Fund and the VTC. This limitation was reflected in the project restructuring of 2013 (provisionally agreed upon in 2012), whereby further policy level support to the Council was eliminated, though the project maintained support to the operational mechanisms and “enabling conditions.” 2.2 Implementation 32. Any VET or skills development operation entails a lengthy and complicated preparation and its implementation requires strong ownership and sustained commitment from policy makers given the multiplicity of actors in both the public and the private sectors. This was not the case for this Project, which started off with adequate political backing and a generally supportive context but with the changing political situation in the region; and the frequent Cabinet reshuffles, , the whole governance fabric shifted. While, skills development continues to be a high priority at all levels, as clearly reflected in the recently approved National Employment Strategy, there is no longer an effective platform for national policy dialogue that can address related issues as they arise; and there is still no overhaul of the E-TVET Council and sector related policy directives. 33. Bearing the above context in mind, the, implementation of the three main project components lagged seriously behind schedule. And, in such a shifting and distracted context, attentive supervision efforts by the Bank team were insufficient to turn around the Project. The efforts made by the DCU and implementing entities also proved to be insufficient in arising the political challenges and implementation constraints to the Project. 34. The rating of implementation progress was downgraded to “moderately satisfactory” in April 2010 and to “moderately unsatisfactory” in October 2010, where it remained through closing. The E-TVET Council has not been effective in fulfilling its mandate of providing employer-driven policy directives to its stakeholders, and the Ministry of Education and Ministry of Higher Education did not prove to be willing partners to the E-TVET Council’s work. As a result, the work of the E-TVET Council has been limited to looking after the needs of the MOL’s own programs, and even there it achieved only modest results. This has been documented in the National Employment Strategy which notes that the E-TVET Council failed to play the umbrella role envisioned and was limited to setting the strategy for MOL programs, VTC, and the NET. 28 35. There were several reasons for this disconnect between project aspirations and project achievements, involving a combination of exogenous events and a failure on the part of the Government – particularly but not exclusively the Ministry of Labor -- to deliver on agreed implementation actions: 28 Page 41, National Employment Strategy 2011-2020, 9 • The E-TVET Council’s work focused on operational and implementation issues involving the VTC and the E-TVET Fund rather than on strategic and policy issues affecting TVET more generally. Be it a result or a cause of this narrower focus, the MOE and MOHE did not play the expected strategic role in the E-TVET Council and the Council failed to play the intended broader strategic role The Government did not act on the repeated recommendations of Bank supervision missions to shift the Council’s attention to the broader strategic function agreed at appraisal. • Although the MOL at the time of appraisal was supportive of the project, the successive six Ministers of Labor were not as supportive. The political commitment and focus at project conception dissipated as implementation progressed and as priorities evolved under successive changes in Ministry leadership. Apart from the varying degrees of project support under successive Ministers of Labor, the frequent changes in MOL management (every six months on average since 2011) led to inefficiency as new Ministers needed to be brought up to speed in understanding the goals, activities, and implementation challenges of the project. • The Government did not firmly pursue implementation of the 2006-2015 National Agenda, which had provided the strategic framework for the EDSDP. • In 2011, the National Agenda was replaced by a new strategic orientation in the form of the 2011-2020 National Employment Strategy (NES). 29 The NES included a more detailed analysis of skill needs and TVET training issues and more specific recommendations for TVET reform than the National Agenda, including a recommendation to make vocational training more responsive to private-sector needs. 30 Despite its initial enthusiasm, evidence of government adoption of the NES is yet to be seen. • In January 2010, the Government suspended the 1% training tax levy that was the main source of financing for the E-TVET Fund. Suspension of the training levy meant that the E- TVET Fund relied essentially on financing received from expatriate workers permit fees. Moreover, 70 percent of work-permit revenues were allocated to the National Employment and Training Company, whose training programs were primarily remedial in nature rather than oriented to the priority skill needs of employers. All this entailed that the intended purpose of the Fund continued to be diverted. Although E-TVET Fund allocated a smaller share of its revenues to the National Employment and Training Company during the final two years of implementation, it still did not fulfill the expectation of allocating Fund resources to private-sector training needs as intended under the project and National Employment Strategy. • Budget caps imposed by the Ministry of Finance (MOF) on all budget expenditures, including loan-financed activities, and the lack of proper planning to include the project’s expected expenditures within these caps, delayed a number of project activities during the final stages of EDSDP implementation. These delays were the practical reason leading to the deletion of some project activities under the 2013 restructuring, especially within Component1. • Technical assistance (TA) under the project suffered from ineffective management by the Ministry of Labor, the E-TVET Council, and the DCU involving: (i) reluctance to revisit 29 The project’s design remains fully appropriate under the new National Employment Strategy. The National Employment Strategy essentially maintains the employment and training goals of the National Agenda but adds significant detail in the diagnosis of employment and training issues in Jordan and in the specificity of recommended actions. It underlines the urgency of better planning and governance of vocational education and training, where “the challenge is not so much the diversity of providers, but rather the poor state of governance of the sector”. It also recommends improvement in the Employment, Technical and Vocational Education and Training (E-TVET) Fund, to make it more effective in meeting actual skill needs of employers. This activity was not specifically included in the National Agenda, but was included in the EDSDP PAD. 30 Page 19, National Employment Strategy 2011-2020, Jordan Ministry of Labor and Ministry of Planning and International Cooperation, 2011. 10 TORs that were deemed inappropriate over the course of activities’ implementation; (ii) lack of optimal representation or appropriate skills on the part of technical assistance TA evaluation committees, or to subordination of quality factors to cost factors in evaluation, and (iii) insufficient supervision of consultants by the E-TVET Council Secretariat, VTC and the DCU. These deficiencies contributed to several of the implementation delays experienced under the EDSDP, including the need to rebid TA to design and implement the VTC restructuring. MOPIC delays in approving termination of the initial TA for VTC restructuring also contributed to the delay in rebidding this vital activity and contract signing till May 2011. Management by the VTC of the activities under component 2 lacked dynamism and was generally reactive rather than proactive. This was reflected in cumulative delays with shared responsibility between the VTC and the TA firm, and unrealistic action plans to address the delays. 36. Project implementation suffered from a lack of an effective leadership by the Government – particularly, by the Ministry of Labor. Although the Minister of Labor at the time of appraisal was firmly committed to the project, subsequent ministers did not share that commitment. Due to a lacking legal framework, the E-TVET Council failed to motivate cooperation by the MOE and MOHE. But if the Government had been genuinely committed to the objectives of the National Agenda, and later the NES and of the project, the Prime Minister could have exercised leadership to ensure that the E-TVET Council performed its intended role in E-TVET strategy and policy development involving all stakeholders. 37. An issue that has affected many of the projects in Jordan’s portfolio, including the Employer Driven Skills Development project, is the lack of timely disbursements from the IBRD loan which in turn slows down the payment of contracts and the implementation of projects. From the perspective of the line ministry, disbursements from the loan are held back because of spending caps set by the Ministry of Finance. However, these caps actually reflect the line ministry’s budget allocations at the time the government’s annual budget is agreed, are in line with national fiscal and debt targets, and cannot be changed throughout the course of the year. As such, it is important for the line ministry to ensure that the necessary amount of the IBRD loan is factored into the ministry’s spending plan for the year and is included in the budget request to (and eventual allocation from) the Ministry of Finance. This requires that a realistic and credible spending/disbursement profile is developed on the basis of which the allocations can be made. The Bank team should work together with the line ministry counterparts to facilitate the development of such a profile and to ensure that it is prepared or updated in time to inform the national budget process. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 38. M&E Design. The project’s M&E system was designed to track implementation progress against agreed targets, and to monitor achievement against the project development objectives. The Original Performance indicators for implementation targets included staffing and staff training of the E-TVET Council Secretariat and appointment of Board members, drafting and passage of by-laws for the restructured VTC, and development of a new business model for the VTC. Performance targets for development outcomes included the number of firms receiving training under the E-TVET Fund, the percentage of VTC financial resources allocated against vocational training institute (VTI) performance, and the level of employer satisfaction with quality and relevance of VTC training. Three monitoring instruments were to be used to assemble this information: (i) tracer studies and structured surveys; (ii) key informant interviews and focus group discussions; and (iii) tracking of internal processes. 11 39. After more than four years of implementation, performance indicators were modified under the project restructuring to provide a more realistic set of expected achievements (Sections 1.6 and 2.2). As shown in the Restructuring Paper, most of the key performance indicators were dropped; targets for others were revised; and several new indicators were added. 40. M&E Implementation. M&E implementation was rated as “satisfactory” in the first ISR, then downgraded to “moderately unsatisfactory” in the second ISR (in June 2009) because the initial M&E actions were not carried out as agreed and the M&E officer in the DCU resigned. M&E was rated as “moderately satisfactory” in all subsequent ISRs through January 2013. Although progress reports were prepared and baseline studies carried out as planned 31 during this period, supervision missions continued to recommend improvements to data collection and reporting. The approval of the restructuring in March 2013 brought M&E accomplishments into alignment with the reduced expectations of the revised results framework; M&E performance rating was upgraded to “satisfactory” in the June 2013 ISR. The ICR concurs with this interpretation. 41. M&E Utilization. Because of the delays in implementation of components 1 and 2, the main uses of M&E findings have been to inform management of project implementation by the DCU and the design and implementation of the public information and awareness efforts during the final year of project implementation. 2.4 Safeguard and Fiduciary Compliance 42. Financial management implementation was essentially problem-free throughout implementation, with timely and satisfactory financial reports and unqualified audit opinions. The presence of a qualified full-time financial officer in the DCU, outsourced from outside MOL, played an instrumental role in ensuring problem-free financial management performance. Financial management implementation was rated as “satisfactory” in the first 10 ISRs for the project, and downgraded to “marginally satisfactory” during the January 2013 mission due to the exogenous factor of insufficient budget funds allocated for the project under the 2013 national budget restrictions and lack of clear arrangements for funding the DCU finance officer and M&E officer during the grace period of the project. The Bank provided an exceptional 6-month grace period that ends in March 2014 so that all payments can be processed under the 2014 national budget. It was agreed during the March 2013 supervision mission that the costs of DCU finance officer and M&E officer were to be covered by counterpart funds. 43. Procurement implementation was rated as “satisfactory” in the first three ISRs, reflecting proper procurement processing and quality of reporting, evaluations of proposals, and contract management. The rating was downgraded to “marginally satisfactory” in February 2010 due to the delay in settling the dispute that rose after termination of the technical assistance contract for VTC restructuring. The “moderately satisfactory” rating for procurement implementation was retained through project closing, reflecting the stagnation in procurement and disbursements by comparison to the appraisal schedule as a result of lack of decision making from the different project stakeholders. The rating was retained for the final mission in September 2013 due to the inability to process procurement as a result from the MOF-imposed budget restrictions. Although not directly caused by procurement actions, these shortcomings seriously affected implementation of planned procurement activities. 31 Ministry of Labour, VTC Employer Satisfaction, February, 2010 and E-TVET Awareness Raising Campaign, Knowledge, Attitudes & Perceptions, Quantitative Research Study - Baseline Survey July 2010, Amman, Jordan. 12 44. No environmental or other safeguard issues were identified during appraisal or arose during project implementation. The project did not trigger any safeguard policies. 32 2.5 Post-completion Operation/Next Phase 45. The DCU will conduct a number of dissemination activities for the key project deliverables. These include policy tools and concrete manuals of procedures, as well as reviews and assessments. Such dissemination has a potential value added in the development of the sector as it ensures that the building blocks created by this Project can eventually be put into use by policy makers and practitioners alike. It was agreed that these events will be held both in Amman and in other selected governorates to enhance adequate outreach to different stakeholders. This will be done in collaboration with the implementing entities/ project beneficiaries and in synergy with the project’s ongoing awareness campaign so that a coherent and consistent message is transmitted to all audiences. 46. E-TVET relevant tools have been developed, but are yet to be fully adopted. Under Component 1, a number of activities were achieved that are considered acceptable project outputs as well as relevant to the implementation of the National Employment Strategy (NES). These are: (i) Roles and responsibilities of the E-TVET Council and Secretariat; (ii) E-TVET Sector Action Plan, to be developed and updated by the Secretariat; (iii) adopting a participatory approach – engaging employers – while developing the procedural manual to create employer-driven occupational profiles and initial set of six profiles for demonstration purpose; (iv) the know-how to identify priority sectors and the identification of three economic priority sectors closely correlated with employment opportunities; (v) a TVET performance assessment system, including sector performance indicators, tools for data collection and data analysis, and data reporting process (on-going); and (vi) a communication strategy driven by market surveys to better understand attitudes and perceptions of the potential TVET sector beneficiaries. Still, the issues in the institutional set-up, inadequate staffing of the Secretariat and the lacking sustainable financing of the sector have prevented putting such tools into use. 47. Still a number of the main products of Component 1 have been put into operation by the E-TVET Council Secretariat, including the decision support system (DSS), sector performance assessment system, and the awareness campaign (whose preparation was financed under the loan and implementation with local resources). The Center for Accreditation and Quality Assurance (CAQA) has adopted and is applying the occupational profiles and occupational profiles manual. The priority sectors identified under the project have been adopted by the E-TVET Council and are to be revisited as part of the Council’s ongoing work in updating TVET strategy over the coming 3-5 years. 48. Under the piloting of the new VTC business, training and governance models, local business leaders worked with VTI staff to assess training needs and to adapt VTC training programs to meet those needs. The VTC is committed to implementing those new models in all of its VTIs as soon as the enabling by-laws are approved by the Cabinet. In the meantime, the VTC has taken concrete steps to launch the models on a pilot basis in three VTIs -- Yajooz, Mafraq, and Aqaba -- and has developed a two-year roadmap for extending the new models to other VTIs. VTC experts are being trained in Malaysia under the TA supported by the project to prepare them to participate in the scaling-up of the new models. 32 PAD, Annex 10. 13 49. GOJ is strongly encouraged to seek alternative routes to establish a platform for TVET policy dialogue that offers concrete policy directives to all TVET stakeholders and is binding to all TVET providers. This may not necessarily be the revival of the Higher Council for Human Resources Development, but could very well be streamlined within an inter-ministerial committee as suggested by the Bank Team during project implementation. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Relevance of Objectives: High Relevance of Design/Implementation: Significant 50. The original PDO of developing employer participation in the sector policy formulation, institutional development and reform, and design and delivery of training programs in order to improve the quality and relevance of training was highly relevant to Jordan’s development priorities and to the Bank’s country assistance strategy at the time of appraisal. This objective appears as a priority under both the National Agenda and the National Employment Strategy. As implementation revealed the limitations of achieving this objective, the PDO was changed in order to restore the PDO’s relevance to what was possible to achieve in the remaining months of project implementation in a situation of reduced political support. The revised PDO remains highly relevant to Jordan’s development priorities and to the Bank’s country partnership strategy. The project´s contribution to the sector, albeit not fully accomplished due to the MOF-imposed budget cap and the delayed delivery of the VTC pilot programs, remains relevant and valid. This is evident, for instance, in the on-going EU and ILO support to the sector, where both organizations are drawing and building on this Project’s deliverables and outcomes. In addition, now that the NES is hosted by the King Abdulla Fund, several of its key activities and interventions are being supported by the E-TVET Council and the E-TVET Fund. 51. Given the changing country context since the time of project preparation, it is worth mentioning to state that EDSDP project design was relevant to the country needs and priorities; and the results achieved, even if less than originally anticipated, are relevant as well. The Project has developed a set of critical tools at the level of institutional development and training program design that can potentially be adopted and scaled up if and when there is the political will to do so. 3.2 Achievement of Project Development Objectives 52. Original PDO: “to align policy formulation with Employment, Technical and Vocational Education and Training (E-TVET) operational mechanisms through the development of employers’ participation in (i) sector policy formulation, (ii) institutional development and reform, and (iii) skill development program design and delivery.” Rating: Modest 53. Revised PDO: “to realign the E-TVET sector with the National Employment Strategy by enhancing the enabling conditions for employers’ participation in (i) TVET institutional development, and (ii) skills development program design and delivery by the Vocational Training Corporation.” Rating: Modest 54. Enhancing the enabling conditions for employers’ participation in TVET institutional development (Significant). The Technical Assistance supporting the institutional aspects of the sector through its support to the E-TVET Council and Secretariat has provided effective “enabling conditions” that fit into the implementation of the National Employment Strategy (NES). The tools are now readily available to be put in action, once there is a clear institutional 14 set-up, funding, and policy directives to do so. a. Developed and piloted employer-driven Occupational Profiles Manual, currently adopted by CAQA: adopted a participatory approach – ensuring employers participation in developing employer driven occupational profiles and their adoption by Center of Accreditation and Quality Assurance (CAQA) both as a modus operandi as well as using the developed profiles in the accreditation process; b. Introduced an E-TVET Sector Performance Assessment system. Developed the E-TVET sector Performance Assessment System, including data collection tools on key sector level indicators, and producing the first and second TVET sector performance assessment report through the joint effort of TVET providers and industry representatives; c. Contributed towards a TVET Communication Strategy. Developing a participative TVET Communication Strategy driven by the views and perspectives of TVET stakeholders (including MOE) that led to a well-designed awareness campaign. d. Developed and piloted an employer-driven modality to identify priority sectors for new training programs. The Project has developed a mechanism for the identification of three economic priority sectors closely correlated with employment opportunities; the assessment of TVET provider’s capacity to respond to the needs expressed by the priority economic sectors; and the gap analysis between demand and supply of TVET services for each of them. And, this mechanism is now available for the sector to dynamically identify priority sectors, as well as assess TVET providers’ capacity to respond to the needs. e. Established a baseline for public perceptions of TVET. The Project has contributed towards baseline surveys pertaining to “knowledge, attitude, and perceptions” of the general public towards the TVET sector, and employers’ satisfaction with VTC programs. 55. All the above remain of solid value at the governance, regulatory and institutional levels, with employers and other public TVET providers (MOE, MOHE, and Balqaa University) actively involved in the development and the piloting of the tools. They all took part in piloting these tools over the course of the project implementation. Thus, there has been tangible progress at the working/technical level. What remains, once a political commitment and ownership emerges is to step up into the E-TVET Council level to draw on the outcome of those tools and actually produce “policy directives” that are well-informed and driven by the employers’ needs. Enhancing the enabling conditions for employers’ participation in skills development program design and delivery by the Vocational Training Corporation. (Modest) 56. Although the piloting of the models and the three pilot training programs were not fully conducted by the time of project closing 33, most of the necessary background work leading to those pilots and a number of key enabling conditions were introduced in the Vocational Training Corporation: (i) a business, governance, and training model adhering to employer-driven training delivery with clear lines of authority and accountability both at the level of VTC, the regional offices, as well as the VT Institutes (VTIs) were developed and approved. This also included a 33 Training Delivery was not launched prior to project closing. 15 modular approach towards VTC training programs that allows for open entry and exit for students and trainees; (ii) a steering committee at the level of VTC headquarters comprised of a majority of employers was created to review and take an active role in the restructuring of VTC; (iii) three pilot VTIs were effectively introduced to a potential paradigm shift in the way the VTI communicates with the employers community and in the way a Strengths, Weaknesses, Opportunities, and Threats (SWOT) analysis is carried out towards a self-assessment to be further validated by peers and with the active participation of the MOL, VTC regional office and the employers; and (iv) a rigorous M&E/MIS system, including tracer studies, is being developed. 57. Indicators measuring enrollment and completion rates were retained by the restructuring of the project. Both rates were envisaged to increase in response to the potential improvement in public perception of TVET and skills development programs as a result of the renewed training offer – employer driven – after the restructuring of the VTC and, to some extent, of the awareness campaign. Yet, considering both the evolution of the Jordanian labor market and more importantly the changes introduced in opening paths from VET to higher education, it is clear that the increase in enrolment in VTC programs cannot be attributed exclusively to the effects of the activities supported by the Project. On the other hand, a partial impact cannot be denied either, particularly in the area of hospitality and tourism training programs. 58. The Project has provided concrete building blocks and a clear set of enabling conditions for the GOJ to build on and for donors to draw on while designing new interventions, as detailed below. Most remarkably, the Bank Supervision Team field visit in March 2013 to the three pilot VTC Institutes provided solid reasons to be optimistic about the pilots, as the managers and staff of the three VTIs: (a) showed high commitment to the new business models; (b) appreciated the training received and the self-assessment process and procedures initiated; (c) pointed to the strategic value of having established internal quality units and external committees that institutionalize the dialogue with employers; (d) firmly believed that the new models could easily be extended and scaled up to other specializations and VTIs; and (e) showed clear awareness of the need to conduct tracer studies of their graduates (the Center in Aqaba is actually doing it as of this academic year). 59. Overall. As shown in Annex 2, the project delivered most of the outputs that were expected of it under the restructuring, but the expected development outcome has not been achieved because: • For Component 1, the E-TVET Council has not been effective in fulfilling its mandate of providing employer-driven policy directives to its stakeholders, although a number of relevant tools has been developed under the project that contribute to more employer- responsive TVET programs. All those tools can be quickly implemented once the institutional set-up, funding and policy directives are ready. • The imposed MOF cap on project funds led to the cancellation of strategic importance activities (despite not having major budget implications) that were meant to contribute to the sector’s enabling conditions. These activities included: (a) technical assistance to support a tri-partite working committee under Component 1 to facilitate the policy dialogue among the three TVET Councils/Boards to promote awareness of their roles and responsibilities. It would have been very useful to adequately rationalize the links between the VTC and the MOE-Vocational Education in terms of resources, infrastructure, teaching capacity, practical training, and outputs relevant to the market needs; (b) dissemination of project deliverables; (c) follow-up survey on knowledge, attitudes and perceptions towards TVET to determine impact of the awareness campaign; and (d) media spots included in the awareness campaign. 16 For Component 2, the new VTC business, governance, and training models have not yet been fully piloted, pending delivery of the pilot training programs and the Cabinet approval of the VTC new by-laws that would support the introduction of the revised organization structure in the VTC. It should be noted that this Component suffered from extensive delays originally caused by the need to rebid the TA for reasons completely beyond the control or responsibility of the DCU and VTC; and then by challenging implementation plans proposed by the TA firm and the VTC itself that were difficult to adhere to. Subsequently, the piloted training programs were not delivered at the time of project closing. Still, the ICR mission ascertained that there is a clear intent at the level of the three VTIs where the models are being piloted to push forward with implementation and to potentially replicate to other specializations. Similarly, the VTC management has expressed its commitment to adopt the models fully, once the revised by-laws are approved. 3.3 Efficiency Rating: Modest 60. As project efficiency is addressed, this needs to be situated in an overall country and portfolio context. As detailed above, the Project was implemented in an ever changing policy environment and critical fiscal deficit, which was most notably reflected in MOF financial cap on government budget, including the related Bank loan. The high turn-over of ministers with diversity of background has not only had an impact of EDSDP, but other Bank-supported projects as well. For it is not conducive to repeatedly invest in presenting the new minister’s with project issues at such high frequency. During the recent Country Portfolio Performance Review, a drop in the overall portfolio was noted and this is essentially due to exogenous pressures from neighboring countries; including the Syrian refugees crises. 61. The EDSDP project approach involved an effort to improve the quality and labor-market relevance of TVET programs by providing the sector with enabling conditions and by involving employers in the operational mechanisms as well as the design and delivery of the long- established TVET programs delivered through Government agencies – notably, the Vocational Training Corporation. 34 As such, it should be clear that this is not a “Training Delivery Project” but one that aimed at setting up a new governance framework and a new set of strategic incentives to change the way TVET programs are conceived, designed, set up, delivered and evaluated. 62. The cost-benefit analysis provided in the PAD was admittedly based on incomplete information, and “mostly serves to illustrate the relation between number of graduates, costs, wages, and economic rates of return.” 35 In the absence of evidence on the potential impacts of the project on earnings, employment rates, and costs of revised TVET programs, it assumed values for these key parameters. Yet, it goes without saying that even a fully successful four-year VET or skills development project would not have enough time to produce a tangible impact on employment and unemployment rates, let alone earnings. This ICR does therefore not try to update the PAD cost-benefit analysis because any quantitative evaluation of benefits on the basis of currently available information would be only speculative. Meaningful quantification of the benefits of the project-supported interventions will require that the improved business, 34 The Vocational Training Corporation is a semi-autonomous agency. But the Minister of Labor chairs its Board of Directors and thus exerts considerable influence on its operational decisions. 35 Page 86, Annex 9, PAD. 17 governance, and training models are in operation for a sufficiently long period of time and allow observation of actual earnings and employment outcomes for completers of restructured training programs, as well as costs of those programs. 63. All that said, this ICR attempts to assess other aspects of efficiency. All technical assistance related to this project involved competitive bidding with a balanced mix of cost and quality. The DCU, together with the implementing entities and with constant support from the Bank Team reviewed all deliverables to ensure acceptable quality and meaningful value added to the whole sector. A participatory approach was adopted throughout the set of institutional- oriented activities to ensure that the resulting deliverables/tools will be indeed adopted by other entities and not limited to MOL. 64. Each of the deliverable produced under Component 1 would contribute to improved responsiveness to the job market needs, graduates employability, and workers potential productivity, including: (i) Roles and responsibilities of the E-TVET Council and Secretariat; (ii) E-TVET Sector Action Plan, to be developed and updated by the Secretariat; (iii) adopting a participatory approach – engaging employers – while developing the procedural manual to create employer-driven occupational profiles and initial set of six profiles for demonstration purpose; (iv) the know-how to identify priority sectors and the identification of three economic priority sectors closely correlated with employment opportunities; (v) a TVET performance assessment system, including sector performance indicators, tools for data collection and data analysis, and data reporting process (on-going); and (vi) a communication strategy driven by market surveys to better understand attitudes and perceptions of the potential TVET sector beneficiaries. 65. For example, the communication strategy that was approved by the E-TVET Council now served MOE awareness activities as well. Similarly, the tools developed to support the development of occupation profiled manual are now being used by CAQA and thus allowing the newly-established CAQA a quick-start with a modality already tested and found acceptable to relevant stakeholders. 66. Another key dimension is related to the restructuring of VTC, which aimed to ensure market-relevant training courses. In doing so, there were also savings from the discontinuation of supply-driven courses which would not be easy to quantify; still efficiency gains out of the restructuring of the VTC could clearly be claimed. 67. On cost-effectiveness, in the context detailed above and in the absence of specific numerical data information, an estimate can still be reached to offer the order of potential benefits that this Project would trigger. If the tools for a more e employer-driven and efficient E-TVET system, and the models for a demand-driven training provision by VTC that were developed under the project are eventually put into operation systematically, the project could prove to have contributed to improving the quality, relevance, and efficiency of VTC training programs currently costing over US$25 million per year, as well as offering a replicable model for improving TVET programs under the MOE and MOHE. This outcome remains possible given the VTC’s plans to proceed with training delivery for the piloted programs, as well as plans to replicate to other programs/ other VTIs – could yield benefits in excess of the US$4.3 million 36. 68. Yet, this ICR can draw on the successful experience of the USAID-supported initiative “SIYAHA” where VTC benefitted from improved infrastructure, demand-driven curricula, well- 37 Implementation Completion and Results Report Guidelines, Attachment to Appendix B, World Bank OPCS, August, 2006 (10/05/2011 Update) 18 targeted awareness activities; strong employers engagement, including internships; as well as relatively functional M&E system. This has been, and continues to be a show-case for the benefits that can arise from offering a truly demand-driven training course; in terms of increased enrollment, higher completion rates and highly satisfactory employment records in the field of hospitality and tourism. 3.4 Justification of Overall Outcome Rating Rating: Moderately Unsatisfactory 69. Under World Bank guidelines, 37 ICR final outcome ratings of projects with formally revised project objectives are to average the interim outcome ratings before and after restructuring, weighted by the respective shares of project disbursements before and after restructuring. Weighting of performance before and after restructuring is not an issue for the EDSDP because although progress against expected outputs improved after restructuring, progress against expected outcomes did not improve. 70. Based on the above assessment of highly relevant objectives, modest relevance of design and implementation, along with modest achievements of both PDOs and modest efficiency, the overall rating is Moderately Unsatisfactory (both before and after restructuring). Although the project delivered most of the outputs that were expected of it under the restructuring, the expected development outcome has not been achieved because: (i) the E-TVET Council has not been effective in fulfilling its mandate of providing employer-driven policy directives to its stakeholders, and (ii) the new VTC business, governance, and training models have not yet been fully delivered 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 71. As observed in the PAD, there are important gender issues in employment in Jordan: Labor-force participation rates are far lower for women than for men; unemployment rates are higher; and earnings for equivalent levels of education are lower.38 The project made a start in addressing these gender issues through the public awareness campaign under component 1, which provided information about training opportunities for women in specializations with good earnings and employment prospects. (b) Institutional Change/Strengthening 72. The focus of the entire project was on institutional change. Despite the delays in piloting the new business, governance, and training models for the VTC, the project did make progress in strengthening institutional capacity for the E-TVET Council Secretariat and CAQA (Section 2.5), as well as VTC. Conducive institutional tools are available to be adopted and used once there is a political commitment towards an effective TVET sector. (c) Other Unintended Outcomes and Impacts (positive or negative) Not Applicable. 37 Implementation Completion and Results Report Guidelines, Attachment to Appendix B, World Bank OPCS, August, 2006 (10/05/2011 Update) 38 PAD, page 87. 19 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops Not Applicable. 4. Assessment of Risk to Development Outcome Rating: High 73. As indicated earlier, the Project has produced a number of relevant deliverables towards the sector’s institutional infrastructure. And a number of these deliverables have already been adopted at the technical and operational level. In addition, the 3 pilot vocational training institutes seem intent on fully adopting the new models in training delivery and in potentially using the models in additional specializations. Yet, a the policy level, a sustained ownership and commitment is yet to be demonstrated for the E-TVET Council to fulfill its mandate and to fully adopt the project deliverables. In addition, the Cabinet has not yet approved the VTC By-laws that are necessary to enable employer participation and delivery of VTC training programs in an institutional and consistent manner. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory 74. Project Preparation was launched within a favorable context characterized by a clear vision and high political commitment on the part of the GOJ to adopt an institutional approach towards skills development. Although insufficient political commitment and weak leadership proved to be a serious obstacle to attainment of EDSDP’s development objective, this risk was anticipated and reflected in project design. It was decided to proceed with the project despite this known risk that was clearly acknowledged in the PAD. The ICR finds that this approach was appropriate since the prospective benefit of successful implementation was large enough to justify the recognized risk that lack of political commitment that could ultimately frustrate attainment of the project’s objective. A 2010 portfolio review of Bank-supported education projects by the Independent Evaluation Group (IEG) found that the most prevalent factor contributing to unsatisfactory project performance was “overambition in relation to strength of political commitment.” 39 Nevertheless, as discussed above, the down-stream route that the Project took could not have been anticipated during project preparation. Accordingly, Bank performance in project design is rated as “moderately satisfactory”. 75. Extensive efforts were invested during project design to review the country and sector context (including PHRD-supported preparing documents that fed into each component’s design). Moreover, the Bank Team did adequately secure an effective arrangement to support the E-TVET Fund and maintained a symbolic amount of US$60,000 for component 3. This certainly was not sufficient to safe-guard a rather promising TVET funding mechanism that was envisaged to be the “leverage” arm for the E-TVET Council over TVET providers, within MOL, MOE, MOHE, and others. In any case, the EDSDP implementation experience suggests that there were two shortcomings in project design: (i) the objective of establishing an effective sector council that can issue policy directives within the current non-obliging legal framework of the E-TVET Council law was not realistic enough; and (ii) the minimal level of involvement of the E-TVET 39 Page 40, Independent Evaluation Group, World Bank Support to Education Since 2001: A Portfolio Note, 2010. 20 Fund under the EDSDP did not allow for a substantive enough engagement that could anticipate and deal with the decline in government support and the reduction in – and eventual cancellation of – CIDA’s support and financial commitment. (b) Quality of Supervision Rating: Moderately Satisfactory 76. Quality of supervision is rated as “moderately satisfactory” as the project benefited from intensive technical and operational support through frequent missions as well as informal channels (including skype and audio, as well as extra meetings while team members were in Amman to supervise other projects) throughout project implementation, all of which provided consistent handholding support. Among other things, this is reflected in the continuity of key project staff throughout project preparation and implementation. Even if there were a total of three task team leaders, continuity of the core task team and sector manager throughout preparation and implementation helped to ensure consistent high supervision standards and strong institutional memory to carry the initial project vision through to the close of the project. Sector management, together with extensive and continued support from the Country Management Unit (CMU), provided consistent support for the team’s work. 77. Supervision missions were frequent and timely. Performance ratings in the fourteen ISRs were well justified and well documented. The mission Aides Mémoire and post-mission letters to Government clearly identified implementation problems and recommended appropriate corrective actions by implementing agencies and, especially, by MOL management and Government. In most cases, these involved more proactive management by Government in carrying out agreed actions under the project, including (i) adoption of more strategic and sector-wide oversight by the E-TVET Council; (ii) an expedited approach towards rebidding of the restructuring of the VTC and project restructuring; (iii) a more firm commitment towards the E-TVET Fund; and (iv) adoption of the new by-laws for VTC restructuring. Supervision missions also offered technical assistance in carrying out the recommended actions. The frequent changes in Ministers of Labor, who played a key role as Chairman of the E-TVET Council and Chairman of the Board for the E- TVET Fund and the VTC, help explain why the recommended actions to improve implementation performance were often not acted upon. This risk was reflected in the project design and appropriate action was taken to minimize it by working to build enduring institutional capacity particularly on the part of the E-TVET Council, its Secretariat, and the VTC. 78. The Mid-Term review of the project was carried out in May 2011 as originally scheduled, with a preparatory mission in March 2011. The February 2010 supervision mission had proposed advancing the mid-term review to July 2010 in order to resolve the already-apparent gap that had arisen between planned and actual project performance. The proposal to advance the date of the mid-tem review was not subsequently pursued, in recognition that more time would be needed to prepare for the restructuring. Project restructuring was initially agreed and designed right after the Mid-term review in May 2011. A specific restructuring mission took place in September 2011 to finalize all the technical details of the restructuring. Despite MOL agreement, MOPIC was reluctant to give the green light to the restructuring and delayed the formal request despite the team´s daily follow up and the involvement of the CMU in at least two portfolio review exercises. MOPIC agreed “in principle” in February 2012, and again in May 2012. Eventually, MOPIC submitted an incomplete formal request in July 2012 that was amended by another letter in October 2012. In the interim, the Bank Team had already drafted the restructuring paper and thus was able to quickly proceed with internal clearing processes, including that of Country Director. On the Bank side, the restructuring underwent lengthy management reviews that resulted in the restructuring package not being approved by the Board until March 2013 and subsequently 21 countersigned by MOPIC with additional delays in May 2013. 79. Project restructuring in March 2013 addressed several of the exogenous developments that affected project implementation – including the dysfunctional E-TVET Council, low government commitment towards the E-TVET Fund, and the adoption of the National Employment Strategy -- as well as the shortcomings that progressively limited the scope of what could be expected under the five years and two months of project implementation. Although full attainment of the goal of genuinely employer-driven skill training will take many years to achieve, the actions supported under the project constitute an important start in this effort. 80. The Bank has also demonstrated a clear sense of proactivity throughout project implementation as shown, among others, by the following: (i) proceeding with a level I – instead of a level II which would have been easier to process – restructuring as it was acknowledged already in the mid-term review that a change in the PDO would be required; (ii) helping to keep the dialogue open with – and between – MOPIC and MOL during the long restructuring process; (iii) the Bank’s rejection of GOJ’s request for the extension of the closing date and the subsequent dialogue with the GOJ to explain and agree on the reasons for such rejection; and (iv) the Bank’s quick reaction in raising the issue of the potential risks for project implementation stemming from the MOF-imposed budget cap. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 81. Overall Bank performance is rated as “moderately satisfactory”, reflecting the “moderately satisfactory” rating for Bank performance in ensuring quality at entry and the “moderately satisfactory” rating for quality of Bank supervision. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Unsatisfactory 82. As chair of the E-TVET Council, the MOL failed to motivate genuine cooperation by the MOE and MOHE in addressing sectoral policy directives and eventually fulfilling the Council’s mandate in addressing sector fragmentation and moving towards a responsive E-TVET sector. It is not unusual for ministries of education and higher education to resist any perceived dilution of their monopoly over the content of education programs. But leadership by the Prime Minister in the spirit of the National Agenda and later the National Employment Strategy could have made a difference in motivating better cooperation by the MOE and MOHE. This was a crucial missed opportunity. As of project closing, the MOL had made only very initial progress in aligning even the training programs of the VTC, in additional to the set of tools developed through a participatory approach with other TVET providers at the technical level. In part, this modest performance reflects the frequent changes of MOL leadership and priorities. More generally, it reflects a lack of commitment by the entire Government to the objectives of the project. Despite its enthusiastic initial support for the National Employment Strategy, the Government has so far shown little evidence as it its commitment to implement the strategy, including the enactment of measures to make vocational training more responsive to skill needs of the private sector– a key element of the National Employment Strategy and of the EDSDP. 83. Borrower performance is rated as “moderately unsatisfactory” because of the lack of leadership and support for key project actions as described in Section 2.2 – in particular, for: (i) the lack of Government leadership in eliciting E-TVET Council’s engagement on broader TVET 22 sector strategy, (ii) the lack of Government leadership in focusing the E-TVET Fund on priority skill gaps of employers, (iii) reactive rather than proactive leadership by VTC management, (iv) delays by MOPIC in approval of rebidding the technical assistance for VTC restructuring and in moving ahead with project restructuring, and (v) failure to provide timely budget financing under the budget ceilings that were imposed by the MOF during the final year of project implementation. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory 84. Implementation agency performance is rated as “moderately satisfactory.” When defining the implementation Agency of the Project, this should include the DCU, the E-TVET Council Secretariat, as well as the VTC. In doing so, it should be clear that being engaged in a World Bank operation was a novelty that is demanding, complicated and at times overwhelming. This was reflected in the way technical assistance activities were handled starting from drafting a TOR, shortlisting, awarding, and contract management as detailed in section 2.2 above. In that context, it should be noted that all parties (with varying degrees of capacity) have indeed climbed a learning curve that would now allow them to potentially benefit from the project deliverables and to make better use of forthcoming interventions from the Bank as well as other donors/agencies. The DCU discharged its role responsibly within the constraints of inadequate staffing and frequent staff turnover during early implementation. The transition from first director to the subsequent directors was smooth, first to the former DCU procurement officer then to the former DCU financial management officer. The DCU monitoring and evaluation officer also effectively managed the implementation of component 1, including technical assistance to support the development of the decision support system and sector performance assessment, as well as overseeing implementation of the VTC management information system. Similarly responsible work was carried out by the DCU officer responsible for coordination of component 2 and by the DCU communication officer. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Unsatisfactory 85. Overall Borrower performance is rated as “moderately unsatisfactory,” reflecting the “moderately unsatisfactory” rating for Government performance and the “moderately satisfactory” rating for implementing agency performance. The former is rated most heavily due to its importance in impeding achievement of the project’s development objective. 6. Lessons Learned 86. Lesson 1 – In highly fragmented sectors, the need for a Gradual Process of Change should be recognized. EDSDP project design was prudently selective and actions appeared feasible with the supported interventions. But the implementation experience revealed that it was overly ambitious. The main lesson learned from this is that the paradigm shift from supply- driven to demand-driven E-TVET program design and delivery involves a longer-term process – longer than that foreseen by the project. This entails: (i) a gradual buy-in on the part of employers on the demand side, providers on the supply side, as well as the different government entities; (ii) a willingness and capacity on the part of employers and training providers to assume a lead role in revisiting the content of training programs to meet the actual skill needs of employers; and (iii) building this willingness and capacity, as is the development of awareness on the part of parents, trainees, and the community that this change provides better preparation for productive and remunerative employment than the former, familiar programs from the past. A closely related 23 lesson learnt is the need to adopt a more urgent approach towards concluding a project restructuring process, which is a shared responsibility between MOL and MOPIC, as well as the World Bank. EDSDP experience suggests that once the need for project restructuring is identified, time was of great essence to finalize the scope of such project restructuring, and conclude the process through the signing and countersigning of the Legal Amendment. 40 Similarly, once a restructuring process is launched, the World Bank is mandated to effectively respond in a timely manner to maximize the chances that impact will be realized over the remaining implementation time. 87. Lesson 2 – Successful Reform Requires Ongoing Support by Key Advocates. Project design and supervision attempted to minimize the political risk that was identified at appraisal by situating the project firmly within the framework of the Government-endorsed National Agenda – and later the National Employment Strategy -- and by developing institutional capacity and broader ownership among stakeholders. At appraisal, the Government’s high-profile endorsement of the National Agenda appeared to offer adequate assurances that the Government would provide the leadership to achieve the objectives of the project. The Government’s subsequent endorsement of the National Employment Strategy appeared to confirm that commitment. But despite its endorsement of these strategies and of the project, the Government did not provide the leadership necessary in a number of areas (Section 5.2a) to achieve the project’s development objective. Conceivably, the Bank might have been able to motivate Government action through higher-level advocacy in the context of its broader country assistance strategy. But apart from that possibility, the project experience illustrates the irreducible risk -- in this project and all projects that involve policy reform -- to development outcomes arising from changing policy priorities under successive governments, and or inability of governments to provide the leadership to prevail over the vested interests that oppose reform. To the appraisal team’s credit, the PAD fully acknowledged that risk. 88. Lesson 3 - Effective participation of all parties is critical in addressing Technical Assistance design and implementation Risks. As described in the various mission Aide- Mémoires over the past five years, many of the delays in project implementation resulted from problems with technical assistance (TA), which contributed to the incomplete achievement of project targets. Symptoms of the problem include rejected inception reports and the need to rebid one major TA contract. Problems with technical assistance occur in many projects, but it is rare for TA problems to affect project activities as consistently as they did for the EDSDP. There are important lessons to be drawn where the project encountered three kinds of problems with technical assistance: (i) unrealistic or unclear terms of reference; (ii) lack of optimal representation or appropriate skills on the part of technical assistance TA evaluation committees, or to subordination of quality factors to cost factors in evaluation (including English language, evaluation of technical proposals vis-à-vis TORs); and (iii) insufficient supervision/ contract management of consultants. As the ICR describes more fully, EDSDP implementation experience suggests suitable safeguards to prevent a recurrence of these TA implementation problems in future projects: ensure the effective participation of all relevant stakeholders in all stages starting from drafting of the terms of reference (TOR) up to implementation of the activity. EDSDP experience does suggest that better results were achieved in cases where the implementing entity 40 In case of EDSDP, this process took from September 2011 until the amendment was countersigned by MOPIC in May 2013; where it was approved by the World Bank Board in March 2013. 24 and the DCU were more actively involved in the drafting of the TOR, included in the composition of the technical evaluation committee, and took part in the effective contract management of the activity. A key lesson is specifically drawn from the rebidding of the VTC related contract, which is a shared responsibility between MOL and MOPIC, which led to extensive delays of over a year in finalizing the rebidding package. EDSDP experience suggests that once a rebidding decision is made, the rebidding process should be launched to avoid double adverse impact that resulted from the termination of the original contract and the delays in rebidding. The second contract was only signed in May 2011 while the original contract was terminated in February 2010. As a related note, while the Bank does clear TORs, the responsibility of ensuring they are adequate for the implementation context lies with the implementing entity; including the responsibility to amend the contract and revisit the respective tasks and deliverable. 89. Lesson 4 - Meaningful steps can be taken, even in the Absence of a Supportive Legal Framework. Management attention during early implementation of Component 2 for the “Restructuring of the VTC” focused on putting in place an improved legal framework (revised VTC by-laws) for implementation of the new business, governance, and training model, which the project developed for the Vocational Training Corporation. Although prospects are good for eventual approval of the new By-Laws for the three models, implementation experience has shown that it is sometimes possible to launch implementation even in the absence of a supportive legal framework. An important element of the new VTC business plan is the signing of production contracts with firms for the delivery of training in the context of specific production agreements. Cabinet approval of the new By-Laws is required before VTIs are authorized to sign production contracts. But even in the absence of the new By-Laws, the three pilot VTIs initially managed to assemble the necessary equipment, materials and instructors to launch the piloting of new training modules; with required delegations provided by the VTC Director General. The Mafraq VTI is in the process of signing a Memorandum of Understanding with a local firm that was involved in development of the program in order to start training program delivery for air- conditioning. 90. Lesson 5 – Staff Continuity – and more importantly sustained commitment – Promote (but does not Ensure) Success. EDSDP supervision benefited from an unusual degree of staff continuity, with the same core team (albeit three Task Team Leaders), sector manager, and sustained support from the Country Management Unit from project identification through project completion. Staff continuity provided a strong consistency of vision and supervision standards throughout implementation, and contributed to the “moderately satisfactory” rating for Bank performance. Yet, as it is clearly evident in this project, high quality supervision cannot compensate for lack of Government commitment and weak ownership. . 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies Not applicable. (b) Cofinanciers Not applicable. (c) Other partners and stakeholders Not applicable. 25 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) 26 (b) Financing Project Financing Percentage of Actual/Latest Appraisal Estimate Percentage of under Restructured Restructured Source of Funds Estimate (USD millions) Appraisal Project Project (USD millions) (USD millions) Borrower 1.33 0.96 72.1% 1.3 72.1% International Bank for Reconstruction and 53.6% 7.50 3.37 44.9% 6.29 Development 27 Annex 2. Outputs by Component Revised Overall Project Outcome Indicators Performance Indicators Baseline Target Values Year 5 Results Achieved (Cumulative year 5 ) 6 OPs were developed and endorsed for: • Metal Fabrication “Welding and Sheet Metal” Number of employer- • Metal Fabrication “Metal driven occupational Machining” profiles (OPs) developed 0 6 • Automobile Repair and endorsed by the E- • Retail and Sales TVET Council • Electricity “House Electricity” • Construction and Press Packaging and Printing - Organizational Structure accepted in April 2013 - Organizational audit Organizational audit, - Organizational Audit accepted in May 2013 Accepted staffing plan and - Staffing Plan accepted in June 2013 None - Staffing Plan Accepted organizational structure - VTC revised by-laws (Finance, Supplies, HR, - Organizational Structure introduced in VTC Administrative) accepted by the VTC in June Endorsed 2013 and sent to Cabinet for endorsement in July 2013 Number of employer- - 3 training programs (Carpentry, Air- driven skills development 0 3 Conditioning, and Metal Fabrication) designed programs designed and in March 2013 implemented 28 Revised Intermediate Outcome Indicators, Component 1 Performance Indicators Baseline Target Values Year 5 Results Achieved (Cumulative year 5) Development and - E-TVET Sector Annual - PAS revised and approved in February 2013 introduction of a TVET Assessment Report - Data collection tools designed and approved in Sector Performance None generated March 2013 Assessment System (PAS) - E-TVET Sector Annual Data collected through March – July 2013 Assessment - Second assessment report received in September Report discussed & 2013 endorsed by the E-TVET Council - TVET providers are generating PAS-based reports Increasing public awareness - Terms of Reference drafted in March 2013. for E-TVET sector 3.4 3.4 – 3.6 - The survey will be conducted following implementation of the awareness campaign to measure its effectiveness Revised Intermediate Outcome Indicators, Component 2 Performance Indicators Baseline Target Values Year 5 Results Achieved (Cumulative year 5 ) Piloting of employer-driven - New Business Model - Organizational Structure accepted in April Business, Training and Piloted 2013 Governance Models in three None - New Training Model - Organizational Audit accepted in May 2013 VTC centers Piloted - Staffing Plan accepted in June 2013 - New Governance Model - Trainers’ program conducted in June 2013 Piloted - VTC revised by-laws (Finance, Supplies, - New Organizational HR, Administrative) accepted by the VTC in Structure Piloted June 2013 and sent to Cabinet for endorsement in July 2013 - Training Plan accepted in September 2013 - Organizational Manuals accepted in September 2013 - Training Completion Report and project Completion Report accepted in September 2013 Increased enrolment rate at § Regular - 2% increase in regular - 12% increase from baseline in 2012 for VTC courses: 7603 courses regular courses § Short-term - 5% in short-term courses - 11% decrease from baseline in 2012 for courses: 5194 short-term courses Increased female enrolment § Regular - 2% increase in regular - 36% increase from baseline in 2012 for rate at VTC courses: 1938 courses regular courses § Baseline - 4 % increase in short-term - 42% decrease from baseline in 2012 for value for courses short-term courses short-term courses: 2031 Completion Rate of VTC 65% 70% Completion rate for 2012 is 73% graduates 29 Annex 3. Economic and Financial Analysis (including assumptions in the analysis) Not Applicable (see Section 3.3) 30 Annex 4. Bank Lending and Implementation Support/Supervision Processes Task Team members Responsibility/ Names Title Unit Specialty Lending Afifa Alia Achsien Senior Program Assistant MNSHD Administrative Arup Banerji Sector Director HDNSP Financial Robert Bou Jaoude Program Coordinator SACPK Management Hyacinth D. Brown Division Manager CTRLA Disbursement Financial Diana C. El Masri Consultant MNAFM Management Linda English Lead HD Specialist AFTHD Mourad Ezzine Education Sector Manager MNCMI Lina Fares Senior Procurement Specialist MNAPC Procurement David Freese Disbursement Technical and Amira Kazem Senior Operations Officer MNSHE Operational Support Kirsten Majgaard Education Economist AFTEW Economic Analysis Kenneth Mwenda Program Manager HRDVA Legal Soren Nellemann Country Sector Coordinator ECSH2 TTL Harry Patrinos Sector Manager HDNED J. Roger Pearson Consultant MNSHE Technical Expert Susan R. Razzaz Senior Country Economist SASEP Kanta K. Rigaud Lead Environment Specialist CPF Safeguards Alberto Rodriguez Sector Manager ECSH2 Haneen Ismail Sayed Lead Operations Officer MNSSP Colin S. Scott Lead Specialist OPSOR Operational Eileen Brainne Sullivan Health Specialist EASHH Support/costing Alexandria Valerio Senior Economist HDNED Arvil Van Adams Consultant Peer Review Sophie Warlop Operations Officer HDNSP Ai Chin Wee Consultant CSABI Supervision/ICR Afifa Alia Achsien Senior Program Assistant MNSHD Administrative Financial Mona El-Chami Sr Financial Management Specialist MNAFM Management Aissatou Dicko Senior Executive Assistant MNSHD Administrative Emma Paulette Etori Program Assistant MNSHE Administrative Lina Fares Senior Procurement Specialist MNAPC Procurement Technical and Amira Kazem Senior Operations Officer MNSHE Operational Support 31 Financial Jad Raji Mazahreh Sr Financial Management Specialist MNAFM Management Juan Manuel Moreno Lead Education Specialist, TTL MNSHE TTL Soren Nelleman Country Sector Coordinator ECSH2 TTL J. Roger Pearson Consultant MNSHE Technical Expert Haneen Ismail Sayed Lead Operations Officer MNSSP Moukim Temourov Senior HD Economist MNSHD TTL (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY07 16.83 140 747.96 FY08 22.27 177 298.90 Total: 39.10 318 046.86 Supervision/ICR FY09 12.89 96 325.64 FY10 23.45 94 885.90 FY11 19.17 71 453.63 FY12 20.25 98 230.63 FY13 17.60 95 241.00 FY14 5.41 29 688.13 Total: 98.77 485 824.93 32 Annex 5. Beneficiary Survey Results (if any) Not Applicable 33 Annex 6. Stakeholder Workshop Report and Results (if any) Not Applicable 34 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR THE HASHEMITE KINGDOM OF JORDAN T H E M I N I S T R Y OF L A B O U R DEVELOPMENT COORDINATION UNIT (DCU) EMPLOYER DRIVEN SKILLS DEVELOPMENT PROJECT (EDSDP) IMPLEMENTATION COMPLETION REPORT JANUARY 2014 Prepared By: DCU – EDSDP/ MoL 35 SECTION 1: PROJECT BACKGROUND, DEVELOPMENT OBJECTIVES AND DESIGN Country Background In 2008, when the design for the Employer Driven Skills Development Project was initiated it was based on assessment of country, macroeconomic and sector background. At time, The Hashemite Kingdom of Jordan has averaged 6% economic growth per year over the past five years. Jordan had a 40% labor market participation rate, one of the lowest in the region (67% of males and 14% of females), totaling a labor force of 1.4 million. Government employment marked 30%. Half of the remaining employment was dominated by micro enterprises (less than 4 persons) mostly in vocational/technical skills. Unemployment was at 14%, particularly among youth (21.7% among females and 11% among males). 60,000 citizens are entering the labor market every year, posing a major challenge for the government and society at large. Unemployment in Jordan is affected with three main factors: (i) geographic mismatch with most of the unemployed in the rural areas and most of the new jobs being created in Amman; (ii) skills mismatch arising from the inadequate and poor quality provision of public training (e.g. Vocational Training Cooperation VTC) and limited private training; and (iii) market failure and the lack of capacity to match job- seekers with job opportunities, fueled by the reluctance of applicants to work in less than satisfactory working conditions (including low salaries). TVET Sector at time of project design faced the following issues: Socio-economic Fragility and Social Cohesion: Given the regional tensions, the demographic composition and youth unemployment, sector policies are conducted on ad hoc basis with a short-term focus making any consistent reform involving institution building and employer participation inherently difficult. Weak Technical and Vocational Education and Training TVET System and Limited Use of Market Driven Mechanisms: The internal and external efficiency of the current TVET system is very weak. Weaknesses identified in the current TVET system include: (i) a governance model that lacks clarity in terms of mandates, accountabilities, prospects for conflict of interest and not inclusive of social partners’ views, especially employers; (ii) the consequent absence of mechanisms for employers to assume a leading role in the articulation of required occupational profiles and related competencies, or to participate in a meaningful way in the development of training programs and the measurement of training outcomes; (iii) inadequate facilitation of school-to- work transition, including career guidance and job placement support; (iv) a need to improve labor market flexibility; (v) lack of an accreditation and certification body; (vi) lack of qualified training providers; and (vii) lack of awareness of the concept of life-long learning within a comprehensive package to upgrade firms’ productivity and alleviate unemployment. Supply-Driven and Poor Quality of Training Delivery: There are some obvious problems with the TVET system, a certain level of reading and writing ability and technical skills is mandatory for a trainee’s success in the labour market. The employer’s ability to “customize” the employee’s technical skills to fit that specific work environment is greatly affected by the employee’s level of literacy. Absence of Employers’ Participation in Workforce Planning and Development: The absence of employers in the participation and decision making of most aspects of workforce development is the current dominating characteristic of the system. 36 Diverted Training Fund: Another concern is the E-TVET Fund. It was stipulated to work with the Employment-Technical and Vocational Education and Training E-TVET Council to develop policies, regulations, budgetary and subsidization decisions, there has been little evidence of interaction or support from the Council. Government TVET Strategy In 2006, the Government of Jordan embarked on a reform of its E-TVET sector in response to the National Agenda which identified E-TVET as one of the country’s priorities. The Agenda proposed restructuring of the institutional framework of the E-TVET system including the establishment of the Higher Council for Human Resource Development HCHRD to ensure efficient and effective collaboration among the main three public TVET providers (Ministry of Education MoE, Ministry of Higher Education MoHE and Ministry of Labour MoL/the VTC). An E-TVET Council was established to guide a holistic approach to the development of the TVET sector with policy directives being provided primarily by the employer community. In early 2007, nine priority areas were identified and agreed among stakeholders. They are: 1. Establish policy making bodies: Strengthen E-TVET Councils; 2. Stakeholder participation in E-TVET strategy: enhance the participation of relevant stakeholders in the planning of employment aspects of the E-TVET strategy; 3. Capacity building and institutional development: enhance capacity of MoL to promote employment; 4. Strengthen public-private partnerships: develop cooperation between MoL and private sector to meet the demand of employment, especially for small and medium enterprises (SME); 5. Improve capacity of training providers: develop the capacity of all public and non- governmental E-TVET providers in the field of planning, policy design and decision- making; 6. Enhance information systems: develop and enhance the Human Resources Information system and build capacity for its use for policy making; 7. Review financing of public and private training provision, including E-TVET Fund; 8. Initiate the development of the design of a Jordan National Qualification Framework; and 9. Promotion/awareness of TVET Sector. Rationale For World Bank Assistance The Bank has extensive experience and a comparative advantage in supporting the Government of Jordan in transforming its TVET structure into a more demand-driven and functional system, drawing upon the experiences of relevant operations in other countries worldwide. Their assistance aimed to contribute to developing the institutional capacity through: (i) operationalizing the National Agenda including the empowerment of the E-TVET Council with the close involvement of the various stakeholders; (ii) transforming supply-driven systems of training into demand-driven systems; and (iii) creating incentives for the employers to benefit from market-relevant and higher quality services. The Employer Driven Skills Development Project EDSDP aimed to contribute to the policy dialogue to improve the E-TVET sector. The project was designed to run from 2008 – 2013 under the leadership of the MoL as an implementing agency and managed by the Development Coordination Unit DCU within the Ministry. EDSDP Original Results Monitoring Framework Original Project Development Objective PDO 37 PDO/ sub-components Outcome Indicators PDO is:  Functional E-TVET system (functional E-TVET Council; functional Secretariat). Realign policy formulation with E-TVET operational  Autonomous VTC (new business and training mechanisms through the development of employers’ delivery model, increased percentage of linking participation in (i) sector policy formulation, (ii) financial resources with training centers’ institutional development and reform, and (iii) skill fulfillment of performance targets). development program design and delivery.  Improved VTC’s internal efficiency through increased trainee/instructor ratio (6.4 in 2006 to 10 in 2013).  Increased number of firms receiving training through E-TVET Fund. Beneficiaries E-TVET Council and Secretariat, Vocational Training Corporation, E-TVET Fund, TVET Sector Stakeholders, Private Sector involved in TVET,TVET Trainees. EDSDP Original Components Component 1: E-TVET System and Council Development with Employer Participation (total cost US$2.49 million). 1.1 E-TVET Council Orientation and Capacity Development. The objective of this sub-component was: (i) to build a common understanding of employment driven TVET systems, structures, and operational practices amongst the E-TVET Council members and TVET Stakeholders, and (ii) build the capacity of the Secretariat to provide policy options analysis in the TVET sector, and establish an agenda and action plan for the work of the Council, based on an analysis of its mandate and the surrounding regulatory framework. 1.2 E-TVET Sector Policy, Planning and Organizational Development with Active Employers Participation. The objective of this sub-component was to support the E-TVET Council and the Secretariat to: (i) formulate national policy directions for the TVET sector, (ii) establish and harmonize explicit policy objectives and accountabilities for the respective public and private TVET providers, (iii) initiate the establishment of common performance based planning and management procedures amongst public sector TVET providers within MTEF framework, (iv) recommend resource allocations for public TVET providers, and (v) progressively build the national qualification, certification and accreditation bodies and procedures. 1.3 E-TVET System Performance Assessment with Involvement of Training Providers and Employers. The objective of this sub-component was to build the capacity of the Secretariat as the Council’s technical arm to: (i) establish performance indicators for the E-TVET system derived from national policy objectives, (ii) design data gathering tools and reporting mechanisms, (iii) undertake performance analysis and recommend remedial action, and (iv) ensure feedback into the policy assessment cycle. 1.4 E-TVET Sector Promotion and Awareness among TVET Stakeholders. The objective of this sub- component was to engage employers and TVET Stakeholders in enhancing and promoting understanding and awareness of occupational opportunities in the TVET sector. 38 Component 2: Vocational Training Cooperation (VTC) Restructuring (total cost US$7.48 million). Component 2.1 Establishment of the Mandate, Governance Structure, and Regulatory Framework for the new Skills Development Agency. The objective of this sub-component was to support the first phase of the reform program leading to the establishment and the implementation of the legal framework and governance structure of the new organization. Component 2.2: Organizational Restructuring and Staff Development. The objective of this sub- component is to support the development of an organizational structure, operational capability, and business plan consistent with the assigned purpose of the new organization. Component 2.3: Reorientation of the Training Delivery Model to Reflect Employer Defined Occupational Competencies. The objective of this sub-component was to support the development and implementation of an employer driven business and training service delivery model consistent with (i) employer community expectations, (ii) the mandate assigned by the E- TVET Council, and (iii) the National Qualifications Framework currently under development. Component 3: Strengthening the E-TVET Fund Development (total cost US$ 0.06 million). The objective of this component was to support the institutional capacity of the E-TVET Fund in the fulfillment of its mandate to identify, finance and facilitate the delivery of skill development initiatives that will enhance the operational efficiency of the productive sector. The E-TVET Council will provide policy guidance for implementation of the component. Component 3.1: E-TVET Fund Development: Institutional Development of the E-TVET Fund under the E-TVET system reform. The objective of this component is to enhance the institutional capacity of the Fund in fulfillment of its mandate to identify, finance, and facilitate the delivery of skill development initiatives that will enhance the operational efficiency of the productive sector. This component was supported by Canadian International Development Agency CIDA through parallel financing with an amount of US$1.9 million. The following activities were funded: Awareness and Outreach; Institutional Development of E-TVET Fund; MIS Development; Monitoring& Evaluation System. Component 4: Implementation Support (total cost US$1.06 million). The objective of this component was to support project implementation arrangements. This component funded key staff hired for the implementation of the proposed project. This will, inter alia, include an Executive Director, Procurement Officer, Financial Management Officer, Monitoring &Evaluation M&E Officer and VTC Component Coordinator. It also funded activities related to the overall project implementation including minor goods, materials and operational expenses and services. 39 SECTION 2: KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES Assessment of Project Original Design According to the Project Appraisal Document PAD, the appraisal of the EDSDP was done as a team work of members from the World Bank, MoPIC, National Center for Human Resource Development NCHRD, VTC and E-TVET Fund; which mean that all implementing entities were involved in the design of the project. At time of project appraisal and negotiation during 2007-2008, project components, sub-components, activities, expected results and time-line had seemed realistic with highly optimistic assumptions. Through the life of the project, several challenges were faced which proved that the project assumptions where highly optimistic: lengthy government procurement procedure required to initiate project activities; the need to go to several layers of approval to endorse deliverables; obstacles faced to engage more the private sector in the development process; and the fact that development projects in the public sector usually require a long period of time to show results (that usually show impact after the project life). Having said that, raises a question on how adequately all these assumptions were considered when the time- line for project activities, goals and objectives, risk analysis and expected results were planned. EDSDP Restructuring As a result of the Mid-Term Review (MTR) conducted in December 2010, the need to restructure the EDSDP was raised early 2011. The MTR aimed at: (i) assessing the progress of the project during the first half of operation, (ii) validating relevance of project objective, (iii) addressing critical issues and agree on remedial actions if necessary, including possibly restructuring project components/ activities and (iv) planning the second half of the project life. The MTR indicated that the project development objective was still relevant, nevertheless; the original design of project activities was not achievable as well as several of the expected intermediate objectives and outcomes by component. This raised the necessity for project restructuring to use the remaining life of the project to focus on implementing realistic activities that were expected to provide fruitful results, support the achievement of project objectives and reduce the risks the project could face by removing activities that are neither relevant nor realistic. In September 2011; restructuring scope was discussed with all related parties including: implementing entities, World Bank, sector stakeholders and Ministry of Planning and International Cooperation MoPIC and agreed upon. The restructuring aimed to draw on the following: Project support to Component 1 (E-TVET System and Council Development with Employer Participation) to continue and activities under this component to be revised. Support to Component 2 (Restructuring of the Vocational Training Corporation (VTC) to continue as designed. Due to the symbolic contribution of the EDSDP to Component 3 (Strengthening the E-TVET Fund), the weakness of the project impact on the challenges this component is facing and other donor’s interventions in this direction, the need to drop out this component from the project design was raised. Component 4 (Project Management of the EDSDP) remained as in original design and the RMF was revised to be more coherent with project activities,more directly linked to the Project's Development Objective and reflect implementation progress as accurately and as transparently as possible. Annex 1: Restructuring Document. 40 Revising the EDSDP Results Monitoring Framework After the restructuring of the EDSDP a revised RMF was adopted effective on 27th of May 2013 when EDSDP restructuring was finalized. The project PDO was revised and new performance indicators were identified in cooperation with all related parties including the E-TVET Secretariat, the VTC, the DCU, the World Bank and MoPIC in order to provide accurate and transparent measurements of project progress. New baseline values were identified as well as target values. The revised RMF was used in reporting for the last two progress reports generated in 2013. Revised EDSDP Results Monitoring Framework Revised PDO Realign the E-TVET Sector with the National Employment Strategy by enhancing the enabling conditions for employer participation in (i) TVET institutional development, and (ii) skills development program design and delivery by the Vocational Training Corporation. Project Outcome Indicators Number of employer-driven occupational profiles developed and endorsed by the E-TVET Council Organizational audit, staffing plan and organizational structure introduced in VTC Number of employer-driven skills development programs designed and implemented Component One Indicators Development and introduction of a TVET Sector Performance Assessment System (PAS) Increasing public awareness for E-TVET sector Component Two Indicators Piloting of employer-driven Business, Training and Governance Models in three VTC centers Increased enrolment rate at VTC Increased female enrolment rate at VTC Completion Rate of VTC graduates Revised Financing Agreement Project Costs (US$ m) Components Original Revised Loan Gov. Total Loan Gov Total 1. E-TVET Council Development with Employer 1.90 0.90 Participation 1.6 0.3 0.6 0.3 2. Restructuring of Vocational Training Corporation 5.24 0.9 6.14 5.24 0.9 6.14 3. Strengthening the Training and Employment Fund 0.06 0 0.06 0 0 0 4. Project Management 0.6 0.1 0.70 0.45 0.1 0.55 Total 7.50 1.3 8.8 6.29 1.3 7.59 41 SECTION 3: ASSESSMENT OF OUTCOMES Achievement of Project/Program Development Objectives The EDSDP was one of the main vehicles of the E-TVET reform. Its goals and objectives were highly relevant for developing the sector and improving its overall performance. However; the operational and procedural delays the project faced did not allow it to fulfill all these objectives. The project design was relevant to major policy priorities under Jordan National Agenda and the National Employment Plan. It was effective in achieving enabling conditions to achieve most of the targeted outputs and outcomes. Project Relevance The Project was highly relevant as it supported the Government‘s major education and economic sector policy priorities under the National Agenda and the National Employment Strategy. It focused on the importance of TVET in improving the human resource development as an instrument of socio-economic development. The Project conformed to World Bank‘s objectives to reduce poverty and its emphasis on human resource development through skills training and employment promotion. Project Effectiveness The scope and activities of the EDSDP were largely appropriate and its interventions were timely in addressing the rising demand for skilled workforce and technicians in a rapidly changing labour market that is in need for TVET skills. The project contributed in developing a number of building blocks for the E-TVET sector, a decision support system for the E-TVET Sector which enhanced the E-TVET Council’s ability to issue policy directives and to assess the extent to which the E- TVET stakeholders' performance is aligned to such policies. The set of guidelines for the creation of Occupational Profile and the selection of 6 occupation subsectors for which occupational profiles were developed as well as the identification of 3 economic priority sectors. Moreover, the assessment of TVET providers’ capacity to respond to the needs triggered by the priority economic sectors as well as a gap analysis between demand for and supply of TVET services which help in aligning macro level employment demand projections and occupational profiles to guide training providers’ planning. The project also contributed in developing a Communication Strategy driven by the views and perspectives of TVET stakeholders. The Project developed an employer- driven training model for the Vocational Training Corporation which is characterized by responsiveness, relevance, cost-effectiveness, efficiency, accountability and sustainability. The model includes internal quality control procedures integrated with a business model for a decentralized vocational training delivery system. It was understood from the beginning that this project‘s goal would be to lay the foundations for this transformation into employer-driven training delivery, a process which will need more additional time and resources in order to be fully completed. Efficiency of the EDSDP The project’s internal operation was efficient and matching the regulations of both the World Bank and the Government. The project tried its most capacity to utilize all available resources however, the several outside challenges and delays the project faced did not allow showing the efficiency of internal operation. The project issued the requested M&E progress reports according to the approved RMF. The IFRs were issued on a quarterly basis as well as the VTC Restructuring monthly 42 progress reports. The procurement procedures were conducted according to the World Bank guidelines and the government guidelines. Regular meetings with stakeholders were conducted to discuss views on project, achievements, and challenges. All implementing entities were involved in reviewing deliverables and providing their acceptance on the final deliverables. In order to ensure that project deliverables were matching requirements DCU outreached various experiences to add value to the project's final outputs/outcomes. The auditor issued unqualified “clean” opinion on all audited financial statements. Most of the major deliverables agreed in the project (before and after restructuring) were delivered, however not all were endorsed and are currently in use. Preliminary Assessment of Sustainability EDSDP deliverables represent a major contribution to the E-TVET reform. Deliverables were accepted by all implementing entities however full endorsement and implementation of the deliverables is expected to be witnessed in the near future. This should take into consideration the availability of needed resources which should be secured by implementing entities and their commitment to implement these deliverables. Coordination with donors’ intervention is also important to build on what is available and avoid duplication. Implementing entities are encouraged to include implementing these deliverables within their action plans. Deliverables of the EDSDP Component 1: E-TVET System and Council Development with Employer Participation 1.1: E-TVET Council Orientation and Capacity Development: Deliverables are:  Stakeholders’ Roles and Responsibilities including that of the E-TVET Council Secretariat which the only officially endorsed by the E-TVET Council,  An Action Plan that is endorsed by the E-TVET Council and is being updated by the Secretariat, though it does not include either cost estimates or financial resources,  Capacity Assessment Tool for the stakeholders responsible for the implementation of the E- TVET action plan,  Secretariat’s Training Plan and Development Plan which is being partially adopted,  A number of capacity building activities targeting the E-TVET Council members and the Secretariat,  E-TVET Council Secretariat Operations Manual,  A Decision Support System built according to the Sector Performance Assessment System developed under sub-component 1.3. 1.2 E-TVET Sector Policy, Planning and Organizational Development with Active Employers Participation: Deliverables are:  Identification of three economic priorities sectors and recommendations for further development with justifications for the choices and alignment with macroeconomic setting and labor demand projections. The three sectors are: (i) Manufacturing, (ii) Whole Sale, Retail and Maintenance of Vehicles and (iii) Transportation, Storage and Communication. Sectors were endorsed by the E-TVET Council,  A set of operational guidelines for creation of Occupational Profiles and a selection of six occupational subsectors for the three selected sectors. Occupational Profiles are for: (Metal Fabrication “Welding and Sheet Metal”, Metal Fabrication “Metal Machining”, Automobile Repair, Retail and Sales, Electricity “House Electricity”, Construction and Press Packaging and Printing). Occupational Profiles were endorsed by the E-TVET Council are were disseminated to TVET training providers through the Center of Accreditation and 43 Quality Assurance CAQA,  An assessment report of the capability of TVET providers to respond to the needs of identified economic sectors.  A final evaluation report of the assignment including Action Plans for each of the three sectors for further development. The first report was endorsed by the E-TVET Council. 1.3 E-TVET System Performance Assessment with Involvement of Training Providers and Employers: Deliverables are:  Workplan template to plan for sectoral evaluation, endorsed by the E-TVET Council and Secretariat,  Performance Assessment System, endorsed by thee-TVET Council and Secretariat,  Data collection and analysis tools including: questionnaires, organizing focus groups, conducting key informant interviews, endorsed by thee-TVET Council Secretariat,  Assessment findings’ matrix, and endorsed by thee-TVET Council Secretariat,  Reporting formats where the first and second assessment reports were generated. The first report was endorsed by the E-TVET Council and disseminated to sector stakeholders. The second report is still pending finalization to be endorsed by the E-TVET Council for dissemination.  As a result of this assignment a national committee for M&E was established as a reference for future sector assessment. 1.4 E-TVET Sector Promotion and Awareness among TVET Stakeholders: Deliverables are:  Baseline surveys of Knowledge, Attitude and Perception of TVET sector (Quantitative, Qualitative and Secondary research studies)  Shared image among TVET sector stakeholders,  Communication Strategy,  Creative Development,  Implementation plan  M&E Plan  Media Plans  National campaign to introduce TVET sector which included using all media channels and social media to implement the campaign on a national scale. Component 2: Vocational Training Cooperation (VTC) Restructuring (total cost US$7.48 million). 2.1 Restructuring of the VTC: Deliverables are: New Business, Governance and Training models (Modular Competency Based Education & Training MCBET),endorsed by the VTC, Organizational Assessment Report and new Organizational Structure, endorsed by the VTC, Staffing Plan and Training Plans, endorsed by the VTC, Organizational Manuals, endorsed by the VTC, Project Completion Report, endorsed by VTC, The new models are being piloted in three selected centers: North, Middle and South. It has started in the North and the MoU was signed meanwhile the Middle and South will be started during the first quarter of 2014 pending finalizing of the infrastructure preparation. 44 Component 2.3: Improving the Infrastructure at the VTC. A local firm was contracted in March 2013 to develop a M&E System for the new structured VTC, however, the firm was not able to deliver all required deliverables as agreed in the ToR/ contract. Following the firm’s inability to provide the agreed upon software (Strat and Go) for the system that was mentioned on their offer, the counterpart team and the DCU agreed that the firm will not be able to deliver a satisfactory M&E system. Therefore, the contract is the process of termination. 45 SECTION 4: ASSESSMENT OF BANK AND BORROWER PERFORMANCE Bank Performance World Bank performance was generally satisfactory in reference to the several review missions conducted, mid-term review and restructuring of the EDSDP mission. The Bank closely supervised and monitored the project and provided guidance through implementation on technical aspects, on procurement and recruitment of consultants. During project design, the World Bank consulted with representatives from the government and local implementing entities that will be directly engaged in implementation to ensure full understanding and mutual agreement among all related parties. As per the loan agreement, the World Bank team engaged in the project conducted two supervision missions every year one in March 41 and one in September. The World Bank team met during all missions with the DCU team, implementing entities, government officials and donors who have interventions in the TVET sector. After each mission, an Aid Memoire AM was submitted from the Bank to all sector stakeholders clearly identifying mission findings. An updated action plan was also attached to each AM in order to follow up on implementation progress. The World Bank team was continuously in contact with EDSDP to follow up on progress and achievement of results. They provided consulting as possible and advised on possible remedial actions to push implementation forward in certain cases. Borrower Performance The Ministry of Labour MoL hosted the EDSDP and was one of the main leaders of the E-TVET reform. As per the loan agreement, a project management unit – the Development Coordination Unit DCU - was established at the Ministry to follow up on implementation. The unit was also expected to be responsible to coordinate donor’s interventions in the E-TVET sector. The DCU was responsible to coordinate and support the implementation of project activities and the roles and responsibilities of all partners and implementing entities to ensure effective and efficient implementation of project activities. Despite that the DCU’s role was not fully understood by MoL and sector stakeholders, it was generally accepted. The DCU was able to manage the EDSDP to the best of its capacity to ensure achievement of expected results; however, its role in terms of donors’ coordination was not fully obtained. The DCU needed more support and empowerment from the MoL and MoPIC to perform its roles and responsibilities effectively and efficiently. The DCU’s role would have been strengthened if it had enough power to force effective and efficient review/approval of deliverables to save the long time the review process used to take and the several layers of approvals the process was going through. Except for the first and last years of the project three missions were conducted. Also in 2012, the March 41 mission took place in May 2012. 46 Partner entities such as the VTC were heavily engaged in the implementation of EDSDP activities while the E-TVET Council and Secretariat were expected to show more involvement and cooperation, this was due to the lack of staff members. 47 SECTION 5: LESSONS LEARNED During the life of the project, several challenges were faced and resulted in lessons learned. Those lessons were taken into consideration during the EDSDP implementation and would be useful for future project planning as well. The following represents a selection of the most important lessons learned: A clear ToR should be developed and discussed with all related stakeholders and agreed upon: a number of meetings should be conducted with the awarded firm / consultant and the implementing entity to discuss the TOR thoroughly and to agree upon the objectives, activities and deliverables. These meetings should be documented and a succinct minutes of negotiation should be developed and signed by all partners. Any changes during implementation of any assignment should be reflected in the TOR, the contract and the project time frame. Dealing with retendering an activity: retendering an activity requires a long procedure and consumes time. The time consumed in implementing this activity should be discussed and compensated in the time frame of the project once it is raised. The retendered activity should be retained with the same time frame and content without losing its outcomes as a result of retendering. Change requires Time and Funds: restructuring institutions/corporations needs more time for implementation and adoption of the new concepts. Funds should be allocated and post action plan should be designed to ensure sustainability. M&E Reporting: The quarterly progress reports should be issued solely by the project without any intervention from the implementing entities to ensure its objectivity and credibility. Secure of Government Budget Allocation: the necessity of increasing the capital allocations for the Ministry when implementing a World Bank loan projects especially during the last years of the project. Provide training on World Bank Procurement and disbursement guidelines at the beginning of the project to the MOL financial department and deputies of the Audit Bureau and the Ministry of Finance at MOL as well as the Special Tendering Committee. 48 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Not Applicable. 49 Annex 9. List of Supporting Documents 1. European Training Foundation, Technical and Vocational Education in Jordan: Areas for Development Cooperation, Turin, 2006. 2. International Bank For Reconstruction And Development and International Finance Corporation, Country Assistance Strategy for The Hashemite Kingdom of Jordan for the Period FY 2006-FY 2010, World Bank Report No. 35665-JO, April 6, 2006. 3. Japanese PHRD Grant funded reports (2007 - preparation stage) 4. Jordan Country Assistance Strategy 5. Loan Agreement (2008) 6. Ministry of Labour, VTC Employer Satisfaction, February, 2010 and 7. Ministry of Labour, E-TVET Awareness Raising Campaign, Knowledge, Attitudes & Perceptions, Quantitative Research Study - Baseline Survey July 2010. 8. National Agenda Steering Committee, 2006-2015 National Agenda: The Jordan We Strive For, 2006.National Employment Strategy (2011). 9. Project Appraisal Document on a Proposed Loan in the Amount of US$7.5 Million to the Hashemite Kingdom of Jordan for an Employer Driven Skills Development Project, World Bank Report No. 41218-JO, May 2, 2008. 10. Project Aide Memoires 11. Project ISRs 12. Loan Amendment (2013) 13. QER - November 6, 2007 memorandum from QER panel to the task team leader, “Quality Enhancement Review Panel Report, Jordan Employer Driven Skills Development Project, P100534” 14. Restructuring Paper on a Proposed Project Restructuring of Jordan Employer Driven Skills Development Project, Approved on June, 2008 to the Hashemite Kingdom of Jordan, World Bank Report No. RES9540, January 30, 2013. 15. World Bank, Resolving Jordan’s Labor Market Paradox of Concurrent Economic Growth and High Unemployment, Report No. 39201-JO, March 31, 2007. 50 IBRD 33424 35°E 36°E 37°E 38°E 39°E 34°N 34°N n Se anea r a er dit JORDAN Me 33°N 33°N To To Zefat Damascus Lake To Tiberias Baghdad Um Qais Irbid IRBID IRBI D Ar Ruwayshid AJLUN Ajlun Al Mafrak River JARASH Jarash MAFRAK n Arda Mahattat al Halif Jorda BALQA Az 32°N As Salt 32°N Zarka AMMAN ZARKA Azraq ash - - Shishan Madaba To Jerusalem MADABA AMMAN Dead Sea To Al Jawf Al Mazra’ah - Al Qatranah To Al Karak Beersheba KARAK -- As Safi 31°N 31°N Ard TAFILAH as At Tafilah Sa w - Al Rashadiyah - Ba’ir aa n Ash Shawbak MA'AN Petra Al Jafr 0 0 50 Kilometers Ma'an 0 25 50 Miles 30°N Ra’s an Naqb 30°N 38°E 39°E AQABA Ad Disi To Nuweiba JO R D A N Jabal Ram Aqaba (1,734 m) SELECTED CITIES AND TOWNS aba GOVERNORATE CAPITALS Al Mudawwarah of Aq This map was produced by NATIONAL CAPITAL To the Map Design Unit of The World Bank. The boundaries, Al B'ir RIVERS Gulf colors, denominations and any other information shown 29°N on this map do not imply, on MAIN ROADS the part of The World Bank To Group, any judgment on the RAILROADS Al B'ir legal status of any territory, or any endorsement or acceptance of such GOVERNORATE BOUNDARIES boundaries. INTERNATIONAL BOUNDARIES 35°E 36°E 37°E JANUARY 2005