The World Bank (P128378) REPORT NO.: RES35837 RESTRUCTURING PAPER ON A PROPOSED PROJECT RESTRUCTURING OF SKILLS DEVELOPMENT AND INNOVATION SUPPORT PROJECT APPROVED ON JANUARY 28, 2014 TO MINISTRY OF FINANCE EDUCATION EUROPE AND CENTRAL ASIA Regional Vice President: Cyril E Muller Country Director: Linda Van Gelder Senior Global Practice Director: Jaime Saavedra Chanduvi Practice Manager/Manager: Harry Anthony Patrinos Task Team Leader: Bojana Naceva The World Bank (P128378) ABBREVIATIONS AND ACRONYMS EMIS Education Management Information System ENQA European Association for Quality Assurance in Higher Education FITD Fund for Innovation and Technological Development HE Higher Education HEAEB Higher Education Accreditation and Evaluation Board HEIs Higher Education Institutions HEMIS Higher Education Management Information System IBRD International Bank for Reconstruction and Development MOES Ministry of Education and Science MOF Ministry of Finance NETs National Expert Teams NTTO National Technology Transfer Office PDO Project Development Objectives SDIS Skills Development and Innovation Support TVET Technical and Vocational Education and Training VET Vocational Education and Training The World Bank (P128378) BASIC DATA Product Information Project ID Financing Instrument P128378 Investment Project Financing Original EA Category Current EA Category Partial Assessment (B) Partial Assessment (B) Approval Date Current Closing Date 28-Jan-2014 31-May-2019 Organizations Borrower Responsible Agency Ministry of Finance Ministry of Education and Science Project Development Objective (PDO) Original PDO The Project Development Objective (PDO) is to improve transparency of resource allocation and promote accountability in higher education, enhance the relevance of secondary technical vocational education, and support innovation capacity in Macedonia. OPS_TABLE_PDO_CURRENTPDO Summary Status of Financing Net Ln/Cr/Tf Approval Signing Effectiveness Closing Commitment Disbursed Undisbursed IBRD-83320 28-Jan-2014 07-Feb-2014 11-Mar-2014 31-May-2019 24.00 8.61 14.02 Policy Waiver(s) Does this restructuring trigger the need for any policy waiver(s)? No The World Bank (P128378) I. PROJECT STATUS AND RATIONALE FOR RESTRUCTURING 1. This restructuring proposes a 13-month extension of the project closing date from May 31, 2019 to June 30, 2020. This would be the first extension of the project. The extension would allow for completion of key activities that would support the achievement of the Project Development Objective (PDO) and enhance project sustainability, namely, to: a) complete the comprehensive Technical Vocational Education and Training (TVET) reform, including procurement of the equipment for TVET schools in accordance to the new curriculum; b) provide support for establishment and support the work of the National Committee to complete the new financing model for Higher Education; c) provide support for establishment of the National Quality Assurance Agency for Higher Education and provide capacity building for its staff; d) fully utilize the allocated funds for innovation instruments at the Fund for Innovation and Technological Development (FITD), 2. The Skills Development and Innovation Support (SDIS) Project was designed to support interventions that foster education and skills relevant to the job market and enhance the innovation capacity and activity of firms in the Republic of North Macedonia. The project contributed to outcomes of the previous Country Partnership Strategy FY15-FY18 and will contribute to Focus Area 2 (Human Capital and Inclusive Growth) of the new Country Partnership Framework for North Macedonia for the period January 1, 2019 – June 30, 2023. 3. Progress towards the PDO and overall implementation progress are rated moderately satisfactory. Since its effectiveness on March 2014, project implementation has proceeded unevenly, with Component 2 (Modernization of Secondary TVET) and Component 3 (Improving the Innovative Capacity of Enterprises and Collaboration with Research Organizations) progressing well, whereas the activities within Component 1 (Improving Transparency of Higher Education) have encountered delays. The reasons for the delays in finalizing the two major activities, establishment of a sound quality assurance system and a transparent model for financing of the higher education (HE) sector are due to the long period of political transition and the time needed to develop the draft Law on Higher Education, which regulates provisions related to the quality assurance and HE finance. The implementation of the project activities accelerated significantly in the last eight months and resulted in adoption of all program documents for TVET reform and in setting up the relevant mechanisms and structures in the education and innovation systems. After the adoption of the new law for Higher Education, the government initiated establishment of a Quality Assurance Agency and National Council, critical structures for improvement of the quality of Higher Education as well as bodies that will be responsible for implementation of the new accreditation and financing models. In addition, the legal framework for the innovation instruments was concluded and calls for the two remaining innovation instruments were launched. This progress suggests that the project will complete the remaining activities and achieve its PDO during the extension period. 4. Implementation status per component: a) Component 1: Improving Transparency of the Higher Education. The higher education elements of the project have experienced notable delays in implementation as stated above. However, the new Law on HE was adopted, which inter alia stipulates establishment of an Agency for Quality Assurance, responsible for accreditation and institutional evaluation of the HE Institutions, as well as the establishment of a National Council as a new independent body of experts, responsible for HE, science and research. The The World Bank (P128378) National Council will have a mandate to propose a Rulebook on the Measures and Criteria for the Financing of the HE. This body, as well as the Agency for Quality Assurance will be constituted by May 19, 2019. The project supported external institutional assessment of all public universities has been completed for the first time in the country by the impartial international evaluation body (European University Association). The results of this activity are a detailed assessment accompanied by action plans for achieving necessary reforms for each university, as well as a comprehensive report for the overall HE system in the country. These reports are also used by the government in establishment of the legal framework that will govern the new Agency for Quality Assurance. The Agency will have a mandate to stabilize the accreditation system and ensure sustainability regardless of political instability as well as support improvement of institutional quality assurance mechanisms across all higher education institutions (HEIs). A model of Central Database for HE has been designed along with the assessment of the information systems of the accredited HEIs. The results of the analysis underpinned the design of a comprehensive Education Management Information System (EMIS) for the accredited HEIs. b) Component 2: Modernization of Secondary Technical Vocational Education and Training. The secondary education TVET reform is in its final stage, and most of the remaining activities will be completed by the original closing date, with the exception of procurement of the school equipment. So far, 152 new occupational standards were developed by employer-led National Expert Teams (NETs) and approved by the Ministry of Education and Science (MOES), involving around 250 representatives from the economy and education sectors, including university teachers. In addition, 53 new Qualification Standards and associated Syllabi were developed by education-led NETS, involving 220 representatives from the economy and education sectors, including university teachers. For the purpose of smooth implementation of the new concept in the TVET secondary schools, the TVET Center supported by the international consortium have trained 28 master trainers who then trained around 2000 TVET and general education teachers, and managers from all TVET secondary schools in the country. The second round of the school-industry collaboration grant program was launched in September 2018. 25 schools were awarded 3 grant measures amounting around 300,000 Euro. The program is governed by the grant manual which was revised based on the findings from the evaluation of the ten grant-projects that have been implemented in the previous school year. The implementation of the new reform in the vocational secondary education schools will start in September 2019. c) Component 3: Improving the Innovative Capacity of Enterprises and Collaboration with Research Organizations. This component supported the operationalization of a key segment in the institutional infrastructure for innovation in the country, the Fund for Innovation and Technological Development (FITD). The project contributed to the design and piloting of 4 financial instruments at FITD – from mini grants for start-ups (proof of concept, prototype), matching grants for innovation commercialization, technology extension, to business acceleration that helped firms to develop, improve and realize their capacity for innovation in new products, processes and services. The government took advantage of the experience and governance structure at FITD and launched new measures for innovation in the third pillar (implemented by FITD) under a Plan for Economic Growth, a new program for enterprise support, funded by the Government in 2018. Since project inception, the FITD supported 52 firms in total through the SDIS Project and over 70 firms through the Government’s Plan for Economic Growth (2018). Since project inception, around 3 million EUR has been committed and a total of EUR 1,005,310 has been disbursed for the innovation instruments. Four calls for mini-grants for start-ups and spin-off companies (instrument 1) and for matching grants for commercialization of innovation (instrument 2) have been completed and have resulted in funding support for innovation in enterprises from different sectors. From these calls, a total of 31 beneficiary companies completed their projects and 15 companies have paid The World Bank (P128378) royalties to the FITD. Further, a call for accelerators (instrument 3) was completed successfully, and as a result, three accelerators were awarded with a total amount of 1.4 million EUR and 0.4 million EUR of matching contribution. The contracts with the accelerators were concluded for 36 months. A call for the technology extension program (instrument 4) resulted in 3 projects selected for funding at nearly 1 million from the SDIS Project and 1.4 million in matching contributions from the beneficiaries. The FITD has also assessed firm capabilities and facilitated training of beneficiary firms. The proposed extension of project duration by 13 months will contribute to full disbursement of the undisbursed funds for the financing instruments and implementation of the commitments already made (accelerator and commercialization of innovation) at FITD. 5. The current disbursement ratio is 41.56%1. The implementation of many subcomponents has advanced significantly lately which has resulted in a total of 11,325,185 EUR (or 64% of loan proceeds) committed and/or disbursed. Budget allocation of funds for the project is appropriate and the team does not anticipate any problems with the Ministry of Finance imposing an allocation ceiling for the IBRD project implementation as it was in the recent past. Disbursement Category Original Disbursed as of Committed as of Total Allocation February 2019 February 2019 Disbursement + Commitment 1.Goods,works, non-consulting 7.847.000,00 5,105,013.38 715,772.62 5,820,786.00 services, consultants ‘services Training and Operating Costs for Parts 1, 2.1, 2.2 (i), 3.1 and 4 of the Project 2.School Grants 649, 000.00 97,802.74 280,398.26 378,201.00 3. Innovation Grants 9,159,750.00 2,109,600.81 2,972,348.12 5,081,948.93 4. Front and Fee 44.250,00 44,250.00 0 44.250,00 Total 17.700.000,00 7,356,666.93 3,968,519.00 11,325,185.93 (expressed in Euros) 6. In terms of financial management and procurement arrangements, the project has consistently been rated satisfactory. The client is in compliance with the audit covenants and there are no outstanding audits under the project. The financial project audit for 2017 had an unmodified (clean) opinion and no internal controls issues have been mentioned in the related management letter. 7. All legal covenants have been complied with. 8. Environmental Safeguards. The project rates satisfactorily under a category “B.” 9. The rationale for Restructuring. The country experienced a long political crisis that resulted in frequent changes of the Minister of Education and Science and subsequently slow implementation progress of the project activities. However, the MOES is very committed to achieving the PDOs, the legal framework for all project activities has been adopted, and its implementing units are sufficiently staffed to support the project implementation over the 13 additional months. The project extension would allow for finalization of all initiated 1 Client connection The World Bank (P128378) activities and improve the likelihood of achieving the project’s development outcomes. In particular, the 13- month extension would allow the project to finalize the below-described activities: a) Finalization of the procurement of equipment for the TVET schools. The process has been initiated and it is estimated that all procedures including delivery of goods to all TVET schools will be completed by May 2020. b) Support in establishing and initial capacity building to the new Agency for Quality Assurance in Higher Education (HE) and introducing the new model for HE financing. The procedure for establishment of the Agency and the National Council for Higher Education & Science and Research-- responsible for a new funding model--were initiated. In November 2018, the Ministry of Education and Science put into Government procedure two sets of documents initializing the formation of the Agency for Quality Assurance and the National Council for Higher Education & Science and Research. These two documents have clear duties and tasks for various Government and Non- Government organizations (Chambers of Commerce, Interuniversity Conference etc) that they have to adhere to in order for these bodies to be formed by May 2019, as prescribed by the law. Moreover, the SDIS project has already hired a consultant that will prepare the basic documents for the operation of the Agency for Quality Assurance so that this agency can have a smooth start once formed. The project has also funded excellent technical advisory work to support the reform in the HE financing model for North Macedonia, so that the new National Council does not start from scratch once established. c) Design and operationalization of the Higher Education Management Information System (HEMIS). The procurement procedure, along with the software development and implementation, as well as the delivery of hardware is estimated to be completed by May 2020. The HEMIS and the planned interconnectivity with HEIs is an important tool for the Ministry’s decision-making. Once both systems are operational, they can communicate, and the data can be used to provide input data for the new financing formula. d) Establishment of the National Technology Transfer Office (NTTO). The legal amendments and terms of reference for the NTTO were developed under the project. The Government has decided that the NTTO should be established under the FITD. The Bank team working on innovation is supporting the next steps and timeline for NTTO establishment and operationalization. e) Full implementation of FITD’s innovation funding instruments to disburse EUR 7,050,150 currently available, which includes EUR 2,972,348 in commitments, through existing grant agreements and EUR 4,077,802 that is uncommitted. FITD closed a call for mini-grants for start-ups in December 2018 and the evaluation and selection process is ongoing. It is expected that the selection will result in contracting EUR 1.5 million. FITD also plans to launch a call for commercialization of innovation in March 2019, with an allocation of EUR 5 million, that will include the remaining uncommitted funds from the loan proceeds and government funding. f) Strengthen the citizen engagement aspects of the project. The project has done a good job in supporting and ensuring the collection of feedback and perceptions on the TVET reforms from all stakeholders including students. The project has supported an inclusive reform of the TVET system, mainly driven by the private sector with the support of the MOES and its agency for vocational education and training. The feedback, collected via the surveys and focus group discussions with stakeholders and The World Bank (P128378) students, were used to inform the development of the qualification standards and revision of the grants program for school-industry cooperation. The restructuring will further strengthen and start to monitor the ongoing citizen engagement activities by: (i) continuing the participatory planning activities (i.e. repeating the students’ focus groups before project closing, collecting their feedback and laying the focus on discussions how previous feedback was used to inform and improve project activities); (ii) introducing a participatory grant application criterion (i.e. ensuring that students’ participation during grant preparation is well documented in the grant application); (iii) conducting one final satisfaction survey (following up on previous progress); and (iv) including an indicator to monitor the implementation of the feedback into the TVET reforms. II. DESCRIPTION OF PROPOSED CHANGES The proposed restructuring seeks authorization to (i) extend the Project’s closing date by 13 months, (ii) revise the Results Framework, and (iii) strengthen citizen engagement and beneficiary feedback monitoring. Specifically, this restructuring seeks approval for the following revisions to the project: 1. The thirteen-month extension of the project from May 31, 2019 to June 30, 2020, requested by the Ministry of Finance in the letter dated October 26, 2018, would provide additional time for finalizing some of the key project activities, which would contribute to the sustainability of the established bodies and enable the government to support initial capacity building to the Agency for Quality Assurance, and draw the lessons from the evaluations to be carried out during the extended period of time. 2. The restructuring will provide the opportunity to further strengthen the citizen engagement activities (participatory planning through focus groups, participatory grant application criterion, final satisfaction survey) and regularly monitor beneficiary feedback through the introduction of an additional indicator. This will ensure that the SDIS project is in compliance with World Bank corporate commitments on citizen engagement. 3. The PDO remains unchanged, though changes to the Results Framework are needed to improve the measurement of progress towards project outputs and outcomes. The revision includes two PDO and five intermediate indicators. One PDO indicator is revised to reflect the legal changes in the higher education area, and the PDO indicator on TVET is revised to improve the measurement of its progress. The changes to the intermediate indicators reflect the extended timeline as well as changes to the implementation of several project activities. In particular, this restructuring includes revision to the following indicators: Table 1. Summary of proposed changes to the Results Framework. Before revision Proposed revision by MOES (October 2018) PDO Results Indicators The World Bank (P128378) Indicator Name Indicator Name Comments % of public universities % of study programs of According to the Law on Higher Education, the accredited/reaccredited, public universities existing universities cannot be accredited again, utilizing new quality accredited/reaccredited, utilizing new quality assurance and accreditation assurance and accreditation utilizing new quality measures developed in accordance with Bologna- measures developed in assurance and accreditation defined EU norms and practices, since they accordance with Bologna- measures developed in already have valid accreditation, and institutional defined EU norms and accordance with Bologna- reaccreditation is not mandatory. practices. defined EU norms and Study programs should be reaccredited every 5 practices, by the newly years and new study programs will undergo established Agency for accreditation /reaccreditation, utilizing new Quality Assurance. quality assurance and accreditation measures developed in accordance with Bologna-defined End target in 2019 - 80% End target in 2020 – 80%of EU norms and practices by the newly established submitted study programs Agency for Quality Assurance of HE. Additionally, according to the new Law on HE, the HEAEB should be replaced with Agency for Quality Assurance by May 2019 % increase in number of % of secondary TVET The number of enrolled students varies each year, secondary TVET students students benefiting from and this is the reason to reformulate this indicator benefiting from practical practical training in SME since it cannot be compared to the baseline and training in SME and large - and large-sized firms. should be measured as a percentage of enrolled sized firms. End target in 2020- 40% students in 2nd, 3rd and 4th year for the End target in 2019 -40% corresponding period. Intermediate Results Indicators Indicator Name Indicator Name Comments The World Bank (P128378) Board of Quality Agency for Quality According ENQA membership criteria, new Assurance and Assurance for HE achieves established agency should have been operational Accreditation achieves affiliate membership in for at least 2 years and should have completed at membership in ENQA (the ENQA (the European least 5 review reports in order to become a European Association for Association for Quality member as well meet all the eligibility criteria for Quality Assurance in Assurance in Higher ENQA membership. Higher Education) Education) End target in 2019: End target in 2020: Agency Agency for Quality for Quality Assurance of Assurance of HE achieves HE achieves affiliate membership status in membership status in ENQA ENQA IP related cases handled Deleted Given that the NTTO has not been established so for industry as measured far and given the lack of time to achieve results of by number of patent such scope until the project closing date, this landscape analysis (PA), indicator is obsolete patents filed (P) and technology in-licensing (importation of technology) Domestic inventions by Deleted RTIs commercialized as Given that the NTTO has not been established so measured by number of far and given the lack of time to achieve results of patents filed(P), such scope until the project closing date, this licenses(L), and contract indicator is obsolete research (C ) Share of updated TVET Deleted All TVET secondary school programs are being secondary school updated, therefore, there is no need to distinguish programs targeting key them by key sectors of local economy. sectors of the local economy. N/A. (New one) The satisfaction survey will assess students’ Citizen Engagement satisfaction regarding their engagement during Indicator: Percentage of the grant preparation. students that participated in the Grant Program satisfied with their engagement during grant preparation. Baseline: TBD End target: 60 % The World Bank (P128378) III. SUMMARY OF CHANGES Changed Not Changed Results Framework ✔ Loan Closing Date(s) ✔ Disbursement Estimates ✔ Implementing Agency ✔ DDO Status ✔ Project's Development Objectives ✔ Components and Cost ✔ Cancellations Proposed ✔ Reallocation between Disbursement Categories ✔ Disbursements Arrangements ✔ Overall Risk Rating ✔ Safeguard Policies Triggered ✔ EA category ✔ Legal Covenants ✔ Institutional Arrangements ✔ Financial Management ✔ Procurement ✔ Implementation Schedule ✔ Other Change(s) ✔ Economic and Financial Analysis ✔ Technical Analysis ✔ Social Analysis ✔ Environmental Analysis ✔ The World Bank (P128378) IV. DETAILED CHANGE(S) OPS_DETAILEDCHANGES_LOANCLOSING_TABLE LOAN CLOSING DATE(S) Original Revised Proposed Proposed Deadline Ln/Cr/Tf Status Closing Closing(s) Closing for Withdrawal Applications IBRD-83320 Effective 31-May-2019 30-Jun-2020 30-Oct-2020 OPS_DETAILEDCHANGES_DISBURSEMENT_TABLE DISBURSEMENT ESTIMATES Change in Disbursement Estimates Yes Year Current Proposed 2014 1,300,000.00 500,000.00 2015 1,500,000.00 1,000,000.00 2016 2,300,000.00 2,000,000.00 2017 1,700,000.00 2,000,000.00 2018 8,200,000.00 3,500,000.00 2019 9,000,000.00 4,000,000.00 2020 0.00 5,000,000.00 2021 0.00 6,000,000.00 . The World Bank (P128378) . Results framework COUNTRY: North Macedonia Project Development Objectives(s) The Project Development Objective (PDO) is to improve transparency of resource allocation and promote accountability in higher education, enhance the relevance of secondary technical vocational education, and support innovation capacity in Macedonia. Project Development Objective Indicators by Objectives/ Outcomes RESULT_FRAME_TBL_PDO Indicator Name DLI Baseline End Target Improving Transparency of Higher Education % of public universities receiving financing based on new, 0.00 80.00 transparent funding model. (Percentage) % of study programs accredited /reaccredited utilizing new quality assurance measures developed in accordance with 0.00 80.00 Bologna-defined norms by the new Quality Assurance Agency (Percentage) Action: This indicator has been Revised Modernization of Secondary Education Vocational Education and Training % of secondary TVET students benefiting from practical training 36.30 40.00 in SME and large-sized firms. (Percentage) Action: This indicator has been Revised Improving the Innovative Capacity of Enterprises and Collaboration with Research Organization Share of private funding mobilized as a percentage of FITD 0.00 25.00 investments in innovation activities. (Percentage) % of beneficiaries that sign collaborative agreements between 0.00 20.00 firms and academia. (Percentage) The World Bank (P128378) PDO Table SPACE Intermediate Results Indicators by Components RESULT_FRAME_TBL_IO Indicator Name DLI Baseline End Target Improving Transparency of Higher Education The new funding model is not developed Implementation of new higher education funding model. (Text) New funding model implemented loped Number of public universities that have completed an external evaluation utilizing new quality assurance measures developed in 0.00 5.00 accordance with Bologna-defined EU norms and practices. (Number) National TTO established and operational (Text) Not in existence TTO operational Agency for Quality Assurance and Accreditation achieves affiliate Board of Quality Assurance and Accreditation NOT a Agency for Quality Assurance achieves affiliate member membership in ENQA (the European Association for Quality member of ENQA status in ENQA Assurance in Higher Education) (Text) Rationale: Action: This indicator has been Revised IP related cases handled for industry as measured by number of patent landscape analysis(PA), patents filed(P) and technology in- 0.00 30.00 licensing(importation of technology) (Number) Action: This indicator has been Marked for Deletion Domestic inventions by RTIs commercialized as measured by number of patents filed, licenses, and contract research 0.00 15.00 (Number) Action: This indicator has been Marked for Deletion Modernization of Secondary Technical Vocational Education and Training The World Bank (P128378) RESULT_FRAME_TBL_IO Indicator Name DLI Baseline End Target TVET action plan that promotes general and broader technical Not in existence Model adopted education and competence- based learning adopted (Text) % of developed TVET occupational standards by professional 0.00 100.00 fields (Percentage) Share of updated TVET secondary school programs targeting key 0.00 70.00 sectors of the local economy. (Percentage) Action: This indicator has been Marked for Deletion % of updated TVET curricula in line with occupational standards 0.00 100.00 (Percentage) % Increase in number of companies providing practical training 0.00 30.00 to secondary TVET students. (Percentage) Manual and training materials for quality delivery of new No Yes curricula adopted (Yes/No) Improving the innovative capacity of enterprises and collaboration with research organizations Share of resources dedicated to Fund for Innovation and Technology Development (FITD) instruments disbursed 0.00 100.00 (Percentage) Number of companies established via the FITD Accelerator 0.00 15.00 program (Number) Number of beneficiary firms to introduce new/improved 0.00 55.00 products (Number) Action: This indicator has been Revised % of beneficiaries receiving training and mentoring (Percentage) 0.00 70.00 Number of beneficiary firms to introduce new processes 0.00 4.00 (Number) Project Management and Monitoring and Evaluation The World Bank (P128378) RESULT_FRAME_TBL_IO Indicator Name DLI Baseline End Target Evaluations for Innovation grants designed, conducted and Not in existence Evaluation reports completed and results disseminated reports reported (Text) Skills Observatory established, providing information about TVET Not in existence The info on platform being updated at least twice per year and HE performance to the public (Text) Percentage of students that participated in the Grant Program satisfied with their engagement during grant preparation 0.00 60.00 (Percentage) Action: This indicator is New IO Table SPACE The World Bank (P128378)