WORLD BANK GROUP Annual Progress Report on the Reimbursable Advisory Services Program in Romania January 1 – December 31, 2017 Table of Contents Abbreviations…………………………………………………………………………………………………………………………………………..2 Introduction ................................................................................................................................................... 4 Overview of the World Bank RAS Program in 2017 ...................................................................................... 5 Complementary Bank-Financed Activities Carried Out in 2017 .................................................................. 10 Coordination Arrangements ........................................................................................................................ 10 Implementation Challenges and Other Issues ............................................................................................ 10 RAS Results .................................................................................................................................................. 11 Lessons Learned .......................................................................................................................................... 15 Going Forward ............................................................................................................................................. 16 Annex 1. Summary Description of Each Ongoing RAS in 2017 Using European Funds ............................... 17 1 Abbreviations ANAP National Public Procurement Agency ANRE National Energy Regulatory Agency CDSACP County Directorates for Social Assistance and Child Protection CPF Country Partnership Framework CPS Country Partnership Strategy CPM Chancellery of the Prime Minister CY Calendar Year CLLD Community Led Local Development EBRD European Bank for Reconstruction and Development EC European Commission EIB European Investment Bank ERDF European Regional Development Fund ESCF European Structural Funds and Cohesion Funds ESIF European Structural Investment Funds EU European Union FY Fiscal Year FRs Functional Reviews GoR Government of Romania GSG General Secretariat of the Government IFI International Financial Institution ISP Institutional Strategic Planning MA Managing Authority MAP Modernization of Administration Program MARD Ministry of Agriculture and Rural Development M&E Monitoring and Evaluation MEWF Ministry of Environment, Waters and Forests MESR Ministry of Education and Scientific Research MoU Memorandum of Understanding MoNE Ministry of National Education MoPF Ministry of Public Finance MLFSPEP Ministry of Labor, Family, Social Protection and Elderly Persons MDRAPFE Ministry of Regional Development, Public Administration and European Funds MoT Ministry of Transport NACS National Agency for Civil Servants NACLR National Agency for Cadaster and Land Registration NACRPA National Authority for Children’s Rights Protection and Adoption NSVFSA National Sanitary veterinary and Food Safety Authority OP Operational Programme PNDR Operational Programme for Rural Development POCA Operational Programme for Administrative Capacity POAT Operational Programme for Technical Assistance 2007-13 POCU Operational Programme for Human Capital PODCA Operational Programme for Administrative Capacity Development 2007-13 POR Regional Operational Programme 2007-13 POSCCE Operational Programme for Increased Economic Competitiveness 2007-13 POSDRU Operational Programme for Human Resources Development 2007-13 RAS Reimbursable Advisory Services RIA Regulatory Impact Assessment RCC Romanian Competition Council SABER WfD Systems Approach for Better Education Results -Workforce Development 2 SCD Systematic Country Diagnostic SU Strategy Unit TTL Task Team Leader UNICEF The United Nations Children's Fund 3 Introduction 1. Through Bank Reimbursable Advisory Services (RAS), the Government of Romania accesses international experience and global best practice in pursuit of strengthening Government capacity in policy and strategy formulation, analysis, diagnosis, sector and project/program planning, design, and implementation. By signing the MoU in January 2016, the Government of Romania acknowledged the value of the Bank’s knowledge and advice as an important element of the effective implementation of its priorities, strategies, and projects financed by the European Structural and Investment Funds (ESIF) during the 2014-20 programing period. Thus, the MoU facilitates access to a large pool of technical assistance funds--the European Fund for Regional Development, the European Social Fund, the Cohesion Fund, and for the first time, the European Agricultural Fund for Rural Development and the European Maritime and Fishery Fund. 2. As part of the reporting activities agreed with the Government under the MoU, the Bank prepares an annual progress report on the RAS program. This is the second progress report on the RAS program paid for by the Government during the 2014-20 EU programming period. The report covers the period January 01, 2017 – December 31, 2017. The report summarizes and discusses the results of the 2017 RAS program, considers its challenges and lessons, and looks at the way forward for this program. 3. This progress report is intended for R A S government beneficiaries and managing authorities. This includes the Ministry of European Funds as the coordinator of the Bank’s RAS activities under its mandate to coordinate advisory services provided by all international financial institutions. The report will also be of interest to other ministries and government institutions, Managing Authorities and Intermediary Bodies. The European Commission is also considered an important audience for this report. 4 Overview of the World Bank RAS Program in 2017 4. Following the 2016 MoU signing, the RAS portfolio increased rapidly reaching a total of 11 agreements for EUR 27.79 million (as of the end of 2017). One agreement ended and two new RAS agreements totaling EUR 11.42 mil were signed in 2017, as shown in Figure no. 1. The RAS agreements signed in 2017 continued to focus on institutional capacity building, and to support the Government further in preparing for the management and prioritization of investments in key areas. The overall volume of the program returned to levels similar to the previous programming period, with increases in both the average duration of RAS to 30 months, and average signed amount to EUR 2.53 mil, from 15 months and EUR 1.15 mil respectively under the previous MoU. The evolution of the average size and duration of a RAS agreement since 2010 is presented in Figures 2 and 3. As the Bank consolidated the program by focusing on key areas of its comparative advantage, the number of projects decreased but the volume of active RAS program is similar to peak levels in 2014-15. The Bank is now engaged in longer-term activities, including strengthening the administrative capacity in beneficiary institutions, where the client can maximize the benefits from the Bank’s knowledge and role. Figure 1 ROMANIA RAS SIGNED USING EU FUNDS/ PROGRAMMING €18.00m PERIOD SIGNING BY CY 2012 - 2017 (EUR million) 20 €16.00m 18 18 16 €14.00m 14 €15.87m €12.00m Number of agreements 12 €10.00m 11 Signed Amount 10 €16.37m €14.99m €8.00m 9 8 8 €11.42m €6.00m 6 €13.04m €4.00m 4 €3.18m €2.00m €0.88m €0.38m 2 2 2 2 1 €0.00m 0 2010 2011 2012 2013 2014 2015 2016 2017 Calendar Year 2007-2013 programming period, Functional Reviews 2007-2013 programming period 2014-2020 programming period # of Agreements 5 Figure 2 RAS Average amount/ Programming period CY 2010 - 2017 €3.00m €2.50m €2.00m €1.50m €2.53m €1.00m €0.50m €1.15m €0.00m 2007-2013 programming period 2014-2020 programming period Figure 3 RAS average duration in months/ Programing Period 35 30 25 20 15 30 31 10 18 15 5 0 2007-2013 programming period - 42 RAS signed 2014-2020 programming period - 11 RAS signed 2010-2015 2016-2017 2007-2013 programming period, RAS average original duration 2007-2013 programming period, RAS average amended duration 2014-2020 programming period, RAS average original duration 2014-2020 programming period, RAS average amended duration 5. Line ministries and agencies remained the main beneficiaries of advisory services in 2017. The two new agreements were signed with one of the past RAS clients: the Ministry of Regional Development, Public Administration and European Funds and one “new” RAS client-institution: Romanian Agency for Quality Assurance in Higher Education. 6 Table 1. World Bank - Romania ongoing Advisory Services Agreements, in 2017 Source Signing Agreement Title Government Institution of EU year Funds 1 Assistance for Establishing a Strategy Unit General Secretariat of the Government POCA 2016 2 Development of the Capacity of Central Public General Secretariat of the Government POCA 2016 Administration to Carry out Impact Studies 3 Assistance on Informed Decision-Making on National Center for Technical and POCA 2016 Investments in Infrastructure Vocational Education and Training Development 4 Development of Plans for Deinstitutionalization National Authority for Children Rights POCA 2016 of Children Deprived of Parental Care and their Protection and Adoption Transfer to Community-based Care 5 Assistance on Strategic Planning and Budgeting General Secretariat of the Government POCA 2016 Capacity 6 Assistance for Capacity Development for Ministry of National Education POCA 2016 Monitoring and Evaluating the Implementation of Education Strategies (ex-ante conditionalities) & Improving Education Policies 7 Support to the Implementation of the Public National Authority for Public Procurement POCA 2016 Procurement Strategy 8 Establishment of a Framework for using Ministry of Public Finance POCA 2016 Derivatives and Executing Swap Transactions 9 Informing Project Evaluation Ministry of European Funds (currently POCU 2016 Ministry of Regional Development, Public Administration and European Funds) 10 Assistance to Enhance Quality Assurance in Romanian Agency for Quality Assurance in POCA 2017 Higher Education System in Romania Higher Education 11 Supporting the Implementation of Romania’s Ministry of Regional Development, Public POCU 2017 Human Development operational Programme Administration and European Funds (POCU) 2014-2020 6. During 2017, the ongoing RAS agreements continued to cover core priority areas: education, strategic planning, impact assessments, children ’s rights, public procurement, use of financial instruments and project evaluation. The RAS Informing Project Evaluation, signed in 2016 and closed in 2017, led to continued engagement in this area by signing, in 2017, the biggest RAS agreement in the Romania portfolio to date with the Ministry of Regional Development, Public Administration and European Funds. Ongoing RAS agreements continue to assist the line ministries on policy formulation, strategy development and capacity building, as is the case for inf orming decisions on education infrastructure and on the deinstitutionalization of children. 7. Reflecting the evolution of the Government needs, the Bank activities now concentrate more on capacity building and implementation support. Bank support for programming period 2007-2013 focused on supporting the Government in its preparation for the next programming period, in particular, in carrying out assessments and formulating strategies. The RAS program under the 2016 MoU goes one step further and supports enhancing the institutional capacity and advising the Government on implementing the strategies and priorities adopted by the Government. The capacity building elements remain at the core of the current RAS programs, as all RAS agreements under the 2014-20 programming period include such elements. For the Government to be able to meet some delayed action requirements related to ex-ante conditionalities and unlocking access to EU funds under the 2014-20 period, a few of the agreements signed for programming period 2014-2020 still support strategy formulation at the request of selected ministries. Figure 4 reflects the change in Government needs, with an increased focus on capacity building and implementation support, and a significant drop in strategy formulation. 7 Figure 4 RAS type of assistance/ Programming Period 120% 100% 100% 100% 86% 82% 80% 64% 60% 55% 40% 19% 18% 20% 0% 2007-2013 programming period - 42 RAS signed 2014-2020 programming period - 11 RAS signed 2010-2015 2016-2017 Assesment Capacity building Strategy Formulation Implementation support 8. The RAS agreements signed in 2017 are paid for by the client institutions using EU funds through two Operational Programmes (OPs) under the 2014-20 programming period. One RAS agreement was signed using funds from the Operational Programme for Administrative Capacity 2014 – 2020 (POCA), for EUR 1.42 million, and one RAS agreement was signed using European funds through the Operational Programme for Human Capital 2014-2020 (POCU), for EUR 10 million. 9. In 2017, 25 percent of the total outputs envisaged under the ongoing RAS were delivered. The Bank mobilized more than 300 experts to carry out activities under these eleven agreements. During the 2017 calendar year, 19 contractual outputs and 16 progress reports were submitted to the client institutions. Some outputs continued to support the Government in meeting actions related to accessing EU funds under the 2014-20 programming period, although not constituting ex-ante conditionalities per se. More than 110 events (workshops, trainings, etc.) took place during 2017 to consult and inform the preparation of outputs or to contribute to institutional capacity building. 10. Seventeen invoices were issued in 2017, out of which only one was unpaid at the end of the year. The invoices are issued upon approval of corresponding outputs submitted by the Bank and in line with the timeline agreed in the RAS agreement. 11. Additional requests for RAS under preparation. During 2017, several requests for technical assistance were received by the Bank covering a range of Government priorities (table below). At the end of 2017, the RAS pipeline to be funded from EU funds consisted of four requests listed in the table below. One of the requests resulted in signing in 2018: RAS on Developing Unitary Human Resources 8 Management System Within the Public Administration. The three remaining requests are expected to be signed by the end of 2018. The Bank has also received requests from local authorities for assistance to be provided by the Bank and to be paid using local budget funds. Table 2. World Bank - Romania Advisory Services for which public institutions requested assistance Agreement Title Agreement Client/ Government Source of EU Institution Funds (to be confirmed) 1 Development of a Monitoring and Ministry of Labor and Social Justice POCA Evaluation System for Social Inclusion 2 Enhancing the capacity of the Romanian National Institute of Statistics POCA National Statistical System 3 Developing and Improving the HR General Secretariat of the POCA Management System Government 4 Urban Wastewater Treatment Ministry of Water and Forestry POCA POAT: Operational Programme for Technical Assistance 2014 – 2020 POCA: Operational Programme for Administrative Capacity 2014 – 2020 POCU: Operational Programme for Human Capital 2014 – 2020 POR: Regional Operational Programme 2014-2020 PNDR: Operational Programme for Rural Development 2014 - 2020 12. Leveraging its rich experience, the Bank continued to take actions to improve its ability to respond to intensive client demand and ensure RAS complementarity and synergies. These include: (i) identifying a series of new international and local Bank experts to be team leaders for the RAS program in Romania; (ii) building on the knowledge of the local staff (all new and old staff in the World Bank office are involved in one or more RAS); (iii) increasing the number of international-level Romanian staff and experts; (iv) strengthening the RAS monitoring and coordination team in the Romania Office and improving knowledge sharing and learning, and (v) revising and improving quality control mechanisms required by Bank management for the RAS, both at entry and during implementation. 9 Complementary Bank-Financed Activities Carried Out in 2017 13. In parallel with the development of the new RAS program, dissemination activities for RAS completed before 2017 continued to allow for knowledge sharing within the government and beyond. 14. The Bank-led evaluation of the RAS program in Romania (2012-2015) completed in 2016 informed the preparation of the evaluative review of the Bank’s Country Partnership Strategy (CPS), FY2014-18, and its new strategy for Romania, the Country Partnership Framework, FY19-23 in FY2017. A substantial number of the CPS objectives was met through achieved RAS results. Even in cases where the CPS objectives were not met; the RAS intervention was a step forward for the targeted institutions. The current RAS program (programing period 2014-2020) aims to continue to help the Romanian Government to achieve its objectives. Coordination Arrangements 15. The MoU stipulated that its implementation will be monitored through coordination meetings chaired by the Ministry of Regional Development, Public Administration and European Funds every six months. Such meetings aimed to ensure coordination of activities and enhance synergies and complementarity of projects and initiatives financed through similar reimbursable advisory services provided by other eligible international financial institutions. Although such meetings were postponed until the RAS program matured, the Bank kept the Ministry of Regional Development, Public Administration and European Funds informed on the status of the ongoing activities through regular meetings. In addition, the Bank had several coordination meetings with the European Commission, as well as with the EIB and EBRD on areas of common interest to enhance synergies and complementarity of activities, and avoid any potential overlaps. 16. The RAS program continues to be anchored in the Bank’s CPS and i s monitored regularly alongside the rest of the Bank’s portfolio in Romania. Implementation Challenges and Other Issues 17. Frequent changes at management and political levels have slowed RAS implementation. After the legislative elections in December 2016, in early January 2017 a new cabinet with 26 ministers was confirmed. This resulted in two main changes affecting RAS: 1) the Ministry of Regional Development and Public Administration was merged with the Ministry of European Funds, the two ministries were separated again in January 2018, and 2) the Ministry of Economy was divided into several smaller entities. At the of end-June 2017, there was another change of government. Furthermore, for most ongoing RAS, management of the client institution changed at least once. With each change, the Bank had to reengage with the new government team and brief it on the Bank’s program in the country as well as the specific RAS. Frequent changes have also led to slow feedback of technical teams of the client institutions who sometimes required the management endorsement before providing feedback to the Bank. 18. For two RAS, the frequent political changes also led to changes in the originally-signed RAS agreements during implementation, as well as the use of resources not anticipated at RAS design. The RAS Assistance on Strategic Planning and Budgeting Capacity team had to prepare 13 instead of the originally agreed 10 reports on Institutional Strategic Planning (ISP) due to the creation of additional ministries during 2017. The RAS on Enhancing the Quality Assurance in Higher Education System in Romania, originally prepared in 2016, was signed in July 2017 with a reduced scope of work compared with the initial approved financing request only to be amended in October 2017 to return to the initially approved extended scope of work. 19. Working with multiple line ministries and agencies within a single RAS is a complex undertaking 10 that requires a focused effort on coordination, communications, and commitment. In 2017, the Bank had three ongoing RAS agreements with the GSG involving multiple line ministries and agencies as direct beneficiaries. Even though the contracting institution is GSG, it is the ministries and agencies that set priorities and policy decisions that could affect RAS implementation. In some instances, the coordination between multiple stakeholders has proved to be challenging. The lack of commitment, poor communication and coordination amongst third-party public institutions, the GSG and the Bank created issues in the implementation of RAS agreements, which sometimes led to significant changes to the initially agreed RAS scope of work. In addition, GSG had to align the provisions of the RAS program with the EU financing agreement. This alignment is made difficult by the fact that major changes to the scope of work of the financing contract during the evaluation period are not permitted by EU funding rules. In the case of the RIA2, the pilot with the National Authority for Energy (ANRE) was supposed to offer inputs on the order of smart metering system for electrical power use. In February 2017, during the implementation of the RAS pilot, ANRE announced the new order and timeline for the implementation of smart metering systems for electricity at the national level without benefitting from the RAS inputs. Therefore, the Agreement was amended, changing the activity from a RIA Pilot on smart electricity metering to a report on how to do RIA on smart electricity metering. In the end, the new order was cancelled in autumn of 2017. 20. Signing a RAS agreement funded through non-EU funds was a challenge for the client. In 2016, some local public institutions requested Bank assistance and were ready to allocate own resources to pay for RAS. While the MoU was explicitly crafted to provide the principles of engagement for RAS paid by client institutions using EU funds, it does not preclude the clients from using other resources. Signing the first RAS agreement paid from local funds was difficult considering there is no clear guidance and procedure for those who want to engage RAS using local funds. Following the first successful agreement, further requests were received in 2017 from other authorities, one of which resulted in signing the second RAS agreement paid from non-EU funds in March 2018. RAS Results 21. RAS on Support for the Establishment of a Strategy Unit • The Strategy Unit RAS supported the Romanian Government in conceptualizing and establishing a new strategic management system in the Center of Government. The RAS has reviewed and evaluated all national strategies resulting in a process of creating a national inventory of valid strategies that has now been placed on the GSG website. More than 360 strategies were reviewed and the 116 valid strategies were made available to the public. The valid strategies were analyzed at a high-level with a further 16 strategies analyzed in detail. Moreover, the analysis of the strategies led to the identification of strengths and weaknesses in current strategy formulation. The Bank team used this information to update the methodology for strategy formulation and develop a guide to help public officials draft new strategies. • The SU RAS has also supported the creation of the strategic management function and the reorganized structure of the GSG that corresponded to (i) Strategy formulation, (ii) Policy formulation and (iii) Implementation. The government has taken a decision to establish a strategy unit not as a standalone department but as a well-defined unit within the Directorate for Coordinating Policies and Priorities. This decision was adopted through a Government Note in February 2017. The definition functions and supporting procedures are detailed in the materials that the World Bank team has produced. • The RAS has also successfully completed a pilot to test the newly created methodology and accompanying guide. The first pilot supported the development of Romania’s strategy for the EU presidency at a time when the Brexit decision effectively moved up the timetable by 6 months as the UK pulled out of hosting the EU presidency in 2019. 11 22. RAS on Development of the Capacity of the Central Public Administration to Carry Out Impact Studies • The RAS successfully completed the ex-ante impact assessments in three policy areas: (i) “Better employment system within Romania’s central public administration”, for the National Agency for Civil Servants (NACS); (ii) “Potential risk to human health of fruit and vegetables from semi-subsistence farms” for the National Sanitary veterinary and Food Safety Authority (NSVFSA); and (iii) Design and Implementation of a national School after School program” for the Ministry of national Education. The impact assessment helped the public authorities to substantiate the relevant regulations and /or the policy formulation. • The RAS developed a methodology that was successfully implemented for data collection and analysis in view of preparing the impact assessment study for the Emergency medicine residency program. • The RAS has also successfully implemented a series of activities, such as training and workshops aiming at building the RIA community of practice and to raising awareness on the importance of using RIA instrument in the public administration. 23. RAS on Informed Decision – Making on Investments in Infrastructure • The final version of the draft Strategy for Infrastructure Investments in Education Institutions was delivered. This strategy was a requirement from the EU for the country to unlock €428.5 million (€362 million from ERDF and €66.5 million as the national contribution) for investments in education infrastructure, considering only the Thematic Objective 10 of the POR 2014-2020. Local authorities and public universities are now eligible to apply for these funds for rehabilitation, construction, rehabilitation, modernization and equipment for: 1. Nurseries, kindergartens, and primary and lower secondary education schools. 2. Technological high schools and Vocational Education and Training schools. 3. Universities. • On September 29, 2017, the team also delivered the Geospatial Education Infrastructure System. This is a tool for authorities at the central, regional and local levels make informed decisions about investments in education infrastructure, based on the prioritization criteria that is part of the Strategy for Infrastructure Investments in Education Institutions. 24. RAS on Development of Plans for the De-Institutionalization of Children Deprived of Parental Care and their Transfer to Community-Based Care. • The RAS informed the preparation of the List of eligible institutions to be financed from EU Funds, within the Regional Operational Programme Specific Objective 8.3 – Increasing the coverage of social services. Under the RAS, a thorough assessment of the resources currently available to the residential centers was carried out, a multi-criteria evaluation model of the residential centers was developed and applied using the collected data, and all centers in the country were ranked by their scores with and the centers with the highest being higher on the list of priorities for being closed down. • A thorough guide for designing the plans for closing down individual residential centers (together with a template for the closure plans) was prepared and approved by the National Authority for Children’s Rights Protection and Adoption (NACRPA); NACRPA requested the County Directorates for Social Assistance and Child Protection to use the guide when preparing their closure plans and to fill in all the information from the template to make sure that all the necessary steps are being followed and that all the necessary information is being provided by the applicants. The closure plan prepared by the CDSACPs is a prerequisite for applying for EU Funds (the NACRPA should give their approval in the first place). • A set of multi-disciplinary instruments was delivered as part of the guide to assess the children 12 living in residential centers to make sure that the children’s needs are identified and the right solutions for them are proposed; the instruments were applied for all the children from the centers for which closure plans were prepared (more than 1700 children) with the close hands-on support of more than 50 WB experts (social workers, sociologists and research assistants). 25. RAS on Strengthening Planning and Budgeting Capacity and Supporting the Introduction of Performance Budgeting • Through this RAS, the Bank helped make important progress toward performance-based budgeting with the adoption of a framework for program-based budgeting. This RAS is helping to strengthen the strategic planning and monitoring processes in Government to enhance the efficiency of public spending. The General Secretariat of Government (GSG) has adjusted its structure and functions to implement a new methodology for developing institutional strategic plans (ISPs) linked to program budgets. 26. RAS on Assistance to MNESR (MoNE) for Capacity Development for M&E the Implementation of Education Strategies • A Monitoring and Evaluation (M&E) Methodology and Instruments was delivered as part of the activities under the RAS Capacity Development for M&E the Implementation of Education Strategies. This Methodology is aimed at supporting the Ministry of National Education (MoNE) to undertake effective M&E techniques, and to coordinate in a unitary and integrated way the M&E of the implementation of the education strategies representing ex-ante conditionalities for Romania’s access to EU funds under the Programming Period 2014-2020: Strategy to Reduce Early School Leaving, the Strategy for Increasing Tertiary Education Attainment, Quality and Efficiency, the Strategy for Lifelong Learning, and the Strategy for Vocational Education and Training. Relevant MoNE staff as well as representatives of other agencies benefited from training, coaching and mentoring and applied the M&E Methodology as part of the first MoNE monitoring and reporting exercise initiated in 2017. • The SABER Workforce (WfD) Development Study delivered on June 30, 2017. This study is aimed at supporting a structured policy dialogue with decision makers and stakeholders based on a systematic assessment of Romania’s policies and institutions for TVET. In this respect, an evidence-based tool was introduced to the MoNE and applied - the Systems Approach for Better Education Results -Workforce Development (SABER WfD). System-level data on existing policies and practices (as reflected in laws, policy and operational documents, and stakeholder interviews), were collected and analyzed across policy goals. As a result, MoNE currently has a comprehensive diagnostic, including findings and detailed recommendations for advancing the WfD agenda in Romania. • The SABER Teachers Study delivered on December 22, 2017. This study provides informed policy guidance based on an analysis of the information collected to assess the extent to which the teacher policies in Romania are aligned with policies shown by research evidence to have a positive effect on student achievement. The Bank introduced MoNE to the SABER–Teachers evidence-based tool and specific aspects related to the education system’s progress in achieving most important teacher policy goals such as attracting the best into teaching; preparing teachers with useful training and experience, matching teachers’ skills with students’ needs, leading teachers with strong principals, etc. As a result, the MoNE benefited from policy recommendations and implementation options to further improve Romania’s teacher policy framework. 27. RAS on Support to the Implementation of the Public Procurement Strategy • The RAS team continued to support the ANAP in the preparation of a Web-based Guide (second stage), which was made publicly available at www.achizitiipublice.gov.ro in December 2016. The second stage of the Web-based guide includes: 1. Content related to the open procedure including guidance, checklists, and templates for all phases of the procurement cycle, including further content related to the planning stage 2. An interactive tool and a forum with instructions on how to use the Web-based Guide 13 3. 4 sets of standard bidding documents (including conditions of contract) for: intellectual services, design services, execution of works and supplies 4. Technical requirements for purchasing the information technology hardware and software required to operate the Web-based Guide elaborated. • Three dissemination sessions delivered, with the scope to ensure a consistent approach by disseminating the new operational methodology among the main stakeholders of the public procurement system. 28. RAS on Establishment of a Framework for the Use of Derivatives and the Execution of Swap Transactions in Romania • The MoPF already took action to amend primary legislation as per the recommendations of the Legal Output of the RAS, by making changes to Government Decision Nr. 1470 of December 6, 2007 on approving the Methodological Norms for application of provisions in the Government Emergency Ordinance Nr. 64/2007 on Public Debt. • The World Bank RAS team has also worked closely in recent months with the MoPF and an outside MoPF vendor of the FTI Star Treasury Management system, which the MoPF had purchased a few years ago but made only limited use, to considerably advance MoPF staff use of the FTI Star’s functionalities for swap valuation and accounting purposes. • The MoPF established an inter-departmental working group comprising of staff designated to implement the MoPF’s program to use derivatives for hedging purposes, based on the guidance and recommendations of this RAS program. • Over 35 MoPF and other government agency staff have been extensively trained in the areas under the RAS components described earlier. 29. RAS on Informing Project Evaluation • A draft Guidebook regarding the analysis of marginalized communities aiming (i) to be a reference document for the evaluators regarding good practices in carrying out a needs assessment at the level of marginal Roma and non-Roma communities (also including a minimal set of criteria based on which the evaluators can evaluate the analyses); and (ii) to support future project applicants in carrying out analyses of marginalized communities. • Two training sessions of evaluators for specific objectives 4.1. and 4.2., Axis 4, under the Human Capital Operational Program to evaluators aiming to increase their capacity in evaluating analyses of marginalized communities. Technical support/help desk to evaluators during the period of the contract for the evaluation of the projects that requested such analyses. • The WB team provided continuous support to the evaluators over the 6 months of contract in helping them understand how to assess from a technical point of view the proposals they received from applicants regarding the assessment of the marginalized communities. The RAS is an example of what can be achieved under extremely tight time constraints and with a low budget if the topic is at the core of what the Bank does and if a local team can be made available rapidly. 30. RAS on Technical Assistance to Enhance Quality Assurance in Higher Education System in Romania • The Draft Methodology and Indicators for Classification of Universities in Romania was delivered under the RAS Enhance Quality Assurance in Higher Education System in Romania. This Methodology in draft was published by the Ministry on their website for public consultations to receive feedback from universities. 31. RAS on Supporting the Implementation of Romania’s Human Development Operational Programme (POCU) 2014-2020 • As part of POCU RAS, the team has provided hands-on support to the client (POCU Managing Authority) in the process of evaluating local development strategies submitted for CLLD funding by 14 Romanian municipalities. This activity has directly contributed to the mobilization of funds more than EUR 200 million. It is important to note that as per EU regulation 1303/2013 these strategies had to be selected no later than December 31 of last year, which the client would not have been able to carry out without support from the RAS team. Lessons Learned 32. A combination of several factors including client commitment, frequent communication and the right staffing mix is key for a successful RAS. High-level (political) and technical level commitment by the client is necessary but not sufficient; it has to be complemented by frequent communication and the right staffing mix. Successful RAS agreements have a local TTL or senior specialist available on the team to facilitate collaboration and relevant international expertise bringing in best practice examples. The use of local experts helps to ensure that the diagnosis and examples are appropriate for the current local context. 33. Enhanced coordination with IFIs, counterparts and stakeholders is needed to avoid overlaps and to ensure complementarity of activities across IFIs. On RAS on Support to the Public Procurement Strategy the Bank’s team has worked closely with the EIB team providing advisory support to the National Public Procurement Agency under a separate agreement to ensure consistency and complementarity. Team members from both development partners met at least monthly in Bucharest and provided peer reviews of each other’s documents when useful. Similarly, the Bank has worked with the UNICEF on the deinstitutionalization of children. The Bank teams have also discussed reforms with other relevant partners, including unions, business associations, and stock market participants, as well as other stakeholders, including the Parliament and political parties, when appropriate. 34. RAS can be extended to the subnational level. Relative political stability at the local level has allowed the Bank to more effectively contribute to and pilot new approaches and resources for regional development. For example, the city of Constanta pursued its own funding sources to engage with the Bank on integrated urban development planning and inter-municipal coordination. 35. The demand for RAS deepened the policy dialogue and allowed for continuing the support in key reform areas. RAS allowed the Bank to respond to client requests for advice and support that could not be funded within the existing Bank’s budget. 36. Enhanced public access to knowledge generated under RAS benefits both its direct beneficiaries and the wider public. W h i l e according to signed RAS agreements, the Bank classifies all outputs “for official use only”, a wider dissemination of c e r t a i n reports and recommendations is coordinated with the beneficiaries. The Bank has worked with the relevant line ministries and bodies to obtain their permission to organize workshops to present the results of the advisory services. Broader access to this knowledge seems desirable. More focus on dissemination of knowledge and extracting relevant lessons from the analysis of past projects would be needed from both parties for going forward in this direction. 37. Institutional capacity building is a complex and lengthy process requiring long-term political commitment. As part o f the Bank’s policy dialogue and knowledge activities, RAS foster change of values, focus on results and modernization of management practices in the public administration. While responding to client demands, RAS also encourages the government to focus on long-term issues, sustain reforms and mitigate the backsliding. RAS is an integral element of the Bank’s country program, closely related to other activities including the lending and investment operations. 38. Weak institutional capacity and practices addressed by RAS are among the greatest bottlenecks for more efficient use of EU funds and faster growth. Removing these bottlenecks would lead to a faster convergence with the EU average living standards. 15 Going Forward 39. The Bank will start the evaluation of activities under the programming period 2014 – 2020 in 2018. The analysis contained in this report and the feedback from the client will contribute to the evaluation of RAS performance over the current period. 40. The dissemination of RAS results of the current and previous programming periods will continue in 2018. Such efforts will be reflected in the Bank’s Romania RAS website, which will continue to provide the publicly available information about the past and current RAS programs. 41. New RAS agreements are under preparation by ministries, national agencies, and municipal authorities. Several government institutions requested Bank’s assistance for the 2014-20 programing period, as presented in the Pipeline table. A list of activities has already been discussed with the government and it will be updated regularly. Several RAS agreements are expected to be signed in the next twelve months, depending on the timeline of the opening of new calls and the completion of financial applications by beneficiaries. 42. RAS have contributed to the Systematic Country Diagnostic (SCD) that is to be finalized in the first half of 2018 and it will take stock of key development challenges and inform the new Country Partnership Framework (CPF) for period 2019-2024. Advisory services remain the anchor of the Bank’s engagement in Romania along with investment operations and are expected to continue to feature in the new CPF. The RAS program completed and ongoing will help draw lessons and inform the RAS program and priorities going forward. 16 Annex 1. Summary Description of Each Ongoing RAS in 2017 Using European Funds ROMANIA: PROVIDING SUPPORT FOR THE ESTABLISHMENT OF A STRATEGY UNIT Signed: February 25, 2016 Key Partners: General Secretariat of the Government Duration: 25 months Value: RON 6.43 million (US 1.44 million) Project objectives: To strengthen Romania’s central strategic planning function. The RAS will help the Prime Minister’s office achieve this objective through the establishment of a Strategy Unit and a design of strategic planning and monitoring processes and tools. The project has four main components: (i) design of the National Strategy Management Framework and Procedures; (ii) support for the establishment of the Strategy Unit; (iii) methodology and procedures for preparing strategic documents; and (iv) monitoring and evaluation. ROMANIA: DEVELOPMENT OF THE CAPACITY OF THE CENTRAL PUBLIC ADMINISTRATION TO CARRY OUT IMPACT STUDIES Signed: March 9, 2016 Key Partners: General Secretariat of the Government Duration: 25 months Value: RON 12.67 million (EUR 2.84 million) Project objectives: To strengthen the capacity of the Romanian administration for evidence based policy making. The main types of activities to be undertaken in order to achieve the project goal are: impact assessment studies carried out in selected institutions, training modules, the set-up of a community of practice, and advocacy events. Through these activities, the proposed RAS will seek to achieve a set of outcomes, which will be tracked by a number of key indicators. The main goals of this project are to increase administrative capacity required to undertake an evidence –based policy making process, by enhancing relevant knowledge and skills, develop comprehensive impact studies following best practice methodologies, and raise awareness across the political and civil society spectrum regarding the need for evidence-based regulation. The project aims to: (a) create and expand technical capacities within the Romanian administration for the use of RIA by providing best practice examples of impact assessments for relevant legislation through the development of five impact studies in collaboration with selected institutions within the Romanian administration, emphasizing a learning-by-doing approach; (b) strengthen the technical capacities within the public administration through the provision of training, study tours, technical assistance, and knowledge exchanges; and (c) increase public awareness regarding the role and importance of RIA through advocacy and awareness events for universities, civil society, NGOs, journalists. 17 ROMANIA: ASSISTANCE TO THE MINISTRY OF EDUCATION AND SCIENTIFIC RESEARCH FOR INFORMED DECISION-MAKING ON INVESTMENT IN INFRASTRUCTURE Signed: March 30, 2016 Key Partners: National Centre for Technical and Duration: 36 months Vocational Education and Training Development Value: RON 7.8 million (EUR 1.75 million) Project objectives: To strengthen the strategic and monitoring capacity of the Ministry of Education and Scientific Research of Romania to make informed decisions on investments in education infrastructure. Specifically, this RAS will (i) support the preparation of a draft Strategy for Infrastructure Investments in Education Institutions, and (ii) build and enhance capacity within the Ministry of Education and Scientific Research for the preparation and monitoring of the National Strategy for Infrastructure Investments in Education Institutions. The Ministry of Education and Scientific Research will prepare the National Strategy for Infrastructure Investments in Education Institutions (NSIIEI) based on the draft Strategy for Infrastructure Investments in Education Institutions (SIIEI) to be developed under this RAS. The NSIIEI will be fundamental to inform decisions on investments in infrastructure, and to increase the absorption of European Union Structural and Investment Funds for the 2014-2020 programming period. ROMANIA: DEVELOPMENT OF PLANS FOR THE DE-INSTITUTIONALIZATION OF CHILDREN DEPRIVED OF PARENTAL CARE AND THEIR TRANSFER TO COMMUNITY-BASED CARE Signed: May 12, 2016 Key Partners: National Authority for the Protection Duration: 29 months of Children Rights and Adoption Value: RON 12.38 million (EUR 2.76 million) Project objectives: To: (a) develop a complex diagnostic of the centers for children deprived of parental care (residential institutions) and of the needs of their beneficiaries, to prioritize the closure of these centers; (b) carry out an inventory of the available alternative care modalities (e.g. foster parents, extended family) and preventive services, as well as the resources available at local, district and national level that could be mobilized to finance the deinstitutionalization process (including here the improvement of services aimed at preventing the separation of the child from his/her family); (c) develop a methodology for the closure of residential institutions for children deprived of parental care; and (d) develop a unitary system to identify and enroll the children and families at risk of separation, to diminish the inflows into the specialized child protection system. The project has four broad components: a) Prioritization of the process of closing down institutions for child protection, including classic residential units as well as modulated residential units; b) Routes of alternative care for children from classic residential units; c) Methodology for closing down institutions for child protection; and d) Monitoring and evaluation plan of the deinstitutionalization process together with a procedure of identification, registration, referral and support of children at risk of being separated from their family. ROMANIA: ASSISTANCE ON STRENGTHENING PLANNING AND BUDGETING CAPACITY AND SUPPORTING THE INTRODUCTION OF PERFORMANCE BUDGETING Signed: June 8, 2016 Key Partners: General Secretariat of the Government Duration: 35 months Value: RON 14.2 million (EUR 3.15 million) Project objectives: To strengthen the strategic and monitoring processes in the Center of Government (CoG), Ministry of Public Finance (MoPF) and line ministries with the purpose of enhancing the efficiency of public spending. Specifically, the assistance will focus on a) further strengthening the existing Institutional Strategic Plan (ISP) development processes of the MoPF and line ministries under the guidance of the Chancellery of the Prime Minister (CPM); b) monitoring ISP updates and implementation with a common IT tool, developed with support from the Bank; and c) developing a central dashboard in the CPM for high level ISP priorities of the selected ministries. 18 ROMANIA: ASSISTANCE TO THE MINISTRY OF EDUCATION AND SCIENTIFIC RESEARCH FOR CAPACITY DEVELOPMENT FOR M&E IMPLEMENTATION OF EDUCATION STRATEGIES Signed: June 29, 2016 Key Partners: Ministry of National Education Duration: 36 months Value: RON 6.46 million (EUR 1.43 million) Project objectives: To build the capacity of the Ministry of Education and Scientific Research to (i) monitor and evaluate the implementation of strategies for reducing early school leaving, and improving tertiary education, life-long learning and vocational education and training, and (ii) apply an evidence-based tool--the Systems Approach for Better Education Results (SABER) --to key policy areas with the purpose of fostering a structured policy dialogue with decision makers and stakeholders. The MESR will undertake a M&E process of the implementation of four complex education strategies - three of them (ESL, TE, and LLL) developed with Bank assistance under previous RASs and formally approved by the Government in June/July 2015, and one (VET) prepared by the MESR separately and currently processed for final approval. While all these strategies represent ex-ante conditionalities for Romania’s access to European Structural and Investment Funds (ESIF), resources for their implementation will also incur national budgets as well as other sources of funds. The M&E is expected to be a process of monitoring, learning and adjusting, as necessary, while leading to the achievement of the objectives of the strategies and EU2020 targets. At the same time, the MESR will seek to evaluate and improve key policies against evidence based global standards. This RAS will assist the MESR in the above mentioned process through capacity building activities. The capacity building process is expected to foster a broader policy dialogue with decision makers and stakeholders on how to strengthen the effectiveness of the education system. ROMANIA: SUPPORT TO THE IMPLEMENTATION OF THE PUBLIC PROCUREMENT STRATEGY Signed: September 1, 2016 Key Partners: National Agency for Public Duration: 35 months Procurement Value: EUR2.38 million Project objectives: To support the Romanian Ministry of Public Finance and its subordinated National Public Procurement Agency to reform the public procurement system in line with the PP strategy by transforming the procurement policy and practice into a strategic management function with focus on value for money and integrity in the whole project cycle. ROMANIA: ASSISTANCE ON INFORMING PROJECT EVALUATION Signed: September 9, 2016 Key Partners: Ministry of European Funds Duration: 8 months Value: US$0.05 million (EUR 0.04 million) Project objectives: To provide: (a) support to evaluators of marginalized community analyses under the Operational Program Human Capital by developing a good practices guide in carrying out needs’ assessment in marginalized communities (Roma and non-Roma communities); (b) training to evaluators to increase technical capacity; and (c) support to evaluators on technical aspects of methodologies and analyses, as well as input to requests for clarifications. ROMANIA: ESTABLISHMENT OF A FRAMEWORK FOR THE USE OF DERIVATIVES AND THE EXECUTION OF SWAP TRANSACTIONS Signed: September 30, 2016 Key Partners: Ministry of Public Finance Duration: 18 months Value: RON 2.6 million (EUR 0.58 million) Project objectives: Help the Ministry of Public Finance (MoPF) develop legal and operational framework (policies, procedures, norms) and provide the necessary training for MoPF staff to use interest rate and cross currency swaps as part of the funding program. 19 ROMANIA: SUPPORTING THE IMPLEMENTATION OF ROMANIA’s HUMAN DEVELOPMENT OPERATIONAL PROGRAMME (POCU) 2014-2020 Signed: October 12, 2017 Key Partners: Ministry of Regional Development, Duration: 38 months Public Administration and European Funds Value: EUR 10 million (US$10.71 million) Project objectives: To enable the Government of Romania to make more effective and efficient use of resources from the Human Development Operational Programme (POCU). This will be achieved by supporting (i) streamlining the implementation process of POCU within the managing authority and intermediary bodies, (ii) improvement of the design of POCU calls and (iii) improvements in the appraisal and evaluation of calls through clarifying roles and responsibilities, and introducing adequate controls and accountability mechanisms. ROMANIA: TECHNICAL ASSISTANCE TO ENHANCE QUALITY ASSURANCE IN HIGHER EDUCATION SYSTEM IN ROMANIA Signed: July 24, 2017 Key Partners: Romanian Agency for Quality Duration: 37 months Assurance in Higher Education Value: RON 6.48 million (US$1.64 million) Project objectives: To improve the capacity of the Ministry of National Education and the Romanian Agency for Quality Assurance in Higher Education (RAQAHE/ARACIS) to govern the quality assurance system of higher education in Romania with the purpose of supporting increased attainment to and quality and efficiency of Romania’s higher education system. This Advisory Services is designed to support the improvement of the administrative capacity of the MESR and the RAQAHE by developing procedures for the organization of its internal and external activities, by training its own staff, and also by developing and implementing consultation mechanisms to ensure that all stakeholders are participating in the decision-making process and continuing to develop efficient, citizen-oriented public policies. 20 21