WORLD BANK GROUP Annual Progress Report on the Reimbursable Advisory Services Program in Romania January 1 – December 31, 2015 Table of Contents Introduction ................................................................................................................................................................. 3 Background .................................................................................................................................................................. 3 Overview of the World Bank RAS Program in Romania .............................................................................................. 4 RAS Results .................................................................................................................................................................. 8 Coordination Arrangements ...................................................................................................................................... 11 Implementation Challenges....................................................................................................................................... 11 Lessons Learned ........................................................................................................................................................ 12 Going Forward ........................................................................................................................................................... 13 Tables......................................................................................................................................................................... 14 Table 1. Advisory Services Completed 2010-2015 .................................................................................................... 14 Table 2. RAS supporting the Government to prepare strategic documents to fulfill ex-ante conditionalities......... 18 Table 3. Proposed RAS Agreements in pipeline (under 2016 MoU) ......................................................................... 19 Abbreviations CPM Chancellery of the Prime Minister EBRD European Bank for Reconstruction and Development EC European Commission EIB European Investment Bank EU European Union DPL Development Policy Loan FRs Functional Reviews GoR Government of Romania IFI International Financial Institution IMF International Monetary Fund MA Managing Authority MAP Modernization of Administration Program MARD Ministry of Agriculture and Rural Development MEF Ministry of European Funds MEWF Ministry of Environment, Waters and Forests MESR Ministry of Education and Scientific Research MoU Memorandum of Understanding MoPF Ministry of Public Finance MLFSPEP Ministry of Labor, Family, Social Protection and Elderly Persons MRDPA Ministry of Regional Development and Public Administration MoT Ministry of Transport NACLR National Agency for Cadaster and Land Registration OP Operational Programme POAT Operational Programme for Technical Assistance 2007-13 PODCA Operational Programme for Administrative Capacity Development 2007-13 POR Regional Operational Programme 2007-13 POSCCE Operational Programme for Increased Economic Competitiveness 2007-13 POSDRU Operational Programme for Human Resources Development 2007-13 RAS Reimbursable Advisory Services RCC Romanian Competition Council SCF European Structural Funds and Cohesion Funds 2 Introduction 1. This is the third annual progress report which takes stock of the preparation and implementation of advisory services activities in 2015 (January 1 – December 31, 2015). As part of the reporting activities agreed with the Government under the Memorandum of Understanding (MoU) signed in 2012, every year the Bank prepares a progress report on the reimbursable advisory services (RAS) program. Since this is the last progress report to cover advisory services provided to the Government under the MoU signed in 2012 and paid under the 2007-13 EU programming period, the report provides also an overview of all activities implemented and completed until the end of December 2015. 2. This progress report is intended for concerned Government beneficiaries and managing authorities. This includes the Ministry of European Funds as coordinator of the Bank’s reimbursable advisory services activities and that of the international financial institutions advisory services. These services have been financed by the clients (i.e. various Ministries and public institutions) using funds from the European Structural and Cohesion Funds available under the 2007-13 Programming Period. The European Commission is also considered an important audience for this report, as well as other government institutions. 3. The Report includes an overview of the RAS program and its results and briefly touches upon some implementation challenges and lessons learned. The primary focus is on the outputs delivered, but there is also some discussion of outcomes based on the preliminary findings of an ongoing portfolio-level evaluation. Background 4. Since 2010, the Government of Romania and the World Bank have been strengthening their partnership to make Romania’s economic growth path more inclusive and sustainable. Key to this has been the continued and sustained effort to modernize Romania’s public administration, including through support for implementing EU Structural and Cohesion Funds. 5. Although Romania utilizes the full range of lending and technical assistance instruments availed to it as an upper-middle income World Bank member, reimbursable advisory services has emerged as one of the centerpieces of t h e Romania’s program. It has evolved into an important platform for broadening and deepening the reform dialogue, and become an essential complement to the Bank’s other program tools to transfer knowledge, build capacity, and achieve results. 6. Functional reviews (FR) were the first set of the Bank’s reimbursable advisory services (then called “fee- based services”) in Romania following both EU accession and the 2008 crisis, establishing an in- depth analytical foundation for implementing reforms across 13 ministries and agencies. As the functional reviews diagnosed the challenges to improved effectiveness and performance in the institutions and the sectors they covered, these reviews have been the key underpinnings of Government action plans for strengthening public administration capacity. 7. The Memorandum of Understanding (MoU) signed in January 2012 between the Government and Bank provided a framework agreement for the Bank’s provision of reimbursable advisory services. Since the signing of the MoU, the Bank’s RAS program has expanded beyond the eight original priority areas identified at the time1 and some areas of engagement, such as regional development, have sustained continuous, programmatic engagements that deepened with each successive agreement of advisory services. Since 2012, additional areas have been added in line with the evolving needs of the Government to prepare for the next financing cycle (i.e. areas of ex-ante conditionality). 1 These areas are: (i) environment; (ii) transport; (iii) policies for poor areas and less favored communities; (iv) adapting to climate change; (v) regional development; (vi) public administration reform including central/local government relations; (vii) support to coordination of actions at the central government level, and (viii) human resources development. 3 Overview of the World Bank RAS Program in Romania 8. Between 2010 and end December 2015, 42 advisory services agreements were signed and completed by the World Bank teams, for a total equivalent to €48.18 million. In particular, these can be grouped as: two agreements for conducting functional reviews of the public administration, 14 agreements for modernizing the public administration (MAP program) worth €12.75 million and 26 agreements to prepare for the 2014-20 programming period worth €32.43 million (MoU program). 9. In 2015, two new agreements were signed. As of January 1, 2015, the RAS portfolio included 19 agreements (16 under the MoU and 3 under MAP) valued at €31.99 million in the program. Two agreements were signed in 2015 for an amount of 0.87 million EUR, under the MoU: Assistance to the Muntenia Sud Development Agency for the Ploiesti growth pole and Support to the Implementation of the National Strategy for Digital Agenda. By the end of 2015, all agreements paid by the Government using EU funds ended, activities were completed and outputs delivered. 10. Support for modernizing the public administration counted 14 advisory service agreements out of which three were still in implementation in 2015. At the beginning of 2015, only three of these agreements were still in implementation (assistance to the Ministry of Education, to the Competition Council and to the Ministry of Transport). Additionally, one agreement on rail transport was under consideration in 2014 but it was dropped by the Ministry of Transport. The 14 agreements included activities for institutional capacity strengthening, strategic planning and internal management (improving human resources management, organizational structure, monitoring and delivery functions, IT systems, etc.) and implemented selected Government actions within ministries’ action plans, mostly derived from adopted functional review recommendations. The Government designed action plans to implement the priority recommendations from the functional reviews and mobilized Bank support for their execution. Table 1. Advisory Services for Modernizing Public Administration – status on December 31, 2015 Agreement Source of Status end finance Dec 2015 Public finance (debt management, tax administration) 1 Strengthening debt management PODCA Closed in 2014 2 Strengthening institutional capacity in fiscal policy formulation PODCA Closed in 2014 3 A better solution for providing clarifications, interpretations and ruling on tax PODCA Closed in 2014 issues 4 Improve human resources management instruments and mechanism for PODCA Closed in 2014 strengthening the institutional capacity of MoPF Agriculture and rural development 5 Strategic planning for the agricultural administration PODCA Closed in 2014 6 Implementation of an internal management system at MARD and its PODCA Closed in 2014 subordinated structures 7 Developing an integrated financial management system at MARD PODCA Closed in 2014 8 Agri-food sector strategy formulation PODCA Closed in 2014 4 Agreement Source of Status end finance Dec 2015 Education 9 Development of administrative capacity of the MESR PODCA Closed in 2015 Competition 10 Assistance to the Romanian Competition Council PODCA Closed in 2015 Transport 11 Strengthen strategic planning in the transport sector PODCA Closed in 2014 12 Improve capacity to prepare and manage Public Private Partnerships PODCA Closed in 2015 Center of Government 13 Assistance to the Center of the Government for policy planning PODCA Closed in 2011 and coordination Judicial system 14 Assistance to the Ministry of Justice to Assess the Performance of the Judicial PODCA Closed in 2013 System 11. Advisory Services to help Romania prepare for the 2014-20 Programming Period counted 26 agreements out of which 14 were in implementation at the beginning of 2015. In addition to these, two new agreements were signed in 2015 for an amount of EUR 0.87 million. By the end of December 2015, all 16 advisory service agreements (€27.36 million) ended. The advisory services contributed to medium-term strategy development, prioritization of government interventions and strengthening of decision-making processes. Table 2. RAS supporting the preparation of the 2014-20 period – status on December 31, 2015 Source of Status end Agreement finance Dec 2015 Enhancing government functions (prioritization of investment and management of resources) Improving the national framework for preparing and implementing public 1 POAT Closed in 2013 investment projects Assessment of the communication and collaboration between MAs and 2 Intermediate Bodies of the (ROP) and facilitation of proactive and direct POR Closed in 2013 support for beneficiaries 3 Analysis of capacity building activities in the public administration PODCA Closed in 2014 4 Technical assistance for the identification of project selection models POR Closed in 2014 Enhancement of Risk Based Systems of the Sectoral Operational Program for 5 POSDRU Closed in 2014 Human Resources Development 6 Strengthening regulatory impact assessment POAT Closed in 2015 7 Establishing a Delivery Unit POAT Closed in 2015 8 Performance appraisal system POAT Closed in 2015 9 Public investment management POAT Closed in 2015 Harmonizing State and EU funded Projects for the benefit of subnational 10 POAT Closed in 2015 governments Competitiveness 11 Competitiveness Enhancement and Smart Specialization Policies POAT Closed in 2013 5 Source of Status end Agreement finance Dec 2015 Regional development 12 Enhanced Spatial Planning as a Precondition for Urban Development POAT Closed in 2014 13 Upgrade growth poles strategic planning and economic impact POR Closed in 2013 14 Assistance to the Intercommunity Development Agency for ITI POAT Closed in 2015 15 Assistance to the Ploiesti Growth Pole POAT Closed in 2015 Sustainable development 16 Climate Change and Low Carbon Green Growth Program POAT Closed in 2015 17 Danube Delta Integrated Sustainable Development Strategy POAT Closed in 2015 Education 18 Reducing Early School Leaving POSDRU Closed in 2015 19 Preparing a Strategic Framework for Increasing Tertiary Education Attainment, POSDRU Closed in 2015 Quality and Efficiency 20 Preparing a Strategic framework for Lifelong Learning POSDRU Closed in 2015 Land Administration 21 Basis for National and EU Policies POAT Closed in 2015 Social inclusion 22 Elaboration of integration strategies for poor areas and disadvantaged POR Closed in 2014 communities 23 Study on Diagnostics and Policy Advice for Supporting Roma POSDRU Closed in 2014 Integration in Romania 24 National Strategy on Social Inclusion and Poverty Reduction POSDRU Closed in 2015 25 National Strategy for Elderly and Active Ageing POSDRU Closed in 2015 Social inclusion 26 Support to the Implementation of the National Strategy for Digital Agenda POSCCE Closed in 2015 12. The areas covered by all advisory activities evolved to meet Government needs. In addition to the areas identified as priority under the MoU, the Bank was subsequently asked to support strategic activities and deliverables required for Romania to meet the EU’s ex-ante conditionalities. In 2014, advisory services of high relevance were added to the program for: providing advice to a new Delivery Unit in the Prime Minister’s office, assisting with regulatory impact assessment, developing a performance management system, supporting the Government prepare for meeting ex-ante conditionalities (inputs to strategies for social inclusion and poverty reduction, and for active aging), strengthening public investment management and providing targeted advisory services to the Inter- community Development Association of Tulcea, for the implementation of the national digital agenda strategy and the development of local development plan for the Ploiesti Growth Pole. 13. The RAS agreements were paid by the client institutions using EU funds through various Operational Programmes (OPs) under the 2007-13 programming period, depending on the type of services provided. Activities implementing actions derived from the recommendations of the FR are paid using funds from the OP for Administrative Capacity Development (PODCA) 2007-13. Under the MoU program, most of activities (13) were financed under the Technical Assistance Operational Programme (POAT) which supports horizontal needs of the Government. Four regional development related activities were financed by the Regional OP 6 (POR). Seven activities that support diagnostics and analysis on Roma integration and financial risk management, as well as policy formulation and strategy development in education, social inclusion and active aging are covered under the Human Resources Development OP (POSDRU) and one from PODCA. One agreement was paid under the Increased Economic Competitiveness OP (POSCCE). Figure 3 provides an overview of the sources of funding used by the Government to pay for RAS activities under the 2007-13 period, RAS agreement amount vs. RAS billed amount. (as of March 31, 2016) 14. In 2012-2015, invoices issued by the Bank amounted to €43.38 million, out of which €18.16 million only in 2015. This includes all invoices issued under agreements covered by the MoU, as well as agreements implementing activities supporting public sector administration reforms (paid entirely from PODCA). The comparison between number of days worked and number of days billed through corresponding invoices and activities per MA are included in Figure 4. (as of March 31, 2016) 15. The Bank engaged with a broad range of public institutions . The Bank has provided assistance to 16 different line ministries and government agencies (12 public institutions had agreements signed under the 2012 MoU). The first RAS agreement under the MoU was signed with the Ministry of European Funds. The largest RAS program was with the Ministry of Regional Development and Public Administration (eight agreements), followed by the Ministry of Public Finance and the General Secretariat of the Government/Chancellery of the Prime Minister (five agreements). Ex-ante conditionalities for the Romanian government to gain access to EU 2014-20 programming period funds required selected ministries to develop and adopt national strategies. Through the RAS program, the Bank provided inputs to the Ministry of Labor, Family, Social Protection and Elderly and the Ministry of Education and Scientific Research. 7 Highlights of 2015 16. In 2015, the RAS program was at its peak in terms of volume and intensity of activities. Some agreements signed before 2015 required extensions to allow for more time for consultations and reviews or additional activities and funds have been allocated to complement tasks. Two new agreements were signed in 2015. All RAS activities signed under the MoU ended by December 2015 given the closing of the 2007-13 period. This required a tremendous effort from the Bank and coordination with counterparts to ensure timely feedback in order to be able to deliver on multiple tasks by the agreed deadlines such that the Government can meet the requirements agreed with the European Commission and related to the preparation for the 2014-20 period including ex-ante conditionalities. Often, such deadlines were very tight and required amendments to the agreements with the Government and agreed in consultation with the European Commission to allow for more realistic dates for the outputs without affecting quality standards. 17. Dissemination activities were intensified to allow for knowledge sharing within the Government and outside. Various outputs were produced under the RAS agreements and the valuable knowledge needs to be made available to relevant stakeholders. Under various RAS assignments, the Bank team jointly with the counterpart engaged in dissemination activities within the country, as well as with the European Commission to inform on progress made. Such dissemination activities will continue throughout 2016. 18. 2015 was a challenging and demanding year for the Government while handling activities related to two programming periods. The Government had to implement and monitor the remaining activities paid from the EU funds available under the 2007-13 period as well as to take the necessary steps to successfully close the programming period. In parallel, the Government handled the preparatory work and institutional reorganization with a view to accessing EU funds under the 2014-20 period. 19. Engagement with IFIs was reinforced through the preparation of new MoUs. The Ministry of European Funds led the preparation and discussions with each IFI (EIB, EBRD and the World Bank) to continue the engagement under the 2014-20 period and conclude a MoU with each institution. Whle all the discussions and preparatory work was carried out in 2015, the MoU between the Government and the World Bank was signed and became effective on January 11, 2016. 8 RAS Results 16. The World Bank responded to the substantial, time-sensitive demand for knowledge and advisory activities in Romania and delivered an impressive number of outputs. Throughout 2012 - 2015, the Bank has (i) delivered over 440 outputs, (ii) mobilized over 900 experts, including seasoned international specialist and local experts well familiar with the country’s specificities and context. By expanding the range of instruments to meet Romania’s specific needs, reimbursable advisory services enhanced the Bank’s role as reform advocate, neutral stakeholder and trusted adviser to the Government. Satisfaction with the quality of the Bank’s RAS work helped to strengthen the close collaboration with the European Commission and other international financial institutions. 17. Preliminary findings from a comprehensive Bank-led evaluation of the RAS program in Romania (2012- 2015) provide evidence of how the knowledge and advisory activities and outputs are contributing to the achievement of Romania’s development goals. Based on the methodology tested through four RAS case studies in 2015, the evaluation is systematically examining the extent to which the Bank’s transfer of knowledge to individuals and groups is helping to strengthen the local ownership of and commitment to development objectives, improve the policy environment, and increase the effectiveness of organizations. 18. RAS engagements are increasing the effectiveness of public agencies. Despite the time required for improving the performance of organizations, 23 of the 39 RAS engagements have documented preliminary evidence of increased organizational capacity. Some areas of improvement included clarifying the organization’s mission, roles, and responsibilities related to achieving a development objective (20); improving communication with stakeholders (10); increasing effectiveness as demonstrated by producing more or better outcomes (8); and improving adaptability and use of up-to-date information to adjust operations as needed (8). 19. RAS outcomes have contributed to better legislation, policies, and strategies that efficiently guide stakeholder actions to support Romania’s development agenda. Overall, 27 of the 39 RAS teams reported some evidence of how the RAS had helped to improve the policy environment as the Bank’s outputs were used to inform strategic documents, policymaking, and prioritization decisions prepared by the GoR. 9 Examples of these types of changes include making a policy instrument more consistent with other related documents in support of a development objective (13); improving the transparency of a policy or strategy (11); clarifying roles and responsibilities across organizations (9); improving a policy or strategy to reduce unintended negative effects (6); and approving a new strategy to satisfy ex-ante conditionality. 20. RAS activities have also helped to facilitate broad demand-side changes to promote the achievement of development goals. Evidence of such changes was reported by 24 of the 39 RAS teams. The most common improvements included increasing the participation of stakeholders in setting priorities (10); increasing the commitment of one or more key leaders to achieving a development objective (8); and shifting widespread beliefs or behaviors to be more supportive of the development objective (8). 21. Evidence of these three types of institutional capacity changes continues to emerge for the RAS agreements that completed implementation during 2015. The evaluation explored how changes in the knowledge, skills, relationships, and motivation of individuals and groups supported by RAS activities and outputs often represent key intermediate steps for addressing institutional capacity challenges. This diagnostic approach allows the Bank to understand better not only the immediate results of RAS engagements but also what ongoing efforts will likely be needed to sustain the capacity changes over the longer term. Selected examples from the recently-closed RAS illustrate the value of this approach. 22. The Bank’s support for the Ploiesti Growth Pole resulted in a new integrated urban development strategy that serves as an example for other urban areas in Romania. RAS activities brought diverse stakeholders together from the public and private sectors, academia, and civil society to determine an integrated package of priority projects through 2023. Several other cities have already started to use this strategy as a planning model: the Brasov Growth Pole, the Iasi Growth Pole, the Cluj Growth Pole, the Constanta Growth Pole, and the City of Targoviste. 23. Critical progress has been achieved for upgrading the current regulatory impact assessment system. Following the Bank’s diagnostic exercise and constructive dialogue with the Department for Coordination of Policies and Programs, the GoR has published the RAS reports and recommendations to generate a sound debate about streamlining the RIA system. RAS activities also built capacities within the Romanian administration by providing training, hands-on coaching, and pilot exercises. Intermediate outcomes from these advisory services reflect needed steps toward improving organizational and procedural arrangements to establish evidence-based decision-making aligned with international good practice. 24. RAS activities and outputs have provided needed frameworks and guidance for strengthening the effectiveness of skills and programs for education and labor market inclusion. Most notably, five projects completed during 2015 supported the approval and adoption of new national strategies for reducing early school leaving, for tertiary education, lifelong learning, poverty reduction and social inclusion and active aging. All five strategies contributed to satisfying ex ante conditionality for accessing ESIF under the 2014- 2020 programming period. 25. The Climate Change and Low Carbon Green Growth Program has contributed to important incremental improvements for fostering high-level commitment and adopting a coherent strategy and action plan with clearly defined roles and responsibilities across organizations. In order to enhance inter-ministerial collaboration, the Ministry of Environment, Waters, and Forests (MEWF) drew on RAS recommendations and outputs to reform the National Commission on Climate Change (NCCC) in the spring of 2015. The NCCC was restructured (by GD no. 1026/20.11.2014), to enhance its role and ensure a broader representation of ministries or agencies with competencies in climate change area. This is an important early step for implementing the climate change agenda to achieve development impact. 26. Leveraging its rich experience, the Bank has taken actions to increase its ability to respond to client intensive demand and ensure RAS complementarity and synergies. These include: (i) identifying a series of new international and local Bank experts to lead the RAS program in Romania; (ii) building on the knowledge of the local staff (all new and old staff in the World Bank office are involved in one or more RAS); (iii) increasing the number of international level Romanian staff and experts; (iv) strengthening the RAS 10 monitoring and coordination team in the Romania office and improving knowledge sharing and learning, and (v) revising and improving quality control mechanisms required by Bank management for the RAS, both at entry and during implementation. Coordination Arrangements 27. In 2012 the Ministry of European Affairs (currently, the Ministry of European Funds) put in place a technical level coordination mechanism to ensure coherence, avoid potential overlaps and address potential issues stemming from interventions under different advisory services as quickly as possible. This mechanism consists of monthly monitoring of the status of each advisory activity based on status updates provided by IFIs and concerned beneficiaries, as well as quarterly meetings with concerned stakeholders. The Coordination Committee for Technical Assistance reviews progress in the preparation and implementation of advisory services and follows up with concerned parties to address challenges and bottlenecks. The committee is led by the Directorate of System Coordination and Technical Assistance. A matrix is circulated and updated every month to keep track of the activities of each IFI, improve synergies and avoid overlaps. 28. The Bank has participated in all high level coordination meetings organized. In addition to the coordination at technical level, the government organizes biannual high level meetings with the participation of representatives of the managing authorities, beneficiary ministries or agencies, IFIs and the European Commission. High level meetings review the status of advisory services implementation, EU funds absorption and preparation for the next programming period, and address issues at management level. These meetings allowed for discussing and checking for potential overlaps, complementarity and synergies with various ongoing activities touching upon common topics or areas. These also encouraged coordination and knowledge exchange between institutions. In addition, the Bank participated in joint meetings with the Government and the European Commission with a view to inform and exchange views with various units of the Commission to ensure the Bank led advisory services and recommendations take into consideration the views and guidelines of the EC. 29. The RAS instrument has become an integral part of the Bank’s program in Romania. The Bank has included the RAS as part of its bi-annual, portfolio monitoring exercise that it regularly undertakes with its main counterparts. This process is led by the Ministry of Public Finance and by the Ministry of European Funds for the RAS. This is done to ensure a higher level of monitoring and quality control of the Bank’s own work and because RAS are considered an integral part of the Bank’s work in Romania. RAS have increased the Bank’s ability to reach out to a variety of clients and allowed for deeper policy and sector dialogue, in cooperation with the Bank’s other more traditional instruments. RAS have proved essential for addressing Romania’s challenges to achieve inclusive and sustainable growth. Implementation Challenges 30. The Bank’s experience under the RAS carried out in 2012-2015 can be considered very positive, especially given how new this instrument is for Romania and the speed at which the demand for service has grown in less than three years. The bulk of RAS deliverables have been submitted on a timely basis as per the schedule set in the advisory agreements, sometimes earlier than scheduled and other times according to a revised schedule agreed with the client institutions. The RAS are meeting overall quality standards and clients’ requirements. However, some challenges identified during implementation are worth highlighting. 31. The signature of some RAS suffered delays as new beneficiaries of assistance and stakeholders were less familiar with the legal, billing and pricing requirements of the RAS and wanted to engage each time in negotiations of the standard legal clauses of the agreements. In some cases, the dialogue has been very constructive with the beneficiary of the services. In other instances the challenge has come from the need to make sure the RAS agreement was in accordance with the financing contract between the beneficiary and the Managing Authority. The Bank has also needed time to understand and learn the Romanian and EU’s rules and regulations under which the MAs and beneficiaries operate. 11 32. The very tight calendar for the delivery of outputs and time sensitive preparation for drafting strategies and meeting ex-ante conditionalities has unduly compressed the deliverables timetable under several RAS. Consequently, several financing contracts between the beneficiary and managing authority had to be amended to extend the duration of delivery. However, the extensions of the closing date of some agreements allowed for some flexibility to ensure appropriate exchanges and consultations for relevant outputs. 33. In some cases, there has been the challenge of the World Bank mandate to ensure quality of its outputs and the client’s need to receive the outputs . Experience has shown that implementation schedules were not always realistically set (at the preparation stage) for the envisaged depth of analysis, complexity of tasks or sensitiveness of the covered topics, particularly for advisory activities of shorter duration (six months). Given the GoR’s high need to use remaining EU funds under the 2007-13 period and urgency to prepare for accessing funds under the next EU programming period, selected outputs have been required a compressed delivery timetable. 34. The choice was made to utilize only time-based agreements and embed the entire cost of the RAS in simple, lumped daily fees. While this form of agreement has worked well in some cases, in others it has created challenging situations. Daily rates were often misunderstood as average salary rates and hence appeared high (in fact, such rates embed travel and all other direct and related expenses). Also, this billing modality requires amendments for any minor change in the number of days per category of staff (international and local), even when there is no change in the output being delivered or the price of the agreement. To overcome the challenges with invoicing, the Bank will change the costing and invoicing methodology from time-based to output-based agreements under the new MoU. This type of agreements is used for most of the Bank’s RAS in other countries. 35. In some cases throughout implementation, clients’ payments to the Bank for issued invoices have been delayed. Such delays were caused by lengthy administrative and approval rules, special nature of fiscal treatment (VAT), applicable exchange rates of advanced payments, or the need to provide accurate payment details. Lessons Learned 36. The collaboration with the client and government counterparts has been one of the keys to success. Romanian public officials have generally extended timely collaboration and support to Bank advisory service teams, including but not limited to sharing available data, providing relevant documentation and background information, being available to meet frequently with local and international experts, organizing and making logistic arrangements for consultations and workshops, etc. Without their time and dedication, it would not have been possible for the Bank to deliver a RAS program of this size and depth. 37. Enhanced public access to knowledge generated under RAS would benefit both the direct beneficiaries and the overall public. The dissemination of reports and recommendations is coordinated with the beneficiaries. The Bank has worked with the relevant line ministries and bodies to organize workshops to present the results of the advisory services. According to signed RAS Agreements, the Bank classifies all outputs “for official use only”. Public access requires client’s approval of content and format. Broader access to this knowledge seems to be desirable. More focus on dissemination of knowledge and extracting relevant lessons learned from the analysis would be needed from both parties going forward. 38. In 2015, the RAS program was at its peak in terms of volume and intensity of activities in implementation and brought in additional challenges. The Government had to implement and monitor remaining activities paid from the EU funds available under the 2007-13 period in order to successfully close the programming period. Two new agreements were signed in 2015. Some agreements signed before 2015 required extensions to allow for more time for consultations and reviews or additional activities and funds have been allocated to complement tasks. All RAS activities signed under the MoU ended by December 2015 given the closing of the 2007-13 period. In parallel, the Government handled the preparatory work and institutional reorganization with a view to accessing EU funds under the 2014-20 period. 12 39. RAS activities became larger and more complex in recent years, evolving from short implementation duration of about six months with one major output on a single sector or thematic area to considerably longer, more complex, cross-sectoral activities spanning over two years. Such complex activities required mobilization of very large teams, close coordination and collaboration among experts from different sectors within the Bank and across ministries. This trend has proved to be good, allowing going beyond studies and reports, and permitting to include larger TA and capacity building components into the RAS. A strong and stable counterpart team, including from senior management level, is required to ensure that results of the RAS are properly steered, absorbed and disseminated. Frequent changes of Ministers and senior officials sometimes limit impact and delay decision making on funding or actual adoption of the new policies, strategies or processes developed under RAS. The Bank also suffered from its own internal procedure that made it sometimes difficult to move fast between RAS identification and having experts on the ground. Identifying appropriate teams and mix of skills and expertise needed on short notice proved challenging. 40. Advisory Services strengthened the Bank’s engagement with Romania. RAS allowed the Bank to respond to client requests for advice and support that could not be funded within the existing Bank’s budget. As demand for RAS grew in volume and complexity, this deepened the policy dialogue and engaged the Bank in key reform areas (i.e. public investment management, Roma inclusion, regional development, climate change, competitiveness, land registration, education). Weak institutional capacity and practices in these areas are among the greatest bottlenecks for more efficient use of EU funds and faster growth. Removing these bottlenecks would lead to convergence with EU average living standards. These would have a potential transformative impact. However, this has challenging for the Government to reform. 41. Institutional capacity building is a complex and lengthy process requiring long-term political commitment. The Bank’s knowledge program encourages the Government to focus on long-term issues, sustain reforms and mitigate the backsliding to improve the functions of state. RAS is an integral element of the Bank’s country program, closely related to other activities including the lending and investment operations. The Bank’s policy dialogue and knowledge activities foster change of values, focus on results and modernization of management practices in the public administration. Going Forward 42. An evaluation exercise of the advisory services provided by international financial institutions (IFI) in Romania was carried out by the Ministry of European Funds. This reviewed the assistance provided by the main IFIs (EIB, EBRD, World Bank) following the memoranda of understanding signed in 2012, drew lessons to inform the preparation of new advisory services. The findings and recommendations of this evaluation informed the preparation of the new MoU and provided guidance on how to improve the implementation arrangements of the new set of activities to be paid by clients using ESIF under the 2014-20 period. 43. A new MoU was signed with the Government of Romania on January 11, 2016 to cover for advisory services to be provided by the Bank and paid by the client institutions mainly through EU funds available under the 2014-2020 period. This will allow access to a larger pool of technical assistance funds, including the rural development funds. The new MoU is a flexible document, spelling out the strategic engagement of the cooperation in line with the Romania Partnership Agreement 2014-20 and the Government priorities. Output-based agreements will be used starting with this MoU, instead of time-based agreements. The new MoU builds on the lessons learnt and the experience accumulated in the past years. 44. New RAS agreements are in advanced stage of preparation. Several Government institutions requested Bank’s assistance under the 2014-20 period. A list of activities has already been discussed with the Government and this will be updated regularly based on demand (Table 3 in the Annex). Nine RAS agreements are expected to be signed in 2016, depending on the completion of procedural aspects related to beneficiary institutions securing the needed EU funds for paying for RAS activities. Internal procedures of evaluation and approval between the Managing Authorities and Beneficiary institutions applying for EU funding, as well as establishing mechanisms for payments are still being established by the relevant institutions to allow for smooth implementation of EU funds under the 2014-20 programming period. 13 Tables Table 1. Advisory Services Completed 2010-2015 Crt. No. Agreement Title Framework Agreement Source Agreement Agreement Agreement Client/Government of EU Sign Date End Date Amount Institution Funds (EUR mill) 1 Functional Review phase I - Agriculture FR General Secretariat PODCA 1-Mar-10 31-Mar-11 1.59 of the Government Functional Review phase I - Education Functional Review phase I - Transport Functional Review phase I - Finance Functional Review phase I - Competition Functional Review phase I - Center of Government 2 Functional Review phase II - Regional Development FR 26-Nov-10 25-Jun-11 1.59 Functional Review phase II - Labor and social Protection Functional Review phase II - Economy/Energy Functional Review phase II - Health Functional Review phase II - Environment and Forestry Functional Review phase II - Research and Development Functional Review phase II - Higher Education Functional Review phase II - Cross-Cutting Issues 3 FBS CoG Policy Planning and Coordination MAP General Secretariat PODCA 22-Jul-11 21-Oct-11 0.36 of the Government 4 Development of Administrative Capacity of the Ministry of 25-Oct-12 24-Sep-15 1.57 MoNE Education and Scientific Research 5 Strengthening Debt Management Ministry of Public 1-Oct-12 31-May-14 0.49 6 Improving the HRM Instruments and Mechanism to Finance 13-Dec-12 12-Apr-14 0.28 Strengthen the Institutional Capacity of MoPF 7 Strengthening Institutional Capacity in Fiscal Policy 14 23-May-13 22-Sep-14 0.32 Crt. No. Agreement Title Framework Agreement Source Agreement Agreement Agreement Client/Government of EU Sign Date End Date Amount Institution Funds (EUR mill) Formulation 8 A Better Solution for Providing Clarifications, 23-May-13 22-Dec-14 0.32 Interpretations and Ruling on Tax Issues 9 Strengthen Strategic Planning in the Transport Ministry of Transport 19-Mar-12 8-Dec-14 3.00 Sector 10 Improve its Capacity to Prepare and Manage Public 6-Sep-12 11-Dec-15 0.82 Private Partnerships 11 Assistance for Agri-food Sector Strategy Ministry of 28-Sep-12 27-Jun-14 0.97 Formulation Agriculture and Rural 12 Assistance for Strategic Planning for the Agricultural Development 28-Sep-12 27-Sep-14 0.89 Administration 13 Assistance for Implementation of an Internal 28-Sep-12 27-May-14 0.50 Management System at the MARD and its Subordinated Structures 14 Developing an Integrated Financial Management 28-Sep-12 27-Sep-14 0.26 System at the MARD 15 Assess the Performance of the Judicial System Ministry of Justice 17-Feb-12 1-Mar-13 0.73 16 Assistance to the Romanian Competition Council Competition Council 4-Jun-12 3-May-15 2.24 17 Romania Enhanced Spatial Planning MoU Ministry of Regional POAT 26-Sep-12 25-Jan-14 0.57 18 Technical Assistance for the Identification of Project Development and POR 20-Nov-12 19-Feb-14 0.59 Selection Models Public 19 Upgrade Growth Poles Strategic Planning and Administration 20-Nov-12 19-Nov-13 0.69 Economic Impact 20 Elaboration of Integration Strategies for Poor Areas 20-Nov-12 19-Apr-14 0.45 and Disadvantaged Communities 21 Assessment of the Communication and 20-Nov-12 19-Dec-13 0.38 Collaboration between MAs and IBs of the ROP and Facilitation of Proactive and Direct Support for Beneficiaries 22 Analysis of Capacity Building Activities in the Public PODCA 10-May-13 9-Jan-14 0.29 Administration 23 Danube Delta Integrated Sustainable Development 15 POAT 4-Sep-13 4-Nov-15 2.35 Crt. No. Agreement Title Framework Agreement Source Agreement Agreement Agreement Client/Government of EU Sign Date End Date Amount Institution Funds (EUR mill) Strategy 24 Harmonizing State and EU funded Projects in 27-May-14 26-Nov-15 2.50 Regions 25 Providing Support to the Establishment of a Chancellery of the 31-Jan-14 4-Dec-15 4.68 Delivery Unit Prime Minister 26 Strengthening the Regulatory Impact Assessment 24-Apr-14 15-Nov-15 1.46 Framework in Romania 27 Reducing Early School Leaving Ministry of POSDRU 12-Jun-13 11-Oct-15 1.49 28 Preparing a Strategic Framework for Increasing Education and 17-Jul-13 16-Oct-15 1.45 Tertiary Education Attainment, Quality & Efficiency Scientific Research 29 Preparing a Strategic Framework for Lifelong 16-Sep-13 15-Jan-15 0.84 Learning 30 Study on Diagnostics and Policy Advice for Ministry of Labor, 17-Jun-13 16-Apr-14 0.41 Supporting Roma Integration in Romania Family, Social 31 Enhancement of Risk Based Systems of the Sectoral Protection and 28-Jun-13 27-May-14 0.15 Operational Program for Human Resources Elderly Development 32 Preparation of a Draft National Strategy and Action 26-Feb-14 30-Nov-15 1.36 Plan on Social Inclusion and Poverty Reduction 33 Preparation of a Draft National Strategy Regarding 26-Feb-14 25-Feb-15 0.95 Elderly and Active Ageing 34 Support to Strengthening Public Investment Ministry of Public POAT 16-Sep-14 15-Dec-15 1.19 Management Finance 35 Improving the National Framework for Preparing Ministry of European 27-Jun-12 27-Apr-13 0.78 and Implementing Public Investment Projects Funds 36 Establishment of a Performance Appraisal System 11-Jun-14 10-Jul-15 0.50 for Government Officials Managing European Union Funds 37 Climate Change and Low Carbon Green Growth Ministry of 23-Jul-13 30-Nov-15 4.71 Program Environment, Waters and Forestry 16 Crt. No. Agreement Title Framework Agreement Source Agreement Agreement Agreement Client/Government of EU Sign Date End Date Amount Institution Funds (EUR mill) 38 Real Estate - Basis for National and EU Policies National Agency for 3-Apr-13 30-Nov-15 2.63 Cadaster and Land Registration 39 Assistance to the Muntenia Sud Development Muntenia Sud 9-Jan-15 8-Dec-15 0.13 Agency for the Ploiesti growth pole Development Agency 40 Competitiveness Enhancement and Smart West Region 31-Aug-12 16-Aug-13 0.74 Specialization Policies in the West Region Development Agency 41 Assistance to Intercommunity Development Tulcea Country 23-Dec-14 22-Nov-15 0.39 Association ITI Danube Delta Council 42 Support to the Implementation of the National Ministry of POSCCE 2-Apr-15 1-Dec-15 0.74 Strategy for Digital Agenda Information Society Total completed 48.38 FR Functional Review MAP Modernization of Public Administration (supporting the implementation of FR actions) MoU Activities under Govt - WB Memorandum of Understanding 2012 POAT Operational Program for Technical Assistance 2007-2013 PODCA Operational Program for Administrative Capacity Development 2007-2013 POSDRU Operational Program for Human Resources Development 2007-2013 POR Regional Operational Program 2007-2013 17 Table 2. RAS supporting the Government to prepare strategic documents to fulfill ex-ante conditionalities Agreement Title Government Institution Gov Agreement Agreement Amount Status on end Source of Signing Date End Date EUR million December 2015 Preparation of a Draft National Strategy Min of Labor, Family, POSDRU EU Funds 26-Feb-2014 30-Nov-2015 € 1.36 Closed and Action Plan on Social Inclusion and Social Protection and Poverty Reduction Elderly Reducing Early School Leaving Min of Education and POSDRU 12-Jun-2013 11-Oct-2015 € 1.49 Closed Scientific Research Preparing a Strategic Framework for Min of Education and POSDRU 17-Jul-2013 16-Oct-2015 € 1.45 Closed Increasing Tertiary Education Attainment, Scientific Research Quality and Preparing EfficiencyFramework for a Strategic Min of Education and POSDRU 16-Sep-2013 15-Jan-2015 € 0.84 Closed Lifelong Learning Scientific Research Preparation of a Draft National Strategy Min of Labor, Family, POSDRU 26-Feb-2014 25-Feb-2015 € 0.95 Closed Regarding Elderly and Active Ageing Social Protection and Elderly 18 Table 3. Proposed RAS Agreements in pipeline (under 2016 MoU) Proposed RAS Engagement Client Institution Agreements expected to be signed in the first half of 2016 1 Assistance for Establishing a Strategy Unit General Secretariat of the Government 2 Development of the Capacity of Central Public Administration to Carry out Impact General Secretariat of the Government Studies 3 Assistance on Strategic Planning and Budgeting Capacity General Secretariat of the Government 4 Support to the Implementation of the Public Procurement Strategy National Agency for Public Procurement 5 Assistance on Informed Decision-Making on Investments in Infrastructure National Center for Technical and Vocational Education and Training Development 6 Assistance for Capacity Development for Monitoring and Evaluating the Implementation Ministry of National Education and Scientific Research of Education Strategies (ex-ante conditionalities) and for Improving Education Policies 7 Assistance to Enhance Quality Assurance in Higher Education System in Romania Ministry of National Education and Scientific Research 8 Establishment of a Framework for using Derivatives and Executing Swap Transactions Ministry of Public Finance 9 Development of Plans for Deinstitutionalization of Children Deprived of Parental Care National Authority for Children Rights Protection and and their Transfer to Community-based Care Adoption Potential RAS in early stage of discussion and for which the Bank received a request for assistance 1 Support for Enhancing the Delivery System Chancellery of the Prime Minister 2 Assistance to Strengthen the Management of Macroeconomic and Fiscal Risks in the Ministry of Public Finance Area of SOEs 3 Assistance on Corporate Governance, Reporting and Auditing Ministry of Public Finance 4 Assistance on Insolvency Ministry of Justice 5 Development of a Monitoring and Evaluation System for the Strategies Coordinated by Ministry of Labor, Family, Social Protection and Elderly the Ministry of Labor 6 Strengthening the Agriculture Food Chain Ministry of Agriculture and Rural Development 7 Low Carbon Growth Mitigation Strategy for the ARD Sector Ministry of Agriculture and Rural Development 8 Transforming Romania’s Competition Architecture Competition Council 9 Assistance to the Financial Surveillance Authority Financial Surveillance Authority 10 Assistance on Land Administration National Agency for Cadaster and Real Estate 11 Follow up on the implementation of the action plan derived from FR recommendations Ministry of Environment, Waters and Forests for the Ministry of Environment 19