85009 v2 IDENTIFICATION OF PROJECT SELECTION MODELS FOR THE REGIONAL OPERATIONAL PROGRAMME 2014-2020 The findings, interpretations, and conclusions expressed in this report do not necessarily reflect the views and position of the Executive Directors of the World Bank, the European Union, or the Government of Romania. Project co-financed from the European Regional Development Fund through the Regional Operational Programme 2007-2013. February 20, 2014 Introduction..................................... 7 6 An optimal Project Selection Model should also have a good Monitoring and Evaluation (M&E) Background......................................................... 9 system to facilitate continuous improvement.......... 37 Lessons from other Member States................................... 20 7 While keeping in mind the need to ensure a stable and predictable playing field for all applicants and Recommendations beneficiaries, the ROP 2014-2020 should feature for improving the ROP Project Selection Model 25 flexible approaches, as needed.............................. 39 1 Do not sacrifice ABSORPTION for the sake of IMPACT.................................................................. 27 Proposals for Project Selection Models............................. 41 2 There should be a purposeful focus on a number of strategic projects, with an appropriate allocation for Scenario 1 (“Status Quo+”)......................... 44 such types of investments...................................... 29 Scenario 2 (“Semi-Competitive”)................. 45 Scenario 3 (“Competitive”).......................... 47 3 Key strategic projects should be drawn from 31 integrated development strategies.......................... 4 The ROP 2014-2020 programmatic document can play a strategic role in steering beneficiaries 33 toward a set of core types of investments............... 5 Principles of fiscal prudence should be applied for the ROP 2014-2020, ensuring that beneficiaries do not overburden their financial capacity to support projects..................................................... 35 Table of contents 5 Romania has the opportunity to benefit from significant EU structural assistance through the Regional Operational Programme (ROP). In preparation for the 2014 – 2020 ROP, a key task at hand is to enhance not only the absorption rate, but also the impact of ROP-funded investments. One critical component in this process is improved project selection models to help boost the quality of ROP investments, have a stronger linkage to strategic development priorities, and focus on getting the highest “value for money” projects. The proposed selection models for the ROP 2014-2020 should build upon the strengths and correct the weaknesses of the 2007-2013 framework, as well as incorporate best practices from other EU countries, appropriately tailored to the Romanian context and capacity. 7 Background 9 The Regional Operational Programme (ROP) is one of seven operational programs (OPs) financed through EU funds during the 2007-2013 programming period. The EU dedicated EUR 3.7 billion to the ROP to support Romania’s regional development in the post-accession period. This amounts to 19% of the total structural funds available through the National Strategic Reference Framework (NSRF) between 2007 and 2013, making the ROP the third largest program after transportation (23.7%) and environment (23.5%). In 2011, projects financed under the ROP accounted for nearly 27% of projects financed by external grants and more than 12% of total public investment. The total allocation of the 2007-2013 ROP is more than EUR 4.5 billion, of which the EU contributes nearly 82%. Background 11 Relative performance of EU structural programs in Romania (as of November 30, 2013) It is important to note from the 120% 118% start that the ROP has been 100% 107% 103% a top performer for Romania 80% 78% 85% 82% 80% in terms of absorption rates, 60% despite a complex and 40% 48% 44% 45% 40% 46% heterogeneous portfolio of projects. 33% 30% 27% 22% 22% 19%18% 21% 20% 18% Because of its broad focus on regional development issues, the ROP 0% addresses a wide-ranging set of needs at the level of each region, from ROP Environment Transport Competitiveness Administrative Human Technical capacity resources assistance urban development to transport infrastructure, social infrastructure, % contracted of total ROP funds available % paid to beneficiaries % paid by EC to Romania business development, and tourism. In this context, the ROP has achieved the best absorption rates out of all structural programs (as of November 30, Source: Ministry of European Funds (www.fonduri-ue.ro) 2013), particularly when it comes to payments from the EC to Romania, which stand at 44% of the total available ROP allocation. Background 13 Performance of different OPs at the end of 2012 The ROP has also performed Operational Programme Commitments by end of 2012 Absorption by end of 2012 relatively well when compared to similar Operational Programmes from other ROMANIA 96% 34.5% Regional Operational Programme GERMANY Nordrhein-Westfalen ROP 95.1% 47.3% Member Countries. As the table below indicates, although absorption rates were relatively lower ESTONIA 87.7% 54.3% compared to the other OPs at the end of 2013, the Romanian ROP had OP for the Development of the Living Environment one of the highest commitment rates. This indicates that the ROP system functions relatively well up to the project implementation stage, where most SLOVENIA 86.6% 69.5% delays in absorption are caused by systemic issues (e.g., an inefficient Strengthening the Regional Development Potential OP public procurement system, poor quality technical documentation, property rights issues, etc.). UK 103% 34% West Wales and the Valleys Convergence Programme POLAND 88.4% 48.5% Slaskie Regional Operational Programme Background 15 That said, the commitment and absorption rates should be neither the only, nor the most important indicators for the ROP’s success. A high rate of absorption may be a sign that projects have been chosen for financing simply because they were ready to be implemented, rather than for their potential contribution to development objectives. In Romania and more broadly across the EU, policymakers and key stakeholders should pay increased attention to the impact of EU-funded investments in terms of achieving social and economic goals – which, after all, is the ultimate aim of such interventions. Background 17 Lessons from other Member States 19 One of the first steps in elaborating an improved selection model is to assess the approaches of other Member States. This way, best practices can be collected and improved upon to generate even better outcomes for the ROP. In the full report, project selection models from 8 Member Countries (Germany, Poland, Estonia, Italy, Slovenia, the UK, Ireland, and Lithuania) are analyzed to identify potential suggestions for enhancing the ROP selection model for the 2014-2020 programming period. The operational programmes analyzed were chosen based on their similarities to the Romanian ROP. Lessons from other Member States 21 A number of key lessons can be drawn from the analysis of these eight case studies: 1 Each Member State used unique approaches for project selection, with some focusing on the pre-selection of a number of large strategic projects (e.g., Lithuania), some focusing primarily on competitive calls (e.g., Ireland, Estonia), and yet others, particularly the larger countries (e.g., Germany, Italy, Poland), deploying a mix of methods. 2 The evaluation and selection systems used by these countries are by-and- large simpler than the system used under the ROP 2007-2013. For example, most countries did not use the Cost-Benefit Analysis (CBA) for evaluating projects. Germany was one of the exceptions, and it only used CBA for large strategic projects. Similarly, the ROP’s evaluation and selection grids appear much more granular and detailed than comparable frameworks in other Member States. 3 For the most part, the case studies analyzed revealed a keen focus on impact and also used weights for individual criteria to differentiate between more important and less important evaluation factors. Lessons from other Member States 23 Recommendations for improving the ROP Project Selection Model 25 The ROP should balance absorption and impact in a legitimate and The improvement of the ROP 2007-2013 Project Selection Model, through a functional selection system stronger focus on impact, can be achieved in a number of ways, keeping the following recommendations in mind: 1 Absorption Impact Appropriate focus for 2007- Do not sacrifice ABSORPTION for the sake of IMPACT. Higher priority for 2014-2020, 2013 to reduce the risk of funds after 2007-2013 learning curve decommitment Impact and value for money as In hindsight, optimal decision to key objectives prioritize quantity and speed Having projects that generate development impact is the main underlying aim of the ROP. Projects that lack actual impact are equivalent to a waste of resources and may in fact lead to high operation and maintenance In costs that will pose a burden on limited local resources. However, having addition to a few impactful projects and low absorption rates is not an optimal absorption and impact, an optimal situation either. selection framework also depends on: Legitimacy: Romania showcases the poorest absorption rates of any Member State, How credible is the system? and as an average across all its Operational Programmes. In fact, Romania Capacity: is some distance away (14 percentage points) from the second worst How functional is the system? performer – Bulgaria. This is obviously not an ideal situation. Low absorption means that substantial and critical resources are not Average absorption rates by Member States, in June 2013 used in a timely, effective manner, which also implies that resources 80.00% are wasted. In some ways, this may be a worse situation than having 70.00% an insufficient focus on impact. For example, even if the funds are not 70.95% 70.83% 68.81% spent on the newest, most impactful projects, but on the rehabilitation of 67.56% 60.00% 62.58% 60.82% 59.14% 58.82% 58.65% 58.16% 57.45% existing infrastructure (to a large extent, this is what ROP funds have been 56.73% 55.11% 50.00% 54.32% 53.67% 53.34% 52.97% 52.41% 49.95% 49.22% 47.71% used for and will continue to finance in 2014-2020), this may decrease 45.51% 40.00% 41.95% 40.86% 40.27% 40.00% 30.00% operation and maintenance costs for current infrastructure. Thus, even 26.20% 20.00% though the funds may not generate a significant development impact, 10.00% 0.00% they will help reduce costs and improve the quality of life for people – an important achievement in of itself. Estonia Lithuania Portugal Ireland Sweden Germany Poland Spain Latvia Finland Austria Greece Belgium The Nederlands Luxembourg United Kingdom France Slovenia Denmark Cyprus Hungary Slovakia Czech Republic Malta Italy Bulgaria Romania Source: http://ec.europa.eu/regional_policy/thefunds/funding/index_en.cfm Recommendations for improving the ROP Project Selection Model 27 2 There should be a purposeful focus on a number The ROP 2007-2013 primarily focused on small and medium-sized projects (situation as of August 2013) of strategic projects, with an appropriate allocation for such types of investments. Number of Average value Total value % of total projects of project of projects contracted Currently, the ROP primarily focuses on small and medium-sized projects, (in RON) (in RON) projects with a total share of commitments for large projects of less than 4% of available funds. Of course, the fact that a project is large does not make Small Projects 1,952 841,553 1,642,712,364 5.74% it necessarily impactful. Also, there are very small projects that can have (<1.5 mln RON) very large impact (e.g., well-targeted information campaigns). However, when it comes to infrastructure development, it is usually the larger Medium-sized Projects 1,554 16,750,343 25,923,350,459 90.51% projects that generate the most significant outcomes. (1.5 - 100 mln RON) The longer preparation time that large projects entail can be countered Lower-middle-sized projects (1.5 - 30 mln RON) 1,279 8,017,773 10,254,732,124 35.8% by simplified application and selection procedures for small projects. For example, the ROP 2007-2013 processed over 1,800 small projects, which underwent similar evaluation and selection procedures as larger projects, Upper-middle-sized projects (30 - 100 mln RON) 275 56,976,793 15,668,618,334 54.71% but represented only 5.6% of allocated funds. Simply put, evaluation and selection procedures should reflect the value, complexity, and expected risks/costs/benefits of a particular investment. This should ensure that Large Projects (>100 mln RON) 9 119,471,996 1,075,247,961 3.75% the overall evaluation and selection system remains functional and has enough capacity to accommodate peak-time demand. TOTAL 3,515 8,148,310 28,641,310,783 100% Source: MRDPA Recommendations for improving the ROP Project Selection Model 29 3 Sub-national entities have limited budgets for capital expenditures and they should spend these budgets wisely Key strategic projects should be drawn from integrated development strategies. Prudent Margin for Capital Expenditures Such strategies should represent the overall developmental vision for the btw. 2014-2022 area covered and should not be drafted for the sole purpose of attracting 35,442,983 - 50,000,000 EU funds, but present needs, priorities, and projects regardless of the ultimate financing source. New strategies should only be drafted when 50,000,001 - 100,000,000 addressing an area that is not already covered by an existing strategy (e.g., new metropolitan areas) and reflect, if available, the development 100,000,001 - 150,000,000 vision set in the strategies of constituent administrative units. Where . 150,000,001 - 200,000,000 integrated development strategies already exist (as is the case of Growth Poles and Urban Development Poles), they should only be updated. 200,000,001 - 250,000,000 To ensure consistency in approaches across development strategies, 250,000,001 - 300,000,000 a number of key items should be required (e.g., to have a clear vision and objectives, a list of projects to help achieve those objectives, and 300,000,001 - 324,121,969 corresponding funding sources). Also, to ensure a proper prioritization of projects, tools such as Capital Investment Planning (CIP) could be used to ensure that limited budget funds are targeted at key strategic projects and are not spent in an opportunistic manner, while ongoing operations Prudent Margin for Capital Expenditures btw. 2014-2022 and maintenance (O&M) costs remain reasonable. Integrated approaches also entail that EU and state-budget-funded 0 - 50,000,000 projects are not treated differently or separately, but that they are part of a single, coherent development vision and help achieve the same overall 50,000,001 - 100,000,000 goals. This means that similar evaluation and selection criteria and procedures should apply to such investments, regardless of the funding 100,000,001 - 143,852,516 source (i.e., EU funds or state-budget financing). Recommendations for improving the ROP Project Selection Model 31 4 The ROP 2014-2020 programmatic document Key investment priorities for Romania (as identified in the Competitive Cities report) can play a strategic role in steering beneficiaries toward a set of core types of investments. While local and county authorities define priorities on their own, as they know well the needs of the communities they represent, the ROP can Shorten the distance to large markets globally by improving help guide them toward the type of investments that are likely to have the infrastructure and encouraging cross-border flows of people, capital, and ideas highest impact. The Competitive Cities report (World Bank, 2013) makes a number of recommendations in this respect, taking Romania’s current development level into consideration. It is also important to keep in mind the main policy aims and thematic Improve connections between leading and lagging areas within Romania objectives set in the Europe 2020 Strategy. This is a key document that to enable efficient concentration of resources and spillover effects sets the priorities for the EU’s development as a whole, but obviously the needs of individual Member States may vary considerably. As such, priorities should be adapted to the Romanian context, with an enhanced focus on those objectives that are likely to be more relevant for Romania. Improve connective infrastructure between cities & Foster good institutions surrounding (basic services infrastructure, areas education, health, land markets, etc.) Promote quality-of-life investments Design and implement targeted measures for marginalized groups Recommendations for improving the ROP Project Selection Model 33 Some sub-national entities have spent more on capital investments than is sustainable in the long term, as indicated by the Glenday and Giosan 5 Indicators Glenday Indicator for county residences Glenday Indicator for county councils Principles of fiscal prudence should be applied for the ROP 2014-2020, ensuring that beneficiaries do not overburden their financial capacity to support projects. For example, beneficiaries should ensure that, once they make capital investments, they also have the necessary budgetary resources to cover the associated operation and maintenance costs. The fiscal health of individual beneficiaries can be assessed with the help of the Glenday Glenday Indicator 14% - 20% Glenday Indicator 21% - 30% Indicator and/or the Giosan Indicator. These two indicators are explained (Average for 2010-2012) (Average for 2010-2012) in more detail in the full report. 21% - 40% 31% - 40% 41% - 50% 41% - 50% For large strategic projects, which also require substantial co-financing from the state budget, it is important to have the Ministry of Public 51% - 71% 51% - 101% Finance (MoPF) involved, to ensure that funds are spent in an efficient and sustainable manner. The MoPF can ensure that basic fiscal prudence Giosan Indicator for county residences Giosan Indicator for county councils principles are followed and it could also set up a system for prioritizing and coordinating the development of large projects at the national level. Giosan Indicator 17% Giosan Indicator -2% - 15% (Average for 2010-2012) (Average for 2010-2012) 18% - 20% 16% - 20% 21% - 30% 21% - 30% 31% - 58% 31% - 64% Note: Territorial Administrative Units with a Glenday indicator above 50%, or a Giosan Indicator Recommendations below 15%, may have problems covering operations and maintenance costs for infrastructure investments they are making now. for improving the ROP Project Selection Model 35 6 An optimal Project Selection Model should also A proper M&E system with optimal feedback loops have a good Monitoring and Evaluation (M&E) Needs / Problems socio economic environment Impacts system to facilitate continuous improvement. Outcomes / Results A good M&E system serves several purposes. For one, it helps track the performance of a specific project and the extent to which this project has reached its output indicators – this is a mandatory task that the European Commission requires of all Operational Programmes. However, in Programme addition to what the EC asks for, a good M&E system can help evaluate Objectives Inputs Outputs if projects, and the overall program that finances them, have achieved planned outcomes, and if they have had an actual development impact. In particular, it can provide information on whether the project selection model is appropriate and has delivered on its intended goals, making sure that the ROP finances investments that have the biggest impact – the same way a farmer selectively breeds crops using the strongest seeds from the last harvest. A comprehensive M&E framework can assess not only the degree to which impactful projects are selected and prioritized, but also whether the programme delivers optimal absorption rates, a Evaluation Efficiency functional system with appropriate capacity to process applications, and Relevance a high level of credibility among applicants and beneficiaries. Effectiveness Utility / Sustainability (Longer Terms) Source: European Commission. 2006. “Indicative Guidelines on Evaluation Method” Recommendations for improving the ROP Project Selection Model 37 7 While keeping in mind the need to ensure a stable and predictable playing field for all applicants and beneficiaries, the ROP 2014-2020 should feature flexible approaches, as needed. For example, even if most of the available funds will be pre-allocated (for strategic projects) or allocated using a First-In First-Out (FIFO) approach, the programme could adopt a competitive selection model for one priority axis or one key area of intervention (e.g., business infrastructure). For example, many Member Countries use competitive calls for private beneficiaries (e.g., SMEs) or for the development of business infrastructure. Another option is to enable the ROP’s shift to a competitive approach after a mid-term claw-back (e.g., if some beneficiaries have not used pre-allocated funds in due time). Recommendations for improving the ROP Project Selection Model 39 Proposals for Project Selection Models 41 With the above recommendations as key guidelines, three project selection models A framework for defining an optimal selection model for Romania’s ROP are proposed for the ROP 2014-2020. The development of the scenarios followed a framework for assessing various OPTIMAL PROJECT SELECTION MODEL options based on six fundamental requirements (efficiency, effectiveness, clarity, Evaluation > Selection > Contracting fairness, transparency, and feasibility) and four key objectives (absorption, impact, legitimacy, and capacity). Core objectives 1 2 3 4 ABSORPTION IMPACT LEGITIMACY Capacity The figure across conveys precisely this Does the model Does the model Is the model Is the model ensure a high optimize for impact, legitimate for all feasible given absorption value for money? stakeholders? current capacity? message, making the case that an optimal rate? A EFFICIENCY: Quick, smooth process that ensures an optimal absorption rate project selection model for the next ROP Fundamental Requirements B EFFECTIVENESS: Selected projects optimize for impact and value for money will need to accomplish all four targets. C CLARITY: Clear selection procedures and requirements minimize the room for interpretation Put differently, a model may be great at promoting quick absorption of EU funds, but it may fail to deliver value for money and resources could be wasted on D FAIRNESS: Fair selection criteria apply to all submissions low-impact interventions. Similarly, if a model is not legitimate (i.e., with clear and transparent evaluation criteria and fair selection procedures), not enough E TRANSPARENCY: Transparent procedures enable full visibility and accountability of all parties ROP applicants will demonstrate interest in the program, so there will be an inherent limitation on what the ROP can accomplish. Last but not least, any F FeasibilITY: The institutional system is able to properly implement the model proposed change needs to be more than just a theoretical framework that promises to deliver great results in the abstract; indeed, selection models have to work in practice, keeping in mind the system’s current and expected capacity constraints. Proposals for Project Selection Models 43 Three scenarios are worth considering in terms of the overall project selection model: Scenario 1 Scenario 2 The main characteristics of the system This variation applies different selection remain unchanged, with a few tweaks (e.g., methods based on project size and beneficiary excluding the full Cost-Benefit Analysis from type. Project size tends to be a good proxy for (“Status Quo+”): the formal assessment, making the Detailed (“Semi-Competitive”) both complexity and potential impact. Under Technical Design optional at the evaluation this scenario, large regional projects (type phase, etc.) to correct current weaknesses. 1) would enjoy dedicated funding. Medium- This option capitalizes on the ROP system’s sized investments by counties, growth poles, accumulated experience and leverages and urban development poles (type 2) would the benefits of predictability. Essentially, be selected in two phases: (1) dedicated the two selection procedures remain the funding for a limited period of time; and (2) same: pre-allocation for growth poles and selection based on FIFO for the remaining urban development poles (with the potential pool of funding allocated to these types of inclusion of all county seats as UDPs); and beneficiaries. For other cities, as well as for first-in-first-out for all other projects. private and non-profit applicants (type 3), the FIFO rule would apply, regardless of whether the project is small (under 1.5 million RON) or more significant (over 1.5 million RON). Ultimately, this is a “semi-competitive” model, as a “middle-of-the-road” type of option, although applications are not compared or scored against each other under either FIFO or the dedicated funding model, regardless of the type of project under consideration. Proposals for Project Selection Models 45 The three scenarios considered present a range of different adjustments to the status quo model Scenario 3 This scenario maintains the FIFO selection rule for a limited set of projects, namely the (“Competitive”) smallest ones under 1.5 million RON, while Scenario Scenario 1 Scenario 2 Scenario 3 also setting aside dedicated funding for Project “Status Quo+” “Semi-competitive” “Competitive” the largest regional projects. For counties, Types growth poles, and urban development poles, Type the funding would be allocated in two phases, 1 Size: >100 million RON Beneficiary: Associations of Non-competitive, dedicated funding Non-competitive, dedicated funding much like for the previous scenario, with a Counties or Regions key difference: the phase 2 reallocation is done on the basis of competitive ranking for Type Minimal adjustments to Non-competitive: Semi-competitive: projects under each axis. 2 the status quo system, resulting in virtually no Dedicated funding Dedicated funding Size: >30 million RON differentiation in selection (phase 1) (phase 1) Counties, Growth Poles, model based on Competitive, ranking for Development Poles FIFO project size. each axis (phase 2) (phase 2) Type Selection model is: 3 Size: <100 mil. RON, >1.5 mil. RON) Dedicated funds for Growth Poles and FIFO Competitive, based on ranking for each axis Public authorities, private sector, Development Poles NGOs FIFO for all others The figure across summarizes the main features of the three options for each Type applicable type of project. As this report argues, choosing an optimal selection 4 Size: <1.5 million RON Public authorities, private sector, FIFO FIFO model is a challenging task and may involve several quintessential tradeoffs: NGOs between impact (effectiveness) and absorption (efficiency); between needed changes and predictability/continuity; and between ideal outcomes and practical feasibility given capacity constraints. Proposals for Project Selection Models 47 Simplified comparative assessment of proposed Project Selection Models Importantly, the current work does not Absorption Impact Legitimacy Capacity To select this option for propose selection models that vary by sector. Scenario ROP 2014-2020, you would (How quick?) (How effective?) (How credible?) (How feasible?) have to believe… Instead, the size of a proposed intervention is the key variable that determines how a project should be evaluated, selected, and contracted. This is in line with broader thinking on integrated interventions (something the ROP is uniquely 0 Scenario The current system is the best possible positioned to deliver on because of its focus on the broad field of regional The costs of any changes to the system are too high compared development) and a move away from sector-specific procedures. In fact, there to the potential benefits (i.e., the “Status Quo” system is extremely rigid) is value in putting forth standardized selection models and criteria that allow for easy comparisons across projects, regardless of their thematic focus. Regardless of the MA’s ultimate choices in Major changes to the system are 1 Scenario not advisable, predictability is valuable in and of itself terms of the 2014-2020 ROP’s selection Marginal improvements are needed and sufficient (e.g., dropping the CBA requirement, “Status Quo+” model and criteria, the fundamental aim adopting FIDIC, more rigorous evaluation of IDPs) FIFO is, ceteris paribus, preferable to a more competitive of the programme should remain front and center – to promote Romania’s sustainable 2 selection, even for larger Scenario projects and inclusive development. The balance between absorption and impact should still lean toward the former for 2014-2020 “Semi- The system (MA, IB, competitive” beneficiaries) has enough capacity to implement the In that endeavor, the importance of the ROP, both as an indispensable source of proposed changes funds and as a unique catalyzer of administrative capacity building, can hardly Competitive selection is be overstated. Some of this impact will continue to be very visible across the 3 generally preferable to FIFO Scenario country, changing people’s lives in very concrete terms: a new road; a better The balance between absorption and impact can lean more equipped, modern hospital; a proper school with comfortable temperatures heavily toward the latter The system (MA, IB, year-round; a new business center employing hundreds or thousands of people; “Competitive” beneficiaries) has enough capacity to implement the a renovated museum, etc. Other types of impact, including the motivation of proposed changes countless public servants, the shift in mentalities, and the improved capacity to deliver key services to citizens, may be harder to quantify, but no less important. Note: A larger black-fill in the pie-charts indicates the degree to which each individual scenario fulfils a specific criterion Ultimately, this is the power of EU convergence at its best, and Romania’s ROP will continue to play a key role in this historic effort. 49