Return on Investment of Public Support to SMEs and Innovation in Poland ROI / Effectiveness Analysis Technical Note September 2019 A technical assistance project funded by the European Commission and implemented by the World Bank Group, in collaboration with the Government of Poland. 0 ACKNOWLEDGEMENTS This Technical Note was produced by Shawn W. Tan, Trang Tran, and Maciej Koniewski, with research assistance and support from Damian Iwanowski, Jerzy Toborowicz, Bartłomiej Skowron, Sarah Hebous, Esther Bartl, Dea Tusha, Youpei Yan and Hoa Nguyen. Special thanks go to Statistics Poland who partnered with the World Bank Group (WBG) on the data preparation and econometric analysis, including Dominik Rozkrut, Dominika Rogalińska, Dominika Nowak, Jacek Kotowski, Jarosław Kalski, Joanna Sulik, and Rafał Trzciński. The project was financed by the Directorate-General for Employment, Social Affairs and Inclusion (DG Empl), of the European Commission (EC) and benefited from close collaboration with the EC’s Directorate- General for Regional and Urban Policy (DG Regio). Special thanks go to Judit Törökne Rozsa and Rafał Janas (DG Empl) for the financing, leadership, and guidance, as well as Peter Berkowitz, Marek Przeor, Christopher Todd, Wolfgang Munch, and Karolina Tilman (DG Regio) for their support and feedback. Overall, the project was implemented by the Finance, Competitiveness, and Innovation Global Practice of the WBG. The WBG team, led by Thomas Haven and Łukasz Marć, included Arti Grover, Juan Rogers, Damian Iwanowski, Jerzy Toborowicz (needs assessment, portfolio mapping, and functional analysis); and authors listed above for the ROI/effectiveness analysis. Peer review was provided by Natasha Kapil, Miriam Bruhn, and Leonardo Iacovone. The report was prepared under the leadership and guidance of Marialisa Motta and Carlos Piñerúa. The initial phase of the work benefited from the leadership, guidance, and contributions of Paulo Correa, Marina Wes, Christine Qiang, Roberto Echandi, Erik von Uexkull, and Isfandyar Khan. We are also grateful to Bartłomiej Skowron for research support as well as Małgorzata Michnowska and Barbara Skwarczyńska for organizational support. The team would like to thank the government of Poland, and particularly, Katarzyna Kaczkowska from the Ministry of Investment and Economic Development, and Beata Lubos from the Ministry of Entrepreneurship and Technology Development, for their support and feedback throughout the process. We would also like to thank the National Center for Research and Development (NCBR), the Polish Agency for Enterprise Development (PARP), the Polish Development Fund, Bank Gospodarstwa Krajowego (BGK), and the regional Marshals’ Offices for their extensive participation and contributions to the different pieces of the project. 1 TABLE OF CONTENTS Project Overview........................................................................................................................................... 3 1 Introduction .......................................................................................................................................... 6 2 Background ........................................................................................................................................... 6 2.1 Poland’s innovation performance................................................................................................. 6 2.2 Poland’s innovation-related operational programs.................................................................... 10 3 Review of program evaluation studies ............................................................................................... 14 3.1 Direct effects of innovation support on firm performance ........................................................ 14 3.2 Indirect Effects of Government Support Programs on Firms ..................................................... 16 3.3 Evaluation of the innovation support programs ......................................................................... 19 4 Empirical Methodology ....................................................................................................................... 21 4.1 Evaluation of direct measures .................................................................................................... 22 4.2 Spillover effects of firm-level support on non-beneficiary firms ................................................ 26 4.3 Spillover effects of support to research institutions and business support institutions ............ 27 5 Aggregate impact and return on investment (ROI) ............................................................................ 28 6 Data ..................................................................................................................................................... 29 7 Empirical results .................................................................................................................................. 30 7.1 Direct effects with single treatment analysis ............................................................................. 30 7.2 Direct effects with heterogeneity ............................................................................................... 33 7.3 Direct effect with multiple treatment analysis ........................................................................... 36 7.4 Indirect effects ............................................................................................................................ 39 7.5 Return on Investment ................................................................................................................. 40 8 Concluding Remarks............................................................................................................................ 45 References .................................................................................................................................................. 46 Annex 1: List of Direct Measures in Innovative Economy and Regional Operational Programs ................ 52 Annex 2: List of Indirect Measures in Innovative Economy and Regional Operational Programs ............. 70 Annex 3: List of Measures in OP IE and ROPs EXCLUDED from the Evaluation .......................................... 81 Annex 4: Studies on the effects of innovation support programs on firms’ R&D expenditure decisions .. 87 Annex 5: Additional regression results ....................................................................................................... 88 Annex 6: Data cleaning and estimation procedures ................................................................................... 90 8.1 Data Management ...................................................................................................................... 90 8.2 Estimation ................................................................................................................................. 103 8.3 Practical issues .......................................................................................................................... 123 2 Project Overview A “Smarter Europe” is a top priority of the European Union (EU), the core of which is innovation, economic transformation, and more competitive small and medium enterprises (SMEs). These themes account for a huge part of EU spending in the past, present, and (likely) future programming periods. Despite high expenditures, impacts on the economy often appear modest or are not well understood. EU, national, and regional policymakers want to know where and how to invest to get the highest return on investment (ROI). This project responds to a request from the European Commission to develop and pilot methodologies to measure and improve innovation and SME support instruments. Poland was selected as the pilot country, since it is the largest recipient of EU funding, and has a rich set of support measures and implementing bodies. Drawing on the World Bank’s framework for Public Expenditure Reviews in Science, Technology, and Innovation, the project starts by examining the inputs to the innovation ecosystem, including economic priorities, government policies, and support instruments. It then analyzes the functioning of the support instruments, and ultimately the impacts on firm performance (that is, the return on investment / effectiveness). A combined understanding of these different dimensions gives public authorities a view into where and how innovation and SME support mechanisms can be improved. To our knowledge, this is the most comprehensive and robust analysis of EU support to SMEs and innovation to date. 1. Needs assessment and portfolio mapping analysis A brief overview of Poland’s innovation ecosystem was prepared to understand the country’s needs and policy priorities. This “needs assessment” highlights the main weaknesses in innovation-related outcomes and challenges in Poland compared to its regional and aspirational peers. Examples of the identified challenges relate to the productivity growth of new firms, the sophistication of production and exports, and firms’ capabilities (for example, management skills). The characteristics of all the current (2014–2020) innovation and SME-related EU and nationally funded instruments were then mapped. At the national and regional levels, 182 support measures— corresponding to about €21 billion—were mapped across 14 dimensions, including: objectives, beneficiary types, support mechanisms, implementing bodies, and others. The 11 largest instruments account for half of all funding, while the remainder is split between 171 instruments. The combination of the needs assessment and the portfolio mapping permitted an analysis of the ‘policy mix’. The policy mix analysis compares whether the current set of support instruments is aligned with Poland’s innovation- related and strategic needs and priorities, and seeks to identify potential gaps and overlaps. Four recommendations are offered: • Reduce fragmentation and overlap between support instruments, including through increased coordination between national and regional authorities • Consider narrowing the focus of regional instruments to adapt to local conditions • Improve support for management and export capabilities, and young firms • Improve the use of evidence when choosing support mechanisms 3 2. Functional analysis The functional analysis examines how the design, implementation, and governance of selected support instruments in the current EU financing perspective compare to international best practices. It aims to identify how the functionality of support instruments could be adjusted to increase the return on investment of public support. Through a series of semi-structured interviews with the managers of 21 support instruments (primarily at the regional level), detailed information on 31 dimensions of instrument design, implementation, and governance was obtained. Examples of good practices in instrument design include the identification of products and outputs, and the consideration of relevant stakeholders. Examples of good practices in instrument implementation include efficient application and selection processes, as well as project and instrument closure procedures. Additionally, the functionality of the national instruments appears somewhat better than the regional ones. Identified challenges relate to the improvement of: • Coordination between national and regional programs • Flexibility in the design and implementation of instruments (particularly at the regional level) • Instrument justification, and aligning instrument design with the desired systemic change • Operational planning, particularly with respect to better targeting instruments toward firms where the impact of public support is likely to be the highest 3. Return on investment / effectiveness analysis An ex post impact evaluation of the 2007–2013 EU-funded innovation and SME-related support instruments was undertaken. In partnership with Statistics Poland, the direct impact of the support was estimated using firm-level data and a difference-in-differences (DID) with propensity score matching approach. Indirect/spillover effects were estimated as well. Multiple estimation models were used to make the analysis as robust as possible. The analysis provides evidence on the impact of different types of support on: firm revenues, profits, exports, value added, research and development (R&D), and productivity as well as, ultimately, on the return on investment (ROI) of public support. The analysis found positive impacts on firm employment, sales, value added, and exports. This result is consistent across the different estimation models. In contrast, there is mixed evidence on the impact on productivity, depending on the productivity measure used. However, productivity impacts were higher for firms in the manufacturing sector. Lastly, there is minimal evidence that the programs affected investments (either physical assets or R&D) and profits. These findings are broadly similar to earlier studies, such as GUS (2015), which examined the impact for a more limited set of programs or firms. The estimated impact from the support measures translates to an estimated average cost per job gain that compares favorably to available estimates of programs in other countries. Support measures emphasizing R&D, innovation, exports, and access to markets appear significantly more cost-effective in job creation than programs with small capital grants or those focusing on access to finance. A Summary Report brings together the findings from the different pieces of the analysis. It provides insights on where and how innovation support mechanisms could be adjusted to improve the return on investment during the upcoming EU financial perspective (2021–2027). It also offers some considerations for the application of the methodology in the future, both in Poland and other countries. Key messages concern: 4 • Designing and targeting instruments to maximize impact (based on clear needs assessments), including a smarter approach to supporting young firms, exports, small firms, and firm capabilities • Reducing inefficiencies (fragmentation and overlap, for instance) and improving coordination and learning Technical Notes detailing the full methodologies and findings were prepared for each of the three dimensions. The subject of this Technical Note is the Return on investment / effectiveness analysis. 5 1 Introduction This Technical Note describes the evaluation of EU support to innovation in Poland. The aim is to measure the effectiveness of support measures in the 2007-2013 perspective, demonstrate impact at the firm level, and derive relevant conclusions for future project design. The methodology can also serve as a guide for future analyses of public spending in Poland and other countries. The analysis aims to be comprehensive in the programs included in the evaluation. As the main interest is on support for innovation (broadly defined), innovation-related programs were included based on a portfolio mapping exercise and consultations with the Polish government. During 2007-13, the EU funds were used to create measures within the Operational Program for Innovative Economy (IE) and the Regional Operational Programs (ROP) to boost innovation of firms through, for instance, grants for capital equipment, R&D equipment, access to business services and access to trade fairs. Thus, the evaluation is conducted for over 100 measures during the 2007-2013 perspective, which have a variety of project objectives and outcomes. The wide variety of support measures (or instruments) makes it challenging to conduct a rigorous impact evaluation. The approach taken in the report is agnostic about the estimation models, testing various specifications to ensure that the effects of the support measures on firms are consistent across different models. In addition, the evaluation is comprehensive as it uses administrative firm-level data in Poland that covers the universe of firms (with at least 10 employees). Lastly, using the estimated impact on firm performance outcomes and the costs of providing and administering the support measures, the report calculates the return on investment of the programs. Our findings suggest that there are positive impacts on firm employment, sales, value-added and exports. This result is consistent across the different estimation models. However, there is mixed evidence about the impacts on productivity, depending on the productivity measures used. Lastly there is minimal evidence that the programs affected investments (either physical assets or R&D) and profits. These estimates suggest that the return on investments (ROI) in terms of job creation for these programs is in with international estimates of cost effectiveness of other job creation schemes. In addition, the estimations show that there is high cost effectiveness for programs emphasizing R&D, innovation and access to markets. The report is divided into eight sections. Section 2 describes the background on Poland’s recent innovation performance and innovation-related EU-funded operational programs. Section 3 provides a review of the economic literature that will help to situate the evaluation of the programs. It argues that while Poland exhibited improvements in various measures of innovation and competitiveness during the period 2007- 2013, a comprehensive evaluation of the support programs is needed to understand the aggregate impact on firms. Sections 4 and 5 discuss the impact evaluation methodology and the cost effectiveness calculations. Section 6 describes the data sources. Section 7 presents the results and Section 8 concludes. 2 Background 2.1 Poland’s innovation performance From 2007, there was an increase in innovative activities in Poland, R&D expenditure increased, firms became more innovative, and the quality of research institutions improved. The increase in R&D 6 expenditure coincided with the period of increased EU support for innovation in the 2007-13 perspective. This represented a marked change from Poland’s past. Support for innovation in Poland was put aside during the early transition years in the 1990s. Justifiably, the Polish government prioritized structural and institutional reforms and omitted to establish a systematic direct support for innovation in enterprises (Szcygielski, 2007). Compared to peer countries that acceded to the EU at the same time, Poland had a low R&D expenditure share of GDP (Figure 1). Poland spent only 0.6 percent of GDP on R&D in 2004, compared to almost double that share in the Czech Republic and Slovenia. Poland had a markedly low share of R&D expenditure by businesses, with only Cyprus having a lower share among the peer countries. As economic development started to gain steam and Poland acceded to the EU in 2004, the Polish government turned their attention to innovation and R&D support. Poland’s R&D expenditure as a percentage of GDP almost doubled between 2004 and 2014 (Figure 2): R&D expenditure increased from 0.56 percent of GDP in 2004 to 0.97 percent in 2016. Figure 1: Small share of R&D expenditures in Figure 2: Poland’s R&D expenditures as a share Poland compared to other EU accession of GDP nearly doubled, 2004-2016 countries in 2004 1.6 1.2 R&D expenditures (% of GDP) R&D Expenditures (% of GDP) 1.4 1 1.2 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 Year Business Enterprise Government Higher Education Business Enterprise Government Higher Education Source: Eurostat. Source: Eurostat. The large increase in private sector investment in R&D coincided with the development of the innovation eco-system in Poland and the increase in EU funding to support more private sector innovation activities. The Polish Agency for Enterprise Development (PARP) was established in 2000, focusing on supporting entrepreneurship through innovation, developing human capital, and expanding markets. The National Center for Research and Development (NCBiR) was established in 2007 to manage scientific R&D programs and the diffusion of scientific research to the market. Both PARP and NCBiR are responsible for the disbursement of EU funds. Concurrently, Poland received a large amount of support for innovation from the EU during 2007-2013. EU-funded support for innovation increased steadily since Poland’s acceded into the EU and accounted for 11 percent of R&D expenditures in 2014 (Figure 3). 7 Figure 3: R&D Expenditure boosted by EU funds, Figure 4: Poland still lags behind other EU countries 2002-2014 in R&D expenditure, 2014 100 R&D expenditures in EU 28 (2014) 90 3.5 80 3 70 R&D expenditure (% of GDP) 60 2.5 50 2 40 1.5 30 20 1 10 0.5 0 0 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 FI SE AT DK DE BE SI FR NL CZ UK IE EE HU IT LU PT ES LT PL SK EL BG HR MT LV CY RO Country European Commission Business Enterprise Government Higher Education Other Business Enterprise Government Higher Education EU28 Average Source: OECD. Source: Eurostat. Note: Other category refers to funding from private non-profits, foreign sources other than the EC, and not elsewhere classified. Despite nearly doubling R&D expenditures as a percent of GDP, Poland still lagged behind other EU countries in 2014 (Figure 4). Poland was in the bottom third of the EU28 countries as it spent 0.94 percent of GDP on R&D, which was almost half of its peer countries (the 2004 EU accession countries). Slovenia spent 2.4 percent of its GDP on R&D, Czech Republic spent 2.0 percent, and Estonia spent 1.4 percent. A 2016 Deloitte study on R&D expenditures of Polish firms showed that the surveyed firms did not recognize R&D as an essential source for economic growth (Deloitte, 2016). In 2015-2016, only 17 percent of surveyed firms spent more than 10 percent of their revenue on R&D; 33 percent of surveyed firms spent more than three percent of their revenue on R&D. The study identified that the main drivers that can motivate more R&D investments by Polish firms include the availability of more and different types of benefits such as grants, tax deductions, and the availability of skilled researchers. At the firm level, the IE and related programs may have contributed to an increase in the number of innovative firms. The share of manufacturing firms that introduced new processes and new products increased from 2008 to 2014 (Figure 5). In 2008, 26 percent of manufacturing firms introduced new processes (to the firm) and this increased to 40 percent in 2014. Similarly, there was a 10-percentage point increase in the share of manufacturing firms that introduced products new to the firm and new to the market between 2008 and 2014. In addition, the share of firms that obtained a quality certification and use foreign technology increased between 2009 and 2013 (Figure 6). According to the WEF Global Competitiveness Survey, the sophistication level of production processes in Polish firms increased over the same period (Figure 7). While there were improvements in firms’ capacity for innovation between 2008 and 2013, the improvements only brought Poland back to its 2007 levels (Figure 8). The increase in innovative firms may have contributed to the increase in the quality of local suppliers (Figure 9). Higher quality suppliers can enable firms to participate in global value chains and provide better inputs to other firms. Finally, the support programs may also have positively affected the number of exporters. There was a five-percentage point increase in the share of firms exporting between 2009 and 2013 (Figure 10). 8 Figure 5: Share of manufacturing firms that Figure 6: Share of firms obtaining quality introduced new processes and products certification and foreign technology, 2009-2013 increased, 2008-2014 70 35 Percentage of Manufacturing Firms (%) 30 Percentage of Firms (%) 60 50 25 40 20 15 30 10 20 5 10 0 0 Have internationally- Using technology licensed New to firm New to market New to firm New to market recognized quality from foreign companies Process Products certification 2008 2014 2009 2013 Source: Eurostat. Source: World Bank Enterprise survey. Figure 7: Poland is using more advanced Figure 8: Improvement in the firm’s capacity for technology in its production processes innovation, but returning to 2006 levels Production Process Sophistication Capacity for Innovation 4.3 4.5 4.2 4 4.1 3.5 4 3 3.9 2.5 3.8 2 3.7 1.5 3.6 1 3.5 0.5 3.4 0 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 Source: WEF Global Competitiveness Survey. Source: WEF Global Competitiveness Survey. Note: Survey question is “In your country, how sophisticated Note: Survey question is “In your country, to what extent do are production processes?”, where 1= production uses labor- companies have the capacity to innovate?”, where 1 = not at intensive processes & 7=production uses latest technologies. all and 7 = to a great extent. Figure 9: Polish local suppliers are increasing in Figure 10: Increase in share of firms exporting, quality 2009-2013 Local Supplier Quality 25.0 Percentage of Firms (%) 5.2 20.0 5 15.0 4.8 10.0 4.6 5.0 4.4 0.0 4.2 Percent of firms exporting directly or indirectly (at least 10% of sales) 4 2009 2013 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 Source: WEF Global Competitiveness Survey. Source: World Bank Enterprise survey. Note: Survey question is “In your country, how do you assess the quality of local suppliers”, where 1 = not at all and 7 = to a great extent. 9 The IE program, as well as ROP’s dedicated measures and instruments also supported research institutions and promoted linkages between research institutions and industry. There was a perceived improvement in the quality of Polish research institutions between 2008 and 2012, but a sharp drop in 2013 (Figure 11). Concurrently, there was a perceived increase in collaboration between industry and universities in Poland, particularly from 2008-2010 (Figure 12). Figure 11: Improvement in the quality of Figure 12: Higher levels of collaboration research institutions but a sharp drop from 2013 between university and industry Quality of Scientific Research Institutions University-Industry Collaboration in R&D 4.2 3.8 4.1 3.6 4 3.4 3.9 3.2 3.8 3 3.7 2.8 3.6 2.6 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 Source: WEF Global Competitiveness Survey. Source: WEF Global Competitiveness Survey. Note: Survey question is “In your country, how do you assess Note: Survey question is “In your country, to what extent the quality of scientific research institutions”, where 1 = do business and universities collaborate on research and extremely poor—among the worst in the world and 7 = development (R&D)”, where 1 = do not collaborate at all extremely good—among the best in the world. and 7 = collaborate extensively. 2.2 Poland’s innovation-related operational programs Poland received a large increase in funding from the EU Structural and Cohesion Funds since its accession into the European Union. Over 2007-2013, Poland receive a total of 67.3 billion Euros for operational programs, out of which 12 percent or EUR 8.3 billion went towards the OP Innovative Economy (IE) (Figure 13). Innovation and competitiveness-related support was also included in the Regional Operational Programs (ROPs), as well as OP Development of Eastern Poland, OP Infrastructure and Environment, and OP Human Capital to a lesser extent. In total, over 35 percent of EU funds went towards innovation between 2007-2013. While the discussion here focuses on the OP IE as the largest source of innovation- related funding, relevant measures from the ROPs and other OPs are also be included in the analysis. 10 Figure 13: Over 35 percent of the EU-funds between 2007-13 went towards innovation- related programs Others OP 7% Innovative Economy OP 12% infrastructur e and Environment 42% OP Human Capital Regional 14% operational programs 25% Source: Ministry of Regional Development of Poland. Note: “Others” combines various operational programs (OP): OP Technical Assistance, OP of European Territorial Co - operation, OP Development of East Poland and Reserves. While the main innovation-related program is titled “Innovative Economy”, the program is not solely devoted to promoting research and development or technology adoption per se. The IE program has a broader focus on increasing the innovative activities and improving the competitiveness of firms and regional economies, with three focus areas: (1) stimulating the business environment; (2) intensifying cooperation between state-run research and development institutions, academia, and the private sector; and (3) increasing the public sector’s capacity for conducting advanced research on innovation. The IE program aimed to increase R&D activities of the science sector, improve access to external financing sources for innovative activities, increase the competitiveness of enterprises through the application of new technology, and foster the diffusion of innovation by providing entrepreneurs with high quality services and infrastructure. It also sought to improve Poland’s global image as an attractive economic partner and a good place for trade, as well as stimulate the digital economy. The IE and related programs were operational between 2007 and 2013 but the implementation of the programs extended beyond 2013.1 These IE and regional programs disbursed over 63 billion PLN (around 15 billion euro) to 29,000 beneficiaries. The programs started providing subsidies in 2007, but the amount of subsidies disbursed peaked in 2009 and then reduced slightly in 2011 (Figure 14). At its peak, there were about 13 billion PLN (3 billion euro) given to firms and other institutions (government and research). The number of beneficiaries follow a similar pattern and peaked in 2010 with just over 5,400 beneficiaries (Figure 15). While the programs ended in 2013, there were still contracts disbursing funds to beneficiaries in 2014 and 2015. 1 The data on the programs in this section refer to the OP IE, as well as innovation-related measures in the ROPs, and OP Eastern Poland. One measures from OP Human Capital is included (POKL 02.01) is included in the econometric analysis. 11 Figure 14: Increasing number of amount of Figure 15: Peak number of beneficiaries in 2010 grants disbursed, peaking in 2009 Total amount of grants Number of beneficiaries 14 6000 Billions zł 12 5000 10 4000 8 3000 6 2000 4 1000 2 0 0 2007 2008 2009 2010 2011 2012 2013 2014 2015 2007 2008 2009 2010 2011 2012 2013 2014 2015 Source: Authors’ calculation using SIMIK database. Source: Authors’ calculation using SIMIK database. Note: The start date of the contract was used. Note: the beneficiaries are the unique firms or other institutions that received the subsidies. The IE and regional programs consisted of 145 measures targeting different types of beneficiaries – firms, scientific entities, finance institutions – administered by authorities at the national and regional (or voivodeship) level. The measures have direct effects on firms, where they receive the grant directly for the supported activities; or indirect effects, where firms receive the benefit indirectly through the research or business support provided to universities, research institutes, and business support institutions. Some measures will be excluded from the evaluation. One set of measures supports general conditions for competitiveness but they are not specific on their interventions. It is difficult to capture the effects of these measures on firms and so these measures are not included in the evaluation. Another set of measures provides funding to government agencies, financial institutions and funds, and grants to improve the national and regional business environment. There are 35 such measures in the OP IE and regional program.2 There are 64 direct measures that address a broad range of issues related to firm competitiveness. For example, the measures can focus on the technology adoption of firms by supporting R&D activities, implementation of R&D projects, and the application of new technology in firms. The measures can also promote exports, which can increase the productivity of firms and encourage them to innovate and produce new products for new markets. Thus, the direct measures can be grouped into seven categories based on their objectives and interventions: technology adoption, competitiveness, export promotion, firm capabilities, financing, environmental, and digital adoption. Many measures have a narrow focus to address one issue such as technology adoption, export promotion or increasing firm capabilities. Almost half of the measures have a broad objective and support activities that can increase the competitiveness of firms through a mix of the aforementioned objectives, and they are direct measures in the regional OPs administered by the Voivodeships. In addition, the other operational programs on Human Capital and Infrastructure and Environment also have measures that benefited firms. The OP Human Capital has one sub-measure (POKL.02.1.1) to develop the human capital in firms by supporting training and consulting services. This sub-measure can be placed under the “firm capabilities” category of direct measures. The 2 See Annex 3 for a breakdown of these measures excluded from the evaluation. 12 OP Infrastructure and Environment has a priority axis IV that provides support firms to reduce their environmental burden by improving their environmental management and standards. The priority axis IV can be placed under the “environmental” category of direct measures. The list of direct measures is included in Annex 1. There are 46 indirect measures that provide support to research institutions, business support institutions, and local governments, and whose ultimate beneficiaries are firms. The measures can be grouped into two categories based on who are their immediate beneficiaries. First are the research institutes that receive grants to support R&D activities through projects or related infrastructure and equipment. Second are the business support institutions (BSI) that receive grants to support the creation and development of various centers, tech-park, incubators and other similar institutions that provide business support to firms to participate in innovative activities. These indirect measures are listed in Table 1 and details are provided in Annex 2. Table 1: Categories of indirect measures in relevant OPs Category Description of Measures National / Number of Regional Measures Research Support universities and research institutes in R&D activities Both 13 Institutions directly and to purchase equipment and other investments related to the R&D activities. Business Support the creation and development of BSIs (such as tech- Both 33 Support parks, incubators, and business centers) that provide Institutions finance, advice, and information to firms. The 110 measures included in the evaluation form a majority of grants in the IE and related programs. These programs provided 51 billion PLN, through almost 36,000 contracts to over 21,000 beneficiaries. The average number of contracts per firm was 1.68 and the average amount each firm received was 2.4 million PLN. Most of the grants were disbursed from direct measures and to beneficiaries located in the Mazowieckie Voivodeship, where Warsaw is located Figure 16). Similar, most beneficiaries received grants from direct measures are located in the the Mazowieckie Voivodeship (Figure 17). Figure 16: Most grants were disbursed from direct Figure 17: Most beneficiaries received grants measures and to beneficiaries in the Mazowieckie from direct measures and are located in the Voivodeship Mazowieckie Voivodeship 12 4500 Billions zł 10 4000 3500 8 3000 6 2500 2000 4 1500 2 1000 500 0 0 Direct Indirect Direct Indirect Source: Own calculation using Simik database. Source: Own calculation using Simik database. 13 3 Review of program evaluation studies In this section, the report is situated within three branches of the economic literature on program evaluation. The first branch examines how different government support programs on innovation, export promotion, or technology adoption has direct effects on firm. These studies examine how programs can improve firms’ performance in R&D activities, sales, exports and productivity levels. The second branch examines spillover effects of support to firms and research institutions. The third branch are studies that examined the effect of IE and ROP on firm performance in Poland. 3.1 Direct effects of innovation support on firm performance The literature on the effects of innovation programs has largely focused on how these programs affect a firm’s decisions to invest in innovation. Innovation programs usually provides some subsidy or credit towards a firm’s innovation expenditures, so many studies have focused on the policy question of whether the innovation programs substitute or complement a firm’s R&D expenditure. Innovation support programs can complement a firm’s R&D expenditure (“crowding-in effect”) by generating positive private returns for the firm as well as social benefits in the form of knowledge and training spillovers. On the other hand, the public funds may reduce private R&D expenditure as firms re-direct their own funds towards other purposes (“crowding-out effect”). These studies also examine whether the innovation programs supported projects with high social returns and activities that would not be carried out by firms in the absence of the programs (Aerts et al., 2006). There are many studies that examine the crowding-in and crowding-out effects of innovation programs on firms. These programs are at the EU, national and regional level and in both developing and developed countries. To isolate the treatment effect, researchers have used a variety of different methods such as instrumental variables, structural models, and treatment and control groups. Annex 4 provides a list of these studies. Effects on firms tend to be heterogeneous. Some innovation programs show a larger effect on older and domestic firms’ R&D expenditure decisions.3 Older or more established Argentinian firms were more likely to substitute their R&D expenditure for public funding than the new innovators (Chudnovsky et al., 2006). Goerg et al. (2007) show that small and medium sized grants increased private R&D expenditure of domestic firms in Ireland, large sized grants crowded out private R&D spending of these firms. However, the grant had no statistically significant effect on foreign establishments in Ireland regardless of the size of the grant. While the effects on firms’ R&D expenditure decisions are important for policymakers, the more important question is whether these innovation programs have improved firm performance. Many studies show that an increase in R&D expenditure can boost firms’ employment, sales, productivity and exports. For example, Choi et al. (2014) find positive effects of R&D firm intensity on sales growth in China from 3 There is no clear direction on how firm size can influence the effect of public innovation programs on R&D activities. Czarnitzki et al. (2002) find that innovation support led to lower patenting for firms in Eastern Germany, which tend to be smaller and less productive due to their communist heritage compared to firms in Western Germany. On the other hand, Gonzalez et al. (2008) find that the innovation support in Spain had a larger impact on small manufacturing firms: small manufacturing firms increased the R&D expenditure by 10.7 percent compared to 6 percent in large firms. 14 2001 and 2005 and Lachenmaier et al. (2004) find a positive effect of private innovation expenditures of German manufacturing firms on their exports ranging from 6.8 to 8.2 percentage points in 2002. These effects are also heterogeneous and depend on firm characteristics. Mitra et al. (2015) find a positive effect of R&D expenditures of Indian firms on employment from 1998 to 2010 with different magnitudes across industries. Doraszelski et al. (2013) conclude that between 1990 and 1999 private R&D expenditures had positive effects on labor productivity of Spanish manufacturing firms with differing magnitudes across industries. Moreover, R&D expenditure can provide firms with an innovation advantage, especially if the innovation level of the average firm is low. Griffith et al. (2004) examine the positive effects of firms’ R&D expenditures on their TFP levels across 12 OCED countries and find that the effect is stronger if the firm is in a country with relatively low TFP levels and further away from the technological frontier. Compared to the literature on R&D expenditure, the literature examining how innovation programs affect firm performance is relatively scarce. A handful of studies have examined how innovation programs have affect firms’ employment, sales and productivity (Table 2). Table 2: Studies that examine the effect of innovation programs on firm performance Papers Effects of innovation program Countries (year of study) Employment Wallsten (2000) No effect USA (1992) Link and Scott (2012) Increase (but small effect) USA (2005) Sales Czarnitzki (2011) Increase Canada (1997-1999) Innovative Activities Bruhn and McKenzie (2017) Increase Poland (2007-2013) Labor productivity Griffith (2006)4 Increase Germany, France, Spain, U.K. (1998-2000) Hall et al. (2009)5 Increase Italy (1995-2003) Chudnovsky et al. (2006) No effect Argentina (2001-2004) Employment, capital-labor ratio, output, productivity Benkovskis et al. (2019) Increase (capital-labor ratio, Latvia (2007-13) employment, output) No effect (productivity) Note: The positive effects presented in the table are statistically significant. Innovation programs can create jobs but the effect may be small. Two studies have examined the effects of the Small Business Innovation Research (SBIR) program in the U.S. The program has awarded more than US$1 billion per year in R&D grants to small firms to increase innovation commercialization since the 1970s. Wallsten (2000) investigated the SBIR program on employment of U.S. firms in 1992. While the author finds that firms with more employees win awards, the awards do not have a statistically significant effect on employment. On the other hand, Link and Scott (2012) focused on the effects of the SBIR program on small firms in 2005. They find that, on average, the direct impact of the SBIR program on employment is small as many firms retain little or none of their workers after the projects is completed. 4 Griffith (2006) separately focuses on the impact of EU and national funding on R&D expenditures of firms and the impact of such expenditures on labor productivity. 5 Hall et al (2009) separately examine the effect of government subsidies on R&D expenditures, the impact of those expenditures on product and process innovation and on labor productivity. 15 However, the authors show that firms retained more workers if they generated intellectual property from their funded projects, and firms created more jobs if the government assisted in creating a market for the firms’ products and services. Innovation programs have also promoted firms’ output, especially of innovative products. Czarnitzki (2011) examines the effects of the Canadian Federal and Provincial R&D tax credit programs. The programs were used by more than a third of manufacturing provincial enterprises and by close to two-thirds of firms in high-technology sectors from 1997 to 1999. To correct for a potential selection bias, the author used propensity score matching procedure to create a control group consisting of all innovating and non- innovating firms that did not receive the R&D tax credits. The author finds that the beneficiaries (tax credit recipients) had more product innovations and higher sales of new products. No effect was found on firms’ profitability or market share. In a recent study, Bruhn and McKenzie (2017) employ the regression discontinuity method to examine the effects of funding from Poland’s In-Tech program on firms. The program is financed by the Polish National Center for Research and Development (NCBiR) and supports collaboration between research entities and firms. The authors use the application data to examine firms that are just above and below the funding cut-off threshold to robustly examine the effects of the funding on firms’ innovative activities. They find that the In-Tech program resulted in more collaboration, more patents and publications and some commercialization of those products that resulted from the project. Perhaps the closest to our study, Benkovskis et al. (2019) performs a similar evaluation of EU-funded programs in Latvia between 2007-13. They examine the effect of these programs (assuming the same effect across all programs) for about 1000 firms using a propensity matching difference-in-difference method. They find that firms increase their capital-to-labor ratio, employment and subsequently sales after receiving the program support. The programs do not have any effect on firms’ productivity (whether measured as labor productivity or TFP). Lastly, innovation programs can increase firm productivity. Griffith et al. (2006) compare the effects of EU, national and local R&D funding on R&D expenditures of firms and their impact on labor productivity across France, Germany, Spain and the U.K between 1998 and 2000. Without differentiating between the three sources of public funding on innovation in a Tobit model, the authors find statistically significant positive effects on labor productivity for all countries, with differing magnitudes across countries and firm sizes. The exception is Eastern Germany, which shows statistically significant negative effects. One deficiency in Griffith et al. (2006) is the use of labor productivity as a measure of firm productivity. Labor productivity does not control for any increase in capital, which is the likely result in most innovation programs. The report will remedy this by using total factor productivity as a measure of firm productivity. 3.2 Indirect Effects of Government Support Programs on Firms The indirect effects of government support programs can occur through two channels. First, firms receiving support from the government can indirectly affect neighboring firms (spatial spillovers), firms within their sector (horizontal spillovers) and firms that purchase or sell to them (vertical spillovers). Non- beneficiary firms can receive knowledge, technology, and higher quality inputs from beneficiary firms. 16 Second, research and business support institutions supported by the government can benefit firms that are located near them as these institutions provide knowledge, services and highly-skilled workers. There is a rich literature that has examined the spillover effects of firms that export, are foreign-owned (through foreign direct investment), or possess some special quality (such as high productivity or better technology).6 Many of these studies use a similar method to estimate the spillover effects: either by the presence (or number) of firms in the sector or location. A selection of studies that examine whether firms provide any knowledge spillovers are summarized in Table 3. Table 3: Selected studies that examine knowledge spillovers from firms Paper (year of publ.) Spillover type Effects on performance Country (sample period) Productivity (TFP) Goodridge, Haskel, Wallis (2012) Vertical Increase U.K. (1992-2007) Greenstone, Hornbeck, Moretti Spatial, Increase U.S. (1973-1998) (2010) vertical Hanel (2000) Horizontal, vertical Increase Canada (1974-1989) Sales Bloom, Schankerman and Van Horizontal, spatial Increase U.S. (1981-2001) Reenen (2013) Goya, Vaya and Surinach (2016) Vertical, horizontal - Ambiguous (vertical) Spain (2004-2009) - Increase (horizontal) Note: The positive effects presented in the table are statistically significant. Many studies show that better or higher performing firms have positive effects on the performance and productivity of other firms. Greenstone et al. (2010) show that U.S. plants in counties that had attracted a large manufacturing plant – the Million Dollar Plant (MDP) – saw substantial increases in TFP compared to plants in other counties that had not attracted the new plant for the 1973-1998 period. Five years after the opening of the MDP, the TFP of plants was 12 percent higher in the so-called “winning” counties than in the “losing” counties. Hanel (2000) finds positive horizontal and vertical R&D spillover effects generated by domestic and foreign industries on TFP growth of Canadian manufacturing industries from 1974 to 1989. Goodridge et al. (2012) finds positive correlations between external R&D knowledge stock growth and industry TFP growth in seven UK industries over the 1992-2007 period, which is consistent with vertical R&D spillovers. The study defines external R&D knowledge stock growth as outside industry knowledge stock growth weighted by matrices based on flows of intermediate consumption and workers. R&D spillovers can also affect firm performance such as sales. Bloom et al. (2013) investigates two different types of spillover effects: first, technology (or knowledge) spillovers, which may affect the productivity of other firms that operate in similar technology areas and second, product market spillovers, which may cause negative business stealing effects from R&D. The authors show that both types of spillovers exist and that the gross social returns are 34 percentage points higher than the private returns. Finally, Goya et al. (2016) investigate the effects of vertical and horizontal spillovers on sales of Spanish 6 For examples of studies that examine spillovers from exporters and foreign direct investment, see Koenig et. al (2010), Javorcik and Spatareanu (2008, 2011), Fernandes and Tang (2014), and Javorcik (2004). 17 firms from 2004 to 2009. The study concludes that horizontal spillovers had mostly statistically significant positive effects on firms’ sales with larger effects for low-tech and large firms. Studies have also shown that universities and research institutions create knowledge spillovers to firms located near them. Knowledge diffusion can occur through the geographical concentration of firms around universities and research institutions, knowledge transfer though mobile employees, and informal knowledge exchange through social networks (Klepper, 2007; Breschi and Lissoni, 2003; Singh, 2005). Knowledge spillovers from research institutions are more evident through formal collaboration linkages. George et al. (2002) investigate effects of business-university alliances (formal agreements) on firm innovation and performance outcomes for a sample of publicly traded biotechnology companies in the U.S in 1995. They find that firms with established links to universities produce more patents, enhance product development and reduced R&D spending than firms without such links. However, the study did not find statistically significant differences in financial performance measured by the ratio of net sales to assets. Rothaermel and Thursby (2005) examine the effects of incubator firm-university linkages on incubator firm performance for a sample of start-up firms incubated in the Advanced Technology Development Center at the Georgia Institute of Technology for the 1998-2003 period. The results show that a firm’s link to the sponsoring university reduced the probability of new venture failure and retarded timely graduation with stronger effects the stronger the university link. Belderbos et al. (2004) examine the impact of R&D collaboration in 1996 on subsequent productivity growth (labor productivity and productivity in innovative sales) for a sample of Dutch innovating firms between 1996 and 1998. The analysis differentiates between four types of R&D partners – competitors, suppliers, customers and universities & research institutes. The results show that supplier and competitor cooperation had a significant impact on labor productivity growth, whereas competitor cooperation and collaboration with universities & research institutes positively affected growth in innovative sales per employee. Additionally, innovative sales were stimulated by incoming spillovers from customers and universities. The study shows that university cooperation was instrumental in creating and bringing to market radical innovations generating sales or products novel to the market, thereby improving the growth performance of firms. While the literature is less rich, there are studies that show some positive impacts of business support institutions on firm performance. Cheng et al. (2014) show that the presence of science parks at the national, municipal and local levels significantly increased the probability of attracting more high-tech SMEs in the Chinese region of Shenzhen in 2007. They find that high-tech SMEs and large firms prefer the national-level science parks that have more favorable policies and a more active R&D environment. Colombo and Delmastro (2002) compare outcomes of Italian new technology-based firms (NTBFs) located inside and outside of incubators (or science parks) and business innovation centers (BICs) in 2000. The results show that firms in incubators and BIC outperformed non-incubator firms on education of the workforce, adoption of innovative ICT, participation in EU-sponsored research projects, and the ability to take advantage of the scientific and technical services provided by research organizations. Incubated firms are more likely to establish formal commercial and technical relationships with universities. Finally, Schwartz (2013) conducts an empirical analysis of German start-up firms from five German incubators over a 10-year time span to determine the impact on long-term business survival. The results find no 18 statistically significant higher survival probabilities for firms located in incubators compared to firms outside of them in all five incubator locations. 3.3 Evaluation of the innovation support programs Several studies have examined the effects of the IE and ROP programs on firms in Poland but they focus only on a subset of measures, do not use a comprehensive dataset, do not examine the full period when the programs were operational or do not employ robust methodologies. In the context of this report, a review of the existing studies published in English and Polish reveals that there are approximately 30 ex- post evaluation of EU and national funded programs implemented in 2007-13. Among these studies, nine studies used robust methodology similar to the approached used in this report. Some of these studies are summarized below: • Marshall’s Office of Pomeranian Voivodship in collaboration with WYG PSDB (2013) focuses on the Regional Operational Program (ROP) for the Pomorskie Voivodeship for 2007-2011. 7 The study examines the effects of the grants and subsidized loans on innovation and competitiveness indicators of micro-firms and SMEs. The data for this study was collected through surveys conducted online or through phone calls, which can restrict the sample size or introduce sample selection bias. While a matching procedure was used to create a control group, the study does not include details on the procedure. The study finds that the regional program had a positive impact on the innovation and competitiveness of beneficiaries: there was an increase in employment, revenue growth from the sale of new or significantly improved products and services, and clients. • A study by the Ministry of Infrastructure and Development (2014) examines the effects of three programs supported by the EU Cohesion Policy, which includes the IE program, but does not distinguish between the programs in the evaluation. The study focuses only on large firms with more than 249 employees and over EUR 50 million gross turnover or EUR 43 million total liabilities. Data was collected through interviews so the sample size was small. • Polish Agency of Enterprise Development in collaboration with CEAPP UJ (2015) is one of the more robust evaluations of the IE program. The study uses a propensity score matching and a difference-in-difference method to examine the effect of the IE program. While the methodology is robust, the study focuses on only four measures in the IE program.8 Nonetheless, the study finds that the IE program increased firms’ employment, the offering of unique products and services, and the quality of products and services. Additionally, the program increased the share of companies that introduced new or significantly improved products, business systems, or logistics solutions. 7 The measures that WYG PSDB examined in their study is RPPM.01.01.00, RPPM.01.02.00, RPPM.01.03.00 and RPPM.01.04.00. 8 The study focused on four measures: POIE 04.01.00, 04.02.00, 04.04.00, 08.02.00. 19 • Similar to Polish Agency of Enterprise Development in collaboration with CEAPP UJ (2015), the Central Statistical Office (GUS) evaluated an expanded list of measures (with three more national measures and regional measures) from the IE program (GUS, 2015). The results from GUS (2015) study is noteworthy as it is the closest in coverage of measures and methodology as the analysis in this report. While the study uses the same robust methods as evaluation described in the paragraph above, it evaluates the effects from national measures individually and regional measures as a group. Using the same method, the study finds that the regional measures increased employment and fixed assets of some beneficiaries, but the effect is not present in all regions. No significant impacts of the regional measures are observed for profits, revenues and R&D expenditure or exports. The study does not find any effect of the national measures on net revenues or profits. However, there are some positive effects on employment, fixed assets, export activities, and R&D activities for firms receiving support from certain measures. • A study by the Ministry of Development in collaboration with WYG PSDB (2016b) examines the effects of a long list of measures in the IE program on firm performance over 2007-2013. As the study collected information through a telephone survey, no data was collected on firm characteristics and outcome variables before the IE program. The study found that the IE program was fairly effective in support of innovative activities, especially measures POIE.04.01.00 and 01.04.00, stimulating R&D activities in firms. In contrast, the development of linkages between enterprises and business support institute (measure POIE.05.01.00) was not as effective in improving firm performance. Impact of OP IE measure managed by NCBR (POIE.01.05.00) was evaluated based on a similar methodology in study by the National Centre of Research and Development in collaboration with Taylor Economics (2017). Overall, previous evaluations of the IE and related programs have not been comprehensive. These studies may not use robust methods or complete data but their results are useful to build upon. They show that the effects of the measures are different and the proposed evaluation will need to be careful about how the measures are grouped. Moreover, the regional measures from the Voivodeships may have a different effect from the national measures as they tend to be broader in their focus, i.e. the measures that focus on competitiveness (a broad slate of interventions) are generally regional measures. In addition, the evaluation of the programs can consider whether external factors such as business environment, educational attainment, and economic conditions of the regions influence the effect of the measures on firm performance. There are other studies that have used different methods that can inform the results in these studies and the proposed evaluation (Polish Agency of Enterprise Development in collaboration with CEAPP UJ (2015); Ministry of Development in collaboration with WYG PSDB (2015); GUS (2015); Ministry of Development in collaboration with WiseEuropa (2017)). For example, a report commissioned by PARP (WiseEuropa and PIB, 2017) uses macro modeling techniques (i.e. Dynamic Stochastic General Equilibrium, or DSGE) to evaluate the projected effects of the IE programs implemented by PARP on the macroeconomy. 9 There are also ongoing studies being prepared by PARP that can complement the 9 DSGE methods are used to forecast the longer-term effects on the macro economy but they are based on strong assumptions about the economic models. The DSGE methods are not comparable to the methods in this report. Nonetheless, the study finds that IE programs administered by PARP can contribute to a relative increase in capital investment but there is no sustained effect. 20 evaluation (Polish Agency of Enterprise Development in collaboration with ARC and Exacto (2015); Polish Agency of Enterprise Development in collaboration with WiseEuropa and OPI (2018)). Moreover, the focus of these studies is on the direct effect on firm performance but there are many measures with indirect effects on firms. First, some indirect measures provide grants to research institutions for R&D activities and investment, which can generate knowledge spillovers to firms located near the institutions. These research institutions can also provide specialized labor to the sector, increasing the skill level and knowledge in these firms. Ex-post analysis of effects of measures of the PO IG II Priority Axis, which were managed by the NCBR, provides evidence of the such projects (National Centre of Research and Development in collaboration with Agrotec Polska (2017a)). Second, other indirect measures provide grants to develop business support institutions that can help firms become more innovative. Lastly, beneficiaries of grants from the direct measures can also indirectly improve the performance of non-beneficiary firms. Non-beneficiary firms can benefit through the knowledge spillovers from beneficiary firms in the region, or from the specialized labor that these firms may produce. They can also benefit through vertical industrial linkages by purchasing better inputs or selling more output to beneficiary firms. There have been studies on the effects of the business parks and business support institutions (Ministry of Development in collaboration with WYG PSDB and Evalu (2016a); Ministry of Regional Development (2012)) that use different methods from our proposed evaluation but are nonetheless useful in complementing our evaluation of the indirect effects. Additional information on macroeconomic effects of the Cohesion Policy in Poland in relation to the National Strategic Reference Framework (NSRF) are included in numerous analyses performed on the behalf of the Ministry of Economic Development (and summarized in the report of the Ministry of Development in collaboration with ImApp (2017b)), however its scope does not include measuring impact of specific measures on innovativeness of enterprises. 4 Empirical Methodology While there were improvements observed in many innovation and competitiveness indicators between 2007 and 2013, it is difficult to causally attribute the improvement to the operational programs without a proper evaluation. The improvements can be a result of a variety of reasons unrelated to the operational programs, such as a growing economy and an improvement in the quality of European research institutions. First, many indicators relate to outputs, such as amount of R&D expenditure, introduction of business processes, and use of intellectual property, but they do not reveal the impact of the programs on actual firm performance outcomes such as product/market diversification, revenue, employment, and productivity. Figure 18 presents our theory of change. Under certain market imperfections, public support for innovation is expected to improve intermediate outputs such as R&D activities, which in turn is expected to have an impact on firm survival, growth, and productivity, both directly and through spillover effects. 21 Figure 18: Theory of Change for Innovation Support Second, many of the support measures are targeted at firms, and it is difficult to understand the impact without looking at responses at the firm level. As measures in the programs are varied and will possibly have different effects on firms, different methods should be employed for the different categories of direct and indirect measures. The evaluation can examine how these measures affected performance variables (such as sales, revenue, employment, export activities, and productivity levels) of firms who directly receive funding. Indeed, some measures will have more effect on certain variables: the export promotion measures will likely increase export activities, while the technology adoption measures will likely increase productivity levels. In addition, they can affect non-supported firms indirectly through changing competition or the composition of both output and input markets. Given the different types of measures in the IE and related programs, we will perform four groups of evaluations to consider the full effects. The first component is an evaluation of the effects of the direct measures on beneficiary firms where the evaluation will be conducted based on the identified categories of measures. The second component evaluates the spillover effects from beneficiary to non-beneficiary firms. The third component evaluates the spillover effects from the grant-receiving research institutions on firms. The last component is an evaluation of the effects of grant-receiving business support institutes on firms. 4.1 Evaluation of direct measures 4.1.1 Outcomes of interest The program’s direct support measures can affect firms’ performance in many ways. The grants and other financing instruments subsidize firm’s purchases of tangible or intangible assets and other business services. If recipient firms previously underinvest due to constraints such as access to finance or lack of information about available services or technologies, these measures should increase their investments in innovative activities and consequently increase firm’s output, employment, and productivity. More specifically, we are interested in both intermediate outputs and longer term outcomes (see Figure 18 above). Most of these outcomes—survival, employment, investment, sales, exports, value added, profits, 22 value added per worker—can be derived directly from firm-level data. Total factor productivity (TFP) will be calculated based on the index approach as in Asker et al. (2014).10 4.1.2 Estimation methodology: matching and difference-in-difference (DID) There are two key challenges in identifying a causal relationship between access to grants/loans and firm performance. First, firms might self-select into applying for the grants based on their assessment of needs and success likelihood. Second, non-random allocation of the program measures might further select beneficiary firms that are systematically different from non-beneficiary firms. Therefore, any observed differences in performance maybe due to inherent differences in firm’s characteristics rather than due to the impact of receiving support. This is likely to be the case if the aim of these measures is to select “better” firms in the first place. Indeed, program data as of 2011 indicate that applicants for R&D support are disproportionally in knowledge-intensive sectors activities while applicants to capital investment support are dominated by low and medium-tech manufacturing. As described above, the Polish implementing agencies do select firms based on a set of selection criteria such as firm size and sector. There is some evidence suggesting that risk-aversion during the application review process results in the bulk of R&D funding channeled to large firms and low to medium-tech sectors (Kapil et al. 2013). To mitigate concerns with the endogenous selection of firms into these policy measures, we propose to use a propensity score matching (PSM) methodology to account for observable differences in characteristics between beneficiary (treated) and non-beneficiary (untreated) firms. Intuitively, PSM is a process of replicating an experimental dataset by estimating a firm’s likelihood of receiving support (the treatment) based on their observed characteristics. Under the unconfoundedness assumption – i.e. after controlling for observable characteristics, treatment exposure is independent of firm outcomes – and the common support assumption – i.e. given any values of observable characteristics, each firm has a non- zero probability of receiving a treatment – then controlling for the propensity score is sufficient to eliminate bias (Rosenbaum and Rubin, 1983). That is, non-treated firms with similar propensity score can be a valid control group for the treated firms. In the context of our evaluation, there are several advantages of using the PSM methodology. First, we are interested in evaluating the impact of the overall innovation-related support, which involves over a hundred measures. The sheer number of measures renders selection on observables the most likely viable method. 11 Second, rich firm-level data in Poland enables us to control for a large set of firm characteristics, making the unconfoundedness assumption more plausible. 10 We do not use the control functions approach such as in Olley and Pakes (1996) or Ackerberg, Caves, and Frazer (2015) to estimate TFP as they are particularly prone to measurement problems in our context (see section 6.1.2 on selection problem). More specifically, the productivity process in these estimation procedures is assumed to depend only on past productivity and an exogenous shock. However, if innovation is expected to increase productivity, then the productivity process needs to account for whether firms receive innovation funding. To ensure the productivity shock remains exogenous (to the firm capital and labor) requires no misspecification in this relationship, which is difficult to achieve. 11 As opposed to methods requiring detailed institutional knowledge of each program such as Instrumental Variables or Regression Discontinuity Design. 23 Despite these advantages, there are multiple shortcomings with the standard PSM method that we will attempt to mitigate. First, as there is extreme heterogeneity in the support measures, it is both difficult to establish a valid control group with only one treatment group as well as less useful in terms of identifying the effect of different support instruments. To account for this complication, the PSM model can be extended to multivalued or multiple treatment cases to consider different grant amounts (Imbens, 2000; Lechner, 2001) or categories of measures (Joffe and Rosenbaum 1999, Imai and van Dyk 2004). In our case, grant amount is only one dimension of the program heterogeneity. Given the very different objectives, financing instruments, and eligibility conditions of different program measures, we choose to categorize them into groups based on those conditions. Grouping based on these dimensions helps to avoid having a prohibitively large number of treatments while preserving the critical heterogeneity across policy measures. More specifically, suppose we have grouped the types of measures into M support schemes. Denote the treatment status of each firm as () = ̅̅̅̅̅̅ 0, where = 0 if a firm is never treated and = if the firm receives the support in group . M can range from 1 in simple models with a single treatments to multiple groups in multiple treatment models. The probability of being in support scheme m can be estimated as: ( ) = ( = | ) = ( ) + (1) Here, the potential set of observables includes the firm employment, revenue, profit, tangible and intangible assets, average wage, sector, region, age, growth rate of past outcomes, and other characteristics measured before treatment. The function can be estimated parametrically using a multinomial logit or non-parametrically.12 Once we obtain an estimate of ̂ ( ), the treatment effects can be estimated, via matching or inverse probability weighting (IPW), which can be implemented in a weighted regression model. To mitigate this concern with misspecifications of the propensity score and the treatment effect models, we will use the doubly robust estimation by further controlling for covariates in a weighted regression with inverse propensity score weights (Wooldridge 2010). Finally, the methods described above only control for observable characteristics. Even with rich data, there likely exist unobservable firm characteristics that can influence both treatment propensity and firm performance, violating the unconfoundedness assumptions. To account for this possibility, we will combine matching with the difference-in-differences (DID) estimator to control for time-invariant characteristics, such as managerial capabilities, that may influence firm performance. More specifically, this amounts to defining a trimmed or matched sample for the treated firms and estimating a DID model on that sample. As the firms receive grants at different times throughout the period when the programs were operational, both the matching procedures and DID specification need to account for the different timing at which 12 In a multinomial logit model: ( ) = , where can include polynomial terms and interactions. 1+ ∑=1,−1 24 firms receive the funding. One candidate matching procedure is for each year of the programs (that is between 2007-2013), we find controls for the firms who started receiving treatment in the same year using pre-treatment firm characteristics. 13 This approach has the advantage of allowing the program selection process to vary each year, which is robust to considerations such as the rush at the end of the programs to disburse money. For each year (where = ̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅ 2007, 2013), we will estimate the following fixed effect (FE) regression:14 = + + ∑ + + (3) Where = when firm i starts receiving treatment is the outcome variable for firm in year t, as discussed in section 4.1.1 and denote the year and firm FE, respectively = 1 if the firm is receiving the policy group at year , and 0 otherwise denotes the set of firm characteristics. is the estimated treatment impact of policy group m for firms starting treatment in year k In this specification, the overall ATE of the program can be estimated as: 1 = ∑ 4.1.3 Heterogenous treatment effects: short vs. long term impact and the role of firm characteristics and regional business environment on program effectiveness The matched DID specification can also be modified to allow for heterogenous treatment effects, by including interactions with the treatment status: = + + ∑ + ∑ × + + (4) Where the variables are defined as above and is the set of additional controls and is the coefficient that captures the heterogeneous effects based on the control . The set of potential controls include: • Baseline firm characteristics such as size, age, sector: to examine whether the impact of support varies by types of firms. • Years after receiving support to identify whether the impact varies with time. • Regional characteristics such as education level, infrastructure: to examine whether the effectiveness of support measures depends on the quality of a region’s investment climate. 13 Examples of sample selection procedures are: (i) trimming to discard observations with extreme propensity scores, (ii) nearest neighbor matching to restrict the control sample to those most observationally similar to the treated firms. 14 This specification relaxes the unconfoundedness assumption from [ ()| , ] = [ ()| ( )] to [ ()| , , ] = [ ()| ( ), ] where is a firm fixed effect and = (1 , . . , ) is a time series of all treatments. For examples of DID matching applications, see Smith and Todd (2005), Gorg et al. (2008), Bradley and Migali (2012). 25 4.2 Spillover effects of firm-level support on non-beneficiary firms Beside the direct effects on beneficiary firms, policy measures that provide support at the firm level can also affect non-beneficiary firms through their interactions with the beneficiaries. There are three channels through which these spillovers can happen: • First, through geographical proximity: firms located near beneficiary firms can take advantage of knowledge spillovers through operating in a close environment and having repeated interactions.15 • Second, through horizontal industry linkages: similar to the ways that firms can learn through geographical proximity, firms in the same industries can also learn by operating in the same market, such as through employee turnover and industry association meetings. As a result, support in an industry can result in a positive impact on non-supported firms in the same industry. On the other hand, there can be negative spillover effect if supported firms compete away demand from non- supported firms. The overall impact will depend on the strength of each effect, on the degree of market competition, and on tradability of the sector. Since the program supports innovation and exports, we would expect see the demand substitution effect to be small if supported firms succeeded in accessing external markets or create new product demand. • Third, through vertical industry linkages: Non-beneficiaries may also benefit as they purchase higher quality inputs from beneficiaries or sell more or higher quality output to the beneficiaries (“vertical linkages”). To examine these spillover effects, we will estimate for the non-treated firms (that is, firms that never received any direct support throughout the sample period), how the extent of their exposure to treated firms – either spatially or through industrial linkages – impact performance: = + + + + Θ × exposurerst−1 + (5) where , , , and denote the year, firm, sector and regional fixed effects respectively and the subscript s denotes the NACE rev. 2 sector. Our identifying assumption is that exposure to treatment, which will be calculated at the regional or sector level, is exogenous to the firm. While treatment status is potentially endogenous at the firm level, an individual firm cannot control how much other firms receive support. Hence exposure calculated based on total support received by other firms at the regional or sector is likely exogenous to the firm. We control for firm fixed effects to further remove time-invariant unobserved firm, region, or sector level characteristics that can affect both industry/regional level exposure as well as performance of the firm. Since knowledge and other supply/demand shocks often diffuse slowly over time, we control for exposure terms in lags rather than in levels. To measure exposure, we construct the following terms: 15 A caveat to the analysis on geographical spillovers is that the data may only provide the location of the firm’s headquarters and not its production facilities. Many Polish firms have their headquarters in Warsaw and can have factories in other regions. In this case, we may overestimate the geographical spillovers for firms in Warsaw and underestimate the geographical spillovers for firms in other regions. 26 • The first variable captures the extent of innovation support in a region: . = ∑ (Tirt > 0) where ( > ) is an indicator variable that equals one if firm i is a beneficiary in region r, is the grant amount received by firm i in year t. • The second variable captures exposure through horizontal linkages, or other firms in the sector: ℎ. = ∑ (Tist > 0)/N or ℎ. = ∑ (Tist > 0) where ( > ) is an indicator variable that equals one if the firm i is a beneficiary in sector s in year t, is the grant amount received by firm i, is the total number of firms in sector s. We will experiment with both the (i) total amount of grant and (ii) share of firms receiving grants within a sector for two reasons. First, the total amount of grant matters as it indicates the intensity of support and potential upgrading happening in a sector. At the same time, the share of firms receiving grants indicates the demand substitution effect based on the share of competitors receiving a subsidy. • The third and fourth variables captures backward and forward exposure through vertical linkages: . = ∑ → × ℎ. ≠ . = ∑ → × ℎ. ≠ where → is the share of output from sector s sold to sector k in its total output and → is the share of input that sector s buys from sector k over its total inputs. Both are available as technical coefficients from the input-output table. . captures how much support is received by upstream sectors, which is then weighted by their importance in terms of demand for sector s. Similarly, . captures how much support is received in downstream sectors, which is then weighted by their importance in terms of supply to sector s.16 The four spillover variables can be included separately in a regression of firm performance variables of non-beneficiaries. They can also be included together to examine the relative importance of each variable. 4.3 Spillover effects of support to research institutions and business support institutions Indirect measures that promote R&D activities or provides grants for R&D infrastructure in research institutions can also generate spillovers to firms in several ways. Support to research institutions can increase the supply of high-skill labor or create commercial spinoffs of R&D outputs to firms in the region. Proximity to researchers can reduce cost for firms to access information about new technologies. Another type of indirect measure is through the development of business support institutions (such as techparks, 16 → More specifically: → = ∑′ →′ . An alternative measure to capture the total network effect (as opposed to only direct effect) is based on the use of the coefficients from the Leontief matrix: = ( − )−1 (see Lane 2017). 27 incubators, business centers) that provide important business services to firms. Evaluating the impact of BSIs is similar to evaluating the direct measures, but information about which firms have been supported by each BSI is needed. Firms receiving support from the BSI can presumably form another treatment group where treatment propensity is estimated simultaneously with other treatments through equation (1). However, estimation of the direct effect of BSIs on firms is excluded due to data constraints as these firms are usually small and cannot be matched with the firm level data. Nonetheless, these firms are still considered in the evaluation of the indirect effects. To capture the effect of support to research institutions and BSIs, we modify equation (5) to estimate the impact of being geographically close to a beneficiary research institution on firm performance: = + + + + Θ × r. exposurert−1 + (5’) Here r. exposurert−1 captures firms’ exposure to research institutions and BSIs that received funding, which can be measured as the total amount of grant that location received for its research institutions and BSIs. 5 Aggregate impact and return on investment (ROI) To inform policy decisions on public spending for innovation in Poland and other countries, it is necessary to know not only the impact of the program, but also ROI. To estimate ROI, first, aggregate program impacts can be estimated using the (average) direct and indirect effects of the various programs on firms. Under the assumption of constant linear treatment effect, the aggregate benefit from the program can be calculated as a simple product of the average treatment effect multiplied by the number of treated firms across types of treatment. More specifically, the aggregate direct and indirect benefit can be calculated as follow:17 ̂ × + ̂ 1 × ) + Θ ΔY = ∑ ∑ ( ̂ × 0 + ̂ 2 × 0 (6) =1 Where ΔY denotes the aggregate impact for an outcome of interest; ̂ denotes the estimated direct treatment effect of receiving the direct support in group m starting in year k; ̂ 1 denotes the estimated indirect treatment effect from support to research institutions and BSIs; ̂ denotes the estimated spillover effects from direct support measures on firms who did not Θ receive such direct support; ̂ 2 denotes the estimated indirect treatment effect from support to research institutions for firms who did not receive any direct support; denotes the number of firms who received direct support in group m starting in year k; 0 denotes the total number of firms who never received any direct support. 17 Note that in this formulation, we ignore spillover effects on treated firms when they are not yet treated, hence the aggregate impact is likely underestimated. We also sidestep the effects of firm entry and exit, which would not be feasible to estimate without specifying a model of how firm outcomes would have evolved for exited firms. 28 Second, a “back-of-the envelope” calculation would allow us to estimate the overall cost effectiveness of the program as follow: ΔY = where program costs should include both the grant/support provided for the firms and administrative cost for all the programs.18 We are interested in evaluating ROI on the following outcomes: • $ per job gain • $ per $ value added gain (in present value) 6 Data The main firm-level data used for the analysis is the census of all Polish firms with at least 10 or more employees between 2006-2017, collected by Statistics Poland (GUS). This dataset includes information on firm characteristics (size, age, sector, location, etc) and other standard financial data on employment, output, assets, and profits. In addition, the firms census is merged with Customs data to construct measures on firm’s exports patterns. To identify beneficiaries from EU support, this data is merged with the SIMIK database that lists the beneficiaries of the OP IE, ROPs, and related programs, using the taxpayer ID number. In addition, we supplement these two main firm-level datasets with other sector and regional level data required for the analysis. First, we use 2-digit level price deflators to calculate real values of output, materials, capital, and wages. Second, we use data on education attainment and infrastructure at the regional level in specifications aimed to estimate how the impact of support is affected by regional investment climate characteristics. Finally, we use the 2009 input-output table to construct measures on vertical linkages to capture spillovers effects. Given that the data from the firm census and Custom are confidential, all of the data processing, matching, and analysis were performed by GUS with technical support from the World Bank. The full detail of the working arrangement the processes for data cleaning, management and estimation are described in the Annex 6.19 Table 4 presents the summary statistics for the main outcome variables used in the analysis. Between 2006-2016, the data includes more than firm-year 600,000 observations. Of these, over 7000 firms can be identified as beneficiaries from various support programs. 18 To calculate administrative costs, certain assumptions need to be made about how staff time/cost at the implementing agency is allocated towards different support programs. These assumptions are validated through conversations with implementing agencies. 19 Stata codes are also included in the appendix to assist the practitioner in replicating the estimation process. 29 Table 4: Summary statistics of main outcome variables, 2016-2016 Variable Observations Mean Standard Min Max Dev. ln(Employment) 622468 3.53 1.14 0.00 11.48 Export Value (‘000 PLN) 622468 73.54 1146.63 0.00 154835.67 Total Sales (‘000 PLN) 622468 458.44 4660.42 -2695.82 843176.19 Value-added (‘000 PLN) 622468 346.09 4493.85 -3303.10 1338778.17 Profits (‘000 PLN) 622468 23.09 408.57 -48934.72 82812.51 Investment in Tangible Fixed Assets 563134 7.76 364.94 -58633.18 114689.70 Investment in Intangible Fixed Assets 563134 0.45 102.12 -25486.02 39864.24 R&D Expenditure (‘000 PLN) 585740 0.30 16.31 0.00 8273.03 Average Wages (FTE) (‘000 PLN) 616579 0.44 1.13 0.00 321.64 Labor productivity 616579 4.95 29.23 -16.03 10880.20 ln(TFPR) 573147 2.33 1.07 -8.71 10.55 7 Empirical results This section presents the results for the direct and indirect effects estimation and the corresponding ROI calculations.20 7.1 Direct effects with single treatment analysis The starting point of the analysis is a simple model where we do not distinguish the types of support that firms receive but only consider whether firms received any support in the sample period. However, we allow for differences in the selection and average impact over time by estimating the impact separately for each treatment year between 2008 and 2014. As firms can receive multiple grants across time, we define treated firms in, for example, year 2008, as firms who first received any support in 2008. (see Annex 6 for a more detailed description of the estimation procedures). For each treatment year, we estimate a propensity score using a logit function of firm size class, age, sector, legal status, employment, revenue, and whether the firms reported positive exports, imports and R&D expenditure, all measured in the year immediately prior to treatment.21 Effectively, by distinguishing treatment by year, we also allow the control group to vary over time. We estimate the treatment effects using two models, nearest neighbor (NN) matching and Inverse Probability Weighting (IPW). In Figure 19, we show the density distribution of the propensity score by treated and control group in the full sample and in the matched sample using NN caliper matching for treatment year 2014 (see Annex 6 for further details). Figure 20 plots the standardized mean differences 20 For brevity, we only present results for the direct effects (both in the single treatment and multiple treatment specifications) using an ATE estimators. Results using ATT estimators are very similar in both significance in magnitude. Similarly, we also tested for robustness of results where data have been winsorized at 1% upper and lower tails. Results are broadly identical to what reported here. 21 More complex specifications with additional firm characteristics and growth rate resulted in non-convergence. Because we do not have direct access to the firm data, we were constrained in the number of specifications we could experiment with. 30 in the covariates in the full sample and matched samples for treatment year 2014. It shows that while there is substantial heterogeneity between the beneficiary and non-beneficiary firms, we were able to find a group of control firms that are similar in selected observable characteristics.22 Figure 19: Overlap in full sample and on common support, NN matching Figure 20: Covariate balance, before and after NN matching Table 5 summarizes results from the DID match model with NN matching. Each cell presents the coefficient estimate of across different outcomes from equation (3). Recall, the coefficient for each year represents the average treatment effect for firms first receiving the treatment in that year up to 2016. 22 Results on balance and overlaps are very similar across treatment years, hence we only present 2014 results here. 31 For many firms, receiving government support has a positive effect on employment, output, exports and value-added. On average, treatment effect across all year cohorts will increase employment by 14.5 percent, value-added by 18.7 percent, sales by 19.2 percent and export value by 28.7 percent. These effects do differ for each year cohort. For example, a company that received funding for the first time in 2009 will have 15.4 percent higher employment in 2016 compared to a similar company that did not receive the funding. There is some positive treatment effect on average wages especially for firms receiving government support towards the end of the program (in 2012 and 2014). There is little to no treatment effect on fixed assets investments (tangible and intangible), R&D expenditure and profits. Lastly, there are mixed results on firm productivity levels, depending on which productivity measure is used. The two productivity measures – labor productivity and TFPR – imposes different assumptions and structure on the productivity calculations. For labor productivity, with the least amount of assumptions and structure, there is a positive treatment effect and on average, the treatment effect across all year cohorts is a 2.7 percent increase in labor productivity. For TFPR, with the most structure and assumptions in the production function, there is a negative treatment effect and on average, TFPR decreases by 6.0 percent due to the treatment. Table 5: Treatment effect by outcome and treatment year Outcome 2008 2009 2010 2011 2012 2013 2014 Employment (ln) 0.197 0.154 0.143 0.154 0.116 0.0984 0.156 (0.0113)*** (0.00996)*** (0.0125)*** (0.0134)*** (0.0171)*** (0.0169)*** (0.0219)*** Sales (ln) 0.213 0.193 0.185 0.194 0.176 0.151 0.235 (0.0142)*** (0.0136)*** (0.0192)*** (0.0186)*** (0.0212)*** (0.0212)*** (0.0322)*** Export (ln) 0.332 0.327 0.272 0.249 0.193 0.341 0.297 (0.0581)*** (0.0535)*** (0.0741)*** (0.0779)** (0.0816)* (0.0843)*** (0.128)* Value added (ln) 0.213 0.189 0.170 0.197 0.184 0.144 0.212 (0.0155)*** (0.0146)*** (0.0201)*** (0.0197)*** (0.0218)*** (0.0221)*** (0.0310)*** Tangible Fixed assets Inv. 9.350 -0.869 0.294 0.148 0.120 6.433 -0.236 (9.424) (0.643) (0.129)* (0.125) (0.124) (6.325) (0.274) Intangible Fixed assets Inv. -0.000986 0.00112 0.00168 0.00209 0.00663 0.350 0.00396 (0.00200) (0.00129) (0.00108) (0.00152) (0.00396) (0.352) (0.00199)* Average wages (ln) -0.00475 0.00452 0.00910 0.00750 0.0191 0.0159 0.0390 (0.00655) (0.00595) (0.00755) (0.00745) (0.00889)* (0.00915) (0.0137)** R&D Exp. 0.000668 0.00179 0.000375 -0.000242 0.000295 0.0271 -0.000161 (0.000310)* (0.00111) (0.000756) (0.000523) (0.000705) (0.0314) (0.000270) Profits 23.52 1.455 6.527 -2.703 -2.656 2.587 0.723 (11.78)* (17.20) (11.45) (1.867) (14.21) (2.172) (1.914) Labor Prod. (ln) 0.0148 0.0330 0.0226 0.0410 0.0651 0.0467 0.0452 (0.0133) (0.0121)** (0.0166) (0.0165)* (0.0191)*** (0.0182)* (0.0268) TFPR (ln) -0.0276 -0.0421 -0.0468 -0.0525 -0.0508 -0.0585 -0.139 (0.0128)* (0.0117)*** (0.0177)** (0.0194)** (0.0230)* (0.0194)** (0.0339)*** Notes: Standard errors in parentheses where * p<0.05, ** p<0.01 and *** p<0.001. All regressions include controls on firm size, firm ownership type, sector, export participation, firm age, and regions. 32 These results are very similar with or without DID, and with or without regression adjustments (that is controlling for covariates in the matching model). The regression results with IPW are presented in Annex 5. 7.2 Direct effects with heterogeneity The analysis continues to assume one treatment group but examines examine whether the program impacts vary by firm characteristics.23 Heterogeneous impacts were examined for employment, TFPR, labor productivity, profits and R&D expenditure. There are minimal heterogenous effects for profits and R&D expenditures so these results are omitted here. Other firm characteristics were also included (such as region and export status) but there were minimal heterogeneous effects so these results are also omitted. For brevity, the full results of coefficients are not presented; the average treatment effects across all year cohorts described below. In general, there are larger treatment effects experienced by large firms, young firms (less than 3 years old) and firms in the manufacturing sector. 7.2.1 Firm size On average, large firms appear to benefit more from the support. Large firms have a higher increase in employment, albeit the difference from small and micro firms is small (0.9 percentage points) (Figure 21). The treatment effect on TFPR for large firms is positive while it is negative for small and micro firms, and about six percentage points higher than medium size firms (Figure 22). Labor productivity increases are also higher for large firms, where it is three percentage points higher than medium-size firms (Figure 23). There is no treatment effect on labor productivity for small and micro firms. Figure 21: Employment changes by firm size Figure 22: TFPR changes by firm size 20.0% 18.4% 18.8% 10.0% 8.3% 17.9% 18.0% 16.0% 5.0% Percentage increase (%) Percentage increase (%) 1.9% 14.0% 12.0% 0.0% 10.0% 8.0% -5.0% 6.0% 4.0% -10.0% 2.0% 0.0% -15.0% -13.5% Small & Micro Medium Large Small & Micro Medium Large 23 Additional results may be included in this section which examines the heterogenous effects by firm location and regional characteristics. 33 Figure 23: Labor productivity changes by firm size 7.0% 6.3% 6.0% Percentage increase (%) 5.0% 4.0% 3.6% 3.0% 2.0% 1.0% 0.0% Small & Micro Medium Large Source: Authors’ elaboration. 7.2.2 Firm Age Young firms – those that 3 years old and younger – benefit more from the support. The difference in employment increases between young firms and older firms is large (Figure 24). The treatment effect on TFPR for older firms is negative compared to a no effect for young firms (Figure 25). The increase in labor productivity is higher for young firms compare to older firms, with a difference of about 2.5 percentage points (Figure 26). Figure 24: Employment changes by firm age Figure 25: TFPR changes by firm age 0.0% 40.0% 36.6% 0.0% 35.0% -1.0% 30.0% Percentage increase (%) Percentage increase (%) -2.0% 25.0% -3.0% 20.0% 16.3% -4.0% 15.0% -5.0% 10.0% 5.0% -6.0% -6.2% 0.0% -7.0% Young (<3) Old (≥3) Young (<3) Old (≥3) 34 Figure 26: Labor productivity changes by firm age 10.0% 9.2% 9.0% 8.0% Percentage increase (%) 7.0% 6.5% 6.0% 5.0% 4.0% 3.0% 2.0% 1.0% 0.0% Young (<3) Old (≥3) Source: Authors’ elaboration. 7.2.3 Sector We find different impact on productivity in different sectors: manufacturing, trade (retail and wholesale), construction, and all other sectors (agriculture and other services). There is a positive effect for firms in the manufacturing sector and a negative effect for firms in the trade sector (Figure 27 and Figure 28). There is minimal effect for firms in construction and the remaining sectors. Figure 27: TFPR changes by sector Figure 28: Labor productivity changes by sector By Sector By Sector 30.0% 10.0% 19.0% 6.3% Percentage increase (%) 20.0% Percentage increase (%) 5.0% 10.0% 0.0% 0.0% 0.7% 0.7% 0.0% 0.0% -10.0% -5.0% -20.0% -30.0% -10.0% -40.0% -29.2% -15.0% -13.2% Source: Authors’ elaboration. 7.2.4 Grant amount We find that the impact on employment appears to increase with the grant amount that firms received. The estimated elasticity of employment to grant amount is 0.04: that is, a one percent increase in grant amount is associated with a 0.04 percent increase in employment. For comparison, moving from the 50th percentile to 75th percentile in the co-financing amount among beneficiaries translates to a tripling in co- financing amount. Thus, doubling the grant amount (i.e. a 100 percent increase) can translate to a 4.4 35 percent increase in employment. Surprisingly, we did not find evidence that conditional on treatment, grant amount has further impact on other outcomes such as productivity. 7.2.5 Business environment characteristics Finally, we test for whether the impact of support is affected by the initial business environment that the firm operates in, using proxies for regional skills supply (number of graduates and faculty in higher education) and infrastructure (public expenditure for infrastructure). The results suggest that the impact of support on most outcomes is largely unaffected by these business environment proxies. However, a fairly consistent result found is the impact on wages tend to be smaller in regions with higher skill supplies or infrastructure spending. Table 6 presents the coefficient estimates on the interaction between treatment (i.e. receiving support) and log of public infrastructure spending, number of faculty and graduates in higher education. It suggests, for example, that a 1 percent increase in public infrastructure spending is associated with a 2.4 percent lower treatment impact on wages. One interpretation is that support in less developed regions is more effective in raising wages. However, given the lack of similar results for other firm outcomes, there is not enough evidence to explain why that might be the case. Additional research would be needed to further explore and confirm the influence of business environment quality on the effectiveness of innovation support. Table 6: Wage impact changes based on business environment characteristics Average coefficient estimates (2008-2014) (Ln) Public infrastructure spending*treatment -0.0243 (Ln) Higher ed (faculty)*treatment -0.0504 (Ln) Higher ed (graduates)*treatment -0.053 Note: all coefficient estimates are significant at the 5 percent level 7.3 Direct effect with multiple treatment analysis To perform our multiple-treatment analysis, we create 2 categorial variables associated with 3 different grouping criteria. For each grouping, firms can only fall into one of the mutually exclusive treatment statuses. In the first grouping, support programs are categorized based on a combination of their objectives and grant size. In the second grouping, programs are categorized solely based on their stated expected results outlined in the program document. The grouping was done subjectively using our reading of project documents and a portfolio mapping exercise, and in consultation with Polish government counterparts. From our knowledge of the support programs, we have chosen the criteria so as to maximize homogeneity of firms within the groups while ensuring sufficient sample size for the analysis. The criteria were also chosen such that the distinction across groups has meaningful policy implications. Table 7 reports the number of observations and average firm characteristics by grouping and category. There are substantial differences in firms across treatment groups. In the first grouping, for example, on average, firms receiving grants for R&D works and implementation are larger in multiple dimensions (employment, assets, revenue, value added), pay higher wages, export more, and have higher profits. 36 These differences suggest that results from the single treatment analysis likely mark significant underlying differences in the impact across different types of support programs. Table 7: Number of firms and average outcomes by treatment category Grouping criteria 1 N Employment Total assets Wages EBT Value added Exports Revenue 1. Small K grants, low expected innovation 28,629 56 14289.26 0.39 12.63 175.28 27.61 227.42 2. Large K grants, high expected innovation 9,250 163 103041.40 0.40 65.26 494.02 303.84 939.40 3. Grants for R&D works/implementation 2908 520 348217.80 0.53 279.78 1379.20 816.21 2033.43 4. Supporting export 4,345 54 14497.26 0.44 12.97 157.92 20.70 203.53 Grouping criteria 2 N Employment Total assets Wages EBT Value added Exports Revenue 1. Increased innovation 5,337 420 315887.04 0.49 231.82 1226.20 769.40 2053.90 2. Access to finance 1,134 64 20744.15 0.40 20.56 170.62 76.68 293.65 3. Access to services 656 54 16654.35 0.50 11.14 246.52 23.95 272.25 4. Access to markets 6,193 112 37411.17 0.47 27.06 324.91 102.62 484.38 5. Overall competitiveness 34,733 99 41715.67 0.40 28.14 447.43 70.17 553.59 We estimate the relative effect of different treatment groups through an IPW DID model. The propensity score is estimated with multinomial logit, using a similar set of covariates as in the single treatment case, defined before treatment. For this model, we need to specify one common treatment year as it is not feasible to calculate the impact by year due to sample size constraints. Hence, we use 2007, the first year where the programs started to operate fully to define the treatment year (see Annex 6 for further details on the procedure). Figure 29 plots the density distribution of the estimated propensity score by treatment group, using the first grouping criteria. It shows that the estimated propensities for all treatment groups have their density mass in a similar region, suggesting the overlap assumption is not violated. 37 Figure 29: Propensity scores distribution, Multinomial logit model (2007 only) Note: T2 = 0 indicates the control group. T2 = 1,4 indicates the following 4 treatment groups: (1) Small K grants, low expected innovation, (2) Large K grants, high expected innovation, (3) Grants for R&D works/implementation, (4) Supporting export The overall direction of effect from the multiple treatment analysis is presented in Figure 30 and Figure 31. The full results are presented in Annex 6. The treatment effects of both groupings are consistent with the results with the one treatment effect. There is a positive, significant impact in most treatment groups on employment, sales, value-added, export participation and export value. Similarly, the results for productivity are mixed with positive effects for labor productivity and negative effects for TFPR. Figure 30: Summary of results for Grouping 1 Note: full results are presented in Annex 5. Positive coefficients are represented with “+” in green, negative coefficients are represented with “-“ in orange, and non-significant results are blank. 38 Figure 31: Summary of results for Grouping 2 Note: full results are presented in Annex 5. Positive coefficients are represented with “+” in green, negative coefficients are represented with “-“ in orange, and non-significant results are blank. 7.4 Indirect effects We estimate the indirect effects of support to two groups of beneficiaries: (i) firms receiving support (i.e. firms included in the direct effects) and (ii) research institutions and business support institutes. For the first group, we examine the indirect effects at the regional and industry level (i.e. horizontal, backward and forward linkages as defined in Section 4.3). We present the results that includes all four spillover measures. For the second group, we examine the indirect effects at the regional level. In both groups, we focus on how spillovers affected the non-beneficiary firms’ employment, sales, value-added, profits, TFPR and labor productivity. Table 8, panel (a) presents the results of the spillover effects from beneficiary to non-beneficiary firms. In general, we find that the programs have largely insignificant indirect effects on non-beneficiary firms and if they did, the effects are very small. First, there is no evidence of spillover effects through backward linkages, that is, through spillovers to suppliers of beneficiaries. However, there is a negative effect on non-beneficiary firms’ TFPR through forward linkages. That is, increasing support in upstream (selling) sectors has a negative effect on non-beneficiary (downstream) firms. Similarly, there is a negative effect on non-beneficiary firms’ labor productivity when there are more beneficiary firms in its sector (i.e. horizontal spillovers). These negative indirect effects suggest that beneficiary firms are exerting some negative pressures on non-beneficiary firms they sell to or those in the same sector. As we observed in the direct effects results (Section 7.2), the support programs increased the employment and sales of beneficiary firms, which will strongly increase their demand for labor and inputs. Non- beneficiary firms can face higher input prices or a tighter labor market, thereby lowering their TFPR and labor productivity. On the other hand, there is a positive indirect effect on employment when there are more beneficiary firms, regardless of sector, in the region. This indirect effect on employment is likely due to the positive demand shock experienced in the region. When beneficiary firms increase their employment, sales, and wages, demand for goods and services of non-beneficiary firms could increase. 39 Since there is a lack of evidence of spillover effects through backward linkages, the positive demand shocks are more likely a result of higher incomes of workers in beneficiary firms. The significant results in Table 8, however, indicate that the effects are very small. For example, the coefficient for the regional effect on employment is 1.67e-11. Thus, effects may be minimal as an increase of PLN 1 million in a region can result in a 0.00167 percent increase in employment in the same region (regional effect) and a PLN 1 million increase in an industry can result in 0.01 percent decrease in labor productivity in the same industry (horizontal effect). The results are slightly stronger for the indirect effects of research institutions and BSIs receiving support as presented in Table 9, panel (b). There are positive effects of support to research institutions and BSIs on non-beneficiary firms’ labor productivity, TFPR and profits in the same region. However, similar to the previous results, the magnitude of effects may be minimal. An increase in PLN 1 million of financing to the research institutes and BSIs in the region can result in: 0.17 percent increase in average firm profits, 0.00053 percent increase in labor productivity, and 0.00049 percent increase in TFPR. To summarize, these results suggest that there have been limited spillover impacts of the innovation support on other firms – either through geographical proximity, within industry interactions, or buyer- supplier linkages. Support to BSIs and research institutions appears beneficial, but still minimal in magnitude. Table 8: Indirect effects by outcome Employment Sales Value-added Profits Lab. Prod. TFPR (a) Indirect effects of firms receiving support Backward -5.43e-10** -3.97e-10 -2.23e-10 -7.80e-09 2.89e-10 2.34e-10 (2.14e-10) (3.02e-10) (3.90e-10) (2.60e-08) (2.43e-10) (2.84e-10) Forward -1.82e-10 -8.28e-10* -7.43e-10 7.20e-08 -5.27e-10 -9.82e-10*** (2.51e-10) (4.21e-10) (5.08e-10) (4.47e-08) (3.37e-10) (3.52e-10) Horizontal 0 -0 -7.71e-11 -5.87e-09 -1.07e-10** -8.35e-11 (7.49e-11) (8.36e-11) (1.01e-10) (7.31e-09) (5.04e-11) (5.45e-11) Regional 1.67E-11** 0 0 6.98e-10 -0 0 (7.17E-12) (0) (0) (2.11e-09) (0) (0) (b) Indirect effects of research institutions and BSIs Regional -0 0 0 1.72e-09** 5.27E-12*** 4.88E-12** (0) (0) (0) (5.96e-10) (1.78E-12) (2.28E-12) Note: Robust standard errors in parentheses clustered at sector level for regressions in group (a) and at the regional level for regressions in group (b). *** p<0.01, ** p<0.05, * p<0.1. All outcome variables are in natural logs and spillover measures are included in lags. All regressions contain firm, sector, year and region fixed effects. 7.5 Return on Investment To arrive at ROI estimates, we make a range of assumptions about administrative cost of the support programs to account for the full cost beyond the direct value of government support. First, on the government size, our research on program documents suggests that the administrative cost averages around 2.7 percent of total grant value. On the firm size, we rely on selected interviews with consulting firms to derive the estimates for firms to apply for funding and other M&E costs, differentiating between SMEs and large firms. Based on 2 scenarios that we received, we take the average cost. 40 In all calculations, we take into account the direct effects only, given the negligible indirect effects that we found (see section 7.4). Table 9: Assumptions on administrative costs Administrative cost as a share of grant value Government side SME 2.7% Large 2.7% Consulting cost as a Firm side share of grant value Fixed Option 1 SME 7.5% 2500 Large 3.0% 25000 Option 2 SME 8.0% 0 Large 5.2% 0 7.5.1 Cost per job estimates Figure 32 presents the cost per job gain averaged for the treatment years 2008-2014, estimated based on equation (6) and the cost assumptions described above and the average treatment effects in Table 5. These estimates assume the same impact on employment but the differences in cost per job gain reflect the different cost assumptions. While there is a lack of comparable international evidence, our finding suggests that the average cost per job gain from the innovation support programs, at 53,000 PLN and 85,000 PLN (around or less than 20,000 USD) is in line with available estimates of other programs in other countries. An estimate in Tunisia, for example, suggests that the average cost per job gain from a public investment program is at 30,000 USD.24 An estimate in the US puts the estimated cost per job gain from 240 tax break packages for large corporations at 465,000 USD.25 24 https://blogs.worldbank.org/jobs/how-much-does-it-cost-create-job 25 https://www.goodjobsfirst.org/megadeals 41 Figure 32: Cost per job gain Cost per job (PLN), 2008-2014 Large 84,916 SME 52,913 0 20,000 40,000 60,000 80,000 100,000 Source: Authors’ calculations Figure 33 present results from a similar calculation using the estimated treatment effect from the multiple treatment analysis. When the impact is allowed to differentiate across programs, we find that the cost per job gain can vary substantially between 7,000 PLN to over 100,000 PLN. In general, programs emphasizing R&D, innovation, exports, and access to markets appear significantly more cost effective in job creation than programs with small capital grants or those focusing on access to finance. Figure 33: Cost per job gain by treatment group Cost per job gain 2008-2014 (PLN) Cost per job gain 2008-2014 (PLN) - Grouping 1 - Grouping 2 0 20000 40000 60000 0 50000 100000 150000 small K grants, low expected increased innovation 16603 innovation 55718 large K grants, high expected better access to finance 101133 innovation 37937 better access to services 41754 grants for R&D work/implementation 16369 better access to markets 6552 supporting export 13171 overall competitiveness 32458 Source: Authors’ calculations 7.5.2 Value added gain per $ investment The second measure of ROI that we consider is value added gain per $ investment. We calculate the stream of discounted benefits from value added gain in $ amount in a 15-year period. The gain in value added is calculated based on the estimated treatment effect in the single treatment model (Table 5) as well as assumptions about the persistence of impact. We consider 3 different scenarios. In the first scenario (A1), impact is assumed to be constant for the first 10 years and drop to 0 afterwards. In the 2nd and 3rd scenarios (A2, A3), we assume the impact decrease by half and 1/3 every 5 years. 42 Figure 34: Assumptions on the persistence of impact on value added over time 1.2 1 0.8 0.6 0.4 0.2 A1 A2 A3 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Years after treatment Source: Authors’ calculations With these assumptions and the cost assumptions described above, our estimates suggest that $100 investment in innovation support programs can result in between $18.6 to $24.8 in value added again on average (see Table 10). Table 10: Value added gain per $ investment Benefit Cost ratio, by scenario A1 A2 A3 2008 0.17 0.15 0.13 2009 0.16 0.14 0.12 2010 0.24 0.21 0.18 2011 0.44 0.38 0.33 2012 0.43 0.37 0.32 2013 0.35 0.31 0.27 2014 0.86 0.74 0.65 Aggregate 0.248 0.215 0.186 Source: Authors’ calculations Assumption 1 (A1) assumes that the value-added gains is constant over 10 years after receiving support, and zero benefits from 11th year onwards. Assumption 2 (A2) assumes that the effect is constant over 5 years after receiving the support, halved in the 6th year and halved again in the 11th year. Assumption 3 (A3) assumes that the effect is constant over 5 years after receiving the support, reduced to 1/3 in the 6 th year and 1/3 again in the 11th year. 7.5.3 Fiscal impact Finally, we consider the fiscal implications of the increase in employment and wages due to the support programs. We focus on the potential increase in Personal Income Tax (PIT) revenue that can be generated through the increase in employment and wages. Since it is not possible to recover the full distribution of wages to determine which tax rate will apply, these calculations are based on a key simplifying assumption that all wages are taxable at the same average PIT rate of 8.7 percent, based on Poland’s average PIT rate 43 in 2007-2013.26 As mentioned above, given the minimal impact found on non-beneficiaries firms, the total gain in PIT revenue is calculated based on the direct impact on the average employment and wages in beneficiaries firms only. In addition, we estimate the total PIT gain up until 2016 as the sum of PIT gain in each year following the treatment, where annual PIT gain is assumed to be constant over time. This follows from the assumption of a constant linear treatment effect used in the estimations above. Table 11 shows the results by treatment year. Our results suggest there has been a substantial benefit to fiscal revenue due to the increases in employment. The estimated annual tax revenue gain between 2008 and 2014 ranges approximately between PLN 80-280 million. To put this in perspective, the cumulative tax revenue gains for a given treatment year (up until 2016) cover between 13 percent to 35 percent of the cost of support for that treatment year. (The cumulative gains for more recent treatment years are naturally lower due to fewer years of available data.) Table 11: Estimated gain in Personal Income Tax revenue, by treatment year Impact on Impact on Average Number of Annual Total annual Personal employment wages (%) wage bill beneficiaries wage bill wage bill Income Tax (%) per firm increase per increase ('000 gain up to ('000 PLN) firm ('000 PLN) 2016 (million PLN) PLN) 2008 20% 0% 3659.99 3620 721 2,610,085 1818 2009 15% 0% 3816.01 5505 588 3,235,099 1971 2010 14% 0% 4077.77 3572 583 2,082,909 1088 2011 15% 0% 4208.51 2335 648 1,513,338 659 2012 12% 2% 4273.33 2168 587 1,272,171 443 2013 10% 0% 4288.05 2266 422 956,125 250 2014 16% 4% 4490.29 1150 903 1,038,364 181 Source: Authors’ calculations Impact on employment and wages are taken from the direct effect estimations (see Table 5). The implied increase in the annual firm-level wage bill is calculated as Average wage bill* increase in average employment * increase in average wages. This is then multiplied by the number of beneficiaries to arrive at the total annual wage bill increase. The total tax revenue gain cumulative to 2016 is then calculated as the annual gain times the number of years in the period between treatment year and 2016. There may be fiscal implications for value-added taxes (VAT) on the increases in value-added due to the support programs. However, these fiscal implications are difficult to calculate as there may be different value-added tax rates applied on products and exemptions given to firms in certain sectors and activities. The standard VAT rate in Poland is 23 percent with reduced VAT rates of 5 and 8 percent for certain foodstuff, medical products, restaurant and hotel services, and printed books, and a zero rate for certain supplies of vessels and aircrafts and air and sea transport services. More importantly, firms engaging in international trade can claim an exemption and apply a zero VAT rate on all exports. Thus, it is difficult to arrive at a single average VAT rate, as it will miscalculate the fiscal implications of increases in value added. 26 This PIT rate is calculated based on the EC (2018)’s Taxation Trends report 44 Nonetheless, assuming an average VAT rate of 9 percent,27 the resulting increases in VAT revenue, in net present value and under the same assumptions as in Table 10, would range between 509 to 678 million PLN, or 1.9 to 2.1 percent of the total support cost. 8 Concluding Remarks The ex post evaluation of Poland’s SME and innovation support programs over 2007-13 finds that there are positive impacts on firm employment, sales, value-added and exports. This result is consistent across the different estimation models. In contrast, there is mixed evidence about the impacts on productivity, depending on the productivity measure used. There is minimal evidence that the programs affected investments (either physical assets or R&D) and profits. These findings are broadly similar to earlier studies, such as GUS (2015), which examine the impact for a more limited set of programs or firms. The estimated impact from the innovation support programs, in turn, translate to an estimated average cost per job gain that compares favorably to available estimates of other programs in other countries. Instruments emphasizing R&D, innovation, exports, and access to markets appear significantly more cost effective in job creation than instruments with small capital grants or those focusing on access to finance. Finally, evidence of the indirect effects of support on non-beneficiary firms is minimal, while the fiscal impact from increased employment and wages appears to be substantial. The limited impact on investments in physical assets or R&D expenditure may be partially driven by a crowding-out effect if firms lack the capacity to absorb capital efficiently. Instead of increasing investment, firms may choose to redirect private funds towards other purposes (for example, increase hiring). In turn, the lack of impact on investment and intangible assets in particular could be one of the reasons for the muted (mixed) effect on productivity. The limited impact on investments could be a possible driver. A recent IMF study (2019) has found that Polish firms with no or low investments in intangible assets have lower TFP growth than firms with the highest investments in intangible assets. It has also been shown in Southern Europe that large capital inflows in the short run led to an increase in capital misallocation and a subsequent decline in productivity (Gopinath et al, 2017). Pinpointing the precise causes for these results would require further analyses that is beyond the scope of this study. 27 9 percent is a rough estimate arrived at by taking the simple average of the standard 23 percent rate and the 0, 5, and 8 percent reduced rates. 45 References Ackerberg, D. A., Caves, K., & Frazer, G. (2015). “Identification properties of recent production function estimators”. Econometrica, 83(6), 2411-2451. Asker, J, A Collard-Wexler and J De Loecker (2014). “Dynamic Inputs and Resource (Mis)Allocation”, Journal of Political Economy 122:5: 1013 – 1063. Aerts, Kris and Dirk Czarnitzki (2006). The Impact of Public R&D-funding in Flanders. IWT-Studies 54. Instituut voor de Aanmoediging van Innnovatie door Wettenschap en Technologie in Vlaanderen: Brussels. Belderbos, R., Carree, M. and Lokshin, B. (2004). “Cooperative R&D and firm performance”. Research policy, 33(10), pp.1477-1492. Beņkovskis, K, O Tkačevs and N Yashiro (2019), “Importance of EU Regional Support Programmes for Firm Performance”, Economic Policy, forthcoming. Bloom, Nick, Mark Schankerman and John Van Reenen (2013). “Identifying technology spillovers and product market rivalry”, Econometrica, 81 (4): pp. 1347-1393. Bradley, S. and Migali, G., (2012). The joint evaluation of multiple educational policies: the case of specialist schools and Excellence in Cities policies in Britain. Education Economics, 20(3), pp.322-342. Breschi, Stefano and Francesco Lissoni (2003): Mobility and social networks: localised knowledge spillovers revisited. CESPRI Working Paper 142. Bocconi University: Milan. Bruhn, Miriam and David McKenzie (2017). “Can grants to consortia spur innovation and science-industry collaboration? Regression-discontinuity evidence from Poland.” World Bank Policy Research Paper No. 7934. Central Statistical Office of Poland (2015). Raport Końcowy. Ustalenie wartości wybranych wskaźników ekonomicznych dla beneficjentów Regionalnych Programów Operacyjnych, Programu Operacyjnego Innowacyjna Gospodarka oraz dla dobranych grup kontrolnych. Cheng, F., van Oort, F., Geertman, S. and Hooimeijer, P. (2014). “Science Parks and the Co-location of High- tech Small-and Medium-sized Firms in China’s Shenzhen”. Urban studies, 51(5), pp.1073-1089. Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W. and Robins, J. (2017). “Double/debiased machine learning for treatment and structural parameters”. The Econometrics Journal. Choi, S.B. and Williams, C. (2014). “The impact of innovation intensity, scope, and spillovers on sales growth in Chinese firms”. Asia Pacific Journal of Management, 31(1), pp.25-46. 46 Chudnovsky, Daniel, Andres Lopez, Martin Rossi and Diego Ubfal (2006). Evaluating a program of public funding of private innovation activities. An econometric study of FONTAR in Argentina. Working Paper OVE/WP-16/06. Office of Evaluation and Oversight. Inter-American Development Bank: Washington, D.C. Colombo, M.G. and Delmastro, M. (2002). “How effective are technology incubators?: Evidence from Italy”. Research policy, 31(7), pp.1103-1122. Czarnitzki, Dirk, Petr Hanel and Julio Miguel Rosa (2011). “Evaluating the impact of R&D credits on innovation: A microeconometric study on Canadian firms”. Research Policy, 40: pp. 217-229. Deloitte (2016). Corporate R&D Report 2016. Poland. Deloitte: Warsaw. Doraszelski, Ulrich and Jordi Jaumandreu (2013). “R&D and Productivity: Estimating Endogenous Productivity”. Review of Economic Studies 80 (2013): pp. 1338-1383. European Commission (2017). Operational Programme ‘Innovative Economy’. Poland. InfoRegio. Online: http://ec.europa.eu/regional_policy/en/atlas/programmes/2007-2013/poland/operational-programme- innovative-economy, retrieved on 08/07/2017. European Commission (2018). Taxation Trends in the European Union: Data for the EU Member States, Iceland and Norway. 2018 Edition. Fernandes, Ana P and Heiwai Tang (2014). “Learning to export from neighbors”, Journal of International Economics, 94 (2014): pp. 67-84. Ferraro, P.J. and Miranda, J.J., 2017. “Panel data designs and estimators as substitutes for randomized controlled trials in the evaluation of public programs”. Journal of the Association of Environmental and Resource Economists, 4(1), pp.281-317. George, G., Zahra, S.A. and Wood, D.R. (2002). The effects of business–university alliances on innovative output and financial performance: a study of publicly traded biotechnology companies. Journal of Business Venturing, 17(6), pp.577-609. Goerg, Holger and Eric Strobl (2007). “The Effect of R&D Subsidies on Private R&D”. Economica, 74: pp. 215-234. Gopinath, G., Kalemli-Özcan, Ş., Karabarbounis, L. and Villegas-Sanchez, C. (2017). “Capital allocation and productivity in South Europe”. The Quarterly Journal of Economics, 132(4), pp.1915-1967. Görg, H., Henry, M. and Strobl, E. (2008). “Grant support and exporting activity”. The review of economics and statistics, 90(1), pp.168-174. 47 Goodridge, Peter, Jonathan Haskel and Gavin Wallis (2012). Spillovers from R&D and other intangible investment: evidence from UK industries. Discussion paper 2012/9. Imperial College London Business School: London. Goya, Esther, Esther Vaya and Jordi Surinach (2016). “Innovation spillovers and firm performance: micro evidence from Spain (2004-2009)”. The Journal of Productivity Analysis, 45: pp. 1-22. Greenstone, Michael, Richard Hornbeck and Enrico Moretti: (2010). “Identifying Agglomeration Spillovers: Evidence from Winners and Losers of Large Plant Openings”. Journal of Political Economy, 118 (3) (June 2010): pp. 536-598. Griffith, Rachel, Stephen Redding and John Van Reenen (2004). “Mapping the Two Faces of R&D: Productivity Growth in a Panel of OECD Industries”. Review of Economics and Statistics, 86 (4): pp. 883- 895. GUS. (2015) Ustalenie wartości wybranych wskaźników ekonomicznych dla beneficjentów Regionalnych Programów Operacyjnych, Programu Operacyjnego Innowacyjna Gospodarka oraz dla dobranych grup kontrolnych. Hall, Bronwyn H., Jacques Mairesse and Pierre Mohnen (2009). Measuring the Returns to R&D. Working Paper. National Bureau of Economic Research: Cambridge. Hanel, Petr (2000). “R&D, Interindustry and International Technology Spillovers and the Total Factor Productivity Growth of Manufacturing Industries in Canada, 1974-1989”. Economic Systems Research, 12 (3): pp. 345-361. Hirano, K., Imbens, G.W. and Ridder, G. (2003). “Efficient estimation of average treatment effects using the estimated propensity score”. Econometrica, 71(4), pp.1161-1189. Hussam, R., Natalia Rigol, and Benjamin Roth (2016). “Targeting High Ability Entrepreneurs Using Community Information: Mechanism Design In The Field”. Unpublished Imai, K. and Van Dyk, D.A. (2004). “Causal inference with general treatment regimes: Generalizing the propensity score”. Journal of the American Statistical Association, 99(467), pp.854-866. Imbens, G.W. (2000). “The role of the propensity score in estimating dose -response functions”. Biometrika, 87(3), pp.706-710. Imbens, G. and Wooldridge, J. (2007). “What’s new in econometrics” lecture 5. NBER Summer Institute. IMF (2019). Republic of Poland: Selected Issues January 2019. 48 Javorcik, Beata S. (2004). “Does Foreign Direct Investment Increase the Productivity of Domestic Firms? In Search of Spillovers through Backward Linkages”. American Economic Review, 94 (3) (June): pp. 605- 627. Javorcik, Beata S. and Mariana Spatareanu (2008). “To Share or Not To Share: Does Local Participation Matter for Spillovers from Foreign Direct Investment?”. Journal of Development Economics, 85: pp. 194- 217. Javorcik, Beata S. and Mariana Spatareanu (2011). “Does it matter where you come from? Vertical spillovers from foreign direct investment and the origin of investors”. Journal of Development Economics, 96: pp. 126-138. Joffe, M.M. and Rosenbaum, P.R. (1999). “Invited commentary: propensity scores”. American journal of epidemiology, 150(4), pp.327-333. Kapil, Natasha, Marcin Piatkowski, Ismail Radwan and Juan Julio Gutierrez (2013). Poland Enterprise Innovation Support Review: From Catching up to Moving Ahead. The World Bank: Washington, D.C. Klepper, Steven (2007). “Disagreements, spinoffs, and the evolution of Detroit as the capital of the U.S. automobile industry”. Management Science, 53: pp. 616–631. Koenig, Pamina, Florian Mayneris and Sandra Poncet (2010). “Local export spillovers in France”. European Economic Review, 54: pp. 622-641. Lachenmaier, Stefan and Ludger Woessmann (2004). Does Innovation Cause Exports? Evidence from Exogenous Innovation Impulses and Obstacles Using German Micro Data. IFO Institute for Economic Research at the University of Munich: Munich. Lane, N., (2017). “Manufacturing Revolution: Industrial Policy and Networks in South Korea”. Working paper. Lechner, M. (2001). “Identification and estimation of causal effects of multiple treatments under the conditional independence assumption”. Econometric evaluation of labour market policies, pp.43-58. Lee, B.K., Lessler, J. and Stuart, E.A. (2010). “Improving propensity score weighting using machine learning”. Statistics in medicine, 29(3), pp.337-346. Link, A.N. and Scott, J.T. (2012). Employment growth from public support of innovation in small firms. Economics of Innovation and New Technology, 21(7), pp.655-678. McCaffrey, D.F., Griffin, B.A., Almirall, D., Slaughter, M.E., Ramchand, R. and Burgette, L.F. (2013). “A tutorial on propensity score estimation for multiple treatments using generalized boosted models”. Statistics in medicine, 32(19), pp.3388-3414. 49 Marshall’s Office of Pomeranian Voivodship in collaboration with WYG PSDB (2013). Ocena efektów wsparcia Osi Priorytetowej 1 ze szczególnym uwzględnieniem wsparcia przedsiębiorstw w ramach Regionalnego Programu Operacyjnego dla Województwa Pomorskiego na lata 2007-2013. Ministry of Development in collaboration with WYG PSDB (2015). Ocena efektów PO IG w zakresie ochrony własności intelektualnej. Ministry of Development in collaboration with WYG PSDB and Evalu (2016a). Ocena efektów wsparcia ośrodków innowacyjności w ramach działania 5.3 Wspieranie ośrodków innowacyjności PO IG oraz ich potencjału w świadczeniu usług proinnowacyjnych. Ministry of Development in collaboration with WYG PSDB (2016b). Ocena wpływu Programu Operacyjnego Innowacyjna Gospodarka na zwiększenie innowacyjności przedsiębiorstw. Raport Końcowy. Ministry of Development in collaboration with WiseEuropa (2017). Wpływ Polityki spójności 2007-2013 na Konkurencyjność Przedsiębiorstw I Rozwój Przedsiębiorczości W Polsce. Ministry of Development in collaboration with z ImApp (2017b). Efekty Polityki Spójności 2007-2013 w Polsce. Ministry of Infrastructure and Development (2014): Ocena efektów wsparcia dużych przedsiębiorstw w ramach realizacji polityki spójności w Polsce. Warsaw. Ministry of Regional Development (2012). Ewaluacja komplementarności i efektywności wsparcia instytucji otoczenia biznesu świadczących usługi dla przedsiębiorców. National Centre of Research and Development in collaboration with Taylor Economics (2017). Podsumowanie projektów systemowych realizowanych przez NCBR w ramach Działania 1.5 POIG. Raport Końcowy. National Centre of Research and Development in collaboration with Agrotec Polska (2017a). Podsumowanie realizacji działań na rzecz infrastruktury badawczej w ramach II osi priorytetowej PO IG. Raport Końcowy. Mitra, Arup and Amit Kumar Jha (2015): “Innovation and employment: a firm level study of Indian industries”. Eurasia Business and Economics Society, 5: pp. 45-71. Olley, G. S., & Pakes, A. (1996). “The Dynamics of Productivity in the Telecommunications Equipment”. e Polish Agency of Enterprise Development in collaboration with CEAPP UJ (2015). Analiza efektów netto wybranych Działań PO IG skierowanych do przedsiębiorstw, z wykorzystaniem podejścia counterfactual impact evaluation. BAROMETR INNOWACYJNOŚCI. 50 Polish Agency of Enterprise Development in collaboration with ARC and Exacto (2015). Ewaluacja on- going POIG w ramach BAROMETRU INNOWACYJNOSCI. Polish Agency of Enterprise Development in collaboration with WiseEuropa and OPI (2018). Ewaluacja efektów pośrednich POIG metodą przepływów międzygałęziowych. Roaf, James, Ruben Atoyan, Bikas Joshi, Krzysztof Krogulski and the IMF Staff Team (2014). 25 Years of Transition: Post-Communist Europe and the IMF. Regional Economic Issues. Special Report. International Monetary Fund: Washington, D.C. Rosenbaum, P.R. and Rubin, D.B. (1983). “The central role of the propensity score in observational studies for causal effects”. Biometrika, 70(1), pp.41-55. Rothaermel, F.T. and Thursby, M. (2005). “University–incubator firm knowledge flows: assessing their impact on incubator firm performance”. Research Policy, 34(3), pp.305-320. Schwartz, M. (2013). “A control group study of incubators’ impact to promote firm survival”. The Journal of Technology Transfer, 38(3), pp.302-331. Singh, Jasjit (2005). “Collaborative networks as determinants of knowledge diffusion patterns”. Management Science, 51: pp. 756–770. Smith, J.A. and Todd, P.E. (2005). “Does matching overcome LaLonde's critique o f nonexperimental estimators?” Journal of econometrics, 125(1), pp.305-353. Szczygielski, Krysztof, Wojciech Grabowski, Mehmet Teoman Pamukcu and Vedat Sinan Tangodan (2017). “Does government support for private innovation matter? Firm-level evidence from two catching-up countries”, in: Research Policy, 46: pp. 219-237. Wallsten, Scott J. (2000). “The Effects of Government-Industry R&D Programs on Private R&D: The Case of the Small Business Innovation Research Program”. The RAND Journal of Economics, 31 (1) (Spring): pp. 82-100. WiseEuropa and PIB (2017) Analiza Wybranych Działań PO IG Na Poziomie Sektorowym i Makroekonomicznym Za Pomocą Modelu Przepływów Międzygałęziowych. Wooldridge, Jeff. (2012) “Treatment effect estimation with unconfounded assignment”. Lecture notes from the FARS workshop, Chicago. 51 Annex 1: List of Direct Measures in Innovative Economy and Regional Operational Programs (Note: the measures in the Innovative Economy start with “POIE” and the measures in the Regional Operational Programs start with “RP” followed by two letters denoting the voivodship.) Total amount Number of Number of Measure Objective Description of support of grants contracts beneficiaries (PLN) The objective of the measure is to increase the innovation of entrepreneurs due to making use of POIE.01.04.00 Grants on R&D works in enterprises 2,363,061,036 970 728 R&D results being a consequence of the project realized within the measure 1.4. The objective of the measure is to increase access to external financing of SMEs in early stages of Risk capital funds invest in SMEs in early stages of development, POIE.03.02.00 370 144 133,8 158 158 development with particular regard to innovative particularly of innovative character or R&D activity SMEs or conducting R&D activities. Implementation of the measure will enable to support entrepreneurs The objective of the measure is to activate the in preparing them to acquire external sources of financing. The market of private investors through creating measure also aims at complementary support for networks of private favorable conditions for initiating the cooperation of POIE.03.03.00 investors and increasing entrepreneurs’ awareness of benefits and 121,088,827 231 223 private investors with SME entrepreneurs searching services offered by such networks. The support will also include the for financial resources to implement their innovative creation of platforms associating investors and entrepreneurs who undertakings. search for financing and exchange of experiences between investors. Measure 4.1 is the second stage of measure 1.4 realized within the The objective of the measure is to improve the level priority 1 which includes co-financing of projects covering technical, of innovativeness of enterprises by supporting the technological or organizational undertakings (industrial research implementation of R&D activities realized within and/or development works) conducted by entrepreneurs – on their POIE.04.01.00 1,309,891,422 424 349 measure 1.4 and the implementation of R&D own or at the request of entrepreneurs by scientific entities or other activities as a result of the program “Technological entrepreneurs able to realize research activities. Within this measure Initiative I”. co-financing includes the implementation of results of R&D activities of entrepreneurs. 52 The measure aims at strengthening enterprises Grants for R&D in companies, including transforming entrepreneurs conducting R&D works. It will contribute to the into CRD (Centres of Research and Development); support in the POIE.04.02.00 709,909,189 194 171 increase of using industrial and functional design by scope of elaborating industrial and functional designs and entrepreneurs as one of the competitive advantages. implementing them into production BGK grants SMEs a promise of technological credit on the basis of which commercial banks (credit) grants SMEs technological credit The objective of the measure is to support the from their own funds. Technological credit concerns financing the investment in implementing new technologies by investment to implement new technology, own or acquired, granting technological credit to SMEs with the POIE.04.03.00 launching production of new goods or modernizing produced goods 1,703,679,147 665 577 possibility for partial repayment from resources of or rendering new services on the basis of this technology. Payment of the Technological Credit Fund in a form of technological premium by BGK constitutes an earlier partial technological premium. repayment of technological credit borrowed by SMEs from a commercial bank. Investment projects (including necessary training and counseling The objective of the measure is to support activities) to purchase and implement new technological solutions in production and service enterprises making new production and services applied worldwide for no longer than three POIE.04.04.00 investments and projects of consultancy and training 8,065,995,998 582 539 years or having a rate of expansion worldwide not exceeding 15% courses covering acquisition of innovative (purchase of fixed assets as well as intangible and legal assets technological solutions. connected directly with purchase and exploitation of fixed assets). 53 New investments of innovative character (e.g. innovative technologies and products) including the purchase or implementation of technological solution which is applied worldwide for no longer than three years or its grade of expansion worldwide in a given branch does not exceed 15% and fulfill the following conditions (in total): a) eligible expenditures not less than PLN 160 million and b) The objective of the measure is to improve net employment growth not smaller than 150 people. competitiveness and the level of innovativeness of New investments including the purchase of fixed assets as well as economy through supporting production and service intangible and legal assets and employment growth not smaller than enterprises making new, of a large value investments POIE.04.05.00 100 people leading to the creation or development of: 3,032,072,516 126 118 of a high innovative potential generating a large a) Common services center (e.g. finances, accountancy, human number of jobs. The preferred investments will be resources management, administration, bank and insurance connected with start and development of R&D background, market research), activities in enterprises. b) IT centers. New investments with eligible costs exceeding PLN 2 million referring of the commencement or extension of R&D activity concerning purchase of fixed assets as well as intangible and legal assets and net employment growth not smaller than 10 persons of R&D staff leading to the creation or development of activities in R&D centers (e.g. engineering services centers, quality centers) The instrument aims to support companies that are planning to implement their own or acquired POIE.04.06.00 innovations. This will help to increase the use of Grants for implementation of the invention (first time in Europe) 882,179,379 97 96 inventions by entrepreneurs as one of the competitive advantages. The objective of the measure is to improve the Support of expenses related to protecting industrial property; functioning of the innovation market and the flow of support of expenses connected with a proceeding within: cancelation POIE.05.04.00 innovative solutions through popularizing of the patent, protection law for functional design or registration law; 83,317,304 512 240 intellectual property law, in particular through ascertainment of the expiry of the patent, of protection law for gaining protection of industrial property. functional design or of the registration law; The objective of the measure is to strengthen the Two types of projects: stage I – consultancy within the scope of position of Polish economy through promotion of preparing the Export Development Plan; stage II – implementation POIE.06.01.00 466,243,866 4297 3406 Poland as an attractive economic partner and a place the Export Development Plan worked out in stage I, using at least two of making business contacts as well as to increase of the following actions: a) participating in international fairs and 54 the number of Polish exports and the number of exhibition events as an exhibitor; b) organizing and participating in entrepreneurs that run an export business. economic missions abroad; c) searching and selecting partners in target markets; d) obtaining documents necessary to introduce a new product or service into a target markets; e) counseling on strategies of financing export undertakings and export activity; f) counseling on developing an image of an entrepreneur in chosen target markets. Strengthening of competitiveness of economy by improving the image of Polish economy among Organization and support for comprehensive sector programs POIE.06.05.00 international partners and providing better access to promoting Polish export specialties: goods and services, culture 385,732,625 690 547 information about Poland and possibilities of products, using different forms of export promotion developing economic relationships. The objective of the measure is to stimulate Subsidy for implementation of individual projects of micro and small development of the market of services provided in entrepreneurs, running economic activity not longer than one year POIE.08.01.00 electronic form (eServices) by means of supporting 1,350,358,571 2434 2432 aiming at provision of eServices. The project may cover creation of micro and small entrepreneurs, running economic digital products necessary for provision of this eService. activity not longer than one year The aim of the measure is stimulation of establishing The support will be granted to technical, IT and organizational POIE.08.02.00 joint business undertakings conducted in electronic undertakings, leading to implementation of business processes in 1,890,455,408 4091 3793 form. electronic form, including three or more cooperating entrepreneurs. The measure aims at creating a possibility of direct provision of access to broadband Internet service at the so called "last mile” level for the target group The measure covers projects based on co-financing construction of a (providing Internet directly for user) by means of POIE.08.04.00 dedicated tele-information infrastructure between the nearest or 789,358,471 536 306 supporting micro, small and medium sized most effective point of Internet distribution and target group(s). entrepreneurs, who plan to provide the service in areas, where such activity according to market rules is unprofitable. 55 Investments in technologies aimed at eliminating harmful impact on environment through prevention and limitation of pollution – modernization and / or replacement of electrostatic precipitators, Prevention and limitation of pollution produced by dedusting systems, desulfurization, implementation of technologies enterprises by adopting existing industrial POIS.04.03.00 reducing demand for energy, water and raw materials with special 174,609,678 17 15 installations to requirements of the best available focus on the secondary use of waste heat and elimination of waste technology (BAT) generation, investments aimed at limiting emissions of pollutants, implementation of changes required to comply with environmental regulations. Support of the investment and R&D process in RPDS.01.01.00 enterprises (SMEs in particular). Support of the Grants for investments in SMEs 824,074,167 775 647 regional tourism potential. Purchase of hardware and software enabling usage of the Internet in Increase in the use of ICT technologies among firms business activities, reorganization of enterprises in order to enable RPKP.04.03.00 22,271,261 97 91 – development of commercial e-services broader use of ICT technologies and creation of digital platforms for the exchange of information supporting development of SMEs. Grants for investments in SMEs - expansion of the companies, RPKP.05.02.00 increase the competitiveness of enterprises product diversification etc. (two sub measures - 5.2.1 targeted at 0 0 0 micro-companies and 5.2.2 targeted at all companies) adaptation of enterprises to environmental Implementation of the efficient, environmental management RPKP.05.03.00 standards and other requirements arising from 27,757,193 26 25 systems. Adoption and use technologies which prevent pollution national and EU legislation Increase of the competitiveness and the level of innovativeness of the economy by supporting RPLB.02.01.00 microenterprises in implementation of investments Grants for investment 87,332,865 206 192 with high innovative potential, generating jobs, contributing to the value of production / services. 56 Increase of the competitiveness and the level of innovativeness of the economy by supporting small and medium sized enterprises in implementing RPLB.02.02.00 Grants for investment 206,915,025 189 151 investments with high innovative potential, generating jobs, contributing to the value of production / services. Increase the competitiveness of the SME sector (micro, small and medium entrepreneurs) by acquiring the ability to adapt to changing market conditions by access to a variety of advisory services, in particular those related to quality improvement, the implementation of management systems, grants for advisory (1st type of projects), grants for participation in RPLB.02.03.00 5,187,495 154 87 strategic and operational planning, economic and missions abroad, business events etc. financial analysis and sectoral and through the internationalization of enterprises. Two types of projects were implemented (I - advisory, II - participation in missions abroad, business events etc.) Using the potential of science and R&D entities to increase the competitiveness of the region's economy and to create a "knowledge-based economy". Two types of projects were implemented - I: Purchase of equipment and development of R&D Grants for investment projects related to the acquisition of fixed RPLB.02.04.00 152,505,864 26 23 infrastructure; II: Creation or extension of assets necessary for conducting R & D work. technology parks, science and technology parks, incubators, including infrastructure and equipment of science and technology parks, industrial incubators Grants for direct investment in enterprises, understood as investment Support for modernization and development projects; Investment in multimodal transport; Support for investments in enterprises, promotion of products RPLD.03.02.00 environmental management systems for SMEs; Participation or 717,981,250 768 515 offered on target markets and creation of regional organization of participation in fairs, missions and economic events products as a recognizable regional brand abroad Grants for purchase and implementation of R&D works; Supporting the R&D activity in enterprises to Implementation and implementation of targeted projects (industrial RPLD.03.03.00 increase their competitiveness and strengthen their 101,281,513 135 100 R&D); Support of purchase of fixed assets necessary for R&D works in innovative potential in the market. enterprises,; Support of acquisition of fixed assets ne 57 Increasing the competitiveness of micro and small Grants for direct investment in micro and small enterprises, enterprises. Measure was designed for micro- and understood as investment projects (including necessary training or small enterprises that operate for more than 12 advisory activities as part of investment projects). Grants for RPLD.03.06.00 109,183,952 721 616 months and was aimed to support running the participation of entrepreneurs in international fairs, exhibitions and business or change the profile of the business business missions abroad, including the preparation and presentation activity. of the company's offer. Purchase and implementation of assets necessary for providing on- line services using modern ICT tools (e.g. e-commerce, e-education and training, networking), support of research in the field of technical Investment support for SMEs in the area of ICT in RPLD.04.03.00 and organizational processes, which result in the implementation of 51,467,906 154 145 order to increase their competitiveness B2B digital platforms and co-financing of projects in the field of production and television transmission in the digital technology with development of on-line viewing platforms investment projects in the companies - purchase of machinery and RPLU.01.01.00 Increasing the investment capacity of start-up's 101,122,302 406 382 equipment Improvement of the competitiveness of micro- investment projects in the company - necessary to conduct RPLU.01.02.00 enterprises, operating for no more than two years, 321,967,609 715 628 innovative projects implementing innovative investment projects Improvement of the competitiveness of small and investment projects in the company - necessary to conduct RPLU.01.03.00 medium-sized enterprises implementing innovative 466,795,505 468 361 innovative projects investment projects Adaptation of enterprises to environmental Implementation of the efficient, environmental management RPLU.01.04.00 standards and other requirements arising from 0 0 0 systems. Adoption and use technologies which prevent pollution national legislation and EU. The objective is to increase the competitiveness of Support for the development and implementation of new tourism RPLU.01.05.00 micro, small and medium-sized enterprises operating regional products/expansion of existing tourism products; 184,325,400 153 148 in the Lublin province in the tourism sector Construction and expansion of accommodation facilities 58 Investments in fixed assets, i.e. the infrastructure and laboratory equipment, and intangible assets related to the transfer of Support of the development and transfer of modern technology through the acquisition of patent rights, licenses, know- RPLU.01.06.00 technologies to enterprises. Strengthen the capacity 6,207,060 27 20 how, including technical knowledge used to conduct R&D activities in of enterprises in conducting R & D works enterprises; Investments in fixed assets, i.e. the infrastructure and laboratory equipment, used to conduct R & D activities in enterprises Specialized consulting services for companies provided by accredited Increase of the competitiveness of enterprises by RPLU.01.07.00 consultants (consulting on running business in the EU, entering new 6,723,320 119 100 facilitating access to specialized advisory assistance markets, establishing cooperation, etc.). Participation of the enterprises from the SME sector in exhibitions RPLU.02.04.00 promotion of the attractiveness of the region 0 0 0 and fairs (including foreign). The research part provides for the financing of projects involving Increased transfer of innovation to the economy by industrial research and / or experimental development carried out by RPMA.01.02.00 supporting investment in research and development 21,868,487 24 24 both entrepreneurs themselves and commissioned / redeemed from projects scientific units. Improve of the competitiveness of micro and small Implementation of the new investments in SMEs involving: fixed enterprises by adapting to market requirements, assets and intangible assets connected with creation of a new RPMA.01.05.00 830,850,952 974 865 including ensuring access to new technologies, enterprise; expansion of an existing enterprise; diversification of the certification and quality systems. company's production by introducing new product Business Support for Best Available Techniques (BAT) will address: technology changes to eliminate harmful effects by preventing and reducing emissions to the environment; Changing technologies to Prevention of the formation and reduction of reduce energy, water and raw materials demand, with particular RPMA.01.08.00 pollution of various components of the environment emphasis on the secondary use of waste heat and the elimination of 28,806,270 22 19 by adapting companies to BAT requirements waste generation; Changes in technology aimed at limiting the emission of certain substances and energy to the level specified in national and Community legislation and in the BAT reference documents; 59 Implementation of IT integrated enterprise resource planning Improving the competitiveness of micro-enterprises systems (ERP), implementation of IT customer relationship RPMA.02.03.00 and SMEs through providing access to ICT management systems (CRM), provision of advisory services in the 38,908,863 126 123 technologies area of application of ICT technologies in the enterprise, construction or reconstruction of ICT infrastructure in the enterprise Creation of a strong and competitive SME sector through direct financial assistance and providing alternative sources of financing for business operations. In addition, a strong and well-functioning business environment is an important factor supporting the development of enterprises. 4 RPMP.02.01.00 grants for investment in different groups of beneficiaries 0 0 0 schemes of support were implemented in the measure: 1) Direct investment support in SMEs 2) Creation of cooperation links between enterprises, including creation of clusters 3) Grants for business environment institutions 4) Support of financial instruments for enterprises Increase of the level of innovation and competitiveness of companies through the Grants for: 1) implementation and / or purchase of industrial research implementation of research projects in a results, experimental development work, including the purchase of RPMP.02.02.00 cooperation with a science sector. 2 schemes of 0 0 0 the necessary research infrastructure 2) investments in fixed assets, support were implemented within the measure: 1) intangible assets necessary to implement R&D results research projects 2) Investment projects in R&D in companies 60 1. Grants to individuals intending to start a business and establish a micro-enterprise (StartUp). 2. Micro grants for investment expenditures related to: Promotion of employment by stimulating the - development of the micro-enterprise; formation of new and development of existing micro - construction, reconstruction and renovation of micro-enterprise enterprises; Increasing the competitiveness of micro infrastructure, inter alia aimed at expanding the scope or RPOP.01.01.02 106,197,104 410 375 enterprises in the Opolskie Voivodeship by diversification of economic activity (diversify); modernizing their product and technological offer - purchase and modernization of equipment necessary for running a and increasing their investment opportunities business; - purchase of intangible assets related to project implementation. 3. Advisory services related to the introduction of innovation in micro-enterprise aimed at increasing its competitiveness Investment in fixed assets and intangible assets related to the establishment of a new company; Expansion of the existing company Improvement of competitiveness of companies from (including the introduction of new solutions for the sale of products RPOP.01.03.02 Opolskie voivodship through support of investment 253,733,719 212 160 and services); Support for diversification of business activity by leading to higher innovativeness introducing new additional products or making a substantial change to the production process in existing enterprise Support for new investment in companies - investment in fixed assets Increasing the competitiveness and innovativeness and intangible assets related to: creation of a new enterprise; of SMEs operating in the Podlaskie Voivodeship by RPPD.01.04.01 Expansion of an existing enterprise; Diversification of company's 107,664,155 318 275 supporting their development projects - micro production by introducing new, additional products; A fundamental enterprises change in the production process in an existing enterprise. Support for new investment in companies - investment in fixed assets Increasing the competitiveness and innovativeness and intangible assets related to: creation of a new enterprise; of SMEs operating in the Podlaskie Voivodeship by RPPD.01.04.02 Expansion of an existing enterprise; Diversification of company's 294,923,798 287 220 supporting their development projects. - small and production by introducing new, additional products; A fundamental medium-sized enterprises change in the production process in an existing enterprise. 61 Scheme A: Capital support of guarantee and loan funds and other financial instruments to support businesses, Scheme B: Direct investment subsidies to improve the competitiveness of micro, small and medium enterprises, for: investments primarily based on Support the development of existing enterprises by innovations and knowledge and the purchase of patents and new increasing the availability of external financing. Two RPPK.01.01.00 technologies; Investments in the field of computerization, as well as 837,279,323 1008 833 schemes of support - Schema A: Support for financial the application and use of e-business technology and information and instruments Scheme B: direct support for enterprises communication technologies (ICTs); Investments in the scope of changes in the production process or mode of providing services, as well as modernization of the equipment necessary to conduct business activity etc. Support for expansion of a company's business; Implementation of fundamental changes in the production process; Investment in fixed assets necessary to run and develop a company's business; Rationalization of the logistics of selling goods and services; Improvement of the investment capacity of Construction, extension or reconstruction of installations and RPPM.01.01.00 enterprises through co-financing of investment equipment conducive to the saving of raw materials and energy and 0 0 0 projects located in the Pomeranian Voivodship. the reduction of harmful emissions into the environment; IT investments including enterprise management systems; Adapting companies to national and European standards (certification of products and services); Grants for participation in national and international economic events such as fairs and exhibitions 62 Support for purchasing R&D results, intellectual property rights, including patents, licenses, know-how or other unpatented technical knowledge related to the product or service being implemented; Support for obtaining exclusive rights (e.g. patents) for own technical solutions; Investment support for R&D activities in enterprises; RPPM.01.02.00 Development of innovative enterprises 80,943,725 113 103 Support for implementing innovative processes of manufacturing products and services, organizational systems and market solutions, including the implementation of R&D results; Support for implementation and commercialization of innovative products and services and product and technology platforms. Support for expansion of an existing enterprise leading to the introduction of new or improved products / services; Support for fundamental changes to the manufacturing process in the company. Adaptation of technology and products to the requirements of the EU directives, in particular the harmonized standards and health and RPSL.01.02.01 Increase of the competitiveness of enterprises 45,062,832 393 351 safety legislation, environmental protection; Application and use of e- business technology; Application and use of information and communication technologies (ICTs) in enterprise management processes; Entrepreneurs' participation in international fairs and exhibitions and trade missions related to fairs and exhibitions abroad. 63 Support for expansion of an existing enterprise leading to the introduction of new or improved products / services; Support for fundamental changes to the manufacturing process in the company. Adaptation of technology and products to the requirements of the EU directives, in particular the harmonized standards and health and safety legislation, environmental protection; Application and use of e- business technology; Application and use of information and communication technologies (ICTs) in enterprise management processes; Entrepreneurs' participation in international fairs and RPSL.01.02.02 Increase of the competitiveness of enterprises 155,443,626 523 386 exhibitions and trade missions related to fairs and exhibitions abroad. Advisory services on new products or services, marketing plans, company development strategy, quality management and other management support systems, environmental management, health and safety and certification of products, services, raw materials, machinery and equipment. Advisory services on developing export activities. Consulting services on the use and use of information technology in the enterprise. Advisory services on mergers between small and medium-sized enterprises Support for implementation and commercialization of innovative technologies and products; Support for undertaking or developing R&D activities in micro, small and medium enterprises; Consulting services for implementation of enterprise development strategy RPSL.01.02.03 Increase of the competitiveness of enterprises based on new technologies and innovative solutions. Advisory 229,733,196 488 428 services on the establishment of high technology companies or intending to start a business consisting in the production of new products or services on the Polish market or significantly improved compared to existing ones on the market. 64 Support for expansion of an existing enterprise leading to the introduction of new or improved products / services; Support for fundamental changes to the manufacturing process in the company. Adaptation of technology and products to the requirements of the EU directives, in particular the harmonized standards and health and safety legislation, environmental protection; Application and use of e- business technology; Application and use of information and communication technologies (ICTs) in enterprise management processes; Entrepreneurs' participation in international fairs and RPSL.01.02.04 Increase of the competitiveness of enterprises 399,129,744 1657 1186 exhibitions and trade missions related to fairs and exhibitions abroad. Advisory services on new products or services, marketing plans, company development strategy, quality management and other management support systems, environmental management, health and safety and certification of products, services, raw materials, machinery and equipment. Advisory services on developing export activities. Consulting services on the use and use of information technology in the enterprise. Advisory services on mergers between small and medium-sized enterprises Financing of investments related to business development, modernization of equipment related to economic activity, introduction of technological and / or organizational innovations, implementation of R&D results. Support for implementation and commercialization of innovative technologies and products. As part of Increase of the innovativeness and competitiveness RPSW.01.01.00 the investment project, the costs of employment could be financed 412,133,249 542 485 of micro, small and medium sized enterprises (including partial coverage of staff salaries plus mandatory social security contributions for a period of no more than 2 years). As a complement to the investment project, it was possible to subsidize the organization or participation in trade fairs and exhibitions and trade missions related to fairs and exhibitions abroad. 65 Strengthening the competitiveness and Co-financing of joint investment projects undertaken by a group of innovativeness of enterprises in the Swietokrzyskie companies; Co-financing the costs of use of communication voivodeship through cooperation and technologies (ICTs) in the processes of industrial cluster management; RPSW.01.02.00 51,400,465 21 20 implementation of joint ventures carried out by Co-financing the costs of purchasing research and development groups of companies, industrial clusters or corporate results or industrial property rights by a cluster of companies. Co- consortia financing of the costs of extension of clusters of regional importance. Co-financing of the creation of new or extension of existing loan Support for the development of enterprises and funds and guarantee funds. Recapitalization of loan funds providing growth of economic activity of the region's RPSW.01.03.00 assistance to companies operating in the Świętokrzyskie Voivodeship. 116,000,000 7 6 inhabitants by facilitating access to external sources Recapitalization of loan guarantee funds providing assistance to of investment financing companies operating in the Świętokrzyskie Voivodeship. Investment subsidies for micro, small and medium-sized enterprises, Increase of the competitiveness of enterprises by including: major changes in production process or change in the promoting products and processes environmentally RPWM.01.01.05 provision of environmental services - purchase of equipment 20,919,506 40 40 friendly; increasing the investment capacity of necessary to modernize and develop the company's environmental enterprises friendly operations Support for investments of large companies focused Grants for investments of large companies, focused on new methods on new methods of production (process innovation). of production (process innovation). Support for delivery of new or RPWM.01.01.06 Support for delivery of new or significantly improved 64,381,236 27 21 significantly improved goods and services (product innovation) or goods and services (product innovation) or organizational changes (organizational innovation) organizational changes (organizational innovation) Grants for new investments for micro, small and medium sized enterprises aimed at increasing their competitiveness by building or RPWM.01.01.07 Increase of the competitiveness of enterprises 230,738,088 347 272 expanding their infrastructure, equipping with modern equipment and technologies Support for joint industrial and scientific projects, i.e. industrial and RPWM.01.01.08 Increased competitiveness of enterprises pre-competitive research completed with the implementation of the 20,206,412 9 7 results of work in enterprises. 66 Investment grants for micro, small and medium-sized enterprises, including: extension of business activities; Making fundamental RPWM.01.01.09 Increased competitiveness of enterprises 372,612,012 729 622 changes in production process; Purchase of equipment necessary for the modernization and development of the company's business. Supporting the creation of regional products connected with the Warmińsko-Mazurskie Voivodeship; Preparation of promotion Increased competitiveness of enterprises through programs for branded products related to Warmińsko-Mazurskie financial support directed at the creation of regional RPWM.01.03.00 Voivodeship; Projects related to regional marketing (promotion of 20,058,572 34 23 products and recognizable brands and their regional economy through organization of regional and supra-local promotion with the use of modern marketing tools. events); Participation in fairs, conferences, seminars, exhibitions, etc.; Organization and promotion of local and regional events Development and promotion of e-services in following areas: e- commerce, tourism, cooperation networks, creation or participation in Network-Virtual-Organizations, creation and development of IT Intensification of the development of high-quality tools and systems in SMEs leading to increased competitiveness RPWM.07.02.02 digital services and applications for small and (including integration of business activities between cooperating 66,925,949 214 153 medium-sized enterprises in the region companies, information exchange, e-circulation of documents, electronic documents archiving systems, database systems), establishment and development of social websites and portals as well as on-line services for companies. Support for investment projects (scheme I): Investment in fixed assets; Investment in intangible assets (investment in technology transfer through the acquisition of patent, license, know-how or non- The economic and social development of the region patented technical knowledge); Support for advisory projects through direct support to micro-enterprises in the (Scheme II): advisory on obtaining and renewing certificates of RPWP.01.01.00 initial period of operation (up to 2 years of activity) conformity for products, services, raw materials, machinery and 0 0 0 by increasing their investment capacity and equipment, control and measuring equipment or qualifications of supporting creation of new jobs personnel; The use of advanced information technology in the enterprise; Introduction of the product by the enterprise to new foreign markets; Gaining external financing for business development, etc. 67 Support for investment projects (scheme I): Investment in fixed assets; Investment in intangible assets (investment in technology transfer through the acquisition of patent, license, know-how or non- patented technical knowledge); Support for advisory projects The economic and social development of the region (Scheme II): advisory on obtaining and renewing certificates of through direct support to SMEs in order to increase conformity for products, services, raw materials, machinery and the level of business innovation, expand their RPWP.01.02.00 equipment, control and measuring equipment or qualifications of 0 0 0 business, to allow for more responsive behavior to personnel; The use of advanced information technology in the the needs of the regional economy. Job creation in enterprise; Introduction of the product by the enterprise to new line with the Europe 2020 Strategy foreign markets; Gaining external financing for business development, etc. Scheme III: Support for implementation new technologies in enterprises (Used in the world for not more than 3 years) Investment projects located in the Zachodniopomorskie Voivodeship aimed at improving the company's competitiveness by: expanding the business scope, diversifying the production or service of the increase of the competitiveness of micro-enterprises RPZP.01.01.01 enterprise by introducing new products and services, changing the 84,383,117 315 300 through increasing their investment potential product or service; Fundamental changes in the production process or change in the way of providing services, start-up of the activity of newly established companies Investment projects located in Zachodniopomorskie Voivodeship aimed at improving the company's competitiveness by: expanding the business scope, diversifying the production or service of the increase of the competitiveness of SMEs through RPZP.01.01.02 enterprise by introducing new products and services, changing the 182,531,285 217 191 increasing their investment potential product or service; Fundamental changes in the production process or change in the way of providing services, start-up of the activity of newly established companies 68 Improving the competitiveness and innovation of Investment projects located in Zachodniopomorskie Voivodeship micro, small and medium-sized enterprises by aimed at improving the company's competitiveness by: expanding the implementing new technologies in the enterprise business scope, diversifying the production or service of the RPZP.01.01.03 that will enable the production of new or enterprise by introducing new products and services, changing the 263,253,620 190 164 significantly improved products, processes or product or service; Fundamental changes in the production process services and are not used in the world for more than or change in the way of providing services, start-up of the activity of a 5 years. newly established companies Support for consulting services on the use of advanced information Sub measure 1.3.1: increase of the competitiveness technology in the enterprises; Consultancy on development of of micro, small and medium-sized enterprises by enterprise development strategy; Consultancy on obtaining external RPZP.01.03.01 2,686,502 132 118 facilitating their access to specialist consulting financing for business development - excluding equity financing; services Consulting on the quality management in the enterprises; Investment planning consultancy Acceleration of the process of internationalization of Support for establishing contacts with potential foreign business RPZP.01.03.02 West Pomeranian enterprises and promotion of the partners and development of export activities. Participation of 3,298,190 154 93 region abroad enterprises in international fairs and exhibition events Source: Total amount, and number of contracts and beneficiaries based on own calculations from SIMIK database Note: 1. Asterisk (*) indicates the number of beneficiaries the total number of grants for the whole measure. Final number of beneficiaries according to SIMIK database. Total number of grants according to period report (30.06.2016). 2. The measure code denotes whether it is a national or regional program. Measure codes that start with “PO” is national programs; measure codes that start with “R” are regional programs. 69 Annex 2: List of Indirect Measures in Innovative Economy and Regional Operational Programs Measure Objective Description of support Total amount Number of Number of of grants (PLN) contracts beneficiaries Research Institute POIE.01.01.00 The objective of the measure is to direct Grants for research in most cases with the use of the foresight 2,028,836,019 77 53 research for scientific fields and disciplines approach which may have a great influence on the country’s civilization and economic development and for the establishment of a knowledge-based economy. POIE.01.03.00 The objective of the measure is to acquire and Grants for science studies which can be directly applied in the 1,598,000,340 432 112 increase the use of new solutions essential for business the development of economy and improvement of the competitive position of entrepreneurs and development of the Polish society. POIE.02.01.00 The objective of the measure is to develop the Grants for R&D infrastructure (laboratories, equipment, etc.) at 3,406,573,871 51 35 infrastructure of scientific entities in centres of scientific entities, universities, etc. high research potential enabling conducting high quality research. POIE.02.02.00 The objective of the measure is to develop the Co-financing of the investment projects meaning: construction 1,376,585,894 16 15 research infrastructure facilities to establish of a new common scientific and research equipment; scientific cooperation between different transferring of scientific and research equipment. national research centers and to gain greater benefits from funds and technical infrastructure due to synergy of measures. POIE.02.03.00 The objective of the measure is to provide IT investments 1,962,238,395 99 69 Polish scientific circles with permanent and safe access to advanced IT infrastructure, to enable conducting modern research using technologies of information society and provide scientific entities seated in Poland with communication with international scientific telecommunication networks. POPW.01.01.00 Preparation of universities to active Construction and purchase of a scientific and research 1,587,502,031 26 21 participation in creation of a competitive equipment, including laboratories, computer laboratories, economy. libraries; Construction and other construction work on university infrastructure (e.g. student housing, administrative facilities) 70 RPKP.05.04.00 Exploiting the potential of science and R&TD Financial support for projects aimed at technology transfer and 144,793,495 15 11 units for increasing the competitiveness of the improvement of cooperation networks including equipment region. resources and resources related to the creation and development of cooperative relations between enterprises and R&D institutions. In addition to projects aimed at the purchase R&D services by companies. RPLD.03.01.00 The objective of the measure was to support Grants for purchase or production of research equipment, 2,354,778 3 3 research and development activities of investments related to the creation or maintenance of scientific units. information science infrastructure, including advanced regional network infrastructure and the development of advanced ICT applications and solutions in scientific units RPLU.02.02.00 Increase of the potential of regional Projects on a specialized laboratory base exclusively used for 598,079 1 1 universities and research units in the field of commercial research in favor of companies interested in commercial R&D acquiring new technological solutions and providing specialized services with documented market demand. RPMA.01.01.00 Increased transfer of innovation to the Grant for construction, reconstruction or modernization of high- 394,193,315 30 22 economy by strengthening the infrastructural tech research facilities, including those operating on the basis of potential of the research and development a scientific-industrial consortium necessary for research and sphere development; investments in equipment and fast IT networks connecting research centers; Purchase of specialist equipment and intangible assets necessary for conducting research and development in laboratories RPSW.02.01.00 Improving the competitiveness of the regional Investments related to the construction, modernization and 176,736,375 52 14 economy by raising the level of innovation, equipment of laboratories, carried out by scientific units; including increasing the transfer of modern Customizing laboratories to the requirements of EU directives, technological, product and organizational in particular harmonized standards and health and safety solutions to enterprises and institutions. legislation, environmental protection. Construction, reconstruction and modernization of didactic and research facilities of universities; Modernization of specialized facilities of health care institutions RPWM.01.01.01 Increase of the innovation potential in the Investments in R&D infrastructure in the area of research for 12,332,141 5 1 region and thus the competitiveness of construction services regional enterprises by promoting cooperation with sphere of science, research and development and technology transfer in the construction services sector. RPZP.01.02.02 support of R&D activity by enhancing the Support for projects concerning investments in the equipment 57,883,811 18 14 ability of entities (higher education of specialized research laboratories. It should guarantee the use institutions, research units etc.) to provide of a laboratory base for research conducted for companies research and development services interested in gaining new technological solutions and providing specialist services with documented market demand. Sub- measure 1.2.2 provides support for projects involving the development of research and development infrastructure of 71 research units destined for R&D activities, should be used in the business practice RPDS.02.01.00 Improvement of access to broadband and Construction or expansion of telecommunications backbone and 171,760,572 118 10 secure Internet in the Lower Silesian distribution, implementation or expansion of access points, Voivodship. purchase of equipment and modernization of network management centers, establishment of Public Internet Access Points (PIAP), construction and implementation of secure data transmission systems using modern technologies, organization of international and interregional cooperation (seminars, conferences) on exchanging good practices and experiences in the area of information society. RPMP.05.01.00 Strengthening the role of Kraków Metropolitan Preparation of the concept of establishing of innovation centers 120,038,028 6 4 Area as a thriving research and development in the region, investments in the creation of innovation centers center as well as the center of modern and in the region, construction, extension or modification of innovative economy laboratories and modernization of equipment for research laboratories, establishment of laboratories that provide specialized services for enterprises with documented market demand, creation of certified laboratories and support of laboratories applying for certification. RPPM.02.02.01 Creation of the basis for the development of Construction and extension of regional and municipal IT 48,846,914 79 4 the information society backbone and distribution networks, in particular based on fiber optic technology, construction and extension of existing local broadband access networks outside of urban areas, in particular based on wireless technology, establishment and extension of urban broadband access networks (METRO Ethernet), creation of Public Internet Access Points (PIAP, hot-spots), development of Internet telephony (VoIP) for use in public institutions RPWM.07.01.00 Provision of access to broadband Internet to Construction, extension and modernization of a broadband 95,026,105 144 16 households, enterprises, administration units Internet network at the level of housing estate, universities, and educational institutions throughout the schools, comune and poviat, extension of the fiber-optic entire region, especially where investments of network at the local level through synergy with public private investors are not profitable investments in the construction, repair of roads, water supply, sewage, heating networks etc., construction of wireless broadband Internet access networks, establishment of new and further development of existing Public Internet Access Points (PIAP), creation and development of IT network management platforms, provision of audit and evaluation activities of implemented ICT solutions. RPWP.02.08.00 Creation of the possibility of direct broadband Construction, extension or modernization of broadband 108,392,240 137 8 access to the Internet in the whole region, in backbone and distribution networks, construction, extension or particular in areas which are at risk of digital modernization of ICT management centers with purchase of exclusion necessary equipment. 72 RPWP.02.09.00 Creation of conditions for provision of public e- Implementation of internal networks in public institutions, 50 4 services by public administration construction of regional e-healthcare services platform, implementation of wireless network infrastructure in higher educational institutions, purchase of equipment necessary for conduct of e-learning activities by universities. Business Support Institutes POIE.03.01.00 The objective of the measure is to increase the The implementation of the measure comprises the selection of 840,257,272 77 69 number of enterprises operating on grounds of institutions that will provide two-stage support for newly innovative solutions. founded innovative enterprises. These institutions search and select innovative ideas, help in establishing a new enterprise (so-called pre-incubation) and subsequently invest in the newly founded enterprise. The co-financing of the project comprises two components that is the subsidy for the pre-incubation and for the capital reinforcement of the newly founded entrepreneur. The capital entry occurs in the situation when the results of actions taken up in the scope of pre-incubation will indicate the economic reasonability of conducting economic activity based on the innovative solution and the considerable chances of a commercial success of a newly founded enterprise are identified. The capital entry can only include entities, which went through the stage of pre-incubation (it is impossible to take part in the second stage only). The resources obtained after completing the investment (i.e. the exit from the investment) are sent to the institution supporting the establishment of innovative enterprises with the intention to continue the activity of the same character. POIE.05.02.00 Better accessibility of comprehensive, high Support for a Business environment institutions (BEIs/IOBs) - 281,581,773 34 23 quality business services from the point of promotion of the cooperation within the network, exchange of view of innovative activity for the experience, common customer service and the development of entrepreneurs from all over the country. pro-innovative services. POIE.05.03.00 The measure consists in supporting the Support for creation and development of science and 1,192,969,947 12 12 creation and development of centers located technology parks aimed at making a favorable conditions for the in areas of high development potential. development of enterprises from the new technology areas Support for abovementioned centers is to operating on modern solutions. provide access to comprehensive services for both entrepreneurs aiming at implementation of new solutions and scientists who want to start an economic activity. 73 POIE.06.02.00 For entrepreneurs across the country Support for COIE including preparation and development of 164,286,484 98 77 facilitating the access to comprehensive, high counselling services package within carrying out export and quality services within internationalization of investment activities the economic activity and increasing the level of investments through boosting the attractiveness of localization for investment projects. POPW.01.03.00 Improvement of conditions for business Scheme I: Support for the development of innovation centers - 2,313,563,849 84 56 operations - development and diffusion of projects involving the construction, extension, modernization innovative undertakings. and start-up of industrial parks, technological parks and business incubators. Construction, extension and start-up of centers of excellence and centers for technology transfer; Scheme II: Support for equipment - projects involving the construction of facilities and the execution of other construction works in facilities for the continuous pursuit of scientific research or development and the furnishing of such facilities; Scheme III: Support for R&D infrastructure - R&D infrastructure projects in enterprises; Scheme IV Support for the preparation of investment areas - projects involving the preparation of areas for investment activities related to parks and incubators or the preparation of investment areas for the creation of production areas and zones of modern services. RPDS.01.02.00 The objective is to strengthen the potential of Grants for business advice for enterprises and business 3,663,906 94 88 the Lower Silesian enterprises, in particular environment institutions. Grants for participation in business SMEs, by providing expert consultancy, to events (Fairs, missions) foster new business contacts and increase the capacity of Business environment institutions RPDS.01.04.00 Strengthening the potential of research and Investment projects aiming at establishment and development 90,334,193 24 14 development sector (including infrastructure) of infrastructure for innovation, e.g.: science and technology to support innovation and entrepreneurship in parks, incubators, technology transfer centers, innovation the region. centers, etc. Creation and development of the infrastructure of scientific institutions, centers of excellence which are involved in research and development works for companies RPKP.05.01.00 Providing high quality services to facilitate IV types of projects: I Establishment and management (acting as 249,542,319 23 14 running and growing a business. Facilitating the Manager) Trust Funds; II projects aimed at improving the access to external sources of finance for SMEs quality and access to services related to advanced business in the region support; III Supporting existing loan funds (micro-lending) 74 RPLB.02.05.00 Stimulating the growth of the economic Grants for investment projects related to 1) projects for the 46,700,519 12 9 activity of the region and facilitating the development of regional and local business environment functioning of enterprises in open market institutions, including the creation of new IOBs (investment conditions through business environment projects); and 2) - projects for the recapitalization of loan and institutions. Two types of projects were guarantee funds. implemented: 1st type - projects for the development of regional and local business environment institutions, including the creation of new IOBs (investment projects); Type II - projects for the recapitalization of loan and guarantee funds. RPLD.03.04.00 Developing a business environment to improve Grants for development of business environment institutions, 281,212,183 29 25 collaboration between science and business supporting cooperation of science and business (technology and facilitating access to finance for transfer); Recapitalization or creation of financial instruments; companies. Support for study and conceptual works on the creation of investment areas RPLU.02.03.00 supporting business environment institutions 1. Transfer and commercialization of new technologies in 0 0 0 and knowledge transfer through investment technology transfer centers, 2. The development of new and and advisory services. strengthening existing business incubators, technology incubators and technology centers aimed at new enterprises, 3. The creation of clusters of entrepreneurs within the framework of technological parks, business zones, industrial parks and clusters etc. RPMA.01.04.00 Development of business environment (IOB) Support for the establishment and development of business 214,978,835 9 8 networks and increased availability of environment institutions; The support of industrial parks and consultancy and advisory services, and technological parks of both new and existing ones; Support for increased availability of companies for external recapitalization of loan and guarantee funds capital through the creation of a financial support system for enterprises RPMA.01.06.00 Development of a network of business links by Creation and development of regional clusters; Advice on 55,562,563 15 12 supporting the emergence and development of development of cluster development plans; Supporting the clusters and cooperative relationships between activities of cluster operators; Cluster promotion activities to enterprises as well as between enterprises and attract new businesses to participate in the cluster; the R&D sphere. Implementation and commercialization of innovative technologies and products. RPOP.01.01.01 Sub measure 1.1.1: Strengthening the business Investments that allow improving the quality of services 85,514,277 32 25 environment and facilitating business access to provided to enterprises or enable the provision of new services professional business services to increase the related to the creation and / or extension of the scope of competitiveness of enterprises in Opolskie activities of business environment institutions (including Voivodship. clusters and branch business centers). Investments in business environment institutions; Land development around the business environment institutions; Purchase of equipment necessary to conduct business activities of these units. 75 RPOP.01.03.01 Sub measure 1.3.1: Increasing the offer of R&D Purchase of machines, equipment and intangible assets for 32,782,327 27 5 entities for the economy; Encouraging laboratories in research institutions and other scientific units enterprises to use the results of R&D as a conducting R&D works crucial factor for enterprise development. RPPD.01.01.00 Increase of the competitiveness of the region's Support for the creation and development of infrastructure 89,334,568 24 20 economy through the development of adapted to the needs of technologically advanced companies, innovative potential of enterprises and other i.e. technology parks, science and technology parks and entities as well as support for the R&D sector industrial incubators. Support for projects in the area of creating centers: investor service, promotion, professional development, advanced technology, research and development, technology transfer, competence, logistics, etc. The construction of parks, incubators and centers includes development of the area around these facilities and access roads. Support for conducting scientific research with a special focus on industrial research and experimental development (pre-competitive research carried out directly in enterprises). RPPK.01.02.00 Development of a network of business Support to improve the quality of the existing services for 18,000,637 31 19 environment institutions. enterprises, preparation and implementation a new service package; Support for participation in local and regional networks; The investment support necessary for the proper functioning RPPK.01.03.00 Creation of conditions for economic Support for the creation and development of infrastructure 402,918,341 52 31 development and growth of regional adapted to the needs of technologically advanced companies, innovation potential and transfer of i.e. technology parks, science and technology parks and knowledge. industrial incubators. Support for projects in the area of creating centers: investor service, promotion, professional development, advanced technology, research and development, technology transfer, competence, logistics, etc. The construction of parks, incubators and centers includes development of the area around these facilities and access roads. Support for conducting scientific research with a special focus on industrial research and experimental development (pre-competitive research carried out directly in enterprises). 76 RPPK.01.04.00 Increasing investment attractiveness and Scheme A. Comprehensive preparation of investment areas. 93,065,565 183 92 supporting interregional and international Scheme B: Non-investment projects - Organization and cooperation. Two schemes of support were implementation of promotional activities, including fairs, available: A. Comprehensive preparation of exhibition events, or participation in fairs, business missions, investment areas. B: Non-investment projects - study visits; Development and implementation of promotional Organization and implementation of campaigns and marketing plans; Creating new and developing promotional activities, including fairs, existing business information systems; Promotion of the exhibition events, or participation in fairs, products of the region business missions, study visits; Development and implementation of promotional campaigns and marketing plans; Creating new and developing existing business information systems; Promotion of the products of the region RPPM.01.04.00 Strengthening of system supporting Development / improvement, testing and pilot implementation 28,421,329 4 3 enterprises - projects aimed at comprehensive of new advisory and information services and how they are regional actions improving the quality, scope implemented; Creation / improvement and pilot and availability of information, advisory and implementation of new financial services and their financial information services provided by implementation; Provision of specialist advisory services and business environment organizations (IOBs) to information services other than individual information services entrepreneurs. (e.g. information seminars, publications); Expansion of core financial services (loans and sureties); Improvement of practical qualification of business advisers in IOB; Research and analysis on the activity / potential of Pomeranian SMEs, including the study of demand for services and financial support instruments; Support for participation in national and international IOB organizations; RPPM.01.05.00 Creation of efficient mechanisms for the Support for construction, extension, reconstruction, renovation 0 0 0 generation, transfer and commercialization of of facilities for the creation or development of science and innovation in enterprises - creation and technology parks, centers of advanced technology, centers of development of infrastructure for the excellence, education and implementation centers, business development of innovative companies, incubators and other similar institutions (including necessary through investments targeted at science and technical infrastructure and environmental management); technology parks, business incubators, high technology centers, RPSL.01.01.00 Sub measure 1.1.1: Increase of the value of Support for construction (including extension, reconstruction) 0 0 0 direct investment in the region. and overhaul of industrial and technological parks along with comprehensive development of business investment areas; Construction, reconstruction and renovation of business support 77 RPSL.01.03.00 Technology and innovation transfer, Construction/ reconstruction and overhaul of infrastructure, as 150,425,541 63 56 implemented by strengthening the potential of well as investment in specialized equipment for industrial and technology parks, research and innovation technological parks, technological and technology transfer transfer service providers, stimulating and centers of local and regional importance, which are providing developing networking and cooperation specialist services in the area of innovation and / or technology between research and development transfer; Consultancy services for the improvement of institutions and entrepreneurs, and the innovation and / or technology transfer, including the development of local and regional clusters. establishment and development of a network of cooperation between the research and development sector and entrepreneurs; Creation and development of a network of local and regional business environment institutions and their cooperation with international networks in the area of innovation and / or technology transfer; Customizing laboratories providing economic services to the requirements of EU directives, in particular harmonized standards and health and safety legislation, environmental protection; Creation and development of clusters of local and regional importance. RPSW.01.04.00 Support for business development and Support for construction, renovation and modernization of local 11,457,422 8 6 increase economic activity of the region's and regional business environment institutions, including the inhabitants by strengthening business purchase of equipment to improve the quality of services environment institutions provided (or to provide new services). Support for purchase of fixed assets and tangible and intangible rights, enabling the implementation of new service packages for entrepreneurs; Co- financing of investments related to the establishment and development of industrial parks, science and technology parks and technological incubators (including incubators of academic entrepreneurship), innovation centers. RPSW.02.04.00 Creation of new or improvement of the quality Creation and expansion of existing investment areas (owned by 63,285,299 11 9 of already existing areas for economic local government units), through the comprehensive investment in order to increase the development of the area for business etc. competitiveness of local centers. RPWM.01.01.02 Sub measure 1.1.2: Creation and development Construction, extension and equipping with a specialized 21,180,504 10 8 of technology parks, industrial and business equipment technological parks, industrial parks and business incubators of local and regional importance incubators RPWM.01.01.03 Sub measure 1.1.3: Support for the creation Construction, extension and equipping with a specialized 30,932,646 20 15 and development of local and regional links equipment of joint industrial and scientific projects, i.e. between scientific and research and industrial and pre-competitive research aimed at the development units, research organizations, implementation of results of completed research in enterprises. enterprises and other entities. RPWM.01.01.10 Sub measure 1.1.10: Creation of the favourable Establishment of business zones - comprehensive preparation of 59,979,295 17 14 conditions for businesses to invest and conduct areas for investment, mainly in the area of land development. business activities through comprehensive preparation of investment areas. 78 RPWM.01.01.11 Sub measure 1.1.11: Extension and refinement Support for the creation, extension and modernization of the 9,638,953 1 1 of services provided by the regional innovation infrastructure of the Regional Support System for Innovation, support system with a goal to expand and refine the provided services - Monitoring of the innovativeness and competitiveness of the Warmińsko-Mazurskie Voivodeship; Creation of a database for promotion and implementation of the innovation in the region RPWM.01.02.00 Sub measure 1.2.1: Support for projects that Support for expansion and modernization of the infrastructure 0 0 0 increase and maintain standards of services of a regional network of business environment institutions provided to entrepreneurs by business providing non-profit consulting and information services for environment institutions. enterprises (adapting the premises and purchase of equipment to raise the standard of services provided to entrepreneurs); RPWP.01.04.00 Strengthening the regional innovation system Scheme I: Creating new and developing existing business 0 0 0 by strengthening the R&D potential and environment institutions and intermediaries between science business environment institutions. and business, such as business incubators; Industrial parks; Science and technology parks; Technological incubators; Incubators of academic entrepreneurship; Technology transfer centers; Other centers of innovation. Scheme II: Construction and extension of facilities and purchase of equipment necessary for conducting research and development work. Scheme III: Creation and management of the Trust Fund, which allocates repayable funds in financial engineering instruments (JESSICA - Joint European Support for Sustainable Investment in City Areas) RPWP.01.06.00 Increasing innovativeness of enterprises in the Support for development of clusters and cooperative 11,069,844 33 23 region through the support of co-operation of relationships of regional and local importance. The projects companies with business environment focus on cluster promotion purchase of professional advisory institutions, research and development services related to the services provided by clusters; actions centres, universities, self-government focused on improving the level of cluster innovation (including authorities and other entrepreneurs. technology transfer / know-how), investment activities (including expenditure on the purchase of construction works and purchase of fixed assets and intangible assets necessary for the cluster's functioning). RPZP.01.02.01 Improving the quality and availability of Construction, reconstruction and renovation of facilities along 93,268,230 7 7 business support instruments that contribute with the purchase of the necessary equipment to create or to: increasing the capacity of West Pomeranian develop business environment institutions by providing new or enterprises to implement innovative solutions significant improvements to existing pro-innovative services; and stimulating innovation awareness of the Investments in IOB equipment (fixed assets and intangible SME sector; Enabling the transfer of assets) necessary to provide new or significant improvements to technology between enterprises and between existing pro-innovation services; research centers and enterprises; Creation of a new and development of existing enterprises based on new technology, know-how, idea or business model. 79 RPZP.03.01.00 Provision of universal access to secure Construction or extension of local or regional backbone, 72,747,367 54 17 broadband Internet distribution and access broadband networks integrated with other regional and national ICT infrastructure, establishment of open and publicly accessible hot-spots, purchase of equipment for network management centers, creation of Public Internet Access Points (PIAP). POIE.05.01.00 The objective of the measure is to strengthen Common projects of groups of entities aimed at preparing a 439,473,701 53 48 the position of enterprises’ competitiveness common innovative product and/or service and introducing it throughout supporting the development of on the market relations between enterprises as well as between enterprises and the business environment institutions including scientific entities. Source: Total amount, and number of contracts and beneficiaries based on own calculations from SIMIK database Note: 1. Asterisk (*) indicates the number of beneficiaries the total number of grants for the whole measure. Final number of beneficiaries according to SIMIK database. Total number of grants according to period report (30.06.2016). 2. The measure code denotes whether it is a national or regional program. Measure codes that start with “PO” is national programs; measure codes that start with “R” are regional programs. 80 Annex 3: List of Measures in OP IE and ROPs EXCLUDED from the Evaluation Measure Objective Description of support Total amount Number of Number of of grants contracts beneficiaries (PLN) POIE.01.02.00 The objective of the measure is to encourage young people There was a list of indicated "individual projects", which 421,244,967 6 1 to take up scientific careers and at the same time stimulate could receive a grant for implementation of the project. the quality development of science staff and international A support was used for financing a research of young cooperation. students/PhD students, scientists coming back to Poland etc. POIE.01.05.00 Implementation of system projects aimed at invention/test Grants on R&D works in enterprises 747,180,540 6 1 of model support instruments within R&D sector for 2014- 2020 perspective. POIE.03.04.00 The instrument aims to provide capital in the form of loans Support in a form of a loan for investment in new 100,000,000 1 1 to micro and small enterprises, which plan the products and services implementation of innovative projects. POIE.04.07.00 The objective is to test the potential of innovative SMEs to Guarantees for bank loans given to enterprises 0 0 0 make use of bank guarantees to facilitate access to bank financing. POIE.06.03.00 Strengthening the competitiveness of the economy by Support for carrying out marketing test and analyses in 258,395,934 2 1 promotion of Poland as an attractive country with respect the area of tourism or a purchase of such tests and to tourism. analyses as well as distribution of those results; service and development of Polish national system of tourist information together with instruments enabling booking; creating strategies and plans of promotion; POIE.06.04.00 Support of investment to create competitive and innovative Investments in infrastructure of unique tourism 578,279,467 21 21 tourism products of unique and supra-regional character products, projects implemented in the formula of while maintaining absorbability of tourist regions and their partnership among several entities; support for objects tourist capacity. being unique tourist attractions listed in the UNESCO World Cultural and Natural Heritage or regarded by the President of Poland as Monuments of History. POIE.07.01.00 Establishing broad nationwide ICT infrastructure enabling to grants for e-services in public administration (ICT 4,039,819,420 40 18 transfer data between particular platforms of electronic infrastructure enabling to exchange data among services, thematic portals, electronic registers and offices thematic platforms of electronic services, electronic themselves which is the basis for public electronic services registers and offices. Infrastructure will include provided for citizens and entrepreneurs (back-office). administration of all levels etc.) 81 POIE.08.03.00 The measure is aimed at providing Internet access for the Projects constituting of the following components: - 1,332,070,764 740 609 people threatened by digital exclusion caused by financial subsidy completely or partially covering the costs of difficulties or disability. Internet access in households on the area covered by the project (max. for three years), - covering the costs of providing installations POPW.01.02.00 Improvement of entrepreneurs' access to external sources Support of the guarantee system using specialized 175,632,740 1 1 of financing at an early stage of the company's activity. financial engineering instruments; Support of loan Improvement of investment readiness of SMEs. funds using specialized financial engineering instruments; Information and promotion activities on financial instruments. POPW.01.04.01 Component I (Promotion)Increased interest in the economic Preparation and implementation of the project of 100,520,000 1 1 offer of Eastern Poland. economic promotion of five Voivodeships of Eastern Poland (organization of business trips, economic missions, etc.). Creation and maintenance of common databases, portals, web pages related to the network offer; Data acquisition and preparation of joint investment offers; Development, production and distribution of common catalogues containing investment offers for all members of the network; Organization of meetings, in particular working groups responsible for the creation of a supra-regional network of investor service centers, seminars, conferences on presentation of the offer, increase of innovativeness of the services provided, exchange of experience in the service of the investor. 82 POPW.01.04.02 Component II (Cooperation) Increased interest in the Support for projects in which cluster members 22,408,651 2 2 economic offer of Eastern Poland. represent a minimum of two Voivodeships of Eastern Poland; co-financing for: organization and functioning of the cluster office; Development of cluster operational documents, including development strategies, marketing strategies, clustering and internal communication policies; Monitoring the effectiveness of cluster actions; Design, creation and maintenance of databases, portals, services and websites for cluster development, promoting clustering; Promotion and acquisition of new cluster participants, identification of new cluster initiatives; Organization of thematic meetings of cluster members, seminars, conferences promoting clustering potential, stimulating cluster development, clustering links, popularizing clustering in Eastern Poland; Organization of foreign industry missions; Raising the cluster's innovation and competitiveness through the acquisition of licenses, research, analysis, expertise, technology, know-how, patents and non-patented technical knowledge; etc. RPDS.01.03.00 Facilitating access to the external sources of finance for JEREMIE initiative (loans, guarantees) 405,732,728 1 1 SMEs in the region. RPKP.05.05.00 Strengthening the attractiveness of the Kujawsko- Implementation of projects aimed at supporting the 110,705,490 312 170 Pomorskie region promotion of regional and local products; Funding costs of participation in fairs and exhibitions abroad or in Poland (company as an exhibitor); Covering the costs of tender preparation and implementation of the promotional activities in the field of branded product. Support for participation in economic missions abroad (as a participant) etc. RPKP.05.06.00 improvement of location attractiveness of the region to Support comprehensive land investment. 137,419,960 18 16 increase the level of investment, including foreign direct investment. RPLU.02.01.00 Facilitating companies' access to external financing Creating new and recapitalization of existing local and 0 0 0 resources. regional loan and guarantee funds. Granting loans to entrepreneurs from the SME sector 83 RPLU.03.01.00 Improvement of the investment attractiveness of the Projects on comprehensive development of greenfield 70,406,605 10 8 voivodship and, as a consequence, an increase in the inflow and brownfield investment areas of less than 15 of foreign investment capital as well as domestic capital. hectares RPMA.01.03.00 Comprehensive preparation of investment areas for Investment projects on comprehensive land allocation 276,818,537 33 22 business in the local spatial development plan for investment RPMA.01.07.00 The objective of the Measure was to promote Mazovia as a Support and integration of marketing and promotion of 189,752,282 125 119 business-friendly region and new technologies. the region's economy through: organization of events and promotional and marketing campaigns; Participation of entrepreneurs in fairs and exhibitions; Participation in industry business missions aimed at finding and selecting partners in target markets; RPOP.01.02.00 Business growth through the creation and development of The recapitalization of emerging and operating 102,000,000 5 3 external financing of enterprises guarantee funds, loans and microloans and other public financial institutions offering returnable financial instruments. RPOP.01.04.01 Growth of the importance of tourism, recreational and Support for construction, reconstruction, renovation 161,664,714 114 102 sport services as a factor stimulating the socio-economic and equipment of tourist sector objects (including development of the Opolskie Voivodship. accommodation, catering and leisure facilities); Raising the standard (construction, reconstruction and equipment) of the infrastructure accompanying tourist facilities and the development of the area around them, enhancing the attractiveness of the facility; Adaptation of the existing infrastructure of tourist facilities to the needs of the individuals with disabilities RPPD.01.02.01 Creation of the conditions for the development of Comprehensive commercial investment area 89,214,332 12 10 entrepreneurship, creation of new innovative companies development, including roads and water supply, and increasing the competitiveness of Podlasie companies sanitation and rainwater systems, electricity, operating on the domestic and international market - telecommunications, gas, etc. investment areas RPPD.01.02.02 Creation of the conditions for the development of Construction, extension and modernization of trade 2,638,726 58 40 entrepreneurship, creation of new innovative companies fairs and exhibitions. Support of participation as an and increasing the competitiveness of Podlasie companies exhibitor at trade fairs, exhibitions in Poland and operating on the domestic and international market - abroad. Organization and participation in economic Economic promotion of the region missions in Poland and abroad. Support for promotional campaigns promoting the region in Poland and abroad. 84 RPPD.01.03.00 Easier access to external sources of investment financing Support for establishment of new loan and guarantee 130,768,287 8 6 and creating favourable conditions for entrepreneurship funds. Recapitalisation of the existing loan and development. guarantee funds. RPPM.01.03.00 Improving access to capital for SMEs. Establishment and management (performing the 287,394,442 1 1 functions of the Operator) Trust Fund allocating funds in financial instruments RPPM.01.06.00 Support for international business activity and investment Support for organization of economic missions and 0 0 0 attractiveness of the region study visits for enterprises; Facilitating the establishment of foreign business contacts of Pomeranian companies (e.g. electronic platforms for information exchange or interregional cooperation of public entities); Research and analysis of the export potential of enterprises, leading to a better orientation of activities supporting export development. RPWM.01.01.00 Sub measure 1.1.4: Support for the creation and Support for investment projects: joint investment 0 0 0 development of clusters projects of cluster members, creation of a common research and scientific infrastructure within the cluster, purchase of machines and equipment, purchase of intangible assets for the cluster members. Projects related to marketing activities RPWM.01.02.02 Sub measure 1.2.2: Recapitalization of guarantee and loan Support for creation or extension of existing capital: 127,396,500 4 2 funds loan funds, guarantee funds. RPWM.01.02.03 Sub measure 1.2.3: increased and maintained standards of Support for creation or improvement of services 12,531,671 3 1 services provided to entrepreneurs by the investor service provided by the investor service system at the regional system at the regional level. level. Development of systems to help entrepreneurs and investors access information on investment opportunities in the region RPWP.01.03.00 Improving access to micro, small and medium enterprises of This measure was implemented through JEREMIE 501,300,000 1 1 the region to external sources of funding. initiative- Joint European Resources for Micro to Medium Enterprises. The financial resources available under this measure are used to create the Trust Fund RPWP.01.05.00 Construction of a comprehensive system of management of Support for participants in promotional activities, both 11,363,076 1 1 investment and economic promotion of Wielkopolska, public and corporate, carried out within the project covering program activities, information, institutional (participation in fairs, trade missions, business forums). capacity building, promotional projects and support for regional promotional activities. 85 RPWP.01.07.00 Creation of the favourable conditions for businesses to Projects of comprehensive development of land for 57,516,704 15 14 invest - preparation of the investment areas in the region investment RPZP.01.03.03 Sub measure 1.3.3: Improving conditions for business Preparation of the investment zone with the necessary 186,536,038 22 13 activity. Creation of conditions for new investments in the technical infrastructure, in particular water supply, region sewerage, energy, gas, rain, telecommunications, specialized networks and internal roads etc. RPZP.01.03.04 Sub measure 1.3.4: Supporting the development of micro, The Fund Manager is obliged to allocate the Fund's 280,000,000 1 1 small and medium-sized enterprises operating in the resources to the instruments of external financing of Zachodniopomorskie Voivodeship by improving their entrepreneurship, primarily credit guarantee funds and availability to capital by offering them a comprehensive, loan funds flexible and effective system of financial engineering instruments. (JEREMIE Initiative - Joint European Resources for Micro to Medium Enterprises.) Source: Total amount, and number of contracts and beneficiaries based on own calculations from SIMIK database 86 Annex 4: Studies on the effects of innovation support programs on firms’ R&D expenditure decisions Paper Country (Year of Study) Effect on Firm investment in R&D expenditure Aerts et al. (2004) Belgium (Flanders) (1998-2000) Increase Aerts et al. (2008) Belgium (Flanders), Germany (1998-2000, Increase 2002-2004) Almus et al. (2003) Eastern Germany (1995, 1997, 1999) Increase Aschhoff (2009) Germany (1994-2005) Increase Berube et al. (2009) Canada (2005) Increase Bloom et al. (2002) Australia, Canada, France, Germany, Italy, Increase Japan, Spain, U.K., USA (1979-1997) Chudnovsky et al. (2006) Argentina (2001-2004) Increase Czarnitzki et al. (2002) Germany (1996, 1998) Increase Czarnitski et al. (2006) Germany (1994, 1996, 1998) Increase Czarnitzki et al. (2007) Germany, Finland (1996, 2000) - Increase (Finland) - No effect (Germany) Duguet (2004) France (1985-1997) Increase Ebersberger (2005) Finland (1994-1996, 1998-2000) Increase Goerg et al. (2007) Ireland (1998-2002) - Increase (domestic plants with small grants) - Decrease (domestic firms with larger grants) - No effect (foreign firms) Gonzalez et al. (2008) Spain (1990-1999) Increase Kaiser (2006) Denmark (2001) Increase Loof et al. (2005) Sweden (1998-2000) - Increase (small firms) - No effect (medium and large firms) Szcygielski et al. (2017) Poland, Turkey (2008-2010) - Increase (Poland, Turkey, national fund.), - Increase/no effect (Turkey, EU fund.), - Decrease (Poland, EU fund.) Zhu et al. (2006) China (1993-2002) Increase Note: The possible effects of innovation programs can be increase, no effect and decrease. All presented effects in the table are statistically significant. 87 Annex 5: Additional regression results Results with 1 treatment group matching with IPW Outcome 2008 2009 2010 2011 2012 2013 2014 Employment (ln) 0.141*** 0.0958*** 0.0694* 0.0394 0.0745* -0.0203 0.102 (0.0297) (0.0206) (0.0291) (0.0354) (0.0363) (0.0340) (0.0546) Sales (ln) 0.0811 0.0929*** 0.101** 0.0325 0.143** -0.00107 0.0861 (0.0417) (0.0266) (0.0353) (0.0448) (0.0472) (0.0467) (0.0705) Export (ln) 0.00513 -0.0857 -0.000175 0.228 -0.0737 -0.178 -0.327 (0.121) (0.0981) (0.146) (0.177) (0.185) (0.172) (0.429) Value added (ln) 0.0512 0.0907*** 0.0802* 0.0419 0.143** 0.00456 0.0635 (0.0432) (0.0272) (0.0374) (0.0452) (0.0474) (0.0448) (0.0707) Tangible Fixed assets Inv. 5.834 0.166 -4.172 -0.252 0.631 1.626 0.313 (6.552) (0.817) (4.503) (0.479) (1.804) (2.104) (0.310) Intangible Fixed assets Inv. -0.138 0.397 0.000638 0.000778 0.0998 0.0125 0.00528* (0.137) (0.412) (0.00127) (0.00292) (0.0943) (0.155) (0.00206) Average wages (ln) -0.0793*** -0.0236* 0.00714 -0.00995 0.0147 -0.00982 0.0644* (0.0141) (0.0113) (0.0158) (0.0181) (0.0242) (0.0180) (0.0292) R&D Exp. 0.00119*** 0.00169 0.000995 0.000513 0.000737 0.353 -0.000151 (0.000341) (0.00131) (0.000634) (0.000318) (0.000471) (0.351) (0.000622) Profits 25.48 4.057 20.32 -6.305 -0.434 -0.258 1.652 (18.65) (9.587) (26.78) (4.250) (7.252) (2.853) (1.846) Labor Prod. (ln) -0.0896** -0.00574 0.00401 -0.00177 0.0607 0.0225 -0.0583 (0.0284) (0.0235) (0.0280) (0.0366) (0.0389) (0.0349) (0.0625) TFPR (ln) -0.140*** -0.135*** -0.120*** -0.143*** -0.153*** -0.137*** -0.338*** (0.0319) (0.0212) (0.0307) (0.0361) (0.0406) (0.0330) (0.0680) Notes: Standard errors in parentheses where * p<0.05, ** p<0.01 and *** p<0.001. Results with multiple treatments matching Grouping 1 ATE (1) vs. 0 (2) vs. (0) (3) vs. (0) (4) vs. (0) Employment (ln) 0.191 0.554 0.362 0.185 (0.0173)*** (0.0376)*** (0.0884)*** (0.0452)*** Sales (ln) 0.382 0.671 0.338 0.431 (0.0241)*** (0.0526)*** (0.141)* (0.0712)*** Value-added (ln) 0.330 0.524 0.317 0.405 (0.0242)*** (0.0522)*** (0.127)* (0.0712)*** Exporter status (=1) 0.0907 0.177 0.255 0.283 (0.00649)*** (0.0142)*** (0.0279)*** (0.0208)*** Export value (ln) 0.304 0.698 0.664 0.640 (0.0258)*** (0.0606)*** (0.106)*** (0.0811)*** Average wages (ln) 0.0333 0.0482 0.144 0.0842 (0.0101)*** (0.0199)* (0.0465)** (0.0297)** Labor productivity (ln) 0.111 -0.0786 -0.0479 0.198 88 (0.0199)*** (0.0426) (0.0910) (0.0588)*** Profits 1.002 7.476 7.833 -0.316 (0.925) (2.596)** (3.077)* (2.155) TFPR (ln) -0.136 -0.332 -0.607 0.0181 (0.0198)*** (0.0422)*** (0.119)*** (0.0633) R&D expenditure (ln) 0.000932 0.00926 0.0336 0.00320 (0.000790) (0.00423)* (0.0130)** (0.00314) Tangible fixed assets Inv. -0.00205 0.0431 0.275 0.00126 (0.00577) (0.0180)* (0.0955)** (0.0150) Intangible fixed assets Inv. 0.000642 0.00958 0.0501 0.00335 (0.00141) (0.00549) (0.0361) (0.00341) Notes: Standard errors in parentheses where * p<0.05, ** p<0.01 and *** p<0.001. Grouping 2 ATE (1) vs. (2) vs. (0) (3) vs. (0) (4) vs. (0) (5) vs 0 (0) Employment (ln) 0.477 0.334 0.326 0.207 0.295 (0.0918)*** (0.153)* (0.0713)*** (0.0578)*** (0.0248)*** Sales (ln) 0.479 0.514 0.495 0.215 0.480 (0.122)*** (0.197)** (0.120)*** (0.0894)* (0.0381)*** Value-added (ln) 0.408 0.400 0.513 0.155 0.400 (0.124)*** (0.164)* (0.126)*** (0.0914) (0.0386)*** Exporter status (=1) 0.197 0.0684 0.162 0.254 0.114 (0.0349)*** (0.0542) (0.0502)** (0.0259)*** (0.0119)*** Export value (ln) 0.466 0.915 0.239 0.921 0.519 (0.143)** (0.302)** (0.150) (0.109)*** (0.0537)*** Average wages (ln) 0.0825 0.149 0.171 0.0360 0.0784 (0.0478) (0.0761) (0.0544)** (0.0368) (0.0152)*** Labor productivity (ln) -0.177 0.0378 0.147 -0.0765 0.0721 (0.105) (0.0927) (0.108) (0.0794) (0.0318)* Profits 5.710 5.192 1.219 -0.257 1.406 (6.442) (.) (2.217) (3.644) (1.168) TFPR (ln) -0.478 -0.162 -0.0580 -0.232 -0.171 (0.0930)*** (0.148) (0.115) (0.0856)** (0.0317)*** R&D expenditure (ln) 0.0226 -0.0325 0.000699 -0.00173 0.00287 (0.0140) (0.0199) (0.000204)*** (0.00270) (0.00269) Tangible fixed assets Inv. 0.0689 0.0635 -0.0221 0.0329 0.00836 (0.0265)** (0.0779) (0.0227) (0.0158)* (0.00872) Intangible fixed assets Inv. 0.0271 -0.0206 0.00386 -0.000139 0.00266 (0.0201) (0.0137) (0.00232) (0.00317) (0.00132)* Notes: Standard errors in parentheses where * p<0.05, ** p<0.01 and *** p<0.001. 89 Annex 6: Data cleaning and estimation procedures This appendix describes the processes for the practitioner to replicate the estimation methodology used in the report. Due to legal limitations, Statistics Poland does not make any micro-data available to external entities. The appendix covers the steps the World Bank team took to process and manage the data and conduct the estimation procedure. The appendix includes technical details with accompany statistical codes based on the STATA software. Lastly it discusses practical issues on the working arrangement and lessons learnt from the World Bank team. 8.1 Data Management A6.1.1 Data processing The main dataset, financial and trade data on companies, has been prepared based on guidelines and in accordance to detailed codebook provided by the World Bank team. The codebook specified variables sources, i.e. from which cells in reporting forms/ surveys collected from companies data are to be pulled, along with data storage parameters: variables names, labels, formats/ types (e.g. byte, int, long, float, str), decimals, units (e.g. thous. PLN for monetary variables), length, theoretical min/ max values, number of expected unique values, coding schemes. The codebook has been consulted and includes suggestions made by Statistics Poland in order to minimize future efforts to amend the manner in which data was stored for the project. Along with codebook the World Bank team provided Statistics Poland with Stata do-files to proceed with elementary data recodes, data restructuring, etc. That also includes restructuring data into panel form. The variables used in this project could be grouped into following types: companies’ identifiers, monetary variables (at the beginning/ end of fiscal year; e.g. assets, liabilities), composites based on monetary variables (at the end of fiscal year; e.g. EBIT, EBT, EAT), trade data (e.g. value of imported/ exported commodities), companies’ characteristics that are mostly stable in time (e.g. size, sector of operation) and those which vary in time (e.g. employment). The primary data has not been changed to large extent; missing values have not been imputed. Categorial variables based on companies’ characteristics were created, which later served as covariates to match on. Outcome variables (mostly monetary variables) have been logged and winsorized using the following code: ** Logarithmize outcomes foreach v of varlist $outcomes { gen ln_`v' = ln(`v') } ** Winsoring/ replacing extreme values at the right side of the distribution with p99 value for all outcomes foreach v of varlist $outcomes { qui sum `v', d replace `v'=`r(p99)' if `v'>`r(p99)' } 90 A6.1.2 Deflating monetary variables All monetary variables covering time span of 2006-2016 were harmonized using deflators available at Statistics Poland website. Company’s main sector of operation expressed in NACE Rev. 2 codes were used as a base for deflating monetary variables. Deflators for the section level were used and for C and H sections, selected division deflators were used. Available data published by Statistics Poland did not allowed for more precise price indices at more disaggregated sector levels, i.e. groups or classes. The following deflators were used, with 2010 as a base year (=100): • K_deflator (Consumption of fixed capital) • VA_deflator (Gross value added) • CPI (Consumer price index) • PPI (Producer price index). The monetary variables were harmonized using the following formulas: **Generate the outcome variables (in real values where needed) //All monetary variables in real values have prefix ‘rv_’ //Suffix ‘E’ means that data is taken from the end of fiscal year gen rv_RDworksCostsE = RDworksCostsE / CPI gen rv_fixAssE = fixAssE / K_deflator gen rv_tangFixAssE = tangFixAssE / K_deflator gen rv_intangFixAssE = intangFixAssE / K_deflator gen rv_machTechToolE = machTechToolE / K_deflator gen rv_assTotalE = assTotalE / K_deflator gen rv_revNetTotal = revNetTotal / PPI gen rv_wages = wages / CPI gen rv_expValue = expValue / PPI gen rv_valueAdded = (revNetTotal - matEnergCosts) / VA_deflator gen rv_EBT = ((revNetTotal - operCostsTotal + wages) / VA_deflator) - (wages / CPI) gen currentRatio = assTotalE / liabE A6.1.3 Outcomes computation The following list contains the potential outcomes defined from the data. For the final analysis, a shorter list was used with selected outcomes based on, for e.g., reasonably low percentage of missing data: • RDwkCostRev – R&D expenditure (% of revenue) • faInv – Investment in fixed assets (total) • faInvPct – Investment in fixed assets (total) (% change) • faInvRev – Investment in fixed assets (total) (% of revenue) • tangFaInv – Investment in tangible fixed assets • tangFaInvPct – Investment in tangible fixed assets (% change) 91 • tangFaInvRev – Investment in tangible fixed assets (% of revenue) • intFaInv – Investment in intangible fixed assets • intFaInvPct – Investment in intangible fixed assets (% change) • intFaInvRev – Investment in intangible fixed assets (% of revenue) • machInv – Investment in in machinery • machInvPct – Investment in machinery (% change) • machInvRev – Investment in machinery (% of revenue) • machYoY – Fixed assets in machinery (% of last year’s assets) • impInput – Use of imported inputs (% of total inputs) • impInputRev – Use of imported inputs (% of revenue) • useForeignSuppl – Use of foreign suppliers (1/0 indicator) • rv_RDworksCostsE – R&D expenditure (total) • rv_machTechToolE – Fixed assets in machinery (total) • imporingComp – Use of imported inputs (1/0 indicator) • importShare – Use of imported inputs (% of total inputs) • avgWagesFTE – Average wages per FTE • ROE – Return on equity • ROA – Return on assets • rv_valueAdded – Value Added • labrProdRev – Labor productivity (revenue per FTE) • labrProd – Labor productivity (VA per FTE) • lnEmplFTE – Employment (ln) • emplFTEYoY – Employment in FTE year-on-year (% of last year’s empl.) Note: outcomes with prefix ‘rv_’ are monetary variables in real values (deflated); ‘YoY’ means year-on- year change; prefix ‘ln’ means that variable are in natural logs; variables with ‘S’ at the end of their names have values from beginning (start) of given fiscal year; variables with ‘E’ at the end of their names have values from end of given fiscal year. ** Intermediate outcomes (composite outcomes): gen RDwkCostRev = rv_RDworksCostsE / rv_revNetTotal gen faInv = rv_fixAssE - rv_fixAssS gen faInvPct = (rv_fixAssE - rv_fixAssS) / rv_fixAssS gen faInvRev = (rv_fixAssE - rv_fixAssS) / rv_revNetTotal gen tangFaInv = rv_tangFixAssE - rv_tangFixAssS gen tangFaInvPct = (rv_tangFixAssE - rv_tangFixAssS) / rv_tangFixAssS gen tangFaInvRev = (rv_tangFixAssE - rv_tangFixAssS) / rv_revNetTotal gen intFaInv = rv_intangFixAssE - rv_intangFixAssS gen intFaInvPct = (rv_intangFixAssE - rv_intangFixAssS) / rv_intangFixAssS gen intFaInvRev = (rv_intangFixAssE - rv_intangFixAssS) / rv_revNetTotal gen machInv = rv_machTechToolE - rv_machTechToolS gen machInvPct = (rv_machTechToolE - rv_machTechToolS) / rv_machTechToolS gen machInvRev = (rv_machTechToolE - rv_machTechToolS) / rv_revNetTotal 92 sort REGON year by REGON: gen machYoY = rv_machTechToolE / rv_machTechToolE[_n-1] if rv_machTechToolE > 0 & rv_machTechToolE[_n-1] > 0 gen impInput = currentRatio / operCostsTotal gen impInputRev = impValue / revNetTotal recode rsupplNo (0=0) (1=1) (2=1), gen(useForeignSuppl) ** Intermediate outcomes (not a composite indicators, already in the data): * rv_RDworksCostsE * rv_machTechToolE * imporingComp //created by command: recode expValue (0=0) (nonmissing=1), gen(imporingComp) * importShare //comparable with import input ** Final outcomes (composite outcomes): gen avgWagesFTE = rv_wages / emplFTE gen ROE = EAT / equityE gen ROA = EAT / assTotalE * rv_valueAdded //already in data gen labrProdRev = rv_revNetTotal / emplFTE gen labrProd = rv_valueAdded / emplFTE gen lnEmplFTE = log(emplFTE+1) sort REGON year by REGON: gen emplFTEYoY = emplFTE / emplFTE[_n-1] //Employment in FTE year-on-year (% of last year’s empl.) A6.1.4 Total factor productivity (TFP) computation The index approach was utilized: following Asker et al. (2014) who use a sales generating function. At the industry i level, the input shares are calculated, input X(L, M): b_L and b_M as share of the input’s expenditure in sales: b_X = median_i(p_X*X/S) Constant returns to scale is assumed, so the input share for capital is computed as b_K: b_K = (e-1/e) - b_L - b_K Then the formula to calculate the TFP is as follows: TFPR = S - b_K*K - b_L*L - b_M*M 93 ** (1) The index approach // the revenue production function coefficient equals the share of the input’s expenditure in sales: bysort NACE year: egen b_L = median(rv_wages/rv_revNetTotal) bysort NACE year: egen b_M = median(matEnergCosts/revNetTotal) // Create beta_K (assume epsilon = 4) gen b_K = 0.75 - b_L - b_M // For each year: replace negative b_K with average b_K if b_K <= 0 (that is b_L + b_K >= 0.75) levelsof year, local(year) foreach y of local year { summ b_k if year == `y' & b_K > 0, d replace b_K = r(mean) if b_K <= 0 & year == `y' } //Generate TFP = log(Revenue) - beta_K * log(K_it) - beta_L * log(L_it) - beta_M * log(M_it) gen lnY = log(rv_revNetTotal) gen lnK = log(rv_fixAssE) gen lnL = log(emplFTE) gen lnM = log(rv_matEnergCosts) gen TFPR = lnY - b_K*lnK -b_L*lnL - b_m*lnM label var TFPR "TFPR index approach (Asker et al. 2014)" A6.1.5 Identification of direct and indirect beneficiaries and merging with main data for matching on one treatment group Direct beneficiaries and business support institutions (BSI) were identified by the list provided by the former Ministry of Infrastructure and Development and linked to data from Statistics Poland using company’s tax identification number (NIP). Since a particular company could be a beneficiary more than once during period 2006-2016, the project data was collapsed to the company level, i.e. chronologically, each company was started being a beneficiary in the evaluated program based on the first project it received. The start year was extracted from the contract number of the grant disbursement contract. ** Collapse beneficiaries to 1 obs. per firm; create treatment, i.e. the 1st year // They ever received any grant in the database. Ignore non-firms; Non-firms should not merge with Statistics Poland data use BSI_SIMIK, clear sort NIP startYear collapse (firstnm) startYear …, by(NIP) merge 1:m NIP using "SP_panel_data.dta" 94 A treatment status was assigned to companies based on the start year of its first subsidized project and following years. * Create a treatment variable where D = 0 before treatment year, * D = 1 after treatment year, (D = 0 for all non-beneficiaries) gen treated = 0 if mergeBSI_SIMIK == 2 // BSI beneficiaries are not considered as control - otherwise the impact is relative to BSI treatment replace treated = 1 if simik_ben == 1 //use only simik_ben to avoid marking bsi_ben as treated for direct effect estimation gen after = 0 replace after = 1 if year >= startYear & year != . & startYear != . gen D = treated*after //we prevent benef. from years where they are non-benef. become controls; our PS estimation will be with D`y' which will be missing for firms starting treatment other than year `y' // check distribution of treatment overtime --> use this information for the repeated cross-section: how many firms actually receive grants starting in later years tab year D forval y = 2008/2015 { gen D`y' = D if startYear == `y' // D`y' will be missing for firms starting treatment earlier or later than `y' replace D`y' = 0 if treated == 0 // we still need D`y'=0 for control group } foreach v of varlist D* { tab `v', m } forval y = 2008/2015 { tab year D`y', m } ** Check if those who did not reported to GUS indeed have missing values on GUS vars egen temp = rownonmiss(_all), strok gen nonmiss = temp / c(k) * 100 drop temp 95 tab nonmiss //it is expected that mergeBSI_SIMIK==1 would have share of nonmissings close to 0% table nonmiss mergeBSI_SIMIK, m // also check that SIMIK_firm indicator correctly identifies firms vs. non-firms tab SIMIK_firm if mergeBSI_SIMIK == 1 // they should all be zeros drop if mergeBSI_SIMIK==1 //drop those who did not reported to GUS A6.1.6 Identification of direct and indirect beneficiaries and merging with main data for matching on multiple treatment groups For the analysis with multiple treatment groups the companies, which overlap groups at interest were excluded from the dataset. / Check share & pattern of firms that have overlapping treatments bysort NIP T1D: keep if _n == 1 & T1D != . keep NIP T1D gen treatment = 1 reshapewide treatment, i(NIP) j(T1D) // pattern of treatment --> to many combinations to create new treatment groups egen pattern = concat(treatment*) tab pattern // check share of firms with more > 1 treatment groups => drop firms with multiple treatments egen mult = rowtotal(treatment*) tab mult bysort NIP (T1D): gen mult = 1 if T1D[1] != T1D[_N] drop if mult == 1 | T1D == . A6.1.7 Summary statistics reporting Generating summary statistics is fairly straightforward but the long list of outcome variables may be unwieldly. The STATA codes below enhance the easy reporting of basic statistics. First, the lists of variables were specified, categorical and continuous ones separately. #delimit ; global categorical size compLife_upTo3yr regions jointStockComp C_Manufacturing …; 96 global continuous fixAssE machTechToolE assTotalE operCostsTotal wages …; #delimit cr Now the summary statistics could be generated. A default *csv format was set. In the same manner, the results could be generated into *.tex or *.html format. The *.html is especially helpful for easy and quick data reviewing. ssc install estout, replace format %9.2fc $continuous //set display format for continuous vars ** Export tabulated results for continuous variables // Yearly stats qui forval i = 2006/2016 { eststo clear eststo: estpost sum $continuous if year == `i', detail esttab using "sum_stats_cont_`i'.csv", cell("count mean(fmt(2)) sd(fmt(2)) min(fmt(2)) max(fmt(2)) skewness(fmt(2)) kurtosis(fmt(2)) p1(fmt(2)) p5(fmt(2)) p10(fmt(2)) p25(fmt(2)) p50(fmt(2)) p75(fmt(2)) p90(fmt(2)) p95(fmt(2)) p99(fmt(2))") label nodepvar nomtitle nonumber title(Descriptive Statistics for `i') replace } // All stats together eststo clear qui eststo: estpost sum $continuous, detail esttab using "sum_stats_cont_2006-16.csv", cell("count mean(fmt(2)) sd(fmt(2)) min(fmt(2)) max(fmt(2)) skewness(fmt(2)) kurtosis(fmt(2)) p1(fmt(2)) p5(fmt(2)) p10(fmt(2)) p25(fmt(2)) p50(fmt(2)) p75(fmt(2)) p90(fmt(2)) p95(fmt(2)) p99(fmt(2))") label nodepvar nomtitle nonumber title(Descriptive Statistics for 2006-16) replace ** Export tabulated results for categorical variables // Yearly stats qui forval i = 2006/2016 { foreach var of varlist $categorical { eststo clear eststo: estpost tab `var' if year == `i', miss 97 esttab using "sum_stats_cat_`i'.csv", cell("b(label(freq)) pct(fmt(2)) cumpct(fmt(2))") label nonumber title(Frequencies for `i') varlabels(`e(labels)') append } } // All stats together qui foreach var of varlist $categorical { eststo clear eststo: estpost tab `var', miss esttab using "sum_stats_cat_2006-16.csv", cell("b(label(freq)) pct(fmt(2)) cumpct(fmt(2))") label nonumber title(Frequencies for 2006-16) varlabels(`e(labels)') append } A6.1.8 Input-output Coefficients for indirect estimation Building on the data work done in the direct effects estimation, the indirect effects estimation will only require some additional work to create the sector weights obtained from the Input-Output (IO) tables. Poland’s IO table are available on their website and the input-output coefficients are processed before they were submitted to Statistics Poland. The IO coefficients were created using these codes: * Applying formula to turn Input-Output quantities from current to constant prices (base year 2010): foreach x of varlist S1 - STotal tot_use{ gen r`x'=`x'/PPI*100 label var r`x' "Output to sector `x' in real terms, b.y. 2010" } * Trying to produce output coefficients, with current prices: foreach x of varlist S1-STotal tot_use{ gen outto`x'= `x'/tot_use label var outto`x' "Output coefficient to `x', current prices" } foreach x of varlist rS1-rS95 rSTotal rtot_use{ gen routto`x'= `x'/rtot_use label var routto`x' "Output coefficient to `x', constant prices, b.y.2010" } * Deriving input coefficients for industry y by dividing inputs from each industry x by the total amount of supply used in industry y. 98 foreach x of varlist S1-STot_use{ gen inputsfrom`x'= `x'/STot_use label var inputsfrom`x' "Input coefficient from `x', current prices" } rename inputsfromSTotal Stotinputshare rename inputsfromSTot_use STotuseshare *Deriving input coefficients in constant prices by dividing by total use in constant prices. foreach x of varlist rS1-rS95 rSTotinput rSTot_use{ gen rinputsfrom`x'= `x'/rSTot_use label var rinputsfrom`x' "Input coefficient from `x', constant prices, b.y.2010" } A6.1.9 Creating spillover variables for indirect estimation The indirect estimation will require the construction of three types of spillover variables: • Horizontal spillovers: the sum of all grant amounts in the sector. • Vertical spillovers: the weighted sum of all grant amounts in the buyer and supplier sectors, using the input-output coefficients as weights. • Regional spillovers: the sum of all grant amounts in the region. In order to construct the variables, the beneficiaries has to be identified and matched with the firm data (which provides the region and sector information). First the horizontal spillover variables were created: /*Horizontal spillover variables - measured as number of firms in a sector, in a year that got a grant and as amount of cofinancing in a sector, in a year:*/ * Generating firm-year identifier: egen nip_year=group(nip year) * Generating firm-year-beneficiary type identifier: egen nip_year_bentype=group(nip year direct_ben) sort nip_year_bentype * Generating cofinancing measure by firm, year and beneficiary type by nip_year_bentype: egen cofin_nip_year_type=sum(cofinancing) * Making cofinancing variables by beneficiary type separately: gen dir_cofin_nip_year=. replace dir_cofin_nip_year=cofin_nip_year_type if direct_ben==1 99 gen i_cofin_nip_year=. replace i_cofin_nip_year=cofin_nip_year_type if indirect_ben==1 bysort nip_year: egen cofin_nip_year=sum(cofinancing) recast double dir_cofin_nip_year i_cofin_nip_year cofin_nip_year cofin_nip_year_type *Generating new variable to identify firms that have been either a direct or an indirect beneficiary at least once: sort NIP year egen ben_firm=max(direct_ben), by(NIP) distinct NIP if ben_firm==1 distinct NIP if ben_firm==0 distinct NIP if ben_firm==. *dis 157+79633+15234 distinct NIP distinct NIP if ben_firm!=. distinct NIP if ben_firm==. distinct NIP save SP_panel_for_hor, replace * Horizontal spillover variable by number of beneficiary firms, direct spillovers. use SP_panel_for_hor.dta egen dir_ben_year_nace=group(NIP year nacerev1) if direct_ben==1 sort dir_ben_year_nace gen nace_no_dir_bens=. by dir_ben_year_nace: replace nace_no_dir_bens=1 if [_n]==[_N] &!missing(dir_ben_year_nace) sort year nacerev1 by year nacerev1: egen dir_no_year_nace=total(nace_no_dir_bens) if !missing(dir_ben_year_nace) * Horizontal spillover variable by number of beneficiary firms, indirect spillovers. egen i_ben_year_nace=group(NIP year nacerev1) if indirect_ben==1 sort i_ben_year_nace gen nace_no_i_bens=. by i_ben_year_nace: replace nace_no_i_bens=1 if [_n]==[_N] &!missing(i_ben_year_nace) sort year nacerev1 100 by year nacerev1: egen i_no_year_nace=total(nace_no_i_bens) if !missing(i_ben_year_nace) * Horizontal spillover variable, by amount of cofinancing, direct spillovers: by year nacerev1: egen dir_cofin_year_nace=total(dir_cofin_nip_year) if !missing(dir_ben_year_nace) * Horizontal spillover variable, by amount of cofinancing, indirect spillovers: by year nacerev1: egen i_cofin_year_nace=total(i_cofin_nip_year) if !missing(i_ben_year_nace) Next the vertical (backward and forward) spillover variables are created using the horizontal spillover variables: /*For each sector x, in year t and upstream/downstream sector y, we assign a value for horizontal spillover variables equal to the value of horizontal spillover variables of the upstream/downstream sector y that sector x is selling output to/buying inputs from in year t. We do this for all measures of direct and indirect horizontal variables. */ foreach x of varlist dir_no_year_nace- i_cofin_year_nace { gen hor_`x'=. } levelsof year, local(years) foreach x of local years{ levelsof nacerev1, local(sectors) foreach y of local sectors{ foreach var of varlist dir_no_year_nace- i_cofin_year_nace{ sum `var' if year==`x' & nacerev1=="`y'", meanonly local dirno=r(mean) replace hor_`var'=`dirno' if year==`x' & other_nace=="`y'" } } } sort nacerev1 year other_nace /*We build yearly backward spillover variables by adding up all horizontal variables in downstream sectors and weighting them by the "output to" coefficients. We subtract the sector's own contribution*/ 101 local amount "no cofin" foreach x of local amount { local bentype "dir i" foreach j of local bentype { egen tot_`j'_`x'_backward=total(routtorS* hor_`j'_`x'_year_nace), by(nacerev1 year) gen `j'_`x'_backward=tot_`j'_`x'_backward - (routtorS*`j'_`x'_year_nace) if other_nace==nacerev1 xtset nace_year xfill `j'_`x'_backward, i(nace_year) } } /*We build yearly forward spillover variables by adding up all horizontal variables in upstream sectors and weighting them by the "input from" coefficients. We subtract the sector's own contribution*/ local amount "no cofin" foreach x of local amount { local bentype "dir i" foreach j of local bentype { egen tot_`j'_`x'_forward=total(rinputsfromrS* hor_`j'_`x'_year_nace), by(nacerev1 year) gen `j'_`x'_forward=tot_`j'_`x'_forward - (rinputsfromrS*`j'_`x'_year_nace) if other_nace==nacerev1 xtset nace_year xfill `j'_`x'_forward, i(nace_year) } } Lastly the regional spillover variables are created: *********************************************************************** *Building regional spillover variables based on amount of cofinancing. *********************************************************************** sort nip year ben_region_code no_cofin egen dir_ben_year_reg=group(nip year ben_region_code) if direct_ben==1 count if missing(ben_region_code) & measure_type==2 * Building direct cofinancing variable per region, per year: sort year ben_region_code by year ben_region_code: egen dir_cofin_year_reg=total(cofinancing) if direct_ben==1 102 ************************************************************** * Same thing for indirect beneficiaries: egen i_ben_year_reg=group(nip year ben_region_code) if indirect_ben==1 * Building indirect cofinancing variable per region, per year: sort year ben_region_code by year ben_region_code: egen i_cofin_year_reg=sum(cofinancing) if indirect_ben==1 8.2 Estimation A6.2.1 Defining outcomes for final estimation First the skewed outcome variables are transformed to natural logs. ** Define list of outcomes: investment, R&D, employment, sales … exports, profits, value added, wages, LP, TFP foreach y in emplFTE rv_revNetTotal rv_expValue rv_valueAdded avgWagesFTE labrProd { gen ln_`y' = ln(`y') } global list_outcomes "ln_emplFTE ln_rv_revNetTotal ln_rv_valueAdded exportingComp ln_rv_expValue ln_avgWagesFTE ln_labrProd rv_EBT TFPR ln_RDwkCostRev ln_faInvRev ln_intFaInvRev" Some covariates were expressed as year-on-year change using value in the start year and a value from one year before. sort REGON year foreach var in revNetTotal expShare assTotalE { gen `var'YoY = `var'/`var'[_n-1] } Values for the rest covariates were taken from one year before company’s first subsidized project started. New lagged variables have prefix ‘L’. // take laggeg values sort REGON year foreach var of varlist $covariates_cont $covariates_binary $covariates_categorical { gen L_`var' = `var'[_n-1] } 103 A6.2.2 Covariates diagnostics A diagnostics of the covariates are performed before matching to examined the missing observations and treatment status of firms. ****************************************** ** Diagnostic on covariates before matching forval y = 2008/2014 { //last analyzed year is 2014 as very little obs. are available for later ** Missing data diagnostic on covariates foreach v of varlist $X1_spec1 { qui egen nmK_`v' = rowmiss(`v') if year==`y' } gen nmisSumD`y' = 0 qui foreach v of varlist nmK_* { replace nmisSumD`y' = nmisSumD`y' + `v' } di "" di "" di as result "Missing data diagnostic on: " "`v'" "in " "`y'" tab nmisSumD`y', miss drop nmK* nmisSum* sort D`y' bysort D`y': mdesc $X1_spec1, ab(32) ****************************************** ** Check covariates distribution across treatment status di "" di "" di as result "Check covariates distribution across treatment status in " "`y'" foreach var of varlist $covariates_binary_L1 $covariates_cat_L1 { tab `var' D`y' if year==`y', miss chi nokey } 104 } A6.2.3 Local diagnostics of model for PS estimation The team employed five selected indices to identify outliers and influential observations: • standardized residuals – standardized Pearson residuals; adjusted for the covariate pattern • deviance residuals • leverage – Pregibon leverage • DBETA – Pregibon delta beta influence statistic • Delta Chi-Sq influence statistic – Hosmer and Lemeshow change in Chi-Sq influence statistic Then liberal cut-offs were used to classify a given company as outlier or influential observation. These cut- offs are: • standardized residuals: shouldn’t exceed range of -4 to +4 • deviance residuals: shouldn’t exceed range of -1 to +1 • leverage: shouldn’t be larger than +1 • DBETA: shouldn’t be larger than +2 • Delta Chi-Sq influence statistic: shouldn’t be larger than +2 ** Evaluating model for PS estimation forval y = 2008/2014 { capture noisily { * PS estimation logit D`y' $covariates if year == `y', nolog vce(robust) iterate(100) * Local fit measures predict lfit_stdr if year == `y', rstandard //std residuals predict lfit_dr if year == `y', deviance //deviance residuals predict lfit_leverage if year == `y', hat //leverage predict lfit_dbeta if year == `y', dbeta //DBETA predict lfit_dx2 if year == `y', dx2 //Delta Chi-Sq influence statistic gen suspectedStdr = 1 if (lfit_stdr <(-4) | lfit_stdr >4) & lfit_stdr!=. gen suspectedDr = 1 if (lfit_dr <(-1) | lfit_dr >1) & lfit_dr!=. gen suspectedLev = 1 if lfit_leverage >1 & lfit_leverage!=. gen suspectedDbeta = 1 if lfit_dbeta >2 & lfit_dbeta!=. gen suspectedDx2 = 1 if lfit_dx2 >2 & lfit_dx2!=. egen susp`y' = rowtotal(suspected*) //a new indicator created to spot outliers } 105 A6.2.4 PS estimation and model global diagnostics The procedure dropped suspected outliers and influential observations based on a global diagnostics procedures: • linktest – Used to detect a specification error. If the model is properly specified, one should not be able to find any additional predictors that are statistically significant except by chance. • lfit – Hosmer and Lemeshow’s goodness-of-fit test • fitstat – Measures of fit for logit models. There was a focus on the McFadden's Adj R-Sq, which should not be too low for the model to have some predictive power, but it should not be too high either to ensure wide enough common support area to match within. ** PS estimation in final model with outliers excluded forval y = 2008/2014 { capture noisily { * PS estimation; skip companies suspected on more than 3 out of 5 outliers indicators logit D`y' $covariates if year == `y' & susp`y' <= 3, nolog vce(robust) iterate(100) predict xtlhat`y' if year == `y' & susp`y' <= 3, pr //the propensity score to match on * Global fit measures linktest, nolog lfit, group(10) table fitstat } } A6.2.5 Matching of one treatment group Matching was conducted based on the nearest-neighbor propensity score (which was done for the one joint treatment group estimation). The procedure for nearest-neighbors propensity score matching (one joint treatment group) is as follows: • matching is performed separately for each year-cohort of the beneficiaries • year-cohorts defined by start-year of 1st subsidized project run by a particular company • 7 year-cohorts of beneficiaries (2008-2014) • covariates to match-on values taken from one year previous to 1st project start year • outcome variables values taken from years 2007-2016 An illustration is provided below: 106 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 C C C C C C C T T T T T T T O O O O O O O O O O Note: C – covariates; T – treated; O – outcomes. These matching parameters were used: • Random sorting of observations to avoid bias, as NN procedure is sensitive to data ordering • Caliper defined as ¼ standard deviation of PS • With option ‘ties’ we not only match nearest neighbors but also other controls with identical (tied) pscores • Matching performed only within common support area • For each beneficiary up to 20 controls can be matched; this is to utilize large sample available, while wrapping up computations in reasonable time ** Nearest-neighbors matching (NN) gen random = runiform() //created for data sorting before NN matching forval y = 2008/2014 { capture noisily { qui sort random //sort using random numbers to avoid bias during NN matching local caliper=r(sd)/4 //caliper defined as one fourth of PS std. dev. psmatch2 D`y', pscore(xtlhat`y') ties common neighbor(20) cal(`caliper') sort _id //a new identifier created for all observations sum _weight //it holds the frequency with which observation is used as a match pstest $covariates , graph treated(D`y') both //test balance across treatment status graph export pstest_PSM_NN_1_20_`y'.png, replace //name of exported graph drop _id _n1-_n20 //drop to clear up database of redundant variables tab _nn //for every beneficiary it stores the number of matched controls * Create the matched sample indicator //generate indicator to keep observations on common support (by year) gen support`y' = 1 if _support == 1 //equals 1 if the observation is on the common support and 0 if it is off the support 107 gen _weight`y' = _weight gen _treated`y' = _treated gen _nn`y' = _nn } } //after this, we have both xtlhat`y' for IPW estimation and support`y' for DID in matched sample By default, the code line placed above, i.e. pstest $covariates , graph treated(D`y'), both produces balance plot. An example for the 2014 cohort is presented: ** Diagnostic on covariates after matching forval y = 2008/2014 { //last analysed year is 2014 as very little data is available in following years * Check covariates distribution across treatment status after matching di "" di "" 108 di as result "Check covariates distribution across treatment status after matching in " "`y'" foreach v of varlist $covariates_categorical { tab `v' D`y' if year==`y' & support`y'==1, miss chi nokey } foreach v of varlist $covariates_continuous { bysort D`y': sum `v' if year==`y' & support`y'==1 } } ** Visualizing PS distribution before and after matching across treatment status on matched samples forval y = 2008/2014 { capture noisily { qui tabstat support`y', s(n) save qui mat total = r(StatTotal) * Common support area plot before matching #delimit ; twoway (kdensity xtlhat`y' if D`y'==1, lpattern(solid) lcolor(blue*1.25)) (kdensity xtlhat`y' if D`y'==0, lpattern(dash) lcolor(red*1.25)), legend(off) title("Before matching in `y'", size(small)) subtitle("", size(small)) xtitle("Propensity Score", size(small)) ytitle("No. of companies", size(small)) ylabel(none) xlabel(#5,labsize(small)) caption("", size(v. small)) saving(commonSuppBefore`y', replace) ; #delimit cr * Common support area plot on matched samples #delimit ; twoway (kdensity xtlhat`y' if D`y'==1 & support`y'==1 [fw=round(_weight*100)], lpattern(solid) lcolor(blue*1.25)) (kdensity xtlhat`y' if D`y'==0 & support`y'==1 [fw=round(_weight*100)], lpattern(dash) lcolor(red*1.25)), legend(label(1 "Beneficiaries") label(2 "Controls")) 109 legend(size(small)) legend(region(lwidth(none))) legend(ring(0)) legend(position(2)) legend(col(1)) title("After matching in `y'", size(small)) subtitle("", size(small)) xtitle(" ", size(small)) ytitle("", size(small)) ylabel(,labsize(small)) xlabel(#5,labsize(small)) caption("", size(v. small)) saving(commonSuppAfter`y', replace) ; #delimit cr gr combine commonSuppBefore`y'.gph commonSuppAfter`y'.gph, col(2) iscale(1) ycommon graph export commonSuppBeforeAfter`y'.png, replace } } The above codes also produces for each a visual assessment of the matching performance, which will be useful for the practitioner. An example of the matching for the 2014 cohort is presented: 110 A6.2.6 Direct effect estimation for one treatment group These are the processes used for the direct effects estimation where there was one joint treatment group. • The effects were estimated with the use of two alternative weights obtained in matching procedure (matching weights and ATT weights) • Then the effects were estimated as difference-in-differences of average outcome variable values taken from all years after and all years before treatment (treatment defined as 1st project start year) between treated and controls. This provides an illustration of the estimation for each year cohort and what was used in the outcome variables. 2007 |---| 2008 |----------------------------------------------------------------------| |-------------| 2009 |-------------------------------------------------------------| |-----------------------| 2010 |---------------------------------------------------| |---------------------------------| 2011 |-----------------------------------------| |-------------------------------------------| 2012 |--------------------------------| |-----------------------------------------------------| 2013 |----------------------| |---------------------------------------------------------------| 2014 |------------| 2015 2016 111 before |----| after ** DID effects on matched sample * Run DID using matching weights //with option ties and k-nearest neighbours matching _weight holds the normalized matching weight egen fid = group(REGON) //xtreg is not accepting factor/string variables sort fid xtset fid year // First: fill in weight and support variables forval y = 2008/2014 { xfill _weight`y', i(fid) xfill support`y', i(fid) } // Now run regressions on the support sample with weights foreach outcome of varlist $outcomes { est clear forval y = 2008/2014 { capture noisily { di "" di "" di as result "DID effects for: " "`outcome'" "in " "`y'" reg `outcome' treated i.year D`y' if support`y'== 1 [aw=_weight`y'], cluster(REGON) robust eststo xtreg `outcome' i.year D`y' if support`y'== 1 [aw=_weight`y'], fe cluster(REGON) robust eststo } } eststo dir cap noisily esttab est* using "DID NN matching_`outcome'.csv", r2 ar2 /// order(D2008 D2009 D2010 D2011 D2012 D2013 D2014) se staraux /// nodepvar nonumber wide nogaps replace title("DID NN matching for: `outcome'") //similarly results could be exported to other formats } ** Create ATT and ATE weights for regressions 112 forval y = 2008/2014 { gen treated`y' = treated replace treated`y' = . if D`y' == . } forval y = 2008/2014 { * Filling in estimated PS xfill xtlhat`y', i(fid) * Now create weights based on treatment status /*Need yearly treatment status; otherwise the weights can be different across years as PS is year-specific: later year values will replace earlier year values*/ gen double w_att`y' = cond(treated`y'==1, 1, xtlhat`y'/(1-xtlhat`y')) gen double w_ate`y' = cond(treated`y'==1, 1/xtlhat`y', 1/(1-xtlhat`y')) } ** Effects estimation using ATT weights (in similar code ATE weights could be used) foreach outcome of varlist $outcomes { estimates clear forval y = 2008/2014 { capture noisily { di "" di "" di as result "DID effects using ATT weights for: " "`outcome'" "in " "`y'" reg `outcome' treated i.year D`y' [pw=w_att`y'], cluster(fid) robust eststo xtreg `outcome' i.year D`y' [pw=w_att`y'], fe cluster(fid) robust eststo } } eststo dir cap noisily esttab est* using "DID IPW_ATT_`outcome'.csv", r2 ar2 /// order(D2008 D2009 D2010 D2011 D2012 D2013 D2014) se staraux /// nodepvar nonumber wide nogaps replace title("DID NN matching for: `outcome'") //similarly results could be exported to other formats } 113 ** Visualizing distribution of matching weights across years; the code below regards ATT weights (w_att), however the same code could be used for ATE (w_ate) and NN matching (_weights) weights #delimit ; twoway (kdensity w_att2008, lpattern(solid) lcolor(blue*0.75)) (kdensity w_att2009, lpattern(dash) lcolor(brown*1.25)) (kdensity w_att2010, lpattern(dot) lcolor(cyan*0.75)) (kdensity w_att2011, lpattern(shortdash) lcolor(green*1.25)) (kdensity w_att2012, lpattern(longdash) lcolor(lime*0.75)) (kdensity w_att2013, lpattern(solid) lcolor(mint*1.25)) (kdensity w_att2014, lpattern(dash) lcolor(purple*0.75)), legend( label(1 "2008") label(2 "2009") label(3 "2010") label(4 "2011") label(5 "2012") label(6 "2013") label(7 "2014")) legend(size(small)) legend(region(lwidth(none))) legend(ring(0)) legend(position(2)) legend(col(1)) title("ATT weights", size(small)) subtitle("", size(small)) xtitle("ATT weights", size(small)) ytitle("Density", size(small)) ylabel(,labsize(small)) xlabel(#10,labsize(small)) caption("", size(v. small)) ; #delimit cr graph export att_weights.png, replace The codes produce the diagram of the ATT weights that will be useful for the practitioner. The diagram produced in the report is presented here for illustration: 114 A6.2.7 Effects heterogeneity analysis The estimation examined the heterogeneous impact of program on different outcome variables, such as employment, profits, productivity. Firm characteristics were interacted with the treatment variables. ** Specify interaction terms with: firm size, age, sector, region, exporter, has R&D, pre-treatment growth * Check and if not available recreate lagged values from year before company’s first project launch foreach var in medium large compLife_over3yr C_Manufacturing … { forval y = 2008/2014 { capture confirm var `var'_`y' if !_rc { xfill `var'_`y', i(fid) di in green "`var'_`y' exist" } else { gen `var'_`y' = L_`var' if year == `y' xfill `var'_`y', i(fid) } } 115 } * Create interactions, i.e. take value for each year foreach var in medium large compLife_over3yr C_Manufacturing … { forval y = 2008/2014 { capture confirm var D_`var'_`y' if !_rc { di in green "D_`var'_`y' exist" } else { gen D_`var'_`y' = D`y'*`var'_`y' } } } global int1 D_medium_20* D_large_20* global int2 D_compLife_over3yr_20* global int3 D_C_Manufacturing_20* D_F_Construction_20* D_G_Trade_20* global int4 D_NACElev2_20* global int5 D_regions_20* global int6 D_RDwkCostRev01_20* global int7 D_exportingComp_20* global int8 D_emplFTEYoY_20* global int9 D_revNetTotalYoY_20* ** Program related variables: subsidy amount, length of treatment foreach var in amount { forval y = 2008/2014 { capture confirm var D_`var'_`y' if !_rc { di in green "D_`var'_`y' exist" } else { gen D_amount_`y' = D`y'*ln(cofinancing) } } } global int10 D_amount_20* global int11 D20* 116 foreach v of varlist POIG POKL regional RPDS RPKP RPLB RPLD RPLU RPMA RPMP RPOP /// RPPD RPPK RPPM RPSL RPSW RPWM RPWP RPZP { tab `v' recode `v' (1/100=1) } foreach v of varlist POIG POKL regional RPDS RPKP RPLB RPLD RPLU RPMA RPMP RPOP /// RPPD RPPK RPPM RPSL RPSW RPWM RPWP RPZP { forval y = 2008/2014 { capture confirm var `var'_`y' if !_rc { xfill `var'_`y', i(fid) di in green "`var'_`y' exist" } else { gen `var'_`y' = `var' if year == `y' xfill `var'_`y', i(fid) } } } foreach v of varlist POIG POKL regional RPDS RPKP RPLB RPLD RPLU RPMA RPMP RPOP /// RPPD RPPK RPPM RPSL RPSW RPWM RPWP RPZP { forval y = 2008/2014 { forval y = 2008/2014 { capture confirm var `var'_`y' if !_rc { di in green "D_`var'_`y' exist" } else { gen D_`var'_`y' = D`y'*`var'_`y' } } } global int12 D_POIG_20* global int13 D_POKL_20* global int14 D_regional_20* global int15 D_RP*_20* forval i = 1/10 { foreach y of global int`i' { 117 di "" di "global int`i':" ds `y', varwidth(32) } } ** Outcome of interest global outcomes "ln_emplFTE ln_rv_revNetTotal ln_rv_valueAdded … " ** Check the values of the interaction terms sum $int1 $int2 $int3 $int4 $int5 $int6 $int7 $int8 $int9 $int10 /*$int11*/ /// $int12 $int13 $int14 $int15 ****************************************** ** DID effects on matched sample with interaction terms ** Run DID using matching weights // Now run regressions on the support sample with weights foreach outcome of varlist $outcomes { est clear forval y = 2008/2014 { capture noisily { eststo: xtreg `outcome' i.year D`y' D_medium_`y' D_large_`y' if support`y'== 1 [aw=_weight`y'], fe cluster(REGON) robust eststo: xtreg `outcome' i.year D`y' D_compLife_over3yr_`y' if … eststo: xtreg `outcome' i.year D`y' D_C_Manufacturing_`y' D_F_Construction_`y' D_G_Trade_`y' if … eststo: xtreg `outcome' i.year D`y' i.D_regions_`y' if … eststo: xtreg `outcome' i.year D`y' D_RDwkCostRev01_`y' if … eststo: xtreg `outcome' i.year D`y' D_exportingComp_`y' if … eststo: xtreg `outcome' i.year D`y' D_emplFTEYoY_`y' if … eststo: xtreg `outcome' i.year D`y' D_revNetTotalYoY_`y' if … eststo: xtreg `outcome' i.year D`y' D_amount_`y' if … eststo: xtreg `outcome' i.year D`y' D_POIG_`y' D_POKL_`y' D_regional_`y' if … eststo: xtreg `outcome' i.year D`y' D_POIG_`y' D_POKL_`y' D_RPDS_`y' D_RPKP_`y' D_RPLB_`y' D_RPLD_`y' D_RPLU_`y' D_RPMA_`y' D_RPMP_`y' D_RPOP_`y' D_RPPD_`y' D_RPPK_`y' D_RPPM_`y' D_RPSL_`y' D_RPSW_`y' D_RPWM_`y' D_RPWP_`y' D_RPZP_`y' 118 } } eststo dir cap noisily esttab est* using "DID NN matching het impact_`outcome'.csv", r2 ar2 /// se staraux nodepvar nonumber wide nogaps replace title("DID NN matching for: `outcome'") } A6.2.8 Matching and direct effect estimation for multiple treatment groups The estimation for multiple treatment groups were conducted using inverse probability matching. The process is as follows: • matching was performed jointly for all year-cohorts of the beneficiaries. • covariates to match-on values taken from 2007 for all beneficiaries. • outcome variables values taken as average from years 2008-2016. • effects estimated as difference-in-differences of average outcome variable values taken from all years 2008-2016, i.e. after treatment and 2007, i.e. before treatment. • different grouping schemes of the interventions has been tested to ensure the final groupings are stable and interpretable. An illustration of how the before and after variables were constructed for the 2007 and 2008 cohort is as follows: Before |------------------ after ----------| 2006 |------------------------------2008-2016--------------------------------------| 2006-08 |------------------------------2009-2016--------------------------------------| // Collapse twice then merge to avoid the error of repeating variables preserve sort REGON year collapse (mean) T2 $covariates_cont_1 (lastnm) $covariates_binary_1 $covariates_cat_1, by(fid after) tab T2 tempfile X1 save `X1' restore collapse (mean) $list_outcomes, by(fid after) merge 1:1 fid after using `X1' drop _merge 119 * Gen outcome of interest as the before-after difference of outcomes cap egen fid = group(REGON) xtset fid after foreach o in $list_outcomes { capture confirm variable diff_`o', exact if !_rc { display "diff_`o' exists" drop diff_`o' } gen diff_`o' = `o' - `o'[_n-1] } * Gen lag covariates for the propensity score estimation foreach c in $X1 { capture confirm variable L_`c', exact if !_rc { display "L_`c' exists" drop L_`c' } gen L_`c' = `c'[_n-1] } * Keep only after data, run teffects & export results * Note that MNL can take long to converge & teffects overlap is computationally intensive keep if after == 1 global ps_tolerance1 "0.001" global ps_tolerance2 "0.000000000000000000001" // very low tolerance set to avoid code break if companies with predicted 0 or 1 PS occured foreach o of varlist $list_outcomes { cap noisily { // X1_spec1 & outcome levels qui teffects ipw (`o') (T2 $X1_spec1) , ate vce(robust) control(0) pstolerance($ps_tolerance1) osample(v_`o'_1) nolog iterate(50) // X1_spec1 & B-A outcome differences (equivalent to DID) qui teffects ipw (diff_`o') (T2 $X1_spec1) , ate vce(robust) control(0) pstolerance($ps_tolerance1) osample(v_`o'_3) nolog iterate(50) } 120 } est clear foreach o of varlist $list_outcomes { cap noisily { // X1_spec1 & outcome levels capture confirm var v_`o'_1 if !_rc { teffects ipw (`o') (T2 $X1_spec1) if v_`o'_1!=1 , ate vce(robust) control(0) pstolerance($ps_tolerance2) nolog iterate(50) eststo m1_`o' teffects overlap graph export NoBSI_T2_ipw1_`o'.png, replace } else { teffects ipw (`o') (T2 $X1_spec1) , ate vce(robust) control(0) pstolerance($ps_tolerance2) nolog iterate(50) eststo m1_`o' teffects overlap graph export NoBSI_T2_ipw1_`o'.png, replace } // X1_spec1 & B-A outcome differences (equivalent to DID) capture confirm var v_`o'_3 if !_rc { teffects ipw (diff_`o') (T2 $X1_spec1) if v_`o'_3!=1 , ate vce(robust) control(0) pstolerance($ps_tolerance2) nolog iterate(50) eststo m2`o' teffects overlap graph export NoBSI_T2_ipw3_`y'.png, replace } else { teffects ipw (diff_`o') (T2 $X1_spec1) , ate vce(robust) control(0) pstolerance($ps_tolerance2) nolog iterate(50) eststo m2`o' teffects overlap graph export NoBSI_T2_ipw2_`y'.png, replace } } } 121 foreach format in "html" "csv" "tex" { cap n forval i = 1/2 { cap n esttab m`i'* using "NoBSI_T2_teffects`i'.`format'", se staraux nodepvar nonumber title(teffects ipw: `o') wide nogaps replace } } A6.2.9 Indirect effects estimation The indirect effects estimation examines the effect of each spillover variable on firm outcomes. * Defining list of outcomes: investment, R&D, employment, sales, exports, profits, value added, wages, LP, TFP #delimit ; global list_outcomes ln_emplFTE ln_rv_revNetTotal ln_rv_valueAdded exportingComp ln_rv_expValue ln_avgWagesFTE ln_labrProd rv_EBT TFPR ln_RDwkCostRev ln_faInvRev ln_intFaInvRev ; #delimit cr /* 1. Running regressions for DIRECT horizontal, vertical (backward + forward), horizontal + vertical, regional, all 4 types of spillovers on the list of outcomes on non-beneficiaries only.*/ foreach x of varlist $list_outcomes { capture noisily xtreg `x' dir_cofin_year_nace i.year i.rnacerev1 i.region_code if touse==1, fe vce(cluster nacerev1) nonest outreg2 using unbalanced_cofin_direct.xls, excel drop(i.rnacerev1 i.year i.region_code) addtext(Sector FE, YES, Region FE, YES, Year FE, YES) capture noisily xtreg `x' dir_cofin_backward dir_cofin_forward i.year i.rnacerev1 i.region_code if touse==1, fe vce(cluster nacerev1) nonest outreg2 using unbalanced_cofin_direct.xls, excel drop(i.rnacerev1 i.year i.region_code) addtext(Sector FE, YES, Region FE, YES, Year FE, YES) 122 capture noisily xtreg `x' dir_cofin_year_nace dir_cofin_backward dir_cofin_forward i.year i.rnacerev1 i.region_code if touse==1, fe vce(cluster nacerev1) nonest outreg2 using unbalanced_cofin_direct.xls, excel drop(i.rnacerev1 i.year i.region_code) addtext(Sector FE, YES, Region FE, YES, Year FE, YES) capture noisily xtreg `x' dir_cofin_year_reg i.year i.rnacerev1 i.region_code if touse==1, fe vce(cluster region_code) nonest outreg2 using unbalanced_cofin_direct.xls, excel drop(i.rnacerev1 i.year i.region_code) addtext(Sector FE, YES, Region FE, YES, Year FE, YES) capture noisily xtreg `x' dir_cofin_year_nace dir_cofin_backward dir_cofin_forward dir_cofin_year_reg i.year i.rnacerev1 i.region_code if touse==1, fe vce(cluster nacerev1) nonest outreg2 using unbalanced_cofin_direct.xls, excel drop(i.rnacerev1 i.year i.region_code) addtext(Sector FE, YES, Region FE, YES, Year FE, YES) } /*2. Running regressions for INDIRECT regional spillovers on the list of outcomes on non-beneficiaries only.*/ foreach x of varlist $list_outcomes { capture noisily xtreg `x' i_cofin_year_reg i.year i.rnacerev1 i.region_code if touse==1, fe vce(cluster region_code) nonest outreg2 using unbalanced_indirect.xls, excel drop(i.rnacerev1 i.year i.region_code) addtext(Sector FE, YES, Region FE, YES, Year FE, YES) } 8.3 Practical issues A6.3.1 Working arrangements with Statistics Poland Due to legal limitations, Statistics Poland does not make any micro-data available to external entities.28 ln the course of the above counterfactual evaluations, a model of collaboration with Statistics Poland was developed to allow the use of the micro-data. ln simplified terms, the procedure was as follows: 1. A list of beneficiaries was submitted to Statistics Poland. The list of beneficiaries is publicly available, so the data was downloaded and cleaned by the World Bank team. 2. Based on the beneficiaries’ identification numbers (REGON and NIP), Statistics Poland identified these enterprises and marked them in its databases. 3. Next, the World Bank team developed appropriate codes for data analysis (script) and handed them to the Statistics Poland. 28 Cf. the Act of 29 June 1995 on Official Statistics (Dz. U [Journal of Laws] of 1995, No 88, item 439, as amended). 123 4. Statistics Poland, using the same statistical analysis software (Stata), ran the code, which in step one, matched “statistical twins" to the beneficiaries (from the population of remaining non- beneficiaries companies whose data are at Statistics Poland’s disposal), and then, in the second step, calculated specific indicators of intervention effects. 5. Finally, Statistics Poland submitted the results of the analyses to the World Bank team who assessed the quality of the match obtained and the stability of estimated effects. 6. The process was repeated until a satisfactory match and stable estimated effects are obtained. A6.3.2 Lessons learnt and good practices Through the process of working with Statistics Poland, the World Bank team identified some lessons and good practices that may be useful for future practitioners who face similar working arrangements: • The practitioner should not aim to create general codes that attempt to perform all planned analyses together, as it is too ambitious a task. It would be more effective to prepare and test smaller code chunks on the data and progressively create from them more comprehensive codes. • While requesting results of the analyses from Statistics Poland, it is crucial to have in one batch of results all the files produced during analyses along with *.log/*.smcl file, and most importantly, the final codes used. As there may be many iterations, this will be useful for the practitioner to keep track of the codes used and results produced. • If the practitioner is faced with a similar working arrangement with the Statistics office (where there is no direct access to the micro data), it is imperative to have a local expert to provide just- in-time support on the codes and to facilitate the collaboration process. This local expert should have experience working on similar evaluations and a good knowledge of the structure of the micro data. 124