How to Make Grants a Better Match for Private Sector Development Review of World Bank Matching Grants Projects CIIP Competitive Industries and Innovation Program Financed by in partnership with © 2016 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judg- ment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. Report design by Voilà Information Design chezvoila.com Table of Contents Acknowledgements  VI Abbreviations and acronyms  VII Executive summary  1 1. Introduction  9 1.1. Objective  9 1.2. Structure of the report  10 2. Background and literature review  11 2.1. Background on MGs  11 2.2. Literature review  13 3. Study methodology  17 3.1. Selection criteria  17 3.2. Overall project distribution  18 4. Design features of MG schemes  21 4.1. Program scale and number of beneficiaries  21 4.2. Implementing agency  25 4.3. Beneficiary eligibility criteria  29 4.4. Sector focus  37 4.5. Eligible expenses  43 4.6. Selection mechanism  47 4.7. Grant modalities  49 4.8. Provision of diagnostics and TA to firms  57 4.9. Link with other project components  59 4.10. Series of projects  59 4.11. Implementation and disbursement of funds   61 5. Design characteristics of successful projects  63 5.1. Features of successful and unsuccessful projects  63 5.2. Econometric analysis  65 6. Defining and measuring success  69 6.1. Institutional arrangements for M&E   69 6.2. What do we measure?  69 6.3. Is this the right thing to measure?  70 6.4. Design and implementation tailored to success   71 7. Conclusions and recommendations  73 7.1. Economic rationale  74 7.2. Design and implementation  74 7.3. Monitoring & Evaluation  75 References  76 Annex 1: Results indicators used in reviewed projects  78 Annex 2: List of reviewed MG projects implemented by the World Bank Group  79 Annex 3: Correlation between projects’ design features and outcomes  83 IV Case studies 1: Alternative and complementary instruments  14 2: Choosing the appropriate implementing agency  24 3: Large and micro firms in MG schemes  28 4: MG and groups of firms  30 5: Provision of MGs to BDS providers  32 6: MG schemes in agriculture  36 7: Use of MG to foster exports  40 8: Use of MG to purchase equipment  42 9: Competitive allocation of MG  46 10: Multiple MG windows  48 11: Advance payment of eligible expenses  52 12: Provision of services by public institutions  54 13: Matching requirement for initial technical assistance  56 14: Repeated MG projects  58 15: High and low disbursement of MG projects  60 16: Highly successful and unsuccessful projects  66 17: Randomized allocation of MGs for impact evaluation  68 18: Economic rationale for MG projects  74 Structure This document is a systematic review of a number of features of matching grants schemes to identify whether they are associated with successful project outcomes or not. The reports successive sections describe different design features and present relevant case studies of projects implemented by the Bank for each. Case studies Feature description V Acknowledgements This report was written by Diana Hristova and Antoine Coste, both in the Trade and Competitiveness (T&C) Global Practice of the World Bank, under the guidance of Ivan Rossignol (Chief Technical Spe- cialist, T&C) and Michael Wong (Lead Private Sector Specialist and Program Manager, T&C). The report benefited from inputs by Iradj Alikhani, Andres Garcia, and Abhishek Saurav, as well as from comments provided by Francisco Campos, Xavier Cierra, Leonardo Iacovone, David McKenzie, Jose Guilherme Reis, Sunita Varada, and Panos Varangis. The team is grateful for the support of the Competitive Industries and Innovation Program (CIIP), a trust fund in T&C. VI Abbreviations and acronyms ACP African, Caribbean, and Pacific Group of States Secretariat BDS Business Development Services CEDF Community Economic Development Fund CPIA Country Policy and Institutional Assessment EMAF Export Market Access Fund FPD Finance and Private Sector Development GDP Gross Domestic Product ICR Implementation Completion and Results Report IEG Independent Evaluation Group ISR Implementation Status and Results Report M&E Monitoring and Evaluation MG Matching Grant MSME Micro, Small, and Medium Enterprise PAD Project Appraisal Document PIU Project Implementation Unit R&D Research and Development SME Small and Medium Enterprise T&C Trade and Competitiveness TA Technical Assistance WTO World Trade Organization VII Executive summary Matching grants (MGs) have been implemented by the World Bank for over two decades. They remain a very popular instrument for private sector development interventions, despite often challenging implementation and insufficient evidence of impact. The objective of this study is to synthesize the current knowledge on MGs and to review the experience with this instrument, as designed and imple- mented by the World Bank from the early 1990s to the present. In doing so, we hope to equip teams in charge of ongoing and planned MG operations with a better understanding of the instrument and to help them choose the design and implementation arrangements that are best fitted to their ob- jectives. We look at both the “why” and the “how” of MG programs, focusing on those aiming to foster private sector development and small and medium enterprises (SMEs) competitiveness primarily through the use of business development services (BDS). We review a number of core design features and implementation arrangements most often used in MG schemes across a sample of 106 projects and present an aggregate view of the frequency of their use, offer descriptive case studies to show how each MG is implemented in practice, and correlate each feature with success. Based on the evidence available, we find that there is no design blueprint as no design features are systematically correlated with a specific outcome and that therefore programs need to be appropriately tailored to local circumstances and capacities to be successful. Table 1 sum- marizes the different design features we investigated and offers questions that could guide project teams when selecting features. Based on our descriptive analysis and review of existing studies, we offer a number of general principles to guide the design and implementation of MG programs. We also look at how success is defined and question the way the outcome of MG programs is often assessed. While 75 percent of projects in the sample were deemed to have some degree of success, the definition of success rarely reflects a measurement of broad and sustainable economic benefits that would justify the subsidization of private enterprises with public funds. We argue that this is linked to a common weakness in identifying a sound economic rationale, such as a specific market failure to be solved, and in subsequently not focusing the design and measurement of the MG on the fund’s additionality beyond the private benefit of recipients. We conclude that a robust definition of the economic rationale is a critical prerequisite for the use of MG programs and should include, among others, an identification of a specific market failure, consid- eration of alternative instruments, cost-benefit analysis, assessment of the potential for additionality and spillovers, and a realistic exit strategy. We recommend an equally robust monitoring and evalua- tion (M&E) system tied directly to the economic rationale, which is essential for real-time assessment of impact, potential course correction, and learning, and could be utilized to gauge additionality and sustainability. Increased attention to these elements could help teams make the most of this potential- ly powerful instrument for private sector development and competitiveness. 1 Table 1: Options for the design of MG programs Economic rationale Main variables and options Consideration for teams Context-specific analytical underpinnings ¢¢ Has a targeted population clearly been identified? Is there enough in- and demand assessment formation about this group’s needs and about the supply and demand for BDS? What are the main market failures identified and what would Project economic and financial analysis be the impact of lifting them? (e.g., cost-benefit, least-cost) Use of complementary or alternative ¢¢ To what extent is a MG scheme the most cost-effective instrument instruments available to tackle these market failures and/or foster these spillovers in this specific context? Can complementary interventions increase the scope and sustainability of benefits? ¢¢ Can full or substantial additionality of public MG funds be ensured? In addition to direct impact on grant recipients, are spillovers likely to materialize? ¢¢ What is the exit strategy and how sustainable are project benefits likely to be? What is the expected impact on supply and demand in the BDS market at the end of the prog ram? Is there a risk that an increased demand for BDS results in higher prices rather than higher supply (e.g., restrictions on the entry of new providers or on imports of services)? Program scale and number of beneficiaries Main variables and options Consideration for teams Amount of the MG fund ¢¢ How many MGs will be extended and what proportion of the target population will this represent? Number of beneficiaries targeted Maximum size of a single grant ¢¢ Is the scale of the program sufficient to have a meaningful impact on the targeted population? To what extent does the project impact rely Possibility of multiple grants per on indirect benefits/spillovers? beneficiary and maximum cumulative amount ¢¢ What will be the average and maximum amount of grants? Is this in line with targeted beneficiaries’ needs? 2 Implementing agency and institutional arrangements Main variables and options Consideration for teams Agency type (e.g., Project ¢¢ Is there an agency with experience with MGs? Is there scope to build Implementation Unit [PIU], private public capacity through temporary external assistance before and/or contractor, professional association, or during MG implementation? public-private body) ¢¢ Does the selected agency have the required technical, human, and Governance and oversight arrangements administrative capacity? Is training planned for its staff? Will an MG Incentive structure (e.g., payment manual be prepared before implementation? modalities and performance ¢¢ Which implementation model is likely to be the most cost-efficient? requirements) ¢¢ Are there risks of administrative bottlenecks outside the control of the implementing agency? ¢¢ What is the risk of political capture? Are checks and balances ade- quate? Will the private sector be involved in program oversight and grant allocation decisions? ¢¢ What are the incentives for the implementing agency to perform? Are there mechanisms in place to ensure that staff of the implementing agency are recruited based on qualifications? Beneficiary eligibility criteria Main variables and options Consideration for teams Size of firms ¢¢ Do the project’s objectives require to focus on (or exclude) certain cat- egories of firms? If yes, what are the design elements needed to adapt Possibility of group application to their specific needs? Eligibility of BDS providers ¢¢ Are the eligibility criteria, documentary requirements, and application Other (e.g., start-up vs. established firms, procedure clear and transparent? Are they likely to make the program informal vs. officially registered, domestic attractive and accessible to the target population? Do they strike the vs. foreign ownership, public vs. private right balance between openness and targeting? ownership, and location) ¢¢ Are the criteria and application appraisal method directly linked to the project objectives? Do they focus on activities not already routinely undertaken by firms? Do they maximize the chances of generating additionality and spillovers? ¢¢ Would there be additional economic benefits from allowing joint ap- plications and would such applications be likely to be submitted? ¢¢ Is there a market failure restricting the quantity and quality of BDS sup- ply? If yes, could MG be extended to service providers or would other interventions be more appropriate? 3 Sector focus Main variables and options Consideration for teams Sector restrictions ¢¢ Do the project objectives imply a focus on one/several specific sectors or activities, including exports? If yes, is this best achieved by restrict- Focus on exports vs. general ing eligibility to corresponding firms or by focusing outreach efforts competitiveness while leaving eligibility criteria open? ¢¢ If a sector or export focus is adopted, is the program consistent with international trade commitments? Eligible expenses Main variables and options Consideration for teams Eligibility of equipment and capital ¢¢ Which types of services and activities should be made (in)eligible for goods, recurrent expenses, etc. in grant support to best achieve project objectives (e.g., acquisition of addition to services intangible assets such as certification, preparation of feasibility study for future investments, and building of new capacities through train- ing)? Which services are the most likely to generate additionality and externalities? ¢¢ Is there an economic rationale for financing tangible assets and/or re- curring expenses? If yes, should these be restricted to common/public goods? Are there better alternative interventions in the short and/or medium term that could help firms acquire those assets? ¢¢ If financing of equipment is justified, are the risks of doing so ade- quately addressed (e.g., resale of subsidized goods)? What will be the sustainability of the impact after the end of the project? Selection mechanism Main variables and options Consideration for teams First-come, first-served for eligible firms ¢¢ What is the likely capacity of the implementing agency to identify the vs. competitive grant attribution projects most likely to generate additionality and spillovers? ¢¢ Which award mechanism is likely to generate the highest take-up? ¢¢ Which selection mechanism is best suited for the planned project eval- uation strategy? 4 Grant modalities Main variables and options Consideration for teams Level of match and number of funding ¢¢ Is there a need for a single or multiple funding window(s) and/or sub- windows and fixed versus variable match sidization rates? Have the pros and cons of each been considered (for level (over time and across types of example, administrative costs and targeting)? Should multiple grants beneficiaries or activities) per firms be authorized? Beneficiary contribution (cash versus ¢¢ Does the selected level(s) of match strike an adequate balance be- in kind and partial or full grant tween several objectives (for example, attractiveness of the scheme, reimbursement in case of success) beneficiary commitment, and risks of abuse)? Have other elements of Modality of payment to beneficiaries (for program attractiveness been leveraged to reduce the need for an ex- example, advance versus reimbursement cessive subsidy level? Is the subsidy level consistent with the expected basis and tranches upon partial delivery) additionality and spillovers? Consultant selection (for example, free ¢¢ Are the requirements for beneficiaries’ contribution and grant payment selection by beneficiary provided basic adequate for their capacity? Are advance payments or partial payment eligibility criteria are met, selection for agreed milestones needed for certain firms because of liquidity from a roster of preapproved providers constraints? Are the mechanisms in place likely to result in prompt maintained by the implementing agency, payment of grants? Is there a significant risk of nonpayment or de- and mandatory competitive award) layed payment by beneficiaries to address? ¢¢ Are there sufficient providers of quality BDS and should the BDS supply side be supported with MGs or another intervention? Is there a lack of information on which providers are reliable? Diagnostic and technical assistance (TA) to beneficiaries Main variables and options Consideration for teams Two-stage application process with ¢¢ How specific/complex are the projects to be supported through MGs mandatory diagnostic first phase to and are many non-eligible applications likely to be submitted? screen eligible and most-deserving firms ¢¢ What is the average capacity level of the targeted firms? Do they need Free or partially free technical support to support to elaborate/implement a business plan? Is a sufficient pipe- firms to strengthen general business skills line of applicants likely to materialize in the absence of such support? and/or elaborate a business plan to apply for an MG ¢¢ What human, financial, and material resources are or could be made available to provide technical support to applicants and beneficiaries? Continuous assistance and training to MG beneficiaries during project implementation 5 Link with other project components Main variables and options Consideration for teams Complementary project components ¢¢ What external constraints can undermine the impact of the MG to provide firm-level support and/or scheme and what interventions could best address them? address general business environment constraints (for example, institutional ¢¢ To what extent will/could synergies between the MG program, other capacity building, business environment project components, and/or other projects be leveraged? and regulatory reforms, and access to ¢¢ Are the project’s institutional arrangements amenable to coordination finance) between different components? Coordination with other projects and policies Series of projects Main variables and options Consideration for teams Follow-up MG schemes building on ¢¢ Is there experience in the country with MGs, financed by the World previous country experience Bank or other institutions? If yes, have lessons been drawn through evaluation? What design features should be kept/changed? ¢¢ Given the country capacity or MG program characteristics, would it be preferable to start with a limited-scale pilot and subsequently expand in case of success? 6 Monitoring and evaluation Main variables and options Consideration for teams M&E responsibilities and arrangements ¢¢ Has M&E been considered as an integral element of the project? Will (e.g., budget, internal, and/or external) the implementing agency have adequate resources to perform con- tinuous M&E and final project evaluation (e.g., dedicated staff, informa- Impact evaluation strategy (e.g., ex tion system, and M&E manual)? Can internal project M&E be comple- ante, ex post, experimental, or quasi- mented by independent external evaluation? experimental designs) ¢¢ Has an adequate results framework been established? Are the indica- tors well defined? Do they measure not only output, but also outcome and impact? How will the results of M&E be used to adapt the pro- gram design during implementation? ¢¢ Has an evaluation strategy been built from the start in the project de- sign? Has baseline data been collected? What are the data reporting requirements from beneficiaries? How will data be collected for a com- parable group of non-beneficiary firms? Will data be collected after project completion to evaluate the evolution of impact over time? ¢¢ How will the evaluation assess the additionality of funds, the attribut- ability of results, the spillovers generated, etc.? Is a randomized design for grant allocation politically, financially, and technically realistic? Other important considerations Main variables and options Consideration for teams Communication, marketing, and ¢¢ What marketing and outreach initiatives are planned to ensure that stakeholder engagement the MG reaches the intended beneficiaries (e.g., type of firm and lo- cation)? Have private sector representatives been consulted on the Ownership and alignment with design of the scheme? government priorities Project external environment ¢¢ Is there broad political ownership for the program and is this likely to continue? Is the MG program fully aligned with national development Design flexibility and adaptation (e.g., strategies and policies? Are there adequate mechanisms in place to scope, eligibility criteria, subsidy level, ensure interagency coordination? application procedure, and institutional arrangements) ¢¢ Is the overall economic environment appropriate for the implemen- tation of an MG program and expected to remain so (e.g., macroeco- nomic and political stability, business environment, and level of corpo- rate depth and competition)? ¢¢ Are mechanisms in place to collect beneficiaries’ feedback? Is there enough flexibility to adapt the program during implementation and address any unforeseen issue or erogenous shock? 7 Case study 8 1. Introduction 1.1. Objective MGs have been a staple of development interventions for decades, favored by institutions such as the World Bank as well as by many governments around the globe. The instrument has been used exten- sively to support a large number of policies on issues ranging from innovation and entrepreneurship to agriculture. The objective of this report is to synthesize the current knowledge on MGs and to re- view the experience with the instrument, as designed and implemented by the World Bank from the early 1990s to the present.1 In doing so, the report focuses on the main types of projects using MGs, namely schemes aimed at fostering private sector development and SME competitiveness through the use of BDS. Despite the accumulated implementation experience, evidence of impact appears to be weak. A recent Independent Evaluation Group (IEG) review of World Bank SME operations conclud- ed that “[there is] limited evidence of the benefits of matching grants and advisory services” (IEG 2014). This study analyzes a number of different design features and implementation arrangements for MGs, uses descriptive case studies to show how each is implemented in practice, and correlates each fea- ture with success or failure. The study intends to equip teams in charge of ongoing and planned MG operations with a better understanding of the instrument and to help them choose the design and implementation arrangements that are best fitted to their objectives. Given the regional distribution of matching grants projects, the study will particularly benefit African, Caribbean and Pacific (ACP) coun- tries and guide their policy-makers in designing effective enterprise support policies .The full database with all projects in the sample and information on their various characteristics (e.g., MG amount, op- erating costs, implementing agency, expected and actual number of beneficiaries, eligibility criteria, subsidy level, payment modalities, outcome etc.) is available separately for interested readers who can tabulate the data according to their needs to get more granular information. For the purpose of this study, an MG is defined as a short-term temporary subsidy targeting knowl- edge and BDS acquisition by SMEs, which is provided to the private sector on a cost-sharing basis. BDS typically include a wide variety of nonfinancial services such as labor and management training; extension, consultancy, and counseling; marketing and information services; or technology develop- ment and diffusion. 1 Many other international, regional, and national institutions have designed and implemented MG programs in recent de- cades. Due to time and information constraints, the projects of other development partners are outside the scope of this study. See annex 1 in Goldberg and Ortiz del Salto (2009) for information about programs implemented by the Inter-American Devel- opment Bank, the Asian Development Bank, and the U. S. Agency for International Development. 9 1.2. Structure of the report The structure of the report is as follows: Section 2 provides background on MGs and a brief overview of the literature on the instrument. Section 3 provides details on the study methodology and how the sample was selected. Section 4 describes each of the features in turn and provides examples of implementation in practice. For each of the features analyzed, the section also provides a breakdown of success or failure in the subset of projects implementing it. Section 5 presents the findings of a re- gression analysis motivated to identify any correlation between project outcomes and different design features once various aspects of country context are controlled for. Section 6 delves further into what the definition of ‘success’ for MG programs generally is, what it should be, and how it can be measured. Section 7 concludes and the annexes include additional information such as a list of the projects re- viewed and results indicators used. 10 2. Background and literature review 2.1. Background on MGs MGs have been one of the most common policy tools used by governments and development part- ners to promote SME competitiveness and growth. Phillips (2001) reports that the first initiatives of this nature were piloted in Europe starting with Ireland in the early 1960s, while the earliest World Bank support for such a scheme was in projects in India and Indonesia in 1986. Since then, over a 100 World Bank projects have been implemented with an MG component. Most of these projects have support- ed private sector and SME development. A significant proportion supported agriculture and rural de- velopment in the 2000s, although the number of such projects seems to have declined in more recent years (figure 2.1). First-generation projects in private sector and SME development tended to target actual or potential exporting firms and focus on trade promotion. In comparison, more recent projects have adopted a broader emphasis on firm-level competitiveness, ranging from basic management capacity for small firms to innovation and research and development (R&D) (Goldberg and Ortiz del Salto 2009). Figure 2.1 Approval of World Bank projects with an MG component (fiscal years) Number of projects 30 25 20 15 Private sector development Agriculture 10 5 0 1991-1995 1996-2000 2001-2005 2006-2010 2011-2015 11 The theoretical economic rationale for the use of MGs is well understood.2 Campos et al. (2012) pro- vides a useful synthesis of the possible justifications for the use of public funds through MGs. In a nut- shell, such subsidies generally aim at addressing a perceived underinvestment by SMEs in BDS within two main possible contexts: • BDS are a profitable investment, but various market failures prevent firms from purchasing them. This can, for instance, be because of the credit constraint faced by SMEs with limited working capital, as lenders are reluctant to finance ‘soft’ services that cannot be collateralized and whose return on investment is difficult to assess ex ante. SMEs may also be risk averse and avoid invest- ing in services with a high but uncertain potential return. Finally, SMEs may not have sufficient information on the BDS available and their potential return or on their relative performance com- pared with competitors. Similarly, BDS providers may lack information about the type or quantity of services demanded, leading to inadequate supply and underinvestment from the providers. By providing a mechanism to screen and strengthen business development projects and reduce the effective cost of the needed services for SMEs, MG schemes can address some of these market failures and catalyze the market for BDS. • The expected private return of investing in BDS might not justify its cost for profit-maximizing SMEs, but it could have positive externalities for the economy. Subsidizing the use of such services might still be justified in this case as it brings net benefits for the economy at large. This can, for instance, be the case if MGs are used by a firm to train workers who then leave to work in other companies, or the improvements and innovations introduced by firms as a result of their invest- ment in BDS are copied by other firms. Though the theoretic economic justification for the use of MGs may be convincing, there are serious potential pitfalls in their implementation. In cases where SMEs are not constrained in their capacity to purchase BDS and where such services do not have significant positive externalities, subsidizing these services can create distortions in resource allocation and constitute a public subsidy for a private gain. The most common criticism of MGs is the risk of limited additionality, meaning that MGs may benefit firms that would have paid for the services anyway. Finally, if MGs give SMEs only a temporary incen- tive to procure more BDS but do nothing to solve the inherent market failure that prevented them from using these services in the first place, the impact of the MG program on the BDS market would be questionable. 2 This study does not aim to reproduce at length the economic rationale and risks of using MGs to foster SME development. For a detailed discussion, see notably Biggs (1999), Phillips (2001), Van der Meer and Noordam (2004), and Campos et al. (2012). 12 2. Background and literature review 2.2. Literature review Significant work was undertaken in the late 1990s to study the justification and modalities of interven- tion in BDS markets. A dedicated working group of the Donor Committee on Enterprise Development organized several conferences on this topic between 1998 and 2000. This resulted in the publication of a set of basic guiding principles (e.g. root interventions in a serious assessment of the BDS market, careful use of subsidies to avoid distortions and ensure cost-effectiveness, and focus on sustainability and impact assessment) (DCED 2001).3 Turning specifically to MGs, a number of studies on the impact of these programs have been carried out since the early 2000s, most of which attempt to draw lessons learned and best practices. While earlier reports were mostly nonexperimental assessments based on samples of projects or single case studies, a few more recent papers present the results of impact evaluations based on randomized de- sign. This section briefly summarizes some of the most relevant studies to date.4 a. General reviews Phillips (2001) is one of the first studies providing a comprehensive discussion of MG schemes based on the experience in developing countries. The provision of reasonable, targeted, transparent, and temporary subsidies is argued to be justified if it solves market failures, generates positive externalities in a cost-effective manner, and durably strengthens the BDS market. The study draws mixed con- clusions from a review of 10 World Bank-funded projects; while outputs for beneficiaries are usually positive (e.g., sales and exports), doubts are raised regarding the additionality and sustainability of benefits in the absence of adequate indicators. The study also notes the generally excessive operating costs and slow implementation of these programs. It concludes by emphasizing the need to step up efforts to (a) establish a sound economic justification for each MG project grounded in evidence on local market conditions and (b) develop a cost-effective design, with a few basic principles laid out (e.g., governance and management arrangements and grant modalities). One design consideration discussed is the need to balance two objectives: on the one hand, to target the subprojects with the highest potential for spillovers/additionality (through specific selection criteria and rigorous appraisal of applications) and, on the other hand, to ensure streamlined procedures to disburse funds quickly and create momentum for BDS market development. More recently, Goldberg and Ortiz del Salto (2009) analyzed a sample of MG programs supported by the World Bank and other donors to identify lessons learned from successful and failed projects and to derive best practices for program design, based on program documents and interviews with task managers. The study also identifies the main risks for these types of projects (e.g., limited take-up by firms, disbursement delays, and fraud) and discusses impact evaluation issues. Finally, a couple of recent comprehensive reviews addressed MGs as one of several different tools to support SMEs. The World Bank Group’s IEG published a review of the World Bank’s experience with targeted support to SMEs over the period 2006–2012 (IEG 2014). It finds that MG projects have gener- ally achieved successful outcomes, but with frequently serious implementation difficulties. The report concludes that projects targeting SMEs are often weakly justified and/or have limited potential for additionality. Piza et al. (2016) reviewed the available literature on the impact of SME development interventions in low- and middle-income countries. They found evidence that MGs can improve firms’ performance and help create jobs, although the evidence reviewed is generally insufficient to ensure attributability and cost-effectiveness. 3 See also www.enterprise-development.org/page/bds 4 This literature review focuses on interventions targeting SME development. For a discussion of grant programs in agricultural and rural development projects, see Van der Meer and Noordam (2004). 13 1: Alternative and complementary instruments Case study 1 Alternative and complementary instruments Beyond MGs, several other related mechanisms, which are includes both an MG scheme and a subloan scheme outside the scope of this study, have been used to foster administered through partner financial institutions. private sector growth and productivity gains, including Subloans are provided at market rate and borrowers vouchers, sub-loans, business plan competitions and reim- are expected to cover at least 20 percent of subproj- bursable instruments: ect costs from their own resources. This instrument has had limited success to date, in a context of excess • Vouchers:5 Instead of grants paid to a beneficiary on liquidity in the banking system. a reimbursement basis, some projects have provid- ed vouchers for micro firms with limited cash flow • Business plan competitions: Since 2011, the Nigeria to procure basic services. For instance, the Kenya Growth & Employment Project (P103499, FY11) Micro and Small Enterprise Training and Tech- has supported the annual YouWiN! Competition. nology Project (P001353, FY1994–2003) featured The competition is aimed at aspiring entrepreneurial a voucher program for micro firms in the informal youth, who are provided with equity funding, training, sector, which required a small contribution by bene- and mentoring. Results from an impact evaluation ficiaries. This project had mixed results, as it allowed suggested that the grants have had lasting positive over 34,000 beneficiaries to receive training but was impact on beneficiaries, including higher business affected by significant implementation shortcomings survival, sales, profits, and employment (McKenzie (World Bank). The Guatemala Competitiveness 2015). Project (P055084, FY01–09) included both an MG program for small firms and a voucher scheme for • Reimbursable instruments: The Mauritius Manufac- micro firms to use BDS from prequalified vendors, turing and Services Development and Compet- who would redeem vouchers for cash after services itiveness Project (P112943, FY10–13) was not were provided. Vouchers would have had a face value implemented because of a change in government below market prices to ensure beneficiaries would priorities, but the project envisioned an MG scheme contribute part of the cost. This scheme was never with a ‘classic’ window (50 percent nonreimbursable implemented because of lack of commitment from grant) and a second window offering ‘payback grants’ the Government, and the MG scheme also had very of 90 percent, to be repaid over several years as roy- limited results. alty on additional sales. Likewise, the Innovation Serbia Project (P126229, FY12–16) featured an • Subloans: The Jamaica Private Investment and MG scheme to support R&D projects under which Export Development Project (P007485, FY1994– successful beneficiaries benefiting from higher sales, 2001) featured an interest-free subloans program thanks to the technology developed, would be re- to finance up to 80 percent of the cost of an export quired to repay up to 120 percent of the financing development plan, which was initially intended to be received as royalty on sales. an MG scheme. This design change made implemen- tation more cumbersome at first because of the need to find a way to guarantee the subloans. The Vietnam Inclusive Innovation Project (P121643, FY13) 5 See Phillips (2001) for further discussion and references on experi- ences with voucher schemes. The study argues that vouchers can yield positive results but that they can require costly management arrange- ments to provide successful coordination, guard against fraud, and en- sure quality service provision. 14 2. Background and literature review b. Nonexperimental case studies In-depth case studies of MG projects often utilize both qualitative and quantitative analyses. Several studies attempt a comparison with a counterfactual by building ex post a control group of similar non-beneficiary firms, although it is acknowledged that this does not address selection bias concerns. An early example is the evaluation of the Mauritius Technology Diffusion Scheme by Biggs (1999). The study finds that, although the program had a sound rationale relevant to local conditions, shortcom- ings in design and implementation prevented it from selecting the firms and projects with the highest potential for additionality and externalities, limiting the program’s overall economic benefits for the Mauritian economy. The study goes on to suggest possible ways to improve additionality and selectiv- ity of grant financing, as well as the durability of impact on the BDS market. Gourdon et al. (2011a, 2011b) use a similar approach to evaluate the impact of an export-focused MG program in Tunisia. They find that, compared to a control group, beneficiary firms had significantly faster growth of exports volume, destination markets, and employment (especially for new exporters), although this effect was only temporary. The study also emphasizes the value of planning an ex ante evaluation design in MG programs. c. Experimental impact evaluations The need for more evidence on the impact of MG programs has led some recent programs to incor- porate experimental designs randomly selecting beneficiaries among eligible firms. This has allowed the first rigorous impact evaluations of such programs to be conducted for projects in the Republic of Yemen and Mexico, which are respectively described in McKenzie et al. (2016) and in Bruhn et al. (2013). In both cases, the studies find some evidence of additional impact of the grants on different dimensions of performances (for example, innovation activities, productivity, and employment). On a related topic, Campos et al. (2012) describe various obstacles that prevented other projects from implementing randomized evaluations of MG programs and derive lessons learned for future project evaluations.6 The present study builds on this literature and aims to complement it by (a) covering virtually all World Bank projects with MG components implemented over the last two decades, (b) providing a compre- hensive discussion of the different options for program design features, and (c) correlating elements of design/implementation with project outcomes. 6 See case study 17 in section 6.2 for more details on these three studies. 15 Case study 16 3. Study methodology 3.1. Selection criteria The sample of projects reviewed in this study was compiled by using the World Bank classification system of sector and theme codes, which in the absence of a specific MG flag were the closest proxy for an initial screening of the World Bank portfolio. The authors used relevant sector and theme codes to filter all active and closed lending operations.7 The resulting few hundred projects were manually re- viewed for mention of MGs in the Project Appraisal Document (PAD), and the final list for review of 106 projects was generated in June 2015. Any changes to project status or the addition of new projects since then are not reflected in the database. Due to the imperfect nature of the screening, the study may have missed some projects. Information on the different MG schemes’ design features and implementation arrangements were collected from project documents, such as PADs and, where available, MG manuals. The outcomes of the selected MG projects were assessed using the rating provided in Implementation Completion and Results Reports (ICRs). Since ICRs do not always assign outcome ratings to specific components, the authors assigned an implied rating to each operation that was either closed or still active but close to fully disbursed. This implied rating was deduced from the text of ICRs and Implementation Status and Results Reports (ISRs) and is the main dependent variable of the analysis in the following sections. In both cases, these ratings are subjective and based on project-specific pre-identified indicators, and section 6 discusses in more detail how the projects in the sample define and measure the success of MG programs. 7 The initial filter included sector codes FZ - General Finance Sector, FH - SME Finance, BT - Public Administration, Industry and Trade, EV - Vocational Training, YA - Agro-Industry, Marketing, and Trade, and YZ - General Industry and Trade, and the theme codes 0041 - Micro, Small, and Medium Enterprise Support, 0044 - Other financial and private sector development, 0045 - Export development and competitiveness, 0048 - Technology diffusion, 0049 - Trade facilitation and market access, 0039 - Infrastructure for private sector development, 0066 - Education for the knowledge economy, 0099 - Other private sector development, and 0076 - Rural non-farm income generation. 17 3.2. Overall project distribution The 106 projects in the sample are very unevenly distributed between the six regions. Over half of all the MG projects in the sample are in Africa, followed by 21 percent in Latin America (figure 3.1). Figure 3.1 Regional breakdown of the sample of MG projects Africa Latin America & Caribbean Europe & Central Asia East Asia & Pacific South Asia Middle East & North Africa 0% 10% 20% 30% 40% 50% 60% Share of projects To put this regional distribution in perspective, we looked at only the subset of projects in the sample that were mapped to the former Finance and Private Sector Development (FPD) Network and com- pared the distribution of FPD MG projects to the overall distribution of FPD projects over the same period (figure 3.2). We note that the use of MG instruments is heavily biased toward Africa, a distribu- tion that does not reflect the general regional breakdown of the overall FPD portfolio. Figure 3.2 Regional breakdown of FPD projects All FPD projects FPD MG projects Africa Europe & Central Asia Latin America & Caribbean East Asia & Pacific South Asia Middle East & North Africa Other 0% 10% 20% 30% 40% 50% 60% Share of projects 18 3. Study Methodology Of the 106 projects in the sample, 73 projects have an assigned rating for the MG facility and the dis- tribution is shown in figure 3.3. Overall, 75 percent of the implemented MG facilities have received positive development outcome ratings, while 25 percent were rated Unsatisfactory, Moderately Unsat- isfactory, or were not implemented. Figure 3.3 Breakdown of the sample projects by MG component rating Not implemented Unsatisfactory Moderately unsatisfactory Moderately satisfactory Satisfactory Highly satisfactory 100% 80% 60% 40% 20% 0% 20% 40% 60% 80% 100% Share of projects The regional breakdown of rates of success is presented in figure 3.4. All projects in Middle East and North Africa have a positive rating, but with only four observations. East Asia and Pacific has the lowest success rate with 50 percent positive ratings, but with only six observations. Of the regions with more projects implemented, Latin America and the Caribbean has 80 percent of successful projects, fol- lowed by Africa with 73 percent. Figure 3.4 Success rates by region Not implemented Unsatisfactory Moderately unsatisfactory Moderately satisfactory Satisfactory Highly satisfactory Observations Middle East & North Africa 4 Europe & Central Asia 6 South Asia 5 Latin America & Caribbean 15 Average 73 Africa 37 East Asia & Pacific 6 100% 80% 60% 40% 20% 0% 20% 40% 60% 80% 100% Share of projects 19 Case study 20 4. Design features of MG schemes 4.1. Program scale and number of beneficiaries There is a wide variety in the scale of MG projects approved by the World Bank since the 1990s. The two largest amounts dedicated to MGs were for agricultural development projects in Mexico ($343 million, FY1999–2003) and Nigeria ($331 million, FY09). Without these outliers, the size of MG facili- ties averages $11.5 million, with 17 percent of projects above $20 million and 24 percent below $3 million.8 Even excluding the two very large projects mentioned, the average is significantly higher for agricultural projects ($25 million) than for private sector development projects ($7.9 million), which is explained by the frequent focus on rural infrastructure in the former category. Likewise, there is a large spread in the number of beneficiaries targeted by MG programs, from a few dozens to several thousands, with the average reaching around 450 and the median 240.9 Using pop- ulation and gross domestic product (GDP) size as proxies, there is only a partial correlation between targeted numbers of beneficiaries and the size of the private sector. At the extremes, one recent proj- ect in Vietnam (90 million inhabitants) expected to extend 38 MGs, while an earlier project for Cabo Verde (0.5 million inhabitants) was planning on reaching 200 beneficiaries. While countries with a large population and economy like Indonesia and Pakistan have benefited from some of the largest MG projects, these have remained disproportionately small in comparison with the projects in smaller countries. The maximum cumulative MG funding that a single beneficiary can receive varies from a few thousands of dollars to several hundreds of thousands,10 with an average of $112,000 and a medi- an of $76,000. When restricted to projects focusing on providing BDS or financing-related expenses to single SMEs, the average is around $80,000 and the median $60,000. 8 Based on funds to be used as MGs, excluding operating costs. The amounts used correspond to final amounts, taking into account additional financing or reduced envelopes decided at project restructuring in case of good or disappointing program performances. 9 To maintain comparability, this is computed for projects targeting individual firms or associations. This excludes agricultural projects, which tend to either support alliances or ’productive partnerships’ between actors in a small number of value chains or large numbers of smallholders with small MG amounts for each. 10 It is not always clear from project documents whether the maximum size of a single grant is also the cumulative maximum amount that can be received by a single beneficiary with several grants. 21 Case study 22 There can be a sound rationale for targeting a limited number of beneficiaries, even in large countries. This depends notably on the main objective of the program, types of projects to be funded, absorp- tion capacity of the local private sector, and so on. Some projects may purposely aim at extending a relatively small number of grants because of their focus on a specific geographic area/sector or on niche activities such as R&D (this is the case of the Vietnam project cited above). For instance, a recent project in Ethiopia aims at fostering linkages between firms in two industrial zones and the local pri- vate sector through the provision of MGs. These local conditions justified making firms with up to 500 employees eligible and setting the maximum size of MGs for a single firm at $400,000. In other con- texts, setting relatively low limits for the size of individual grants and the maximum amount that can be received by a single firm can be a way to favor smaller firms, which are likely to have less-expensive projects. There is rarely an explicit discussion of the critical size required for MG schemes to have the demon- stration effects and public good benefits that should justify this use of public resources, for example in projects’ economic analyses. In the absence of such demonstration, the risk is that MG schemes be added automatically and that their size and targeted number of beneficiaries be driven more by the remaining funds in the total project envelopes that have not been attributed to other components. 23 2: Choosing the appropriate implementing agency Case study 2 Choosing the appropriate implementing agency In the case of the Nicaragua – MSME Development In a fragile context, the Afghanistan New Market De- Project (P109691, FY08–15), a competitive selection velopment Project (P118053, FY11), approved in 2011, process to manage the MG facility was considered but has successfully piloted the use of MGs in the country. Giv- rejected to align with the Government’s strategy to en the lack of experience within the Ministry of Commerce strengthen ministries. Despite initial plans to implement and Industry, an international consulting firm was hired the program through a unit of the Ministry of Economy, competitively to undertake full management of the facility there were prolonged discussions within the Government under the Ministry of Commerce and Industry’s oversight. on the best-suited implementing agency after the project While the hiring of the firm took 15 months to complete, had already become effective. The MG program was even- the pace of MG approval and disbursement quickly in- tually transferred to another ministry and started with a creased since then and most of the $11.8 million available 20-month delay. Nevertheless, thanks to a one-year project in grants have been committed with encouraging impact extension, combined with the diligence of the implemen- on beneficiaries. The management contract amounted to tation team and World Bank support, this component ex- $5.5 million, including staff, operating costs, and M&E, and ceeded most of its targets. the project also set aside resources for business plan sup- port services ($0.8 million), communication and marketing The administration of the MG component under the Ethi- ($0.3 million), and core skills development workshops opia Private Sector Development Capacity Building ($0.2 million), setting total operating costs at 38 percent. Project (P050272, FY05–13) was delayed by almost two Despite the contracting of the facility’s management, the years due to inadequate capacity and flexibility of the des- requirement that all reimbursement claims made by ben- ignated implementing agency within the Ministry of In- eficiaries be processed through the Government’s slow dustry. The program only started to show progress after its public financial management system has been highlight- responsibility was transferred to a newly established semi- ed as a cause of significant reimbursement delays limiting autonomous administration fully staffed with competitive- the program’s impact. The authorities have nonetheless ly recruited employees (Ethiopia Competitiveness Facility), expressed their satisfaction with the program and are con- with the oversight of a steering committee headed by sidering a second phase to scale it up after 2016. the Government and involving key private stakeholders. This delay required an 18-month extension of the project, which eventually achieved satisfying results. 24 4. Design feature of MG schemes 4.2. Implementing agency The implementation of MG schemes is typically done by a PIU also responsible for other project com- ponents (often specially hired and not civil servants), by a dedicated private manager contracted by the PIU, by other public or private entities, such as a private sector association, or through a variety of other ad hoc arrangements (for example, public and public-private bodies in charge of enterprise de- velopment, export growth, or innovation). In the sample, private management is the dominant model (49 out of 106 projects), followed by PIUs (28 out of 106 projects) and private sector associations (11 out of 106 projects).11 Several arguments have been advanced to suggest that contracting a private operator to manage MG schemes would lead to better outcomes, the main ones being (a) a perceived higher efficiency of the private sector, especially where public sector capacity is weak and (b) independence in the selection of grantees (Goldberg et al. 2009). The first argument is not supported by the ratings of MG compo- nents in our sample: the rating is moderately satisfactory or higher for all but one scheme managed by a PIU (and another one was not implemented), while a quarter of contracted schemes were rated moderately unsatisfactory or lower (slightly above the sample average). Overall positive ratings for projects with private contractors were 72 percent, compared to a sample overall of 77 percent, while 88 percent of projects managed by PIUs received a positive rating. An independent selection process is certainly a legitimate consideration, though adequate governance and oversight arrangements of publicly managed MG schemes (for example, participation of the private sector in the selection com- mittee) and transparent communication can minimize the risk of political capture. Figure 4.1 Success rate by type of implementing agency Not implemented Unsatisfactory Moderately unsatisfactory Moderately satisfactory Satisfactory Highly satisfactory Observations12 Project Implementation Unit 16 Other 9 Association 9 Contractor 36 All 70 60% 40% 20% 0% 20% 40% 60% 80% 100% Share of projects 11 In a number of cases, the management of the MG scheme was planned to be contracted at the design stage, but was even- tually implemented by the PIU. In other cases, private management is limited to the hiring within the PIU of specialized consul- tants to manage the MG component. Both cases might bias upward the proportion of projects classified as privately managed. 12 As elsewhere in this section, when analyzing outcome distribution, the total number of observations will be presented for each feature, because, owing to missing data, not all features will have information available for all 73 projects with ratings. 25 Case study 26 4. Design feature of MG schemes On the other hand, private management has been criticized as inflating operating costs and failing to build local capacity in the case of contracting to foreign consultants. The data confirms that operating costs tend to be higher for projects involving a private manager (23.8 percent of available funds on average) than for projects managed by a PIU (19.8 percent).13 In one extreme case in Cambodia, the fee of the contracted international consulting firm ended up being twice as high as all disbursements made to beneficiaries of the MG scheme. However, simple percentages can be misleading in the ab- sence of a breakdown of operating costs. In particular, projects providing eligible firms with training and TA for business plan preparation may have justifiably higher operating costs (see case study 2 be- low). Moreover, in several cases where it was decided to contract management firms, factors outside their control (e.g., inadequate preparation and obstacles with the contract award) resulted in severely delayed start of activities. Overall, the choice between public, private, and alternative forms of management should depend on local circumstances and opportunities. Irrespective of the nature of the implementing agency, the focus should be on ensuring that it has the appropriate capacity (technical, human, and administra- tive), incentives, and governance arrangements (including oversight and audit) to efficiently manage an MG scheme or that these will be built before the start of the project. In several cases in the sample, insufficient capacity required a change of agency during implementation, leading to major delays. In cases where contracting is deemed desirable, a clear allocation of roles and responsibilities between the private operator and supporting public institutions is needed. Finally, even the best public or pri- vate agency will not be successful if tasked to implement a poorly designed and prepared MG scheme that is not adapted to local conditions. In this regard, involvement of the private sector in the design of the project is highly advisable, even if the public sector remains responsible for program management. It is often difficult to get a clear view of the level of operating costs, either because project documents do not mention 13 them or because initial figures underestimate actual implementation costs. This review has used only information available in the PADs and ICRs. 27 3: Large and micro firms in MG schemes Case study 3 Large and micro firms in MG schemes The Moldova Competitiveness Enhancement Project The Ecuador International Trade and Integration (P089124, FY06–13) featured a $1.2 million MG facility Project (P040106, FY1998–2003) featured an Entre- designed to promote the use of metrology, standardiza- preneurial Learning and Innovation Fund for SMEs and a tion, testing, and quality services by the private sector, Grassroots Growth Fund pilot program for micro business with the objective to strengthen its competitiveness with less than 12 employees, both of which provided both in export and domestic markets. The facility’s scope MGs. The project financed 30 percent of eligible expenses was later expanded to cover a broader array of BDS with a under the Entrepreneurial Learning and Innovation Fund $1.5 million additional financing. While focused on SMEs, but 50 percent under the Grassroots Growth Fund. The the program’s eligibility criteria did not include any size other half was expected to be self-financed by micro busi- restriction and targeted firms with a strong track record nesses or to be supported by the local nongovernmental of successful business in foreign and/or local markets. organizations providing the services. The program reached The matching proportion (50 percent) was the same for 3,900 micro businesses through 106 projects, significantly all beneficiaries and the maximum grant size was set rel- exceeding initial targets. About 40 percent of beneficiaries atively low at $10,000. The program exceeded its original were women and 56 percent of all beneficiaries were in ru- targets and provided grants to 479 firms. While most were ral areas across 21 provinces. An evaluation reported good SMEs, large firms accounted for about 25 percent of the results in terms of increased sales/exports and improved total with a larger share for the original quality certification business skills, although outcomes varied depending on window. The scheme has been credited for its positive the quality of the sponsor nongovernmental organization. impact in terms of fostering an increased use of metrology, While some organizations performed well, others were standardization, testing, and quality services, although it is overstretched by commitments to too many projects and unclear from project documents to what extent this ap- failed to fully address individual client needs. This points plied to both SMEs and large firms. to the challenge of implementing a program with a large number of (very) small beneficiaries. 28 4. Design feature of MG schemes 4.3. Beneficiary eligibility criteria a. Size of beneficiaries Generally, MG programs target SMEs, as this category of firms is thought to be less likely to appropri- ately value BDS and to have less capacity to pay for them than large firms. About 47 percent of proj- ects in the sample explicitly restrict eligibility to SMEs,14 either through a strict size restriction, by giving priority to SMEs, or by including separate windows for SMEs and other types of firms (e.g., exporters of any size). A number of projects do not mention any size restrictions and a few explicitly allow large firms to apply for MG funding. The eligibility threshold for SMEs varies largely, and depends on country or ad hoc definitions, and often involves a ceiling with regard to the number of employees (usually between 50 and 250) and/or other criteria (e.g., maximum annual turnover). A minority of projects explicitly target micro or one-person businesses, sometimes with dedicated funding instruments such as matching vouchers. These firms are likely to have lower absorptive capacity, making it difficult to compare the impact of MG targeting them with interventions focusing on SMEs. On the contrary, some projects include a minimum number of employees to exclude such micro firms. In some cases, employment or turnover ceilings were modified during implementation to increase the reach of the program. Here again, the optimal eligibility criteria for a specific project depend on its main objective and local environment (characteristics of the private sector, administrative capacity). While targeting SMEs as opposed to large firms is arguably a more justifiable use of public funds through MGs, the level and breadth of eligibility criteria must be carefully set (and revised during implementation, if necessary), as they influence the program’s results with regard to implementation and impact. Generally speaking, making larger firms eligible would require explaining why subsidized grants are needed to change their practices and incentives. As shown in figure 4.2, limitation by size had no bearing on the success rate of projects, with 77 percent of projects with no explicit size limitations receiving positive ratings, compared to 76 percent of projects limiting size and 77 percent of all projects in this sample. Figure 4.2 Success rate by limitation of size Not implemented Unsatisfactory Moderately unsatisfactory Moderately satisfactory Satisfactory Highly satisfactory Observations Yes 34 No 30 All 64 100% 80% 60% 40% 20% 0% 20% 40% 60% 80% 100% Share of projects It is likely that other projects without such explicit restriction in their PAD introduced it in the MG manual and during imple- 14 mentation. 29 4: MG and groups of firms Case study 4 MG and groups of firms Supported jointly with the U.K. Department for Inter- The Haiti Business Development and Investment national Development, the Fund for New Market De- Project (P123974, FY13) was launched with a compo- velopment for the West Bank and Gaza (P110526, nent designed to provide MGs to both single micro, small, FY08–11) encouraged groups of firms with common and medium enterprises (MSMEs) and groups of firms. A new market/product development plans to apply for MGs restructuring of the project in December 2015 doubled through higher percentage of subsidy and grant ceiling the envelope allocated to MGs from $4 million to $8 mil- (70 percent and $100,000, compared to 50 percent and lion and shifted the focus exclusively to the provision of $50,000 for single firms). The rationale provided was nota- grants to groups of MSMEs in the same value chain to bly that doing so would foster cooperation between firms, access new services and increase the quality of their prod- create synergies, and stimulate learning. The project was ucts. The matching contribution by each MSME will vary planning to encourage firms submitting similar individual based on a weighed score of employment and turnover. applications to develop joint proposals and to engage The restructuring also aimed at reducing the number of directly with business representative associations to iden- MGs and increasing their value, to increase their impact tify potential consortia. Prospective areas for cooperation and reduce transaction costs. Two windows were intro- included the development of shared marketing material, duced: (a) MG for Business Improvement to increase value acquisition of market information, training on certification chain productivity and competitiveness (up to $100,000 or export development, and so on. The consortia were per value chain, with a minimum involvement of four required to (a) be made up of at least three firms, each of MSMEs) and (b) MG for Common Services to develop lo- which would benefit equally from the project and contrib- gistical and other services that require economies of scale ute the same amount; (b) be formalized through a Mem- ($500,000 to $1.5 million, with a minimum involvement of orandum of Understanding signed by all members; and 60 MSMEs). This new approach is expected to lead to the (c) appoint a lead firm responsible for handling communi- development of at least 20 mostly agricultural value chains cation and financial transactions with the Facility for New and to be more inclusive by covering the country’s poor- Market Development and for redistributing their share of est regions. the MG to each partner firm. At closing, the project had supported 36 consortia and 567 single firms. The project’s evaluation noted that this modest success stemmed from the difficulty to identify mutually beneficial projects be- yond simple activities, such as joint participation in trade fairs. It concluded that this approach was worthwhile but that (a) expectations in terms of number of joint ventures should be kept low and (b) technical support to interested firms to plan such ventures might matter more than addi- tional financial incentives. 30 4. Design feature of MG schemes b. Groups of beneficiaries A large majority of MG projects allow groups of firms and/or professional associations to apply for a common project or to benefit jointly from BDS. Around 72 percent of projects in the sample al- low group beneficiaries, 9 percent do not, and the rest do not explicitly specify whether groups are eligible. Virtually all of the agriculture projects in the sample allow groups (for example, productive partnerships between a lead agribusiness and producers groups), while private sector development projects more frequently restrict eligibility to single firms. Projects sometimes include separate win- dows for single firms and groups, with varying grant modalities (e.g. higher/lower maximum grant size or matching percentage). Several reasons can be advanced to justify the eligibility of groups of firms or associations to MG programs, such as fostering linkages and cooperation between firms, improved access to services for firms with limited financial resources, reduced program administration cost, etc. Programs accepting applications from groups of firms can have requirements regarding the nature of the partnership (e.g., formalization through a Memorandum of Understanding), designation of a lead firm representing the consortia, and the contribution of individual members. A matching contribution from each firm is generally required to ensure their commitment to the success of projects, which can- not solely be financed through association funds. The eligibility of groups of firms has consequences on the administration of MG schemes and the nature of the grant agreements signed with beneficia- ries, although this is rarely adequately addressed in program documents. Fifty-six projects were rated for outcome and had information available on whether groups of firms are allowed or not. Though the sample size is quite small, 88 percent of projects that do not allow groups of firms had a positive rating versus 75 percent of projects that do. Figure 4.3 Success rate by allowing groups Not implemented Unsatisfactory Moderately unsatisfactory Moderately satisfactory Satisfactory Highly satisfactory Yes 48 No 8 All 56 100% 80% 60% 40% 20% 0% 20% 40% 60% 80% 100% Share of projects 31 5: Provision of MGs to BDS providers Case study 5 Provision of MGs to BDS providers The Mozambique Enterprise Development Project Under the Nigeria Micro, Small and Medium Enter- (P049874, FY00–06) featured a Technical Learning in prise Project (P083082, FY04–12), it was decided to Firms component focused on building the technical capa- use MGs exclusively in a ‘BDS Fund’ aiming at strength- bilities of local firms. In addition to a classic demand-side ening BDS providers’ capacity to provide services in the MG scheme, this component included a $2 million MG quantity, price, and quality range required by MSMEs, as window targeting providers to increase the private supply well as to develop new BDS products in targeted industry of business and technical training. This was expected to value chains and states. The project supported 69 BDS focus on (a) existing local or overseas-developed training providers who provided training to around 3,360 MSMEs, courses for which a local market has not yet developed exceeding the original target of 20 and 1,000, respectively. and (b) additional training courses for which an unmet Cost recovery by supported BDS providers was relatively market demand can be demonstrated. The second win- low at 50 percent, as they reported difficulties in getting dow for providers was seen as needed to reach the small- MSMEs to pay for their services, because of lack of resourc- est firms which would be difficult to support on a one-to- es as well as perceptions that World Bank-funded govern- one basis and for whom generic training in basic business ment programs should not require payments. skills can yield substantial returns. The initial targets of 200 training events and $2 million in grants were largely The Mali Growth Support Project (P080935, FY05– exceeded, with 3,160 training events and $3.69 million in 13) was initially planning to fund a demand-side MG pro- grants disbursed. As an example of activities supported, gram, which was never implemented. However, the proj- 280 small consultancies and individuals outside Maputo ect supported some non-MG activities to strengthen the were trained as Microsoft-certified engineers. The project’s supply of BDS, including a STEP program aimed at training evaluation nonetheless noted that issues were faced re- young graduates to become BDS consultants for micro garding the quality/relevance of some training and that businesses. Around 1,300 such businesses received basic evaluating the final impact of the program was made dif- training under the project (for example, bookkeeping and ficult by the absence of monitoring data on the status of marketing), which they partly had to pay for. trainees following the courses. 32 4. Design feature of MG schemes c. Service providers While most MG programs attempt to develop BDS markets and increase the use of such services by subsidizing potential users, some projects also aim to address the supply side by making service pro- viders eligible for MGs. This is intended to subsidize the cost of providing BDS services and to improve the diversity, quality, and marketing of these services. A few projects in Africa have also intended to provide MGs to financial institutions to incentivize them to develop their capacity to lend to SMEs, but these components have met limited interest and were dropped. Just like for service users, using MGs for BDS providers could be justified, provided a clear market failure leading to undersupply or inade- quate supply can be identified, there is confirmed interest/demand from those firms, selection criteria ensure additionality of the resources to be provided, etc. As an alternative, other projects have opted not to make service providers eligible for subsidies, but instead to strengthen their capacity directly through training. In both cases, the objective is to increase the quality and diversity of BDS available to SMEs, as opposed to recycling existing services. In the sample, about 26 percent of projects make service providers eligible, 35 percent do not, and 39 percent do not make this clear in available project documents. Despite the perception that support to BDS providers was not featured in the first wave of MG projects in the 1990s and has only appeared later on (Goldberg and Ortiz del Salto 2009), there is no clear trend on this aspect over time and a large proportion of recent projects either rejects MGs to BDS providers or does not explicitly include them in the scope of the program. Depending on the project, BDS providers can be eligible just like any other private business or through a dedicated funding window, possibly with different matching percent- age, maximum grant size, etc. Of the 65 projects that have explicit information available on whether service providers are eligible for the MG or not, 45 projects also have an outcome rating. Surprisingly, 88 percent of projects that do not allow MGs to service providers have successful outcomes versus 71 percent of projects that do. This may have more to do with the complexity of implementation arrangements for multiple windows and varied eligibility criteria, or alternatively, the projects that do not allow service providers may have more accurately diagnosed the market failure to be on the demand side. Figure 4.4 Success rate by eligibility of provider Not implemented Unsatisfactory Moderately unsatisfactory Moderately satisfactory Satisfactory Highly satisfactory Observations Yes 21 No 24 All 45 100% 80% 60% 40% 20% 0% 20% 40% 60% 80% 100% Share of projects 33 Case study 34 4. Design feature of MG schemes d. Other eligibility criteria A number of other eligibility criteria are defined in most MG projects, with the purpose to reach the desired type of beneficiaries that will maximize impact. Some projects give higher weight to appli- cations submitted by certain types of firms (e.g. women-owned businesses). Experience has shown that there can be a trade-off between the level of requirements and the reach of the program, which should be carefully assessed during project preparation and rebalanced if needed during implementa- tion. The most frequent eligibility criteria include the following: • Firm age: Eligibility is generally restricted to firms with a minimum time in operation (for example, two years) to the exclusion of start-ups. In cases where project objectives justify making start-ups or very young firms eligible, other requirements can usefully be set to ensure the good use of funds, such as solid business experience of the owner and/or the capacity to prove the medi- um-term commercial and financial viability of the venture. • Registration: Documenting company status is usually a requirement, although depending on the context, this can range from simple registration on some public registry of companies to fully formal status, evidence of compliance with tax obligations, and so on. While such requirements can facilitate tracking of beneficiaries and can constitute an incentive to formalize, they should be adapted to local circumstances and not be so stringent as to disqualify a majority of SMEs. • Ownership: Some projects specify a minimum share of domestic ownership (usually 50 percent) and/or domestic registration. Moreover, eligibility is generally restricted to private or majority-pri- vate companies. • Other: (a) Existence of a sound financial management system, accounting documents, and a sta- ble financial situation; (b) absence of previous public funding for the same activities; (c) conditions on association structure for programs accepting applications by groups of firms or consortia (for example, minimum number of members, governance structure, and presence of both agribusi- ness and smallholders for agricultural projects); (d) location in specific geographical areas covered by the project; and (e) proof of land rights for agricultural projects. 35 6: MG schemes in agriculture Case study 6 MG schemes in agriculture MG funds in agriculture or agribusiness projects tend to The Zambia Agricultural Development Support Pro- address different market failures and objectives compared gram (P070063, FY06–14) aims to advance smallholder to traditional private sector development funds. The two agriculture commercialization along value chains. The MG project examples below highlight some of these differenc- facility supports this objective by offering financial resourc- es, such as the use of the grant to finance equipment or to es for the development of innovative business linkages encourage the development of productive alliances. between smallholders and other actors along the value chains. The grant provided ‘seed’ capital for promoting the The objective of the Mali Agricultural Competitiveness introduction of new products and processes that improve and Diversification Project (P081704, FY06–15) is to the competitiveness of agricultural production by small- improve the competitiveness of supply chains in tradition- holder farmers. The grant considered three main catego- al and nontraditional agricultural crops through targeted ries of windows with a varying degree of match: extension investments to remove critical constraints, improve pro- and technology development (50 percent match); studies ductivity and efficiency, and improve and build organiza- and pilots (60 percent match); and capacity building for tional and institutional capacities along the supply chains. associations, cooperatives, and farmer groups (75 percent The MG fund supports the dissemination of adapted new match). The facility was demand-driven and proposals did technologies for small farmers. The fund is targeted at not have a commodity or geographical focus. It combined smallholders, under 0.5 ha, and covers 70 percent of the innovation with a focus on partnerships with smallholders. cost of infrastructure works linked to the introduction of Eligible applicants would include (a) organized producer new and improved technologies for irrigation or post-har- associations and related smallholder organizations with vest techniques. The project justifies the use of an MG developed or developing linkages to agricultural value on the grounds that even though investments could be chains and supporting the development of smallholder profitable, they typically exceed the self-financing capacity farmers; (b) agribusinesses linked with smallholders, in- of small farmers and cannot be financed on a credit basis volved in processing, trading, and marketing of agricultural due to the long amortization period. The project financed and food products/commodities that help to improve the 125 investment subprojects and microenterprises to be- growth and competitiveness of smallholder farmers; (c) in- come profitable SMEs. It is unclear from existing project put suppliers; (d) nucleus commercial farms working with documentation how the impact and sustainability of the smallholders; and (e) publicly and privately funded agri- MG fund will be assessed; the ICR for the project is forth- cultural and industrial institutions and trusts. At closing, 29 coming. subprojects were implemented and 30 innovative tech- nologies were introduced, either directly to smallholder farmers at the production stage or further down the value chain as part of the processing chain of agricultural com- modities. The ICR provides some evidence of impact with regard to MGs facilitating access to new export markets by a couple of the recipients, but apart from that, there is limited information on the overall adoption of the new technologies or their sustainability and impact. 36 4. Design feature of MG schemes 4.4. Sector focus a. Sector restrictions Of the 106 projects under review, 22 are agribusiness projects. Agribusiness MG facilities are structured differently from MGs in private sector development projects. Case study 6 provides more detail on the typical design features of agribusiness MGs using two project examples. The rest of the section dis- cusses sectoral focus among only the remaining 84 projects that are not agribusiness projects. Of the 84 remaining projects, 47 list at least one sector of focus for the MG facility. Of those that do, very few actually restrict the fund to certain sectors, and when they do, those tend to be sectors or value chains that have been pre-identified by the Government, World Bank, or other donors as having the highest potential for growth, job creation, or exports, based on the overall objective of the oper- ation. Two examples of projects that restrict the fund to pre-identified sectors include the Argentina Unleashing Productive Innovation Project (P106752), with a focus on biotechnology, nanotechnology, and information and communication technology, and Republic of Congo Support to Economic Diver- sification Project (P118561) with a focus on six value chains: agribusiness, transport and logistics, wood transformation, construction material, tourism and hospitality, and arts and crafts. The remaining proj- ects that list specific sectors do not explicitly restrict the fund to them, but rather target these sectors while still remaining open for applications from others. A number of projects present a negative list of activities, which is the standard exclusion list of activities that the World Bank does not finance, such as alcohol, weapons, tobacco, and gambling. A few projects that focus on exports reference World Trade Organization (WTO) requirements as a rea- son not to restrict sectors for the MG fund. To satisfy the requirements of the framework agreement of the WTO, grants are defined as nonactionable subsidies because they are not specific. Specificity un- der WTO rules typically involves “targeting of geographic regions or economic sectors and paragraph 2 of Article 8, principally the exception for research activities carried out on a contract basis for industrial research and for precompetitive development activity.”15 In at least one case, the World Bank discontin- ued an MG fund on the belief that the fund violates WTO rules, which had come into existence since the project was approved.16 Figure 4.5 gives a breakdown of the frequency with which specific sectors are listed in the program documents of the 47 non-agribusiness projects that list at least one sector of focus. The frequency of focus appears to be evenly split between Agriculture, Forestry and Fishing, Manufacturing, and Ser- vices. Figure 4.5 Frequency of sectors Standard Industrial Classification (siccode.com/en/siccode/list/directory) Services 48 Manufacturing 43 Agriculture, Forestry and Fishing 43 Retail Trade 3 Mining 3 Construction 3 Wholesale Trade 1 Transportation, Communication, and others 1 Cambodia Trade Facilitation and Competitiveness (P089196), PAD, page 35. 15 Trinidad and Tobago Business Expansion and Industrial Restructuring Project (P008100), Borrower’s comments to ICR. 16 37 Case study 38 4. Design feature of MG schemes Of the 84 projects that are not agribusiness MGs, 57 have both outcome ratings and information on whether at least one sector was targeted. Whether specific sectors were targeted seems to make no difference to the margin between positive and negative outcome, with 70 percent of projects receiv- ing positive ratings regardless of whether or not they target sectors. There does seem to be a differ- ence in the edges of the distribution, with 15 percent of projects that do not target a sector being highly satisfactory versus 3 percent of those that do, while on the other end, 20 percent of projects targeting sectors are unsatisfactory versus 7 percent of those that do not. Figure 4.6 Success rate by sector targeting Not implemented Unsatisfactory Moderately unsatisfactory Moderately satisfactory Satisfactory Highly satisfactory Observations Yes 30 No 27 All 57 100% 80% 60% 40% 20% 0% 20% 40% 60% 80% 100% Share of projects 39 7: Use of MG to foster exports Case study 7 Use of MG to foster exports17 The Cambodia Trade Facilitation and Competitive- The project disbursed grants to 65 enterprises and asso- ness Project (P089196, FY05–13) aims to promote eco- ciations, 13 of which were new exporters. The ICR esti- nomic growth by reducing transaction costs associated mates that on average firms benefiting from the grants with trade and investment, introducing transparency in generated $275.5 for every $1 spent. The ICR judged the investment processes, and facilitating access of enterpris- fund to have made satisfactory progress in helping firms es to export markets. The MG facility funds 50 percent of access new markets, though it noted that the return on the cost to enterprises for achieving market standards the grants was much higher in some sectors (such as rice) or evidence of compliance with those standards. Those than others and suggested that a focus on a single sector standards include product quality standards, core labor may exhibit larger returns. The evaluation also points out standards, organic product standards, and safety or en- that the management fee for the consulting firm im- vironmental standards. The fund will include TA to help plementing the scheme was nearly double the amount enterprises prepare an export plan and an MG for eligible disbursed to beneficiaries. The consulting firm evaluated activities undertaken within the framework of such a plan, proposals, provided TA, and helped prepare export plans. such as acquisition of information (including software and As much as this support had a successful outcome, the ICR reference material), trips by entrepreneurs to explore spe- points out that given the high upfront costs of evaluating, cific export opportunities and buyers, mailing of samples, a lesson for future operations is to involve government promotion material, and other necessary miscellaneous counterparts in the process to increase capacity and sus- expenses. tainability and reduce management costs, as well as to allow a longer time frame for implementation to allow the fund to support less-developed companies for which the decision to export would be strategic. For more examples of export-oriented MGs, see also case studies 13 and 14 presenting the series of three Tunisia Export Development projects. 17 40 4. Design feature of MG schemes b. Focus on exports Of the 106 projects in the sample, 17 are export-focused and restrict the use of the MG fund to export- ers, potential exporters, or suppliers to exporters. One of those projects restricts access to exporters to a single window of the MG, while the others are entirely export only. In addition to the 17, three other projects heavily focus on exporters or on developing export markets but do not restrict access to the fund. In the sample, 67 projects have outcome ratings and information on whether beneficiaries are limited to exporters. There does not appear to be any difference in the margin of positive ratings with 77 percent of projects restricted to exporters having positive ratings compared to 74 percent of proj- ects that are not restricted to exporters, suggesting that as long as the eligibility of activities is cus- tomized to serve the development objective, there is nothing about exports that is intrinsically more suitable to support through MGs. Figure 4.7 Success rate by exporter targeting Not implemented Unsatisfactory Moderately unsatisfactory Moderately satisfactory Satisfactory Highly satisfactory Observations Yes 13 No 54 All 67 100% 80% 60% 40% 20% 0% 20% 40% 60% 80% 100% Share of projects 41 8: Use of MG to purchase equipment Case study 8 Use of MG to purchase equipment The Sri Lanka Sustainable Tourism Development The project took two and a half years from the start of Project (P113709, FY10-14) aimed to strengthen the preparation to effectiveness in 2011, at a time of rapidly institutional framework for the tourism sector to facilitate changing environment following the end of the civil war environmentally and socially sound investments. The MG in 2009. The government asked to restructure the project facility aimed to contribute to the Project Development and enhance its effectiveness by reallocating most of the Objective by improving and extending the product con- funds from the other two components to the MG. In spite tent and supply chains of SMEs and reimburse 50 percent of the restructuring, the fund did not disburse and the of general business support toward certification, training, project was ultimately canceled in 2014. The ICR highlights promotion, and so on. the design of the MG fund as the main design flaw leading to the cancellation of the operation, and more specifically the provision for 50 percent match for refurbishment and purchase of equipment. This led to a significant oversub- scription with requests totaling more than three times the amount of available funds, but also posed serious gover- nance risks as equipment could be resold for a significant profit. The government also faced accusations of favoritism from the firms that did not receive funding as the grants were awarded on a first-come, first-served basis with fairly wide eligibility criteria. 42 4. Design feature of MG schemes 4.5. Eligible expenses As defined in the scope of this study, the MG instrument under investigation is a short-term subsidy for nonfinancial services and as such does not include funding for working capital or regular operating expenses, and only one project in the sample allows funding for small works for a coffee washing and drying facility.18 Whether funding for equipment should be allowed has been the subject of much debate for private sector development-type MGs. As noted in section 4.4, MG facilities in agribusiness projects on the other hand almost always allow the financing of equipment due to the different na- ture of market failures and the different objectives of those operations. For this reason, we have ex- cluded the 22 agribusiness projects from the discussion of eligible expenses below. Of the remaining 84 projects, only 22 allow the financing of equipment, and in most cases, the eligible types of equipment are very specific and limited. Innovation or science and technology projects often allow the funding of specialized R&D equipment (for purchase or leasing), raw materials, or incremen- tal permanent working capital, but still disallow land, civil works, housing, or operating expenses. In other cases, the MG fund only allows testing and quality control equipment. The MG instrument en- tails a subsidy with public funds and as such is best suited for the financing of public goods, or private goods with positive spillovers or externalities, or for the removal of market failures. Innovation projects often meet this criterion given their potential positive spillovers as well as the uncertain returns on investment that make credit financing difficult. Most other types of equipment, however, fall under the category of marketable private goods and as such should be financed on a commercial basis. The argument can be made that, in fragile or low-income countries where credit is almost nonexistent for firms, MGs can be primarily used as a way to overcome credit constraints in the absence of a mecha- nism to fix the financial sector in the short term. This could in turn justify making capital expenditures eligible for financing through grants. However, this involves specific risks (for example, resale of a sub- sidized movable asset at market price) and raises questions regarding the sustainability of any impact after the end of the project, which would have to be addressed at the concept stage. Case study 8 highlights some of the potential pitfalls of unjustified funding of equipment. The 50 projects that have information on whether equipment is allowed also have outcome ratings. Of those, 55 percent allowing equipment are rated moderately satisfactory and above compared to 74 percent of projects that do not. Figure 4.8 Success rate by equipment eligible Not implemented Unsatisfactory Moderately unsatisfactory Moderately satisfactory Satisfactory Highly satisfactory Observations Yes 11 No 39 All 50 100% 80% 60% 40% 20% 0% 20% 40% 60% 80% 100% Share of projects Haiti Business Development and Investment Project (P123974). 18 43 Case study 44 4. Design feature of MG schemes With regard to services, most MGs allow a wide array of BDS, including technical and management consultancies, design and implementation of marketing campaigns or export plans (for projects di- rected at exporters), market research, quality control and certification, business planning, productivity enhancement, production processes and product design, and travel and trade fairs. Although one MG program can support different types of projects, a distinction can be made between grants (a) for feasibility and design (for example, technical study and business plan) and for implementation of standards, software, machinery, and so on and (b) for acquisition of an intangible asset (for example, International Organization for Standardization certification) and for the building of new capabilities through training. Most projects in the sample provide a list of eligible activities to be funded but rarely restrict the specific type of service as long as it is relevant to the firm’s business plan. One exception is a project19 in the Republic of Congo that excluded TA with direct implications for investments, such as feasibility studies and engineering designs. This, however, was changed during the restructuring and small equipment for select value chains was also allowed. Republic of Congo Support to Economic Diversification Project (P118561). 19 45 9: Competitive allocation of MG Case study 9 Competitive allocation of MG The Nicaragua Micro, Small and Medium Enterprise The project reached 864 firms, which constituted 9.3 per- Development Project (P109691, FY08–15) aimed to cent of all firms with the minimum level of registration improve the competitiveness of MSMEs and the business required by the project. Although no impact evaluation climate that affects them. The MG component assisted was conducted, the ICR compared the recipients with enterprises with pre-investment activities, such as busi- trends among nonparticipants and rated the fund satisfac- ness plans and feasibility studies, which would allow them tory based on the level of sales, the introduction of new to apply for commercial credit. Rather than substituting products and processes, and the implementation of quali- commercial credit, the value added of the fund is to create ty enhancements. a large number of bankable SMEs. In addition, the fund finances investment activities and complements commer- cial credit by focusing on themes where bank financing is not viable. The grants were awarded competitively through a series of rounds with different technical focus (such as quality enhancements and certification, innovation, labor training, clean technologies, or resolving value chain bottlenecks for a specific sector). One benefit of the competitive selec- tion mechanism was the regular and predictable nature of the rounds, which allowed firms to plan ahead their participation in subsequent rounds. The evaluation com- mittee orders the proposals in each round based on their estimated impact on the national economy, and grants are awarded down the ordered list until funds for the particu- lar round are exhausted. 46 4. Design feature of MG schemes 4.6. Selection mechanism Out of the 106 projects in the sample, 85 have explicitly specified the type of selection mechanism based on which grants were awarded to beneficiaries, and the vast majority of them, 80 projects, use a first-come, first-served selection mechanism that awards grants in order of application to any firm that passes the pre-identified eligibility criteria. Only five projects have a true competitive selection mechanism that assigns scores to each proposal and gives grants to the top scoring firms down the list until the money allocated for that round runs out. While selecting the ‘best’ projects is intuitively appealing, the capacity of the implementing agency to do so can be questioned, especially if projects are evaluated on the basis of their additionality and spillover potential, as we argue should be the case, rather than simply growth potential. On the other end, a competitive selection mechanism could generate more interest if the grants are perceived to be limited and their awarding time-bound. Case study 9 looks at an example of a project with competitive selection. Due to the small number of proj- ects choosing to select grantees with a competitive process, it is hard to assess whether the choice of selection mechanism actually brings any systematic benefits. The type of selection mechanism could also have some implications for the design of impact evalua- tions. A competitive selection seems more amenable to techniques such as propensity-score match- ing or regression discontinuity design using a score threshold while both competitive and first-come, first-served could be used for randomized control trials based on oversubscription design. 47 10: Multiple MG windows Case study 10 Multiple MG windows The recently closed The Gambia Growth & Competi- The Honduras Enhancing Competitiveness: Trade tiveness Project (P114240, FY11–16) initially planned Facilitation and Productivity Improvement Project an MG Facility with four different windows with different (P070038, FY04–11) intended to use MGs for multiple objectives and funding levels: (a) horticulture out-grower activities in several components and subcomponents, scheme, (b) quality assurance program for the groundnuts each with different objectives and rules. MGs were notably sector, (c) use of BDS by MSMEs, and (d) support to banks to be used to (a) incentivize MSMEs to purchase services and financial institutions to lend to MSMEs. The second provided by new centers for technological innovation and fourth windows were dropped at project restructuring to be established for the textile, wood, and artisan crafts due to low interest, unfocused delivery, and insufficient industries; (b) help selected government agencies imple- awareness creation. The MG component was refocused ment administrative reforms; (c) improve the quality of on out-grower schemes in selected value chains (mango, private and public laboratories and encourage firms to vegetables, and poultry), as well as BDS support to MSMEs. adopt quality systems and obtain quality certification; and (d) encourage firms to train their workforce. These different The Pakistan Economic Revitalization of KP and FATA MG schemes had limited results and several changes were Project (P124268, FY12 features three MG windows made at restructurings, including to reduce MG funds. accounting, respectively, for 70, 20, and 10 percent of Under the first activity, only the center for crafts was es- total funding: (a) rehabilitation to provide direct support tablished and $21,000 were spent on MGs for MSMEs to to crisis-affected SMEs in restarting their operations, (b) use its services (the center was unsuccessful in attracting upgrade support to SMEs or groups of SMEs (not only larger companies that would have paid the full price). MGs crisis-affected), and (c) capacity building and BDS. The were found to be an inadequate tool to foster administra- third window includes provision of training under the In- tive reforms by government agencies and were discontin- ternational Finance Corporation’s Business Edge program ued. Finally, the improvement of private and public lab- and training of local Business Edge trainers. This design oratories turned out to require investment in goods and allowed the project to support both traditional MG activi- works, which were not adequately addressed by the MGs. ties (second and third windows) and to restart companies The project evaluation notes that insufficient justification in crisis-affected areas through the more flexible first win- was provided for the need to subsidize firms to introduce dow (for example, possibility of advance payment to ben- quality certification and train workers. eficiaries, higher subsidy level and maximum grant size for individual firms, capital expenditures eligible, and in-kind contributions allowed). As of November 2015, the project had supported over 1,000 SMEs under this window, signifi- cantly above the initial target. 48 4. Design feature of MG schemes 4.7. Grant modalities a. Level of match The level of subsidization of beneficiaries’ projects is a central element of MG programs, which involves trade-offs between several dimensions (e.g. attractiveness of the scheme, commitment of the bene- ficiary, additionality of resources, and risks of abuse). While the grant level should not be set too low for the program to have a strong catalytic effect, other key design characteristics, such as simple appli- cation procedures and streamlined reimbursements, can matter equally or more for its attractiveness. Generally speaking, it can be argued that a high subsidy proportion should be justified by a high ad- ditionality and/or spillover potential. However, an evaluation of the Kenya Voucher Program by Phillips and Steel (2003) offers a model of long-term market impact at different levels of subsidy and argues that the higher the subsidy the lower the likely long-term impact as a proportion of subsidy cost as it may lead to ‘adverse selection’ of initial participants who are unlikely to be able to purchase the service at full cost. Even though the level-setting is hardly scientific, it should still be based on an analysis of the specific conditions and objectives, but few projects explain the economic rationale for the level of matching adopted. Two-thirds of the projects in the sample have a fixed cost-sharing proportion, generally set at 50 per- cent although some projects provide a match from 60 up to 95 percent. In a few cases, this percent- age was modified at project restructuring to increase take-up. The rest of the projects includes variable rates and/or separate funding windows for different types of beneficiaries (e.g. micro vs. medium firms and firms vs. associations), project amounts, types of activities, and firm location. Some projects also allow multiple MGs per firm but reduce the percentage of subsidy for each successive grant. In some cases, the project design was very complex, with four to five different funding windows or separate MG schemes in different project components, each with their own rules, criteria, implementation ar- rangements, etc. Targeting different types of projects through distinct funding windows can increase a program’s capacity to reach the intended beneficiaries, but this should be balanced with the cost and ease of administration. There is some evidence in the sample that administration costs may be moderately higher for projects with variable than fixed percentage match (24 percent and 20 percent, respectively). To tabulate the outcome ratings, we looked at the projects with fixed matching rates and isolated only the matching rate for firms (and not for associations, or other eligible types of beneficiaries). Only 67 projects had a fixed matching rate for firms, and the vast majority of those were fixed at 50 percent. The distribution was 47 projects fixed at 50 percent, 15 projects fixed at over 50 percent, and only five projects fixed below 50 percent. Only 50 of those projects had an outcome rating, so the sample size is heavily skewed toward 50 percent, and there was no significant difference in outcome ratings be- tween 50 percent match (74 percent positive) and over 50 percent match (83 percent positive). 49 Case study 50 4. Design feature of MG schemes Figure 4.9 Success rate by level of match Not implemented Unsatisfactory Moderately unsatisfactory Moderately satisfactory Satisfactory Highly satisfactory Observations Over 50% 6 50% 38 Under 50% 6 All 50 100% 80% 60% 40% 20% 0% 20% 40% 60% 80% 100% Share of projects While MG beneficiaries are in most cases expected to bring their contribution in cash, some projects allow contributions in kind and a few others require beneficiaries to contribute to a local fund to fi- nance future common projects (mostly rural development). Finally, a couple of projects experimented with repayment of the grant as royalties on additional sales or licensing generated. 51 11: Advance payment of eligible expenses Case study 11 Advance payment of eligible expenses The Nicaragua Micro, Small and Medium Enterprise The Ghana Private Sector Development Project Development Project (P109691, FY08–15) included (P000960, FY1995–2000) featured a Technology and a $8.6 million MG component for MSMEs. SMEs were only Enterprise Development Fund, which provided MGs to eligible for reimbursement of the grant portion based on SMEs which had limited impact. One of the issues identi- proof of expenditure and verification that the activities fied in the project evaluation was the decision by the fund were completed. Based on the experience from a project manager to pay consultants up front instead of reimburs- supported by another donor and considering the large ing companies for services they procured. This resulted proportion of micro businesses in the private sector, in a significant number of clients not paying for services, advance payment was authorized for firms with five em- paying late, or paying a lower-than-required amount and ployees or less. Advances were to follow a predefined generated disputes. disbursement schedule and be conditional on proof that the previous disbursement had been adequately spent. The project exceeded its targeted number of beneficiaries, although the respective proportions of micro businesses and SMEs are unclear. 52 4. Design feature of MG schemes b. Type of payment In the sample, 72 projects have information about the payment modality of the grant. About two- thirds of them provide MGs only on a reimbursement basis, meaning that beneficiaries who sign an MG agreement need to first advance the full cost of services and receive the grant reimbursement only after the service is rendered and appropriate evidence has been submitted to the program ad- ministrator. A minority of projects work on the basis of advances to beneficiaries (11 percent), same time payment to providers (14 percent), or a combination of these three methods for different types of beneficiaries or project sizes (11 percent). An argument used to justify advance payments is the limited capacity of SMEs to advance the full price of services due to low cash flow. While this may be true for the smallest and weakest micro and small firms, advances entail higher risks of abuse and should be used with the appropriate governance considerations. Some projects try to avoid imposing an excessive cash flow burden on beneficiaries by allowing beneficiaries to submit several requests for partial reimbursement for agreed milestone activities. On the other hand, teams should be careful to select reimbursement without ensuring that there is sufficient institutional capacity to swiftly process payments, as cumbersome procedures or inefficient implementing agencies may lead firms to wonder when or whether they will be repaid, which can discourage take-up. A project20 in the sample had an interesting payment arrangement, where the match to cooperatives was set at 30 percent. The other 70 percent were taken out as a loan from the partner financial institutions administering the program, and the 30 percent match was only disbursed once the loan was paid off. A tabulation of projects with reimbursement (74 percent positive) versus all the other arrangements discussed above (78 percent positive) does not show any significant difference in outcome rating. Figure 4.10 Success rate by type of payment Not implemented Unsatisfactory Moderately unsatisfactory Moderately satisfactory Satisfactory Highly satisfactory Observations Reimbursement 35 All other 18 All 53 100% 80% 60% 40% 20% 0% 20% 40% 60% 80% 100% Share of projects A majority of projects allow beneficiaries to submit multiple applications for MGs over the duration of the program, although some projects set a limit of one or two MGs per firm. In both cases, a maximum cumulative amount of grant is generally established, which ranges from as low as $1,000 to as high as $400,000, depending on the project scope and type of beneficiary. Kyrgyz Republic Agribusiness & Marketing Project (P049724). 20 53 12: Provision of services by public institutions Case study 12 Provision of services by public institutions A series of two Croatia Science and Technology Proj- The Indonesia Industrial Technology Development ects (P080258, FY06–11/P127308, FY13) has aimed Project (P003978, FY1996–2002) aimed at commer- at fostering investment in technology and innovation by cializing public R&D institutions by creating incentives for SMEs by offering MGs to finance applied research projects them to become more demand-driven, increasing sales of subcontracted to participating public R&D institutions and technology services to the private sector, and encourag- universities. One objective was to increase collaboration ing competition between public and private technology and trust between SMEs and public R&D institutions and providers. The project included an MG scheme for SMEs, help the latter expand their industry network and out- which was marketed to and preregistered both public and reach. Under the first project, 22 projects were supported, private institutions. The scheme had limited results due with MGs amounting to €2 million for projects totaling to various design and implementation issues. Few grants €4.7 million. This program was expanded under the ongo- were disbursed and SMEs had limited demand for services ing follow-up project. provided by public R&D institutions. In this regard, the project’s evaluation noted that such institutions were gen- erally ill-equipped to provide services to SMEs with limited technical capacity and less sophisticated needs than their usual larger clients. 54 4. Design feature of MG schemes c. Consultant selection The selection of a consultant who can provide good-quality services to grant beneficiaries is essential to the success of an MG program. Several options are available when designing programs, the most common ones being the following: • Free selection: The fund manager generally verifies that the selected provider has appropriate credentials, that prices charged are in line with market realities, and that there is a genuine arm’s- length relationship with the beneficiary to minimize the risk of abuse. Vetted providers can then be added to a non-mandatory list of providers. Programs can also provide support to beneficiaries to identify reliable service providers. Public institutions are rarely eligible or targeted as service providers for MG programs, although a few exceptions are presented in case study 12. • Selection from a roster:21 A list of prequalified consultants is maintained by the fund manager and beneficiaries can only be reimbursed for services provided by firms on this list. Training courses are sometimes provided for consultants who want to operate under the program. Alternatively, consultants can be required to have a specific certification to be eligible. • Competitive award: MG programs can request beneficiaries to award contracts competitively through different methods (for example, shopping and competitive local tender) depending on the amount or type of activities. This can be combined with the requirement to select potential providers from a roster. Depending on the context, there can be a dearth of quality service providers or a high variance in quality with limited information to single out good providers. The most appropriate selection mecha- nism should be determined taking these considerations into account, and other ways to improve the quality of supply should be explored where needed (for example, extending MG eligibility to providers and leveraging a complementary component targeting the BDS supply side). In the Kyrgyz Republic Agribusiness & Marketing Project (P049724), the MG fund hires and trains local consultants from 21 whom the beneficiaries can choose. If international consultants are needed, the fund hires them as well and lets them choose which local consultant to pair with. 55 13: Matching requirement for initial technical assistance Case study 13 Matching requirement for initial technical assistance Under the first and second Tunisia Export Develop- companies in industrial zones and local firms through ment Projects (P055814, FY1999–2005/P071115, an MG program, the Business-to-Business Linkage Fund. FY04–13), the provision of MGs through the Export Mar- Eligible firms can apply for grants on the basis of a linkage ket Access Fund (EMAF) followed a two-stage process with development plan. Firms that do not have the capacity (a) the preparation by SMEs of an export plan and (b) the to develop such a plan on their own can receive support application for MGs to support the activities and services from a consulting firm hired by the Business-to-Business required by export plans. Under the first project, $1.5 mil- Linkage Fund, but are required to finance at least 30 per- lion was reserved for the provision of TA to develop export cent of the cost of these services if full support is needed. plans. EMAF cofinanced 70 percent of the cost of enter- In addition, the fund manager’s responsibilities include prises diagnostic and export plan preparation if requested, the provision of assistance to prospective clients in com- while the remaining 30 percent were paid by firms. The pleting grant applications, as well as free hand-holding project also provided extensive TA and general training to services to beneficiary firms in the implementation of their beneficiaries during the implementation stage. The sec- linkage development plans. ond project did not continue this cost-sharing approach for export plan preparation but increased the emphasis on TA, training, and implementation support for firms, allocat- ing $4 million for these purposes. Both projects attracted a high demand from the private sector and were credited with positive, if temporary, results with regard to export growth and use of export services by SMEs. The Ethiopia Competitiveness and Job Creation Proj- ect (P143302, FY14) aims at fostering linkages between 56 4. Design feature of MG schemes 4.8. Provision of diagnostics and TA to firms Of the full sample of 106 projects, the majority of MG programs (61 percent) provide some form of diagnostic and/or support to SMEs, or are part of a broader project which does, although almost 40 percent of projects do not seem to feature such assistance. Given the limited capacity of most SMEs in low-income countries, this can prove to be a crucial element to ensure that (a) only the most deserv- ing firms are identified and obtain MGs and (b) selected beneficiaries are able to make the most of the subsidy and external consulting services provided. Common forms of diagnostic or support for poten- tial MG beneficiaries include: • Two-stage application process with a mandatory initial diagnostic to verify eligibility, ensure that the most promising firms are selected, and help firms avoid putting significant efforts into devel- oping a proposal if they are not eligible (for example, export readiness assessment for projects focusing on improving SMEs’ access to external markets). • TA to SMEs to prepare sound business plans that can be used to apply for MGs. In some cases, this assistance is provided freely, while in others, applicants have to pay for a share of the cost involved (see case study 13). • Complementary project component to train SMEs and create a pipeline of applicants for MGs. • Continuous provision of TA and training to MG beneficiaries, from application to project imple- mentation. This can sometimes represent a sizeable proportion of MG schemes’ administrative costs. The types of support listed above have various degrees of intensity, which apart from having different implications on administration costs arguably also have a different degree of effect on outcomes. The review did a binary analysis of whether or not any type of diagnostic or TA was offered.22 The positive effect of a diagnostic appears to be quite strong, with 83 percent of projects offering it having positive outcomes compared to only 60 percent of projects that do not. Thus, despite the budgetary implica- tion of offering extra, often free, TA, the data seems to strongly support an argument for including it in the project design. Figure 4.11 Success rate by provision of diagnostic Not implemented Unsatisfactory Moderately unsatisfactory Moderately satisfactory Satisfactory Highly satisfactory Observations Yes 48 No 25 All 73 100% 80% 60% 40% 20% 0% 20% 40% 60% 80% 100% Share of projects 22 Caveat: Given the budgetary and design implications of providing dedicated TA or a diagnostic to beneficiaries, projects that did not explicitly mention these elements in their PADs were assumed not to offer them. To the extent that some projects featured diagnostics or TA that were not mentioned in project documents, the data may slightly misrepresent the true break- down. 57 14: Repeated MG projects Case study 14 Repeated MG projects The first, second, and third Tunisia Export Develop- after the intervention and no evidence of spillovers were ment Projects (P055814, FY1999–2005; P071115, found (Cadot et al. 2015; Gourdon et al. 2011a, 2011b).23 FY04–13; P132381, FY14) have been implemented since 1999 to improve the trade environment in Tunisia The CEDF was launched in end-2015 with a $22 million and the export competitiveness of firms in the country. MG allocation and $1.5 million to administer the scheme Among other components, the three projects have fea- (6 percent of the total, compared to 29 percent and 15 tured an MG scheme (respectively, EMAF I and II and the percent for the first and second projects, respectively). Community Economic Development Fund [CEDF]) to sub- The targeted number of beneficiaries was significantly sidize the cost of consultant services required to enable increased to 2,000 firms. The project drew the lesson from an SME to access new markets, on the basis of an export its predecessors at several levels: development plan prepared with TA provided under the • Implementation arrangements: While EMAF I and II scheme. relied on international contractors, the CEDF’s man- The first EMAF (1999–2004) disbursed around $10 mil- agement was attributed to CEPEX, a public export lion in MGs, with $4 million allocated to the provision promotion agency, with the explicit objective to build of assistance to SMEs and administration of the fund. more durable institutional capacity to administer MG Exceeding the target of 350, 595 SMEs benefited from the programs in the country. Temporary support to CEPEX scheme, and the project was deemed a major success. from a consultant was planned specifically on the MG program, and a whole project component was dedi- The EMAF II (2005–2010) had a higher allocation of cated to building this agency’s overall capacity. $16.6 million for grants, and received an additional financ- ing of $6 million due to stronger-than-anticipated de- • M&E: An amount of $0.5 million was allocated to es- mand. Again, $4 million were allocated to administer the tablish an M&E system of the MG scheme by following scheme and accompany firms, an amount that remained enterprises throughout the project implementation constant with the additional financing. While the program period, including a control group identified ex ante. was similar overall, some design refinements were in- This was consistent with the recommendation made cluded (for example, increased scope of eligible activities, by the evaluators of the second EMAF to build impact increased provision of TA to firms, allowing associations to evaluation ex ante in the project’s design. apply, and higher maximum grant size). The initial target • Economic impact: An increased emphasis was put of 500 SMEs and 40 professional associations was in- by the Government on the objective to maximize creased to, respectively, 1,000 and 50 after the additional economic benefits from the scheme rather than only financing, and was eventually exceeded as 1,239 firms financial benefits of private firms. The CEDF scheme and 93 associations received grants. An ex post impact was designed with these concerns in mind, by primar- evaluation of the project concluded that it had statistically ily targeting sectors and firms with a high potential to significant positive effects on beneficiaries with regard to increase economic diversification, foster innovation, short-term export growth and diversification, especially for create skilled employment, reduce geographic dispar- first-time exporters. However, no difference between ben- ities, and so on. eficiary and non-beneficiary firms were found three years Cadot et al. (2015) suggest that the impact of the EMAF II program 23 may have been transient because it did not lead to the enhancements in product quality or sophistication, which could have strengthened com- petitiveness durably. Notwithstanding its transient effect, the program is deemed to have been cost-effective because it generated two Tunisian dinars of private profits per Tunisian dinar of program expenditure, and the additional corporate tax revenue just covered the public cost of the program. 58 4. Design feature of MG schemes 4.9. Link with other project components MG components are more often than not part of larger projects with other components. Their allo- cated amounts can represent anywhere between a few percent and all of the total project amount, averaging around a third. In the sample reviewed, the other most common components dealt with institutional capacity building (60 percent of projects), business environment and regulatory reforms (51 percent), and access to finance (34 percent). Synergies can clearly be exploited between the support directly provided to SMEs through MGs and other initiatives to improve the business environment. For instance, Phillips (2001) made the case for coordinating grants for know-how with a public goods element and credits for traditional bankable investments. Likewise, investment climate reforms could aim at reducing the supply-side constraints and market failure that undermine firms’ capacity and willingness to procure BDS. In practice, however, there rarely seem to be strong links between MGs and other components. For instance, the program document of the Tunisia Third Export Development Project (P132381) noted that the efficiency and sustainability of the previous two projects was undermined by the lack of syn- ergies between components (MGs and export finance guarantee facilities). It was decided for the third project to modify institutional arrangements to ensure synergies. 4.10. Series of projects The 106 projects with an MG component reviewed for this study were implemented in 64 different countries. Out of these, 3 countries (Mexico, Nigeria, and Zambia) had 4 projects each, 8 countries had 3 projects, 17 countries had 2 projects, and 36 countries had only 1 project. While for some countries this corresponds to operations with limited mutual links (for example, rural development and SME competitiveness projects), in others, series of similar follow-up operations were implemented (for ex- ample, Colombia, Croatia, Lesotho, Moldova, and Tunisia). One could think that design and implementation experience allows later projects to have more favor- able outcomes than earlier ones in the same country. While this indeed seems to have been the case for some countries, there is no systematic trend and in some cases the opposite is found. This may be explained by the fact that insufficient lessons are drawn from previous experiences, that broader exogenous factors influence the outcome, or that repeat projects are precisely prepared where good results were achieved in the past. 59 15: High and low disbursement of MG projects Case study 15 High and low disbursement of MG projects The Moldova Competitiveness Enhancement Project The Guatemala Competitiveness Project (P055084, (P089124, FY06–13) adopted a two-staged approach FY01–09) featured an MG ($5 million) and micro vouchers to support SME through MGs. It initially featured a $1.2 ($1 million) component, respectively, expected to benefit million MG facility designed to extend 100 to 140 grants 300 SMEs and 10,000 micro businesses by supporting to firms, focusing on quality upgrading and international investment in BDS to increase competitiveness. Only one certification. Based on good results, the facility’s size was round of MGs was initiated, and 448 enterprises applied later increased with a $1.5 million additional financing. As for funding. After an intensive vetting process, four com- foreseen in the original project document, its scope was panies received grant funding, equivalent to 2 percent of expanded to cover a broader array of BDS (e.g., upgrade available funding. No additional competition was held, as labor skills and management practices and introduce new several rounds of bidding did not lead to the contracting products). The program exceeded its original targets and of a suitable implementation agency that had the capac- provided grants to 479 firms. The scheme was deemed a ity and knowledge to roll out the component on a larger success and was credited with having played an important scale throughout Guatemala. Likewise, the micro voucher role in the development and consolidation of the consult- scheme was never implemented. The project suffered ing services sector in Moldova on both the demand and from the lack of commitment by successive governments, supply side. and the internal review of the project also faulted its overly complex and ambitious design. The MG/micro voucher schemes were initially the largest component of the World Bank project, and the bulk of funds were canceled or redi- rected to other project activities. Around the same period, an Inter-American Development Bank MG project was un- dermined by similar implementation arrangement issues, and an Inter-American Development Bank study conclud- ed that Guatemala lacked important preconditions for a successful rollout of MGs. 60 4. Design characteristics of successful projects 4.11. Implementation and disbursement of funds The capacity of MG schemes to disburse the funds allocated and reach the expected number of ben- eficiaries has often been retained as the main measure of success, although as argued in this report, these measures of output do not imply positive outcome and impact. It is nonetheless useful to look at the average performance of MG schemes with regard to disbursement as an indicator of imple- mentation quality. Out of the 55 closed projects for which information was found on both the initial amount allocated to MGs and the amount actually disbursed, the average disbursement rate reaches slightly over 80 percent. Interestingly, no obvious correlation is observable between the size of the MG facilities, or the country’s development level (proxied with GDP per capita), and their capacity to disburse fully. Some projects with disbursement difficulties have reacted by reducing the MG amount, sometimes by as much as 80 percent, or even by dropping the MG component altogether. Other projects have attempted to increase take-up by relaxing eligibility criteria, increasing the level of subsidy to make the scheme more attractive, simplifying application procedures, or providing technical support to prepare applications. However, over a dozen projects disbursed two-thirds of funds or less at closing (with five projects disbursing less than 20 percent). On the other hand, some MG schemes received additional fi- nancing to adapt to higher-than-anticipated demand, in some instances doubling the initial allocation. The targeted number of beneficiaries varies largely depending on several factors, including project size, country size, and type of subprojects supported, with the objectives ranging from a handful of R&D projects to several thousands of microenterprises. Among the 70 projects that specified a target- ed number of beneficiaries at appraisal, the average number was just under 800 beneficiaries and the median 225. Among the 38 closed projects for which both initial target and actual number of bene- ficiaries were found, the latter represented on average 129 percent of the former, meaning that initial targets are often exceeded (and are sometimes revised upward at project restructuring) (figure 4.12). Figure 4.12 Targeted versus actual number of MGs MG scheme Actual number of grants (log) 10,000 1,000 100 10 10 100 1,000 10,000 Targeted number of grants (log) 61 Case study 62 5. Design characteristics of successful projects 5.1. Features of successful and unsuccessful projects This section looks at the design features of projects at the tail end of the outcome distribution. Among the 75 closed projects in the sample, 6 were deemed highly satisfactory and 8 were deemed unsatis- factory. The outcome of the MG component is generally, but not always, correlated with the overall outcome of the project (four of the eight unsatisfactory MG schemes were part of broader projects rated mod- erately satisfactory or higher). There are no evident differences between successful and unsuccessful projects with regard to size of MG facility, implementing agency, share of operating costs, country characteristics, and so on. With regard to design features, T 5.1 would suggest that the former tend to be simpler (for example, well-identified implementing agency, single funding window, and few restrictions on eligibility), to include BDS providers, and to explicitly provide assistance to beneficiary firms. Overall, however, there is no evident design feature or combination of features that seems to be correlated with outcome among the highly satisfactory and the unsatisfactory projects in the sample. Table 5.1 maps out their design features and case study 16 provides examples. 63 Table 5.1 Main design features of successful and unsuccessful projects Restrictions for BDS Funding Applicants Providers TA to Project Years Implementing Agency Windows Size Sector Eligible Beneficiaries Highly Satisfactory Projects Mauritius 1994–1999 Private contractor (P001918) 1 l l Uganda 1996–2003 Private contractor (P035634) 1 l l Madagascar 1997–2003 Private contractor 1 (P001555) Tunisia 1999–2005 Private contractor (P055814) 1 l l l l Uganda 2005–2013 Private association (P083809) 1 l l Mali 2006–2015 Private contractor (P081704) 1 l l l l Unsatisfactory Projects Ghana 1995–2000 Partnership ministry/ 1 (P000960) nonprofit l l l Zimbabwe 1996–2003 Four different agencies 4 (P035628) l l Zambia 1997–2003 Private contractor 1 (P044324) l l Guatemala 2001–2009 Private contractor 1 (P055084) planned but not found l l l Senegal 2003–2012 Private contractor 1 (P051609) l Honduras 2004–2011 Private contractor 5 (P070038) l l Zambia 2005–2012 Private contractor 1 (P071407) l Sri Lanka 2010–2014 Ministry at design, trans- 2 (P113709) ferred to public bank l l during implementation 64 5. Design characteristics of successful projects 5.2. Econometric analysis a. Model We used a linear multivariate regression (detailed in annex 3) to assess the extent to which the design features discussed in section 4 are systematically correlated with the development outcomes of MG programs, controlling for country-specific conditions. The dependent variable is the outcome rating for the MG scheme, which was directly extracted from ICR and ISR documents or inferred based on the information provided in these documents. The variation in project outcomes is examined as a function of country conditions and project-specif- ic design characteristics. With regard to country conditions, the control variables are GDP per capita (current U.S. dollars), Index of Economic Freedom, Doing Business - distance to frontier, Country Policy and Institutional Assessment (CPIA) score (1 to 6, higher is better), and agriculture, value added (per- centage of GDP). For each control variable, the value is assigned for each individual project for the year when the project was approved (or the closest available), which means that multiple projects in the same country will have different control values if they were approved in different years. Since the CPIA score is relevant only for IDA countries, controlling for it significantly limits the number of observations; so each of the models was run with and without the CPIA score. With regard to project-specific design characteristics, nine independent variables were tested in nine separate models, captured in a series of dummies, except for the level of match which is numeric. The variables are PIU, size limit, groups, service providers, sector restrictions, equipment, level of match, reimbursement, and diagnostic. The variable characteristics and the regression results are presented in annex 3. b. Findings • Other things being equal, there is no significant association between any of the country variables and outcome. One exception is the Index of Economic Freedom, which is very slightly but signifi- cantly positively correlated with outcome in the two models that control for reimbursement. • Project design features that show a statistically significant correlation with outcomes are: -- The project being implemented by a PIU is significantly and positively correlated with out- come when CPIA scores are also controlled for. Thus, in IDA countries, when institutional and economic management aspects of the economy are controlled for, projects implemented by a PIU are associated with a unit higher rating on outcome measures. This association becomes insignificant if CPIA is not controlled for, which increases the sample to include all countries. A plausible explanation is the difficulty in procuring large international contracts in low-capacity environments, which can cause delays and affect implementation, or alter- natively the added importance of knowing the local context, which may make PIUs more successful in IDA countries. 65 16: Highly successful and unsuccessful projects Case study 16 Highly successful and unsuccessful projects For some successful projects, there were indications that On the contrary, other projects were deemed to have had gains extended beyond direct beneficiary firms and were no or minimal durable impact on the BDS market and on sustainable. For instance, the Uganda Private Sector SME productivity. The Zambia Enterprise Development Competitiveness Project (P035634, FY1996–2003) Project (P044324, FY1997–2003) was less successful disbursed $3 million to 538 firms (well above the initial with its MG component, which only disbursed less than target of 300) through the Business Uganda Development half of its $2.5 million allocation for a variety of reasons Scheme to support capacity building. Around 40 percent linked to project design and the broader economic en- of the beneficiary firms were located outside the capital, vironment. Given the importance of a few large firms in Kampala, and 16 percent were managed by women. A the Zambian economy, the scheme was intentionally survey of beneficiaries indicated that the grant and TA pro- designed to encourage a higher use of BDS even by large vided were crucial for most firms to undertake their proj- firms and those which could pay for these services. In ects and had significantly contributed to increased sales, the end, only 63 companies received grants, 7 of which improved management systems, and new market knowl- received 42 percent of funds. Some of the largest grants edge. It was estimated that sales growth for beneficiaries went to the strongest companies in the country, which had been 40 percent higher than that of firms outside the had already invested their own funds in similar activities. scheme. Maybe more importantly, about two-thirds of the This outcome was unintended, and the program failed to beneficiaries surveyed indicated their willingness to pay reach smaller beneficiaries despite substantial commu- the full price for future use of similar BDS. nication efforts. While the depressed economic context and complex reimbursement procedures did not help, the main cause of the low performance was the failure to adapt the scheme to Zambia’s lack of corporate depth at the time. 66 5. Design characteristics of successful projects -- Reimbursement as a method of payment is not correlated with outcome when CPIA is con- trolled for, but is negatively correlated with outcome and significant when CPIA is not con- trolled for. So in IDA countries, all things being equal, reimbursement does not seem to be systematically linked with outcome, while in the full sample of rated projects, it seems to be negatively correlated with outcome. A possible explanation could be that processing delays linked with reimbursement negatively affect disbursement or repayment uncertainly affects initial take-up. -- Having a diagnostic is positively correlated with outcome and associated with an increase of 0.8 unit of development outcome when CPIA is controlled for. It remains positively correlated with outcome even when CPIA is not controlled for, though the coefficient goes down to half a unit of development outcome and the R squared is also lower—it explains only 8 percent of the variation in outcome versus 18 percent when CPIA is controlled for. The findings regarding implementation by a PIU and having a diagnostic reflect and confirm the tab- ulations discussed in section 4. Reimbursement shows only a marginally lower share of positive out- comes in the tabulations, though the regression analysis shows it as significant when country variables are controlled for in the full sample of projects. 67 17: Randomized allocation of MGs for impact evaluation Case study 17 Randomized allocation of MGs for impact evaluation Despite the theoretical ease of implementing a random- term effects, the results of which are reported in McKenzie ized evaluation of MG programs, there have been more et al. (2016). MGs do appear to have resulted in additional failed attempts than successfully completed randomized innovation activities being undertaken by firms. For in- evaluations of MG schemes. Campos et al. (2012) describe stance, beneficiaries were more than twice as likely to in- attempts to conduct such evaluations for seven programs troduce a new product as firms in the control group. They in six African countries, which failed due to a mix of im- were also more likely to do more marketing, introduce plementation delays, lack of political will to allow random new accounting systems, make a capital investment, or assignment, and low program take-up. Several factors use a consultant. However, data limitations prevented the contributing to these outcomes are highlighted, including authors from assessing impacts on employment or sales. political economy issues, overly stringent eligibility criteria Several conclusions are drawn: (a) attention in design to that do not take account of where value added may be make the application process better for firms can lead to highest, lack of attention to detail in ‘last mile’ issues, in- high take-up and make it feasible to evaluate through a centives facing project implementation staff, and the way randomized oversubscription design, (b) there is some impact evaluations are funded. The authors draw lessons evidence that MGs can have additional impact beyond from these experiences for both the implementation and what firms would do anyway, and (c) further evaluation on the possible evaluation of future projects. a larger scale is needed to measure other key outcomes such as employment and sales and whether the grants Taking these lessons into account, the Yemen SME Revi- also involve externalities for other firms. talization and Employment Pilot Project (P143715, FY13–16) planned to randomly allocate screened SMEs Another successful randomized trial of a regional MG pro- either to process to receive an MG or to a control group gram for SMEs in Mexico was conducted a few years ago, to later be able to conduct a proper evaluation of the the methods and results of which are reported in Bruhn et program’s additional impact on beneficiaries’ performance al. (2013). This analysis notably concluded that the consult- and employment levels. Demand largely exceeded the ing interventions had a positive short-term impact on the number of grants available, allowing this randomized al- productivity and return on assets of firms in the treatment location at publicly held events. While this program was group, as well as on employment several years after the interrupted by the civil war in early 2015, a partial impact program. evaluation was nonetheless carried out to measure short- 68 6. Defining and measuring success 6.1. Institutional arrangements for M&E Unequal efforts and resources are dedicated by projects to ensure sound M&E arrangement for the MG schemes. While fund managers generally have the responsibility to monitor the use of funds and report on outcomes, this is sometimes more of an afterthought without specific resources allocated to M&E. However, other projects follow best practices by including M&E from the start as a constitutive component of the program. For instance, an impact assessment unit was one of the three compo- nents of a project in West Bank and Gaza (P110526), for which a separate M&E Handbook specifying the approach to track client projects and measure results was prepared alongside the Grants Manual. Likewise, a project in Afghanistan (P118053) made detailed M&E arrangements, notably featuring a dual system with (a) internal M&E (including regular supervision monitoring and rating of each project under implementation, portfolio management at the aggregate level, and initial qualitative and quan- titative impact assessment of each completed project)and (b) annual contracting of independent ex- ternal M&E services for both randomized and representative sampling impact evaluations to continue several years after projects are completed. Importantly, these arrangements must detail the resources and staffing allocated to M&E and clearly spell out the reporting obligations of beneficiaries. Ideally, continuous M&E from the start of the project should allow the continuous adaptation of program design (e.g., eligibility criteria, matching requirement, and authorized activities) to address issues that surface during implementation and maximize impact. 6.2. What do we measure? We extracted from the reviewed projects all indicators used to measure the MG scheme. Nine projects had no indicators at all to measure the MGs, and a further 16 projects had only output indicators, such as number of grants awarded or volume disbursed. This means that a quarter of the projects in the sample had no meaningful indicator with which to gauge success. 24 The other 75 percent of projects used a combination of outcome and impact measures and at the highest levels of their results chains measured firm-level impacts such as jobs, sales, or productivity growth. The type of assessments used to measure firm-level impact also vary significantly from project to project. Satisfaction surveys for a sample of beneficiaries seem to be accepted as the basis for out- come ratings in ICRs, even though they have serious attribution issues and do not have a counterfac- tual with which to compare reported impacts. The best practice to establish causality and attribution and address potential endogeneity concerns is to plan an impact evaluation from the beginning by randomly allocating firms to the treatment (that is, receiving a grant) and control groups and collect- ing baseline data, although this is rarely done due to the cost and complexity of such undertakings (see case study 17 for examples of failed and successful attempts). In the absence of ex ante random- ized allocation of MGs, some projects have attempted second-best ex post strategies, such as collect- Annex 1 to this report includes a list of results indicators used to measure the MG schemes in the reviewed sample. 24 69 6. Defining and measuring success ing information on a control group selected ex post,25 or by randomly selecting a control group of similar non-beneficiary firms and controlling for observable differences with the treatment group with regard to size, age, sector, and so on.26 6.3. Is this the right thing to measure? As discussed above, attribution of firm-level impacts to MG programs is difficult to establish. In addi- tion, there is very little data on how sustainable firm-level impacts are. There are few rigorous longitu- dinal studies of MG beneficiaries and consequently, little is known about whether there is a time lag to observe the benefits from MGs or conversely whether observed benefits disappear over time (IFAD 2012), which presents an additional challenge for measurement, particularly post-project close. But even if firm-level impacts are established, is that the right level at which to measure the success of MG programs? As explained in section 2, the theoretical economic case for MGs and the conditions needed to justify them have been well understood for a long time. However, the justification for the use of this instru- ment is rarely well articulated in the reviewed projects. Common issues in projects’ MG justification include the absence or poor quality of (a) demand assessment and information about the local BDS market, (b) discussion of the specific market failures or the generation of public economic gains that the use of an MG would address, and (c) economic analysis of the project’s costs and benefits com- pared with a counterfactual without the project or the potential use of a different instrument. Even when some of these elements are included in project documents, they are sometimes formulated in generic terms, without sufficient efforts to confirm their validity for the specific context of the proj- ect.27 If an adequate economic rationale for the instrument is missing, many projects subsequently fail to set adequate objectives for the MG, which makes it difficult to measure their success through an M&E framework. For example, in the case of innovation grants, the economic rationale is generally tied to the positive spillover effects of innovation that make its benefits hard to appropriate for specific firms and thus make them underinvest in innovation activities. Thus, the ultimate measure of success for such an intervention would be to try to establish whether the program had a positive catalytic effect on the level of innovation in the country or use proxies to gauge the level of spillovers from recipient firms to other firms and the larger economy. In projects that are not directly funding innovative activ- ities, the most typical justification for the use of public funds to subsidize private firms is one or many of a series of market failures, both from the supply as well as demand side, that result in an underde- veloped market for BDS. Yet, very few projects attempt to measure their impact on the market for BDS and whether the fund, which is intended to be temporary, has had additionality through a demonstra- tion effect and has catalyzed the market and moved it to a higher and sustainable equilibrium.28 On the contrary, several projects that had good intermediate outcomes nonetheless noted an absence 25 See for instance the report prepared by a third-party consulting company for the West Bank and Gaza Fund for New Market Development (P110526): “Facility for New Market Development (FNMD) to Strengthen the Private Sector in the Occupied Pales- tinian Territories – Final Evaluation”, May 2012, Triple Line Consulting, available at: www.oecd.org/derec/unitedkingdom/8_Evalu- ationFacilityNewMarketDevelopmentFNMDStrengthenPrivateSectorinOccupiedPalestinianTerritories.pdf. 26 See for instance the impact evaluation of the EMMAF II MG facility implemented in the context of the second Tunisia Export Development Projects (P071115), presented in Cadot et al. (2015) and Gourdon et al. (2011a, 2011b). For an earlier example on Mauritius, see Biggs (1999). 27 Van der Meer and Noordam (2004) also emphasized that World Bank projects generally did not provide enough evidence on the economic justification for the use of MGs, paid little attention to potential distortive effects on markets, and lacked rigor- ous cost-benefit or cost-effectiveness analyses. The authors notably argue that “loose claims of market failure can easily result in misguided interventions with grants” and discuss alternatives to grants to address various types of constraints on private invest- ment. 28 See annex 1; only four projects out of the 106 have any indicator measuring the fund’s effect on the market for BDS. 70 6. Defining and measuring success of increased willingness or capacity of firms to pay the full price of services, the absence of a supply response from local service providers, or increased prices with higher demand due to inelastic supply. Almost none of the reviewed projects conduct or refer to a demand assessment of the targeted popu- lation of firms to try to identify the precise market failure that is causing the perceived low productivity or low value addition, or lack of growth. Consequently, most lack specific objectives for the MG facility and thus do not select results metrics that can adequately measure the fund’s additionality. Inadequate efforts to ensure the project has a strong economic rationale increase the risks of low grant take-up by targeted beneficiaries, or incorrect targeting that limits additionality compared to what firms would have paid for anyway, absence of sizeable or durable impact on the BDS market, and so on. On the other hand, well-designed projects that are adequately tailored to local circumstances and capacity have been able to show significant additionality, sustainability, and demonstration ef- fects. For this reason, the first step for task teams should always be to establish a strong economic justifi- cation for the project (Phillips 2001). This principle is not specific to MG projects but is worth empha- sizing given the deficiencies presented above. This implies (a) dedicating sufficient resources during preparation to collect information, allowing a real reflection on the current obstacles to BDS use as well as the potential for additionality/externalities and (b) justifying that grants are the best available tool to address these constraints. The preparation of a cost-benefit analysis and computation of an economic rate of return are difficult in the case of MGs (for example, difficulty to assess spillovers and unknown effective demand for grants). However, Van der Meer and Noordam (2004) provide some useful economic principles and highlight a couple of projects that conducted ex ante cost-benefit analyses. At the very least, a qualitative discussion of the economic justification for the project with references to the impact of previous projects in comparable contexts should be provided. 6.4. Design and implementation tailored to success Once a sound economic rationale has been established for the use of MGs to address specific con- straints, the scheme should be designed in a way that directly addresses these constraints and maxi- mizes economic gains from the project, through the choice of eligibility criteria, grant modalities, type of projects supported, and so on. The reviewed projects often provide insufficient details on implementation modalities and their justifi- cation before project launch. Program documents sometimes contain limited specific information and fail to link implementation modalities with the economic rationale for the project.29 A detailed imple- mentation manual to guide the implementing agency’s actions is usually put together during the project preparation phase and its adoption is usually added as an effectiveness or disbursement con- dition, along with the establishment of an MG review committee and recruitment of a fund manager. However, some projects have postponed these steps until after effectiveness and have considered the elaboration of the MG manual as a project output. This is a potential risk for implementation and has been highlighted as a source of delays and implementation difficulties in several projects, for instance in Senegal in the 1990s and more recently in Sri Lanka. Here again, Van der Meer and Noordam (2004) already noted that the frequent lack of information in the project documents 29 means that much decision-making is left for the implementers. 71 18: Economic rationale for MG projects Case study 18 Economic rationale for MG projects The Bangladesh Export Diversification Project activities that were already routinely accomplished by (P049790, FY1999–2004) is a good example of efforts the private sector, credit constraints often precluded in- to provide an economic justification for the use of MGs. As vestments recommended by consultants, there was no part of project preparation, a detailed needs assessment observed increase in SMEs’ willingness to pay for BDS, and demand analysis were prepared for key export sectors and the small number of grants extended was unlikely to based on firm surveys, meetings with firms and profes- have affected the BDS market in the country) (World Bank sional associations, and letters of potential beneficiaries 2005). expressing interest in the scheme. Project documents also included a detailed description of the scheme’s objectives Likewise, in his evaluation of the Technology Diffusion and implementation modalities. The project had good Scheme supported under the Mauritius Technical Assis- outcomes and was credited with having encouraged tance Project to Enhance Competitiveness (P001918, greater use of export support services, resulting in greater FY94–99), Biggs (1999) concludes that the project had a development of new products and diversification into good justification and was well managed, but that short- new markets for beneficiaries. comings in design and implementation prevented it from maximizing additionality and selectivity and hence seri- Project documents for the Burkina Faso Competitive- ously limited its economic benefits in practice. It is note- ness & Enterprise Development Project (P071443, worthy that the self-evaluation prepared at project com- FY03–14) also included a detailed discussion of market pletion deemed the Technology Diffusion Scheme ‘highly failures and firms’ needs, adapted firm and project selec- satisfactory’ on the basis of the project’s outputs. tion criteria to focus on additionality and spillovers, and carried out a cost-benefit analysis of the MG component. The MG component of the Zambia Enterprise Develop- The project which benefited from additional financing ment Project (P044324, FY1997–2003) was deemed during implementation had good results, exceeding its unsatisfactory due to low disbursement and limited ad- targets and improving the performances of beneficiaries. ditionality/selectivity (grants went mostly to some of the largest Zambian companies). Several factors likely contrib- There are also several examples of projects for which insuf- uted to this outcome, from the depressed economic situ- ficient attention to the economic rationale for the project ation and high level of uncertainty at the time the project and design led to disappointing results. was promoted to the excessively complex procedures for small businesses. However, the ICR emphasized the lack of A detailed review of the Indonesia Industrial Technol- corporate depth in Zambia (meaning that the scheme was ogy Development Project (P003978, FY1996–2002) designed without due consideration for the small size of partly blamed the MG scheme’s poor economic benefits the private sector) and noted the need for more rigorous on its insufficient analytical underpinnings and design analytical work and market investigation before appraisal. shortcomings (for example, many subprojects financed 72 7. Conclusions and recommendations After over two decades of use in development projects, rigorous evidence of the impact of MGs is weak. They have channeled hundreds of millions of dollars to SMEs in dozens of countries, often with positive results. Compared to the earlier efforts to improve enterprise technical capabilities through services provided by central public institutions (Briggs 1999), the current demand-driven nature of MGs, which lets private operators decide which services they want to invest in, represents a desirable evolution. However, experience has shown that MGs can also be difficult to implement and that they rarely yield the type of broad and durable economic benefits that would justify the subsidization of private enterprises with public funds. For the most part, the general principles that should guide the design and implementation of the MG programs were laid out in the early 2000s (Biggs 1999; DCED 2001; Phillips 2001; Van der Meer and Noordam 2004). Several studies have completed our understanding of the conditions under which this instrument is more likely to work or not. However, the efforts dedicated to establish a sound rationale for the MG schemes and to design and implement them with due consideration for local circumstances often remain insufficient. Given the long list of potential difficulties at different levels of MG implementation (e.g., take-up level, administrative inefficiency, fraud, and political capture), a more thorough discussion of risks specific to the MG component would also be warranted in project documents. It is worth (re)emphasizing some of these principles concerning (a) the economic rationale, (b) the design and implementation, and (c) the M&E of the MG programs: 73 7.1. Economic rationale MGs should only be used when they are determined to be the most adapted and least-cost tool to reach the project objective in the specific local context. In some cases, other tools such as cash trans- fers might be better suited to subsidize poor entrepreneurs. The use of subsidy must be based on a sound economic rationale, generally the tackling of a well-identified market failure, and directed at clear beneficiary groups with a verified demand. The intervention’s capacity to have a broad and du- rable impact on both the supply and demand for BDS should be presented, with an explicit analysis of the potential for additionality and spillovers and a realistic exit strategy. Based on these elements, a rigorous economic analysis should be prepared early on. All of this requires to dedicate more time and resources to design and particularly to the analytical underpinnings that inform it than is typically done. Experience has shown that failing to do so is likely to result in suboptimal outcomes, such as limited additionality and spillovers, weak demand and disbursements, unintended consequences on the BDS market (e.g., price increase if the supply is inelastic), or nonsustainable impact if the project does not address binding constraints for SMEs (e.g., access to credit to finance intangible BDS).30 7.2. Design and implementation There is no universal blueprint that can be applied for MG schemes, and much depends on the project objectives and local conditions. None of the design features considered in this study has been found to have a systematic impact on outcomes, which means that programs need to be tailored to local circumstances and capacities. The functioning of the MG programs should be clearly presented in project documents (and MG manuals drafted early), but the schemes should remain flexible enough to adapt to unforeseen difficulties as well as to changing environments and external shocks. This study has attempted to present the pros and cons of different design modalities. Based on anecdotal evi- dence and lessons learned from past experience, several general considerations are worth keeping in mind for the design of the MG schemes: i. Although no causal relationship is established, the design modality that seems to be most often correlated with positive outcome is the provision of personalized TA to beneficiary firms before, during, and after the application stage. An active MG scheme seems to be better at supporting beneficiaries and increasing their odds of successfully completing the projects. ii. Implementing agencies can be public or private/international, although the progressive strength- ening of local capacities is advisable where possible. They should in all cases be free from political interferences and have the technical capacity to implement the program before its launch. Broad stakeholder engagement, transparent communication, and involvement of the private sector in the grant attribution process can help minimize the risk of political capture. iii. Generally speaking, beneficiaries should be able to select their service providers, although the choice can be limited to a roster of providers meeting basic eligibility criteria to ensure quality of 30 These principles apply to interventions targeting SMEs in general. As put by IEG (2014), “A credible theory of change for SME interventions must be focused on leaving a sustainable supply of the service (such as financing, BDS, or training) by establishing well-functioning markets and institutions, not simply providing a temporary supply of benefits to a small group of firms during a project’s lifespan. The scale of gaps identified for SME services, especially finance, dwarfs the direct benefits the World Bank Group can deliver. Thus, targeted interventions need to be strategic, leveraging resources to produce broader benefits for in- stitutions and markets. Targeted support for SMEs needs to be firmly rooted in a clear, evidence-based understanding of what distinguishes an SME and how the proposed support will sustainably remove the problems that constrain SMEs’ ability to con- tribute to employment, growth, and economic opportunity.” 74 7. Conclusions and recommendations services. The MG administrators can nonetheless play an active role to help them identify good providers as well as to minimize the risk of fraud or conflict of interest. iv. MGs should be awarded based on objective and transparent criteria, including proven capacity to implement the project as well as demonstration of the additionality of resources (as discussed above, this additionality should be estimated not based on private returns but based on returns to public funding such as demonstration effects, spillovers, and so on). v. The level of subsidy should be sufficiently attractive for targeted beneficiaries but should not be so high that it reduces SMEs’ ownership. vi. Application and disbursement procedures should be streamlined and in line with beneficiaries’ capacities, so as to avoid discouraging them from applying or imposing an excessive administra- tive burden. vii. Marketing and communication campaigns with a clear and accessible messaging geared toward SMEs can help increase take-up. viii. There is often scope for synergies with other project components that can ease some of the sup- ply-side constraints that could undermine the breadth and sustainability of benefits from MGs. 7.3. Monitoring & Evaluation This is an essential but often neglected element of MG programs, which is directly linked to the defini- tion of the economic rationale discussed above. A solid monitoring system is required to identify po- tential issues in the disbursement of funds and execution of subprojects as well as to minimize the risk of fraud. Evaluation systems are essential both during the project’s life, for a preliminary assessment of impact and to adapt the design if needed, and after completion to assess the attributable economic benefits. From the beginning, this notably supposes (a) establishing a sound results framework, which is directly linked to the market failure the project aims to address (with measurable and attributable outcome and impact indicators) and (b) allocating sufficient human and financial resources to M&E to determine baseline and continuous data collection mechanisms for beneficiaries and control groups, and so on. It is good practice to complement the internal M&E systems of the implementing agency with independent performance evaluations. Finally, rigorous impact evaluations (based on random- ized allocation of grants or other quasi-experimental methods), while costly and complex to carry out, can yield valuable information for future operations and should be considered where possible.31 31 McKenzie (2011) discusses the technical difficulties in carrying out statistically valid evaluations of SME programs, which typ- ically target a small number of heterogeneous firms. The recommendations are to try to better target beneficiaries to increase homogeneity and to collect more detailed and frequent data on them. 75 References Biggs, Tyler. 1999. “A Microeconometric Evaluation of the Mauritius Technology Diffusion Scheme (TDS).” Regional Program on Enterprise Development Discussion Paper 108, World Bank, Washington, DC. Bruhn, Miriam, Dean Karlan, and Antoinette Schoar. 2013. “The Impact of Consulting Services on Small and Medium Enterprises – Evidence from a Randomized Trial in Mexico.” Policy Research Working Paper 6508, World Bank, Washington, DC. Cadot, Olivier, Ana M. Fernandes, Julien Gourdon, and Aaditya Mattoo. 2015. “Are the Benefits of Export Support Durable? Evidence from Tunisia.” Journal of International Economics 97 (2): 310–324. Campos, Francisco, Aidan Coville, Ana M. Fernandes, Markus Goldstein, and David McKenzie. 2012. “Learning from the Experiments that Never Happened.” Policy Research Working Paper 6296, World Bank, Washington, DC. DCED (Donor Committee on Enterprise Development). 2001. “Business Development Services for Small Enterprises: Guiding Principles for Donor Interventions.” http://www.enterprise-development. org/page/bds. Goldberg, Michael, and Daniel Ortiz del Salto. 2009. “Matching Grants: A Review of Matching Grants Systems in Private Sector Development Projects.” Unpublished Memo. Washington, DC: World Bank. Gourdon, Julien, Jean-Michel Marchat, Siddarth Sharma, and Tara Vishwanath. 2011a. “Can Matching Grants Promote Exports? Evidence from Tunisia’s FAMEZ II Programme.” In Where to Spend the Next Million? Applying Impact Evaluation to Trade Assistance, edited by Olivier Cadot, Ana Fernandes, Julien Gourdon, and Aaditya Mattoo, 81–106. Washington, DC: World Bank. ———. 2011b. “Ex-Post Impact Evaluation of an Export Promotion Matching Grant: Tunisia’s Second EMAF.” MENA Knowledge and Learning Quick Notes Series No. 40, World Bank, Washington, DC. IEG (Independent Evaluation Group). 2014. The Big Business of Small Enterprises – Evaluation of the World Bank Group Experience with Targeted Support to Small and Medium-Size Enterprises, 2006– 2012. Washington, DC: World Bank. 76 IFAD (International Fund for Agricultural Development). 2012. Matching Grants: Technical Note. Rome, Italy: International Fund for Agricultural Development. McKenzie, David. 2011. “How Can We Learn Whether Firm Policies Are Working in Africa – Challenges (and Solutions?) for Experiments and Structural Models.” Policy Research Working Paper 5623, World Bank, Washington, DC. ———. 2015. “Identifying and Spurring High-Growth Entrepreneurship – Experimental Evidence from a Business Plan Competition.” Policy Research Working Paper 7391, World Bank, Washington, DC. McKenzie, David, Nabila Assaf, and Ana Paula Cusolito. 2016. The Additionality Impact of a Matching Grant Program for Small Firms: Experimental Evidence from Yemen. Washington, DC: World Bank. Phillips, David. 2001. “Implementing the Market-Based Approach to Enterprise Support: An Evaluation of Ten Matching Grant Schemes.” Policy Research Working Paper 2589, World Bank, Washington, DC. Phillips, David, and William F. Steel. 2003. “Evaluating the Development Impact of Demand-Side Inter- ventions for BDS Markets: Kenya Voucher Program.” Small Enterprise Development 14 (4). Pp 39-48. Piza, Caio, Tulio Cravo, Linnet Taylor, Lauro Gonzalez, Isabel Musse, Isabela Furtado, Ana Cristina Sierra, and Samer Abdelnour. 2016. “The Impact of Business Support Services for Small and Medium Enter- prises on Firm Performance in Low- and Middle-Income Countries: A Systematic Review.” 3ie Grantee Final Review. London: International Initiative for Impact Evaluation. Van der Meer, Kees, and Marijn Noordam. 2004. “The Use of Grants to Address Market Failures – A Review of World Bank Rural Development Projects.” Agriculture and Rural Development Discussion Paper 27, World Bank, Washington, DC. World Bank. 2005. Project Performance Assessment Report – Indonesia Industrial Technology Develop- ment Project (P003978). Report No. 32733. Washington DC: World Bank. World Bank. 2005. Project Performance Assessment Report – Kenya Micro and Small Enterprise Training and Technology Project. Report No. 32657. Washington DC: World Bank. 77 78 Annex 1: Results indicators used in reviewed projects The 106 projects used a total of 293 indicators with different wording. We have matched each custom indicator as closely as possible to a generic wording. The table presents the frequency of use of each of the indicators at output, outcome, and impact levels. Output No. Outcome No. Impact No. Long Term No. Number of funded projects/enterprises 67 Share of beneficiaries satisfied with functioning or result of grant 15 Sales revenue/turnover of beneficiaries 36 Jobs 27 Value of dollars disbursed 17 Share of beneficiaries with operational or managerial improvements 13 Value of exports of beneficiaries 21 Income 5 Share of grants successfully implemented 11 Value of private sector investment 11 Unemployment 1 Number of research consortia/productive alliances created 4 Value added per worker/productivity 11 Number of beneficiaries obtaining certification/star classification 4 Number of beneficiaries accessing new markets 10 Number of beneficiaries introducing new Value of procurement contracts signed by beneficiaries 3 10 products or processes Number of recipients obtaining commercial credit 3 Profitability of beneficiaries 6 Number of firms drafting/completing/implementing export 2 Output per dollar spent in grant 4 development plans Value spent by beneficiaries on BDS 2 Export share in sales/GDP 2 Share of female-owned/managed businesses supported with 2 Number of patents 1 advisory services Share of beneficiaries still operating at the end of project 1 Composition of exports, more nontraditional 1 Number of service providers operating in project area 1 Share of cost recovery by participating BDS providers 1 Share of service providers posting gains in activity and range of 1 services Annex 2: List of reviewed MG projects implemented by the World Bank Group32 FY FY Amount MG amount ($, Implied MG Project ID Project Name Country Region Approved Closed Status GP (USD, millions) millions)* Rating P118053 New Market Development Afghanistan SAR 2011 n.a. Active T&C 22 11.2 S P105101 Local Development Project Angola AFR 2010 n.a. Active SPL 81.7 4 n.a. P106752 Unleashing Productive Innovation Project Argentina LCR 2009 n.a. Active T&C 150 60 n.a. P038883 Enterprise Export Development Argentina LCR 1996 2000 Closed T&C 38.5 27 S P008279 Enterprise Development Project Armenia ECA 1997 2003 Closed F&M 16.8 3 MU P146994 Trade Promotion and Quality Infrastructure Armenia ECA 2015 n.a. Active T&C 50 3 n.a. P049790 Export Diversification Bangladesh SAR 1999 2004 Closed F&M 32 12 MS P039882 Private Sector Project Benin AFR 2000 2008 Closed F&M 30.4 4 MU P087925 Land for Agricultural Development Project Bolivia LCR 2008 2015 Closed Ag 15 7.8 MS P120139 Pernambuco Rural Economic Inclusion Brazil LCR 2012 n.a. Active Ag 100 80.2 n.a. P071443 Competitiveness & Enterprise Development Project Burkina Faso AFR 2003 2014 Closed T&C 30.7 5.5 S P119662 Bagre Growth Pole Project Burkina Faso AFR 2011 n.a. Active T&C 115 11 n.a. P074055 Growth & Competitiveness Project Cabo Verde AFR 2003 2010 Closed F&M 11.5 1.8 S P107456 SME Capacity Building and Economic Governance Cabo Verde AFR 2010 2015 Closed T&C 4.5 0.7 MS P089196 Trade Facilitation and Competitiveness Cambodia EAP 2005 2013 Closed F&M 10 1.6 MS P133021 Value Chain Support Project Chad AFR 2014 n.a. Active T&C 10.2 2.5 n.a. P060270 Enterprise Reform Project China EAP 1999 2006 Closed F&M 5 n.a. NI P006870 Export Development Colombia LCR 1993 2000 Closed F&M 50 36 P041642 Productive Partnerships Support Project Colombia LCR 2002 2009 Closed SURR 32 40.1 S P104567 Second Rural Productive Partnerships Project Colombia LCR 2008 2015 Closed Ag 30 24.8 S Congo, Democrat- P092724 Agriculture Rehabilitation and Recovery Support AFR 2010 n.a. Active Ag 120 30 ic Republic of Congo, Democrat- P071144 Private Sector Development and Competitiveness Project AFR 2004 2014 Closed T&C 120 2 MU ic Republic of 79 32 Some projects that are still active may nonetheless have an implied rating for the MG component, if that component has already disbursed and been discussed in ISRs. 80 FY FY Amount MG amount ($, Implied MG Project ID Project Name Country Region Approved Closed Status GP (USD, millions) millions)* Rating Congo, Repub- P118561 Support to Economic Diversification Project AFR 2011 n.a. Active T&C 10 5 n.a. lic of P043736 Private Sector Development TA Cote d’Ivoire AFR 1998 2005 Closed F&M 12 3.6 MU P115398 SME Revitalization and Governance Cote d’Ivoire AFR 2010 2015 Closed T&C 15 3 S P080258 Science & Technology Project Croatia ECA 2006 2011 Closed F&M 40 1.5 S P127308 Second Science & Technology Project Croatia ECA 2013 n.a. Active T&C 26.2 n.a. P040106 International Trade and Integration Project Ecuador LCR 1998 2003 Closed F&M 21 10 S P040824 Competitiveness Enhancement El Salvador LCR 1996 2004 Closed F&M 16 2.3 S P050272 Private Sector Development Capacity Building Project Ethiopia AFR 2005 2013 Closed T&C 24 5.75 MS P098132 Tourism Development Ethiopia AFR 2009 2016 Closed T&C 35 3 n.a. P143302 Competitiveness and Job Creation Project Ethiopia AFR 2014 n.a. Active T&C 250 12.3 n.a. P114240 Growth & Competitiveness Gambia, The AFR 2011 2016 Closed T&C 12 4.6 n.a. P000960 Private Sector Development Ghana AFR 1995 2000 Closed F&M 13 1.9 U P085006 Micro, Small and Medium Enterprise Project Ghana AFR 2006 2014 Closed T&C 45 7.2 MS P055084 Competitiveness Project Guatemala LCR 2001 2009 Closed F&M 20.3 5 U P128443 MSME Development Project Guinea AFR 2013 n.a. Active T&C 10 4.2 n.a. P123974 Business Development and Investment Project Haiti LCR 2013 n.a. Active T&C 20 4 n.a. P101209 Rural Competitiveness Project Honduras LCR 2008 n.a. Active Ag 30 19.3 S Enhancing Competitiveness: Trade Facilitation and Productivity P070038 Honduras LCR 2004 2011 Closed F&M 28.1 3.96 U Improvement Project P076467 Chattisgarh District Rural Poverty Reduction Project India SAR 2003 2010 Closed Ag 112.6 98.3 P084792 Assam Agricultural Competitiveness Project India SAR 2005 2015 Closed Ag 154 23.13 S P003978 Industrial Technology Development Indonesia EAP 1996 2002 Closed F&M 47 6.2 MU P048715 Information Infrastructure Development Project Indonesia EAP 1998 2004 Closed T&ICT 34.5 8.4 MS P007485 Private Investment & Export Development Jamaica LCR 1994 2001 Closed F&M 35 4.6 MS P147665 Foundations for Competitiveness and Growth Jamaica LCR 2015 n.a. Active T&C 50 4.5 n.a. P049721 Agricultural Competitiveness Project Kazakhstan ECA 2005 2012 Closed Ag 24 26.69 MS P085007 Micro, Small, and Medium Enterprise Competitiveness Project Kenya AFR 2005 2012 Closed F&M 22 5 S P049724 Agribusiness and Marketing Kyrgyz Republic ECA 2005 2013 Closed Ag 8.1 1.3 S Lao People’s Dem- P130512 Second Trade Development Facility Project EAP 2013 n.a. Active T&C 4 1.2 n.a. ocratic Republic P088544 Private Sector Competitiveness and Economic Diversification Lesotho AFR 2007 2013 Closed T&C 8.1 1 S FY FY Amount MG amount ($, Implied MG Project ID Project Name Country Region Approved Closed Status GP (USD, millions) millions)* Rating Second Private Sector Competitiveness and Economic Diversi- P144933 Lesotho AFR 2014 n.a. Active T&C 13.1 0.75 n.a. fication Project P001555 Private Sector Development & Capacity Building Madagascar AFR 1997 2003 Closed F&M 23.8 6.8 HS P113971 Integrated Growth Poles and Corridor Project 2 Madagascar AFR 2015 n.a. Active T&C 50 5 n.a. P083351 Integrated Growth Poles Madagascar AFR 2006 2015 Closed T&C 129.8 1 S P103773 Business Environment Strengthening Technical Assistance Malawi AFR 2007 2013 Closed F&M 15 1.6 MS P001747 Private Sector Assistance Mali AFR 1993 2002 Closed F&M 12 MS P080935 Growth Support Project Mali AFR 2005 2013 Closed T&C 55 3.66 NI P081704 Agricultural Competitiveness and Diversification Project Mali AFR 2006 2015 Closed Ag 46.4 9.9 HS P001918 Technical Assistance Project to Enhance Competitiveness Mauritius AFR 1994 1999 Closed F&M 7.7 2.68 HS Manufacturing and Services Development and Competitive- P112943 Mauritius AFR 2010 2013 Closed T&C 20 8 NI ness P048505 Agricultural Productivity Improvement Project Mexico LCR 1999 2003 Closed Ag 444.5 343 S P068290 E-Business for Small Business Development Project Mexico LCR 2004 2008 Closed F&M 58.4 16.4 n.a. P044531 Knowledge and Innovation Mexico LCR 1998 2006 Closed Ed 300 30 MU P089865 Innovation for Competitiveness 1st Phase APL Mexico LCR 2005 2011 Closed F&M 250 29.59 S P089124 Competitiveness Enhancement Project Moldova ECA 2006 2013 Closed F&M 9.8 1.2 S P144103 Second Competitiveness Enhancement Project Moldova ECA 2015 n.a. Active T&C 45 3 n.a. P049874 Enterprise Development Mozambique AFR 2000 2006 Closed F&M 26 12.2 S P106355 Competitiveness and Private Sector Development Mozambique AFR 2009 2016 Closed T&C 25 6.46 S P127303 Integrated Growth Poles Project Mozambique AFR 2013 n.a. Active T&C 100 24 n.a. P109691 Micro, Small and Medium Enterprise Development Nicaragua LCR 2008 2015 Closed T&C 20 6.2 S P127204 Competitiveness & Growth Support Niger AFR 2012 n.a. Active T&C 50 11 n.a. P095210 Agro-Pastoral Export and Market Development Project Niger AFR 2009 n.a. Active Ag 40 13 HS P083082 Micro, Small and Medium Enterprise Project Nigeria AFR 2004 2012 Closed F&M 32 12 S P103499 Growth & Employment Nigeria AFR 2011 n.a. Active T&C 160 28 n.a. P063622 Second National Fadama Development Project Nigeria AFR 2004 2010 Closed Ag 100 58.2 S P096572 Third National Fadama Development Project (Fadama III) Nigeria AFR 2009 n.a. Active Ag 250 186.07 P124268 Economic Revitalization of KP and FATA Pakistan SAR 2012 n.a. Closed T&C 20 14 S P064918 Rural Productivity Project Panama LCR 2007 2015 Closed Ag 39.4 19.8 P002262 Private Sector Development Rwanda AFR 1994 2001 Closed F&M 12 1.5 MS P002376 Private Sector Capacity Building Senegal AFR 1995 2002 Closed F&M 12.5 5 S 81 82 FY FY Amount MG amount ($, Implied MG Project ID Project Name Country Region Approved Closed Status GP (USD, millions) millions)* Rating P051609 Private Investment Promotion SIL (FY03) Senegal AFR 2003 2012 Closed F&M 46 3 U P083609 Agricultural Markets & Agribusiness Development Project Senegal AFR 2006 2015 Closed Ag 35 P126229 YF Innovation Serbia Serbia ECA 2012 2016 Closed T&C 10 3 S P096105 Rural and Private Sector Development Sierra Leone AFR 2007 2016 Closed Ag 30 12.1 P048606 Industrial Competitiveness & Job Creation South Africa AFR 1997 2005 Closed F&M 46 24.47 MS P113709 Sustainable Tourism Development Project Sri Lanka SAR 2010 2014 Closed T&C 18 4.6 U P110588 Revitalizing the Sudan Gum Arabic Production and Marketing Sudan AFR 2010 2013 Closed Ag 7 0.75 S P085009 Private Sector/MSME Competitiveness Tanzania AFR 2006 n.a. Active T&C 95 6.3 S P122326 Private Sector Development Support Project Togo AFR 2011 n.a. Closed T&C 13 1.75 n.a. Trinidad and P008100 Business Expansion and Industrial Restructuring Project LCR 1992 1999 Closed F&M 27 1.45 MS Tobago P055814 Export Development Tunisia MNA 1999 2005 Closed F&M 35 10 HS P071115 Export Development II Tunisia MNA 2004 2013 Closed F&M 36 22.55 S P132381 Third Export Development Project (EDP III) Tunisia MNA 2014 n.a. Active T&C 50 22 n.a. P035634 Private Sector Competitiveness Uganda AFR 1996 2003 Closed F&M 12.3 3 HS P083809 Private Sector Competitiveness II Uganda AFR 2005 2013 Closed F&M 70 4.4 HS P130471 Competitiveness and Enterprise Development Project Uganda AFR 2013 n.a. Active T&C 100 7 n.a. P121643 Inclusive Innovation Project Vietnam EAP 2013 n.a. Active T&C 55 33 MU P108885 Agriculture Competitiveness Project Vietnam EAP 2009 2014 Closed Ag 59.8 10.6 MS West Bank and P110526 Fund for New Market Development MNA 2008 2011 Closed F&M 1.2 1.2 MS Gaza Yemen, Repub- P143715 SME Revitalization and Employment Pilot Project MNA 2013 2016 Closed T&C 3 2.43 MS lic of P044324 Enterprise Development Zambia AFR 1997 2003 Closed F&M 45 2.5 U P071407 Support for Economic Expansion and Diversification (SEED) Zambia AFR 2005 2012 Closed F&M 28.2 2.127 U P070063 Agricultural Development Support Program Zambia AFR 2006 2014 Closed Ag 37.2 3 S P102459 Irrigation Development and Support Project Zambia AFR 2011 n.a. Active Ag 115 15.46 P035628 Enterprise Development Zimbabwe AFR 1996 2003 Closed F&M 70 5.6 U Note: * At approval, excluding any additional financing or restructuring; AFR = Africa; EAP = East Asia Pacific; ECA = Europe and Central Asia; LCR = Latin America and the Caribbean; MNA = Middle East and North Africa; SAR = South Asia Region; HS = Highly satisfactory; S = Satisfactory; MS = Moderately satisfactory; MU = Moderately unsatisfactory; U = Unsatisfactory; NI = Not implemented; n.a. = Not applicable; Ag = Agriculture; Ed = Education; F&M = Finance and Markets; SPL = Social Protection and Labor; SURR = Social, Urban, Rural, and Resilience; T&C = Trade and Competitiveness; and T&ICT = Trans- port and ICT. Annex 3: Correlation between projects’ design features and outcomes The following model is to be estimated: MG development outcome = a + bX + iZ + e where X is the vector of country-level variables, Z is a vector of project-level variables, and e is the random error term, normally distributed. The development outcome is measured by implied ICR outcome rating and ranges from 1 (not imple- mented) to 6 (highly satisfactory). There are no highly unsatisfactory projects in the sample. Ordinary least squares regression is used for the estimation. The means, minimum, maximum, and standard deviations of each of the variables are presented in table A3.1. Table A3.2 presents the results. In cases where running the estimation with the CPIA score and without the CPIA score made a difference to the significance, both cases are presented. In cases where it did not, the case displayed is when CPIA is controlled for. Table A3.1: Means and standard deviations of variables in the analysis Standard Variables Mean Deviation Minimum Maximum Outcome 4.18 1.31 1 6 Country GDP per capita (current U.S. dollars) (in log) 6.93 1.04 4.83 8.97 Index of Economic Freedom (out of 100) 56.7 6.6 38.3 76.3 Doing Business (distance to frontier, out of 100) 53.3 10.2 30.2 73.4 CPIA (scale 1–6) 3.4 0.45 1.8 4.3 Agriculture, value added (percentage of GDP) 20.8 12.8 2.6 54.8 Design feature (for each binary variable yes=1) PIU 0.22 0.42 0 1 Size limit 0.53 0.50 0 1 Groups 0.85 0.35 0 1 Service providers 0.47 0.50 0 1 Sector restriction 0.62 0.49 0 1 Equipment 0.39 0.49 0 1 Level of match 50.59 9.65 30 80 Reimbursement 0.66 0.48 0 1 Diagnostic 0.66 0.48 0 1 83 Table A3.2: Regression analysis results 84 Predictor Models PIU (CPIA Service Reimbursement Diagnostic PIU excluded) Size Limit Groups Providers Sector# Equipment# Level of Match Reimbursement (CPIA excluded) Diagnostic (CPIA excluded) Constant 3.362 1.471 0.944 -0.437 4.253 2.23 -1.209 -2.025 2.657 4.656 -0.782 0.598 Country GDP per capita (current U.S. dollars) -0.368 0.290 -0.046 -0.069 -0.035 -0.173 0.048 -0.227 -0.166 -0.081 0.101 0.369 (in log) Index of Economic Freedom (out of 0.050 0.023 0.043 0.060 -0.008 0.041 0.068 0.060 0.079* 0.061* 0.036 0.018 100) CPIA (scale 1–6) 0.361 --- 0.415 0.405 0.607 0.254 0.209 0.443 0.192 --- 0.600 --- Agriculture, value added (% of GDP) 0.010 0.029 0.017 0.034 -0.002 0.010 0.045 0.032 0.015 0.013 0.019 0.030 Doing Business (distance to frontier, -0.026 -0.024 -0.017 -0.010 -0.039 -0.001 -0.016 -0.026 -0.046 -0.058** -0.033 -0.031 out of 100) Design feature PIU 1.042** 0.338 --- --- --- --- --- --- --- --- --- --- Size limit --- --- 0.041 --- --- --- --- --- --- --- --- --- Groups --- --- --- -0.428 --- --- --- --- --- --- --- --- Service providers --- --- --- --- 0.155 --- --- --- --- --- --- --- Sector --- --- --- --- --- -0.327 --- --- --- --- --- --- Equipment --- --- --- --- --- --- -0.007 --- --- --- --- --- Level of match --- --- --- --- --- --- --- 0.005 --- --- --- --- Reimbursement --- --- --- --- --- --- --- --- -0.534 -0.809* --- --- Diagnostic --- --- --- --- --- --- --- --- --- --- 0.852** 0.604* R squared 0.178 0.049 0.087 0.139 0.055 0.09 0.20 0.15 0.25 0.22 0.18 0.09 Number of observations 47 68 41 38 31 37 32 31 35 49 47 68 Note: # 22 agriculture projects have been excluded; *** p < 0.01; ** p < 0.05; and * p < 0. 1 See Phillips (2001) for further discussion and references on experiences with voucher schemes. The study argues that vouchers can yield positive results but that they can require costly management arrangements to provide successful coordination, guard against fraud, and ensure quality service provision. 2 As elsewhere in this section, when analyzing outcome distribution, the total number of observations will be presented for each feature, because, owing to missing data, not all features will have information available for all 73 projects with ratings. 3 Cadot et al. (2015) suggest that the impact of the EMAF II program may have been transient because it did not lead to the enhancements in product quality or sophistication, which could have strengthened com- petitiveness durably. Notwithstanding its transient effect, the program is deemed to have been cost-effective because it generated two Tunisian dinars of private profits per Tunisian dinar of program expenditure, and the additional corporate tax revenue just covered the public cost of the program. 85 Funding for the report and companion papers was provided by CIIP Competitive Industries and Innovation Program Financed by in partnership with www.theciip.org