Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Resilience, Inclusion and Growth(P162071) Report Number : ICRR0021904 1. Project Data Operation ID Operation Name P162071 Resilience, Inclusion and Growth Country Practice Area(Lead) Turkey Macroeconomics, Trade and Investment L/C/TF Number(s) Closing Date (Original) Total Financing (USD) IBRD-87810 29-Aug-2018 408,605,505.00 Bank Approval Date Closing Date (Actual) 29-Aug-2017 29-Aug-2018 IBRD/IDA (USD) Co-financing (USD) Original Commitment 400,000,000.00 0.00 Revised Commitment 400,000,000.00 0.00 Actual 408,605,505.00 0.00 Prepared by Reviewed by ICR Review Coordinator Group Hjalte S. A. Sederlof Judyth L. Twigg Malathi S. Jayawickrama IEGEC (Unit 1) 2. Project Objectives and Policy Areas a. Objectives The Program Development Objective (PDO) of the Resilience, Inclusion and Growth Development Policy Financing (RIG-DPL) was to increase domestic savings and enhance fiscal transparency, support the economic inclusion of vulnerable groups, and reduce structural bottlenecks to sustainable growth (Program Document, PD, page 1). While Program documentation, including the ICR, is based on three objectives, combining “domestic savings” and “fiscal transparency” as one objective, IEG notes that the Program in fact had four objectives: (i) to increase domestic savings; (ii) to enhance fiscal transparency; (iii) to support economic inclusion of vulnerable Page 1 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Resilience, Inclusion and Growth(P162071) groups; and (iv) to reduce structural bottlenecks to sustainable growth. Efficacy will be assessed on the basis of these objectives. b. Pillars/Policy Areas The PDO indicates three pillars for the operation. However, the analysis will draw on four pillars, separating the first pillar into two pillars. The four pillars of the operation are as follows: Pillar 1: increasing domestic savings. This was to be achieved by introducing automatic enrollment into private pension schemes against the likelihood of a less generous public pension scheme in the future. Pillar 2: enhancing fiscal transparency. This was to be achieved by increasing the amount of fiscal information available in audit reports to enhance accountability and fiscal transparency in public financial management. Pillar 3: supporting the economic inclusion of vulnerable groups. This was to be achieved by expanding the rules for flexible employment, increasing access to childcare to facilitate women's entry into work, and facilitating the integration of foreigners into the labor market on a temporary basis. Pillar 4: addressing structural bottlenecks to sustainable growth. This was to be achieved by legislation encouraging innovation; facilitating access to credit for small and medium enterprises (SME); deregulation of network industries by liberalizing the railways sector; and supporting renewable energy investments. c. Comments on Program Cost, Financing, and Dates Program cost and financing. The commitment under RIG-DPL was US$400 million equivalent. US$408.61 million was disbursed, the difference between the two numbers reflecting exchange rate changes. Dates. The RIG-DPL was approved on August 29, 2017, became effective on October 19, 2017, and closed on August 29, 2018. 3. Relevance of Objectives & Design a. Relevance of Objectives The RIG-DPL’s objectives were relevant to the country situation: decelerating economic growth challenged by domestic and external economic conditions, and long-standing structural rigidities that are threatening gains in poverty reduction and shared prosperity. The objectives were fully consistent with Turkey’s 10th development plan and are partly consistent with its new 11th development plan (2019-2023). The main inconsistencies in the latter relate to refugee employment and financial audits for all general budget Page 2 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Resilience, Inclusion and Growth(P162071) institutions, which no longer are listed as explicit priorities. The objectives were consistent with an evolving Country Partnership Framework (CPF) for the 2017-2021 period. The operation covered all three of the CPF's focus areas: in growth, with its emphasis on containing fiscal risk; in inclusion, with increased labor force participation of women and vulnerable groups; and in sustainability, with generation of renewable energy. Rating Substantial b. Relevance of Design There was a causal link between the actions supported by the operation and expected outcomes for all three objectives. Under Pillar 1, private retirement savings accounts were to increase domestic savings and reduce dependence on external financing; and audited financial statements for all general budget institutions were to enhance transparency and encourage fiscal discipline. Under Pillar 2, economic inclusion of vulnerable groups was to be increased by facilitating their entry into the job market through more flexible work arrangements; for women through easier access to childcare; and for external refugees through temporary work permits. Under pillar 3, some structural bottlenecks to long-term growth were to be addressed through supply-side reforms that were expected to increase private sector productivity and investment (through easier access to credit, competition in the transport sector), and with an emphasis on environmental sustainability. The results chain and the link between the three pillars and the nine prior actions were straightforward. Intended outcomes as measured by the results indicators were monitorable. The macroeconomic framework was deemed adequate for the operation. The economy had proven resilient to crisis owing to strong fiscal discipline and a well-capitalized banking system. Still, continued domestic and external imbalances made the country dependent on external financing, creating risks to growth. The IMF’s Stand-by Arrangement for Turkey was successfully completed in 2008, and the latest Article IV consultation report published in 2017. The report emphasized the need to move forward on the policy reform agenda, notably through policies aimed at increasing domestic savings and raising potential growth. Rating Substantial 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Page 3 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Resilience, Inclusion and Growth(P162071) Objective To increase domestic savings Rationale The law on private pension savings was amended as a prior action to introduce auto-enrollment into the private pension system with an opt-out option. The scheme was introduced in 2017 as a means of creating a sustainable and diversified pension system that would shield participants from any future reductions in the generosity of the current public pension pillar that might be required by any future deterioration in the fiscal situation. As of January 2019, all employees of companies with more than five employees had become part of the system, which had more than 5 million participants, compared to a baseline of zero and a target of 2 million. However, there has subsequently been a fall-off, as participants have opted out; according to the ICR, around two-thirds of the new enrollees do so. This would indicate that the impact of the voluntary pension scheme on domestic savings in the short- to medium term may be less than anticipated. Summary: While the introduction of the private pension scheme initially appears to have exceeded expectations, this may have been driven initially by the automatic enrollment of all eligible employees as participants, judging from the subsequent fall-off. What will happen over the longer term is a function of the credibility that the scheme managers succeed in building. Rating Modest PHREVDELTBL PHEFFICACYTBL Objective 2 Objective To enhance fiscal transparency Rationale To promote transparency in public spending and increase the accountability of public institutions (i.e. encourage fiscal discipline), the audited financial statements for 2015 of all general budget institutions were submitted to Parliament in October 2016 as a prior action. As a consequence, individual financial statements now are published annually, not only for all general budget institutions (45 in 2018), but for a large number of other public institutions. Summary: The fiscal transparency initiative did meet its immediate objective of transparency; it is still too early to determine its longer-term impact on fiscal discipline. Page 4 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Resilience, Inclusion and Growth(P162071) Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 3 Objective To address structural bottlenecks to sustainable growth Rationale Amendments to labor market legislation were expected to facilitate labor market participation of women, youth, long-term unemployed, and foreign refugees (Syrians) under temporary protection, by supporting increased access to childcare (women), flexible employment (youth and long-term unemployed), and integration of refugees into the labor market. As prior actions, the income tax law was amended to reduce the corporate tax paid by private nursery schools; labor legislation was amended to make contractual working arrangements more flexible; and work permits for foreigners under temporary protection (essentially Syrian workers) were introduced. The tax incentives for childcare are operational and have been strengthened by exempting employers from income tax. The prior action relating to flexible work contracts is not yet operational in the absence of accompanying bylaws on implementation arrangements. Action on these bylaws is included as part of the country's new development plan. Access to childcare increased as measured by net enrollment rates in childcare facilities, which rose from 35.5 percent in 2016 to 38.5 percent in 2018. At the same time, the number of private pre-schools increased by some 13 percent over the same period. Women’s labor force participation rose from 32.5 percent in 2016 to 34.2 percent in 2018, against an expected target of 34.9 percent. The increase in women’s labor force participation should not be seen only as due to increased access to childcare facilities, but is also likely to have been influenced by positive trends in the economy and the job market. A survey conducted by the National Statistics Office in 2018 found that only 13 percent of women who work use a formal childcare service. So far, the working arrangements for implementing flexi-work arrangements benefiting women (beyond the impact of childcare arrangements), youth, and long-term unemployed have not been put in place, and this particular action has yielded no results. Page 5 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Resilience, Inclusion and Growth(P162071) The introduction of work permits for recent refugees (essentially Syrians under temporary protection) had been extended to some 22,000 refugees in 2018, meeting the target. However, with some 430,000 refugees able to work, work permits cover only 5 percent of eligible refugees. In summary, the operation had a modest, if positive, effect on women’s labor force participation as a function of easier access to childcare services. Its efforts at flexi-work, not yet in force, had no effect on the intended beneficiaries of that program - women, youth and vulnerable groups. The arrangements for job creation for immediate refugees are likely to have had a minor impact on the potential beneficiaries. Rating Modest PHREVDELTBL PHEFFICACYTBL Objective 4 Objective To address structural bottlenecks to sustainable growth Rationale Legislation was introduced as prior actions, including: (i) modernizing the protection of intellectual property to encourage research and development and innovation; (ii) enabling the use of movable property as collateral for access to finance, especially for SMEs and startups; (iii) liberalizing railway transport by unbundling infrastructure and operations, supporting competition between inter-modal (road, rail) transport options; and (iv) introducing incentive mechanisms towards environmentally favorable production in the areas of energy efficiency and transport. Strengthening the legal framework for the patent system has contributed to a rise in patent applications from 6,450 in 2016 to 8,600 in 2017, exceeding the target of 8,500 set for 2018, but declining to 7,300 in 2018. According to the ICR, a more robust legal framework is likely to have contributed to the increase in patent applications, while the subsequent decline in patent applications was likely to have been influenced by factors such as lower perceived returns to patents. A stronger indicator of the impact of new legislation might be observing longer term trends, which was not possible within the timespan of the operation. The volume of credit supported by movable collateral increased from zero to Turkish Lira (TL) 50 billion in lira terms, with an additional US$8.2 billion and euro 1 billion in foreign currency lending. The ICR argues that all of it can be considered as an outcome of the reform. The Doing Business survey improved its assessment of ease of credit in Turkey from a 2 in 2017 to 7 in 2019. Page 6 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Resilience, Inclusion and Growth(P162071) With the liberalization of railway transport (the unbundling of the railway company), four private sector operators have signed framework contracts and have been licensed to use rolling stock on the national rail system. These services are now operating. The original target was at least one operator. The national railway company is considering infrastructure development to allow further private rail services. With regulations to support the establishment of renewable energy resource areas (zones), the first three areas are under development with the construction of new solar and wind farms expected to be completed in late 2019 or 2020. The estimated value of the projects, which were based on auctions, is US$3.4 billion. When on-stream, they are estimated to have energy capacity of up to 3 GW. The total installed renewable energy capacity that is being developed in these areas is 2,000 MW, which will be a significant addition to the existing renewable installed capacity of 12,000 MW (ICR, page 20). Rating Substantial PHREVDELTBL PHREVISEDTBL 5. Outcome • Relevance of objectives and design were both rated substantial, with minor shortcomings in both areas. Efficacy was rated modest for Objective 1, domestic savings, and Objective 3, economic inclusion; while Objective 2, fiscal transparency, and Objective 4, structural bottlenecks, were rated substantial. These ratings reflect moderate shortcomings in the operation overall, producing an outcome rating of moderately satisfactory. a. Outcome Rating Moderately Satisfactory 6. Rationale for Risk to Development Outcome Rating The reforms under the operation closely paralleled the government’s reform program, and the risk to the development outcomes should be modest. However, in some cases distinctions need to be made. Outcomes under Pillars 1 and 2 have so far consistently had government support, are policy priorities, and are therefore likely to be sustainable. Performance under Pillar 3, notably in terms of job creation for women and immigrant refugees, has not been strong, and a certain degree of political uncertainty attaches to job creation for refugees: it was low under the operation, and understandings between the European Union and Turkey on refugees Page 7 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Resilience, Inclusion and Growth(P162071) remain tenuous. Outcomes under Pillar 4 may have political backing, but in terms of credit, for instance, the volume of credit with movable collateral has risen beyond expectations and may well prove unsustainable in the medium term, leading to more stringent credit conditions for targeted borrowers. a. Risk to Development Outcome Rating Modest 7. Assessment of Bank Performance a. Quality-at-Entry The Bank’s intervention was strategically relevant, focusing on areas that were likely to reduce poverty and further shared prosperity destabilized by domestic and external events. According to the ICR, its prior actions built on extensive prior technical analysis of the policy areas supported under the operation, as well as reforms that had been started under earlier DPLs, notably the Sustaining Growth DPL. Lessons taken into account in the operation's design and signaled in the ICR included: the need for adequate institutional capacity; a basis in knowledge work and synergies with other Bank and stakeholder engagement in the country; and a good understanding of the links between policy and actions (as reflected in a results matrix where causality was clear). The approach, a single stand-alone DPO, was appropriate. It addressed an immediate need to improve fiscal balance, and it responded to political volatility at the time, as the country came out of a recent (failed) coup attempt. But it also introduced uncertainty regarding the longer-term sustainability of policy actions, albeit according to the ICR the government commitment to them was strong. While arrangements for M&E allowed project progress to be monitored, some of the operation’s interventions raised challenges of attribution (see Section 9a). Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision No formal supervision missions were undertaken during the implementation period. Still, dialogue between the Bank team and government was able to continue and progress was monitored despite delays due to organizational changes across government, including a high turnover of staff as the country transitioned to a presidential system. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 8. Assessment of Borrower Performance Page 8 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Resilience, Inclusion and Growth(P162071) a. Government Performance The government, through the Treasury, and later through the Ministry of Finance, oversaw preparation and implementation of the operation. The ICR notes that the government's previous experience with DPOs, and strong links between the contents of this operation and the country's development strategy maintained momentum despite changes in government personnel without extensive disruptions. Still, untimely fiscal and monetary policies and personnel changes did influence some of the intended outcomes of the reforms, as noted in Section 4. Government Performance Rating Moderately Satisfactory b. Implementing Agency Performance Not assessed. Implementing Agency Performance Rating Not Rated Overall Borrower Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The policy and results matrix for the operation provided a sequence of prior actions that underpinned the policy changes. Key indicators for systematic tracking of outcomes were identified in the matrix, including specifications of baselines and targets. In most instances, the indicators were sufficient to measure results, albeit in some cases results were too broadly formulated to allow unique attribution (for instance, aggregate labor market outcomes); and in other cases, meaningful results were hard to attain over the short period of time – a year – that the operation covered. b. M&E Implementation The Treasury was responsible for coordinating data gathering and monitoring of the operation, notably results indicators, drawing on information from relevant ministries and agencies. c. M&E Utilization Page 9 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Resilience, Inclusion and Growth(P162071) The Treasury prepared quarterly evaluation reports and presented semi-annual progress reports to the government. M&E Quality Rating Modest 10. Other Issues a. Environmental and Social Effects No safeguard policies were noted in the ICR. b. Fiduciary Compliance No fiduciary concerns were noted in the ICR. c. Unintended impacts (Positive or Negative) None noted. d. Other The operation helped promote the economic inclusion of women into the job market, albeit with modest results. It also supported the burgeoning refugee crisis by offering work permits for foreigners under temporary protection, again with relatively narrow coverage. 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory --- Satisfactory Risk to Development Modest Modest --- Outcome Moderately Bank Performance Moderately Satisfactory --- Satisfactory Moderately Borrower Performance Moderately Satisfactory --- Satisfactory Page 10 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Resilience, Inclusion and Growth(P162071) Quality of ICR Substantial --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 12. Lessons The following lessons are drawn from the ICR: DPOs can be an effective means to support structural change when they are part of the planning process. The operation was able to help initiate significant reforms over a wide range of subject areas despite a volatile political environment and an uncertain macroeconomy. Program design should recognize macroeconomic risk. Major changes in the political and institutional framework will test a program's strength, and a program should be designed with resilience to such factors in mind: for instance, linking the results framework closely to precise results of prior actions rather than broad aggregates, and assessing trade-offs between shorter and longer time horizons to allow outcomes to be more rigorously assessed. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provided a good foundation for assessing performance on this operation. The review drew a careful balance between the short-term effects of the policies being introduced and their often medium-term results perspective. A more nuanced discussion of the usefulness of an M&E approach that is based on short-term results could have been a thought-provoking addition to the ICR, albeit perhaps a bit outside the strict framework of a core ICR. a. Quality of ICR Rating Substantial Page 11 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Resilience, Inclusion and Growth(P162071) Page 12 of 12