Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sustaining Reforms for Inclusive Growth(P153269) Report Number : ICRR0021076 1. Project Data Operation ID Operation Name P153269 Sustaining Reforms for Inclusive Growth Country Practice Area(Lead) Seychelles Macroeconomics, Trade and Investment L/C/TF Number(s) Closing Date (Original) Total Financing (USD) IBRD-85430 31-Dec-2016 5,000,000.00 Bank Approval Date Closing Date (Actual) 23-Oct-2015 31-Dec-2016 IBRD/IDA (USD) Co-financing (USD) Original Commitment 5,000,000.00 0.00 Revised Commitment 5,000,000.00 0.00 Actual 5,000,000.00 0.00 Prepared by Reviewed by ICR Review Coordinator Group Malathi S. Robert Mark Lacey Malathi S. Jayawickrama IEGEC (Unit 1) Jayawickrama 2. Project Objectives and Policy Areas a. Objectives According to the Program Document (PD), the objectives of this stand-alone Development Policy Operation (DPO) were to raise efficiency of public expenditure and improve the business environment (PD, p. v). b. Were the program objectives/key associated outcome targets revised during implementation of the series? Page 1 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sustaining Reforms for Inclusive Growth(P153269) --- c. Pillars/Policy Areas There were three pillars: 1. Raise efficiency of public expenditure. The operation covered actions to: improve the coherence of social assistance programs by controlling costs and increasing efficiency. The operation also assisted in adopting a National Health Policy to protect the most vulnerable, and to develop a comprehensive teacher management policy. These actions were expected to reduce the fragmentation of the social assistance system (SAS) legislature and improve the Agency for Social Protection’s (ASP’s) capacity to monitor the administered programs, develop a more effective health strategy, and improve the incentives and retention rates of teachers. 2. Facilitate access to credit. The operation supported actions to assist the Central Bank of Seychelles (CBS) to enhance the frequency of reporting to the Credit Information System (CIS), and pave the way for a modern secured transaction system. The reforms were expected to enhance overall reporting, and create a centralized electronic collateral registry. 3. Improve Public Investment. The operation covered actions to improve public investment management (PIM) systems, and government supervision of commercial State-owned Enterprises (SOEs). These measures included establishing a committee for independent review and selection of public investment projects, and approving guidelines for such projects. With respect to SOEs, the operation supported actions to clarify the mandate of the Societé Seychellois d’investissement (SSI) -- the Seychelles Investment Society, and improve its governance. d. Comments on Program Cost, Financing, and Dates The IBRD loan of US$5 million was approved on October 23, 2015. It became effective on January 29, 2016, and closed on schedule on December 31, 2016 after full disbursement. 3. Relevance of Objectives & Design a. Relevance of Objectives Both objectives were highly relevant to the country context. With respect to the first objective, Seychelles had a comprehensive social welfare program, which absorbed 4.3% of GDP in 2015; however, it was skewed, with the elderly accounting for 79% of total social protection spending. The program’s legislation also remained scattered. Seychelles expenditure on social assistance was higher than average for both high and upper middle-income countries and higher than in most other small island states at similar income levels (Systematic Country Diagnostic, June 2017). Poverty targeted Social Welfare Assistance (SWA) received a mere 0.3% of GDP. Seychelles faced a challenge in terms of income disparity and the well-being of the Page 2 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sustaining Reforms for Inclusive Growth(P153269) bottom 40 percent of the population. In the health sector, the country faced new pressures, given the aging population and the growing non-communicable diseases. In education, previous reforms had not been able to address the high teacher attrition rates and other sources of inefficiency. Seychelles also needed to improve the business environment to enhance private sector growth (PDO 2). Facilitating access to credit was expected to positively affect small and medium enterprises (SMEs), where most of the poor were employed. The two objectives were relevant to the Government's Medium-term National Development Strategy 2015- 2019, which focuses on accelerating inclusive economic growth. Within the strategy, the operation supported the key areas of: governance (public sector reforms and public finance); economic development (SMEs, poverty, and inequality); and social development (education, health and social protection and family preservation. (PD, p. 10). The objectives were relevant to the World Bank Group's strategy for the Seychelles, as outlined in the Country Partnership Strategy (CPS) for FY2012-2015 (the latest strategy document available), which emphasizes competitiveness and employment, vulnerability and resilience, and governance and public-sector capacity. The Bank's Systematic Country Diagnostic (SCD), completed in June 2017, identifies continued engagement to improve the efficiency of public spending (including through SOEs), and addressing credit constraints as priorities for shared prosperity. Rating High b. Relevance of Design The operation’s internal logic was mostly relevant to achieving the objectives. For example, to raise the efficiency of public expenditure, the first two prior actions clarified the ASP’s multiple reporting lines and responsibilities, and approved a strategic plan and an action plan for the Agency. These latter addressed the gaps and overlaps in the legislative framework, which made implementation difficult without subjective interpretation. The second prior action broadened ASP’s Management Information System (MIS), and aimed to improve the control of sickness benefits and social assistance. Given the aging population and disease dynamics, the third prior action of approving a new National Health Policy aimed to increase focus on preventive services. In education, the prior action aimed to stem the increasing attrition rates among teachers, which was affecting the quality of education. To improve public investment, the prior actions increased formal scrutiny of larger projects and adopted guidelines for public investment. Under the second objective, the prior actions addressed access to credit, and focused on the reporting capability of the CIS, to pave the way for a modern secured transactions system. This was to enable banks and other creditors to use a variety of collateral and eventually provide more loans to the private sector. It is unclear why improving public investment was separated from improving the efficiency of public expenditure, of which it would normally be an integral part. Page 3 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sustaining Reforms for Inclusive Growth(P153269) Seychelles' macroeconomic program was on track at the time of appraisal. The robust recovery from the 2008 balance of payment and debt crisis had resulted in improved economic and social outcomes (IMF Article IV Consultation, July 2015). On the structural agenda, Article IV discussions focused on fostering inclusiveness and private sector-led growth, while improving economic governance and the focus of SOEs. A stand-alone DPL with a simple design was an appropriate instrument, given the good progress made under the previous three-series DPLs, and the recognition of potential risks related to the upcoming elections. Rating Substantial 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective Raise efficiency of public expenditure Rationale 1. Raise Public Expenditure Efficiency in the Social Sectors There were three prior actions related to this sub-objective. Improving Monitoring and Administration of Social Assistance System (SAS). First, the Cabinet approved a strategy and action plan for ASP. Second, the Government expanded the coverage of the ASP’s MIS to include data on applicants and beneficiaries of its sick leave benefits program. As a result of these actions, the directions for central social assistance interventions were to be more clearly defined and a cohesive approach would replace previous piecemeal solutions. Following the action plan, the DPO supported measures to enhance ASP’s limited monitoring and evaluation (M&E) capacity, which has improved control of social assistance and enhanced policy-making procedures. It also assisted in the establishment of an automated ASP database in processes such as application procedures and payment systems. The ASP now possesses an automated Social Welfare Information System, which, according to the ICR, has reduced the likelihood of fraud and error. Develop a Health Strategy to sustain health services to the population. Third, the Cabinet approved and implemented a National Health Policy. The focus of policy shifted to prevention and wellness services. The government adopted a package of interventions that focus on the most effective interventions, including addressing the disease burden and emphasizing preventive care. The package and related resource allocation can be adapted to evolving diseases and new technologies. The management of inputs (training, medicines etc.) has been streamlined. There were three outcome targets to measure attainment of this sub-objective. The first was to reduce spending on social assistance as a percentage of GDP from a baseline of 3.2% (2014) to 3% in 2016. This Page 4 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sustaining Reforms for Inclusive Growth(P153269) was not met. Social assistance increased to 5.2% of GDP in 2016. The measures were taken in early 2016, and reversed the progress made in cutting social assistance costs. The single largest contributor was the 40% increase in retirement benefits and other benefits linked to the minimum wage, which increased by about 25%. Retirement benefit costs increased by 0.9% of GDP to 2.3%. Invalidity benefits and allowance rates for home care also rose to balance the minimum wage. Each of these costs rose by about an extra 0.3 percentage points of GDP. Political factors related to the electoral cycle undoubtedly contributed to these developments, which the ICR (p. 27) considers a one-off phenomenon. The second target was to reduce spending on sickness benefits from 0.07% of GDP in 2014 to 0.06% of GDP in 2016. This target was exceeded – sickness benefit payments dropped by 48% over the two-year period to 0.03% of GDP in 2016, due, in large measure, to the expansion of APS’s automated MIS to cover sickness benefits, a measure supported by the program. The third target was an increase in the percentage of the national health budget allocated to prevention and promotion services from 6% in 2014 to 9% in 2016. This was also exceeded – the percentage rose to 12.5%, reflecting the revised focus of health policy supported by the program. Improve Teacher Management. Seychelles faced key challenges related to the quality of education. Although teacher salaries amounted to over 90 percent of overall education spending, high attrition rates and low teaching performance reduced returns. Strengthening teacher capacity and increasing retention rates were key priorities in the Government’s Medium-Term Education Strategy for 2013 to 2017. The single prior action for this sub-objective was Cabinet approval of a memorandum for a comprehensive teacher management policy reform that focused on teacher distribution, training, career, and performance. The memorandum reflected views of teachers, teacher trainers, school directors, Ministry of Education (MoE) staff, and other stakeholders. It has guided policy reforms and helped MoE to develop a wide-ranging teacher management policy that links teacher management to the revision of teachers’ schemes. The related outcome target was a fall in teacher attrition rates from 11% in 2014 to 7 percent in 2016. The actual level in 2016 was 7.8% slightly short of the target. 2. Improve Public Investment Management There were two sets of prior actions supporting reforms in this area. First, a committee was established for independent review and selection of public investment projects, the relevant staff were recruited for the committee’s operation, and the Cabinet approved detailed guidelines for preparation and appraisal of public investment projects. Particularly large projects (the threshold is not specified in the ICR) would be subject to a full cost-benefit analysis. Second, the Government revised the mandate of its holding company for SOEs, the SSI. The objective of the revision was to define clearly the unique objectives and activities of the SSI in relation to similar government-owned institutions in line with good international practice on corporate governance of SOEs. The public investment process had been characterized by weak coordination between the institutions involved and lengthy procurement procedures. The related outcome target that all public investment projects over 10 million rupees (about US$725,000 equivalent) included in the 2017 National Budget would have been screened and approved by the committee. This target was met. However, while this reform does Page 5 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sustaining Reforms for Inclusive Growth(P153269) ensure that all major project that are budgeted have been screened, its impact on the quality of public investment projects is likely to be limited. Lengthy and cumbersome procurement procedures and weak contract management are untouched by the reform. Enhancement of SSI’s capacity to monitor and manage SOE investments was expected to lead to greater SOE efficiency, improvements in the services they delivered, a lower fiscal burden represented by SOES, and a higher return accruing to the Government’s investments in them. The holding company’s revised mandate includes a mission statement taking account of the overall macroeconomic situation. It has been endowed with a stronger Board of Directors selected through a more transparent process. According to the ICR (p. 25), the Government also plans to strengthen SSI’s mandate through an investment strategy, operating guidelines and a shareholder policy. Each SOE in the holding company’s portfolio was to sign a performance contract containing both financial and non-financial targets. There was one outcome target related to this area: to increase the number of SOEs under SSI’s purview that have attained their financial and non-financial performance indicators as agreed between the SSI, the Public Enterprise Monitoring Committee and the SOE concerned from a baseline of zero to two thirds of the total number of SOEs in the portfolio by the end of 2016. This target was not achieved. The ICR (p. 8) reports that no SOEs had reached their targets. While statements of corporate intent were produced, they did not include quantitative goals and were not subsequently renewed. In summary, results under this objective were mixed. In healthcare, there were two positive outcomes, which led to greater public expenditure efficiency – sickness benefits were brought under control and fell as a percentage of the health budget; and the proportion of health expenditures devoted to prevention and promotion rose sharply, well in excess of the target. In education, results fell very slightly short of the target. Overall social assistance spending as a percentage of GDP rose instead of falling as intended, largely due to political factors, which the ICR considers to be one-off phenomena. On public investment and of its management, the Development Committee that was created approves all screening criteria for development projects. Particularly large investment projects exceeding 10 million rupees now require comprehensive cost- benefit analysis. The goal of enhancing the effectiveness of the SOE holding company through targeted performance agreements with those SOEs in its portfolio was not reached. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 2 Objective Improve the Business Environment Rationale Under this objective, the program focused on facilitating access to credit. Although the CBS had launched a Page 6 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sustaining Reforms for Inclusive Growth(P153269) CIS in 2012, services had not developed beyond standard credit reports, which were insufficient for assessing credit risk. Moreover, the CIS did not show specific repayment behavior; but only indicated whether a credit was in arrears. There were two prior actions in this area, which aimed to improve the frequency and quality of credit information, as well as making lending more secure through the creation of a collateral registry. First, the Central Bank promulgated regulations in an attempt to ensure that credit data are reported on an event basis (that is, each time there is a change in the credit rather than monthly as before), and that lenders are given access to information reflecting borrowers’ credit-worthiness. In addition, the information system would be extended to include financial institutions not regulated by the CBS (including government agencies, commercial companies and others who engage in hire-purchase activities and financial leasing). Second, the Cabinet approved a draft Secure Transaction Bill for submission to the National Assembly. This latter action was accompanied by program-supported institutional and capacity strengthening to operationalize a collateral registry to which an increased number of lenders would have automated access. There were two outcome targets: an increase in the number of credit reports issued per day from 120 to 250; and an increase to 15 of the number of lenders linked to the collateral register from a baseline of zero. Consultations with the CBS during implementation showed that the baseline of 120 for the first target was inaccurate. The actual baseline in 2014 was 48 for the number of daily average reports. This rose to 56 in 2016. The inaccuracy suggests that the target was of limited operational usefulness for the Central Bank. It is also conceptually questionable, since it is unclear what the desirable number of daily credit reports ought to be in view of the economy’s size, banks’ decision-making processes and the impact of economic changes. The second target was not met. The centralized electronic collateral registry was incomplete at program closure. Consequently, no lenders were connected to it. The ICR attributes the delay to capacity constraints in the Central Bank, technical issues related to information and communications technology, and lack of commitment by stakeholders. Rating Negligible PHREVDELTBL PHREVISEDTBL 5. Outcome The program’s objectives were highly relevant to the country context, the World Bank Group’s strategy and the Government’s Medium-term National Development Strategy 2015-2019. The design is rated substantial. Although public expenditure efficiency in the social sectors was enhanced, achievements fell short of lowering social assistance spending as a percentage of GDP, and in establishing performance targets by SOEs. Regarding the second objective, one outcome target turned out to be inaccurate and conceptually flawed, while the other was not met by program closure. It is, therefore, difficult to see how the measures supported by the program could have had a noticeable impact on improving the business environment, and Page 7 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sustaining Reforms for Inclusive Growth(P153269) efficacy is rated negligible. Overall, the operation’s outcome is assessed as moderately unsatisfactory. a. Outcome Rating Moderately Unsatisfactory 6. Rationale for Risk to Development Outcome Rating Despite good progress made under Seychelles’ Extended Fund Facility program with the IMF, there are moderate macroeconomic risks stemming from the dependence on foreign tourism and direct investment, and on growth in the euro zone. The political environment has, since 2016, become more volatile, so that there is now a notable risk that political and social opposition may lead to the reversal of some reforms. Political cycle issues already led to backtracking under the first objective, i.e. change in social assistance, and lack of progress on SOEs. a. Risk to Development Outcome Rating Substantial 7. Assessment of Bank Performance a. Quality-at-Entry The program was aligned with identified government priorities at the time of preparation. There were regular meetings between the Bank, the IMF and the European Commission during preparation to share knowledge. The operation drew on analytical work by the Bank and the Government that covered all eight prior action areas. The work included Policy Notes on Social Protection (2015), which identified gaps in M&E and social assistance regulation, a Strategic Analysis of the ASP (2014), which identified the challenges in improving the effectiveness of social assistance, a Health Task Force report, a Public Expenditure Review (PER), and Policy Notes on Health, education (2014), and Strengthening Public Investment Management. The Bank also looked at administrative barriers to private sector development in the Seychelles, and prepared a Review of SOE holding and Investment Companies worldwide, to identify good practices for specific recommendations on revising the SSI mandate. The team incorporated lessons from the previous DPO series, especially the importance of knowledge and technical assistance to increase institutional capacity and support reforms, and the need for joint missions and close coordination between donors. While the team recognized the potential risks to sustaining reforms related to the 2015-2016 election cycle, the expectation that this would be mitigated by sustained Government’s commitment to the reform program this turned out to be optimistic. The risk of a major reversal of program goals with regard to social assistance expenditures in relation to GDP, which materialized, might have been mitigated more effectively had there been a thorough political economy analysis. There were also important deficiencies in M&E design, including the choice of a conceptually weak indicator, with a statistically inaccurate baseline for assessing the extent and quality of credit reporting. Seychelles fiduciary framework was strong (based on the PEFA report, 2011, and the implementation of the Public Financial Management Action Plan, 2012-2014), and hence, the Page 8 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sustaining Reforms for Inclusive Growth(P153269) fiduciary risk was assessed as low. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision The ICR reports that there were regular visits by staff and technical specialists to advise the government on all policy areas, and to maintain the policy dialogue with the institutions responsible for program implementation. Coordination between the Bank and the IMF remained strong, and the Government’s Extended Fund Facility program increased resilience and helped the country to cope with unexpected shocks. Staff from the Bank and the IMF participated in joint missions. There were, however, three significant shortcomings. First, lack of progress related to the second objective (access to credit) was not addressed in a timely manner. Second, the delays in implementing the mandate of the SSI were not dealt with. Third, the inaccuracy in the performance indicator for credit information was not replaced, and the delays in completing the collateral registry persisted. This was even though the Bank was providing technical assistance to the CBS as part of a Reimbursable Advisory Service (RAS) program. According to the task team, there seems to have been inadequate communication between the Central Bank and the RAS program on the one hand, and the operation’s supervisory team on the other. Quality of Supervision Rating Moderately Unsatisfactory Overall Bank Performance Rating Moderately Unsatisfactory 8. Assessment of Borrower Performance a. Government Performance During preparation and the early stages of implementation, government commitment to the reform program appeared strong. The Ministry of Finance, Trade and Economic Planning (MoFTEP), the core counterpart agency for the operation, focused on the design and early monitoring of the reform agenda. However, as noted in Section 4, implementation under the first objective of raising the efficiency of public expenditures suffered a significant reversal due to measures taken by the then President in early 2016, outside of the ordinary national budget process. This setback, as the ICR states, was a result of intense political competition ahead of legislative elections in September 2016, and affected the government's reform course. In addition, the Central Bank appears to have had inadequate capacity to implement its part of the reform program in a timely manner, despite receiving technical assistance. Government Performance Rating Moderately Unsatisfactory Page 9 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sustaining Reforms for Inclusive Growth(P153269) b. Implementing Agency Performance The Government implemented the program and there is no separate assessment of implementing agency performance. Implementing Agency Performance Rating --- Overall Borrower Performance Rating Moderately Unsatisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The design called for the Government and the Bank to review progress under the program twice a year, based on the results framework (PD, p, 30). There were eight outcome indicators, all of which had baselines and targets. The Ministry of Finance, was responsible for monitoring the program and providing information to the Bank. Two of the indicators had weaknesses. First, the baseline for number of credit reports issued per day was inaccurate. The appropriate interpretation of the indicator for operational purposes is also unclear (see Section 4 above). Second, the number of SOEs under SSI's purview that established and reached performance indicators combines process improvements (the development of performance targets and their agreement among interested parties) and outcomes (actual performance of SOEs as measured by the targets). Also, it says little about the size, market dominance and particular characteristics of individual SOEs that could significantly affect outcome. b. M&E Implementation M&E implementation was variable. Data on public expenditure in health and education was collected in a timely manner (it was part of the national budget-related tracking and planning process overseen by MoFTEP). However, information required for monitoring facilitation of access to credit was not regularly compiled or evaluated by the Central Bank. Similarly, M&E inputs were inadequate in the case of the results indicator on SOE performance, reflecting design weaknesses. c. M&E Utilization The ICR provides little information on the utilization of M&E beyond stating that information on public expenditure efficiency is highly relevant for policy decisions, especially during the budget cycle. This is to be expected given that information used during the budget cycle was the source of M&E inputs for the operation. Page 10 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sustaining Reforms for Inclusive Growth(P153269) M&E Quality Rating Modest 10. Other Issues a. Environmental and Social Effects Reforms under the DPL were not expected to have any negative effects on the environment and natural resources. Nor were they expected to have any direct distributional impact on the poor. They are, however, expected to have a favorable impact on poverty and social indicators, through raising the public expenditures in human development. b. Fiduciary Compliance The fiduciary framework was considered adequate to support the operation (PD, p. 29), and fiduciary risks were assessed as low. The ICR does not raise any fiduciary issues. c. Unintended impacts (Positive or Negative) None d. Other None 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Moderately Outcome --- Unsatisfactory Unsatisfactory There has already been backtracking on achievements Risk to Development Modest Substantial under the first objective. Outcome Curtailing social assistance and SOE reforms face risks. There were significant Moderately Moderately Bank Performance shortcomings in Quality of Satisfactory Unsatisfactory Supervision Page 11 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sustaining Reforms for Inclusive Growth(P153269) The increase in social spending in relation to GDP during the electoral cycle constitutes an important Moderately Moderately Borrower Performance reversal for the reform Satisfactory Unsatisfactory program. There were weaknesses in the performance of the Central Bank. Quality of ICR Substantial --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 12. Lessons The ICR offers both operation-specific and general lessons. IEG adds the following: • The effectiveness and sustainability of a reform program can be significantly affected by the electoral cycle, and it is important to take political events into consideration in both the design and the timing of the program. Especially in the context of political uncertainty, a careful political economy analysis, including in-depth consultations with the political opposition during preparation, can facilitate program implementation and sustainability. • Clear explanations of the reasoning behind the choice of policy areas, prior actions and outcome targets can reinforce the internal logic and coherence of the reform program supported by a DPO. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR is concise and provides a good discussion of the major factors affecting the operation, and overall achievements. The lessons are also well articulated. One shortcoming of the ICR is that it assesses achievements by pillar rather than by objectives. Page 12 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sustaining Reforms for Inclusive Growth(P153269) a. Quality of ICR Rating Substantial Page 13 of 13