Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NG Edo State FISDO(P151480) Report Number : ICRR0021248 1. Project Data Operation ID Operation Name P151480 NG Edo State FISDO Country Practice Area(Lead) Nigeria Macroeconomics, Trade and Investment L/C/TF Number(s) Closing Date (Original) Total Financing (USD) IDA-56260 30-Apr-2016 75,000,000.00 Bank Approval Date Closing Date (Actual) 29-Apr-2015 31-Dec-2016 IBRD/IDA (USD) Co-financing (USD) Original Commitment 75,000,000.00 0.00 Revised Commitment 75,000,000.00 0.00 Actual 74,396,673.00 0.00 Prepared by Reviewed by ICR Review Coordinator Group Hjalte S. A. Sederlof Robert Mark Lacey Malathi S. Jayawickrama IEGEC (Unit 1) 2. Project Objectives and Policy Areas a. Objectives The program development objective (PDO) of the Edo State Fiscal Improvement and Service Delivery DPO (ESFISDPO) as set out in the Program Document was to support Edo State in the implementation of critical policy and institutional reforms to improve the management of its public resources; increase private sector access to land for investment and improve the quality of education (Program Document, p. v). No statement of objectives was included in the Financing Agreement. Page 1 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NG Edo State FISDO(P151480) Achievement of the PDO will be assessed on the basis of three objectives: (i) improved management of public resources; (ii) increased private sector access to land for investment; and (iii) improved quality of education. b. Were the program objectives/key associated outcome targets revised during implementation of the series? --- c. Pillars/Policy Areas The ESFISDDPO supported four policy areas/pillars corresponding to areas of priority in Edo’s strategic plan: 1. Ensuring fiscal sustainability. To diversify revenue sources for the state government (heavily dependent on central government transfers that in turn depend on oil and gas revenues), the ability to generate internal revenues was to be strengthened by better identification, tracking, monitoring, and managing of taxpayers; and by stricter control of expenditure leakages. 2. Improving budget institutions and practices. To further pursue sound public financial management, budget institutions and practices would be strengthened, including public procurement procedures and procurement capacity, budget management processes, strategic allocation of state resources, and timeliness of state audit reports. 3. Improving access to land for investment. To facilitate more formal private sector growth, access to investment land, considered a focus area for improving the investment climate, was to be facilitated by a more reliable land information system, and a more secure land registration process. 4. Improving the quality of education. To strengthen vocational and technical education (TVET) and basic education – both seen as priorities for raising the quality of education – links between TVET and industry were to be improved, and more rigorous monitoring of teacher and student attendance at the basic education level introduced. d. Comments on Program Cost, Financing, and Dates Program cost and financing. An IDA Credit of US$75.0 million was provided, of which US$74.4 million was disbursed. Borrower contribution. None Dates. The ESFISDDPO was approved on April 29, 2015. It became effective on March 7, 2016, and closed on December 31, 2016, after one extension of the Closing Date from its original April 30, 2016. Effectiveness was delayed due to a political transition in Nigeria shortly after approval of the Credit by the Board and the installation of a new Federal Government. Page 2 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NG Edo State FISDO(P151480) 3. Relevance of Objectives & Design a. Relevance of Objectives The operation’s objectives and key policy areas were aligned with Edo State’s development strategy for the period 2010-2020, and with the World Bank Group’s Nigeria Country Partnership Strategy for 2014-2017. The areas covered contained policy and institutional reforms that were considered essential for better public sector management that could help the State cope with external economic shocks, for expanding the private sector and at the same time enlarging the public revenue base, and reducing the risk of youth unemployment through better basic and vocational education. The focus of the operation was also consistent with the Bank’s strategy for Nigeria, in particular its strategic cluster for social service delivery for social inclusion by strengthening training institutions; and the one for improving governance and public sector management. Rating High b. Relevance of Design The operation had an adequate results framework. The Program Document provided a clear rationale for the choice of policy actions and how they would contribute to the objectives. The theory of change and results chain were clear, and the prior actions could reasonably be expected to lead to the anticipated changes in outcomes. The outcome targets were appropriate for measuring achievement of objectives. Targets were set on the assumption of a two-operation programmatic series, and after the second operation had been cancelled (see below) the original outcome targets were nonetheless retained and are used in assessing the efficacy of the program. Nigeria's macroeconomic situation was broadly satisfactory at the time of preparation. The IMF Staff Report of the February 2015 Article IV Consultation concluded that inflation had remained in single digits for two years, and that the banking sector, which had a strong capital base, was expanding credit. The general government fiscal deficit and public debt were kept low. However, the trade surplus had been declining since the second quarter of 2013 due to lower oil exports and continued strong growth of imports; gross international reserves were falling. The macroeconomic framework deteriorated in 2015-2016, leading to cancellation of the second planned operation. Rating Substantial 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Page 3 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NG Edo State FISDO(P151480) Objective 1 Objective To improve management of public resources Rationale The objective was to be achieved by improving fiscal sustainability (policy area 1) and budget institutions and practices (policy area 2). Fiscal sustainability. Two indicators were to show improvements in fiscal sustainability: changes in the number of taxpayers captured in the state’s tax identification database, introduced as a prior action; and completion of a civil service payroll audit for the state. • By the end of the operation, a total of some 173,400 taxpayers were registered in the database, exceeding the program target of 120,000. Registration continued after project closing, and the ICR indicates that 193,000 taxpayers were registered by end-December 2017. • A full audit of the civil service payroll was completed in December, 2015, and quarterly revalidation audits were introduced in 2016. The first measure contributed to the net tax capture and therefore to improved fiscal sustainability by contributing to an increase in internally generated revenue (IGR). The share of IGR in the state’s revenue composition increased from 24 percent in 2015 to 32 percent in 2017. While changes in IGR can be influenced by external factors, in this instance, in a deteriorating economic climate, taxes mainly benefited from an increased number of taxpayers. The payroll audits had an immediate measurable, albeit modest, impact on fiscal sustainability, reducing spending on wages and salaries of ghost workers, as the number of ghost workers on the payroll was reduced by some 3 percent. Budget institutions and practices. Improvements in budget institutions and practices were to contribute to Objective 1 by strengthening multi-year budget planning, and budget execution and monitoring, and audit functions. Four indicators, underpinned by prior actions, were to measure progress: • Professionalization of the public procurement function (circular issued as prior action) was introduced. At project closure, 15 out of 20 line staff in the procurement agency had the necessary certification in public procurement, against a target of 6. These have subsequently been deployed to state offices to support procurement processes statewide. • An integrated electronic platform for budget preparation was in place an operating (system configuration for a budget module, and a module for processing public agency expenditure transactions were undertaken as prior actions). With the electronic platform, the build-up of expenditure arrears has diminished, from 25.6 percent of total expenditures at the start of the program to 2.7 percent at project closure, against a target of no more than 10 percent. Page 4 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NG Edo State FISDO(P151480) • Medium-term agency-level sector strategies conforming to the state’s strategic plan for 2010-2020 have been introduced in 14 agencies, compared to a target of six, the latter being the target set as a prior action. • To improve timeliness and quality of audited financial statements (prior action), In 2016, the state government’s audited financial statements for 2015 were published within 5 months of the end of the fiscal year, against a target of 7 months. Subsequently, the audited statements for FY2016 were published 4 months after the end of the fiscal year, and within 2 months for fiscal year 2017. Submission of audited statements within 9 months of the end of the fiscal year had been introduced as a prior action. In summary, in terms of fiscal sustainability, the revenue base has been expanded and the audit function has become more efficient – both resulting from improvements in the management of public resources supported by the DPO. They will also contribute to improving revenue generation. In terms of budget institutions and practices, the program has built up a cadre of trained procurement specialists, the government has the technical capacity to monitor and control the budget cycle, and medium term strategic planning has been introduced. The majority of Edo state's new procurement staff has professional certification; the upgrade to the electronic budget management system provides a centralized platform for managing budget execution; and the medium term sector strategy permits better control of strategic priorities. Rating High PHREVDELTBL PHEFFICACYTBL Objective 2 Objective To increase private sector access to land for investment Rationale The objective aimed at creating a more dynamic private sector environment by improving access to land for investment, a focus area for improving the investment climate. This was to be achieved by reducing fees on registering and transferring land titles, and it was to be measured by changes in the land transfer fee as a percentage of land value. As a prior action, a working group was established to make recommendations on changes to land transaction fees. Implementation of the recommendations was to occur under the follow-up operation to ESFISDDPO. Recommendations were made to reduce the land transfer fees by some 34 percent. However, the recommendation has not yet been implemented, although according to the ICR, it is expected to take place sometime in 2018. Page 5 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NG Edo State FISDO(P151480) Rating Modest PHREVDELTBL PHEFFICACYTBL Objective 3 Objective To improve the quality of education Rationale The objective was to improve TVET to promote youth employment; and teacher and student attendance in basic education. As prior actions for the initiatives to be taken to meet this objective, the state allocated funding in the 2015 budget to implement school improvement plans for that year; while the state’s Education Board issued guidelines for monitoring teacher and pupil attendance. Achievement of the objective was to be indicated by the number of partnership agreements that were initiated between TVET colleges and the private sector, share of eligible female students benefiting from such partnership arrangements, and school- based monitoring of attendance. • Memoranda of Understanding were drafted for three private companies to enter into partnerships with TVET colleges. However, no further progress had been made by project closing. • In December, 2016, 60 percent of school-based management committees were reporting on teacher and pupil attendance in basic education schools, achieving the target. • The target regarding the share of eligible female students benefiting from partnership arrangements could not be met since the arrangements had not yet reached the implementation stage. Since program closure, there has been some progress related to this objective. In early 2018 memoranda of understanding have been signed with five companies to enter into partnership with the state’s largest TVET school. Likewise, reporting on teacher and student attendance has expanded, reaching 70 percent of basic education schools. Rating Modest PHREVDELTBL PHREVISEDTBL 5. Outcome Page 6 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NG Edo State FISDO(P151480) The relevance of the PDO is rated high, as it responds to Edo State’s priorities set out in the state’s development strategy, and to the focus of the Bank’s strategy for Nigeria on public sector management and social inclusion. Relevance of design is also rated high based on the relevance of the chosen policy areas. Efficacy is rated high for the first objective. Net tax capture was significant, and the payroll audits also appear to have had an impact on fiscal sustainability in terms of elimination of ghost workers. Efficacy for the second objective is rated modest. The necessary changes to land title fees were agreed on, but they had not yet entered into effect. Efficacy for the third objective is rated modest. While some necessary policy actions were taken, albeit after program closing, their impact on the quality of skills development and youth employment, especially of women, and on basic education, is still to be assessed. a. Outcome Rating Moderately Satisfactory 6. Rationale for Risk to Development Outcome Rating According to the ICR, the Federal Government remains committed to the national reform program, including reforms taking place in Edo State. The State, in turn, has maintained reform momentum beyond program closure, as reflected in post-closing progress in various areas (see Section 4 above). That progress also includes a solidifying of the legal and institutional framework for public financial management (ICR, pages 16 and 17). At the same time, Edo State remains vulnerable to economic risks: oil is the dominant revenue source, and the State remains exposed to oil price fluctuations. The buffers created by the fiscal-financial policies introduced under this program may not be sufficient to eliminate their impact, and external shocks might still slow down future progress on reforms a. Risk to Development Outcome Rating Substantial 7. Assessment of Bank Performance a. Quality-at-Entry a. Quality-at-Entry The operation was strategically relevant in focusing on key policy areas – improving the management of the state’s resources, improving the investment climate in the state, and improving the quality of education. It built on an earlier DPO series (2012-2014), which indicated that the DPO could be an Page 7 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NG Edo State FISDO(P151480) effective and appropriate instrument for driving the State’s reform program forward. The background analysis for the operation was sound, with each prior action informed by analytical work done by the Bank and the recipient. A parallel SEEFOR (State Employment and Expenditure Effectiveness for Results) project provided complementary technical assistance financing. The results indicators were generally aligned with the PDOs and prior actions, albeit not always sufficiently measurable (see Section 4 above). Backing up the design were several assessments/surveys covering fiscal-financial, business, and education challenges in the state (ICR, page 6). Implementation, including M&E implementation, was done jointly by the Bank team and a state government coordinating unit. Risk assessment was adequate, and delays in implementation (notably a delay of almost one year in becoming effective) were driven by unpredictable political developments, rather than program-related issues. Quality-at-Entry Rating Satisfactory b. Quality of supervision The Bank supervision team worked closely with the state government's DPO coordinating unit, conducting several joint implementation support missions, including with the Bank's SEEFOR team (which had the same government DPO coordinator). Delays in implementation were political in nature and lay outside the control of the supervision team. The elimination of the proposed follow-up DPO was driven by macroeconomic factors, and also lay outside the control of the supervision team. However, some of the uncertainty surrounding outcomes in each of the three objective areas might have been avoided by placing a greater emphasis on assessing impacts. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 8. Assessment of Borrower Performance a. Government Performance The operation experienced several delays at the Federal Government level which affected project implementation and contributed to the cancellation of the second operation. Some of them were at least partly subject to government influence, such as delays in receiving parliamentary approval for the external borrowing; slow appointment of the national counterpart (the Minister of Finance) to sign the credit; and a deteriorating macroeconomic framework. Page 8 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NG Edo State FISDO(P151480) Government Performance Rating Unsatisfactory b. Implementing Agency Performance The Government of Edo State was the implementing agency. It was committed to the program and maintained reform momentum during and after project completion. The state's coordinating unit provided documentation on project progress in collaboration with the Bank team, while at the same time supporting the SEEFOR project. Implementing Agency Performance Rating Satisfactory Overall Borrower Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The policy matrix and the results framework for the operation presented 11 outcome indicators, along with baselines and targets. Generally, key indicators measured results that plausibly promoted objectives sought, for instance a gradual increase in the tax base as a means of increasing IGR; or an increase in the number of procurement-certified staff as a means of professionalizing the procurement function. In a few instances, the indicators that were introduced seemed to go only part-way: for improving the quality of education, the chosen indicators focused on means to increase participation, rather than addressing the objective of quality b. M&E Implementation Implementation was a joint effort by the Bank team and its state government counterpart, a unit that also coordinated the SEEFOR. Relevant agencies provided data on a regular basis needed to monitor progress, and the information was tabulated by the coordinating unit. According to the ICR, there appear not to have been any significant M&E implementation issues. c. M&E Utilization M&E data was used in charting progress in implementing the operation, and formed the basis for discussions between the Bank teams for the DPO, the SEEFOR and the state government counterparts. M&E Quality Rating Page 9 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NG Edo State FISDO(P151480) Substantial 10. Other Issues a. Environmental and Social Effects No safeguard policies were triggered b. Fiduciary Compliance The ICR reports no fiduciary issues. c. Unintended impacts (Positive or Negative) None noted. d. Other The operation included a gender component, reflected in two of its 11 indicators: monitoring of teacher and student attendance in general education was to be disaggregated by gender; and the share of female students in TVETs benefiting from the public-private partnership should be 50 percent. Neither target was reported on – the public-private partnership did not get underway, and no gender-based reporting in general education is available. 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory --- Satisfactory Risk to Development Substantial Substantial --- Outcome Moderately Bank Performance Moderately Satisfactory --- Satisfactory Moderately Borrower Performance Moderately Satisfactory --- Satisfactory Quality of ICR Substantial --- Page 10 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NG Edo State FISDO(P151480) Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 12. Lessons The lessons are drawn from the ICR with some adaptation: While a DPO program can incentivize reforms, reform orientation of the recipient is important to ensure sustainability and scale of reforms. In some of the ESFISDO policy areas, Edo State has been able to achieve or exceed the targets set for the program of two operations, even though the second operation did not materialize. In the areas where the State did not achieve the targets (e.g. for the TVET reforms and the land transaction rates), the state has maintained the reform momentum and, as in the case with the TVET reforms, is implementing the reforms in a more robust way than was anticipated by the DPO program. This suggests that the State itself is reform-oriented. Political transitions can cause significant delays in implementation of a DPO. Two political transitions took place during the program period, both of which caused long delays in effectiveness and program implementation, which necessitated rescheduling the first operation. It is important to ensure mitigating any potential delays arising from political transitions in a DPO program. If possible, specific mechanisms to ensure completion of a programmatic series of DPOs should be put in place, especially in high political risk environments. The operation was affected by a delay in parliamentary approval of the external borrowing plan, as well as a political transition. A mitigating mechanism could, for example, seek to ensure that initial parliamentary approval of an entire DPO series would suffice, without the need for subsequent approvals of the individual operations. 13. Assessment Recommended? Yes Please explain The operation had been designed as a two-DPO programmatic credit. The cancellation of the second operation leaves some uncertainty with regard to the actual impact of initiatives under the operation on the three objectives. An assessment involving field work could help to determine whether the operation actually achieved substantial results, especially with regard to the second and third objectives. 14. Comments on Quality of ICR Page 11 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NG Edo State FISDO(P151480) The ICR provides an adequate basis for assessing the operation and understanding the gaps created by eliminating the second operation. It provides a succinct synthesis of the programs, policy areas, and their links to outcomes and State priorities. The evaluation of efficacy is thorough and provides sufficient analytical and evaluative material for assessing achievements. At the same time, a more nuanced discussion of outcomes would have strengthened the document. The ICR at times stops at outcomes that fall short of the underlying intentions - a case in point is the "quality of education" objective that focuses on intermediate outcomes rather than ultimate goals. a. Quality of ICR Rating Substantial Page 12 of 12