Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review FY17 -Chad Emergency DPO(P163968) Report Number : ICRR0021498 1. Project Data Operation ID Operation Name P163968 FY17 -Chad Emergency DPO Country Practice Area(Lead) Chad Social Protection & Labor L/C/TF Number(s) Closing Date (Original) Total Financing (USD) IDA-D2270 31-Dec-2017 66,311,829.71 Bank Approval Date Closing Date (Actual) 29-Jun-2017 31-Dec-2017 IBRD/IDA (USD) Co-financing (USD) Original Commitment 65,000,000.00 0.00 Revised Commitment 65,000,000.00 0.00 Actual 66,311,829.71 0.00 Prepared by Reviewed by ICR Review Coordinator Group Hjalte S. A. Sederlof Judyth L. Twigg Eduardo Fernandez IEGHC (Unit 2) Maldonado 2. Project Objectives and Policy Areas a. Objectives The program development objective (PDO) of the Emergency Fiscal Stabilization Development Policy Operation (EFSDPO) as set out in the Program Document (PD) was to achieve fiscal consolidation and lay the foundation for a functioning social safety net and transfer system. Page 1 of 10 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review FY17 -Chad Emergency DPO(P163968) This ICRR will assess efficacy based on two objectives: (i) achievement of fiscal consolidation; and (ii) laying the foundation for a functioning social safety net. The “transfer system” constitutes an integral part of the safety net and is not assessed separately. b. Were the program objectives/key associated outcome targets revised during implementation of the series? --- c. Pillars/Policy Areas The EFSDPO included two pillars and five policy areas: Pillar 1: Fiscal management involving fiscal consolidation through expenditure rationalization included measures to: (i) reduce and streamline the central government wage bill; (ii) improve oversight over state- owned enterprises (SOEs); (iii) rationalize tax expenditures; and (iv) strengthen public procurement. Pillar 2: Social protection, involving moving from a system of short-term crisis management towards building a longer-term, more permanent safety net. d. Comments on Program Cost, Financing, and Dates Program cost and financing. An Emergency Grant of SDR 47 million (US$65 million equivalent) was provided and fully disbursed on becoming effective. Borrower contribution. None. Dates. The grant was approved on June 29, 2017, and disbursed on becoming effective on July 27, 2017. It closed on December 31, 2017. 3. Relevance of Objectives & Design a. Relevance of Objectives The EFSDPO’s objectives and pillars are consistent with the Country Partnership Framework for FY16-20 that includes the management of public resources, as well as reducing vulnerability and building human capital, as two of its three major themes of engagement in Chad. The former, primary, objective of fiscal consolidation supported the government’s emergency fiscal plan and aligned with donor efforts to reverse Page 2 of 10 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review FY17 -Chad Emergency DPO(P163968) fiscal deterioration. It also followed on earlier fiscal consolidation support grants provided in 2015 and 2016 (P155480 and P162548). Rating High b. Relevance of Design The choice of instrument for the operation was appropriate. It responded to the need for quick-disbursing budget support to address pressing financial needs in the aftermath of external shocks, including falling oil prices, security threats, and an ensuing humanitarian crisis, as well as the need to lay the foundations for a functioning safety net. The operation built on previous interventions focused on maintaining fiscal and macroeconomic stability, and functioned as a bridge to an upcoming medium-term structural DPO program, currently under preparation. It also aligned with other donor-generated programs (International Monetary Fund (IMF), European Union) that supported fiscal consolidation, especially the IMF’s three-year Extended Credit Facility (ECF) introduced in 2017. The ECF helps ensure a broadly adequate macroeconomic framework for the operation and provides the necessary stability for effective decision-making at the political level. The results framework was set out in the policy and results matrix, and it reflected the emergency nature of the operation. The PDO were primarily focused on short-term remedies to reduce pressures on fiscal balance. Four of five prior actions addressed fiscal management issues; the fifth one dealt with the safety net, but primarily as a means of laying the institutional foundations for its longer-term development. Even the fiscal remedies were modest; while all prior actions were met, only one of the targeted results was expected to have an immediate effect on fiscal consolidation, while the others set the stage for longer-term improvements. Rating Modest 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective To achieve fiscal consolidation Page 3 of 10 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review FY17 -Chad Emergency DPO(P163968) Rationale The objective was to be achieved by supporting reforms aimed at rationalizing the public wage bill and tax expenditures; improving SOE oversight; and enabling effective public procurement. Rationalizing the public wage bill Based on an audit to strengthen the effectiveness of the public payroll system, the government set a target to reduce allowances for civil servants by 50 percent effective November 2016 (both prior actions for EFSDPO). These efforts were expected to reduce the public wage bill by 3 percent by mid-2017 as compared with the level in mid-2016. Instead, the wage bill rose by one percent, as the government abandoned the plan due to pressures from affected stakeholders. Improving SOE oversight To identify weaknesses and improve efficiency in SOE management, the government commissioned an audit of the financial performance of SOEs to reduce transfers and subsidies to SOEs (prior action for EFSDPO). To facilitate the audit, the government was to complete an initial inventory of 20 SOEs, and issued a decree to ensure the availability of SOE financial statements for the years 2014-2016 (prior actions for EFSDPO). The process is still under way. Under the operation, only 15 SOEs submitted financial statements and a sub-set of 10 were audited, all of them benefiting from government subsidies. Significant constraints on oversight at the government level – limited human, financial, and physical resources – contributed to delays. Still, by late 2018, almost a year after the operation closed, 29 SOEs had been inventoried. Tax expenditures: the provision of new tax exemptions to SOEs was temporarily frozen as the process to identify means to improve the tax expenditure and subsidy system for SOEs was going on (prior action for EFSDPO). Strengthening public procurement The government established a procurement regulatory agency by appointing the president and members of a national council of regulation, as well as assigning it a regular budget line in the 2017 budget law (prior actions for EFSDPO). Previously, the government had already revised the procurement code, including provisions for contract implementation. With a new code and a new regulatory agency in place, the value of single source procurement contracts as a share of the value of total contracts was expected to decline from 37 percent to 30 percent. Instead, their share rose to 42 percent, influenced by an overall decline in procurement through public tenders (ICR, page 20) that included a slower decline in single-source contracts. In summary, measures to reduce the public wage bill were not successful, as the government yielded to political pressures opposing wage adjustments. The process of improving SOE oversight is still underway, Page 4 of 10 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review FY17 -Chad Emergency DPO(P163968) and neither tax expenditures nor subsidies have so far been addressed. Without further analysis of the procurement portfolio, it is difficult to assess achievements in public procurement. Rating Modest PHREVDELTBL PHEFFICACYTBL Objective 2 Objective To lay the foundation for a functioning social safety net Rationale The objective was to be achieved by moving from the existing emergency approach to addressing the needs of poor and vulnerable households, towards the establishment of a permanent safety net supporting resilience and strengthening livelihoods among the poor. Building on a safety nets unit established in 2016, the government set up a steering committee for its safety net program; recruited key staff; and introduced a system combining geographic and household targeting to select beneficiaries. By end project, a geographical area had been selected and a standard questionnaire applied to area households, drawing up an initial registry of 17,000 households (with 4,500 households identified as beneficiary households) (prior actions for EFSDPO). As a result, the 4,500 eligible households are now receiving cash benefits on a regular basis. The government has asked all donors supporting safety net programs to use the government’s questionnaire for targeting, and to include their findings into building up the registry. Rating Substantial PHREVDELTBL PHREVISEDTBL 5. Outcome Relevance of objectives is related high: objectives were in line with the CPF, and reflected the country situation and government policy. Relevance of design is rated modest: it addressed relevant issues, but with only partial Page 5 of 10 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review FY17 -Chad Emergency DPO(P163968) focus on the primary objective of finding remedies to short-term financing needs. Efficacy for the first objective is rated modest, as efforts at fiscal consolidation were only partly successful. Efficacy for the second objective is rated substantial, as the project significantly advanced in key areas for building the safety net. The PD (pp. 17-18) made an effective case for the primacy of the first objective, referring to it as the "main objective" and to the safety net objective as a "secondary objective." Giving more weight to the first objective, overall efficacy is rated moderately unsatisfactory. a. Outcome Rating Moderately Unsatisfactory 6. Rationale for Risk to Development Outcome Rating The operation was introduced into a volatile economic and political environment. While the government managed to move the policy agenda forward, pressing financial needs, security risks, and a threatening humanitarian crisis continue to pose risks that were rated high already at appraisal. a. Risk to Development Outcome Rating High 7. Assessment of Bank Performance a. Quality-at-Entry The operation was timely; an emergency response in the form of external budget support was appropriate in a situation of economic and social crisis. To address pressing fiscal needs, the operation focused on fiscal stabilization as its primary objective, albeit with only one of four prior actions expected to have an immediate effect on fiscal consolidation; the others set the stage for longer-term improvements. Introducing the safety net objective as a secondary aim directly recognized the longer-term nature of building an adequate transfer system, and in the short term focused on two fundamental building blocks: targeting and registry. At the same time, the urgency of the operation – it was put into place over a period of some four months – may not have sufficiently recognized political constraints to some of the policy changes pursued in trying to control fiscal spending. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision Page 6 of 10 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review FY17 -Chad Emergency DPO(P163968) While there was no formal supervision of the operation – one is not required for a single-tranche operation of this type – some monitoring did occur to the extent that actions under the operation coincided with activities during preparation of the new DPO series, and in supervision of other ongoing projects with a similar focus. According to the ICR (page 25), this was sufficient to allow the Bank team to react to concerns when they arose, even in the absence of sufficient information flows from the government.. However, given the modest results in meeting the operation's primary objective, "engagement" was not sufficient. Quality of Supervision Rating Moderately Unsatisfactory Overall Bank Performance Rating Moderately Unsatisfactory 8. Assessment of Borrower Performance a. Government Performance While the government appears to have been committed to the objectives being pursued under the operation, progress was nevertheless modest. In part this reflected a challenging political and economic environment (national security, civil service strikes, fluctuating export prices) and a lingering humanitarian crisis, all of which diverted government attention from the task at hand. It was also to an important extent influenced by institutional and capacity limitations; while prior actions were met, achievement of intended results faltered. Government Performance Rating Moderately Unsatisfactory b. Implementing Agency Performance Not assessed. Implementing Agency Performance Rating Not Rated Overall Borrower Performance Rating Moderately Unsatisfactory 9. M&E Design, Implementation, & Utilization Page 7 of 10 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review FY17 -Chad Emergency DPO(P163968) a. M&E Design A policy and results matrix included short-term indicators of performance and targets to measure change, reflecting the short-term nature of the operation. Information flows were coordinated by a Negotiations Committee in the Ministry of Finance and Budget (MFB) which had participated in preparing the operation and monitored progress against program objectives. The Committee had a solid track record in collaborating with the Bank in monitoring similar operations. Participating agencies provided information and documentation of progress on their respective programs to the Committee. The safety net component was overseen by a national safety net unit, which already had experience in piloting cash transfers b. M&E Implementation In the absence of formal implementation arrangements by the Bank, monitoring was performed by Bank teams when EFSPDO activities coincided with other Bank operations, and in collaboration with other donors. Rigorous monitoring was hampered by the modest performance of the MFB team and the uncertain flow of information between participating agencies and the MFB (ICR, page 25). c. M&E Utilization The information collected on the operation was used to design the new, upcoming programmatic series. In particular, experience with the failed reduction in the wage bill and slow progress on acquiring SOE financial statements led to adjustments in the targets being set for the new series. M&E Quality Rating Modest 10. Other Issues a. Environmental and Social Effects Not raised in the ICR. b. Fiduciary Compliance Not raised in the ICR. Page 8 of 10 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review FY17 -Chad Emergency DPO(P163968) c. Unintended impacts (Positive or Negative) None reported. d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Moderately Outcome --- Unsatisfactory Unsatisfactory Risk to Development High High --- Outcome With a MU rating for Outcome, and MS for quality at entry Moderately Moderately Bank Performance and MU for quality of Satisfactory Unsatisfactory supervision, Bank performance is rated MU Moderately Moderately Borrower Performance --- Unsatisfactory Unsatisfactory Quality of ICR Substantial --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 12. Lessons An emergency DPO may be the most appropriate instrument to respond to a government’s request for immediate relief to a massive fiscal crisis. Teams need to determine the urgency of addressing a fiscal crisis when determining the nature of the intervention. EFSDPO was implemented when the government had to address an economic crisis, a large financing gap, growing security concerns, and with limited institutional capacity to carry out substantial reforms. An emergency DPO was the appropriate choice. Page 9 of 10 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review FY17 -Chad Emergency DPO(P163968) It is useful to have ongoing longer-term operations that complement an emergency response. When deciding on the nature of the intervention, it should be seen in the broader context of ongoing support. A stand- alone, single-tranche operation such as the EFSPDO, by itself, would be at risk of failing (i.e., not meeting the PDOs) if its PDOs were not aligned with other Bank operations that have longer implementation periods. Supplementary and dedicated technical assistance can help governments stay focused on achieving an emergency DPO. Teams should assess the utility of flexible technical assistance and prepare for it in designing an emergency operation. A stand-alone single tranche DPO does not include budgeted supervision. Moreover, government capacity is often weak, especially during a crisis. In this situation, technical assistance to an implementing agency can be critical to achieving the PDOs. In EFSPDO, there was no supervision required as it was stand-alone, but technical assistance to the government to help rationalize expenditure could have helped in achieving targets. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provided an adequate evaluation of the operation, including a sufficient description of its background and rationale to permit results to be assessed and ratings to be assigned. The ICR was consistent with the guidelines, although the main text of 28 pages by far exceeded the prescribed length of 15 pages. a. Quality of ICR Rating Substantial Page 10 of 10