Document of The World Bank Report No: ICR00004055 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-H4810) ON A CREDIT IN THE AMOUNT OF SDR 6.7 MILLION (US$ 10 MILLION EQUIVALENT) TO THE REPUBLIC OF NIGER FOR A REFORM MANAGEMENT AND TECHNICAL ASSISTANCE PROJECT April 26, 2017 Governance Global Practice AFCW3 Africa Region i CURRENCY EQUIVALENTS Exchange Rate Effective: April 26, 2017 Currency Unit = Franc CFA 1 XOF = 0.00166802 USD 1 USD = 599.51 XOF FISCAL YEAR January 1- December 31 ABBREVIATIONS AND ACRONYMS AFD Agence Francaise de Developpement AfDB/ADB African Development Bank AM Aide Mémoire CAS Country Assistance Strategy CMU Country Management Unit CPF Country Partnership Framework DGB General Budget Directorate DGCF General Financial Control Directorate DGCMP General Public Procurement Directorate DGCPT General Government Accounting and Treasury Directorate DGD General Customs Directorate DGE General Economy Directorate DGI General Taxes Directorate DIF Financial Management Information Systems Directorate DPO Development Policy Operation EC European Commission ENAM National School of Public Administration ESIA Environmental and Social Impact Assessment. EU European Union EU European Union FM Financial Management FMIS Financial Management Information System FSEJ Faculty of Economics and Law at the University of Niamey GDP Gross Domestic Product GoN Government of Niger HIPC Heavily Indebted Poor Countries HR Human Resources ICR Implementation Completion Report ICT Information and Communication Technology IDA International Development Association IDP Internally Displaced People IEG Internal Evaluation Group IFMIS Integrated Financial Management Systems IFR Interim Financial Report ii IGF Inspection General of Finances IRI Intermediate Results Indicator ILI Intensive Learning ICR IMF International Monetary Fund ISR Implementation Supervision Report IT Information Technology M&E Monitoring and Evaluation MDGs Millennium Development Goals MDRI Multilateral Debt Relief Initiative MEF Ministry of Economy and Finance MoP Ministry of Planning MTEF Medium-Term Expenditure Framework MTR Mid Term Review ORAF Operational Risk Assessment Framework PAD Project Appraisal Document PCDS Public Sector Capacity Service Delivery Project PCU Project Coordination Unit PDES Social and Economic Development Plan PDO Project Development Outcome PEFA Public Expenditure and Financial Accountability PEMFAR Public Expenditure Management and Financial Accountability Review PFM Public Financial Management PI PEFA Indicator PIU Project Implementation Unit PMU Project Management Unit PPA Project Preparation Advance PRC Project de Renforcement des Capacites PRSP Poverty Reduction Support Paper RMTA Reform Management and Technical Assistance Project SG Secretary General SONITEL Société Nigérienne de Télécommunications TORS Terms of Reference TTL Task Team Leader UNCTAD United Nations Conference on Trade and Development UNDP United Nations Development Program VAT Value Added Tax WAEMU West African Economic and Monetary Union Senior Global Practice Director: Deborah L. Wetzel Practice Manager: Chiara Bronchi Project Team Leader: Ragnvald Michel Maellberg ICR Team Leader: Anne-Lucie Lefebvre Primary ICR Author: Michael Christopher Jelenic iii REPUBLIC OF NIGER Reform Management and Technical Assistance Project CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 5 3. Assessment of Outcomes .......................................................................................... 13 4. Assessment of Risk to Development Outcome......................................................... 24 5. Assessment of Bank and Borrower Performance ..................................................... 25 6. Lessons Learned ....................................................................................................... 27 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 31 Annex 1. Project Costs and Financing .......................................................................... 32 Annex 2. Outputs by Component ................................................................................. 34 Annex 3. Economic and Financial Analysis ................................................................. 39 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 41 Annex 5. Beneficiary Survey Results ........................................................................... 43 Annex 6. Stakeholder Workshop Report and Results................................................... 44 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 45 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 49 Annex 9. List of Supporting Documents ...................................................................... 50 Annex 10. List of People Met…………………………………………………………51 Annex 11: Map of Niger ……………………………………………………………...53 iv A. Basic Information Niger Reform Country: Niger Project Name: Management and TA Project ID: P108253 L/C/TF Number(s): IDA-H4810 ICR Date: 12/19/2016 ICR Type: Core ICR REPUBLIC OF Lending Instrument: TAL Borrower: NIGER Original Total XDR 6.70M Disbursed Amount: XDR 6.62M Commitment: Revised Amount: XDR 6.70M Environmental Category: C Implementing Agencies: Ministry of Finance Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept 07/15/2008 Effectiveness: 01/14/2010 01/29/2010 Review: Appraisal: 03/09/2009 Restructuring(s): 12/10/2014 Mid-term Approval: 07/02/2009 06/05/2012 06/22/2012 Review: Closing: 04/30/2015 10/30/2016 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Significant Bank Performance: Moderately Unsatisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Moderately Moderately Quality at Entry: Government: Unsatisfactory Unsatisfactory Quality of Moderately Implementing Moderately v Supervision: Unsatisfactory Agency/Agencies: Satisfactory Overall Bank Moderately Overall Borrower Moderately Performance: Unsatisfactory Performance: Satisfactory C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Quality at Entry Project at any time Yes None (QEA): (Yes/No): Problem Project at any Quality of Yes None time (Yes/No): Supervision (QSA): DO rating before Moderately Closing/Inactive status: Satisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central Government (Central Agencies) 30 30 Public Administration - Financial Sector 70 70 Theme Code (as % of total Bank financing) Administrative and civil service reform 30 30 Macroeconomic management 3 3 Other public sector governance 3 3 Public expenditure, financial management and 61 61 procurement Tax policy and administration 3 3 E. Bank Staff Positions At ICR At Approval Vice President: Makhtar Diop Obiageli Katryn Ezekwesili Country Director: Paul Noumba Um Madani M. Tall Practice Chiara Bronchi Anand Rajaram Manager/Manager: Project Team Leader: Ragnvald Michel Maellberg Robert A. Yungu ICR Team Leader: Anne-Lucie Lefebvre ICR Primary Author: Michael Christopher Jelenic vi F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The Project Development Objective is to improve: (i) the credibility and reliability of budgets allocated to budget managers; and (ii) the internal controls to hold budget managers accountable Revised Project Development Objectives (as approved by original approving authority) No Changes to the Project Development Objectives (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Indicator 1 : Ratio of Arrears to Total Expenditure Value > 10% < 5% 4.0% quantitative or Qualitative) Date achieved 06/08/2009 10/30/2016 12/31/2016 Comments Achieved: The ratio of arrears to total expenditure decreased from 28 (incl. % percent in 2009 to 4 percent at the end of 2016, meeting the end of project achievement) target of less than 5 percent (PDO 1). Time Delay in Submission of Financial Statements to the Chamber of Indicator 2 : Accounts and the National Assembly 6 Month delay Loi de Reglement Loi de Reglement - 15 for the - 9 month delay Value month delay production of quantitative or both the Loi de Comptes de Qualitative) Comptes de Gestion - Reglement and Gestion - 6 No submission the Comtes de month delay Gestion Date achieved 06/08/2009 10/30/2016 12/31/2016 Achieved: The time taken to submit financial statements to the Chamber of Accounts and National Assembly, from more than 15 months for the Comments publication of the Loi de Reglement and no release of the Comptes de (incl. % Gestion, respectively, decreased to just 6 months for the publication of achievement) both the Loi de Reglement and Comptes de Gestion (PDO 2), meeting the end of project targets. vii (b) Intermediate Outcome Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Indicator 1 : Deviation in Aggregate Expenditure Value > 10% < 5% +16.5% (quantitative or Qualitative) Date achieved 06/08/2009 10/30/2016 12/31/2016 Comments Not Achieved: Deviation in aggregate expenditure decreased from 37.0 to (incl. % 16.5 percent over the life of the project: however, it did not meet the end achievement) of project target of less than 5 percent (IRI 1). Indicator 2 : Deviation in Aggregate Revenue for Tax and Customs Directorates +16.9% Value > 15% < 5% (tax) (quantitative -10.6% or Qualitative) (customs) Date achieved 06/08/2009 10/30/2016 12/31/2016 Not Achieved: Deviation in aggregate revenue for tax and customs Comments directorates began to show an initial decrease, however, by the end of the (incl. % project tax revenue deviation was 21.6 percent and customs revenue achievement) deviation was 10.6 percent, which marked almost no changed from their 2009 baseline values of 21.8 and 11.0 percent, respectively (IRI 2). Degree of Compliance with rules for processing and recording Indicator 3 : transactions (extraordinary spending) Value 1.8% > 35% < 10% (quantitative or Qualitative) Date achieved 06/08/2009 10/30/2016 12/31/2016 Achieved: The degree of compliance with the rules for processing and Comments recording transactions (as measured by a reduction in exceptional (incl. % spending) met its end of project target of less than 10 percent, as achievement) exceptional spending decreased from 23.5 percent in 2009 to just 1.8 percent in 2016 (IRI 3). Frequency of Complete Accounts Reconciliation between tax/customs Indicator 4 : assessments, collections, arrears, records, and receipts by Treasury All Reconciliations Value Tax, > 1 year Customs, Tax, quarterly are now (quantitative > 9 months Customs, every 6 completed on an or Qualitative) weeks annual/monthly basis as required Date achieved 06/08/2009 10/30/2016 12/31/2016 viii Achieved: The frequency of complete accounts reconciliation between Comments tax/customs assessments, collections, arrears, records, and receipts (incl. % increased over the life of the project, albeit with a slight deterioration in achievement) the last year of the project. However, the final targets were met as of the time of this ICR (IRI 4). Indicator 5 : Timeliness of the Semi-Annual Budget Reports Value < 6 weeks > 8 weeks < 6 weeks (quantitative or Qualitative) Date achieved 06/08/2009 10/30/2016 12/31/2016 Comments Achieved: Over the life of the project the timeliness of semiannual budget (incl. % reports has improved considerably from being delivered more than 8 achievement) weeks to less than 6 weeks after the end of the relevant quarters (IRI 5). G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 12/31/2009 Satisfactory Satisfactory 0.00 2 06/30/2010 Moderately Satisfactory Moderately Satisfactory 0.16 3 05/24/2011 Moderately Satisfactory Moderately Satisfactory 1.21 4 12/26/2011 Satisfactory Satisfactory 2.02 5 07/11/2012 Satisfactory Satisfactory 3.38 Moderately 6 01/29/2013 Satisfactory 4.47 Unsatisfactory 7 07/24/2013 Moderately Satisfactory Satisfactory 5.25 8 02/17/2014 Moderately Satisfactory Moderately Satisfactory 6.53 9 09/13/2014 Satisfactory Satisfactory 7.41 10 06/11/2015 Satisfactory Moderately Satisfactory 8.03 11 02/11/2016 Moderately Satisfactory Moderately Satisfactory 8.52 12 08/16/2016 Moderately Satisfactory Moderately Satisfactory 9.92 H. Restructuring (if any) ISR Ratings Amount Board Restructuring at Disbursed at Reason for Restructuring Approved Date(s) Restructurin Restructurin & Key Changes Made PDO Change g g in USD ix DO IP millions Extension of closing date by 12/10/2014 S S 7.68 18 months until October 30, 2016. I. Disbursement Profile ICR Abstract Since 2000, Niger had initiated a PFM reform agenda with the help of major development partners, but success had been limited. In this context, the Minister of Economy and Finance requested the Bank to help strengthen, modernize, and reform his ministry in order to improve its capacity to manage public finances. World Bank support for the government’s PFM priorities was provided by the Reform Management and Technical Assistance (RMTA) Project, which aimed to improve (i) the credibility and x reliability of budgets allocated to budget managers; and (ii) the internal controls to hold budget managers accountable. Given this objective, the overall outcome rating of the project is Moderately Satisfactory, on the basis of the following:  The overall rating for project relevance is Substantial. In particular: (i) the project objectives were substantial and remain relevant to the Bank and Borrower at the time of the ICR; (ii) relevance of design was modest given the overly ambitious nature of the PDO and poor links to results indicators; and (iii) relevance of implementation was substantial due to adaptive changes to activities that occurred.  The overall project efficacy rating is Substantial. Both of the PDO indicators were met and three out of five intermediate indicators were fully met. The project only suffered moderate shortcomings with respect to the first objective (credibility), which were largely offset by the strong performance of the second objective (internal control).  The project efficiency rating is Modest. While no data currently exists to measure the project’s benefits in economic or financial terms, in years to come the new customs administration software is likely to improve inspection and oversight, resulting in greater collections of customs revenues and less leakages. The overall analysis of the project also takes into account Risks to Sustainability of Results, Bank Performance, and Borrower Performance. Risks are Significant as a result of Political and Governance, Macroeconomic, and Institutional Capacity for Implementation and Sustainability Risk. In terms of Bank performance, Quality and Entry is rated Moderately Unsatisfactory and Quality of Supervision is rated Moderately Unsatisfactory, Overall Bank Performance is rated Moderately Unsatisfactory. Likewise, in terms of Borrower performance, general Government Performance is rated Moderately Unsatisfactory and Implementing Agency Performance is rated Moderately Satisfactory, Overall Borrower Performance is rated Moderately Satisfactory since the project outcome was likewise rated Moderately Satisfactory. Given both the success and challenges noted during project preparation and implementation of the RMTA project, a number of lessons can be drawn from the project. In particular, these lessons can be utilized by both the Bank and the Government of Niger to inform future operations in the country, PFM sector, as well as FCV environments more largely. xi 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. Since gaining independence, Niger has experienced frequent coups d’états and violent rebellions, which have reduced trust in the effectiveness of the government and the stability of the political system. However, with the election of President Mamadou Tandja in 1999, Niger entered a period of improved political stability and relative economic prosperity. President Tandja was reelected for another 5-year term in 2004, and in the same year, a major milestone toward political decentralization was reached with the first ever local elections, which took place across all 265 communes in Niger. 2. Accompanying this return to relative political stability, macroeconomic and fiscal performance likewise improved. Real GDP grew at an average 5.1 percent per year between 2001 and 2007, compared to 1.6 percent per year during the previous decade. In 2008, real GDP growth reached 6 percent, up from 3.3 percent in 2007. Improved political stability and better macroeconomic conditions had been the main reasons for this remarkable achievement; however, these improvements had not translated to improved human development outcomes, especially with respect to progress on the Millennium Development Goals (MDGs). At the time of appraisal, Niger was ranked 174th out of 177 countries on the UNDP’s Human Development Index and was one of the poorest countries in the world, with a 2007 per capital GDP of just US$ 260. 3. Since 2000, Niger had initiated a PFM reform agenda with the help of major development partners, but success had been limited. The results of the DPOs (Public Expenditure Adjustment Credits I and II) provided by the Bank between 2000 and 2003, which were intended to shore up public expenditure management, were modest as budget allocations to priority sectors edged up only slightly. Subsequently, the Bank and the EU worked closely with GoN to finalize a Public Expenditure Management and Financial Accountability Review (PEMFAR), which became the basis of PFM reform in Niger. Following the PEMFAR recommendations of 2005, which were agreed between the government and donors for improving PFM in Niger, there were some notable successes including: (i) adoption of West Africa Economic and Monetary Union (WAEMU) budget nomenclature and its usage in budget preparation and execution; (ii) elaboration and dissemination of the organic budget law and government accounting and performance report; (iii) partial automation of the expenditure chain; (iv) compliance in funding Heavily Indebted Poor Countries (HIPC) expenditure priorities; and (v) integration of a substantial share of externally financed expenditures in the chain of expenditures. 4. Despite these achievements, other areas of PFM reform were still lagging at the time of project appraisal. In particular, budget preparation was managed by the Ministry of Economy and Finance (MEF) with limited inputs from sector ministries and other key stakeholders. As a result, there were often disconnects between sector budgets prepared by line ministries using sector Medium-Term Expenditure Frameworks 1 (MTEFs) and budgets allocated by MEF. The national budget was mainly incremental and done on an annually basis, and the time for budget discussions was limited and not focused on sector strategic outcomes. As a result, there was not enough ownership and accountability by line ministries with respect to their budget outcomes. At the same time, budget execution was plagued by inefficient internal controls and delays in the availability of funds. While the automation of the expenditure chain helped to streamline some of the internal controls, which were causing major delays, the automation process was only partially complete and needed to be extended. Moreover, the limited oversight of financial controllers (only 13 people to control ministries and other governmental agencies) had negative effects on the control and budget execution. The situation was further exacerbated by difficulties in cash management from the unpredictability of revenue inflows as well as a lack of timely information exchanges among the most critical departments in MEF (e.g Tax, Customs, Budget, and Treasury). As a result of these anomalies, the level of budget execution remained low and the stock of internal arrears continued to persist. Finally, transparency and accountability were not greatly improved as the budget process continues to be driven centrally by MEF and as only limited information on public finance management is available to the public. 5. In this context, the Minister of Economy and Finance requested the Bank to help strengthen, modernize, and reform his ministry in order to improve its capacity to manage public finances. Specific areas for support included strengthening of General Directorates; acquisition and usage of decision support tools, training, research; and mobilization of experts from within the country and the diaspora. At the time of this request for support, strengthening capacity of MEF was a critical priority as revenues were starting to flow in from the extractives sector, particularly uranium and oil export revenues. As a result, there was an urgent need to improve MEF capacity to properly manage these additional revenues, to expand public expenditure in social sectors and infrastructure, and to ensure the effectiveness, efficiency, and transparency of these expenditures. 6. World Bank support for the government’s PFM priorities was provided by the Reform Management and Technical Assistance (RMTA) Project (Project de Renforcement des Capacites - PRC). This project was closely aligned with the Bank’s 2008-2011 Country Assistance Strategy (CAS) for Niger, which in addition to its two strategic objectives of (i) accelerating sustainable economic growth that is equitably shared and (ii) developing human capital through equal access to quality social services, included the cross-cutting theme of promoting good governance to ensure that increased revenues from growth are efficiently spent and broadly shared. As such, the CAS designated the RMTA project as one of the main instruments for building capacity and promoting governance in Niger during the period. Moreover, the project was a critical element in helping the government achieve the seven objectives noted in its current PSRP, including those related to good governance and capacity building. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 2 7. The project development objective was to improve: (i) the credibility and reliability of budgets allocated to budget managers; and (ii) the internal controls to hold budget managers accountable. 8. This PDO was designed to be measured by two outcome level indicators and five intermediate results indicators as follows:  PDO Indicator 1: Ratio of Arrears to Total Expenditure  PDO Indicator 2: Time Delay in Submission of Financial Statements to the Chamber of Accounts and the National Assembly  IRI 1: Deviation in Aggregate Expenditure  IRI 2: Deviation in Aggregate Revenue for Tax and Customs Directorates  IRI 3: Degree of Compliance with rules for processing and recording transactions  IRI4: Frequency of Complete Accounts Reconciliation between tax/customs assessments, collections, arrears, records, and receipts by Treasury  IRI 5: Timeliness of the Semi-Annual Budget Reports 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 9. No revisions to the PDO were made during project implementation. 1.4 Main Beneficiaries 10. The primary target group of the project were the institutions and individuals within the Ministry of Economy and Finance (MEF) and the Ministry of Planning (MoP), who benefited directly from capacity building activities and IT systems investments. These include the General Budget Directorate (BDB), General Financial Control Directorate (DGCF), General Public Procurement Directorate (DGCMP), General Government Accounting and Treasury Directorate (DGCPT), General Economy Directorate (DGE), General Customs Directorate (DGD), Financial Management Information Systems Directorate (DIF), General Taxes Directorate (DGI), Inspection General of Finances (IGF), and the Project Management Unit (PRC). Additional primary beneficiaries include local training institutes, including the National School of Public Administration (ENAM) and the Faculty of Economics and Law at the University of Niamey (FSEJ), which benefited from resources to expand and deliver critical courses to government employees. A final group of primary beneficiaries included members of the diaspora selected for contracts to provide capacity support key departments. 11. Secondary beneficiaries included the government as a whole, which benefited from the IT systems upgrading, including the introduction of an internet, intranet, and email system. These beneficiaries include numerous government ministries, departments, and agencies in Niamey, including the office of the President and Prime Minister. Likewise, local governments in Dosso, Tahoua, MMradi, and Zinder benefited from both IT connections within their local government offices in their city centers as 3 well as from service contracts with SONITEL, which provided connections between these regional capitals and Niamey. Finally, other indirect beneficiaries included citizens of Niger, who would ultimately benefit from the more efficient use of public finances. 1.5 Original Components  Component 1: Capacity Development (US$3 million): This component was designed to support activities to ensure long-term capacity development at the individual, organizational, and institutional levels.  Sub-component 1.1: Strengthening Capacity of Key Functions of MEF (US$ 1.5 million): This subcomponent was designed to support the MEF to improve performance in critical areas such as macro-fiscal coordination and policy, medium-term revenues and expenditures forecasting, program budgeting, tax collection, internal controls, cash management, and treasury reform.  Sub-component 1.2: Support to Local Training Institutions (US$ 1.0 million): This subcomponent was designed to strengthen the capacity of the National Institute of Public Administration and Magistrate (ENAM) and the Faculty of Economics and Law at the University of Niamey (FSEJ) in order to provide sustainable capacity building support to the government.  Sub-component 1.3: Support to the Diaspora program (US$ 0.5 million): This subcomponent was designed to leverage the growing number of Nigeriens living abroad who have been exposed to modern management techniques in various sectors and who could lend their expertise to key departments in the Ministry of Economy and Planning.  Component 2: Leverage and Rationalize Financial Management Information Systems (US$5.0 million). This component was designed to rationalize and strengthen financial management information and communications systems in order to provide significant efficiency gains and cost savings.  Sub-component 2.1: Intranet/Intranet Deployment (US$2.5 million). This subcomponent was designed to provide internet connections between regional capitals and Niamey as well as develop a fiber optic network to connect disparate MEF information systems through the negotiation of service contracts with SONATEL.  Sub-component 2.2: Systems Rationalization and Upgrade (US$2.0 million). This subcomponent was designed to upgrade Budget systems, Customs information systems, Tax Authority information systems, and the cash management system. 4  Sub-component 2.3: Office Equipment Upgrade (US$0.5 million). This subcomponent was designed to help fund, jointly with the government and other donors, network equipment, computers, printers, and communication tools.  Component 3: Support to the Project Coordination Unit (US$ 1.5 million). This component was designed to provide staff and the necessary furnishing of a PIU within the MEF. 1.6 Revised Components 12. No revisions to the components were made during project implementation. 1.7 Other significant changes 13. A level 2 restructuring was approved in December, 2014 in order to extend the project closing date from April 30, 2015 until October 30, 2016. This 18-month extension was approved by the Country Director in order to complete the installation of four major information systems, including: (i) customs system migration (SYDONIA World); (ii) payroll; (iii) integration of external investments in the expenditure chain; and (iv) interconnection between the central financial management system with the regions. The change in closing date was the only substantive change to the project, and no changes were made to the PDO, the safeguards category, budget allocation among components, or the results framework. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 14. The project design was based on a solid body of background analysis, which aimed to diagnose key bottlenecks to improved public financial management. Starting in 2004, the Public Expenditure Management and Financial Accountability Review (PEMFAR) became the basis for the PFM strategy in Niger. The PEMFAR made a comprehensive assessment of the Niger PFM systems and laid out an action plan to address shortcomings in the areas of budget preparation, execution, controls, information systems, cash management, and domestic debt management. The findings of the PEMFAR were further validated and reformulated on the basis of the first Public Expenditure and Financial Accountability (PEFA) exercise, which was conducted in September 2008, which provided important benchmarks of PFM performance as well as the baselines for a number of indicators. 15. In addition to these diagnostics, the project design was informed by the lessons learned from several implementation completion reports (ICRs) of earlier PFM reform projects in Niger. A first lesson learned from previous ICRs was that low capacity in public sector—particularly in the MEF—was the root cause of the lack of success of reform programs in Niger. Accordingly, the RMTA project specifically targeted the MEF and its departments as the beneficiaries of capacity building activities 5 and IT systems upgrades. A second lesson learned was the need to combine the design of a major policy and institutional reform program in public finance management with the implementation of comprehensive capacity-building. To meet these needs, the RMTA was designed to support capacity building within the MEF and related training institutions (ENA and FSEJ) as well as by leveraging the Nigerien diaspora expert community. A third lesson learned was that long-term and systemic treatment of existing anomalies of the PFM system rather than a quick short-term technical fix, requires a sequenced approach for building credible and reliable PFM systems. As described below, the project took a “platform approach� to ensure appropriate sequencing, selectivity, and time for reform consolidation. A final lesson learned emphasized in previous ICRs was the importance of creating a project management unit (PMU) inside MEF rather than create an outside parallel structure. Accordingly, a PMU was developed directly within the MEF under the direct oversight of the Secretary General of the MEF. 16. As noted above, it was on the basis of these analytics and lessons learned that technical assistance was provided in a selective manner based on a “platform approach�.1 In particular, the focus was on the first two platforms (budget credibility and financial accountability), due in part to the time horizon and limited budget of the project. As such, the project explicitly did not chose to include demand-side activities for strengthening external controls, transparency, and accountability by the Parliament Finance and Accounts Committee, Court of Accounts, media, and civil society, as other donors were already intervening in these areas (e.g. AFD and EU). Likewise, given the resources and time available for the operation, the project design intentionally did not include the treatment of civil service reform due to existing capacity challenges in the MEF, which would have made it difficult to simultaneously undertake large scale PFM and HR reforms. 17. While ambitious, the PDO of improving budget predictability and credibility as well as improving the internal control environment, was well linked to both the country and sector priorities. As noted, the project was well aligned with the CAS, which prioritized improved management of public resources as well as the medium-term priorities set out in the PRSP in terms of building capacity in public sector and promoting good governance. These priorities were especially pertinent during project preparation, as revenues began to flow from the extractives sector, particularly uranium mining and oil. During project preparation, the MEF was very committed to the project and has taken the necessary steps to ensure ownership and leadership in initiatives designed to establish the basis for long-term sustainability. While there was no evidence of direct stakeholder consultations, there was great participation of MEF staff in project preparation and frank dialogue about critical needs and the type of activities that should be undertaken to meet expected outcomes. 1 The PFM platform approach is a strategy designed to package and sequence reform measures (platforms) which, taken together, progressively aim to produce a higher level of PFM competence. 6 18. Although there was adequate commitment and engagement of the government at the time of preparation, there was not a high degree of donor coordination around the PFM agenda. Despite the involvement of a number of multilateral and bilateral donors2, there was not yet a unique institutional set up within MEF to bring together and align these interventions around a comprehensive capacity development plan. Given this lacunae, there was a strong willingness on the part of the government to upgrade country systems (financial management and procurement management), so that their use would be acceptable to donors in donor financed operations. In addition, it was a parallel aim of the project preparation dialogue to have donors better harmonize their interventions and use the same implementation structure or focal point inside MEF to provide their support for the implementation of PFM reforms. 19. Risks were assessed at project design and mitigation measures were suggested. Specific risks identified included: (i) implementation delays due to the end of the political cycle; (ii) weak mechanisms for monitoring and evaluation of the project’s outputs and outcomes; (iii) inadequate technological capabilities; (iv) inadequate implementation capacities; and (v) inadequate project sequencing. At the time of project preparation, the Bank did not have a standardized risk assessment tool in place such as the Operational Risk Assessment Framework (ORAF). As a result, a number of additional yet relevant risks were not assessed, including, country political and governance risk, macroeconomic risk, stakeholder risks, program and donor risk, as well as social and environmental risk. 2.2 Implementation 20. The Niger Reform Management and Technical Assistance (RMTA) was approved by the Bank’s Board of Executive directors on July 2, 2009, and became effective on January 29, 2010. This initial delay was due in part to the time it took for the project to meet effectiveness conditions, which included the purchasing of proper accounting software (TOMPRO), the preparations of an Operation Procedures Manual, and the setting up of a Monitoring and Evaluation Committee. In order to support these activities, as well as the finalization of a technical study on the roll-out of a government Internet, a Project Preparation Advance (PPA) provided resources to the project until effectiveness conditions were met. Shortly after the project became effective, however, efforts by President Tandja to circumvent a two-term limit resulted in a severe political crisis that led to the February 2010 military coup d’état. For four months, the Bank suspended all projects in Niger under OP 7.30 on Dealings with De Facto Governments. 2 At the time of appraisal, there were a number of donors engaged in supporting the MEF capacity as well as PFM reform more largely in Niger, including inter alia, the European Commission through its Development Fund (Fond Européen de Développent); the French Embassy through its French Cooperation Unit (Coopération Française) as well as the French Development Agency (Agence Française de Développement); the United Nations Development Program (UNDP); the African Development Bank (ADB), the German Agency for Technical Cooperation (GTZ); as well as the International Monetary Fund (IMF) and the US Treasury. 7 The measure was lifted in May 2010, and the project was officially launched in July 2010, representing a cumulative one-year delay in before project implementation began. 21. The implementation period of the project was characterized by protracted institutional instability with respect to government agencies selected to manage the project (see table 1). Following a referendum and the adoption of a new constitution in November 2010 and a series of local, legislative, and presidential elections in early 2011, a new President took office in April 2011. This regime change also brought about the appointment of a new Minister of Finance as well as a change in all senior directors of the ministry, the project coordinator, and the component supervisors of the project unit. At the same time, the Ministry of Economy and Finance (MEF) was split into two separate ministries, the Ministry of Finance (MoF) and the Ministry of Planning (MPRCD), which complicated the institutional management arrangements agreed during project preparation. In particular, the project remained anchored in the Ministry of Finance, with the Secretary General serving as the project coordinator, whereas the Minister of Planning was the World Bank’s Governor who had the responsibility to the coordinate all Bank projects in Niger. 22. In the early stages of project implementation, project management, leadership, and coordination suffered, as the two ministries did not adequately collaborate and there was a constant struggle for resources. As noted in the ICR for the concurrent DPO series (Shared Growth Credits/Grants I-III), the joint structure led to difficulties identifying and implementing reforms as well as coordinating with other sectoral ministries. For instance, there was an inability to operationalize the software to monitor disbursements on external funding due to the divergences between the two ministries, which had a knock-on effect in terms of frictions and tensions at the line- ministry level. However, as implementation progressed, the situation improved, and the Steering and the M&E committees were eventually opened up to the members of both institutions. While the situation appeared to have stabilized later in the implementation period, MEF and MPRCD were again merged in August 2015; only to be separated again into a Ministry of Finance and a Ministry of Planning, following the Presidential elections in the spring of 2016. Table 1: Timeline of Project and Key Institutional Changes Date Key Changes in Institutional and Leadership Arrangements 2009 July, 2009 Board Approval of Project 2010 January, 2010 Project Effectiveness February, 2010 Coup d’état takes place February-May, 2010 OP/BP 7.30 put in place May, 2010 New Minister of Finance and SG appointed July, 2010 Official Project Launch November 2010 New Constitution Enacted 2011 Early, 2011 Presidential and Local Elections April, 2011 New President Takes office 8 April, 2011 MEF Split into Ministry of Finance (MoF) and the Ministry of Planning (MPRCD) September, 2011 New Minister of Finance and SG & New Minister Planning and SG appointed 2012 April 2012 New Minister of Finance and SG appointed June, 2012 Project Mid-term Review 2013 November, 2013 New Assistant Project Coordinator Appointed 2014 February, 2014 New SG Finance appointed December, 2014 Project Restructured 2015 August, 2015 Ministry of Finance (MoF) and the Ministry of Planning (MPRCD) recombined September, 2015 New Minister of Finance & New Minister Planning 2016 April, 2016 Ministry of Finance (MoF) and the Ministry of Planning (MPRCD) split April, 2016 New Minister Planning October, 2016 New Minister of Finance October, 2016 Project Closure 23. As a result of this prolonged institutional instability, the M&E Committee only met on average twice per year (and not at all in 2014), when it was expected that this committee should hold one session every three months. Likewise, there were very few regular and formal meetings between the Project Coordinator and his team including the fiduciary staff, the component supervisors, the focal points, and the administrative staff. ISRs indicate that the Project Coordination Unit, while effective overall, continued to have problems holding regular internal meetings with all stakeholders and tightly monitoring project activities. After the first two years of the project, a new Secretary-General was appointed, and with the help of the assistant project coordinator, implementation improved considerably. In particular, focal points were embedded in various beneficiary units, which were responsible for activity implementation; regular internal meetings with stakeholders were conducted; and activities were closely monitored. 24. Likewise, poor coordination and project management in the early stages of implementation led to intermittent delays of key project activities. In particular, infighting between the Minister of Planning and the Minister of Finance led to a delay with respect to activities related to the implementation of the budget module (CEGIB+), which would allow monitoring of disbursements for externally financed projects. Likewise, while capacity building activities under component one were implemented and disbursed smoothly, IT system activities under component two suffered significant delays given that the procurement process for these IT systems was considerably more complex than anticipated. In particular, there were significant delays in producing the technical specifications and terms of reference for large IT procurements such as the government- wide internet system and some of the larger software packages. At one point, a procurement complaint was lodged by the contractor responsible for the installation of the internet connections in Niamey, which further delayed the roll out of the data transmission network. In order to relieve these bottlenecks, the project recruited an international technical assistant to assist the head of component two to prepare a coherent 9 plan for deployment of different systems as well as prepare the proposed Terms of Reference (TORs). 25. During project implementation, a Mid-Term Review was conducted in June 2012 to do a stock taking of project relevance and performance as well as evaluate what corrective changes might be needed. Shortly after, at the beginning of 2013, it became apparent that slow progress and poor project management and coordination as well as changing project leadership resulted in a lag of the indicators (ISR 6), and for the first time, the project PDO was rated MU. Later that year, it was recommended that the project be restructured in order establish a closer link between activities (inputs) and dependent outcomes. The CMU also acknowledged that the project was unlikely to meet the overly ambitious PDO and that project management had fallen short of expectations (ISR 8), and recommended a restructuring. However, instead of restructuring the project, it was decided that these emerging issues would be addressed in the context of a follow- on PFM and Service Delivery operation, which was approved in 2014 (described in detail below). 26. After a somewhat slow start to the first half of the project, performance generally improved over the second half of the implementation period. While disbursements stagnating at the beginning of the project—standing at just 19 percent after almost two years after project effectiveness—disbursements more than doubled to 43 percent at the end of 2012, 64 percent at the end of 2013, and 79 percent at the end of 2014. This big jump was mainly attributed to the improved implementation of the integrated financial management information systems (IFMIS) component. Six months before the official closing of the project, the disbursement rate was hovering at around 80 percent with a number of key items yet to be completed. Following the introduction of the new project Task Team Leader (TTL) in the middle of 2014, the project was extended through a level II restructuring for a period of 18 months until October 30, 2016, in order to complete the installation of four major information systems, including: (i) customs system migration (SYDONIA World), (ii) payroll; (iii) integration of external investments in the expenditure chain; and (iv) interconnection between the central financial management system with the regions. The project closed on October 30, 2016 with a disbursement rate of 99 percent. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization Overall M&E Rating: Modest M&E Design: Modest 27. While a project theory of change was elaborated in the PAD, there was a significant mismatch between project objectives and the chosen project indicators. For instance, at the PDO level, budget credibility was measured by the ratio of arrears to total expenditures; however, this indicator could be influenced by a number of exogenous macro-economic factors outside the scope of the project, including ongoing budget support operations. Likewise, there was a PDO indicator linked to “improved 10 transparency� (e.g. delay in the submission of financial statements to oversight bodies), yet the PDO formulation targets “internal control� with no mention of budget transparency. Similarly, on the intermediate level, indicators were not clearly aligned with PDO objectives and many focused on outcomes beyond the scope of the project. For instance, intermediate indicators 1 and 2 monitored the deviations in aggregate expenditures and aggregate revenues, which are macroeconomic outcomes significantly beyond the control of a US$10 million technical assistance operation. Likewise, intermediate indicator 5 (timeliness of semiannual budget reports) was selected as an indicator linked to improved internal controls, whereas this is more a measure of Accounting, Recording, and Reporting (accordingly to the PEFA methodology). M&E Implementation: Modest 28. By mainstreaming project management in the MEF, the project intended to support long-term institutionalization of the M&E system for PFM reform implementation. In order to supervise the M&E of the project, the project design included provisions for the recruitment of an M&E specialist, who would be in charge of collecting data, measuring results, conducting seminars, and validating results with various stakeholders. At a higher level, effectiveness conditions were put in place to establish an M&E Committee which would review and validate the reports on performance indicators and recommend corrective action if necessary. However, despite the design of the project, M&E arrangements, which envisioned a robust reporting ecosystem, activities in reality were not executed as originally planned. During implementation, for instance, a dedicated M&E specialist was never appointed and all M&E activities were undertaken by the Project coordinator, who had responsibilities for a variety of other tasks. In addition, the M&E Committee, which was supposed to meet to validate results and make corrective actions, did not met as frequently as stipulated in the project financing agreement. As a result, there were systematic delays in data collection and validation during project implementation, which undermined the monitoring of project performance in real time. M&E Utilization: Modest 29. Shortcomings in indicator selection as well as incomplete and delayed implementation of M&E arrangements necessarily impacted M&E utilization over the life of the project. In particular, systematic delays in updating indicator values (as noted in the ISRs), limited the scope for corrective actions to be taken. Moreover, while problems with project indicators were acknowledged in ISRs as early as 2013, the subsequent project restructuring in December 2014 only extended the project closing date and did not propose any changes to the project results framework or indicators. 2.4 Safeguard and Fiduciary Compliance 30. The project followed a strict project manual of financial, accounting, and administrative procedures and utilized an accounting software (TOMPRO) to meet the requirements of the project, in a manner satisfactory to IDA. The project 11 complied with all fiduciary requirements and submitted all required quarterly Interim Financial Reports (IFRs), although IFRs were submitted with some delay during the early implementation stages of the project. All annual audits were conducted regularly and submitted within six months of the end of the fiscal year, with no evidence of ineligible expenditures. Finally, all legal covenants were complied with during the life of the project, including the recruitment of an external auditor and the adoption of an M&E committee. 31. While the project complied with fiduciary and legal requirements of the Bank, the ex post evaluation noted a potential deviation from Bank policy with respect to safeguard policy. In the PAD, sub-component 2.1 envisions the development of an internet/intranet network to allow for the integration of government public financial management systems. However, it was explicitly mentioned that “the project will not fund the development of a network per se, but will seek to use available services and negotiate payment of monthly or annual fees� (p. 42). Accordingly, it was on this basis that that no safeguards were triggered and the original project TTL and the regional safeguards advisor decided to rate the project with Safeguards Category C. 32. However, during the ICR mission, it was confirmed that IDA funds were used to purchase materials and install a fiber-optic network in Niamey and in four other cities (Dosso, Tahoua, Maradi, and Zinder). In Niamey, this involved the installation of more than 3 kilometers of cable at a depth of one meter in the city center to connect the Ministry of Finance with other institutions, including DGI, DGD, GDTCP, DGCMP, DGIF, as well as an additional 14 ministries, including the President’s and Prime Minister’s office. The installation of these cable connections involved the digging of trenches one-meter-deep as well as using machinery to burrow under and around buildings and streets. For connections to offices further from the city center, radio antennas were installed to allow for the transmission of data. Over the two-and-a-half- year period it took to install the network, it was noted that this caused traffic disruptions in the city center as well as the creation of dust, debris, and noise. In addition, it was noted that the installation involved the temporary relocation of informal vendors (street- side cooks, hawkers, shoe repairmen, etc.) for days and sometimes weeks at a time until the work was completed. 33. Had these activities been appraised at the outset (i.e. during preparation), it would have likely required the Bank to rate the project a Safeguards Category B. Likewise, it would have been necessary for the borrower to prepare a number of safeguards studies, including inter alia, and Environmental and Social Impact Assessment (ESIA) as well as a Resettlement Action Framework (RAF). However, given that these developments occurred after Board approval, the addition of these new activities—not specified in the PAD—should have likely: (i) triggered the safeguards related to Environment Assessment and Involuntary Resettlement; (ii) necessitated a change in Safeguards Category from C to B; and (iii) required the project to be restructured (at level 1, which requires Board approval). However, none of these issues were raised in the process of providing non-objection for the technical specifications or procurement tenders for the internet network, nor were they raised in any ISR or Aide Memoire. 12 2.5 Post-completion Operation/Next Phase 34. The reforms and institutional capacity achieved under the project will be sustained by the support of the Public Sector Capacity and Performance for Service Delivery Project – PCDS (P145261). Originally approved in 2014 and restructured in 2016, PCDS has the objective of strengthening public finance and human resource management to improve service delivery capacity in selected sectors. In particular, the restructured PCDS will build upon results of the project by supporting ongoing PFM reforms as well as building the capacity of the government in the management of public finances. With respect to the former, PCDS will support the ongoing PFM reform agenda through activities aiming to: (i) strengthen tax and custom administration with a special emphasis on VAT reforms; (ii) increase transparency and control in procurement, including publishing of procurement information and audits; and (iii) improve external control. With respect to capacity building, PCDS will support training of public servants in public administration, such as planning, budgeting, budget execution, M&E, procurement, tax management, and customs administration. In doing so, it will continue to support national training institutes (ENAM and FSEJ) in order to improve training customized to the needs of the administration. 35. Likewise, at the policy level, the results will be sustained by the ongoing Second Public Investment Reform Support Credit (PIRSC) DPO series. Among other objectives, the PIRSC series seeks to improve the quality, reliability and accountability of the country’s public financial management and public investment management systems. With respect to the broader PFM agenda, all of the results will continue to be monitored by recurrent PEFA exercises, the most recent of which was conducted in 2016 with the support of the European Union. In particular, PEFA will continue to monitor indicators on budget credibility, execution, transparency, and oversight, which will inform subsequent PFM strategy and engagement. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Relevance of Objectives: Substantial 36. The project was relevant to both Bank and borrower’s strategy documents at appraisal as well as at project closure. During the preparation phase, the project was closely aligned with the Bank’s 2008-2011 Country Assistance Strategy (CAS) for Niger, which in addition to its two strategic objectives of (i) accelerating sustainable economic growth that is equitably shared and (ii) developing human capital through equal access to quality social services, included the cross-cutting theme of promoting good governance to ensure that increased revenues from growth are efficiently spent and broadly shared. Likewise, by simultaneously addressing the improved management of public finances through both improvements to IT and human resources in key budget institutions, the project was designed to support Niger’s PRSP I objectives of (i) accelerating sustained 13 growth that is equitably shared; and (ii) developing human capital through access to quality social services. Finally, the project supported the government’s PEMFAR action plan, which was based largely on the findings of the 2008 PEFA.3 37. At the time of the ICR mission, the project development objectives continue to remain relevant to the government’s 2012 Government Plan for Social and Economic Development (PDES) and the Bank’s 2013/2016 CPF. In particular, the project objectives continue to be aligned with PDES objectives of (i) strengthening the credibility and efficiency of public institutions and (ii) creating the conditions for inclusive, sustainable and balanced development, as well as the cross cutting focus on good governance and building public sector capacity.4 Likewise, the project continues to support the Bank’s 2013/2016 CPF, which aims to (i) promote resilient growth, (ii) reduce vulnerability, and (iii) strengthen governance and capacity for public service delivery across the board. In particular, the project supports a number of critical CPF outcomes, including the consolidation of fiscal performance and improved budget execution and efficiency. Relevance of Design: Modest 38. While well aligned with Country and Bank priorities, the PDO was overly ambitious given the duration and resource envelope. By selectively focusing on project components to build capacity (component 1) and IT systems (component 2)— while leaving out larger civil service HR reform issues and demand driven aspects of PFM reform from project design—it was unrealistic to expect profound changes to the PFM environment in a period of four years. Moreover, with a resource envelope of US$ 10 million, even what was included in the selective “platform approach� could be viewed as overly ambitious given the number of beneficiaries targeted for capacity building as well as acquisition of multiple IT platforms. Finally, given the fragile operating environment and limited institutional capacity to manage and coordinate reform efforts, it was likewise unrealistic to expect the levels of behavioral and incentives change needed to improve budget credibility and control practices over the four-year period. 3 Specifically, the PEMFAR Action Plan formulated ten main recommendations: (i) strengthen links between budget allocations and government priorities defined in PRSP and sector strategies; (ii) identify alternative mechanisms for financing the Treasury (e.g. treasury bills) and the government budget; (iii) improve budget execution, reduce disparities between voted and executed budgets, protect priority expenditures; (iv) simplify and rationalize the chain of expenditure; (v) prepare for a decentralization process through deconcentration; (vi) improve transparency and information systems, and prepare for delegation of commitment authority to line ministries; (vii) gradually integrate externally-financed expenditures into the chain of expenditures; (viii) modernize and restructure the Treasury; (ix) improve accounting practices to provide an accurate picture of government financial and non-financial assets; and (x) strengthen and improve procedures for closing accounts and preparing final government accounts. 4 Other PDES objectives include: (i) Food security and sustainable agricultural development; (ii) Competitive and diversified economy for accelerated, inclusive growth; and (iii) Promotion of social development. 14 39. Likewise, while a project theory of change (results chain) was elaborated in the PAD, there was a significant mismatch between project objectives and the structure of project components. As such, the project objectives of “improved budget credibility� and “improved internal control� were organized across three components, which supported capacity building (component 1), upgrading IT systems (component 2), and project management (component 3). This organization made it difficult to attribute component outputs to the two larger project outcomes; whereas the project design could have been more simply structured around two components directly supporting the two project development objectives. Making the project logic further complex, the original project results framework also specified four intermediate objectives, to which project activities were intended to contribute. These include: (i) strengthened macroeconomic and fiscal projections (outcome 1-A); (ii) adequate controls of non-salary expenditure (outcome 1-B); (iii) effectiveness in the collection of tax payments (outcome 1-C); and (iv) timely generation of budget and financial reports as mandated by existing regulations (outcome 2-A). However, these four intermediate objectives were not well matched to the two PDO objectives of “improved credibility� and “improved internal control� as per PEFA methodology.5 Relevance of Implementation: Substantial 40. In order to ensure that the project remained relevant during the implementation period, the Bank adapted project activities in order to meet emerging priorities. As such, the Bank was able to include activities supporting the swift implementation of the WAEMU directives as well as newly expressed priorities in the Social and Economic Development Plan (PDES) of the Government. For instance, external shocks resulted in a drop in revenues, which made revenue mobilization a critical issue for the government. Accordingly, it was decided to implement SYDONIA World as opposed to SYDONIA ++ as originally planned as well as to extend the closing date to allow for its effective implementation. Likewise, other priorities emerged during the period related to the need to improve PIM management and execution, particularly procurement. Accordingly, the project vigorously supported activities and training related to the rollout of a revamped SIGMAP system. Other priorities were related to the need to accelerate the implementation, transposition, and application of WAEMU directives, which were captured in the design of the follow-on PCDS project in order to ensure that results were sustained in critical areas. While an earlier and more substantial restructuring would have improved the overall design of the project, the follow-on PCDS project was already under preparation at the time when the need for a project restructuring was first noted. Moreover, with more than 80 percent of the funds were disbursed, there was 5 For instance, the results framework notes that credibility will be supported by sub-objectives 1-3; however, it is not clear why, for instance, “adequate controls of non -salary expenditures� (outcome 1-B) would have been included in the results chain for “improved credibility�, while it could have been more appropriately linked to the second outcome of “improved control�. Likewise, “timely generation of budget and financial reports as mandated by existing regulations� (outcome 2 -A), was included as a sub-objective for “improved control� whereas this outcome area is more closely related to the PEFA outcome of improved accounting, recording, and reporting. 15 limited scope for profound modification of the project design. Accordingly, an extension of closing date provided an effective means for concluding key actives, which would be supported by PCDS. 41. In terms of implementation arrangements, the project design envisioned PIU operating within existing structures of the MEP, which would help with capacity development, the strengthening of country systems, transparency, and accountability. However, given that there were a number of institutional changes over the life of the project, significant challenges arose with respect to project management and leadership of the project, which resulted in implementation days. To remedy these delays, the Bank provided substantial technical assistance, at both the operational and technical level, to the PIU in order to ensure that all major project outputs were achieved during the life of the project. 3.2 Achievement of Project Development Objectives 42. The Project Development Objective of the Niger Reform Management and Technical Assistance (RMTA) Project (Project de Renforcement des Capacites - PRC) was to improve: (i) the credibility and reliability of budgets allocated to budget managers; and (ii) the internal controls to hold budget managers accountable. As such, the evaluation of the project development objectives will be based on project- related contributions to (i) “improved budget credibility and reliability� as well as (ii) “improved internal controls.� While the project results framework also noted four intermediate objectives, the overall efficacy of the project will primarily take into account the specific project level objectives included in the PDO—namely “improved budget credibility� and “improved internal control�—while incorporating these intermediate objectives into the larger analysis, but not treating them on individually as they were not discrete objectives. 43. Given the aforementioned shortcomings of the project results framework, the following evaluation will take a two stage approach. The analysis will first take into account the performance of the project indicators at both the PDO and intermediate levels (see table 2). However, given the modest investment of US$10 million as well as the selective approach of the operation, it would difficult to fully attribute the performance of the project to these indicators alone, as they are highly dependent on other external factors, and in some cases, mismatched with the project objectives they were designed to measure. Accordingly, to corroborate the evaluation of the project objectives of “credibility� and “internal control�, secondary evidence will be provided on the basis of successive 2008, 2012, and 2016 PEFA assessments, which provide a broader picture of PFM performance over the period (see table 3). Improved Budget Credibility and Reliability: Modest 44. The project supported a number of measures to improve budget credibility and reliability, as noted in detail in annex 2. In particular, these include capacity building activities, such as the training of MEF staff in order to improve macroeconomic 16 and fiscal forecasting, revenue collection, budget reporting and analysis, procurement, and internal audit and oversight. Project activities also contributed to improved capacity of training institutions themselves, including ENAM and EFEJ, which received support for the development of strategic plans and other IT materials, as well as the initiation of a diaspora technical assistance program. In addition to these capacity building activities, the project helped to revise basic standards and procedures for budget preparation and management, in line with WAEMU standards, in order to improve budget credibility. These include, inter alia, the development of draft macro-fiscal and sector policies; improving the budget calendar to meet existing regulation requirements; formulating a strategy and development plan for the tax and customs directorates (DGI and DGB); development of a strategy, action plan, and internal audit procedures manual for the Inspector General of Finance (DGIF); as well as the elaboration of a strategy and action plan for the Treasury (GDTCP). 45. Despite the attainment of a number of related outputs, there were some shortcomings in the execution of planned activities related to improving budget credibility. These include the limited effectiveness of the training institutions in actually providing full scale implementation of the above mentioned trainings, which as a result, will require further support under the follow-on PCDS project. Likewise, evidence suggests that only 11 experts were recruited under the auspices of the diaspora program, the majority of whom lacked the necessary professional experience to add significant value in critical PFM functions or to support the training programs offered by ENA of FSEJ. Finally, over the life of the project, there has only been limited progress with respect to moving to a multi-year program budgeting system, which will likely be postponed until a later stage in the PFM reform process. 46. Relevant PDO and intermediate indicators suggest that while the project was largely on track to improve budget credibility and reliability, a deterioration in the PFM environment in the final year of the project impacted the final performance of this objective. At the PDO level, the target was achieved with respect to a decrease in the ratio of arrears to total expenditure from 28 percent in 2009 to 4 percent at the end of 2016 (PDO 1). While this shows that the government of Niger now has a more manageable stock of payment arrears, only one of the three associated intermediate indicators reached its end of project targets, suggesting that challenges remain with respect to maintaining budget credibility and reliability. Deviation in aggregate expenditure decreased from 37.0 to 16.5 percent over the life of the project; however, this indicator did not meet its end of project target of less than 5 percent (IRI 1). Likewise, deviation in aggregate revenue for tax and customs directorates began to show an initial decrease; however, by the end of the project, tax revenue deviation was 16.8 percent and customs revenue deviation was 10.6 percent, which marked almost no change from their 2009 baseline values of 21.8 and 11.0 percent, respectively (IRI 2). Despite these challenges, the frequency of complete accounts reconciliation between tax/customs assessments, collections, arrears, records, and receipts increased over the life of the project, and despite a slight deterioration in 2017, reached the final target of annual/monthly publication by the time of the ICR (IRI 4). 17 47. These mixed results in terms of budget credibility and reliability are reflected by equally mixed PEFA results over the 2008-2016 period. Aggregate expenditure outturn has become more realistic in comparison with the originally approved budget, improving from a score of C in 2008 to a score of B in 2016 (PI-1). Likewise, as reflected by the related PDO indicator, the stock and monitoring of expenditure payment arrears have improved from D+ in 2008 and 2012 to a score of C+ in 2016 (PI-4). However, intermediate outcomes related to effectiveness in the collection of tax payments (intermediate outcome 1-c) stagnated—but did not deteriorate—over the period, with aggregate revenue outturn compared to the original approved budget remaining at a score of D (PI-3) as well as effectiveness in the collection of Tax and Customs Payments remaining at a score of D+ (PI-15). Furthermore, the regularity and timeliness of accounts reconciliation decreased, but due to delays in the final year, resulted in a score of D 6 . Finally, the intermediate outcome related to strengthened macroeconomic and fiscal projections (intermediate outcome 1-a) received a PEFA 2016 score of D+, suggesting that significant challenges persist with respect to macroeconomic and budgetary forecasting as well as the capacity to conduct related sensitivity analyses. Improved Internal Controls: Substantial 48. The project has supported a number of measures to improve internal control systems, as noted in annex 2. In particular, the project was able to support the upgrading of a number of financial management IT systems, including those related to: (i) expenditure modules as well as the integration of externally-funded investments in budget information systems; (ii) a customs information systems (SYDONIA World) and related modules; (iii) a new procurement management systems (SIGMAP); as well as (iv) cash management modules to allow for the timely exchange of balance account information between the Treasury (DGTCP) and the Budget directorate (DGB). Underpinning the efficacy of these IT systems have been project outputs related to the improved internet network, which has successfully connected all relevant finance institutions in both Niamey and the regions, and as a result, have enhanced the effectiveness of the expenditure management chain, allowing for better and more timely reporting. 49. Despite these achievements, some challenges remain with respect to maintaining improvements in the internal control environment. These include, inter alia, delayed IT training related to the new procurement management systems and challenges related to behavioral/incentives change given the introduction of the new SYDONIA World system (see section 6 on lesson learned for more details). To remedy these issues, support under the follow-on PCDS project as well as a forthcoming ASA on 6 N.B. While the PEFA score for this indicators remained at D, the intermediate result target was met, since the PEFA exercise took place six months prior to the ICR. 18 customs revenue and informal trade will provide enhanced policy advice on how to better operationalize the system. 50. Given that the project was largely successful in achieving many of the activities and outputs related to improving the internal control environment, all relevant PDO and intermediate indicators were met. In particular, the project was successful in meeting the second PDO indicator related to a reduction in the time taken to submit financial statements to the Chamber of Accounts and National Assembly, from more than 15 months for the publication of the Loi de Reglement and no release of the Comptes de Gestion, to just 6 months for the publication of both the Loi de Reglement and Comptes de Gestion (PDO 2), meeting the end of project targets. Linked to the intermediate outcome area of adequate controls of non-salary expenditure (intermediate outcome 1-b), the degree of compliance with the rules for processing and recording transactions (as measured by a reduction in exceptional spending), met its end of project target of less than 10 percent, as exceptional spending decreased from 23.5 percent in 2009 to just 1.8 percent in 2016 (IRI 3). Likewise, over the life of the project the timeliness of semiannual budget reports has improved considerably from being delivered more than 8 weeks to less than 6 weeks after the end of the relevant quarters (IRI 5) (intermediate outcome 2-a). 51. Similarly, PEFA trends reflect an improvement in the larger internal control environment. PEFA indicators for recording and management of cash balances, debt, and guarantees (PI-17) and the effectiveness of payroll controls (PI-18) have improved from scores of D+ in 2008 to C+ in 2016. Likewise with the revisions to procurement guidelines and introduction of the SIGMAP system, PEFA indicators related to competition, value-for-money, and controls in procurement (PI-19) have increase from B to B+ over the period. While the effectiveness of internal controls for non-salary expenditure (PI-20) remained at C+ over the period, the sub-dimension measuring compliance with rules for processing and recording transactions (which was a core aim of the project) increased from a score of C in 2008 to B in 2016. Finally, PEFA indicators monitoring the quality and timeliness of in-year budget reports (PI-24) and annual financial statements (PI-25) have generally improved. Over the period, the only PEFA control indicator to slightly decline was that related to the effectiveness of internal audit (PI-21), from a score of C in 2008 to D+ in 2016, most likely as a result of the institutional stability during the period. Table 2: Project PDO and Intermediate Indicators 2009 Baseline End 2010 End 2011 End 2012 End 2013 End 2014 End 2015 End 2016 Target Project Development Objective Indicators Ratio of Arrears to Total >10% 29.8% 19.7% 4.5% 4.1% 4.1% 4.7% 4.0% <5% Expenditure (Actual 28%) Time Delay in Submission of >15 months No Data 24 months 18 months 10 months 6 months 6 months 6 months < 9 months Financial Statements to the (Loi de (Loi de Chamber of Accounts and the Reglement) Reglement) National Assembly None (Comptes No Data 10 months 12 months 8 months 6 months 6 months 6 months < 6 months de Gestion) (Comptes de Gestion) Intermediate Results Indicators Deviation in Aggregate >10% -18.6% +16.5% -30.7% +12.3% 17.7% +8.3% +16.5% <5% Expenditure (Actual -37.0%) 19 Deviation in Aggregate >15% -16.0% +15.3% +5.9% +6.7% +8.4% +5.2% 16.8% <5% Revenue for Tax and (tax) (tax) (tax) (tax) (tax) (tax) (tax) (tax) (tax) Customs Directorates (Actual +21.8%) >15% +0.8% +12.3% -0.8% -9.8% -8.7% +36.7% -10.6% <5% (customs) (customs) (customs) (customs) (customs) (customs) (customs) (customs) (customs) (Actual +11.0%) Frequency of Complete Annual Annual Annual Annual Annual Annual Annual Annual Annual Accounts Reconciliation (Tax (Tax between tax/customs Assessments) Assessmen) assessments, collections, Annual Annual Annual Annual Annual Monthly Monthly Monthly Monthly arrears, records, and receipts (Tax (Tax by Treasury Collections) Collections) Annual Annual Annual Annual Annual Monthly Monthly Monthly Monthly (Arrears (Arrears Records Records Annual Annual Annual Annual Annual Monthly Monthly Monthly Monthly (Treasury (Treasury Receipts) Receipts) Degree of Compliance with >35% 30.0% 10.2% 7.0% 3.0% 1.1% 3.5% 1.8% <10% rules for processing and Exceptional Exceptional recording transactions Spending Spending (Actual 23.5%) Timeliness of the Semi- >8 weeks No Data 10 weeks 12weeks 8 weeks 6 weeks 6 weeks 6 weeks < 6 weeks Annual Budget Reports Table 3: Selected PEFA Indicators 2008-2016: Relevant PEFA Indicators 2008 2012 2016* Credibility of the budget PI-1: Aggregate expenditure out-turn compared to original approved budget C D B PI-3: Aggregate revenue out-turn compared to original approved budget D D D PI-4: Stock and monitoring of expenditure payment arrears D+ D+ C+ Predictability and Control in Budget Execution PI-15: Effectiveness in the collection of Tax and Customs Payments D+ D+ D+ PI-17: Recording and management of cash balances, debt and guarantees D+ C C+ PI-18: Effectiveness of payroll controls D+ C C+ PI-19: Competition, value for money and controls in procurement B B+ B+ PI-20: Effectiveness of internal controls for non-salary expenditure C+ C+ C+ (i) Effectiveness of expenditure commitment controls. B B C (ii) Comprehensiveness, relevance and understanding of other internal control rules/ C C C procedures. (iii) Degree of compliance with rules for processing and recording transactions. C C B PI-21: Effectiveness of internal audit C C D+ Accounting, Recording, and Reporting PI-22: Timeliness and regularity of accounts reconciliation D D D PI-24: Quality and timeliness of in-year budget reports D+ C+ B+ PI-25: Quality and timeliness of annual financial statements C+ D+ C+ PEFA 2016 ISSUES Macroeconomic and Fiscal Forecasting** D+ *PEFA 2016 uses an updated methodology to calculate indicator values. Accordingly, to allow for direct comparability between all three PEFA evaluations, the ratings have been standardized based on the 2011 methodology. **Macroeconomic and Fiscal Forecasting was not included as a PEFA indicator until 2016, so scores for the 2008 and 2012 PEFA exercise cannot be presented. Overall Efficacy Rating: Substantial 52. Overall efficacy in achieving project development objectives was substantial. At the project level, both PDO level indictors were met and three out of five intermediate indicators were fully met. Although there were some shortcomings in the objective related to improved budget credibility, this component showed significant improvement over the life of the project, with deteriorations only occurring in the last year of implementation due to exogenous factors related to the the deteriorating security situation and related fiscal pressures (see section 4 on sustainability risks for more details). Despite these setbacks, the objective related to improved internal control was very successful, and 20 all PDO and intermediate indicators met their respective targets, suggesting that the government is now better able to manage and account for the use of public resources. On the whole, the project only suffered moderate shortcomings with respect to its intended development objectives, and for this reason, the overall weighted project efficacy is Substantial. 3.3 Efficiency Overall Efficiency Rating: Modest 53. In terms of overall efficiency, limited quantitative evidence exists to show the economic and financial benefits generated by the project. No quantitative evaluation of efficiency was conducted at appraisal, including, net present value, economic rate of return, cost effectiveness, unit rate norms, service standards, least cost analysis and comparisons, or financial rate of return. As noted in the PAD, the direct and indirect economic and social benefits of this capacity-building project are difficult to quantify. Accordingly, the PAD proposed a number of potential areas where project efficiency can be demonstrated, albeit in qualitative terms, which have been evaluated ex post.  Improved revenue collection and reduced leakages. In comparison with the its predecessor, SYDONIA++, which was originally planned in the project design, the upgrade to SYDONIA World provides the government of Niger with (i) increased customs control capabilities (e.g. risk-assessment, on-line access to external databases, images control); (ii) improved accounting and statistical capabilities, including the development of Post-clearance audit capabilities; as well as (iii) fully automated workflow management, including the use of Electronic Documents and management of the Document Processing Path workflow. As noted by the United Nation’s Conference on Trade and Development (UNCTAD), experience of previous implementation of the system proved that investments are usually amortized in 2 months - 1 year.7 At project closing, the system was only recently put into place, so it is too early to measure these benefits in economic or financial terms; however, in years to come, the new customs administration software is likely to improve inspection and oversight, resulting in greater collections of customs revenues and less leakages.  Medium Term Economic Growth and Budget allocations. The PAD likewise noted that as budget allocations to sector ministries are better aligned with the country’s strategic objectives, the probability of spurring economic growth is likely to increase. As arrears and extraordinary expenditures have been reduced throughout the life of the project, the logic would be that resources can potentially be more directly allocated to priority sectors such as agriculture, education, health, mining, and infrastructure. However, evidence from the most recent PEFA suggests that while 7 http://www.asycuda.org/awbenefits.asp. 21 arrears were reduced from 28 to 4 percent of total expenditures and exceptional spending declined from 23.5 to 1.8 percent of total expenditures, this did not result in an increase in allocations to predefined priority sectors. In fact, sectoral allocations remained relatively constant over the period, with the exception of allocations to the security sector, which increased over the period as a results of deteriorations in the overall security environment (see annex 3).  The PAD noted that considerable public resource savings is likely to occur due to the rationalization and streamlining of budget processes that will be implemented through the project. Indeed, the project has resulted in a reduction in the time taken to submit financial statements to the Chamber of Accounts and National Assembly, from more than 15 months for the publication of the Loi de Reglement and no release of the Comptes de Gestion, to just 6 months for the publication of both the Loi de Reglement and Comptes de Gestion. Likewise, over the life of the project the timeliness of semiannual budget reports has improved considerably from being delivered more than 8 weeks to less than 6 weeks after the end of the relevant quarters. In addition, the PAD predicted that some of the anticipated business processes re-engineering will impact the expenditure and revenue chains, the intranet, rationalization and upgrade of information systems. By reducing transaction costs and improving accuracy, these technical enhancements will likely improve operational efficiency and result in great savings for taxpayers. While customs and procurement IT systems have recently been put into place (SYDONIA World, SIGMAP, etc.), it is still too early to determine the costs savings or efficiency that they can possibly generate.  Improved transparency and accountability of Government decision-making and management of public resources, can improve the business environment for the private sector. As noted in the original PAD, current domestic arrears and perception of corruption in the public sector have had a dampening effect on the role of private sector in the development process of the country. Accordingly, improved government transparency, more largely, may have a signaling effect for the private sectors and create a better environment for investment. As evidenced by several of the results indicators, the government has become more transparent and timely with providing financial statements to oversight bodies. Likewise, arrears were reduced from 28 to 4 percent of total expenditures over the period. However, World Bank Worldwide governance indicators reflect almost no change in Regulatory Quality, Rule of Law, Government Effectiveness or Control of Corruption over the 2009-2015 period. 3.4 Justification of Overall Outcome Rating Overall Outcome Rating: Moderately Satisfactory 54. In order to calculate the overall outcome rating of the project, the ICR team applied IEG’s harmonized rating methodology as follows: 22  The overall rating for project relevance is Substantial. In particular: (i) the project objectives were substantial and remain relevant to the Bank and Borrower at the time of the ICR; (ii) relevance of design was modest given the overly ambitious nature of the PDO and poor links to results indicators; and (iii) relevance of implementation was substantial due to adaptive changes to activities that occurred during the implementation period, which helped to partially compensate for the project design flaws in later stages of the implementation period.  The overall project efficacy rating is Substantial. Both of the PDO indicators were met and three out of five intermediate indicators were fully met. The project only suffered moderate shortcomings with respect to the first objective (credibility), which were largely offset by the strong performance of the second objective (internal control). Given the aforementioned shortcomings related to the selection of project indicators, the success of the project’s objectives was revalidated by largely improved PEFA indicators based on progress from 2008, 2012, and 2016.  The project efficiency rating is Modest. While no data currently exists to measure the project’s benefits in economic or financial terms, in years to come the new customs administration software is likely to improve inspection and oversight, resulting in greater collections of customs revenues and less leakages. 55. On the basis of overall project relevance weighted as Substantial; project efficacy weighted as Substantial; and project efficiency rated Modest; the overall outcome rating of the project is Moderately Satisfactory, in line with IEG harmonized rating criteria noted in table 4 below. Table 4: RMTA Project Ratings Summary Relevance Efficacy Efficiency Overall Objective Design Objective1 Objective 2 Outcome (credibility) (control) Substantial Modest Modest Substantial Modest Moderately Satisfactory 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 55. While it is unlikely that a project of US$ 10 million would have a profound effect with respect to poverty impacts, gender aspects, and social development, the project did contribute to an improvement in government capacity. At a macro level, the project contributes to enhancing the capacity of the government to better manage scarce public resources, particularly with respect to improved human resource capacity, as well as better utilization of IT software to manage public finances. 23 (b) Institutional Change/Strengthening 56. The RMTA project has had a significant impact in terms of strengthening core functions in the Ministry of Finance. Under component one, the project has helped to improve HR capacities in a number of related functions to better forecast, plan, monitor, execute, and report on the budget. Likewise, the project has invested in the capacity of not only individuals, but also core training institutions to provide a sustainable source of capacity building over time. Support to ENA and FSEJ to develop curricula and training capacity relevant to government needs was initiated under the RMTA project and will be continued under the ongoing PDCS operation. Under component two, the project has helped to decentralize the expenditure chain, which will greatly strengthen public finance management in Niger. Until now, access to and processing in the computerized budget execution system was only done through the processing facility (sale pilote) set up the Ministry of Finance. This meant that all budget managers (administrateurs de crédits) and financial controllers (contrôleurs financiers) had to physically come to the Ministry of Finance to process their transactions (e.g. entry of purchasing orders, verification of budget allocation, availability of credits, financial reports, etc.). With the deconcentration of the budget execution system, line ministries are now empowered to manage their finances, which will have a profound change in terms of ownership and accountability. (c) Other Unintended Outcomes and Impacts (positive or negative) 57. Unintended positive outcomes pertain to the rollout of SYDONIA World, which was not initially included in the original project design. While the PAD had only included provisions for SYDONIA ++, the new SYDONIA World modules will allow for greater functionality to: (i) facilitate and improve the calculation; (ii) collect and account Customs duties and other charges related to Customs operations; (iii) speed- up the clearance of goods and prevent smuggling; and (iv) provide the Customs management with timely and accurate information. 58. Unintended negative outcomes relate to potential disturbances caused by the laying of internet fiber/optic cables in Niamey and four other cities. As noted above, these activities were not originally planned in the project design and were only added during the project implementation phase. While it is impossible to provide any evidence of environmental or social impacts ex post, interviews conducted during the time of the ICR mission noted that the installation process caused traffic disruptions in the city center as well as the creation of dust, debris, and noise. In addition, it was noted that the installation involved the temporary relocation of informal vendors (street-side cooks, hawkers, shoe repair men, etc.) for days, and sometimes weeks at a time, until the work was completed. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 59. Since this was not an Intensive Learning ICR (ILI), no beneficiary surveys and/or stakeholder workshops were conducted by the Bank. 24 4. Assessment of Risk to Development Outcome Overall Risk to Development Outcome: Significant 60. Risk to the Development Outcome is Significant as a result of the following large and interconnected risks:  Political and Governance Risk: The security situation remains fragile as Niger is located in an unstable sub-region, surrounded by armed conflict and rebellions. First, the Libyan conflict led to the return of over 100,000 migrants and an outflow of arms which contributed to regional instability. Second, the coup d’état and rebellion in Mali have resulted in an additional inflow of around 64,000 refugees into Niger, compounded existing tensions in the north between Tuareg populations and the Government in Niamey. Finally, Niger has been experiencing attacks from Boko Haram for several years. Two major attacks in June 2016 indicate an increasing security threat, prompting the authorities to seek support from Chad. In addition to the fiscal pressures that the security situation can create, thus undermining progress in the management of a credible budget and a strong internal control environment, political attention may be distracted from the PFM reform agenda as more attention is diverted to managing the domestic security situation.  Macroeconomic Risk: Niger remains vulnerable to volatile commodity prices, an unpredictable climate, and as noted above, a fragile security situation, all of which can significantly derail public finances. First, commodity price shocks may put pressure on the balance of payments and constrain the fiscal envelope, thus increasing the accumulation of arrears. Likewise, a drought could exacerbate the threat of food insecurity and potentially require emergency food imports, and increase budgetary pressures including extraordinary spending. Finally, new and significant deteriorations in the security sector would challenge fiscal stability, by both diminishing public revenues and increasing spending on security as well as supporting refugees and IDPs. Most recently, remittances declined drastically as a result of the massive return of migrants, and pressures were further exacerbated by the financial burden of supporting returnees. The Government has responded to the mounting security issues by stepping up security spending and collaboration with regional and international partners, but this is producing significant fiscal imbalances and slowing down growth.  Institutional Capacity for Implementation and Sustainability Risks: As noted in parallel risk assessments for other governance projects in Niger, the implementation capacity is generally low, and interagency coordination could be significantly improved. However, this risk has been partially mitigated as a result of the capacity building support provided by the RMTA project for core PFM functions. Given that it will be critical to maintain support for relevant training programs and institutions in order not to backslide on the project’s investments, continued capacity development for expenditure and revenue management will be provided by the follow-on PCDS operation, which should mitigate this risk well into the future. Likewise, the 25 concurrent DPF series is working to support improvements to domestic resource mobilization as well as reforms in the real sector. While these parallel operations can help to improve institutional capacity to mitigate fiscal pressures and macroeconomic fragility, as noted above, these categories of risk remain largely out of the control of the government and can result in a significant erosion of PFM gains made under the RMTA project. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry: Rating: Moderately Unsatisfactory 61. The Bank team prepared a project that was relevant to the development objectives of the client at the time of preparation and which responded to CAS priorities. Likewise, the institutional design of the project was adequate given the scope of the operation and proposed beneficiaries, and monitoring and evaluation functions were well defined in the original project documents. However, an overly ambitious project development objective and weak links between the project components and related indicators, represents a significant shortcoming in project design. Moreover, given the limited resource envelope and the project’s selective “platform� approach, some of the project outcomes and related indicators are not directly attributable to the project intervention. While technical needs of the project were sufficiently treated in the design, the project failed to adequately capture the projected financial and economic benefits of the project using a quantitative methodology. Likewise, the project design did not capture the potential environmental and social impacts that the project might generate, nor was there evidence of formal stakeholder consultations during project preparation. Finally, external risks related to macroeconomic and security issues were not assessed in a systematic manner and mitigation measures were not incorporated into project design. Given the generally mixed quality of the project design, the overall rating for Quality at Entry is rated Moderately Unsatisfactory. (b) Quality of Supervision Rating: Moderately Unsatisfactory 62. Project supervision was carried out in earnest on a bi-annual basis throughout the life of the project and all ISRs and Aide Memoires were filed on time. Fiduciary supervision was likewise undertaken semi-annually and all IFRs and audit reports were reviewed and followed-up on when necessary. The Bank was effective at sharing knowledge and providing technical assistance between missions in specific PFM areas—including bringing in treasury, budget, and tax specialists to provide targeted technical assistance to address key bottlenecks—which greatly helped project implementation, especially given the unstable institutional environment. The Bank 26 likewise provided support even when the country was closed for missions by conducting implementation support activities via teleconference. Unfortunately, serious deviations from environmental and social safeguards policy during the implementation period significantly undermined the Bank’s otherwise adequate quality of supervision. In particular, a non-objection was given for the installation of an internet cable network in Niamey and four other cities. As noted above, these physical works were explicitly excluded from the program design; however, they were completed during project implementation without the necessary environmental and social analysis in line with Bank safeguard policy. As noted above, the installation of the fiber-optic cables involved using machinery to dig trenches in Niamey and four regional cities, the installation of several radio antennas, and the temporary relocation of informal venders—any of which could have potentially triggered an inspection panel complaint. In addition to these safeguards issues, the project restructuring only occurred late in the life of project, providing an extension of closing dates to allow for the completion of key activities; however, the restructuring proposed no changes to project indicators, which had been previously identified as problematic. Given this mixed performance, as well as deviation from Bank safeguard policy—which is of paramount significance and could have resulted in serious consequences for both the Bank and Borrower—the overall rating for Quality of Supervision is Moderately Unsatisfactory. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Unsatisfactory 63. Given that Quality and Entry is rated Moderately Unsatisfactory and Quality of Supervision is rated Moderately Unsatisfactory, Overall Bank Performance is rated Moderately Unsatisfactory. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Unsatisfactory 64. As noted above, the project was implemented during a period of significant political and institutional instability, which had significant impacts on the enabling environment. Accordingly, radical changes early in the life of the project necessarily affected government ownership and commitment, as reflected by poor coordination, communication, and overall project management. Despite these initial challenges, evidence suggests that the enabling environment improved significantly during the implementation period, included improvements to project management arrangements. By project closing, improved ownership was noted both on the part of the Ministry of Finance as well as the Ministry of Planning. However, due to the persistent instability, issues emerged with respect to donor coordination as no mechanism for dialogue on PFM reform was in place especially in earlier stages of the project. Consequently, this lack of coordination resulted in some instances of duplicative activities particularly related to 27 customs information systems. Given the shortcomings on the part of the central government agencies, Government Performance is rated Moderately Unsatisfactory. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory 65. Despite the challenging implementation environment, the Project Management Unit within the Ministry of Finance remained the driving force behind the project implementation. During preparation, proper implementation arrangements and key staff were put in place, particularly at the level of the Assistant Project Coordinator, who functioned as the key project manager on a day to day basis, reflecting a high degree of commitment to achieving the development objectives. Throughout the implementation period, all fiduciary issues were dealt with in a timely manner, including, financial management, governance, procurement, reimbursements, and compliance with covenants. Reflecting the high degree of project management at the level of the PIU is a final disbursement rate of more than 99% and the completion of nearly all planned activities and outputs (see annex 2). While there were delays in the early stages of the project, particularly surrounding the generation of relevant Terms of Reference for large IT procurements, as the project progressed, procurement and overall project management improved considerably. The project manager took responsibility for overall monitoring and evaluation of the project results, without the support of a dedicated M&E assistant. Given the challenging enabling environment, and with key challenges acknowledged, Implementing Agency Performance is rated Moderately Satisfactory. (d) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 66. Given that Government Performance is rated Moderately Unsatisfactory and Implementing Agency Performance is rated Moderately Satisfactory, Overall Borrower Performance is rated Moderately Satisfactory since the project outcome was likewise rated Moderately Satisfactory, in line with IEG’s harmonized ratings criteria. 6. Lessons Learned 67. Given both the success and challenges noted during project preparation and implementation of the RMTA project, as well as the depth this ICR took in assessing key problems, a number of lessons can be drawn from the project, which both the Bank and the Government of Niger can utilize to develop a number of targeted lessons learned to inform future operations in the country, PFM sector, as well as FCV environments more largely. 68. A first lesson learned is that knowledge sharing is particularly important in a low capacity environment. In particular, successful and timely implementation of 28 project activities, remains dependent on adequate levels of technical assistance. As detailed above, project implementation was impacted by persistent procurement delays related to the intermittent absence of a dedicated procurement specialist, delays in procurement tenders, delays in the preparation of TORs and technical specifications, and delays in the approval of consultancy works/outputs. By providing the project with an international technical assistant, the Bank was able to expedite the relevant TORs and technical proposals, thus removing a critical bottleneck. In addition to providing this operational assistance, the Bank provided technical assistance where necessary across project components with respect to PFM reform areas. Even when providing assistance remotely (especially when the country was closed for mission travel), the Bank was successful in sharing knowledge, bringing new ideas, and provided alternate approaches, even though the TTLs were not based in the country. 69. A second lesson learned is that capacity building on an institutional level cannot be viewed as just a one-off intervention, but rather, demands ongoing support given that adequate human resources can evaporate quickly. As noted, the project was successful at training MEF staff in order to improve capacities to engage in macroeconomic and fiscal forecasting, revenue collection, budget reporting and analysis, procurement, and internal audit and oversight. However, given the frequent institutional changes in government, which resulted in the periodic reshuffling of department directors and technical specialists, there is a high chance of attrition and/or mismatch between employee competencies and current functions. In addition, as demonstrated by the diaspora program, there is no guarantee that spending resources will translate to long lasting capacity, as it is not guaranteed that the selected candidates will have the appropriate competencies or experience. In order for such initiatives to succeed, it is necessary that such programs be managed in a transparent and meritorious manner, and for the Bank to provide appropriate non-objections of proposed candidates. In order to further capacity development, the Bank will continue to fund sustainable capacity building in Niger by empowering and improving the capacity of training institutions themselves, including ENAMN and FSEJ. It should be noted that capacity building activities in project design need to go beyond producing a manual or a strategic action plan, and need to provide ongoing resources to support implementation. Accordingly, the follow-on PCDS project will continue to support the development of curricula of national training institutes in public administration, economy, and public financial management in order to provide the government with an internal source of capacity development for years to come. 70. A third lesson learned is the importance of appraising all project risks, including exogenous risks which can jeopardize the sustainability of project results. While the PAD captured key operational risks, there was no evaluation of exogenous factors which could negatively affect project results and sustainability. Although the Bank did not have a standardized risk assessment tool in place such as the Operational Risk Assessment Framework (ORAF), many of these external risks would have been foreseeable. In fact, the PAD actually acknowledged that a state of insecurity had persisted since 2007 and this was well documented in the CAS as well; however, no mitigation measures were proposed in the context of the project design. Had these and 29 other related macroeconomic risks been appraised during preparation, the project might have chosen indicators that were not as susceptible to external influences, such as expenditure and revenue deviations, which are easily affected by macroeconomic and security sector developments that cause significant budget deviations. As noted above, a key reason why a number of the indicators were not met is due to the emergence of these risks in the final year of project implementation, which caused a number of targets to be missed. 71. A fourth lesson learned is the importance of incorporating political economy analysis and change management strategies into project design and implementation arrangements. As noted, the project was originally designed to support capacity building in the Ministry of Economy and Finance; however, shortly after the project became effective, the MEF split into the Ministry of Finance and the Ministry of Planning. While the control of the project was retained by the Ministry of Finance, the Minister of Planning served as the Governor of the World Bank and was responsible for the portfolio of Bank projects in Niger. Accordingly, there was a contestation for control over project resources in the first half of the implementation period, which made it difficult for the Bank to balance priorities and manage competing interests. As a result, this may have had the perverse effect of steering decisions on the implementation of the project in the direction of what was political expedient as opposed to what was operationally sound. In the future, such situations could be avoided by conducing the correct political economy assessments ex ante as well as applying change management and Problem Driven Iterative Adaptation approaches—to the extent possible—during project implementation. 72. A fifth lesson learned is that project monitoring and evaluation is foundational, and failure to choose appropriate indicators can impact the overall performance of a project. As evidenced in section 3.2 on the achievement of the project development objectives, project indicators showed an improvement in the internal control environment whereas PEFA indicators showed a stagnation, and in some areas such as internal audit, a decline. The reason for this disconnect, as mentioned above, is that indicators chosen to monitor the outcome of internal control were actually indicators more suited to monitoring transparency, accounting, reporting, recording, and reporting (as per the PEFA methodology). Had more appropriate indicators been linked to this project outcome, it is likely that this disconnect would not have occurred between the indicators and the larger PFM environment, which would have allowed for better monitoring of results. In the case of PFM projects in fragile states like Niger, indicators and the development objectives should be scrutinized more carefully during project preparation and effectively tempered. Effective PFM changes occur over time even in more stable environments and cannot be expected to quickly occur in environments with weak governance structures. Furthermore, in fragile states it may be more appropriate to include indicators that measure whether there is more ministerial cohesion, communication, or positive institutional inclusion. Much of the positive impact from PFM related projects in fragile environments is not necessarily tied to producing budget reports or reducing budget outturns, but from increasing the communication and work flow between ministerial departments as well as between ministries. 30 73. A sixth lesson learned is that a lack of donor coordination and commitment around the PFM agenda can affect key intended outcomes. In the case of the RMTA project, there was no effective mechanism for donor coordination and the dialogue was not led by the government, who has ultimate responsibility for this function. To remedy this, appropriate donor coordination should occur at three levels. First, it is necessary that the Bank is internally coherent in its donor approach to donor engagement. Second, at the donor level, it is necessary that the government take the lead in driving the discussions and ensuring that mutually reinforcing programs are in place. Third, and finally, it is necessary that the Bank and donors have an effective relationship to intermediate the supply and demand side conditions regarding certain reforms, so that there are no duplications or gaps in critical areas. 74. A seventh lesson learned is that small reform projects such as the RMTA will only have a minimum impact unless they address more nuanced problems or are able to mobilize additional donor resources. A key limitation of the RMTA project is that its resource envelop did not allow it to fully meet the ambitious developments objectives set forth. To assume that an envelope of US$ 10 million would allow for large scale improvement to budget credibility and control ignores the larger macroeconomic context as well as the prevalence of other exogenous issues—particularly the inflamed security situation in Niger—which can greatly affect the project results and sustainability. Given this lesson learned, the IDA 18 cycle will seek to focus more particularly on fragile environments by scaling up its resource allocation as well as focusing on governance and institutions as a key issue. Underpinning this thematic focus is a “cascading� approach, which seeks to use Bank and Donor funds as a means to leverage private sector contributions to key areas where public financing is not essential. 75. An eighth lesson learned is that it is necessary to differentiate the experience of countries where OP 7.30 was in place versus environments where there has been an ongoing FCV environment. For instance, due to the coup d’état that occurred in 2009, Niger was placed under OP 7.30 (Dealings with De Facto Governments) for a period of four months. However, during this time, and until the start of the IDA 18 cycle, Niger was never declared a Fragile and Conflict Affect State. While many FCS recommendations resonate with Niger’s context, key lessons can be drawn from dealing with de facto governments under OP 7.30. Key among these is the need to engage, to the extent possible, with the existing government in order to maintain reform dialogue. Once the 7.30 condition is lifted, it is incumbent upon the Bank to rapidly reassess the priorities of the new government and ensure that that PDO matches the priorities and political settlement that emerges. In the case of the RMTA project, the project was never restructured after the coup d’état, which changed many of the assumptions underpinning the agreed institutional arrangements, and led to significant delay in the early implementation of the project. 76. A ninth lesson learned is that IT—and other largescale and expensive systems changes—need to be underpinned by addressing incentive and behavioral changes. Drawing on key messages of the 2017 World Development Report on Governance and the Law, it is necessary to “think not only about the form of institutions, 31 but also about their functions.� As such, the implementation of systems like SYDONIA World may only serve to formalize otherwise inadequate processes. For instance, even though the IT infrastructure has been put into place, some practices continue to prevail, such as the existence of “common rooms� at customs offices where brokers come to declare; no risk analysis system, which means that virtually all declarations are subject to physical inspection but due to this, real controls are scarce and results of physical checks are not recoded; no monitoring indicators are produced on a regular basis; and a large amount of declarations are declared with an unidentified fiscal number (9999) and despite regulations. Given this situation, it is apparent that despite largescale IT changes, civil servants’ incentives and behaviors take longer to change as a result of imbedded processes, which are counterproductive to reform objectives. Accordingly, the key lesson learned in this respect is that IT improvement does not necessarily change behaviors of civil servants if the IT upgrade only reproduces manual procedures. Problem Driven Iterative Adaptation provides a way forward in this case, and existing process level issues are currently being dealt with in the follow-on PCDS operation. 77. A tenth and final lesson learned is that the verdict is still out with respect to mainstreaming or creating stand-alone PIUs in FCV environments. Best practice FCV literature warns against creating multiple structure in favor of creating project units within existing institutions—and this point is well taken as it substitutes local capacity building opportunities with international “quick fixes�. However, in the case of the RMTA project, continued institutional instability undermined the efficacy of the PIU in managing the project, at least in the first half of the implementation period. While the PIU embedded in the MEF created a great deal of origination and buy-in in the reform agenda, subsequent reshuffling of government arrangement made it difficult for it to carry out its role on the ministerial level. Given this experience, a key lesson learned is that the placement and training of the project management unit is especially important in establishing good project outcomes—and that this remains context specific and such be iteratively adapted based on changing circumstances. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies 78. The Borrower provided the Bank with a self-assessment of project performance in the form of a Borrower Completion Report (executive summary provided in annex 7). Likewise, the RMTA Project Coordinator provided written comments on the Bank ICR, which have been taken into consideration in this final draft. (b) Cofinanciers 79. Not Applicable. (d) Other partners and stakeholders 80. None Provided. 32 33 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent)8 Appraisal Actual/Latest Components Estimate (USD Estimate (USD Percentage of Appraisal millions) millions) 3,055,082.87 101.8% Component 1: Capacity Development 3,000,000.00  Sub-component 1.1: Strengthening 2,594,533.14 173.0% 1,500,000.00 Capacity of Key Functions of MEF 141,121.58  Sub-component 1.2: Support to 14.1% 1,000,000.00 Local Training Institutions 55.2%  Sub-component 1.3: Support to the 275,875.94 500,000.00 Diaspora program Component 2: Leverage and Rationalize 2,965,420.25 59.3% Financial Management Information 5,000,000.00 Systems  Sub-component 2.1: Intranet/Intranet 2,511,368.75 100.5% 2,500,000.00 Deployment  Sub-component 2.2: Systems 274,265.47 13.7% 2,000,000.00 Rationalization and Upgrade 179,786.03 36.0%  Sub-component 2.3: Office 500,000.00 Equipment Upgrade Component 3: Support to the Project 76.4% 1,500,000.00 1,146,280.02 Coordination Unit Total Baseline Cost 9,500,000.00 7,166,783.15 76.4% Physical Contingencies 500,000.00 0.00 0.00 Price Contingencies 0.00 0.00 0.00 Total Project Costs 10,000,000.00 0.00 Front-end fee PPF 0.00 0.00 .00 8 The observed discrepancy between Actual/Latest Estimate shown here and the final disbursement position shown in table (b) is due to currency fluctuations, and incomplete reporting within the Project. The Actual/Latest Estimate in Table (b) of USD 9.06 million accurately reflects the Project’s actual expenditures. 34 Front-end fee IBRD 0.00 0.00 .00 Total Financing Required 10,000,000.00 7,166,783.15 71.7% (b) Financing Appraisal Actual/Latest Type of Percentage of Source of Funds Estimate Estimate Cofinancing Appraisal (USD millions) (USD millions) Borrower 0.00 0.00 .00 IDA Grant 10.00 9.06 90.6% Annex 2. Outputs by Component Expected Outputs by Component Achievement of Expected Outputs  Component 1: Capacity Development (US$3 million):  Sub-component 1.1: Strengthening Capacity of Key Functions of MEF (US$ 1.5 million): Improving Macro and revenue forecasts for credible multi-year budget planning Development of a macro-fiscal and sector policies Achieved. A consultant was hired with the support of the project to provide the relevant diagnostics and the recommendations were included in the Niger 2021 strategy. Improved fiscal framework and revenue forecasts Partially Achieved. Human Resource Capacity and relevant IT systems have been put into place; however, recent shocks related to the domestic security system have caused unpredictable deviations in revenue forecasts as well put additional pressure on the budget. Moreover, more work can be done to increase risk management capacities in fiscal forecasting. Development of multiyear program budgets and Partially Achieved. Project support helped to develop strategic alignment guidelines for preparing program budgets at both the sectoral and global levels. Program budgets have been prepared in 9 ministries, including among others, Education, Health, and Finance. However, this process remains incomplete and will be support under the PCDS project. Improved budget calendar to meet existing Partially Achieved. Capacity building support was regulation requirements provided for the timelier delivery of the loi de reglements and comptes de gestion to the Cour de Comptes, which has allowed the government to meet regulation requirements set forth in the oranic Loi de Finance of 2012. However, recent slippages have occurred particularly in 2016 as noted in the results framework. Training of staff in micro-fiscal forecasting, PIM, Partially Achieved. Training has taken place in a program budgeting, and information systems variety of related topics to macroeconomic planning and forecasting as well as the related information systems. However, this process remains ongoing and will be support under the PCDS project. 35 Expected Outputs by Component Achievement of Expected Outputs Training of senior staff in leadership, change Achieved. Staff have been training in a number of management, project management, related competencies based on specific needs communications, and information systems including project management and audit, public policy evaluation, and writing TORs among others. Study tours to neighboring countries to learn about Achieved. Study tours and internships have been MTEFs and linking MTEFs to PSRPs. offered to provide relevant peer learning opportunities. Improving Revenue Collection Formulating of a strategy and development plan Achieved. The project supported the development of a for DGI and DGD human resources needs assessment and related training plan, including strategic planning, organizational audits, and related studies. Training of DGI and DGD staff on various aspects Achieved. Basic trainings have started for these staff of fiscal revenues and operational management and will be continued under the support of the PCDS follow-on operation. Improving Budget Reporting and Analysis Strengthened reporting, analysis, and monitoring Partially Achieved. Relevant texts and directives have functions by sector type to move from line-item to been put in place in line with the Loi Organic des program budgeting Finances 2012 to strengthen reporting and internal control functions; however, there has yet to be a full scale adoption of program budgeting. Improving Procurement Management Training to DGCMP staff on public procurement Partially Achieved. The project financed the management and statistical analysis development of a manual for procurement procedures; however, as of project closing the manual only in draft form and full scale training has yet to take place. Development of a new procurement management Achieved. The SIGMAP system was rolled out in system within DGCMP October 2016; full scale training has yet to take place. Improving Internal Audit and Financial Oversight Development of a strategy and action plan for the Achieved. A strategy for internal audited and financial DGIF directorate oversight was put in place with the support of the project. Development of an internal audit handbook and Achieved. A procedures manual for internal audit was procedures Manuel developed with the support of the project. Establishment of special status for financial Partially Achieved. A procedure for internal audit inspectors to protect them from retaliation was put into place with the support of the project, but significant weaknesses exists with respect to the role of the DGIF. Training for financial controllers and inspectors Partially Achieved. 30 individuals were trained in project audit and inspection functions; however, continuing training will be put into place under the PCDs project. Introduction of assets accounting Not Achieved. No evidence was obtained or provided on the achievement of this output. Improving Cash Management and Supporting Treasury Reform Elaboration of a strategy and action plan for Partially Achieved. With the support of the project a DGTCP study has been undertaken to identify needs which will identify the major actions needed in the future to improve the public accounting capacity. Technical Assistance for implementation of the Achieved. Technical Assistance and training was strategy and action plan provided on a manual of financial controls, accounting procedures, and WAEMU procedures. Training of public accountants and treasury staff Partially Achieved. Training has commenced; however, continuing training will be put into place 36 Expected Outputs by Component Achievement of Expected Outputs under the PCDS project. Drafting and adoption of an instruction governing Not Achieved. No evidence was obtained or provided relations between the centralizing accounts on the achievement of this output. (ACCT-GTA and PGT) and non-centralized accounts Update of the 2003 Chart of accounts to the new Not Achieved. No evidence was obtained or provided entities envisioned by the reforms on the achievement of this output. Upgrade of the Treasury and Accounting Achieved. Treasury and accounting systems have been Information Systems linked as a results of the in connections between various government departments linking the whole expenditure chain as well as the creation of a centralized data center.  Sub-component 1.2: Support to Local Training Institutions (US$ 1.0 million): Capacity needs assessment, strategic update, and Achieved. A capacity needs assessment was developments plan for each institution completed for ENAM and FSEJ and study tours were taken to comparable university in Togo, Senegal, and Benin. Design of new curriculum and courses for MEF Partially Achieved. While updates to the curriculum trainees were made, continuing curriculum development will be provided under PCDS. Limited hardware and software equipment Partially Achieved. Relevant materials were acquired, but will continued to be provided by the follow-on PCDS project. Books and review materials Partially Achieved. Relevant materials were acquired, but will continued to be provided by the follow-on PCDS project. Integration of Diaspora experts into training Not Achieved. Evidence suggest that diaspora experts programs only provided ministry-specific support and nothing at the level of ENAM or FSEJ.  Sub-component 1.3: Support to the Diaspora program (US$ 0.5 million ): Formulation of engagement policies for work Partially Achieved. Polices were developed but not contracting with diaspora experts fully implemented to engage the diaspora community. Identification of Potential Candidates through Partially Achieved. Candidates were identified, but embassy contacts and networks only a limited number of diaspora experts were recruited, including 4 policy experts to assist with negotiations, macroeconomics, and financing structures as well as 3 IT experts to provide ongoing IT support and training. Ticket Payment and ancillary fees to facilitate Achieved. Tickets and ancillary contract payments short term return were covered under the project.  Component 2: Leverage and Rationalize Financial Management Information Systems (US$5.0 million).  Sub-component 2.1: Intranet/Intranet Deployment (US$2.5 million). Network Extension Negotiate the cost of data transfer with providers Achieved. Contract agreements with SONOTEL were to provide network coverage across the country concluded to provide necessary usage bandwidths to connect Niamey with regional capitals. Acquisition of security and connecting equipment Achieved. Relevant security and connecting materials were acquired and installed to connect all relevant MDAs within Niamey to digitize the expenditure chain, including necessary subterranean wring and antennas. Network training for maintenance staff Partially Achieved. Network Training was provided 37 Expected Outputs by Component Achievement of Expected Outputs under relevant services agreements with SONATEL as well as through the diaspora program. However, since PRC has closed, ongoing support will be needed to complete these activities. Intranet Database applications and servers for intranet to Achieved. Applications and servers were acquired; serves as a collaboration tool with the Diaspora however, connectivity with the diaspora remains limited. Content generation and capacity development of Partially Achieved. While a basic intranet has been webmasters and communications specialist established; there has been limited capacity development for content developers, webmasters, and communications specialists. Study visit to another francophone country with a Not Achieved. No evidence was obtained or provided good intranet in place at MEF. on the achievement of this output.  Sub-component 2.2: Systems Rationalization and Upgrade (US$2.0 million). Upgrade of Budget Information Systems Upgrade of Expenditure Module Achieved. Module was put in place under the PRC project restructuring and extension. Upgrade of Payroll Module Not Achieved. Module will be put in place under the follow-on PDCD project. Integration of externally-funded investment Partially Achieved. The CEGIB+ application has expenditures into (CCD) into the chain of been acquired, but it will be deployed with the support expenditures. of the follow-on PCDS operation. Interconnection with other Satellite applications Achieved. Contract agreements with SONOTEL were (DGCMP) which will link central financial concluded to provide necessary usage bandwidths to management systems with the regions connect Niamey with regional capitals. Relevant security and connecting materials were acquired and installed to connect all relevant MDAs within Niamey to digitize the expenditure chain, including necessary subterranean wring and antennas. Upgrade of Customs Information Systems Rationalize customs processes (BPR) Not Achieved. While the SYDONIA World System has been installed and relevant trainings conducted, problems remain with internal processes as well as meeting WTO standards. Implement key modules (transit, sampling, Achieved. The project restructuring allowed for the manifest, etc.) full installation of the SYDONIA World System. Electronic transfer of customs revenue data from Partially Achieved. The project restructuring allowed DGD to other information’s systems, including for the full installation of the SYDONIA World treasury, DGI, and DGB System, including key modules. Improvements to the connective IT structure will ensure that electronic transfer of customs information occurs, but no evidence to suggest that this is done on a full scale basis. Rollout of SYDONIA World Achieved. The project restructuring allowed for the full installation of the SYDONIA World System. In addition, this included the training of 145 individuals. However, this work is still ongoing and will be supported by ENAM and FSEJ under the follow-on operation. Upgrade of Tax Authority Information Systems Review and study different options for system Partially Achieved. The development of the SISIC design and the DGI is considering system was subsequently picked up by the AfDB. 38 Expected Outputs by Component Achievement of Expected Outputs Upgrade of Cash Management System and Support of the Treasury Reform Update of new government accounting chart and Partially Achieved. The project supported budgetary the new budgetary nomenclature nomenclature in line with WAEMU standards. Timely and accurate transfer of revenue collection Partially Achieved. While this started to be done on a data from revenue agencies to the Treasury timely basis, the most recent results from 2016 suggests that significant slippages have occurred. Complete and timely exchange of balance account Partially Achieved. While this started to be done on a information between Treasury and DGB timely basis, the most recent results from 2016 suggests that significant slippages have occurred. Staff Training Partially Achieved. Basic trainings have started for these staff and will be continued under the support of the PCDS follow-on operation. Component 3: Support to the Project Coordination (US$ 1.5 million) Development of annual Work programs, Achieved. Appropriate resources were provided for procedures manuals, and procurement plans the achievement and necessary activities were undertaken for the achievement of this output. Conduct fiduciary and monitoring activities Achieved. Appropriate resources were provided for the achievement and necessary activities were undertaken for the achievement of this output. Coordinate technical work related to the project Achieved. Appropriate resources were provided for components the achievement and necessary activities were undertaken for the achievement of this output. Monitoring project preparation and Achieved. Appropriate resources were provided for implementation the achievement and necessary activities were undertaken for the achievement of this output. Office equipment, accounting software, Achieved. Appropriate resources were provided for incremental costs the achievement and necessary activities were undertaken for the achievement of this output. Figure 2: Results Chain for the PRC Project 39 Annex 3. Economic and Financial Analysis 56. In terms of overall efficiency, limited quantitative evidence exists to show the economic and financial benefits generated by the project. No quantitative evaluation of efficiency was conducted at appraisal, including, net present value, economic rate of return, cost effectiveness, unit rate norms, service standards, least cost analysis and comparisons, or financial rate of return. As noted in the PAD, the direct and indirect economic and social benefits of this capacity-building project are difficult to quantify. Accordingly, the PAD proposed a number of potential areas where project efficiency can be demonstrated, albeit in qualitative terms, which have been evaluated ex post.  Improved revenue collection and reduced leakages. The installation and implementation of the SYDONIA World system for customs administration has the potential to (i) facilitate and improve the calculation, collection, and accounting of Customs duties and other charges related to Customs operations; (ii) speed-up the clearance of goods and prevent smuggling; and (iii) provide the Customs management with more timely and accurate information. In comparison with the its predecessor, SYDONIA++, which was originally planned in the project design, the 40 upgrade to SYDONIA World provides the government of Niger with (i) increased customs control capabilities (e.g. risk-assessment, on-line access to external databases, images control); (ii) improved accounting and statistical capabilities, including the development of Post-clearance audit capabilities; as well as (iii) fully automated workflow management, including the use of Electronic Documents and management of the Document Processing Path workflow. As noted by UNCTAD, experience of previous implementation of the system proved that investments are usually amortized in 2 months - 1 year.9 At project closing, the system was only recently put into place, so it is too early to measure these benefits in economic or financial terms; however, in years to come, the new customs administration software is likely to improve inspection and oversight, resulting in greater collections of customs revenues and less leakages.  Medium Term Economic Growth and Budget allocations. The PAD likewise noted that as budget allocations to sector ministries are better aligned with the country’s strategic objectives, the probability of spurring economic growth is likely to increase. As arrears and extraordinary expenditures have been reduced throughout the life of the project, the logic would be that resources can potentially be more directly allocated to priority sectors such as agriculture, education, health, mining, and infrastructure. However, evidence from the most recent PEFA suggests that while arrears were reduced from 28 to 4 percent of total expenditures and exceptional spending declined from 23.5 to 1.8 percent of total expenditures, this did not result in an increase in allocations to predefined priority sectors. In fact, sectoral allocations remained relatively constant over the period, with the exception of allocations to the security sector, which increased over the period as a results of deteriorations in the overall security environment (see table 4). Table 4: Budget Allocations by Sector (2013-2015) 9 http://www.asycuda.org/awbenefits.asp. 41  The PAD noted that considerable public resource savings is likely to occur due to the rationalization and streamlining of budget processes that will be implemented through the project. Indeed, the project has resulted in a reduction in the time taken to submit financial statements to the Chamber of Accounts and National Assembly, from more than 15 months for the publication of the Loi de Reglement and no release of the Comptes de Gestion, to just 6 months for the publication of both the Loi de Reglement and Comptes de Gestion. Likewise, over the life of the project the timeliness of semiannual budget reports has improved considerably from being delivered more than 8 weeks to less than 6 weeks after the end of the relevant quarters. In addition, the PAD predicted that some of the anticipated business processes re-engineering will impact the expenditure and revenue chains, the intranet, rationalization and upgrade of information systems. By reducing transaction costs and improving accuracy, these technical enhancements will likely improve operational efficiency and result in great savings for taxpayers. While customs and procurement IT systems have recently been put into place (SYDONIA World, SIGMAP, etc.), it is still too early to determine the costs savings or efficiency that they can possibly generate.  Improved transparency and accountability of Government decision-making and management of public resources, can improve the business environment for the private sector. As noted in the original PAD, current domestic arrears and perception of corruption in the public sector have had a dampening effect on the role of private sector in the development process of the country. Accordingly, improved government transparency, more largely, may have a signaling effect for the private sectors and create a better environment for investment. As evidenced by several of the results indicators, the government has become more transparent and timely with providing financial statements to oversight bodies. Likewise, arrears were reduced from 28 to 4 percent of total expenditures over the period. However, World Bank Worldwide governance indicators reflect almost no change in Regulatory Quality, Rule of Law, Government Effectiveness or Control of Corruption over the 2009-2015 period. 42 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Helene Bertaud Lead Counsel LEGES Lead Counsel Senior Finance Wolfgang M. T. Chadab Senior Finance Officer WFALA Officer Lead Governance Helene Marie Grandvoinnet Lead Governance Specialist GGO13 Specialist Robert A. Yungu Senior Public Sector Specialist GGO13 Team Leader Catherine Laurent Senior Public Sector Specialist AFTPM PFM Specialist Macmillan I. Anyanwu Senior Country Officer SACAA Operations Samia Melhem Lead ICT Policy Specialist GTI09 ICT Specialist Yang-Hah Chung-Kong Program Assistant AFTPM Program Assistant (b) Supervision/ICR Robert A. Yungu Senior Public Sector Specialist GGO13 Team Leader Macmillan I. Anyanwu Senior Country Officer SACAA Operations Public Sector Governance Bepio C. Bado GGO13 Governance Consultant Samia Melhem Lead ICT Policy Specialist GTI09 ICT Specialist Immanuel F. Steinhilper Senior Public Sector Specialist GGO18 Governance Alexandre Arrobio Practice Manager GGO18 Governance Michel R. Mallberg Senior Public Sector Specialist GGO13 Team Leader Public Sector Governance Hajarivony Andriamarofara GGO13 Governance Consultant Public Sector Governance Heriniaina M. Andrianasy GGO13 Governance Specialist Public Sector Governance Dolele Sylla GGO13 ICT Specialist Specialist Public Sector Governance ICR Primary Michael Jelenic GGODR Consultant Author Anne-Lucie Lefebvre Senior Public Sector Specialist GGO13 ICR Team Leader (c) Staff Time and Cost Staff Time and Cost (Bank Budget Only) USD Thousands Stage of Project Cycle No. of staff weeks (including travel and consultant costs) 43 Lending FY2008 22.72 119.19 FY2009 53.64 323.12 FY2010 4.74 19.71 Total: 81.10 462.02 Supervision/ICR FY2010 17.34 98.08 FY2011 17.40 86.93 FY2012 11.72 75.69 FY2013 18.23 97.06 FY2014 10.93 91.74 FY2015 10.13 64.11 FY2016 15.70 106.01 FY2017 8.00 81.85 Total: 109.45 701.48 44 Annex 5. Beneficiary Survey Results 1. Not Applicable as this was not an ILI ICR. 45 Annex 6. Stakeholder Workshop Report and Results 1. Not Applicable as this was not an ILI ICR. 46 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR (Executive Summary) 1. Le Projet de Renforcement des Capacités (PRC) a connu un retard au démarrage dû à la méconnaissance des procédures de gestion de projet appliquées par la Banque mondiale. La formulation, en 2009, a été réalisée sans rapprochement avec le plan stratégique de réforme de la gestion des finances publiques qui n’a été validé, entre le Gouvernement et le FMI, qu’en 2012. Les objectifs de renforcement des capacités étaient pertinents en raison des besoins de formations des personnels chargés de la GFP du Niger et en raison de l’obsolescence des équipements et réseaux informatiques et bureautiques. 2. En termes de Pertinence, le PRC s’est relativement inscrit dans le PDES, le PRGFP et dans les stratégies des PTF (sans être vraiment explicite). Le projet était raisonnablement compatible avec les engagements d'efficacité de l'aide et très pertinent pour réaliser les besoins des groupes cibles. La logique d'intervention a été adéquate, bien qu'elle aurait pu faire l'objet d’amélioration en matière de cadre logique, notamment une meilleure hiérarchie des Objectifs, Indicateurs, actions, activités et taches. Les principales parties prenantes ont compris et participé raisonnablement à la conception; des mesures correctives des actions ont été prévues, mais pas totalement appliquées. Les aspects transversaux les plus importants étaient intégrés dans la conception, mais des améliorations pouvaient être apportées. 3. En termes d’Efficacité, la plupart des effets directs sont de bonne qualité, disponibles et utilisés par la plupart des groupes cibles. Des améliorations auraient pu toutefois être apportées. Les objectifs à moyen terme peuvent être atteints avec des restrictions mineures ; les effets négatifs n'ont pas tellement porté préjudice. 4. En termes d’Efficience, La plupart des ressources ont été disponibles dans un temps raisonnable et n’ont nécessité aucun ajustement budgétaire conséquent . La gestion a été raisonnablement transparente et a respecté les procédures de la Banque mondiale. Toutefois, des améliorations auraient pu être apportées en matière de reportage. Malgré les retards de démarrage, la plupart des activités ont été mises en œuvre dans les temps, notamment celles basées sur des résultats attendus possibles liées à une logique d'intervention indiscutable. Certaines activités trop ambitieuses restent en suspens, mais ne portent pas préjudice à la fourniture des produits car elles n’étaient pas techniquement et juridiquement réalisables pendant la durée du projet. La fourniture des produits est conforme au projet, mais une amélioration aurait pu être apportée en termes de qualité, de couverture et de timing. Les structures interinstitutionnelles mises en place ont été raisonnablement effectives, malgré un cadre général de suivi-reportage et d’évaluation inadéquat 10 . Les partenaires ont rempli généralement leur rôle. Toutefois, certaines 10 Voir le cadre de suivi-reportage et d’évaluation de la mise en œuvre du PRGFP, proposé au point « Conclusions et recommandations ». 47 améliorations auraient pu être apportées à l’efficience des formations ENAM et FSEJ. A l’issue des entretiens avec les responsables des deux institutions de formation partenaires, ces derniers ont reconnu que le niveau des enseignements en gestion des finances publiques devrait être amélioré, notamment en matière de réforme de la GAR. 5. En termes d’Impact, il est généralement difficile d’évaluer en fin de projet, quels sont les impacts mesurables. Les impacts des actions n’ont pas toujours correspondu entièrement aux résultats attendus et par conséquent, certains objectifs n’ont pas été atteints totalement. Si le manque de coordination des PTF, lié à l’absence d’unités fonctionnelle de suivi-reportage-et d’évaluation du PRGFP devait perdurer, ce dysfonctionnement majeur pourrait mener à un impact peu significatif. Globalement, les livraisons d’équipements bureautiques et informatiques ont apporté une amélioration immédiate des conditions de travail des fonctionnaires. En revanche, les formations assurées, non diplômantes, auront un impact très limité sur les plans de carrières des agents bénéficiaires, au mieux un impact de vulgarisation. 6. En termes de Viabilité, les formations et les équipements livrés constituent un socle solide de viabilité. La mise en œuvre du PRC a été basée en grande partie sur des structures et institutions qui sont également impliquées à un certain degré dans le processus décisionnel. Pour la suite des appuis (PCDS, Appuis budgétaires, PORCAP, Autres appuis), la probabilité de viabilité est bonne, mais une amélioration structurelle devrait être apportée, en désignant des points focaux dans chaque institution opérationnelle, uniquement chargés du suivi-reportage de tous les projets en appuis directs ou indirects. La sphère politique et les institutions chargées de la mise en œuvre de la politique de réforme de la GFP ont généralement soutenu le programme ou n'ont, du moins, pas entravé son bon déroulement et devraient probablement continuer à le faire afin d’assurer la mise en œuvre du PRGFP. En conclusion, les résultats de la mise en œuvre du PRC sont globalement positifs en termes de renforcement des capacités, d’amélioration des conditions de travail des opérateurs de la GFP et de modernisation du système d’information. 48 EXECUTIVE SUMMARY (UNOFFICIAL ENGLISH TRANSLATION) 7. The Capacity Building Project (PRC) experienced delays at the beginning of implementation due to the Client’s lack of knowledge of the project management procedures applied by the World Bank. In 2009, the project design was carried out in the absence of a public financial management (PFM) strategic plan, which was only validated by the Government and the IMF in 2012. The project’s objectives in terms of capacity-building were relevant and in line with Niger’s PFM staff training needs and took into consideration the IT, office equipment, and network needs. 8. In terms of Relevance, the PRC has been relatively integrated into the Social and Economic Development Plan (PDES) and donors’ strategies (but not explicitly). Key stakeholders understood and reasonably participated in the design. Corrective actions have been foreseen but were not fully implemented. The most important cross- cutting aspects were integrated into the design, but improvements could have been made. 9. In terms of Effectiveness, most direct effects of the project are of good quality, available, and used by most target groups. Improvements could, however, have been made. Medium-term objectives could be mostly achieved. The negative effects have not been very damaging. 10. In terms of Efficiency, most resources were available within a reasonable time and did not require any significant budgetary adjustments. Management was reasonably transparent and followed World Bank procedures. However, improvements could have been made in reporting. Despite delays at the beginning of implementation, most of the activities were implemented on time, in particular those contributing directly to the achievement of results. Some overly ambitious activities remain pending, but do not undermine the outputs, as they were not technically and legally feasible during the project life cycle. The outputs are in line with the project, but improvements could have been made in terms of quality, coverage, and timing. The inter-institutional structures put in place have been reasonably effective, despite an inadequate general framework for monitoring, reporting, and evaluation. Partners have generally fulfilled their role. However, some improvements could have been made to the efficiency of the national public administration school (ENAM) and the faculty of economy and justice (FSEJ) training. At the end of the interviews with the heads of these two partner training institutions, they recognized that public financial management training should be improved, particularly in the area of results-based management. 11. In terms of Impact, it is generally difficult to assess at the end of the project. The impact of activities was not always aligned with expected results; therefore, some objectives have not been fully achieved. If the lack of donors’ coordination, combined with the absence of functional monitoring and evaluation units, persist, this could lead to an insignificant impact. Overall, deliveries of office and IT equipment have 49 brought about an immediate improvement to civil servants’ working conditions. However, non-certified trainings will have a limited impact on civil servants’ careers, and will at best serve as an introduction. 12. In terms of Sustainability, training and equipment delivered constitute a solid foundation of viability. PRC implementation has been based largely on structures and institutions that are also involved to some extent in the decision- making process. The likelihood of sustainability is good for the continuation of support, but structural improvements should be made, with the designation of focal points in each operational institution, solely responsible for the monitoring and reporting of all projects in direct or indirect support. The political sphere and the institutions responsible for implementing the PFM reform policy have generally supported the program or at least have not hampered it and should probably continue to do so in order to ensure implementation of the PDES. In conclusion, the results of the results of the PRC implementation are generally positive in terms of capacity building, improvement of working conditions for PFM operators, and the modernization of the information management system. 50 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders 1. None provided. 51 Annex 9. List of Supporting Documents Independent Evaluation Group. 2008. Public Sector Reform: What Works and Why? An IEG Evaluation of World Bank Support, Washington, D. C: World Bank World Bank. 2008. Country Assistance Strategy for the Republic of Niger for the Period of FY2008-11. Report No. 43443-NE Washington, D. C: World Bank World Bank. 2008. Second Rural and Social Policy Reform Credit of FY2008-11. Report No. 43443-NE Washington, D. C: World Bank World Bank. 2009. Growth Policy Reform Grant 1 (GPRG-1). Report No.44611-N Washington, D. C: World Bank World Bank. 2008 Implementation Completion and Results Report on First Rural and Social Policy Reform Credit (RSRC-1) and Second Rural and Social Reform Grant. Report No. ICROOOO988, Washington, DC: World Bank World Bank. 2007 Implementation Completion and Results Report on a Public Expenditure Reform Credit. Report No. 39304 (IDA-40660 IDA-HI 720) Washington, D. C.: World Bank World Bank. 2007 Implementation Completion Report on a Public Expenditure Reform Credit. Report No. 261 95 (IDA-35760) Washington, D. C: World Bank Last, Duncan, Lynne McKenzie, Roberto Tibana, and Camille Karamaga. 2008. Implementing Public Financial Management Reforms. IMF Fiscal Affairs Department Technical Assistance Mission Report. March 2008. Washington, DC: IMF Niger: 2008 Article IV Consultation and First Review under the Three-Year Arrangement under the Poverty Reduction and Growth Facility. IMF Country Report No. 09/59, Washington, D.C: IMF I 52 Annex 10. List of People Met Name Title/Organization World Bank Staff Michel Mallberg Senior Public Sector Specialist & TTL Abdoulai Garba Former Country Economist Bougadare Kone Environmental Specialist Carlo des Ninno Program Leader Ibrah Rhamane Sanoussi Sr. Procurement Specialist Jaime Mayaki Former Operations Officer Jose Calix Current EFI Program Leader Josue Akre Financial Management Specialist Luc Razafimandimby Senior Economist Paola Ridolfi Country Program Coordinator Robert Yungu Senior Public Sector Specialist & Former TTL Sebastien C. Dessus Former Program Leader, EFI Serdar Yimlaz Former Cluster Leader, PREM Siaka Bakayoko Country Manager Government and Donors in Niamey Abdou Harouna DGI/DCE Abdou Manu European Union Directeur de l’Informatique Financière, Ahe Aboubacar Ministère des Finances et Responsable de la Composante 2du PRC Alio Douada HALCIA Amadou Petitot Oumarou DGD Boubacar Elh Dioffo DGCMP/EF Boubacar Mahaman DGTCP Boukari Oussoumane ENAM Camille France (Ambassade et AFD) Cherif Chako FSEJ Commission Finances AN Assemblée Nationale Grema Yerima Moussa DEP/MF.Div S & V Hassan Gabari SPM PCDS et ancien SPM PRC Directeur des Ressources Financières et du Ibrahim Massalatchi Matériel, Ministère des Finances et RAF du PRC Ibrahim Noma DIF/MEF Ibrahim Souley DGD Assistant Administratif et Financier du Ismael Boudel PRC Issoufou Yahaya Assistant SPM du PRC Mahamadou Haro DGB 53 Mahaman Balarabé Ibrahim DGI Mainassara Assoumane MP/DGPP Malam Souley DGI Maman Issoufou DGOFR Marie-Claire Hanounou Coordonnatrice du PCDS Mme Billo Rahanatou Comptable Principale du PRC Mme Doubou Fatchima Député Nationale Mme Hambally Haoua Directrice Générale des Impôts Mme Hamed Fadouma MP /DGIN Directrice Générale du Contrôle des Mme Hamissou Mariama Marchés Publics et des Engagements Financiers Mme Ousmane Mariama Ibrahim Secrétaire du PRC Mme Rabo Fatchima Directrice Générale du Budget Monsieur Boubacar Djibo Inspection Général des Finances en Chef Monsieur Issaka Assoumane Directeur Général des Douanes Directeur Général des Opérations Monsieur Mahaman Laouali Rafa Financières et des Réformes Monsieur Yakoubou Maman Sani Directeur Général des Investissements Monsieur Yaye Saidou Directeur Général du Plan Directeur Général du Trésor et de la Monsieur Zourkaleini Souleymane Comptabilité Publique Directeur des études et de la Moussa Moha programmation, Ministère des Finances et Coordonnateur du PRC Omar Dieye DGD Salou Daouda Ancien SPM PRC Secrétaire Général du Ministère des Taher Hassane Finances Tarno Maman FSEJ Zouera Kimba Assistante Comptable du PRC 54 MAP OF NIGER 55