Document of The World Bank Report No: ICR00004853 IMPLEMENTATION COMPLETION AND RESULTS REPORT Credit Number 5547-RW ON A CREDIT IN THE AMOUNT OF SDR 65.9 MILLION (US$100 MILLION EQUIVALENT) TO THE REPUBLIC OF RWANDA FOR THE PUBLIC SECTOR GOVERNANCE PROGRAM-FOR-RESULTS June 28, 2019 Governance Global Practice Africa Region The World Bank ICRR for The Rwanda Public Sector Governance Program (P149095) CURRENCY EQUIVALENTS (Exchange Rate Effective {May 15, 2019}) Currency Unit = Rwanda Franc (RWF) RWF 907 = US$1 US$ = SDR 1 FISCAL YEAR July 1 – June 30 Regional Vice President: Hafez M. H. Ghanem Country Director: Carlos Felipe Jaramillo Senior Global Practice Director: Edward Olowo-Okere Practice Manager: Nicola J. Smithers Task Team Leader(s): Enagnon Ernest Eric Adda, Jens Kromann Kristensen ICR Main Contributor: K. Migara O. De Silva, Yoko Kagawa ii ABBREVIATIONS AND ACRONYMS CPS Country Partnership Strategy DLI Disbursement Linked Indicator DO Development Outcome EDPRS Economic Development and Poverty Reduction Strategy GDP Gross Domestic Product ICR Implementation Completion and Results Report ICT Information and Communication Technology IDA International Development Association IEG Independent Evaluation Group IFMIS Integrated Financial Management Information System IMF International Monetary Fund IP Implementation Performance IPF Investment Project Financing IPSAS International Public Sector Accounting Standards IRIs Intermediate Result Indicators ISR Implementation Status and Results Report M&E Monitoring and Evaluation MDA Ministries, Departments, and Agencies MINECOFIN Ministry of Finance and Economic Planning MTEF Medium-Term Expenditure Framework MTR Mid-Term Review NADA National Data Archive NISR National Institute of Statistics of Rwanda NSDS National Strategy for Development of Statistics NST National Strategy for Transformation OAG Office of the Auditor General PAD Program Appraisal Document PAP Program Action Plan PDO Program Development Objective PEFA Public Expenditure and Financial Accountability PFM Public Financial Management PforR Program-for-Results PSI Policy Support Instrument RPPA Rwanda Public Procurement Authority RRA Rwanda Revenue Authority SEAS Subsidiary Entities Accounting and Financial Reporting Systems SCD Systematic Country Diagnostic SPIU Single Project Implementation Project SSP Sector Strategic Plan TTL Task Team Leader VAT Value Added Tax iii TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROGRAM CONTEXT AND DEVELOPMENT OBJECTIVES.................................................... 8 A. Context at Appraisal and Theory of Change .............................................................................8 B. Significant Changes During Implementation (If Applicable) .................................................... 13 II. OUTCOME .................................................................................................................... 14 A. Relevance ............................................................................................................................ 14 B. Achievement of PDO (efficacy) ..............................................................................................16 C. Justification of Overall Outcome Rating ................................................................................. 21 D. Other Outcomes and Impacts (if any) .................................................................................... 22 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 23 A. Key Factors during Preparation .............................................................................................23 B. Key Factors during Implementation....................................................................................... 25 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 26 A. Quality of Monitoring and Evaluation ................................................................................... 26 B. Environmental, Social, and Fiduciary Compliance .................................................................. 27 C. Bank Performance ................................................................................................................ 27 D. Risk to Development Outcome ............................................................................................. 28 V. LESSONS AND RECOMMENDATIONS ............................................................................. 28 VI. ANNEX 1. RESULTS FRAMEWORK, PERFORMANCE ON DISBURSEMENT LINKED INDICATORS, AND PROGRAM ACTION PLAN ............................................................................................. 31 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT AND SUPERVISION .................. 44 ANNEX 3. PROGRAM EXPENDITURE SUMMARY ................................................................... 46 ANNEX 4. BORROWER’S COMMENTS ................................................................................... 47 ANNEX 5. SUPPORTING DOCUMENTS (IF ANY) ..................................................................... 48 iv The World Bank Rwanda Public Sector Governance Program For Results (P149095) DATA SHEET BASIC INFORMATION Product Information Program ID Program Name Financing Instrument Rwanda Public Sector Governance P149095 Program-for-Results Financing Program-For-Results Country IPF Component Rwanda No Organizations Borrower Implementing Agency Ministry of Finance and Economic Planning MINECOFIN Program Development Objective (PDO) Original PDO The PDO of the Rwanda Public Sector Governance Program-for-Results operation is "to enhance Rwanda’s Public Financial Management and statistics systems to improve transparency and accountability in the use of public funds, revenue mobilization and the quality and accessibility of development data for decision making.". Page 1 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) FINANCING FINANC E_T BL Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Administered Financing 100,000,000 100,000,000 93,375,356 IDA-55470 Total 100,000,000 100,000,000 93,375,356 Non-World Bank Administered Financing Borrower/Recipient 0 0 0 Total 0 0 0 Total Program Cost 100,000,000 100,000,000 93,375,356 KEY DATES FIN_TABLE_DAT Program Approval Effectiveness MTR Review Original Closing Actual Closing P149095 31-Oct-2014 11-Dec-2014 16-Nov-2016 31-Dec-2018 31-Dec-2018 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 11-Mar-2017 91.56 Change in Results Framework KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Substantial RATINGS OF PROGRAM PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 11-Feb-2015 Satisfactory Satisfactory 28.74 02 13-Sep-2015 Satisfactory Satisfactory 28.74 03 30-May-2016 Moderately Satisfactory Satisfactory 62.61 Page 2 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) 04 27-Dec-2016 Moderately Satisfactory Satisfactory 91.56 05 28-Sep-2017 Moderately Satisfactory Satisfactory 93.38 06 01-Apr-2018 Moderately Satisfactory Satisfactory 93.38 07 18-Oct-2018 Moderately Satisfactory Satisfactory 93.38 SECTORS AND THEMES Sectors Major Sector/Sector (%) Public Administration 100 Central Government (Central Agencies) 60 Sub-National Government 40 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Economic Policy 5 Fiscal Policy 5 Tax policy 5 Public Sector Management 95 Public Finance Management 25 Public Expenditure Management 20 Domestic Revenue Administration 5 Public Administration 70 Transparency, Accountability and Good 40 Governance Municipal Institution Building 30 ADM STAFF Role At Approval At ICR Regional Vice President: Makhtar Diop Hafez M. H. Ghanem Country Director: Diarietou Gaye Carlos Felipe Jaramillo Senior Global Practice Director: Deborah L. Wetzel Edward Olowo-Okere Page 3 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) Practice Manager: Guenter Heidenhof Nicola J. Smithers Enagnon Ernest Eric Adda, Jens Task Team Leader(s): Yoichiro Ishihara Kromann Kristensen ICR Contributing Author: K. Migara O. De Silva Page 4 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) EXECUTIVE SUMMARY The Rwanda Public Sector Governance Program-for-Results (PforR) was approved on October 31, 2014, with a credit amount of US$100 million, financed by the International Development Association. The Program was one of the early PforRs after this relatively new instrument was introduced. The Program was closed on December 31, 2018. The Program-for-Results (PforR) supported the Government of Rwanda’s two national programs: the Public Financial Management Sector Strategic Plan (PFM SSP) 2013-2018 and the National Strategy for Development of Statistics 2 (NSDS 2) 2014–2019. Both programs were regarded essential to achieve the country’s medium-term goals set out in the second Economic Development and Poverty Reduction Strategy (EDPRS 2). EDPRS 2 defined the role of PFM as the platform for the efficient management of the nation’s resources to achieve the goals of EDPRS 2, highlighting PFM priorities to: (i) increase resource mobilization; (ii) scale up implementation of the Integrated Financial Management Information System (IFMIS); (iii) strengthen PFM systems at the subnational level; and (iv) enhance training, professionalization, and capacity building across all PFM disciplines. Statistics were also regarded as critical for supporting Rwanda’s development agenda. They play a foundational role in PFM by informing policy analysis and strategy formulation; thus, providing critical information for planning and budgeting. Responding to these PFM priorities, the PforR was designed with the Project Development Objective (PDO) of enhancing Rwanda’s PFM and statistics systems “to improve transparency and accountability in the use of public funds, revenue mobilization, and the quality and accessibility development data for decision making.” Consequently, the overall relevance of the PforR is rated as “High.” This is reinforced by the assessment that the Program considerably expedited Rwanda’s implementation of these programs. A review of the government’s PFM SSP reported significant progress on PFM reforms during the period of the SSP and concluded that it was effective and efficient. The overall efficacy is rated as “Substantial.” The discussion of the achievements related to the three elements of the PDO are drawn largely from evidence derived from the Intermediate Results Indicators (IRIs), Disbursement Linked Indicators (DLIs), and other measures directly related to the PDO. This approach has been taken due to inadequacies in PDO Indicators 1 and 2. The recent Implementation Status and Results Reports (ISRs) rated the development outcome as moderately satisfactory because only the PDO indicators were used for those ratings. The PforR achieved all 8 DLIs (many of them exceeded the end targets) and 18 Program Action Plans (PAPs), and most notably made exceptional achievements in information and communication technology (ICT) systems — the implementation on IFMIS, e-Procurement, and e-tax. Significant achievements were made regarding the first element of the PDO to improve revenue mobilization. All three IRIs were exceeded. The implementation of the e-tax system under the PforR has directly increased revenue collected by local governments and allowed small and medium taxpayers to file taxes electronically. The PDO Indicator 1 of the ratio of tax revenue to gross domestic product (GDP) was a poor choice. Such an indicator is rarely used for World Bank programs and projects because of multiple variables typically outside the operation’s control. It was partially achieved in that there was an improvement in line with the latest International Monetary Fund program, but not to the level of the original target. An amendment to the tax code on value added tax (VAT) exemptions was submitted to the Cabinet, and an action plan to improve VAT revenue was prepared under the PforR. These changes are expected to contribute to increased revenue for Rwanda in the long term. 5 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) Very substantial achievements are seen regarding the second element of the PDO to improve transparency and accountability in using public funds. Before the PforR, Rwanda had made significant progress in PFM reforms, particularly after comprehensive ones were launched in 2008. The Public Expenditure and Financial Accountability (PEFA) assessment of 2010 reported that Rwanda’s performance was as good as or better than that of neighboring countries in all categories except accounting, recording, and reporting. DLIs for this PDO element focused on achieving very difficult improvements in PFM systems and functions. With support of the PforR, the IFMIS was implemented as the backbone of PFM transparency and accountability in almost all national and subnational entities. The rollout of e-Procurement implementation to all government procuring entities was another outstanding success of the Program, leading to cost savings and increased transparency, accountability, and value for money in public procurement. Rwanda’s achievement in establishing national systems using cutting-edge technology is remarkable, especially given the struggles other countries have experienced in making similar transitions. PDO Indicator 2 specifically measured submission and publication of monthly reports — this indicator was partially achieved. All public entities submitted monthly budget reports on time, but they were not made public. This Implementation Completion and Results Report agrees with the government of Rwanda and the supervision task team that disclosure of monthly reports would not have substantially contributed to enhanced accountability and was not an appropriate measure of transparency. Using other measures included in the project, the PforR supported significant achievements in improving transparency and accountability. The major increase in unqualified audit reports of the annual financial statements of ministries, departments, and agencies demonstrates improved public accountability for public spending, while implementation of the e-Procurement systems contributed to enhanced transparency in the use of public resources by publishing all bidding opportunities and contract awards on the public website in real time. There was also major success related to the third element of the PDO — to improve the use of public data for decision making. PDO Indicator 3 was fully achieved. Production and dissemination of vital statistics, e.g., through agriculture and enterprise surveys, together with GDP rebasing, contributed significantly to the use of statistics in decision making for economic policy and strategies. For example, the Ministry of Finance and Economic Planning has increasingly utilized statistical data for the determination of fiscal transfers (for block grants) based on population, poverty rates, and location data. A key Program strength was the coherence and synergy of DLIs and PAPs for strengthening institutional capacity. Institutional, organizational, and human capacity constraints at both the central and subnational levels were identified as key risks to PDO achievement. To mitigate these risks, capacity-building elements were included in DLIs and PAPs. The Program’s success factors are multifold. Reform-minded government with strong leadership capacities remains the most critical success factor. The Program also succeeded in terms of “right timing” — building on the achievements of previous national programs, including fundamental institutional arrangements and the capacity to advance reforms. Given Rwanda’s results-oriented culture, the PforR instrument was the “right fit” as it provided the financial incentives to carry out reforms as envisaged by the government. These well-designed reform programs were based on analytical underpinnings, such as the PEFA and the Procurement framework as well as the World Bank’s engagement in Rwanda for more than two decades. The nature of the PforR enabled the Government of Rwanda and World Bank to focus on technical issues rather than ICT procurement processes, which typically consumes supervision of Investment Project Financing. The 6 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) Program required enhanced supervision in terms of the depth of technical expertise and the rigor of monitoring to support achievement of the DLIs. This project was one of the first PforR efforts of the World Bank, and the design and implementation challenges faced demonstrate that it was a learning experience for both the Bank and the client. Both design and implementation efforts were undertaken without detailed guidance from Bank management, which created uncertainty in the selection of PDO indicators and how to effectively respond to incorrectly defined indicators. For this project, the lack of experience with PforR operations did not have an impact on the degree of success in achieving the development objectives but did have consequences for achieving the specific development indicators that had been selected. Despite careful Program preparation and highly dedicated and attentive supervision, the World Bank’s performance is rated “Satisfactory” because Program indicators were not revised once their weaknesses became apparent to the client and the Bank team. The Program’s overall outcome rating is “Satisfactory.” Successful implementation of planned interventions rendered significant improvements to three elements of the PDO: an increase in revenue collection; greater transparency and accountability in the use of public finance; and better data for decision making . Nonetheless, the PforR came short of being rated “Highly Satisfactory” because two PDO indicators were not fully achieved due to overly ambitious and flawed targets. 7 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) I. PROGRAM CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL AND THEORY OF CHANGE 1. Rwanda made a miraculous transformation from its state of devastation by genocide to a good model of international development today. Growth averaged 7.5 percent over the decade from 2007 to 2017 1 while per capita gross domestic product (GDP) grew at 4.7 percent annually. As a result, poverty levels were significantly reduced from 56.7 percent in 2000 to 38.1 percent in 2017, while inequality measured by the Gini coefficient stood at 0.42. Rwanda is one of a few countries in Africa which achieved almost all of the Millennium Development Goals. 2. The Program-for-Results (PforR) operation was prepared to support a subset of the government’s programs, articulated in the Public Finance Management Sector Strategic Plan (PFM SSP 2) for 2013–2018 and the National Strategy for Development of Statistics (NSDS 2) for 2014–2019. The Program was also in alignment with one of the strategic goals set out in the Country Partnership Strategy (CPS) for 2014–2018 to support accountable governance through PFM and decentralization. Both programs were regarded essential to achieve the country’s medium-term goals set out in the second Economic Development and Poverty Reduction Strategy (EDPRS 2). 3. EDPRS 2 defined the role of PFM as the platform for efficient management of the nation’s resources to achieve the strategy’s goals. It highlighted four PFM priorities: (i) increase resource mobilization from domestic and alternative sources of finance, with the objective of meeting increased public expenditure and making Rwanda progressively self-reliant; (ii) scale up the implementation of the Integrated Financial Management Information System (IFMIS); (iii) strengthen PFM systems at the subnational level, including in districts and subsidiary units (sectors, schools, health facilities) to support fiscal decentralization and service delivery; and (iv) enhance training, professionalization, and capacity building access all PFM disciplines to sustain the reforms in the long run. Thus, the PFM SSP 2 for 2013–2018 was formulated. 4. Rwanda made significant progress in PFM reforms particularly after the government embarked on comprehensive PFM reforms in 2008. The Public Expenditure and Financial Accountability (PEFA) assessment of 2010 documented that scores in all but one category improved from the previous 2007 report. The report found Rwanda’s performance was as good as or better than that of neighboring countries in all categories except for accounting, recording, and reporting. Yet, there was a need to consolidate the achievements made and to address Rwanda’s other challenges — a large central government budget compared to GDP and a tax revenue to GDP ratio that was lower than the average for East African countries. Thus, effective PFM was regarded as very important in Rwanda. The next generation of PFM reforms through the PFM SSP 2 focused on both revenue and expenditure sides of PFM at the national and subnational level with an emphasis on new technologies as catalyst. Advancing and implementing PFM reforms, including capacity building and institution strengthening, was also an important pillar of the CPS for 2014—2018. 5. In tandem with the PFM reforms, the sector strategy for statistics — the first National Strategy for Development of Statistics (NSDS) for 2009–2014 — was launched, and significant progress was made. The country moved up 20 places on the World Bank’s Statistical Capacity ranking, from 73 (of 149 countries) at the end of 2008 to 53 at the end of 2013. NSDS 2 2014–2019 was formulated. Statistical capacity building was continued under NSDS 2 with an emphasis on enhancing statistical advocacy and the use of statistics in decision making. At the same time, the important roles of statistics in the PFM cycle was recognized, and production of economic data was stressed in the PFM SSP. 1 The World Bank’s Rwanda Economic Update of December 2018 predicts medium-term growth of 7 to 8 percent. 8 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) Theory of Change (Results Chain) 6. The logic behind the Program was that PFM functional efficiency and effectiveness and production of key economic data would be improved through Program interventions, particularly by meeting Disbursed Linked Indicators (DLIs). Underlying assumptions of the Program Theory of Change are DLIs would provide incentives to improve systems, functions, and data production and accessibility, which are vital outputs necessary to achieve intermediate outcomes (i.e., budget credibility; transparent public procurement; accountable financial reporting and oversight; efficient revenue mobilization; and improved quality and dissemination of major economic statistics). These intermediate outcomes will lead to the three elements of the PDO, contributing to the achievements of the two national programs, PFM SSP 2 and NSDS 2. The diagrammatic presentation of the Theory of Change is reflected in Figure 1 9 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) Figure 1. Program Theory of Change 10 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) Rationale for PforR Support and Program Scope and Boundaries 7. The PforR instrument was regarded as a good fit for Rwanda because the PFM SSP, NSDS 2, and national development strategy (i.e., EDPRS 2) had strong government leadership and commitment, were well defined, and were results based with established monitoring and evaluation (M&E) mechanisms. At the same time, fluctuation in aid from development partners constrained the implementation of government expenditure programs. Therefore, the PforR instrument would contribute to the predictability of budget financing, while anchoring disbursement against results achieved. With regard to this Program specifically, the scope would also have a direct impact on improving the quality of government expenditures. Furthermore, the institutional capacity to deliver the reforms under the Program was demonstrated by the strong track record regarding PFM and statistics reform. 8. The choice of the PforR was supported by the government and other development partners. The World Bank had already forged strong relationships in public sector reforms with other development partners during two decades of engagement in Rwanda. The Bank chaired the PFM Coordination Forum and engaged in the Statistics Steering Committee. By jointly supporting the government’s own programs and expenditures in a complementary manner, the Program was expected to further improve development partnerships. 9. Although the boundaries between the Program and the government’s own programs (PFM SSP and NSDS 2) were defined based on strategic relevance, government demand, and support from other development partners, the Program covered almost all programs and subprograms because they were interrelated and constituted important elements of the PFM cycle. However, the Program supported selected activities within these subprograms and strategies that were essential to achieve the Program’s development objectives. 11 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) Program Development Objective 10. The overall objective of the Program was enhancing Rwanda’s public financial management and statistical systems “to improve transparency and accountability in the use of public funds, revenue mobilization, and the quality and accessibility of development data for decision making.” Outcome Indicators 11. The Program aimed to measure the achievement of the PDO according to the following indicators:  PDO Indicator 1: Enhanced national and subnational revenue collection, measured by tax to GDP ratio which measures national and subnational revenue collection.  PDO Indicator 2: Improved national and subnational transparency and accountability in the use of public funds, measured by the number of entities which were submitting monthly statements by the due date and making them publicly available to enhance transparency and accountability.  PDO Indicator 3: Improved use of development data for decision making, measured by the share of ministries, departments, and agencies (MDA) which use official statistics for both analysis of current developments for short- term decision making and analysis of trends for longer-term policy formulation. 12 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) Program Results Areas and DLIs 12. The DLIs of the three results elements are presented in table 1. Table 1. Disbursement Linked Indicators for Each Element of the Program Development Objective Elements of the Program Development Objective Disbursement Linked Indicator Public Financial Management Improved revenue mobilization DLI 2: Extent to which districts have adopted the automated local government revenue management system Improved transparency and accountability in the use of DLI 1: Extent to which investment plans of pilot ministries public funds comply with budget call circular DLI 3: Extent to which the e-Procurement system has been implemented DLI 4: Extent to which MDA have improved in their compliance of financial management requirements DLI 5: Extent to which sectors are using Subsidiary Entities Accounting and Financial Reporting Systems DLI 6: Extent to which government financial management staff are trained in PFM Statistics Improved quality and accessibility of development data for DLI 7: Extent to which the production and timeliness of decision making dissemination of economic statistics are enhanced DLI 8: Extent to which the variety of data available in the National Archive Data is enhanced B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDO Outcome Targets, Results Areas, and DLIs 13. The first and only restructuring was completed on March 11, 2017. It was a Level 2 restructuring to correct the baselines for two indicators:  PDO Indicator 3: Share of MDA using official statistics for both analysis of current developments for short-term decision making and analysis of trends for longer-term policy formulation  DLI 6: Number of government PFM staff with at least foundation-level professional finance qualifications 14. For PDO Indicator 3, the baseline established in the Program Appraisal Document (PAD) was found to be incorrect. Instead of calculating the share of MDA using statistics for both short-term decision making and long-term policy formulation, the baseline in the PAD of 68 percent was based on the MDA using statistics for either one. After confirming that progress was made on these two elements of the objectives, the baseline and targets were revised:  The baseline from 68 percent to 39 percent  The first year (2014/15) target from 71 percent to 42 percent  The second year (2015/16) no target was established  The third year (2016/17) from 75 percent to 48 percent 13 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) 15. For DLI 6, the original baseline number (292) included students who did not meet the indicator definition (completion of the foundational level). Also, the DLI definition did not include PFM staff who are Certified Accountant Technicians or have an International Public Sector Accounting Standards (IPSAS) certificate. These two important professional qualifications were needed to be included as criteria for the indicator. Accordingly, the definition criteria for DLI 6 was revised and the targets remained unchanged while the baseline was revised to 202 staff. Other Changes 16. No other changes were made. Rationale for Changes and Their Implication for the Original Theory of Change 17. No changes were made to the original Theory of Change. II. OUTCOME A. RELEVANCE A.1 Relevance of PDO 18. Relevance of PDO is rated High. The Development Objectives of the Program aligns with the CPS 2014–2018, as revised and extended until 2020 after the Performance and Learning Review of March 20, 2017. The CPS’s third theme focuses on supporting accountable governance though PFM and decentralization. The Program remains important to the World Bank’s Systematic Country Diagnostic (SCD) for Rwanda of May 2019 as well. The SCD points out a key reform priority to improve alignment of spending with the government strategy and service delivery objectives, while improving control of expenditures to increase value for money in order to progress toward the World Bank Group’s Twin Goals of reducing extreme poverty and increasing prosperity. 19. The World Bank’s Investment Project Financing (IPF, P164807) of the Rwanda Public Finance Management Reform Project approved by the Board on October 23, 2018 is built upon the achievements made by the Program. It aims to (i) enhance the government’s ability to strengthen policy alignment of the budget through the use of performance-based budgeting and the Medium-Term Expenditure Framework (MTEF) and (ii) to ensure predictability of funding for service delivery and control of resources received by service delivery units. Further expansion and improvement in reporting accurate financial information and increasing PFM capacity across all government entities are also supported by the IPF. 20. The Program is also relevant to the PFM SSP 2018–2024 with its mission “to support Rwanda’s socioeconomic transformation through effective and accountable PFM.” The 2018 evaluation by the Independent Evaluation Group (IEG) of the Rwanda Country Program found that the outcomes and objectives of Bank support for improved public resource management was “entirely aligned with the country’s broad goals.” 2 21. Recently, a new development strategy (Vision 2050) was announced in Rwanda. Vision 2050 has ambitious goals of moving Rwanda to upper middle-income status by 2035 and high-income status by 2050. The implementation of the Vision 2 Rwanda Country Program Evaluation, FY09–17, IEG, 2018, Annex L, p. 178. 14 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) 2050 will be operationalized through a series of medium-term strategies, starting with the National Strategy for Transformation (NST 1). The new PFM Sector Strategy Plan 2018–2024 will deliver broader commitments of the NST 1 by supporting the country’s socioeconomic transformation through effective and accountable PFM. A.2 Relevance of DLIs 22. Relevance of DLIs is rated High. The DLIs adhere to the criteria defined in the PAD: (i) they signal and monitor a milestone along the results chain without which the PDO cannot be achieved; and (ii) they signal incentives for rewarding performance (outputs, outcomes) to encourage the practice of managing for results. The definition of the eight DLIs was clearly presented and measurable along with the PAD’s disbursement arrangements and verification protocols and the financing agreement’s withdrawal conditions. All DLIs related to PDO Indicators 1 and 2 were numerical with achievements clearly measured by the number of MDA, districts, sectors (local governments), and staff who were beneficiaries of the operation’s interventions. The measurement of the remaining two DLIs for Results Area 3 laid out clearly the milestones necessary to produce selected economic data and to increase availability of statistical information in National Archive Data (NADA). 23. Eight DLIs were either Program interventions (outputs) or results of the Program interventions (intermediate outcomes) that were necessary to achieve the PDO. 24. The formulation of the Program’s results framework and the selection of DLIs were strongly aligned. Five DLIs were part of the Program’s results framework as they were incorporated as IRIs. The DLIs related to PDO Indicators 1 and 2 provided appropriate triggers for PFM reforms at the subnational level (DLIs 2 and 5) and among line ministries (DLIs 1, 3, and 4) to galvanize and deepen delivery of reforms toward the attainment of PDO Indicators 1 and 2. DLI 6 aimed to increase professional accounting qualifications, which is necessary to attain and sustain achievements brought by the Program. 25. The alignment of the results framework and DLIs was demonstrated in the revenue mobilization element of the PDO, which had one DLI (DLI 2) for implementing the e-tax system at the local government level (output) with two IRIs: time spent in paying tax and e-tax declaration ratio. They related to more efficient and effective tax administration, which is the foundation for increasing tax collection (PDO Indicator 1). This alignment of the DLI and the IRIs was appropriate and effective. The former provided incentives and availability of funds, which stretched e-tax implementation to 30 districts from the targeted 11, and the latter were outcomes at the national level, which require time for interventions to lead to tangible results. 26. Production of feasibility studies of line ministries (DLI 1); implementation of e-Procurement (DLI 3) and utilization of Subsidiary Entities Accounting and Financial Reporting Systems (SEAS) (DLI 5); delivery of unqualified audit reports (DLI 4); and professionalized accountants (DLI 6) are critical systems and functions for improving transparency and accountability of PFM (PDO Indicator 2). 27. The Program succeeded in producing vital economic data from, for example, the survey of agriculture, which is the nation’s largest economic resource; a business enterprise survey; and GDP rebasing by achieving DLI 7. Together with increased information on NADA (DLI 8), new data collection and results contributed to the achievement of PDO Indicator 3. 28. The government confirmed that the amount allocated to each DLI was sufficient to incentivize all beneficiaries to meet the targets, which enabled reforms to expand. Financing allocated to DLIs during the course of the Program was approximately US$12.5 million for each of the eight DLIs. The allocation of financing was frontloaded with a gradual 15 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) decrease in the second and third years to spur momentum for achieving results early and carrying progress through the entire Program implementation. 29. The government took advance payment at the beginning of each year (table A2.1, annex 6), and verification of results achievement was provided at the end of the year. The government’s strategy was first to demonstrate its commitment to results with strong coordination and leadership provided by the Ministry of Finance and Economic Planning (MINECOFIN). Secondly, separate but linked sub-activities across different parts of government would be galvanized and synergized. This approach ensured sharper accountability for DLI-responsible stakeholders by expanding the environment where staff must report on progress consistently and structurally in their areas. It provided a framework whereby the government was in a better position to judge its own PFM performance across several areas. It also helped identify issues and focus dialogue with the World Bank’s task team. 30. As a result, 98 percent of the financing proceeds were disbursed before the Mid-Term Review (MTR), creating “a nice problem to have” for determining payment arrangements for the Program’s last year. The Bank’s task team debated whether to accept the government’s request for advance payment for the third year, thereby disbursing full financing a year before the Program’s closing date. Given achievements reached during the first two years, an arrangement was made to accept the government’s request for advance payment and determine the amount paid for each DLI according to the proportion of its achievements with assurance that remaining targets will be met. 31. Judging from the fact that all DLIs not only fully met the intended targets, but also exceeded them earlier than the Program’s closing date, the ambitiousness of DLIs could be called into question. However, the first six DLIs covered whole PFM cycles, different from the approach typically taken by IPF, which usually focuses on certain subsystems of PFM with the IFMIS implementation at the center. The time required for this type of IPF is seven years on average,3 whereas the Program achieved the objectives within three years. Combined with the establishment of major statistical data, the selected DLIs provided incentives to effectively consolidate and extend basic systems. Rating of Overall Relevance 32. The rating of overall relevance is High. The PDO was well aligned with the Bank’s country strategy as well as Rwanda’s own long-term national development goals. The functional and procedural improvements, indispensable to the attainment of PDO, were driven by DLIs so that comprehensive PFM reforms covering almost all elements of the wide- ranging PFM cycle could be achieved. B. ACHIEVEMENT OF PDO (EFFICACY) 33. This section reviews achievements related to the three elements of the PDO, namely improved revenue mobilization, transparency and accountability in the use of public funds, and the quality and accessibility of development data for decision making. It does so, drawing largely from evidence derived from the IRIs, DLIs, and other measures that are directly related to the PDO. This approach has been taken due to inadequacies in PDO Indicators 1 and 2. As noted, the PforR achieved all 8 DLIs (many of them exceeded the end targets). It also achieved all 18 Program Action Plans (PAPs), and most notably made exceptional achievements in ICT systems (i.e., implementation on IFMIS, e-Procurement, and e- tax). 3 Finance Management Information Systems: 25 Years of World Bank Experience on What Works and What Doesn’t, World Bank, 2011. 16 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) Assessment of Achievement of Each Objective or Outcome 34. The summary of the results framework is in Annex 1. 1. Improved Revenue Mobilization 35. Significant achievements were made in terms of revenue mobilization . All three IRIs were exceeded. By the end of the project, all 30 districts were using the automated revenue collection system (against a target of 11), and over 99 percent of small and micro-taxpayers were filing their declarations using the e-tax internet portal. Movement to electronic platforms helped to drive a reduction in hours spent paying taxes beyond the project target. In line with the performance on IRIs, the DLI (DLI 2) for revenue performance, relating to the use of the automated Local Government Revenue Management System, was exceeded. Over a period of four years, tax to GDP ratio grew by 10.5 percent (or 1.5 percentage points)— from 14.2 percent in 2012/13 to 15.7 percent in 2017/18. 36. PDO Indicator 1 targeted a tax to GDP ratio of 17.4 percent by the end of the project. Problems were encountered with the indicator along two dimensions. The first problem related to attribution. Tax to GDP ratios are influenced by a variety of factors that go far beyond tax policy and administration. Sub-Saharan African economies were (and still are) recovering from a slowdown in 2015/16. As a result, Rwanda, like most other countries in the region, operated under a less favorable external environment, being susceptible to the lost momentum in global trade and industrial activity. In some emerging markets, financial market pressures intensified due to increased concerns about their growing dollar- denominated debt. The U.S. dollar4 was strong, and the Rwandan franc was weakening. The World Bank’s Aide Memoire (November 2016) also indicates that many factors influenced the tax to GDP ratio which could not be controlled by Program intervention.5 Hence, a less than anticipated economic growth has resulted in the downward revision of the Policy Support Instrument (PSI) targets. The Aide Memoire is prescient in observing that the influence of these factors on the indicator “may cause an unfavorable rating of a program which otherwise has performed well.” 37. A second, subsidiary problem related to the level of ambition of the target. The original target was set in line with the target in the IMF’s PSI. That level was subsequently downgraded by the IMF to 15.9 percent, based on changing economic conditions and as a result of the rebasing of past year revenue to GDP calculations in line with an analysis done by the National Institute of Statistics of Rwanda (NISR) in 2017. Rwanda achieved a tax to GDP ratio of 16.3 percent in 2015/16, which exceeded the revised PSI target but was under the PDO indicator target. The Bank’s Aide Memoire of April 2016 noted that PDO Indicator 1 was on track to meet the PSI target but not the 17.4 percent target set by the PAD, and hence it was “on track to being partially met.” The Aide Memoire of November 2016 also notes that — despite achieving lower than the targeted rate, overall — Rwanda “continues to make good strides in improving revenue mobilization.” 38. In 2016/17, revenue mobilization faced major risks (external to the tax policy or tax administration) largely from lower than projected GDP growth. While the July 2014 IMF-PSI projected a 17.4 % target to be achieved in FY 2016/17, 4 Africa’s Pulse: No. 18, October 2018 issue: “An Analysis of Issues Shaping Africa’s Economic Future,” World Bank. 5 It is important to make a distinction between short- and long-run effects of the depreciation on revenue. In the long run, a weakening franc will have an impact on the trade regime. In the short run, Rwanda’s fiscal authorities are interested in a weaker currency for more immediate effects such as higher fiscal revenues from taxation of imports and higher franc denominated revenues from external grants. At present, the cash flow in Rwanda’s budget is such that depreciation improves the fiscal accounts. This will change if or when debt repayments increase and the positive effects of depreciated currency on franc-denominated flows outweigh the losses from a higher demand for cash to repay debt. Rwanda is not there yet, and at least for the next three to four years, fiscal accounts will benefit from depreciation. 17 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) which was the same as in the PAD, a subsequent PSI of IMF projected a conservative tax to GDP ratio of 15.5 percent for 2016/17 and 15.8 percent for 2017/18 in its January 2016 PSI, which was much lower than the projected target of 17.4 percent in the PAD. The target achieved in 2017/18 was 15.7 percent, which was slightly (0.1 percent) less than the IMF’s projected ratio.6 Local government tax law had not been finalized at the time, and the general assumption was that it will become effective in two to three years. However, it took longer than expected. In addition, Rwanda’s proactive use of tax expenditure for “Made in Rwanda” initiatives to support diversification and export promotion7 is also likely to have contributed to underperformance in the tax to GDP ratio. 39. Overall, the ICR judges that the achievement of the PDO with regard to improved revenue mobilization was substantial. Key steps in improving the efficiency of tax collection and tax administration were successfully implemented, and tax revenues increased substantially over the course of the project. The PDO indicators were only partially achieved due to issues in highly optimistic target setting and attribution. Reflecting these concerns, subsequent PforRs and IPFs have generally avoided using the tax to GDP ratio as a performance indicator, and it is used only in exceptional circumstances when a compelling Theory of Change can be established. 2. Improved Transparency and Accountability in the Use of Public Funds 40. There were substantial achievements in improving the transparency and accountability in the use of public funds. A range of measures are discussed to support this assessment. These include IRIs and DLIs as well as PDO Indicator 2. 41. The path to achieving the second objective of improved transparency and accountability in the use of public funds was defined by three IRIs and six DLIs. IRIs focused on improvements in the compliance of MDA with rules on the use and accounting for public money (IRI 2.1., the proportion of MDA receiving unqualified audits); improvement in the reporting of financial matters by subdistricts (IRI 2.2, the proportion of subdistricts using simplified accounts and financial reports); and increased relevance of medium-term planning in budgeting (IRI 2.3, reduction in the gap between the annual budget and the budget predicted in the second year of the MTEF).8 42. All of the IRIs were exceeded. The percentage of MDA receiving unqualified audits increased significantly during the project (from 32% to 60%), demonstrating a substantial improvement in compliance and accountability. As a result of the complete rollout and adoption of SEAS to all 416 sectors (meaning in subnational entities and lower cost and revenue centers), 100 percent of subdistricts filed simplified accounts and financial reports by the end of the project. After successful implementation of SEAS in sectors, the system has been upgraded to be included in the IFMIS. This extended the coverage of IFMIS to almost all national and subnational entities and moved financial reporting in Rwanda to a fully digital format with enhanced transparency and accountability. Finally, multiyear budgeting improved significantly so that gap between the second-year projects and the actual budget shrank to 6 percent. 43. The five DLIs associated with the second PDO (DLIs 1, 3, 4, 5, and 6) were all obtained, reflecting systemic improvement in financial management practices. Two of the DLIs (4 and 6) tracked the IRIs relating to the increase in unqualified audits obtained by MDA and improved financial reporting by subdistricts. Perhaps the most consequential reform was the adoption and rollout of e-Procurement to all procuring entities (DLI 3). This effort far exceeded the target 6 This result is prior to NISR’s rebasing GDP statistics. 7 IMF Country Report 18/335, November 2018, Tenth Review under the Policy Support Instrument. 8 The Bank’s support continued through the successor Rwanda Public Financial Management Reform Project, which was approved in October 2018 and became effective in December 2018. It includes support to strengthen the policy alignment of the budget using performance-based budgeting and MTEF, to more fully apply a strategic multiyear planning process. 18 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) of piloting the system in five entities and transformed the operative standards for transparency, efficiency, and accountability for costs in state contracting. The more targeted increased use of feasibility studies in proposals for large investment projects in five pilot ministries (DLI 2) was also an important step in improving the value for money of capital spending. Finally, increasing the number of government officials professionally trained in financial management (DLI 6) demonstrated a commitment to investing in people as much as in systems, and increased government competencies. 44. PDO Indicator 2 measured submission and publication of monthly financial reports. This indicator was partially achieved because almost all public entities submitted monthly budget reports on time. However, they were not made public. The ICR agrees with the government of Rwanda and the supervision task team that disclosure of monthly reports was not an appropriate measure of transparency or accountability since no other country, including Organization for Economic Co-operation and Development countries, publicly issue monthly budget reports. The government does publish an audited annual financial statement. The client informed the Bank team as early as October 2015 that it intended to propose redefining the PDO indicator.9 Instead of publishing monthly reports, a consolidated financial statement would be made public biannually. However, the Bank team requested the government to stay within the original definition of the PDO indicator.10 At the start of the project, only 40 percent of MDA submitted monthly financial statements, which increased significantly during project implementation. At present, close to 98percent of the MDA submit these monthly reports. 45. The overall assessment of the ICR is that the Program achievement of the PDO with regard to improved transparency and accountability in the use of public funds was substantial. There was a significant improvement in transparency and accountability as a result of the achievements in implementation of reforms to planning, budgeting, and reporting of spending across all government entities. Perhaps most significantly, the implementation of the of e-Procurement system (DLI 3) drastically enhanced public transparency and accountability in state contracting. The Program fully implemented e-Procurement reforms as planned which covered all categories — goods, consultant services, workers, and nonconsulting services (such as mapping and event organization). The current e-Procurement system provides information to the public in searchable and machine-readable format; procurement bids and winners; and all advertisements. All bidders, suppliers, and consultants should register, and online registration is an option. 3. Improved Quality and Accessibility of Development Data for Decision Making 46. There were high achievements in improving quality and accessibility of development data for decision making. Two IRIs were specified for this objective. IRI 3.1 focused on the public dissemination of the results of an annual national agriculture survey. IRI 3.2 centered on the annual production of labor market statistics and their dissemination. Both indicators were fully achieved and achievement of PDO Indicator 3 is therefore rated as High. 47. The NISR finalized and published the full 2016 Seasonal Agriculture Survey (SAS) on its website six months earlier than the target of June 2016. NISR continues to conduct the survey on an annual basis. The 2018 SAS was completed and is publicly available at http://www.statistics.gov.rw/datasource/seasonal-agricultural-survey-2018 . 9 Since the May 2016 mission, the Bank team had been in discussions with MINECOFIN on the proposed amendment related to PDO Indicator 2. The Bank received the official request to amend PDO Indicator 3 and DLI 6 in October 2016, which required a Level 2 restructuring of the Program. It did not, however, include a request to amend PDO Indicator 2 as it was not possible to reach an agreement on an alternative definition. As the Aide Memoire of the November 2016 mission indicates (p. 41), the Bank team agreed with MINECOFIN’s view that it “does not see the relevance of making the monthly financial statements public.” The ICR team considers this statement in the Aide Memoire as an agreement by the Bank team that the PDO indicator was problematic. 10 Aide Memoire of October 2015 mission. 19 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) 48. The first national labor force survey was conducted and disseminated in September 2017, three months later than the original target date. However, the capacity to produce an annual labor force survey has been established, as evidenced by the completion and publication of the 2018 Labor Force Survey on NISR’s website. Beginning in 2019, labor force surveys will be conducted twice a year. 49. Strong performance in improving the collection and dissemination of statistics in agriculture and the labor market has paved the way for the achievement of the two relevant DLIs. The quantity of economic statistics produced and made available to policy makers and the public has continued to climb and has fully met the target set out in DLI 7 (increased production of economic statistics and dissemination in time for strategic planning). By 2017/18, Rwanda produced essential indicators of economic performance, including quarterly GDP figures, trade statistics and Producer Price Indexes, and monthly Consumer Price Index measures. An increasing number of surveys and official data have been made available as part of the Open Data Initiative, as per DLI 8, including data on a range of health indicators and micro-data on tracking poverty. 50. The PDO indicator targeted the share of MDA using official statistics for analysis and policy making. The baseline for the PDO indicator in the PAD was set at a use rate of 68 percent of MDA, and corresponding targets for 2014/15 and 2016/17 were 71 percent and 75 percent, respectively (measured using satisfaction surveys of MDA officials). In October 2015, NISR identified a mistake made in establishing the baseline value for this indicator.11 The corrected target of 48 percent of MDA using statistics for decision making was in fact exceeded, reaching 62.3 percent. 51. The ICR team notes that NISR appreciated the guidance and support received from the Bank during implementation, which enabled the government to achieve tangible results and meet expected targets on time. NISR participated in different EDPRS 2 Sector Working Groups and forums, and supported sectors to integrate the Sustainable Development Goals’ indicators into NST 1, SSPs, and district development strategies. It found that satisfaction surveys were useful to understand the needs of each MDA. The Ministry of Agriculture, which is one of the recipients and users of NISR data and statistics, indicated that this data-driven approach led to a behavior change, i.e., relying more on evidence-based data during policy formulation. The Strategic Plan for Agriculture Transformation 2018–2024 utilized a similar approach in setting up the baselines to determine more realistic targets, which in some areas, such as food security, remain highly necessary. Similarly, statistical data were regularly utilized by MINECOFIN when determining transfer formulas for block grants based on data-driven factors, including the share of population, rates of poverty, and locational data. 52. Therefore, the assessment of the ICR is that the Program achievements of the PDO with regard to improved quality and accessibility of development data for decision making was High. 11Instead of calculating the share of MDA using statistics for short-term decision making and longer-term policy formulation, the PAD baseline was based on the share of MDA using statistics for either short-term decision making or longer-term policy formulation. Level 2 restructuring completed in February 2017 revised the baseline and targets for corresponding years. The PAD baseline for PDO Indicator 3 was set at 68 percent. However, as documented in ISRs and Aide Memoires, NISR informed the World Bank in October 2015 that a mistake was made in setting the baseline value for this indicator. Instead of calculating the share of MDA using statistics for short-term decision making AND longer-term policy formulation, the PAD baseline reflected the share of MDA using statistics for EITHER short-term decision making OR longer-term policy formulation. While progress has been made on both indicators at the time when the Bank received the request for amending this indicator, the baseline and target values in the results framework were based on the “either/or” definition. Hence, NISR made the request to adjust the values in the results framework to reflect the new (“both/and”) definition. 20 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) Rating of Overall Efficacy 53. The rating of overall efficacy is assessed as Substantial. As summarized in table 2, all IRIs overachieved, exceeding their end targets. Given PDO Indicator 1 was partially achieved, the ICR rates the overall efficacy as Substantial. The recent ISRs rated the development outcome (DO) as marginally satisfactory because the ratings utilized the PDO indicators only. The ICR exercise has enabled a more comprehensive assessment of the achievements against the PDO and determines the efficacy to be substantial. Table 2. Summary of Ratings of Results Framework Indicators Results Area 1 Results Area 2 Results Area 3 PDO Substantial PDO Substantial PDO Data for High Revenue Transparency decision accountability making PDO Partially PDO Partially PDO Fully Achieved Indicator 1 Achieved Indicator 2 Achieved Indicator 3 IRI 1 Exceeded IRI 1 Exceeded IRI 1 Exceeded IRI 2 Exceeded IRI 2 Exceeded IRI 2 Exceeded IRI 3 Exceeded IRI 3 Exceeded C. JUSTIFICATION OF OVERALL OUTCOME RATING 54. The overall outcome rating is assessed as Satisfactory (Relevance: High and Efficacy: Substantial) Even though there were minor shortcomings in efficacy, the achievements made by the Program were significant with all DLIs and PAPs fully achieved or even exceeding the targets (see annex 1). Firstly, under the Program, complex and challenging implementation of systems was achieved. In many countries, the introduction of large-scale information technology solutions is often difficult as it requires multiple technical challenges. Given that many countries have failed or gone years beyond the planned and expected time to roll out the IFMIS even at the national level, Rwanda’s full implementation of IFMIS in 416 subnational entities is a remarkable achievement. The rollout of e-Procurement in eight entities (exceeding the DLI target of five) has been expanded to all government budget entities. The system has already led to a significant increase in the transparency of bidding opportunities and contract awards, as well as producing savings by reducing paperwork and costs for advertisements.12 It also reduced delays and restricted opportunities for corruption. The impact of IFMIS and e-Procurement are expected to strengthen control over commitments and payments and help enhance transparency and accountability in the use of public funds. 55. Secondly, the introduction of new systems could not be successful unless it was coupled with change management. In order to ensure that systems are sustainable, and the technical capabilities of existing systems are used effectively, the government’s oversight for change management became increasingly important. Therefore, the issue of capacity and professionalization became a key part of the Program as a cross-cutting priority. To this end, the number of staff with professional accounting qualifications and other PFM-related qualifications became a DLI (6). Even though the government recognized the importance of the issue, the specific targets for the indicator was not established in the results framework for the government’s own strategy, SSP. As part of the Program, the specific targets for this indicator were 12Before e-Procurement was introduced, an advertisement was published in at least three newspapers, costing the government SDR 150,000 per newspaper. 21 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) established and helped turn the ambiguous objective into tangible results. The original results framework for the SSP set out areas for reform, but many of them did not have specified targets. Including DLI 6, the Program helped establish some of the missing targets of the SSP through which the evaluation of the SSP became more measurable and tangible. 56. Combined with DLI 6, the PAPs included the development and adaptation of the PFM Learning and Development Strategy (PAP 7.1), which provides a long-term vision for increasing the government-wide professionalization and capacity in PFM. Including PAP 7.1, there were 10 PAPs (PAP 1.1, 1.2, 2.2, 3.1, 3.2, 3.4, 3.5, 5.2, and SO 4.1) for the development and adaptation of strategies and new templates (PAP 1.3, 6.1). They were intended to enhance the formal rules and procedures governing the organization and management of the systems used to implement the Program. They included an amendment of the tax code on VAT, new procurement regulations, and revision of the internal audit structure. For statistics, an NISR organizational development strategy was developed (SO 4.1). Other PAPs — such as the retention strategy (PAP 2.2) for the Rwanda Revenue Authority (RRA), the skills transfer policy for IFMIS (PAP 5.2), and placement of key staff for SEAS (PAP 6.2) — were key actions for capacity building and sustainability of the Program achievements over time. These PAPs also served as risk mitigation measures since capacity issues were identified as key constraints for the achievement of the PDO. 57. In conclusion, the Program succeeded in increasing the effectiveness of PFM subsystems through the incentives provided by DLIs, while PAPs provided the foundations necessary for the achievement of DLIs . Considering the fact that many countries struggle to install modern technologies (IFMIS, e-Procurement, and e-tax) at the national level, Rwanda’s achievements remain quite remarkable. In essence, through these reforms, the PforR helped Rwanda put in place modern systems and platforms for PFM that improved efficiency and strengthened accountability and transparency. D. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 58. Prior to Program implementation, gender issues were already mainstreamed in government programs. Gender- responsive budgeting became fully operationalized in 2008. Since then, during budget formulation, all ministries and districts are required to submit gender budget statements and the gender distribution of their work force. Gender statistics are available on the NISR website. There were no specific interventions under the Program to foster gender issues; however, the World Bank’s corporate policy on gender mainstreaming helped, although indirectly, sustain the government’s efforts. Poverty Reduction and Shared Prosperity 59. The Program did not directly impact poverty reduction and shared prosperity. Nonetheless, improvements contributed to the goals envisioned in EDPRS 2 — accelerating private sector-led growth and further reducing poverty. As evidenced in the Doing Business Report 2018, time spent for paying taxes was significantly reduced (from 124 to 95 hours), which was one of the intermediate outcomes of the Program. The Program succeeded in strengthening the linkage between the government’s strategic goals and budget. As a result, line ministries are now conducting feasibility studies for large investment programs to ensure adequate scrutiny of their feasibility and the effective use of resources toward achieving national development goals. Rolling out IFMIS at the sector level made easy the consolidation of financial information at the district level, data security, and sector financial accountability. These improvements will feed into better delivery of public services to citizens. The government plans to implement IFMIS at all health centers and secondary schools. The pilot phase to further roll out IFMIS will begin in 2019. 22 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 60. As noted earlier, the Program objectives were fully aligned with the CPS, the country’s medium-term Strategy, EDPRS 2, and the seven-year government program NST1, 2017–2024. The Program established clear objectives that were realistic — given the borrower’s requirements and capacity to implement. It featured well-designed PDO elements, intermediate indicators, and DLIs. PAPs were well-sequenced with the DLIs and included realistic time frames for each PAP. 61. The Program’s results framework included indicators that were operationally significant and easier to monitor. The established baselines were clear and realistic. Overall, the targets established were generally in line with the Program objective and the borrower’s reform agenda and capacity to implement. As noted earlier, the targets established for PDO Indicator 1 were based on criteria that were less than firm as the result of attribution issues and were based on an IMF target that was reduced after rebasing prior year performance (see related section on this indicator). They should have been revisited during Level 2 restructuring when the baseline and the targets were revised for PDO Indicator 3 and DLI 6. 62. The verification protocol for DLIs13 was well designed. It was appropriate to assign the main tasks of gathering data, information, and other relevant evidence of DLI’s implementation progress and achievements to MINECOFIN, which in turn, established a special unit — the Single Project Implementation Unit (SPIU) — within MINECOFIN to carry out these tasks. The borrower requested a 25 percent advance to implement the DLIs and an additional 5 percent once the government achieved the first year DLI 3 on e-Procurement before signing the legal agreement. The verification protocol involved the Bank’s onsite visits as well as similar verifications conducted by an independent body, which was the Office of the Auditor General (OAG). Unlike many other PforRs, the verification protocol for almost all DLIs was conducted by the OAG. One DLI which was carried out by the OAG was verified by an outside independent body. The Bank team’s decision to rely on OAG seems to have been made after assessing its capacity. This proved to be a wise decision as the OAG based its verification of most of the DLIs by applying its high internal standards of value-for-money audits, which went beyond the requirements outlined in the Program. 63. The stakeholders selected for implementing the Program, such as RRA and NISR, were in line with the Program objective. MINECOFIN assigned relevant focal points to engage with each of the stakeholders. This arrangement proved to be effective as shown by good coordination among key stakeholders which continued throughout Program implementation. Outcomes also indicate that the selected group of stakeholders were committed to achieving the established targets and had the requisite levels of capacity to do so. 64. In terms of risk assessment, it is understandable why the overall risk rating was substantial given this was the first PforR on governance reforms, and as a result, many of the stakeholders were not entirely familiar with the PforR instrument. Hence the PAD noted that while the operating environment risks remained moderate, Program risks were substantial — further indicating the key risk areas were technical and fiduciary. Coordination among government leadership was one such technical risk. The selection of appropriate stakeholders substantially mitigated this risk during Program implementation along with the risk associated with human resource capacity challenges. Those identified as fiduciary risks also were relevant. The mitigation measures proposed proved to be effective during implementation. For instance, the prospect of establishing accounting and financial reporting at the district level could pose a potential 13 Please see PAD, p. 33 and annex D, for more information. 23 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) challenge in any developing country, especially when these reform measures require adequate human resource capacity to meet high quality technical standards. As the outcome of DLI 5 indicates, the government of Rwanda was able to achieve the targets for each year with a significant margin.14 65. The operation was one of the first batch of PforR engagements and represented a new type of undertaking for the client, the task team, and the Bank as an institution. Limited experience with the instrument combined with the absence of explicit operational guidance at the time meant that the project design required considerable “learning by doing.” The team succeeded admirably in designing a well-conceived operation that took full advantage of the opportunities afforded by the instrument to focus on meaningful results. That said, there were certain elements — such as the alignment of PDO indicators and DLIs, or the targeting of the PDO indicators — that demonstrated much was left to learn. 66. The mid-term assessment of PforRs conducted by IEG included positive comments on the design of this operation. 15 The ICR team agrees with this assessment. The design of the project, as the first PforR undertaken by the Governance Global Practice, was an important milestone in the development of this new instrument. 14 In assessing the Program outcome in DLI 5, the OAG did a complete examination of 416 “sectors” (subnational units), which involved the entire OAG staff. In Rwanda, for this project, the OAG (as the Supreme Audit Institution) single-handedly did the verification. According to the Auditor General (AG) and his senior staff, it was not difficult for OAG to play this role because it considered this task like other audits that it conducts regularly. Hence the verification process was hardly an “uncharted territory” for the OAG. When verifying DLIs, the OAG officers went beyond simply looking at the numbers (output) to determine that the outcomes would meet the value-for-money objectives. (The AG quoted an old saying “you take your bread where is the butter,” i.e., go for the value of money.) OAG staff conducted fieldwork to make sure the numbers presented by MINECOFIN and the quality of training they underwent could be independently verified. 15 See Program-for-Results: An Early-Stage Assessment of the Process and Effects of a New Lending Instrument, IEG, 2016. 24 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) B. KEY FACTORS DURING IMPLEMENTATION 67. One important factor that made Program implementation successful was the proper coordination of all relevant stakeholders and each one of their active engagement. In fact, the way program coordination worked provides an example that could be emulated when implementing a similar, reasonably complex PforR with multiple stakeholders. First, the commitment of all stakeholders, which was reinforced by the Imihigo, incentivized proper coordination. MINECOFIN provided the necessary leadership through regular interactions, monitoring, and other forms of assistance. Each stakeholder continuously increased its capacity to be able to implement the reform agenda supported by the Program. Except for the “law determining the sources of revenue and property of decentralized entities” which was delayed, Program implementation did not face any challenges from legislative and regulatory barriers that usually impede implementation progress. 68. Rwanda may be a rare exception among many developing and transition economies where good governance remains a strong element encompassing all aspects of economic management. The Bank’s comprehensive study, “Future Drivers of Growth in Rwanda” also endorses Rwanda’s commitment for control of corruption, rule of law, regulatory quality, and government effectiveness and political stability as key indicators of effective governance. One could discern that government agreement on initiating a project carries with it a strong commitment to achieve all or most of the key targets that are linked to the government’s strategy. Both political commitments — going back to the highest echelons of power — and features of good governance have helped the Program reach its successful conclusion, achieving almost all targets. 69. Fiduciary risks became moderate over time. Given the success of e-Procurement and procurement related to PAP, such as the design and construction of a statistics training center managed by NISR, procurement was rated as Moderate in the last mission Aide Memoire of July 2017. Environmental and Social issues never posed any challenges as the risks associated with them were assessed to be low from the start of the Program. 70. The Bank team maintained an active and close engagement with MINECOFIN. One clear example of its proactive approach was the decision for the procurement team to conduct six monthly supervision visits, similar to that of an IPF implementation, during the launch of e-Procurement (DLI 3), which was rolled out to all government procuring entities. The Bank team capitalized on the opportunities provided by the Program to implement such reforms, given the likelihood that a stand-alone e-Procurement project was unlikely to materialize soon. Except for an opportunity whereby the Bank team could have included the revision of PDO Indicators 1 and 2, the Bank team’s quality of support and reporting met the required standards. The borrower expressed its full satisfaction with the Bank’s critical advice and timely assistance. 71. Throughout project implementation, the Bank team repeatedly questioned how best to respond to issues with two of the three PDO indicators. From early on, the team and the client recognized problems with PDO Indicators 1 and 2, acknowledged that retaining the indicators created risks for project rating, and debated possible alternative indicators. Without clear and explicit institutional guidance and with no prior experience with PforR operations, the team ultimately elected to take a middle path. Mid-term revisions addressed technical issues in two areas, but did not take the more significant step of reformulating the PDO indicators. This course of action reflected a lack of individual and organizational experience with this new instrument. It was also undoubtedly influenced by concerns over impeding the very fast progress of project implementation set by the government of Rwanda. 25 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION Monitoring and Evaluation Design 72. Rwanda had already established an M&E mechanism through EDPRS 2 and PFM SSP, both of which were results based. The PforR chose to use the government M&E to the “greatest extent possible” and to augment the government’s effort to strengthen results-based development management. As part of this effort, the Program helped sharpen the results framework for the PFM SSP 2013–2018 by identifying additional indicators and specific targets. The Program supports this government reform initiative by, among others, strengthening statistical capacity in NISR and human resource capacity at the center and the districts. One Program action (P1) was the planning and M&E policy and the manual and results-based management policy.16 73. As mentioned in the Theory of Change section, the logic behind the Program was clear and straightforward, and DLIs were well defined and measurable. Also, DLIs and intermediate outcomes were coherent. Nonetheless, there was decoupling between intermediate outcomes and PDO indicators. Although substantive achievements were made in all three elements of the PDO, they were not fully evidenced by PDO indicators. 74. The Program also supported the establishment of Program M&E, which was well coordinated by the MINECOFIN SPIU, utilizing several monitoring tools, such as joint sector review, PEFA, and mid-term and final reviews. An independent user satisfaction survey was conducted (in PDO Indicator 3) in each of two years to assess the satisfaction of a wide range of stakeholders. They included both central and local governments as well as donors, research institutions, and civil society — with data produced and disseminated by NISR. M&E Implementation and Utilization 75. The results-based management, as envisaged in the M&E, continues with an increasing adherence to date and outcomes in formulating policy. ICR team’s observation, based on the information received, is that M&E data of the Program were collected and analyzed in a consistent and regular fashion. This process could be further institutionalized by establishing a common methodology and periodic evaluations and a more robust arrangement with MDA to share M&E results. This will clearly establish a link between M&E data on performance and Program management. Thus M&E results could be effectively utilized for informed decision making. The verification protocol in the Program was effective. As noted in a previous section, OAG performed a commendable task in carrying out verification to meet high standards, i.e., value-for-money audits which were not required by the agreement. In sum, the choice of OAG for the verification protocol worked well in the Program, and Rwanda’s commitment to establish an effective M&E is shown by these proactive measures. Justification of Overall Rating of M&E Quality 76. The overall M&E quality is rated as Substantial. The existing government system for M&E of the Program was sufficient to assess the outcomes. Data were collected and analyzed by the SPIU for progress reporting and the OAG for verification. M&E was data-driven and informed the decision-making process. However, there was a weak coherence 16The manual and results-based management policy were approved by the Cabinet in September 2015. The 2013 Organic Budget Law includes a clause requesting an annual activity report on gender balance (including plans for gender balance), which strengthens the M&E of gender-responsive budget implementation. 26 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) between PDO and other indicators. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Environmental and Social 77. The PAD rates the Environmental and Social effect as Low. It was consistently rated as low during the entire project implementation. In the Aide Memoire of November 2016, risk in Environment and Social issues was identified as Moderate (from Low in all previous years), but the Aide Memoire fails to provide any explicit justification for this one- time change. Fiduciary 78. The last Aide Memoire of the July 2017 mission indicates that the overall risk rating of the Program was reduced from Substantial to Moderate primarily because of the reduction of fiduciary risks. Fiduciary risks became moderate over time, and as such, the ICR team does not identify any compliance issues that had posed a threat to Program implementation. C. BANK PERFORMANCE Quality at Entry 79. Overall support of the World Bank during Program implementation has been satisfactory both in assuring quality at entry and during Program implementation. The Bank team was able to align the Program with Rwanda’s strategic priorities, which was one of the key factors in assuring the commitment of all key stakeholders during implementation. That said, the program was a new instrument at the time and the Bank had not prepared guidelines to assist its team both at the design and supervision stages. The challenges this posed during program design became more apparent during implementation. For instance, program design should have focused more on selecting a set of PDO indicators with clear links to the DLIs. However, it is important to note that the technical assessment, which was well presented in the PAD, was detailed and comprehensive. The Program’s success has clear developmental implications both in the short and medium term. Rwanda’s planning and budget include gender-responsive budgeting. Overall, as envisaged at the design stage, the Program supported the government’s initiative to promote gender issues, which included gender- disaggregated data collection of essential data such as labor statistics. There were no Environmental and Social issues that came up during Program implementation, and hence, the low risk rating provided in the PAD remains realistic. 17 Quality of Supervision 80. The current PforR is the first governance PforR in Rwanda, and the Bank team justifiably went through a period of “learning by doing.” This explains why some of the potential implementation challenges were not foreseen. Still, the Program paid close attention to strengthening institutional capacity, as clearly indicated in the DLIs. The detailed assessment of DLI outcomes in this ICR provides evidence of the support provided by the Program to enhance sustained institutional capacity and relevance. The Program’s implementation arrangement was well thought out, and MINECOFIN was indeed the best counterpart to play the coordination and monitoring roles during implementation. MINECOFIN SPIU played an invaluable role as a “secretariat” in coordinating Program activities implemented by all key stakeholders. MINECOFIN SPIU provided regular reports on the progress of DLIs and PAPs twice year, progress on expenditure by 17 Except that in the November 2016 Aide Memoire, this risk rating was revised as Moderate. 27 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) programs and subprograms, and audited financial statements of implementation agencies. The Bank team paid close attention during supervision missions to the risks and most of the mitigation measures (except those related to revising PDO Indicators 1 and 2). Where relevant, the team made appropriate adjustments.18 Although the November 2016 Aide Memoire assessed the risk in Environment and Social issues as Moderate (from Low in all previous years), no explicit justification was given for this change. The ISR rating of risk on Environment and Social issues remains low. All stakeholders acknowledged that Bank advice, its timely input, and the team’s continuous engagement met their high expectations. The team recognized issues with the PDO indicators early on in project implementation and discussed with the client options for revising the indicators. The team’s decision to retain the flawed indicators is strongly associated with its lack of experience in dealing with this instrument and the absence of institutional guidance in proactively addressing such problems. Maintenance of the existing indicators was also encouraged by the largely positive independent review of the project contained in the interim mid-term PforR review conducted by IEG. Justification of Overall Rating of Bank Performance 81. The quality of supervision met World Bank standards. The supervision reports were detailed and comprehensive, and they identified issues that required timely attention of both MINECOFIN and key stakeholders. The Bank team included a competent task team leader (TTL) in the field office, and changes in the role of the TTL at headquarters took place without disrupting the Bank team’s ability to provide timely implementation support. This was also the case in providing required assistance during Program closing and after. That said, the supervision mission missed a valuable opportunity to include the revisions of PDO Indicators 1 and 2 during the restructuring, which led to the partial achievement of both. In view of all of these factors, the ICR team rates World Bank Performance as Satisfactory. D. RISK TO DEVELOPMENT OUTCOME 82. Risk to DOs is rated as Low. First, the Program’s achievements that are linked to the government’s ongoing multiyear strategic vision — enshrined in its PFM SSP 2018-2024 and NSDS 2 — are most likely to be sustained due to strong achievement of established goals and the DLIs exceeding expected targets in most cases. A key contributing factor for this is the government’s strong commitment to this reform agenda. None of the fiduciary and technical risks identified during the design of the Program seemed to have affected any stakeholder in achieving the expected targets. Political stability and the strong endorsement from the highest echelons of the government will most likely continue as Rwanda has established an implementation culture rarely seen in most other countries in the region. 83. While internal risks likely to affect the DOs remain low, there could still be high risk external environments, particularly related to macroeconomic conditions, which are currently unforeseen, that could potentially impact the DOs. One encouraging sign was the government’s close collaboration with the PSI, which provided useful guidelines and regularly monitors key macro and fiscal indicators. PSI has already ended but the IMF is preparing a new program which is expected to be finalized by June 2019. The ICR team does not identify any major risk in the external environment that could have a serious impact on DOs. V. LESSONS AND RECOMMENDATIONS 84. The prerequisite for the success of the PforR instrument is a well-developed reform program that has been defined and is fully owned by the government. In this instance, the national program shaped the Program’s design by identifying areas in need of additional support with the World Bank’s value-added engagement. The existence of a clear 18 For example, in the July 2017 Aide Memoire where fiduciary risk was revised from Substantial to Moderate. 28 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) national program of work also clearly delineated the Program boundary. The Rwanda Public Sector Governance PforR was able to align with the country’s development programs because PFM SSP 2013–2018 and NSDS 2 2014–2019 were defined when the PforR was prepared. These two programs were recognized by the government as important vehicles to achieve the country’s medium-term goals for economic development and poverty reduction. 85. The involvement of the Supreme Audit Institution (SAI) in the verification protocol of DLIs can increase accountability of the Program as long as the SAI is proven to be independent and the strong commitment to results is shared at the high level of the government. For Rwanda, the OAG conducted verification of all DLIs except for one, which was related to an unqualified audit opinion. A dedicated team within the OAG was established and conducted verification for the entire Program implementation period. Learning by doing served to strengthen ownership of Program results, provided added clarity in the accountability for achieving results, and built institutional capacity. Overall, involving the OAG in outcome verification improved results and was a good investment for the country. It should be noted that some indicators required extra scrutiny to ensure that the promised targets were met, delaying the verification of achievements. Rwanda’s case suggests that the level of commitment and competency of both OAG and other related agencies providing similar oversight is critical, which needs to be assessed carefully during the preparation of operations. 86. The PDO indicators need to be closely connected to DLIs through a robust Theory of Change. A well-structured results framework based on a set of logically achievable steps reflecting the PDO indicators and the critical role of DLIs is essential for the success of the PforR. Clear linkages between the PDO indicators, intermediate indicators, and the supporting DLIs should be established and incorporated into the Program’s results framework as well as operation interventions. It is recommended that a Theory of Change and results framework be reviewed during Program implementation, particularly during the MTR. All DLIs were achieved or overachieved, and all PAPs were implemented in Rwanda. However, the strong performance in implementing technical reforms did not smoothly lead to achieving the broader PDO indicators. 87. The introduction of new instruments requires a period of learning and adaptation. This operation was one of the first wave of PforR operations, and the lack of experience with the instrument had clear consequences. Most significantly, a team supported by more explicit guidance, would probably have selected different PDO indicators and restructured the indicators during the MTR. The project proved to be a rich learning-by-doing undertaking for the client, the task team, and the institution. It is important for the learning on design and implementation to be captured effectively and made available to future teams. Overall, the lack of familiarity with the instrument largely affected the selection of PDO indicators and not the development impact of the project. 88. For the success of the Program, the Bank’s task team’s close and frequent monitoring and engagement were required, which necessitated greater supervision. The nature of the PforR enabled the government of Rwanda and World Bank to focus on technical issues rather than ICT procurement processes, which typically consumes IPF supervision. However, the depth of technical expertise and the rigor of monitoring for the achievement of DLIs and PAPs — and for proper verification protocols — necessitated greater supervision. Additionally, technical assistance was provided by other donors through a variety of support interventions in parallel with the PforR in Rwanda, which required more of the World Bank’s involvement in donor coordination. 89. The PforR instrument was the most appropriate instrument to be used at the time to support an important subset of government programs given their close linkages, need for careful sequencing of reforms, and focus on strengthening specific institutional capacity: The ICR assessed the counterfactual of whether a DPO or an IPF could have achieved the same results as the PforR. The consensus is that the PforR instrument provided better opportunities to sequence specific reform actions within a sector more intensively. The PforR’s success is also attributed to (i) the Bank’s effective involvement — but in “non-cumbersome ways” as described by the client which might not have been the case 29 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) with an IPF, and (ii) the government’s greater degree of flexibility and opportunities to come up with innovative, context- driven, action plans during Program implementation. The proceeding IPF provided a more targeted approach to addressing some of the remaining PFM reforms, such as the MTEF. 90. Parallel technical assistance (TA) could be effectively utilized to support Program implementation, and future PforR deign should consider the same to build requisite capacity and institutional strengthening. The Program was able to achieve positive results partly due to a number of TA activities and other analytical work conducted in parallel. For example, gaps in tax administration were identified in a tax administration diagnostic assessment conducted by the IMF. Similarly, the Bank carried out an important, comprehensive procurement assessment, which identified areas that needed further capacity building as well as strengthening to help when implementing comprehensive e-Procurement reforms. . 30 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) VI. ANNEX 1. RESULTS FRAMEWORK, PERFORMANCE ON DISBURSEMENT LINKED INDICATORS, AND PROGRAM ACTION PLAN . Annex 1A. RESULTS FRAMEWORK (i) PDO Indicators Objective/Outcome: Extent to which Districts have adopted the automated local government revenue management system Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion PDO Indicator 1: Tax to GDP Percentage 14.20 17.40 15.70 ratio 30-Jun-2013 31-Dec-2018 31-Dec-2018 Comments (achievements against targets): The ICR finds it difficult to assess the extent to which the objective of PDIO 1 is achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion PDO Indicator 2: Improved Percentage 40.00 80.00 98.00 national and sub-national transparency and 09-Oct-2014 31-Dec-2018 31-Dec-2018 accountability in the use of public funds, measured by the percent of entities submitting monthly financial 31 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) statements by due date Comments (achievements against targets): While the PDO 2 targets were met in terms of the percentage of public entities submitting monthly financial statements by due date. However, it was decided not to publish monthly financial statements. Therefore, this PDO indicator is partially met. Objective/Outcome: PDO Indicator 3: Share of MDAs using official statistics for both analysis of current developments Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion PDO Indicator 3: Share of Percentage 39.00 75.00 48.00 62.30 MDAs using official statistics for both analysis of current 14-Dec-2014 31-Dec-2018 11-Mar-2017 31-Dec-2018 developments Comments (achievements against targets): Because the user satisfactory surveys are only conducted every two to three years, the last value available for the indicator, 62.3% was from the survey carried out in 2017. (ii) Intermediate Results Indicators Results Area: Enhanced National and Subnational Revenue Collection Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion 32 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) IRI 1.1 Intermediate Results Hours 113.00 100.00 95.00 Indicator 1: Hours per year spent paying taxes 09-Oct-2014 31-Dec-2018 31-Dec-2018 Comments (achievements against targets): The end actual exceeded the original target value. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion IRI 1.2: Increase tax Number 0.00 11.00 30.00 collection measured by the number of districts using 09-Oct-2014 31-Dec-2018 31-Dec-2018 automated revenue collection system Comments (achievements against targets): The end actual exceeded the target value. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion IRI 1.3 Percentage of active Percentage 69.30 87.00 99.60 small and micro tax payers file taxes using e-tax portal, 09-Oct-2014 31-Dec-2018 31-Dec-2018 against number of active 33 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) taxpayers (percentage) Comments (achievements against targets): The end actual exceeded the target value. Results Area: Improved National and Subnational transparency and accountability in the use of public funds Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion IRI 2-1. Proportion of MDAs Number 32.00 47.00 60.00 receiving unqualified audit opinion (percent) 14-Dec-2014 31-Dec-2018 31-Dec-2018 Comments (achievements against targets): The end actual exceeded the target value. It should be noted that the final actual number reported by the OAG is 50%. However, the OAG's number included district hospitals which were not audited in FY2016-17. Additionally 2 more public Enterprises were included in FY2016-17. Therefore, ICR team uses the number corrected by MINECOFIN and determined that 60% of entities received unqualified audit opinions. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion 34 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) IRI 2.2 Proportion of sub- Percentage 0.00 80.00 100.00 districts (Sectors) using a simplified accounting and 14-Dec-2014 31-Dec-2018 31-Dec-2018 financial reporting appl Comments (achievements against targets): The end actual exceeded the target value. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion IRI 2.3 Gap between annual Percentage 20.00 11.00 5.00 budget and second year MTEF formulated previous 30-Jun-2014 31-Dec-2018 31-Dec-2018 year Comments (achievements against targets): The end actual exceeded the target value. Results Area: Improved use of public data for decision making Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion IRI 3.1 Annual national Yes/No N Y Y 35 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) agricultural survey 30-Jun-2013 31-Dec-2018 31-Dec-2018 conducted (nes/no) Comments (achievements against targets): The target was achieved as full 2016 national agricultural survey conducted, report produced, and disseminated. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion IRI3.2 Annual labor market Yes/No N Y Y statistics collected and disseminated 09-Oct-2014 31-Dec-2018 31-Dec-2018 Comments (achievements against targets): The target was achieved as first national labor force survey conducted and report produced and disseminated by June 2017. ANNEX 1B. DISBURSEMENT LINKED INDICATORS DLI I N0069 4648 TA BLE DLI 1: DLI 1: Number of pilot ministries submitting investment plan in which feasibility studies are conducted for new projects exceeding US$1 million have, per instructions in budget call circular (Number) Baseline FY1: 2014-15 FY2: 2015-16 FY3: 2016-17 Total Original values 0.00 1.00 3.00 5.00 0.00 36 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) Actual values 1.00 3.00 5.00 0.00 Allocated amount ($) 5.00 4.50 3.00 0.00 12.50 Disbursed amount ($) 5.00 4.50 3.00 0.00 12.50 Comments (achievements against targets): All targets for the FY(year) 1, 2, 3 were met and the allocated amounts disbursed fully. Actual DLI in 2017/18 - MINAGRI, MININFRA, RDB, MINEACOM, and MINIRENA have complied with the budget call circular requiring feasibility studies of projects exceeding US$1 million DLI I N0069 4649 TA BLE DLI 2: DLI 2: Number of districts using automated local government revenue management system (Percentage) Baseline FY1: 2014-15 FY2: 2015-16 FY3: 2016-17 Total Original values 0.00 3.00 6.00 11.00 0.00 Actual values 3.00 30.00 30.00 0.00 Allocated amount ($) 5.00 4.00 3.50 0.00 12.50 Disbursed amount ($) 5.00 4.00 3.50 0.00 12.50 DLI I N0069 4650 TA BLE DLI 3: DLI 3: Implementation of e-procurement (Text) Baseline FY1: 2014-15 FY2: 2015-16 FY3: 2016-17 Total Minister approved Functioning of IFMIS The e-Procurement both & e-procurement was operational in -8 No e-procurement interface protocols. pilot entities: Original values technical&financial system implemented MINECOFIN, proposals and roadmap approved. MINISANTE, MININFRA, RTDA, 37 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) RDB, and RBC as well as Gasabo and Kicukiro districts Target results Target results Target results Actual values achieved. achieved.b achieved. Allocated amount ($) 5.00 3.50 4.00 0.00 12.50 Disbursed amount ($) 5.00 3.50 4.00 0.00 12.50 Comments (achievements against targets): All yearly targets were achieved. DLI I N0069 4651 TA BLE DLI 4: DLI 4: Percent of MDAs receiving unqualified audit opinion (Percentage) Baseline FY1: 2014-15 FY2: 2015-16 FY3: 2016-17 Total Original values 32.00 37.00 42.00 47.00 0.00 Actual values 38.00 52.86 60.00 0.00 Allocated amount ($) 5.00 4.00 4.00 13.00 26.00 Disbursed amount ($) 5.00 4.00 4.00 13.00 26.00 DLI I N0069 4652 TA BLE DLI 5: DLI 5: Percent of sub-districts using simplified accounting and financial reporting application (Percentage) Baseline FY1: 2014-15 FY2: 2015-16 FY3: 2016-17 Total Original values 21.00 30.00 60.00 80.00 0.00 38 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) Actual values 91.00 100.00 100.00 0.00 Allocated amount ($) 5.00 4.00 3.50 0.00 12.50 Disbursed amount ($) 5.00 4.00 3.50 0.00 12.50 Comments (achievements against targets): Actual DLI in 2017/18 is 100 DLI I N0069 4653 TA BLE DLI 6: DLI 6: Number of Government PFM staff with at least foundation-level professional finance qualifications (Number) Baseline FY1: 2014-15 FY2: 2015-16 FY3: 2016-17 Total Original values 294.00 350.00 400.00 450.00 0.00 Actual values 311.00 417.00 665.00 0.00 Allocated amount ($) 5.00 4.00 3.50 0.00 12.50 Disbursed amount ($) 5.00 4.00 3.50 0.00 12.50 Comments (achievements against targets): Actual DLI in 2017/18 is 1257 DLI I N0069 4654 TA BLE DLI 7: DLI 7: Increased production of economic statistics and dissemination in time for strategic planning and decision making (Text) Baseline FY1: 2014-15 FY2: 2015-16 FY3: 2016-17 Total 39 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) Integrated business The NISR has enterprise survey updated the base conducted, and year of National report produced and Account from 2011 Last comprehensive Full 2014 national disseminated to 2014 and new agricultural survey agricultural survey DGP data was dates from 2008 conducted based on published on NISR’s (experimental website. lessons learned from agricultural survey was Original values 2013 experimental The following is the conducted in 2013); agricultural survey link for the produced no systematic program and report produced results: of business or labor and disseminated by http://www.statistic statistics, GDP with June 2015 s.gov.rw/publication base year 2011. /gdp-national- accounts-2016 Target results Target results Target results Actual values achieved. achieved. achieved. Allocated amount ($) 5.00 4.50 3.00 0.00 12.50 Disbursed amount ($) 5.00 4.50 3.00 0.00 12.50 Comments (achievements against targets): Actual DLI in 2017/18 - Produced mandatory economic statistics (quarterly GDP, monthly CPI, quarterly PPIs, and related trade statistics have continued to be produced within respected timelines). DLI I N0069 4655 TA BLE DLI 8: DLI 8: Open Data initiative extended to progressively include more surveys and administrative sources of data (Text) Baseline FY1: 2014-15 FY2: 2015-16 FY3: 2016-17 Total 40 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) National Data Archive Micro-data of EICV4 Open Data initiative Micro-data of (NADA), from which poverty survey extended to experimental survey Micro-data can (2013/14) was administrative agricultural survey be downloaded by released on NADA source data (2013) and at least 5 anyone free of charge, collected by the Original values percent of data of established in 2010; Ministry of Health the population and no access to for five development housing census administrative data indicators (2012) released on produced by line NADA ministries Target results Target results Target results Actual values achieved. achieved. achieved. Allocated amount ($) 5.00 3.50 3.50 0.00 12.00 Disbursed amount ($) 5.00 3.50 3.50 0.00 12.00 Comments (achievements against targets): Actual DLI in 2017/18 - The progress in implementation of data revolution policy ANNEX 1C. PROGRAM ACTION PLAN PAP_T BL Achieved Action Timing Completion Measurement (Yes/No) P2-(1) Submission of Amended tax code Due Date 30-Jun-2015 Yes Publication on RRA website on VAT exemptions to cabinet P2-(2) Staff retention strategy Due Date 30-Jun-2015 Yes RRA will share the strategy with the Bank P2-(3) Action plan to improve VAT Due Date 30-Jun-2016 Yes Official letter from MINECOFIN 41 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) revenues P3-(1) IPSAS roadmap which includes Due Date 30-Jun-2016 Yes Official letter from MINECOFIN Government accounting policy for fixed assets P1-(1) Planning and M&E policy and Due Date 30-Jun-2015 Yes Publication on MINECOFIN website manual / results-based management policy P1-(2) Revised national investment policy Due Date 30-Jun-2015 Yes Publication on MINECOFIN website P1-(3) A draft consolidated program Due Date 30-Jun-2015 Yes MINECOFIN will share program template with the template to assist FY2016/17 budget Bank preparation and monitoring and evaluation processes in identified pilot MDA P3-(2) Revised and adopted procurement Due Date 30-Jun-2015 Yes Publication on RPPA website regulations P3-(3) Submission of annual activity report Due Date 30-Jun-2015 Yes Official letter from MINECOFIN accompanied by on how plans for gender balance have annual activity report been implemented (as stipulated in Article 68 of the 2013 Organic Budget Law) P3-(4) Revised internal audit structure Due Date 30-Jun-2016 Yes Ministerial Order by the Minister of MINECOFIN adopted (Ministerial Order issued) P3-(5) Dissemination of Financial Policies Due Date 30-Jun-2016 Yes Submit the updated financial policies and and Procedures procedures to the Bank and disseminate it publicly P5-(1) Roll out of current version of Due Date 31-Jul-2015 Yes Official letter from MINECOFIN 42 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) SmartFMS to remaining entities P5-(2) Development of skills transfer Due Date 30-Jun-2017 Yes Official letter from MINECOFIN policy from IFMIS vendors to staff P6-(1) Development of reporting template Due Date 30-Sep-2015 Yes Official letter from MINECOFIN with supporting for subsidiary entities materials P6-(2) Placement of key staff with Due Date 30-Jun-2017 Yes Official letter from MINECOFIN including names, suggested profile for SEAS roles, and compliance with annex 7 P7-(1) PFM learning and development Due Date 30-Jun-2015 Yes Official letter from MINECOFIN strategy developed P7-(2) Rwandan educational institutions Due Date 30-Jun-2016 Yes MINECOFIN prepares evidence that a Rwandan accredited to provide professional training education institution is accredited under ICPAR, ACCA, AAT Comments: In addition to the Cornerstone Training Institute, three Rwandan Institutions were found accredited by KASNEB. Those institutions are Polytechnic of Byumba, School of Finance and Banking (UR-CBE) and Kigali Institute of Management (KIM). For details, one can follow the link of student resources, accredited institutions and download attachment on which those institutions are numbered 17, 33, and 67 http://kasneb.or.ke/applications-forms/ SO4-(1) Organizational development Due Date 31-Dec-2015 Yes NISR will share the strategy strategy of NISR developed 43 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT AND SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Supervision/ICR Enagnon Ernest Eric Adda, Jens Kromann Kristensen Task Team Leader(s) Nagaraju Duthaluri Procurement Specialist(s) Paul Kato Kamuchwezi Financial Management Specialist Yasmin Tayyab Social Specialist Dolele Sylla Team Member Svetlana Khvostova Environmental Specialist Aparajita Goyal Team Member Sylvie Ingabire Team Member Moses Misach Kajubi Team Member Mulugeta Dinka Team Member Christine Anyango Owuor Team Member Adrian Howard Cutler Social Specialist Tharcisse Musabyimana Social Specialist George Bob Nkulanga Team Member Emmanuel Muligirwa Social Specialist 44 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) A. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY14 31.953 210,547.40 FY15 32.251 181,762.82 FY16 0 0.00 Total 64.20 392,310.22 Supervision/ICR FY15 15.533 -19,536.26 FY16 28.200 120,156.69 FY17 26.838 137,662.16 FY18 19.503 85,123.84 FY19 14.767 100,727.91 Total 104.84 424,134.34 45 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) ANNEX 3. PROGRAM EXPENDITURE SUMMARY Actual Expenditures (Disbursement) Source of Program Type of Co- Estimates at Financing (US$) Financing Appraisal Percentage of Percentage of Actual Appraisal Actual World Bank IDA US$100 million US$ 94 million 58% 69% Borrower US$ 41 million US$ 20 million 24% 15% Other Partners MDTF US$ 30 million US$ 23 million 18% 16% Total US$172 million US$136 million 46 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) ANNEX 4. BORROWER’S COMMENTS The Borrower reviewed the Implementation Completion Report and agrees with the findings. 47 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) ANNEX 5. SUPPORTING DOCUMENTS (IF ANY) 48 of 49 The World Bank Rwanda Public Sector Governance Program For Results (P149095) ANNEX 6. Payments for Achieving Disbursement Linked Indicators Table A2.1. Payments Made for the Achievement of DLIs Payment Achieved DLIs Disbursed Disbursed Date Accumulated (SDR) Disbursement (%) 1st Year Advance Payment 1, 2, 4, 5, and 8 16,475,000 December 2014 25 Achievement 3.1 3,295,000 December 2014 30 2nd Year Advance Payment 1, 2, 3, 4, 7, and 8 15,289,622 September 2015 53 Achievement 4, 5, and 7 8,764,377 September 2015 67 Achievement 6.1 and 2 5,930,750 September 2016 76 3rd Year Advancement 1, 2, 3, 4, 6, 7, and 8 14,860,198 October 2016 98 Achievement 1,285,052 July 2017 100 49 of 49