Document of The World Bank Report No: ICR00004178 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-H6460, IDA-H8870) ON A GRANT IN THE AMOUNT OF XDR 3.5 MILLION (US$5.3 MILLION EQUIVALENT) TO THE UNION OF THE COMOROS FOR A ECONOMIC GOVERNANCE TECHNICAL ASSISTANCE PROJECT June 20, 2017 Governance Global Practice Africa Region CURRENCY EQUIVALENTS (Exchange Rate Effective December, 31 2016) Currency Unit = Comorian Franc (KMF) US$1.00 = KMF 466 US$1.00 = XDR 0.74 FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS ABGE Economic Governance Technical Assistance Project (Projet d’Appui à la Bonne Gouvernance Economique) AF Additional Financing AfDB African Development Bank AIMF International Association of Francophones local Government Executives (Association Internationale des Maires Francophones) ARMP Autorité de Régulation des Marchés Publics (Procurement Oversight Authority) AU African Union CL4D Collaborative Leadership for Development CREF Economic and Financial Reforms Unit (Cellule des Réformes Economiques et Financières) CSR Civil Service Reform DAF Administrative and Finance Director (Directeur Administratif et Financier) DGCPT Directorate for Public Accounting and Treasury (Direction générale de la comptabilité publique et de la trésorerie) DPO Development Policy Operation DNCMP National Department of Public Procurement Control (Direction Nationale du Contrôle des Marchés Publics) ECF Extended Credit Facility EU European Union FM Financial Management GISE Government Human Resources Management Information System (Gestion Informatisée des Structures des Effectifs) HAFOP High Authority of Civil Service (Haute Autorité de la Fonction Publique) HIPC Highly Indebted Poor Countries HR Human Resources HRM Human Resource Management HRMIS Human Resources Management Information System ICBT Citizen Initiative of Budgetary Transparency (Initiative Citoyenne pour la Transparence Budgétaire) ICR Implementation Completion and Results Report IFMIS Integrated Financial Management Information System IMF International Monetary Fund ISN Interim Strategy Note IT Information Technology M&E Monitoring and Evaluation MDAs Ministries, Departments, and Agencies MOF Ministry of Finance PAD Project Appraisal Document PDO Project Development Objective PEA Political Economy Analysis PEFA Public Expenditure and Financial Accountability PFM Public Financial Management PI Performance Indicator PIU Project Implementation Unit PPF Project Preparation Facility PRCI Institutional Capacity Building Project (Projet pour le Renforcement des Capacités Institutionnelles) PRSP Poverty Reduction Strategy Paper RCIP Regional Communication Infrastructure Project RRI Rapid Results Initiative SCA2D Accelerated Growth and Sustainable Development Strategy (Stratégie de Croissance Accélérée et de Développement Durable) TA Technical Assistance TADAT Tax Administration Diagnostic Assessment Tool TTL Task Team Leader Senior Global Practice Director: Deborah Wetzel Practice Manager: Chiara Bronchi Project Team Leader: Anne-Lucie Lefebvre ICR Team Leader: Abel Bove ICR Primary Authors: Emeline Bredy and Abel Bove UNION OF THE COMOROS Economic Governance Technical Assistance Project CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph Executive Summary ............................................................................................................ 1 1.  Project Context, Development Objectives, and Design............................................ 2 2.  Key Factors Affecting Implementation and Outcomes .......................................... 10 3.  Assessment of Outcomes ........................................................................................ 17 4.   Assessment of Risk to Development Outcome ...................................................... 22 5.   Assessment of Bank and Borrower Performance ................................................... 23 4.  Lessons Learned ..................................................................................................... 25 5.  Comments on Issues Raised by Borrower/Implementing Agencies/Partners ........ 28 Annex 1. Project Costs and Financing .............................................................................. 30 Annex 2. Output by Component ....................................................................................... 31 Annex 3. Economic and Financial Analysis ..................................................................... 36 Annex 4. Bank Lending and Implementation Support/Supervision Processes................. 37 Annex 5. Beneficiary Survey Results ............................................................................... 39 Annex 6. Stakeholder Workshop Report and Results ....................................................... 40 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR submitted ........ 41 Annex 8. Comments of Co-financiers and Other Partners/Stakeholders.......................... 44 Annex 9. List of Supporting Documents .......................................................................... 45 MAP A. Basic Information Economic Governance Country: Comoros Project Name: Technical Assistance Project Project ID: P102376 L/C/TF Number(s): IDA-H6460, IDA-H8870 ICR Date: 04/13/2017 ICR Type: Core ICR UNION OF THE Lending Instrument: TAL Borrower: COMOROS Original Total XDR 1.20 million Disbursed Amount: XDR 3.49 million Commitment: Revised Amount: XDR 3.50 million Environmental Category: C Implementing Agencies: ABGE Projet d’Appui la Bonne Gouvernance Economique Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 09/29/2006 Effectiveness: 02/28/2011 11/12/2013 Appraisal: 11/22/2010 Restructuring(s): 04/08/2016 Approval: 01/31/2011 Mid-term Review: 04/30/2012 03/26/2012 Closing: 12/31/2013 12/31/2016 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Development Outcome: Substantial Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Unsatisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Moderately Moderately Quality at Entry: Government: Unsatisfactory Unsatisfactory Implementing Quality of Supervision: Moderately Satisfactory Moderately Satisfactory Agency/Agencies: i Overall Bank Overall Borrower Moderately Moderately Satisfactory Performance: Performance: Unsatisfactory C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments (if Indicators Rating Performance any) Potential Problem Project at any time Yes Quality at Entry (QEA): None (Yes/No): Problem Project at any Quality of Supervision Yes None time (Yes/No): (QSA): DO rating before Moderately Closing/Inactive status: Unsatisfactory D. Sector and Theme Codes Original Actual Major Sector/Sector Public Administration Central Government (Central Agencies) 36 36 Financial Sector Public Administration - Financial Sector 64 64 Major Theme/Theme/Sub Theme Private Sector Development Enterprise Development 13 13 MSME Development 13 13 Public Sector Management Public Administration 31 31 Administrative and Civil Service Reform 25 25 Transparency, Accountability and Good Governance 31 31 Public Finance Management 31 31 Public Expenditure Management 31 31 E. Bank Staff Positions At ICR At Approval Vice President: Makhtar Diop Obiageli Katryn Ezekwesili Country Director: Mark R. Lundell Constantine Chikosi ii Practice Chiara Bronchi John Panzer Manager/Manager: Project Team Leader: Anne-Lucie Lefebvre Aurelien Kruse ICR Team Leader: Abel Paul Basile Bove ICR Primary Author: Abel Paul Basile Bove Emeline Bredy Herimpamonjy Mavoarisoa Ranaivoarivelo F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The development objectives of the Economic Governance TA project are to increase the efficiency, accountability and transparency of public financial management, and to improve the management of civil service human resources and wages. Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years The composition of budget expenditures deviates from the original budget by Indicator 1: less than 5%. (PEFA PI-2) Sectoral classification YR1: n.a. could not be made in the YR2: Sectoral Value 2007 PEFA study as the classification is n.a. n.a (Quantitative or information was not possible. Indicator Qualitative) available for islands. P-I2 is rated D+ Indicator P-I2 was rated YR3: Indicator P- D. I2 is rated D+. Date achieved 12/28/2010 — — — Comments (including % This indicator was dropped in 2016 (restructuring paper Report # RES21945). achievement) Indicator 2: Budget execution reports published (on the web or national media) regularly Budget execution reports YR1: N/A are produced quarterly Value YR2: Budget but with only summary PI-10 rated B+ PI-10 rated D (Quantitative or reports are made information and limited Qualitative) available to the distribution. public iii YR3: The comprehensivenes s of the reports is improved to include social sectors and/or priority sectors. Date achieved 12/28/2010 02/01/2011 10/16/2013 10/31/2016 Comments This indicator and targets were revised in 2013 (AF project paper report # (including % 81241-KM). Revised target not met based on the 2016 PEFA self-assessment achievement) report. The share of civil servants paid through the automated payroll system is above Indicator 3: 95%. The share of civil The share of 99% of the civil Payrolls are established servants paid civil servants servants are paid Value largely on manual basis through the paid through through the (Quantitative or with limited automated automated payroll the automated automated payroll Qualitative) control mechanisms. system is above: payroll system system as of YR1: 50% is above 95% August 2016. YR2: 70% in June 2016. Date achieved 10/16/2013 02/01/2011 10/16/2013 08/31/2016 Comments Targets were revised in 2013 (AF project paper report # 81241-KM). Revised (incl. % target met. achievement) (b) Intermediate Outcome Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Indicator 1: A new organic law for PFM prepared for adoption. YR1: Studies carried out and Existing texts (Organic initial proposals Law on Law of Finance, made on Decree public fundamental A new organic law Value accounting, principles. on PFM (LOLF) (Quantitative Classification, Chart YR2: Draft was adopted in or Qualitative) of account) take no law presented at 2012. account of Comoros’ national validation unique constitutional workshop. evolution. YR3: Law submitted for iv adoption to the Parliament. Date achieved 12/28/2010 02/01/2011 — 12/31/2012 Comments (including % Target achieved. achievement) Indicator 2: Annual budget prepared in timely manner (PEFA PI-11) YR1: Study on a new budget preparation Belated budget approval schedule and by the legislature or methodology. similarly mandated body; PI-11 rating is B YR2: Regulatory Lack of active Indicator PI- based on the June Value act on participatory involvement 11 is rated at 2016 PEFA self- (Quantitative budget preparation of federal and island B. assessment. PEFA or Qualitative) is adopted. administrations in the check obtained in YR3: Budget preparation of budget October 2016. preparation submissions. Indicator calendar is PI-11 was rated D+. respected (FY13) Indicator PI-11 is rated at C. Date achieved 12/28/2010 02/01/2011 10/16/2013 10/31/2016 Comments Targets were revised in 2013 (AF project paper report # 81241-KM). Revised (including % target achieved. achievement) Indicator 3: Limited use of budgetary adjustments during the FY (PEFA PI-16) Budgetary adjustments are relatively frequent PI-16 rating is C+ during the year. reflecting YR1: n.a. based on June 2016 Value a poor anticipation of YR2: PI-16 rated PI-16 rated C+ PEFA self- (Quantitative cash flows and inefficient D+ assessment. PEFA or Qualitative) tracking of expenditures YR3: PI-16 rated check obtained in during the year. C October 2016. Indicator PI-16 was rated D. Date achieved 12/28/2010 — 10/16/2013 10/31/2016 Comments Targets were revised in 2013 (AF project paper report # 81241-KM). Revised (including % target achieved. achievement) Enhanced computerization of budget management and reporting (Indicator PI- Indicator 4: 24). Value Indicator PI- PI-24 rating is D+ PFM information systems YR1: Study on the (Quantitative 24 is rated B based on June 2016 are rudimentary. computerization or Qualitative) PEFA self- v Indicator PI-24 is rated master plan assessment. PEFA D. completed check obtained in YR2: Selection October 2016. and rollout of budgetary module of the applied PI- 24 is rated D+ YR3: Selection and rollout of accounting module of the applied software Publication of a monthly four phase statement by entities and consolidated, consistent with approved budget 45 days after FY period end. Date achieved 12/28/2010 02/01/2011 10/16/2013 10/31/2016 Comments Targets were revised in 2013 (AF project paper report # 81241-KM). Revised (including % target not achieved. achievement) Publication of public tender procurement notice in five targeted ministries for Indicator 5: any public procurement financed by the national budget or by donors according to law No. 11- 027/AU Publication of public tender procurement notice in five targeted ministries for Value any public 90 % of tenders on (Quantitative No publication procurement Not achieved. national budget or Qualitative) financed by the national budget or by donors according to law No. 11- 027/AU Date achieved 10/16/2013 10/16/2013 05/2016 12/31/2016 Comments New indicators with 2013 Additional Financing and revised in 2016 (including % restructuring vi achievement) Enhanced efficiency in controls of civil service controls of civil service wage Indicator 6: bill accuracy (through cross checks across databases, timely updates and internal controls) (PEFA PI-18) YR1: Physical census of civil servants and cleaning of Controls are ad hoc and computerized manual. There is no roster completion automated system to of user manuals PI-18 rating is D+ detect errors or abuses. validation of all PEFA based on June 2016 Value A management tool has compensation indicator PI- PEFA self- (Quantitative been rolled out (with PPF parameters 18 is rated B+. assessment. PEFA or Qualitative) support) but it is not yet including fringe check obtained in operationalized to issue benefits October 2016. payroll. PEFA indicator YR2: Payroll PI-18 was rated D. managed automatically via the new tool. PEFA indicator PI- 18. Date achieved 12/28/2010 02/01/2011 10/16/2013 10/31/2016 Comments Targets were revised in 2013 (AF project paper report # 81241-KM). Revised (including % target not achieved. achievement) Indicator 7: A comprehensive strategy for civil service management reform is adopted. Value The CSR strategy (Quantitative No CSR strategy is adopted and — Achieved or Qualitative) disclosed. Date achieved 10/16/2013 10/16/2013 — 05/31/2016 Comments (including % New indicators with 2013 Additional Financing. achievement) G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (US$, millions) 1 01/01/2012 Satisfactory Satisfactory 0.82 2 07/31/2012 Satisfactory Satisfactory 1.14 3 03/17/2013 Satisfactory Satisfactory 1.34 4 01/01/2014 Satisfactory Satisfactory 1.46 vii 5 06/10/2014 Moderately Satisfactory Moderately Satisfactory 2.06 Moderately 6 11/25/2014 Unsatisfactory 2.37 Unsatisfactory Moderately Moderately 7 05/26/2015 2.66 Unsatisfactory Unsatisfactory 8 10/26/2015 Moderately Satisfactory Moderately Satisfactory 2.99 Moderately Moderately 9 05/10/2016 4.12 Unsatisfactory Unsatisfactory Moderately Moderately 10 11/16/2016 5.03 Unsatisfactory Unsatisfactory H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring and Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in US$, millions Additional Financing to fund the budget and accounting software and reallocation of proceeds. Extension of closing date from December 31, 2013, to June 30, 2016. No changes for the PDO and for project components. Results Framework updated and 11/12/2013 Y S S 1.46 consolidated. A new PDO-level results indicator introduced to monitor progress in public access to key sector fiscal information (indicator C). Two new intermediate results indicators to monitor procurement (indicator 1.5) and civil service management reforms (indicator 2.2). Extension of the project closing date (P102376) for a period of six months, from June 30, 2016, 04/08/2016 N MS MS 3.89 until December 31, 2016, and restructuring of the Results Framework. viii I. Disbursement Profile US$, millions 14 12 10 8 6 4 2 0 2011 2012 2013 2014 2015 2016 Original Formally revised Actual ix Executive Summary 1. The Comoros - Economic Governance Technical Assistance Project was approved on January 31, 2011, and was effective on February 28, 2011. Following the failed secession attempt in 2008 from Anjouan island, the Comoros entered a period of reform consolidation, supported by international partners and leading to the highly indebted poor countries (HIPC) completion point in December 2012. The objective of the project was to increase the efficiency, accountability, and transparency of public financial management and to improve the management of civil service human resources and wages. 2. The project had three components: (a) rationalizing PFM institutional frameworks and processes mostly related budget preparation and execution; (b) supporting civil service reforms, including wage bill management, putting in place the information system to effectively control the wage bill, and improve human resources deployment and management; and (c) demand-driven facility to finance change management activities. The project was financed through a Technical Investment Loan of US$1.8 million equivalent, fully financed by an IDA grant. 3. The project, originally planned for three years, was extended three times, making a total duration of the project nearly six years. The US$3.5 million additional financing (AF) was necessary to close the funding gap in the budget and accounting software (SIMBA), and upgrade the integrated Government Human Resource Management Information System (Gestion Informatisée des Structures des Effectifs, GISE). When the project closed on December 31, 2016, the final disbursements were 99.1 percent of the revised project financing. 4. Overall, owing to the post-conflict political momentum, the project was instrumental in laying out the basis for budget preparation, budget reporting, and wage bill reporting in a cost- effective way and in a context of limited capacity and institutional complexity and fragility. Key achievements were the review of the existing regulatory framework, the adoption of a new organic Public Financial Management law, the reorganization of the financial administration setup to reflect the new organic law, and the Integrated Financial Management Information System (IFMIS) rollout with the SIMBA software. The project, and notably the IFMIS, has helped improve timeliness of budget preparation, and decrease budgetary adjustments during the fiscal year. GISE has been operational since January 2016 as an integrated personnel and payroll management system for the Union and Island Governments. 5. However, the gains in efficiency have not been fully reaped yet. The deployment of SIMBA/GISE interface was not materialized and the deployment of GISE has not yet translated in human resources data consolidation. Transparency in public finance improved only marginally. The capacity building provided to the Citizen Initiative of Budgetary Transparency to encourage a bottom-up demand for transparency was effective only in the last months of implementation. 6. Finally, the strategy for improved human resource management (HRM) supported by the project has not been implemented yet. The main output is the HRM strategy—officially adopted seven months before the closing date, and implementation has not started yet, given the lack of leadership because the head of High Authority of Civil Service (Haute Autorité de la Fonction Publique, HAFOP) was not replaced after 2014. 7. The Implementation Completion and Results Report (ICR) concluded that the overall project outcome rating is Moderately Unsatisfactory. The objective was highly relevant at the time of appraisal and it is still relevant to date. The project design, particularly the Project Development 1 Objective could have been less ambitious. Nonetheless the project was successful in laying out the vital foundational steps for public finance and civil service management. It offers a range of lessons for similar projects, particularly in fragile, post-conflict, and complex small-islands states. 1. Project Context, Development Objectives, and Design 1.1 Country Context at Appraisal Country Context 8. The Comoros is a small and fragile state which at the time of project appraisal faced significant governance challenges. Years of political instability had eroded the legitimacy and accountability of the state. Poor economic governance and inadequate institutional capacity, at all levels of public administration, had contributed to this delegitimization and impeded economic growth and poverty reduction. Poor management of public finances and procurement, lack of controls, and a poor-performing justice system resulted not only in poor quality of services but also in distrust for government institutions. 9. This situation was compounded by disorderly decentralization and federalization processes, following the ‘reconciliation’ agreements of 2001. While far-reaching state decentralization, in essence federalism, embodied in a new ‘Union of the Comoros’ three autonomous islands, formed the cement of the political reconciliation process, its effects on economic management were disastrous. The result was (a) a debilitating conflict over respective competencies of the Union and islands (against a backdrop of constitutional confusion) leading to a near breakdown of policy making and financial management (FM) systems, (b) a major dilution of scarce capacity as many agents left their positions in the central administrations to be promoted in the island structures, and (c) an exponential increase in the civil service headcount as each autonomous island sought to replicate the full range of public administrations without concern for administrative efficiency or fiscal sustainability. 10. In the following years, the authorities gradually took steps to restore some order in public finances and civil service management. To improve budget preparation and monitoring, following a period of several years when the island and Union budgets were prepared and executed in almost complete isolation, a Budgetary Committee was set up in late 2004, including representatives of the Union and island Governments, to coordinate the preparation of a national budget for 2005.1 Since then, the Budgetary Committee has provided a structured and consensual forum for discussing the revenues and expenditure estimates of each entity throughout the fiscal year and for agreeing on a national budget to be submitted to the Union Assembly. With respect to civil service, in light of unchecked civil service growth and mounting internal arrears, the authorities adopted a strategy to better control growth of the wage bill. A High Authority of Civil Service (Haute Autorité de la Fonction Publique, HAFOP) was established by law 2—again with representatives of the Union and islands—whose main mission was to establish and maintain a single consolidated personnel roster for all entities, control new entries into the civil service 1 Previously, Anjouan had operated in complete isolation, including with respect to budget preparation, execution, and reporting. 2 Article 25 of Law 04-006 of the Union Assembly and September 7, 2006, Decree no 6-163/PR. 2 through competitive examinations, and work on a reform plan to rationalize human resources (HR) deployment and ensure fiscal sustainability. 11. A new secessionist crisis in 2008 put a temporary halt to reforms, but, in fine, created the conditions for a fundamental revision of the reconciliation and decentralization/federalization processes. Following Anjouan’s failed secession attempt in 2008, the Comoros entered a period of reform consolidation. After the deposition of ‘rebel’ Anjouanese strongman Mohamed Bacar by federal and African Union (AU) troops, elections brought to power a pro-Union candidate in Anjouan. The following year, in May 2009, Comorians adopted, by referendum, a series of amendments to the 2001 constitution, paving the way for a fundamental reform of the state that would streamline and rationalize Union island relationships and federal administrative arrangements. Specifically, the amendments significantly curtail the autonomy of the islands3 and reaffirm the unity of the state and the preeminence of the federal government. 12. In this context, stronger Government commitment and cohesion triggered renewed external support. The Government initiated a staff-monitored program with the International Monetary Fund (IMF), followed by a three-year arrangement under the Extended Credit Facility (ECF) in September 2009. The authorities also re-engaged with the European Union (EU) and the African Development Bank (AfDB).4 In 2010, the World Bank’s Board of Executive Directors approved the first budget support operation in favor of the Comoros. In June 2010, the Comoros reached the highly indebted poor countries (HIPC) initiative decision point (completion point in December 2012). 13. However, reforms could only take hold if the legal framework was clarified to reflect the new constitutional reality and if local capacity for implementation could be built rapidly. While at the time the Government of the Comoros signaled a strong commitment to reforming the state—including by staking considerable political capital on obtaining popular validation and committing with development partners to a detailed reform agenda—the task of implementation remained almost intact. Implementing an ambitious Public Financial Management (PFM) reform agenda might have brought about a new set of problems if attention was not paid to ensuring prior review and clarification of the legal framework. Moreover, to translate intentions into concrete implementation, the capacity of the administration to carry out and follow through on reforms was so low that hands-on support by the donor community was key. Sector Context at Appraisal 14. Confirming a new commitment to reform, the Union of the Comoros adopted a Poverty Reduction Strategy Paper (PRSP) for 2010–2014, which was endorsed by partners, including the World Bank. The strategy had the following core objectives: (a) stabilize the economy and lay the groundwork for strong and equitable growth, (b) strengthen key sectors by focusing on institution-building and ensuring a broader role for the private sector, (c) strengthen governance and social cohesion, (d) improve the health status of the general public, (e) promote education and vocational training with the aim of developing human capital, and (f) promote environmental sustainability and civilian security. 3 Whose Presidents, Ministers, and Parliaments were demoted to ‘Governors’, ‘Advisors’, and ‘Councils’. 4 After years of absence due to unpaid arrears. 3 15. In support of its PRSP, the Government finalized a comprehensive PFM strategy for the 2010–2019 decade and adopted a detailed action plan for 2010–2012. The proposed strategy was organized around four pillars: 5 (a) modernizing the legislative and regulatory framework and strengthening institutional capacity, (b) rationalizing budget preparation, (c) strengthening budget execution, and (d) increasing transparency through improved internal and external controls. The action plan provided a detailed, sequenced, and budgeted list of actions mapped to specific lead institutions within the administration. Its implementation relied on two committees that included Union and island representatives: the Economic and Financial Reforms Unit (Cellule des Réformes Economiques et Financières, CREF), which acts as a secretariat of the Comité Budgétaire 6 and was responsible for the implementation, as well as monitoring and evaluation (M&E) of the national PFM reform strategy, and HAFOP that was in charge of leading the civil service reform (CSR), including management of personnel rosters, new recruitments into the civil service, and implementation of the organic frameworks. 16. Notwithstanding this new dynamism for PFM reforms, important challenges remain because of the country’s very low starting point. The 2007 Public Expenditure and Financial Accountability (PEFA) review rated the Comoros at the lowest possible score, on 22 out of 28 key indicators. This evaluation emphasized inadequacies at all levels of PFM (from formal institutions and procedures to actual implementation and processes) and put the Comoros squarely in the ‘lower performance league’ of African countries.7 17. Major weaknesses contributed to reduce the Government’s efficiency in budget preparation, execution, and reporting. Despite improvements and efforts to enhance cooperation through the Comité Budgétaire, budget preparation remained non-participatory with consultations between the Union and islands fragmented and unstructured. As a result, the budget was established mainly on the basis of economic and financial constraints and was largely disconnected from policy priorities. With revenues barely covering wages and recurrent expenditures, there was limited scope for using the budget as a tool for resource allocations. In addition, the budget classification was still based on administrative classifications and did not permit establishing a direct link between the budget (preparation and monitoring) and the Government’s policy priorities or tracking priority expenditures through the budget cycle. With respect to execution, the legal and procedural framework was outdated and moreover poorly understood even by those in charge of the administrative phase of execution (credit managers). As a result, the normal procedures were often circumvented through a proliferation of exceptional procedures and significant gaps appeared between the budgeted and actual amounts. Thus the informational and authorization functions of the budget were severely undermined. At the treasury level, serious structural weaknesses were again compounded during the reconciliation period, which saw a breakdown in communication and systems across levels of the Government. As a result, key public accounts documents were never produced. 18. The PFM information systems and financial control mechanisms were generally rudimentary and obsolete and lacked an Integrated Financial Management Information 5 This strategy is further divided into 16 subprograms, which will be rolled out in a series of three-year action plans. CREF has been tasked with implementing the strategy and reporting on progress on a bi-yearly basis. 6 Comité Budgétaire: multi-stakeholder committee part of the executive. 7 For a regional comparison, see Andrews, Matt. 2010. “How Far Have Public Financial Management Reforms Come in Africa” Faculty Research Working Paper, Harvard Kennedy School. 4 System (IFMIS). At the time of appraisal, except for the recently implemented ASYCUDA++ for customs operations, the only systems in place were software applications for wage bill calculation and external debt management. 19. The Comoros’ civil service absorbed an inordinate share of the Government’s resources while delivering poor quality services to the population. During the period of federal decentralization, most central administrative structures were duplicated in each autonomous island without regard for administrative efficiency or financial sustainability. Correspondingly the civil service roster increased from less than 6,000 in 1999 to some 12,000 in 2008, with a concomitant rise of wage and salaries expenditures from 6.2 percent to 9 percent of gross domestic product, a loss of institutional memory, and a dilution of already scarce HR. Moreover, the often discretionary nature of these appointments had weakened the significance of job descriptions within each public institution. 20. The Government took steps to improve the effectiveness of public administration and gradually reduce the wage bill, but much remained to be done at the time of appraisal. HAFOP supervised the establishment of a unified computerized civil service roster (for the Union and islands) and contracted the development of an integrated civil service and wage management software, Government Human Resource Management Information System (Gestion Informatisée des Structures des Effectifs, GISE) first version which became effective in 2011. It has also carried out a vast review of the existing organizational structures of each administration (cadres organiques) an updated version of which was adopted by Parliament in 2010. Nevertheless, there were numerous remaining challenges. The civil service roster could not immediately become a useful management tool because the data validation of the recent physical census was still needed. Also, much work was needed to develop feasible strategies for implementing the new organizational structures over time and with due consideration for social consequences. Rationale for World Bank Involvement at the Time of Approval 21. The AGBE was part of the donors’ effort to support the Government’s PFM and CSR programs. It was aligned with the World Bank’s Interim Strategy Note (ISN) that indicated that the World Bank would help the Comoros consolidate national reconciliation through building state capacity and increasing accountability with a Development Policy Operation (DPO) and technical assistance (TA) operations. Table 1. Ongoing and Planned PFM and CSR Donor Support at the Time of Design IDA IMF AfDB France EU UNDP PFM Fiscal administration and policy — X X X — — Customs administration and policy — — — — — — Budget and public expenditure X X X X Central banking supervision X Statistics and M&E — — X — X X CSR CSR X — — — — — Note: UNDP = United Nations Development Programme. 5 22. The AGBE—together with the AfDB—aimed at supporting the policy reforms that had been agreed by the Government in (a) the IMF’s ECF-supported program, (b) the World Bank’s DPO approved in June 2010, and (c) the HIPC decision point document. 23. The AGBE would help the Government build the capacity needed to implement its policy reforms. It was designed as only a first step in a longer-term effort, to account for the relatively risky environment and with the objective to attain a series of short-term results. The project targeted a modest set of activities, where quick wins could be reaped and focused mostly on building capacity at the level of the ministry of Finance (MOF) and a few institutions recognizing that obtaining the needed buy-in of a broader range of stakeholders (including line ministries) would require more sustained engagement. In turn, building a successful a track record would provide the needed assurances to scale up the World Bank’s involvement going forward. Nonetheless, the project would also pilot change management activities through the proposed demand-driven component to enhance ownership of reforms and prospects for actual implementation. 1.2 Original Project Development Objectives (PDO) and Key Indicators 24. The original PDOs are to increase the efficiency, accountability, and transparency of public financial management and to improve the management of civil service human resources, and wages. 25. Increased efficiency, accountability, and transparency of PFM was to be measured by two PDO-level indicators. The first indicator was PEFA indicator PI-2 as it was considered that this indicator, which addresses the deviation of the composition of budget expenditures from the original budget, would help measure the impact of the project activities on the Government’s ability to effectively prepare the budget. The Union of the Comoros received a score of D in the 2007 PEFA evaluation. The second indicator was to be evaluated on the timely publication and availability of budget execution reports to the public. 26. Improved management of civil service human resources and wages was to be measured by the share of civil servants paid through the automated payroll system. This indicator was intended to capture how the project through its support to the census of civil servants, human resource management (HRM), and Human Resources Management Information System (HRMIS) influenced the extent to which the wage bill would be rationalized and the performance of civil service improved. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and Reasons/Justification 27. No revisions were made to the original PDO. 28. The Results Framework was updated and consolidated during the restructuring and AF process to (a) scale up the cumulative project target values considering lengthened implementation time frame and (b) integrate new indicators to capture the broader scope of project activities. A new PDO-level results indicator was introduced to monitor progress in public access to key sector fiscal information. The original first indicator on the deviation of the composition of budget expenditures from the original budget (PEFA indicator PI-2) was dropped as budget 6 credibility was one of the main issues faced by the client and was too ambitious to be addressed under the project. The final revised three key indicators are as shown in table 2. Table 2. Revised Key indicators Revised Key Indicators Changes Budget execution reports published (on the web or national media) Indicator regularly. PEFA PI-10 Revised 1 Former original key indicator 2. Revision allowed to express the targets in (2013) terms of PEFA indicator PI-10. Budgetary information of union government and islands for key New Indicator sectors are publicly available on the Ministry of Finance website (2013) 2 Introduced in 2013 and revised in 2016. Also measured with PEFA Revised indicator PI-10. (2016) The share of civil servants paid through the automated payroll system Indicator is above 95% Revised 3 (2013) Revised from 90% to 95% to account for AF. 1.4 Main Beneficiaries 29. The primary direct beneficiaries of the project were the institutions responsible for managing public resources and administration as well as civil society. These are mainly units involved in the budget cycle and in HRM: the Union- and island-level budget offices and treasuries; the Fiscal Reform Monitoring Unit of the MOF (CREF); the civil service department (HAFOP - Haute Autorité de la Fonction Publique); the project implementation unit (PIU) of the Economic Governance Technical Assistance Project (Projet d’Appui à la Bonne Gouvernance Economique, ABGE); information technology (IT) units; MOF; Treasury, Directorate of Public Accounting and Treasury (Direction générale de la comptabilité publique et de la trésorerie, DGCPT); Parliament; Procurement Oversight Authority (Autorité de Régulation des Marchés Publics, ARMP); National Department of Public Procurement Control (Direction Nationale du Contrôle des Marchés Publics, DNCMP); Anticorruption Agency; Directorate for Civil Service; Directorate Financial Control, Government and Office of the Inspector General of Finance; and Citizen Initiative of Budget Transparency (ICBT). 30. A more efficient and transparent management of public resources and of the civil service would ultimately benefit the population at large. A more strategic use of public resources—and enhanced tracking thereof—would particularly benefit the poor by ensuring that priority, poverty-reducing expenditures are prioritized in budget preparation and execution. Moreover, better management of the civil service would ensure that considerable savings can be made on wage bill expenditures and that services are improved by rationalizing the deployment of personnel across services and geographic areas. 1.5 Original Components (as approved) 31. The initial project, as described in the Project Appraisal Document (PAD), primarily focused on core PFM and civil service management reforms. To this end, the project sought to clarify PFM institutional and legal framework, strengthen the budgetary processes, and support 7 gradual computerization of public finances. The project components were selected taking into account (a) the modest available IDA resources, (b) the scarce national capacity for implementation, and (c) the absence of track record of reform. To ensure flexibility, the project included an unallocated demand-driven facility to help increase the space for reforms and respond to unforeseen needs during implementation. Component 1: Rationalizing PFM Institutional Frameworks and Processes (US$795,300) 32. This component included three subcomponents, addressing, respectively, PFM institutional framework, budget process, and public finance gradual computerization. The aim of the component was to build capacity in implementing PFM reforms by clarifying the legal framework, rationalize some aspect of budget preparation and execution, and promote the computerization of the expenditure chain. 33. The legal and institutional framework clarification subcomponent aimed, in the context of the 2009 constitutional amendments, at defining the links between the budgets of three islands and the Union and at revisiting the principles governing revenue allocation across entities. The objectives were to help the Government (a) clarify roles, responsibilities, and accountability in key financial administrations (Treasury and Financial Controller) and (b) regroup and, when necessary, update existing texts and procedures to conform with the new constitutional setup. 34. The strengthening of the budgetary process subcomponent aimed at supporting budget preparation and execution to (a) develop a realistic calendar for budget preparation; (b) clarify the budget preparation methodology; (c) clarify and codify credit managers’ roles, responsibilities, and required qualifications;8 and (d) provide training to credit managers. 35. The third subcomponent supported the gradual computerization of public finance and aimed at computerizing 15 positions nationwide, where budget data would be periodically inputted by the relevant services. The first step would be support to the creation of a master plan (schema directeur informatique) and the second step would be the rolling out of a system for budget operations, covering the entire budgetary process from preparation to execution, including an accounting module. Component 2: Supporting Civil Service Reforms, including Wage Bill Management (US$170,220) 36. The objective of this component was to empower the decision makers by providing them with the necessary information and tools to effectively control the civil service wage bill and rethink HR deployment and management. This component included the following three subcomponents: (a) The first subcomponent covered the physical census of civil servants to help, through the furnishing of small equipment, TA, and enumerators, cleaning up the database and collecting additional information on civil servants. 8 Directeurs Administratifs et Financiers. 8 (b) The second subcomponent aimed at supporting HRM through development of strategies and implementation of a new organic framework (that is, staffing plan) of the civil service. This support would include (i) job evaluation, (ii) assistance to HAFOP in developing recruitment methodologies, and (iii) technical support for communication of the strategy. (c) The third subcomponent dealt with the computerization of the wage bill management. This support would be made available through support to the existing software, GISE, developed with the project preparation funding by (i) preparing manuals and providing training to users and (ii) improving GISE by cleaning up and validating the data and ensuring its safety and integrity. Component 3: Demand-driven Facility (US$130,000) 37. This component intended to finance change management activities and give the project flexibility by making resources available to demand-driven initiatives. The objective was to broaden the space for reform and strengthen the communities of practices and support initiatives that may result from change management activities or become necessary due to changes in circumstances. Component 4: Project Management (US$274,701) 38. This component aimed at supporting the Project Coordination Unit to allow for overall coordination, management, and monitoring of the project. 1.6 Revised Components 39. Component 2 was revised at the time of AF as follows: (a) new Subcomponent 2.1: Provision of support to reform civil service, including the wage bill management and the HR inventory, as well as the computerized system for human resources management and (b) new Subcomponent 2.2: Provision of support to prepare a comprehensive civil service management reform strategy. 1.7 Other Significant Changes 40. Intermediate indicators were also adjusted at the 2013 additional financing and the 2016 restructuring. Two new intermediate results indicators were introduced, one to monitor new activities related to procurement (intermediate indicator 5 related to Component 1) and the other on civil service management reforms (intermediate indicator 7 related to Component 2), as shown in table 3. Table 3. Revised Intermediate Indicators Revised Final Intermediate Indicators Changes Indicator A new organic law for PFM prepared for adoption None 1 Indicator Annual budget prepared in timely manner (PEFA PI-11) None 2 9 Revised Final Intermediate Indicators Changes Indicator Limited use of budgetary adjustments during the FY (PEFA PI-16) None 3 Indicator Enhanced computerization of budget management and reporting (PEFA None 4 PI-24) New Publication of public tender procurement notice in five targeted Indicator (2013) ministries for any public procurement financed by the national budget or 5 Revised by donors according to law No. 11-027/AU (2016) Enhanced efficiency in controls of civil service wage bill accuracy Indicator (through cross checks across databases, timely updates and internal None 6 controls) (PEFA PI-18) Indicator A comprehensive strategy for civil service management reform is New 7 adopted. (2013) 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design, and Quality at Entry 41. The project was approved by the World Bank Board of Directors on January 31, 2011. The financial agreement between IDA and the Government of the Union of the Comoros was signed on February 23, 2011, and the project was declared effective on February 28, 2011. The project had an expected closing date of December 31, 2013. The Project Concept Note was prepared in 2006. At the time the World Bank team might have been overambitious regarding the expected approval date (2007) given the high turnover in the Comorian Government. Then the political turmoil that started in 2008 further delayed the approval of the project. 42. The design of project was consistent with the Government of the Union of the Comoros’ priorities in line with the adoption of a 2010–2019 PFM strategy and detailed 2011– 2012 action plan in support of the 2010–2014 PRSP, as well as progress toward the HIPC completion point. 43. At the time of design, the project and its development objectives were also relevant and consistent with the World Bank’s strategy. The project was designed to contribute to the IDA ISN for 2010–20129 which, to put the country on a sustainable path geared toward growth and poverty reduction, proposed a series of DPOs aimed at improving economic governance and linked with TA in the areas of public finance management, budgetary management, and CSR. The ISN further stressed that key entry points for reforms concerned economic governance, particularly including the country’s public finance and budgetary management system. 44. Furthermore, the project was aligned with the overall donors’ efforts to support PFM and civil services reform programs.10 In September 2009, the Government engaged in a three- year arrangement under the ECF (formerly called Poverty Reduction and Growth Facility), with the IMF. As noted in the ISN, progress in the areas of public finance and budgetary management 9 Report 52522-KM, April 29, 2010. 10 EU’s support to Commissariat of Planning, French Cooperation’s support to CREF, and the AfDB’s support to PFM. 10 system would also lead the way to successful implementation of the IMF program, the attainment of HIPC debt relief, and further budget support by development partners. At the time of design of the project, the ongoing and planned PFM and CSR donor support was as described in table 1. 45. More specifically in 2010, the project subcomponents were designed to directly support the World Bank’s DPO, HIPC completion point trigger, or IMF ECF benchmark as described in table 4. Table 4. Links Between the Project and Main Programs, Triggers, and Benchmarks Objective Benchmark or Trigger Linked to TA Budget management Enhanced The budgetary legal framework is revised Subcomponent 1.1: quality of the to reflect the Comoros’ unique institutional Clarifying the legal and budgetary setup. A draft organic law on public institutional framework process finance is prepared. Subcomponent 1.2: (World Bank DPO 2 indicative trigger) Strengthening the budgetary process Improved budget TORs prepared for a feasibility study for a Subcomponent 1.3: implementation comprehensive computerized PFM system Supporting gradual and monitoring (HIPC completion point floating trigger) computerization of public finances Improved budget Enhanced reporting on budget execution Subcomponent 1.1.2: implementation including social sector spending Adapting the administrative and monitoring (World Bank DPO 2 indicative trigger) setup to reflect the new legal framework Subcomponent 1.2: Strengthening the budgetary process Strengthened Implementation of PFM action plan Subcomponent 1.1.2: controls particularly with respect to internal and Adapting the administrative external controls setup to reflect the new legal (World Bank DPO 2 indicative trigger) framework Civil service management Improved wage Completion of civil service census Subcomponent 2.1: bill and HR (IMF ECF 2010 structural benchmark) Supporting a comprehensive management census of civil servants Improved HR Implementation of the organic frameworks Subcomponent 2.2.1: management (IMF ECF 2010–2011 structural Developing administrative benchmark) re-organization strategies Improved wage Full computerization of civil service Subcomponent 2.3: bill management payment roster and adequate staffing of the Computerization of wage bill unit monitoring wage payments management (IMF ECF 2010 structural benchmark) Improved wage Introduction of computerized payslips Subcomponent 2.3: bill management (IMF ECF 2010 structural benchmark) Computerization of wage bill (World Bank DPO 2 indicative trigger) management 11 Objective Benchmark or Trigger Linked to TA Rationalized Existence of a competitive process of Subcomponent 2.2.2: recruitment and recruitment into the civil service managed Strengthening recruitments deployment of by HAFOP and publication of examination civil servants results (World Bank DPO 1 prior action) (World Bank DPO 2 indicative trigger) 46. In addition to the strategic alignment, lessons learned related to the fragile country high-risk environment context were considered in the design of the project. In particular, (a) implementation of the project alongside DPO for mutual reinforcing, as described in table 4; (b) limited and modest set of objectives and deliberately lean program with only two core components; (c) short operation with limited funding; and (d) flexibility, with precise criteria for allocation to allow for any needed modifications. 47. The project implementation arrangements were designed to address the shortcomings of the administration and promote coordination with the AfDB’s Institutional Capacity Building Project (Projet pour le Renforcement des Capacités Institutionnelles, PRCI). The Government agency responsible for the overall implementation of the project was the MOF. The PIU model was adopted because of the inadequate capacity of the Comorian administration and the PIU was responsible for overall management. CREF was responsible for Component 1 and HAFOP was responsible for Component 2. In addition, the project worked with the same Steering Committee, which included representatives of the institutions supported by the project, such as the AfDB’s PRCI. 48. However, the lack of a logical framework and careful sequencing at inception affected the quality of design. Although the PAD provides a discussion of risk and mitigation measures and is attentive to both the scope of the operation and lessons learned, the project failed, at inception, to use a consistent and logical approach for both components. The PFM component first sought to clarify the legal and institutional framework which was a logical first step to build on. However, no clarification of the civil service legal framework was planned as a first step and the preliminary work in that direction was only performed at the end of implementation in 2016 with the completion of the comprehensive CSR strategy. This strategy showed that (a) some legal and regulatory texts governing the civil service were applied or interpreted differently because they are incomplete, contradictory, or obsolete and (b) the institutional and organizational framework of the civil service does not allow coordination and harmonization of management processes. Also, while the risk of CSR in a complex, social, and political environment was identified, sequencing issues were not taken into consideration in the design of the project. Because of this flaw in the project design approach, some realizations were potentially at risk down the line. 49. Furthermore, Subcomponent 1.3, related to the IFMIS implementation, severely underestimated the necessary costs and time. Indeed, in addition to the production of a computerization master plan, part of this subcomponent was planning to support the rollout of a system for budget operations with a goal of 15 computerized positions nationwide. The amount devoted to this subcomponent was only US$0.6 million and the timing of the entire project was 12 initially three years when, on average, an IFMIS implementation project requires seven years.11 Unsurprisingly, a significant funding gap was identified in 2013, which was one of the factors leading to the AF. 50. Finally, no economic analysis was performed at the time of design and no justification was included in the original PAD. However, an attempt to perform an economic analysis was included in the 2013 AF document and it was noted that the impact of the project was mostly institutional and could not be easily isolated or quantified. 2.2 Implementation 51. The project, originally planned for three years, was extended twice for a total of nearly six years. The project’s original closing date of December 31, 2013, was extended a first time to June 30, 2016, with the December 21, 2013, AF. In March 2016, the project’s closing date was further pushed back to December 31, 2016. For the list of outputs for each component, refer to annex 2. 52. During the first two years of implementation, significant progress was made mostly because of the incentives linked to the achievement of the HIPC completion point. The project effectively supported the achievement of the DPO and HIPC completion point in December 2012 and the project’s progress toward the achievement of PDO and the overall implementation progress were both rated Satisfactory. The political momentum provided by the post-conflict environment and DPO incentive combined with the TA explained the Government’s commitment and the PFM reform’s effective implementation. In this period, the project supported (a) the adoption of a new FM bill and related regulations, (b) the appointment of the accountant general of the Union, (c) the development of the IFMIS master plan and the setting up of the Steering Committee, and (d) the development of organic frameworks for civil service. 53. Early 2013, a funding gap was identified and the World Bank agreed to prepare a US$3.5 million AF to fund the budget and accounting software, while the AfDB would finance the IT equipment and upgrade of the buildings. Although they were not successful in conducting joint supervision missions, the two task teams maintained close contact and communicated on an ongoing basis. In addition, although a first version of GISE was developed early on in the project cycle, an IMF audit in June 2013 pointed out important flaws in the first version prompting the need to develop an upgraded GISE. The project’s progress toward the achievement of PDO and the overall implementation progress were still rated Satisfactory during the March 2013 supervision. 54. During the preparation of the AF, the World Bank team performed extensive due diligence to inform its work and maintained strong alignment with the Country Assistance Strategy12 which was developed at the same time and drew from the same underpinning analysis. 11 Hashim, Ali. 2014. A Handbook on Financial Management Information Systems for Government: A Practitioners Guide for Setting Reform Priorities, Systems Design, and Implementation (Based on a compilation of experiences in World Bank-financed projects) World Bank. 12 IDA and IFC Country Partnership Strategy for the Union of the Comoros for FY14–17, Report #82054-KM. 13 Furthermore, the team took into consideration lessons learned from fragile countries and from the literature on PFM in small-islands states. 55. However, the project started to lose momentum in 2014 subsequent to the signing of the AF and achievement of the HIPC completion point, thus affecting implementation. The project suffered delays during the transition period between the original financing and AF. At the time of the AF, the PIU became responsible for the management of two World Bank projects: the ABGE and the US$13.5 million Regional Communication Infrastructure Project (RCIP), which induced changes in PIU management. Those changes translated into a nine-month delay in implementation, because the legal covenants and effectiveness conditions for the AF Legal Agreement were slow to be met. Also, the initial PIU project coordinator was demoted on May 10, 2014, and the position was officially filled again in December 2014. Furthermore, the recruitment of a new procurement specialist and assistant procurement specialist was agreed during the February 2014 World Bank supervision mission. These recruitments were finally effective in March 2015, which slowed the procurement activities during this period. 56. In addition, political economy constraints and an extended election period13 adversely influenced implementation. In 2013, a few months before the end of the original financing, the HR component suffered from delays because of the vested interests related to the fact that the civil service is the country’s main employer14 and perceived as a social safety net more than as a service provider.15 To address the challenges and create consensus around the CSR, the World Bank team organized the first Cabinet Retreat in September 2013, which recommended the realization of a civil service strategy and the use of anthropological analysis. Although the strategy and the anthropological study16 were delivered, the country’s preexisting political economy constraints led to the de facto de-operationalization of HAFOP in 2014 by removing the HAFOP Secretary General (without replacement), hampering the CSR’s dynamic and strategy implementation. Another example is that the international consultant heading the IT cell left the country at the beginning of 2016 for personal reasons. The project was then left to operate with no IT cell head for the implementation and deployment of SIMBA and GISE to improve. Finally, another example of the complex political economy dynamics at play is that subsequent to the elections in 2016 in Ajouan, the entire IT team which had experience in the project was fired in August 2016. The undertaken political economy analysis (PEA) was (a) in the post-preparation phase (b) funded through the Governance Partnership Facility Trust Fund grant, and (c) focused on HRM and was not directly related to project implementation details (‘PEA of civil service recruitment’). 57. To address these challenges, the World Bank took steps to better understand the political economy dynamics, however, a little too late in the project cycle, and commissioned a Fragility Assessment of the Union of the Comoros which was completed in July 2015.17 The 13 Legislative elections in 2014, presidential campaign in 2015, and presidential elections in 2016. 14 According to the 2016 CSR strategy, the wage bill represented 56 percent of gross domestic product in 2015. As a comparison, the West African Economic Monetary Union convergence criteria is 35 percent. 15 Lack of employment alternatives is described in Rose, Jonathan, and Balachandran Gowthaman. 2015. “Civil Service in Comoros, A Case of Political Clientelism in a Decentralized State” Macroeconomics and Fiscal Management Global Practice Group World Bank Policy Research Working Paper 7428. 16 Soutien à l'amélioration des résultats du projet de gouvernance économique aux Comores (ABGE) mai–juin 2016, Mission d’appui au processus d’appropriation nationale des réformes administratives. Rapport de mission du consultant anthropologue, Said Mahamoudou. 09 Juillet 2016. 17 Comoros: Assessment of Resilience and Fragility, Possible Implications for the 2014–2017 World Bank Portfolio, Fragility, Conflict, and Violence Group, World Bank, July 2015. 14 assessment highlighted the need to (a) manage expectations during the elections period, (b) understand social influences on the project, (c) interact with authorities beyond Moroni, and (d) ensure a greater World Bank presence. It is to be noted that the original 2010 PAD, in its annex 5, called for an analytical work program on the political economy of reforms in the Comoros to inform the team’s understanding of relevant stakeholders’ positions and interests. A draft of this work was available in 2011 and was internally shared with the task team leader (TTL) but could not be officially used during the AF preparation to stimulate the policy dialogue. The report was only officially cleared and published in September 2015.18 58. Great effort was made during the last eight months of the project, to address the challenges and deliver key activities owing to Rapid Results Initiatives (RRIs). The selection of the software followed a linear process, from the development of terms of reference to build the required consensus among stakeholders on their specific needs, to a study tour to see how the pre- identified software meeting their needs and constraints (including limited budget, access to source code, and francophone software) were applied. This led to the selection of SIMBA after a lengthy process and a deployment delayed to the beginning of 2016. Steps were taken to accelerate the implementation process with support from the World Bank Collaborative Leadership for Development (C4LD) team. During the September 2013 Cabinet Retreat, implementation issues became clear and the decision to introduce RRIs was made to build consensus around wage-bill reform priorities between (a) the Union and the three Island Governments and (b) between the MOF and Ministry of Civil Service. At the level of the four entities (three islands and the Union), 24 RRIs were launched to support achievement of the indicators—mostly in 2015 and 2016. The RRIs helped improve the implementation of the IFMIS and the HRMIS and embed the change and leadership dimensions within the CSR strategy.19 2.3 Monitoring and Evaluation Design, Implementation, and Utilization 59. Despite the high risk acknowledged at design, targets for the set of indicators chosen appeared too ambitious and sometimes not adapted both for the initial financing and the AF. Six out of the ten indicators (two PDO-level and four intermediate indicators) were based on improvements of PEFA scores from a D rating (or D+), which is the lowest possible when information is available, to a B rating (or B+) for five indicators and C for one indicator. 60. The formulation of indicators was either too broad or not precise enough. The two PDO-level indicators monitoring the increase in accountability and transparency of PFM were redundant and not adapted. Indeed, they were both based on the improvement of the same PEFA indicator 1020 from a D score to a B+ score. First, this indicator has only one dimension, which means that it is not possible to have a ‘+’ in the scoring. Second, this indicator is not granular enough to capture progress in a capacity-limited environment. The intermediate indicator related to the IFMIS implementation was also not adapted: the PEFA PI-24 relates to the quality and timeliness of in-year budget. Provided that it was the initial implementation of the system, this 18 Rose, Jonathan, and Balachandran Gowthaman. 2015. “Civil Service Recruitment in Comoros, A Case of Political Clientelism in a Decentralized State, and, World Bank Macroeconomics and Fiscal Management Global Practice Group” Policy Research Working Paper 7428. This PEA was funded through the Governance Partnership Facility Trust Fund grant focus on civil service recruitment broadly. 19 See details of RRI in annex 2. 20 PEFA 2011 framework. 15 indicator was overly ambitious. Finally, the last PDO-level indicator related to the HR component could have been more precise as it was focused on the share of civil servants paid through the automated payroll system and failed to explicitly include contractual staff and political appointees. 61. Not all objectives, as defined in the PDO, had indicators designed and set to monitor progress toward their attainment. In particular, indicators were missing to measure PDO aspects of (a) PFM efficiency and (c) HRM improvement. The team could have been more thorough in updating the Results Framework to facilitate monitoring of results. Indeed, no direct outcome indicators were available for measuring efficiency in PFM. Similarly, another objective was to improve the management of civil service HR and although Subcomponent 2.2 of the initial financing (dropped and replaced by the preparation of a strategy for civil service in the AF) was supporting this objective, no direct outcome indicators were designed to monitor progress toward this objective. 62. The M&E team was able to measure all project indicators and the 2016 PEFA self- assessment allowed for a measurement of indicators’ success, as defined in the project’s latest Results Framework. In addition, the Government’s end-of-project evaluation was completed in December 2016. 2.4 Safeguard and Fiduciary Compliance 63. The project did not trigger safeguard policies and there were no exceptions to World Bank policies. It was categorized as C for both. 64. The FM and procurement activities were conducted in line with the provisions in the Legal Agreement, in spite of some delays at the time of the AF, and were considered Satisfactory. The project had no overdue audit report and financial information and reports of acceptable quality were produced on time. The audit reports were delivered on time with unmodified opinions, with the last audit report for December 31, 2016, having an unmodified opinion. The project team made significant progress to address the inadequacies related to the procurement management system in the middle of the project, with hands-on support from the World Bank team. As of the end of the project, the contract management system was functioning, with contract deadlines being broadly met. Table 5. Evolution of FM and Procurement Ratings During Implementation ISR 1 ISR 2 ISR 3 ISR 4 ISR 5 ISR 6 ISR 7 ISR 8 ISR 9 ISR 10 Decembe July March Decemb May Novemb May October May Novemb r 2011 2012 2013 er 2013 2014 er 2014 2015 2015 2016 er 2016 FM rating MS S S S S MS MU MS MS MS Procureme S S S S MS MU MU MS MS MS nt rating Note: MS = Moderately Satisfactory, S= Satisfactory, MU = Moderately Unsatisfactory. 2.5 Post-completion Operation/Next Phase 65. Sustainability risks for the results achieved are high. While there have been substantial gains, especially considering the poor state of PFM and HR systems at the beginning of the project 16 period, the results achieved remain highly dependent on continued financial and political support, which is not the case as of March 2017. 66. The financing of the operationalization and maintenance of both the IFMIS (SIMBA) and HRMIS (GISE) is still ensured by donors. The Government is yet to commit financing to support the operations of the data centers, the network connection, maintenance of the system databases, salaries of the IT specialists, and other relevant trainings to build necessary capacities, particularly at the island levels, beyond the project closing date. Currently, all of this funding is provided by donors 21 and the Government will need to ensure that appropriate resources are allocated to support these activities beyond the closing date of the project (2019). 67. In addition, the Government may lack commitment to pursue PFM and CSRs. The institutions established by legislation to guide and sustain reforms will need adequate financing to be properly staffed and operational. The lack of a financing and manager for HAFOP is a concern. Also, should political attention be preoccupied with other public finance reform areas, particularly domestic resource mobilization, the gains seen thus far may be diminished. It is important to note that the sustainably of results is particularly tenuous given that the indicators are not cumulative by nature and can be very quickly reversed (particularly PEFA indicators) if reform momentum, resources, and political will are not maintained. 68. Fiduciary management transition arrangements were taken. The final audit opinion was dated April 28, 2017. The World Bank will also continue to disburse/approve the use of grant proceeds for withdrawal applications received within a specified period of up to four months after the closing date for payments made or payments due for eligible expenditures incurred before the closing date. As the project closed on December 31, 2016, a fiduciary team remained in place, including a procurement manager, a financial and administrative officer, and an accounting assistant who was under contract with the RCIP 4. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design, and Implementation Relevance of Objectives Rating: High 69. At the time of the Implementation Completion and Results Report (ICR), the development objectives remain strongly relevant, as confirmed by the Government’s growth and poverty reduction strategy (Accelerated Growth and Sustainable Development Strategy [Stratégie de Croissance Accélérée et de Développement Durable, SCA2D] 2015–2019); the World Bank’s Country Partnership Strategy (2014–2017); the forthcoming 2016 Comoros Performance Learning Review; and the comprehensive 2016 Civil Service Management Strategy. Going forward, however, the relevance of project objectives may be subject to change, given that an opposition political party recently came into power following the 2016 presidential elections. Although initial dialogue between the Government and the World Bank has indicated a continuing interest in PFM reform (particularly revenue mobilization) as well as a desire to further reduce 21 The AfDB PRCI 2, closing in 2019, will finance an international consultant to manage the IT department and a national consultant to assist him/her. As of mid-March 2017, the recruitment process was still in progress. 17 public sector wages, the newly elected government is yet to fully articulate its official development priorities and strategies. Relevance of Design Rating: Modest 70. The PDO and related indicators were not all adapted and were overly ambitious, especially considering the poor state of PFM and HR systems at the beginning of the project period. After five years of implementation, results have mainly emerged at the output level (for example, rollout of data systems and processes) but have not yet resulted in the necessary behavioral change to materially impact the higher-level objectives noted in the PDO. Relevance of Implementation Rating: Substantial 71. The implementation of the project was consistent with the country’s development priorities and the relevance of implementation was maintained by systematic and regular supervision missions and through three restructurings which aimed at adjusting resources, results, and closing dates to ensure the attainment of results. 3.2 Achievement of Project Development Objectives (Efficacy) 72. The project laid the basics for budget preparation and reporting, significantly improving accountability in public finance management (Substantial). Structuring activities were implemented to overhaul the PFM framework. Key achievements have been the review of the existing regulatory framework, the adoption of a new organic PFM law (Indicator 1.1), the reorganization of the financial administration to reflect the new organic law with the creation of the DGCPT and Directorate of Financial Control, and the preparation and implementation of a communication strategy on the CREF reforms (completed in March 2016). The project, and notably the IFMIS, has helped improve timeliness of budget preparation (Indicator 1.2) and decrease budgetary adjustments during the fiscal year (Indicator 1.3). The IFMIS just went live in January 2017 and users are still at the beginning of their learning curve, still performing a parallel accounting in some cases using the historic system. However, the system is implemented and used, and the AfDB is planning to provide additional support to the Government on this matter. 73. However, the gains in efficiency were not fully reaped (Modest). The computerization of public finances is a recent reality and does not translate yet into efficiency gains. In particular, SIMBA has been used since April 2016 for pilot implementation by the Union and Island Governments to automate budget execution, accounting, and reporting, and went live in January 2017 (the previous target was January 2016). An interface between SIMBA and GISE is developed but is not yet fully deployed. In spite of this progress, the full utilization of SIMBA is still hampered by the lack of legal basis to enforce the use of the system for capturing all critical budget transactions, notably with banks on electronic payments and data reconciliation. Furthermore, a fire that took place on March 4, 2017, in the Treasury building in Moroni interrupted the Union’s access to the server for two weeks. The related indicator (1.4), which measures the level of enhanced computerization of budget management and reporting, has consequently not been achieved by the end-of-project target. Although the application was not in place as of March 2017, 18 the AfDB, through its financing of the IT department until 2019 will continue to support the application and some gains should be reaped down the line. Finally, the other expected efficiency gains due to improved procurement are unlikely as public procurement process are not functioning effectively, as described in the following paragraphs. 74. Transparency in public finance improved only marginally (Negligible). The expected outputs were twofold: (a) the publication and accessibility of the public to at least three of the key budget documents elements22 and (b) the publication of 90 percent of public tender procurement notices for any procurement financed by the national budget or by donors according to law 11- 027/AU. None of the planned outputs were produced. According to the 2016 PEFA evaluation, the administration does not make accessible any of the six mentioned elements to satisfy the PEFA criteria and SIMBA has not yet been harnessed to support and institutionalize cost-effective publication of key PFM documents. Although the documents published did not meet the PEFA indicator target, the MOF’s website does make available some budget execution reports and finance laws.23 Similarly, public tenders are not systematically published. The public procurement regulatory body does not even have access to the comprehensive list of public tenders by the line ministry. 75. However, the project helped make some progress toward transparency with its support to the ICBT in the Comoros and civil society training to encourage a bottom-up demand for transparency, which was financed under the demand-driven facility component. The ICBT is a program implemented by a consortium of civil society associations 24 to assist the Comorian Government in achieving the specific results related to fiscal transparency. The initiative, which was launched with the support of the project on August 23, 2016, aims to promote public access to key budget information.25 Nevertheless, support to civil society was effective only in the last months of implementation (from August 2016) and is dependent on other external factors, such as civil society funding and an enabling environment, to actually translate into higher pressure on the client for access to information. 76. With regard to civil service, the identified basics for improved HRM have not yet been implemented (Negligible). As of the end of the project, the main output is the HRM strategy, which when implemented, will prove to be challenging as long as there is no leadership to pilot the reforms because HAFOP is not operational and alternative oversight strategies are yet to be 22 (a) Annual budget documentation: a complete set of documents can be obtained by the public through appropriate means when it is submitted to the legislature; (b) in-year budget execution reports: the reports are routinely made available to the public through appropriate means within one month of their completion; (c) year-end financial statements: the statements are made available to the public through appropriate means within six months of completed audit; (f) external audit reports: all reports on the Central Government consolidated operations are made available to the public through appropriate means within six months of completed audit; (e) contract awards: award of all contracts with value above approximately US$100,000 equivalent are published at least quarterly through appropriate means; (f) resources available to primary service units: information is publicized through appropriate means at least annually, or available upon request, for primary service units with national coverage in at least two sectors (such as elementary schools or primary health clinics). 23 See annex 9, subsection a. 24 Fédération Comorienne des Consommateurs (FCC), Organisation pour le Développement Communautaire (ODC), Association MAEESHA (Mouvement Associatif pour l’Education et l’Egalité de Chances et ASCOBEF [Association Comorienne pour le Bien Etre de la Famille]). These organizations collaborate with other nongovernmental organizations, such as Ngoshawo, FAWECOM, l’AFP, and MOSC. 25 “L’initiative citoyenne sur la transparence budgétaire est lancée" La Gazette des Comores, 25 aout 2016, http://lagazettedescomores.com/economie/l%E2%80%99initiative-citoyenne-sur-la-transparence-budg%C3%A9taire-est- lanc%C3%A9e.html. 19 implemented. Although the HRM strategy was adopted, its adoption occurred only seven months before the project’s closing date, which did not allow enough time for action plans to be implemented. Besides, the new administration may not have the same interest and ownership in the strategy approved by the previous government, which may jeopardize its implementation. 77. However, wage bill management significantly improved in the given context but much remains to be done (Substantial). Under Component 2, GISE, has been operational since January 2016 as an integrated personnel and payroll management system for the Union and Island Governments. As of March 2017, more than 95 percent of civil servants are paid through the automated payroll system. However, the following risks affects the full achievement of the expected outcome: (a) lack of legal basis to enforce the use of GISE for all salary payments, (b) lack of verification for salary payments, and (c) inadequate IT specialists to manage the system. As a result, the indicator related to enhanced efficiency of controls in the civil service wage bill (Indicator 2.1) was not achieved. 3.3 Efficiency Rating: Modest 78. While rates of return are difficult to determine for this type of TA project, project efficiency can be considered as Modest given the results achieved. At appraisal in 2010, an economic and financial analysis was not prepared. At the time of the AF in 2013, it was noted that the activities supported by the project could likely yield between 0.75 and 3 percent savings26 (US$1.2 million to US$4.8 million) in public expenditures, which would be sufficient to achieve an undiscounted ‘payback’ by the end of the project. In particular, efficiency gains and cost savings were anticipated to be derived from the implementation of unified PFM procedures (treasury management, budget classification, and HRM), as well as from better budget discipline, improved cash management, and closer monitoring of the wage bill. 79. There is no clear evidence that the gain in efficiencies in the activities successfully implemented have materialized by the end of the project, due in part to the fact that cleaning and verification of the HRMIS database was only done in December 2016, not leaving enough time to generate savings. Also, the link between the HRMIS and the IFMIS was not implemented yet, although the interface is developed. The year 2017 is the first fiscal year where SIMBA and GISE will be fully implemented; hence, the productivity gains will start being measurable after the end of this full year. The wage bill clean-up enabled the Government identify 572 ghost workers out of the 12,883 civil servants currently on payroll (that is, 4.4 percent). 3.4 Justification of Overall Outcome Rating Rating: Moderately Unsatisfactory 80. The overall outcome rating of Moderately Unsatisfactory is based on the individual ratings for relevance, efficacy, and efficiency as shown in table 6 and strictly follows the ICR Guidelines. However, it should be noted that the project was instrumental in laying out the basis for budget preparation, budget reporting, and wage bill reporting in a cost-effective way and in a 26 Based on 2012 public expenditures of US$160 million. 20 context of limited capacity and institutional complexity and fragility—even though it is too early to measure its sustainability and impact. Table 6. Calculation of the Overall Project Outcome Rating Criteria Rating Objectives High Relevance of Design Modest Objective 1: Increase efficiency of PFM Modest Objective 2: Increase accountability in PFM Substantial Achievement Objective 3: Increase transparency of PFM Negligible of PDO Objective 4: Improve human resource Negligible (Efficacy) management Objective 5: Improve wage bill management Substantial Efficiency Modest Overall Outcome Moderately Unsatisfactory Note: Annex 2 on the assessment of achievements by objectives captures the PDO while showing achieved outputs by components. 3.5 Overarching Themes, Other Outcomes, and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 81. The project did not explicitly target poverty reduction, and given the lack of data on poverty at the time, it is difficult to assess the poverty impact. However, it is expected that improvement of budget preparation will translate into a more strategic use of public resources may benefit the poor ultimately by ensuring that pro-poor spending is prioritized. The implementation of the IFMIS will support internal control and translate into savings down the line. Also, better tracking of the civil service may ensure down-the-line savings on wage bill expenditures and the rationalized personnel deployment and ultimately better service delivery. However, it is too early to assess those impacts. 82. Gender was not taken into account in the design of the project nor in implementation. (b) Institutional Change/Strengthening 83. The core administration leading and implementing CSR and PFM reforms, especially budget preparation and reporting, were significantly strengthened. As noted in the original PAD and AF document, the core beneficiaries of the project are the institutions responsible for managing public resources and administration, including the Union’s and island-level budget offices and treasuries, CREF, HAFOP, the IT unit, and the credit managers within each public administrative structure. Through the five years of project implementation, the project has worked to increase the capacity of these implementing agencies as well as the PIU with critical resources (staffing, equipment, and training) to support implementation and monitoring of the Government’s reform agenda. Likewise, through the demand-driven facility and related RRIs, the project support was essential in strengthening the leadership in the reform process. 21 84. The project also strengthened M&E capacity to increase public awareness of the reforms and to enhance accountability. After the end of the project, the institutional development impact achieved thus far will only be sustainable if political and financial support is maintained for these institutions following the closure of the project. (c) Other Unintended Outcomes and Impacts (positive or negative) 85. The 2016 PEFA assessment conducted as part of the M&E effort was a self- assessment and was well conducted with a strong sense of ownership by the Government Steering Committee. The knowledge of PEFA methodology has greatly increased in the Government. This will be useful as other financial partners inform their country strategies using the PEFA scoring.27 86. Support to the ICBT has created demand for transparency from the civil society. The initiative was officially launched on August 2016 and delivered (a) capacity-building activities on finance law literacy and training of 20 trainers; (b) a conference with the Comorian parliamentarians in the parliament, debate with the media; and (c) a workshop in October 2016 on the promotion of budgetary transparency and social accountability, including the creation of a related Facebook page.28 The civil society took ownership of the initiative and has started engaging with the Government. In particular, the ICBT had a working session on November 14, 2016, at the Parliament under the patronage of the finance commission to discuss the 2017 finance law and how to achieve the Government’s objective of going from a KMF 41 billion budget in 2016 to a KMF 80 billion budget in 2017. The ICBT 2017–2021 roadmap was validated on December 13, 2016. However, the initiative does not yet have any funding to operate and might not sustain and consolidate the demand for budget transparency nor the dissemination of information to citizens. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops Not applicable. 4. Assessment of Risk to Development Outcome Rating: Significant 87. The overall risk to the development outcome is rated Significant. While the project has been successful in establishing essential PFM and HRM systems as well as building the capacity of key institutions, there remain significant shortcomings in the full utilization of these systems, thus undermining their performance and efficiency enhancing potential. Moreover, while the objectives of the project remain relevant to both the Government and World Bank development priorities in the Comoros, there is a high risk that without the continued financial and political commitment, the results achieved will not be sustainable beyond the end of the project. Also, the uncertainties with regard to financing and maintenance of the IFMIS are a potential risk to the current achievement. However, the AfDB is planning to follow up the World Bank engagement, mitigating this risk. 27 AfDB Group Union of the Comoros Country Strategy Paper 2016–2020 - EARC Department, March 2016. 28 https://www.facebook.com/comorestransparence2016. 22 88. With continuing computerization of the entire PFM system funded by different donors, there is a risk of a piecemeal approach to computerization of the entire PFM system that could lead to donor-driven fragmentation in the absence of strong ownership from the Government. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory 89. Although the project was well aligned with the Government and World Bank strategy and applied lessons from fragile countries by designing a short and lean project, it failed to properly consider the political economy in the Comorian administration and society, especially with regard to human reform management reform, increased transparency in procurement, and increased internal control. 90. Also, the design of some activities were overlooked. First, the lack of attention to the sequencing of activities affected the implementation of the HR component: the timeline (three years) was too short to complete all activities. Second, the total budget (first US$1.8 million then US$3.5 million) was too limited, especially underestimated for the IT systems put in place. 91. Finally, the Results Framework could have been better elaborated to effectively capture progress in a capacity-limited environment. (b) Quality of Supervision Rating: Moderately Satisfactory 92. The World Bank’s supervision has been regular, comprehensive,29 and leading to adequate adaptation of projects when the project implementation started to be considered Moderately Unsatisfactory. First, it should be noted that the project TTL at the beginning of the project was also the DPO TTL, which greatly helped in aligning priorities in project design and achieving results in the first months of implementation. Second, two midterm reviews were conducted in 2012 and 2015. Both led to the necessary project adaptation through an AF, a restructuring, and the inclusion of a new team member from CL4D to provide TA on the Rapid Results Approach (RRA) to accelerate the rollout of the IT systems successfully (2015–2016). Project implementation also improved in the last three years of the project owing to the Country Management Unit’s decision to have a TTL based in Madagascar since March 2013 as well as hands-on World Bank TA to the PIU covered by World Bank budget on (a) procurement,30 (b) FM,31 and (c) RRA.32 29 Between 2011 and 2016, the World Bank conducted 22 formal supervisory missions covering six-month periods and including reviews of the technical aspects of the project, as well as procurement and FM. 30 During the life of the project, the PIU benefited from two specific trainings on World Bank Procurement Guidelines. As advised by the World Bank procurement specialist, the PIU also recruited a short-term experienced procurement consultant to speed up and streamline the procurement processes. 31 During project implementation, the PIU also benefited from two specific trainings on World Bank FM Guidelines. A World Bank FM specialist and a Madagascar-based short-term consultant also provided regular hands-on support. 32 Two regional RRI coaches and a CL4D senior World Bank staff provided regular hands-on support to the PIU through regular quarterly missions and videoconferences. 23 (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 93. While the World Bank performance in ensuring quality at entry has been Moderately Unsatisfactory, particularly proactive World Bank supervision and TA to the project led to a restructuring and a series of adequate solutions enabling the successful implementation of two key elements: the IFMIS and GISE. Consequently, the overall World Bank performance is rated Moderately Satisfactory. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Unsatisfactory 94. This Moderately Unsatisfactory rating is justified by the fact that the borrower’s leadership was not consistent and strong enough throughout implementation to properly support the project. Although the borrower showed strong leadership at entry using political momentum to push for crucial PFM reforms and achievement of the HIPC completion point, leadership was limited during implementation for specific aspects. Indeed, the borrower dismissed the head of HAFOP in 2014 and had not replaced him. This translated into a lack of implementation of most activities in the HRM component beyond the operationalization of GISE. Also, the borrower had difficulties to properly address major IT implementation issues related to Anjouan IT teams’ demotion mid- 2016 in the middle of SIMBA’s rollout and GISE’s upgrade. Finally, the lack of leadership and commitment of authorities on procurement reforms and budget transparency translated into the lack of implementation of the related activities. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory 95. The PIU performed adequately during the first two years of implementation but faced serious difficulties after the AF in 2014, with delays in properly staffing the unit which resulted in implementation challenges. These challenges were addressed by the end of the project and the performance returned to a Moderately Satisfactory level. Project management rating was at least Moderately Satisfactory for 60 percent of the Implementation Status and Results Reports. The final project management rating was Moderately Satisfactory. Performance fell back during the 2014 transition period, as well as briefly during the last year of implementation due to procurement issues not affecting the ABGE but related to the other World Bank project (RCIP 2) managed by the same PIU, which led to investigations by the World Bank’s Integrity Vice Presidential Unit. Fiduciary ratings remained consistently Satisfactory or Moderately Satisfactory, except for the 2014 transition period. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Unsatisfactory 96. While the Government performance has been proactive in pushing PFM reforms in the first two years of the project, it has been particularly unsatisfactory during the second half of the project, significantly affecting the implementation of the HR and transparency-related activities (budget and procurement), and affecting the IFMIS rollout. Consequently, the overall borrower 24 performance is rated Moderately Unsatisfactory, despite the implementing agency performance rated Moderately Satisfactory and World Bank efforts in supervision to support the project’s adaptation which could not address political commitment issues. 4. Lessons Learned 97. In a fragile, post-conflict, and complex small-islands state context, attention to political economy drivers is key to ensure that the project is country-fit and adapts to the changing environment. Factors of fragility are particularly exacerbated in small-islands states, such as social cohesion between communities, elite capture, connectivity issues, and technical capacity. These factors lead to complex political economy equilibrium. (a) Identify political economy drivers at different stages (transition, reconciliation, and alternance) and build consensus to inform design, adapt project, and build ownership. Some post-conflict contexts—such as in the case of the Comoros—have strong popular support and momentum to pass structural reforms that can be short- lived. The project was aligned with the early dynamic but required adaptation when political commitment for reform winded down. To ensure ownership and adaptability, an analysis of the non-technical bottlenecks and potential issues related to sensitive governance reforms would have been helpful. Those reforms might disturb the status quo by either mitigating discretionary powers and rent opportunities or creating clear winners and losers, with different capacity of influence. In the specific case of the project, in addition to the CSR and PFM reform challenges, the federal system combined with the post-conflict context was a reason to pay attention to political drivers of reform. The need for RRIs33 would probably have emerged earlier to foster consensus on solutions and reforms and mitigate the non-technical bottlenecks to reforms. Paying attention to subnational levels in project supervision would also probably have emerged—while transparency-related activities (procurement, budget) or CSR (performance-based reform, staffing plan review) would have been designed differently—for example, by roping in demand-side stakeholders more heavily or earlier—or in a second phase of the project. Finally, a stakeholder’s mapping related to project activities would also have helped identify the potential champion and leverage to lift bottlenecks (for example, the HAFOP leadership). It should be noted that classic PEAs are useful if they focus on specific problems34 and early at design stage but otherwise are not very helpful during implementation. 35 During implementation, ‘quick and dirty’ PEAs specifically related to an issue are more likely 33 The RRA is a structured process that mobilizes teams to achieve tangible results within 100 days or less. This approach is used when change efforts need to get unstuck or accelerated. The RRA is particularly useful when bringing together disparate stakeholders that need to perform as high-functioning teams. The RRA focuses on tackling adaptive challenges through disciplined experimentation and learning, allowing teams to develop the skills and confidence needed to achieve ‘stretch’ results. Specifically, the RRA is structured into a series of projects known as RRIs that help teams make progress toward their goals. These projects can be launched one at a time or in waves of projects for larger-scale change efforts. In the case of the Comoros project, 24 RRIs were launched in 2015–2016, with an estimated cost of US$35,000 financed by IDA and US$40,000 on World Bank budget for training and supervision. 34 Fritz, V., Levy, B. and Ort, R. 2014. “Problem-Driven Political Economy Analysis, the World Bank’s Experience.” 35 Beuran, M., Raballand, G. and Kapoor, K. 2011. “Political Economy Studies: Are They Actionable?” Policy Research Working Paper 5656. 25 to be useful. However, World Bank experience on the latter is still relatively scattered.36 (b) In a post-conflict context related to secessionist movement—and especially in a federal system—engage with subnational governments. Including subnational levels, to some extent, in the Steering Committee would be a way to ensure this. RRI was another way to maintain a concrete policy dialogue, foster ownership, and focus on results. Moreover, although the PIU might be geographically situated at the federal level, World Bank supervision mission should systematically place the same emphasis on each federal entity involved and undertake on-site supervision to encourage constant dialogue at the subnational level and harness the World Bank’s reputation of ‘honest broker’ and convening power. (c) Take into account broader government financial incentives (partners’ alignment, non-traditional partner disruption). During the first years of implementation, the mutually reinforcing effect between the TA and the DPO was a success, partly fostered owing to TTLship alignment. The subsequent budgetary support between 2011 and 2016 37 were also consistent with the TA but were not as well aligned during 2010–2012. The AfDB-World Bank alignment of support with different but overlapping implementation periods was also positive. The World Bank and AfDB projects’ Steering Committee shared the same members and a joint World Bank and AfDB ad hoc committee working on the implementation of the IFMIS, closely coordinated the process as the AfBD was in charge of financing the hardware and World Bank the software. This donor coordination helped maximize complementarity, coherence, and continuity. Finally, non-traditional donors’ support tends to be unpredictable and isolated from the aid community. Hence it becomes critical to design a World Bank portfolio that can mitigate the potential diversion of attention to reform implementation generated by large ‘free’ budget support. 38 Technical relevance is not the only factor, solid policy dialogue from both the Country Management Unit and the TTLs with the client is crucial to mitigate risks. 98. For low-income countries and fragile states where capacity is limited, define modest and realistic PDOs, do not use PEFA as indicators, and provide hands-on supervision and TA. In those environments, PDOs of public sector reform should focus on laying out the basis. Any steps toward progress is valuable and ambitious PDOs are not helpful, as they do not focus on the fundamental building blocks, as confirmed by the World Bank study on non-technical drivers of PFM reforms.39 In this case, the project was instrumental in laying out the basis for budget preparation, budget reporting, and wage bill reporting in a cost-effective way and in a context of limited capacity and institutional complexity and fragility. The Moderately 36 See experience of Governance Filters in ECA (Asli.Gurkan) or Nigeria (Kathy Bain) or recent use of Netmap for PEA in project (Rachel Ort for Water in Africa and Abel Bove for PFM project in Cameroon and for education project in Haiti). 37 Operations P117229 (2010–2011), P122941 (2012–20130) P131688 and P150924 (2014–2016). 38 Specifically, during implementation of the ABGE, in November 2015, Saudi Arabia granted €40 million—about 50 percent of Comoros’ annual budget—free of any conditionality or reform expectations, dwarfing the US$19.3 million from the World Bank over 2010–2016 (combining DPO and ABGE). Budget support from Saudi Arabia was not a first and can be expected in Comoros and other contexts. 39 Fritz, Verhoeven and Sweet, Avenia. Forthcoming. “Non-Technical drivers of PFM Reforms: Insights from Statistical Comparisons and Selected Cases and Implications for Calibrating Dialogue and Operational Engagement”, Draft February 2017. 26 Unsatisfactory rating is more related to the lack of progress on some aspects and the fact that most achievements were recent and impact not visible yet. Also, the PEFA framework is a great entry point for policy dialogue with the client. However, the framework is the same for all countries and must therefore be considered with caution for low-ranking countries. The project indicators can be designed to set the country on the right path but not necessarily be PEFA indicators themselves. Moreover, the mere realization of a PEFA assessment, regardless of the scoring, could also be a good indicator. For example, TTLs in Libya found it extremely helpful to use PEFA as an entry point for policy dialogue during Libya’s post-conflict and recovery period to promote PFM reform but not as a tool to measure project implementation progress.40 Finally, the World Bank managed to turn around project implementation through several restructurings and hands-on TA on project management, fiduciary management, and change management, which paid off ultimately (for example, IFMIS and GISE rollout). 99. The full implementation of the IFMIS/HRMIS and the related behavior change takes time. The review of World Bank experience estimated seven-and-half-year average time to complete the full installation and operationalization of the IFMIS.41 As a consequence, the lessons specifically drawn from this project are the following: (a) Provide for enough time (for example, smaller IFMIS or series of project). Three years to start an IFMIS from scratch was not realistic. Either the project could have downsized the extent of the IFMIS by focusing on a few building blocks to fit into a three-year project and/or designed the project as a series of project in several phases (for example, three phases of three years), this way providing for resources to define the following TA phase. Ideally, the time line would provide for a full fiscal year of implementation after rollout to enable fine-tuning and estimate impact. (b) Provide for enough resources. Allocating US$0.6 million for an IFMIS was not realistic. For example, off-the-shelf solutions just for e-procurement system is about US$1 million. (c) Use the Project Preparation Facility (PPF) phase to inform sizing, choice of system, and cost to save time in implementation. (d) Simple, adaptable, and off-the-shelf solutions are preferable in a low-capacity environment. The 2014 IFMIS handbook suggested small and simple solutions, such as Free-Balance, Agresso, and so on. SIMBA is in the same product line. Here, SIMBA was chosen by the Government of the Comoros given the budget constraints and after seeing what it achieved for the city of Douala (Cameroon). The linear and classic process of IFMIS selection, from developing terms of reference with all stakeholders to ensure consensus on the specific needs to a study tour to witness how the pre-identified software behaved, was a lengthy process of nearly four years (2011– 40 5/21/2016 World Bank GovTalk: Public Financial Management in Post-recovery Libya, Schaeffer and Wesal Ashur, PFM in Post Conflict Libya. 41 Hashim, Ali .2014. A Handbook on Financial Management Information Systems for Government, A Practitioners’ Guide for Setting Reform Priorities, Systems Design, and Implementation (Based on a compilation of experiences in World Bank-financed projects) World Bank. 27 2015) but built ownership and consensus, ensured cost-effectiveness, and enabled rapid deployment in 2016. (e) Plan for pilot-testing before full deployment—to enable not only the fine-tuning of the IFMIS to the client’s need and the specificity of the country PFM system but also the relevance of the trainings and operation manuals. (f) SIMBA advantages: cost-effective at entry and in maintenance. Because SIMBA has been created and promoted by a non-profit organization of practitioners, International Association of Francophones local Government Executives (Association Internationale des Maires Francophones, AIMF) the system is open source, 42 adaptable to specific needs at a reasonable cost compared to off-the-shelf solutions developed for profit, and TA is partly de facto subsidized by the AIMF. (g) Take into account the information and communication technology readiness of the Government and country. The low level of computer literacy of civil servants that will update the system will affect IFMIS deployment, and the lack of IT support staff and experts, within the Government or even within the country, will affect implementation in the mid-term, as the project will rely on volatile and costly international expertise. Assessing information and communication technology readiness would help design activities to mitigate the mid-term risk by training a critical mass of IT support staff in addition to the minimum computer literacy needed to operate the system. 100. Sustaining reform momentum: strengthen the demand side. The reform momentum which tends to be strong at the design phase needs to be sustained during implementation. At inception, a strong political commitment to reform can be driven by donor conditionality (for example, HPIC or DPO/ECF) or domestic momentum (for example, post-conflict context) but is not necessarily sustained or not associated with de facto change (for example, isomorphic mimicry). 43 Building the capacity of civil society on PFM reform contributes to fostering the Government’s commitment, coordination, and collaboration. Also, increasing transparency does not necessarily lead to higher accountability and better access to information does not systematically translate into the information being used to hold the Government accountable. Other elements matter in the environment.44 Building budget literacy of civil society, fostering collective action, facilitating Civil Society-Government dialogue on budget and governance reforms as well as access to information are a key combination. 5. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies 101. A French version of the draft ICR report was circulated to the client by the World Bank team on May 2, 2017, and the comments were received on May 15, 2017, from the PIU. Most of 42 Nearly 10 percent of the contract with the AIMF was conditioned by the delivery of source codes. 43 Andrews, Matt. 2009. “Isomorphism and the Limits to African Public Finance Management Reform” Faculty Research Working Paper, Harvard Kennedy School. 44 Grandvoinnet, Helene, Ghazia Aslam, and Shomikho Raha. 2015. Opening the Black Box: Contextual Drivers of Social Accountability. World Bank. The authors identified five constitutive elements of social accountability: (a) citizen action, (b) state action, (c) information, (d) citizen-state interface, and (e) civic mobilization. 28 the Government formatting suggestions and requests for clarification were included in the final ICR. The client did not contest the overall findings nor the performance rankings. (b) Cofinanciers Not applicable. (c) Other partners and stakeholders Not applicable. 29 Annex 1. Project Costs and Financing Project Cost by Component (in US$, millions equivalent) Total Project Costs (US$) Components Original Proposed AF Total Financing Component 1: Rationalizing PFM Institutional Frameworks and Processes 795,300 2,294,000 3,089,300 Component 2: Supporting Civil Service Reforms, including wage bill management 170,220 395,000 565,220 Component 3: Demand-driven Facility 130,000 160,000 290,000 Component 4 : Project Management 274,701 591,000 865,701 External audit 65,000 60,000 125,000 PPF and contingencies 364,779 0 364,779 Total Project Costs 1,800,000 3,500,000 5,300,000 30 Annex 2. Output by Component Planned Outputs Status Component 1: Rationalizing the PFM Institutional Frameworks and Processes PDO: Increase accountability in PFM 1.1 Clarifying the legal and institutional framework of public finances  Review of the existing regulatory Achieved. Completed in full in 2011 framework  Draft of a new organic PFM law Achieved. Completed in full in 2012.  Reorganization of financial Achieved. Completed in full in 2012 as part of the HIPC administration set up to reflect a completion point. new organic law.  Introduction of a new accounting Partially achieved. In progress in the context of the creation network (réseau comptabilité) of DGCPT.  Redefinition of the roles of Achieved. Completed as part of the HIPC completion point. various accountants and their reporting relations.  Creation of a new Directorate of Achieved. The DGCPT was created and a facility was Public Accounting refurbished in Moroni.  Reorganization of competencies, Achieved. 2012 Decrees 12-159/PR instituting and organizing attributions, and functioning of the financial control of expenses and 12-161/PR on the the Financial Controller’s Office attributions and organization of the Financial Controller’s Office. 2016 Financial Control Operations Manual. PDO: Increase transparency in PFM  Preparation and implementation Achieved. Completed in full in in March 2016. of the national communication strategy on the CREF reforms.  Preparation and implementation Not achieved. of the national M&E strategy on the CREF reforms.  Completion of a national survey Not achieved. Delays in the implementations of GISE caused of public servants to inform the this not to be finished. outcomes of the reforms. PDO: Increase efficiency in PFM 1.2 Strengthening the budgetary process  Development of a new budget Achieved. Completed in full in 2015. Arrété n°15-037/VP- preparation time table MFEBICEP/CAB, of July 14, 2015.  Development of a realistic Achieved. Completed in full in 2012 as part of the HIPC methodology for budget completion point. preparation  Preparation of a budget law. Achieved. Completed and done in 2014, 2015, 2016, and 2017. The budget law was also printed and distributed.  Preparation of decree specifying Achieved. Completed in full in 2012. the professional requirements for finance managers (DAFs), as well as nomination procedures. 31  Development of a training plan Partially Achieved. Diagnostic and recommendation and Operations Manual for completed in 2012 (doc ref 12-04-ABGE) and Operations DAFs. Manual completed in 2012. Training plan not finalized.  Development of a Manual of Not Achieved. This activity has been postponed to 2020. Procedures and Action Plan to allow for results-oriented program budgets by 2018  Audit and recommendation of a Achieved. Completed in October 2014. three-year strategic plan for public procurement entities (ARMP and DNCMP) 1.3 Supporting the gradual computerization of public finances  Development of a Achieved. Completed in full in 2012. computerization strategy of public finances  Installation and Achieved. Completed in June 2016. operationalization of the IFMIS IT solution (SIMBA)  Training of the IFMIS users Achieved. Additional training needs emerged due to new users and staff turnover.  Inclusion of PFM MIS budget Achieved. All SIMBA modules are being used (budget and treasury modules in the allocations, expenditure and revenue transactions, accounting, IFMIS system. and reporting. The problem is the backlog of transactions that should be entered to match manual records (due to technical challenges and capacity issues). If this is referring to the ‘budget planning’ module, this is not in scope of the AIMF contract and will be considered as a part of expansion later. Component 2: Supporting civil service reforms, including wage bill management PDO: Improve HR management 2.1 Supporting a comprehensive census of civil servants.  Comprehensive physical census Achieved. Completed in full 2011. of civil servants completed 2.2 Supporting Human Resource Management  Comprehensive review of each Achieved. Completed in full 2011as part of the HIPC administrations organization completion point. framework (Cadres Organiques)  Adoption of Cardres Organiques Achieved. Completed in full 2012 as part of the HIPC by parliament completion point.  Revision of draft staffing plans Not Achieved. A consultant was hired, but the staffing plan produced by HAFOP was never completed.  Training of HAFOP in Not Achieved. Not done as HAFOP was internally competitive recruitment transitioning during period. processes  Completion of a study on staff Not Achieved. A consultant was hired, but the evaluation was performance evaluation never completed.  Development of an outreach Achieved in November 2015. strategy to communicate reform orientations 32  Update civil service census based Not Achieved. Not completed due to delays in GISE on new data integration.  Improve HAFOP capacity with Not Achieved. Organization does not effectively exist. TA and training PDO: Improve wage bill management 2.3. Computerization of Wage Bill Management  Installation of an integrated Achieved. Rolled out in 2016 with 100% of civil servants human resources and payroll captured in the system. management software HRMIS (GISE)  Development of detailed user Achieved. A manual has been developed and updated with manuals new specifications of the improved GISE.  Training provided to GISE users Achieved. Additional training needs emerged due to new users and staff turnover.  Technical audit to upgrade Achieved. Completed by Africa Regional Technical integrated management software Assistance Centre (AFRITAC) in 2013 and informed the AF for data security and integrity in 2013.  Preparation of a comprehensive Achieved. Completed and approved in May 2016. CSR strategy for civil service management Component 3: Demand Driven Facility  No Activities Defined a priori Transversal (for example, coaching, RRIs: Completion RRIs (please see below for details), motoring, leadership, change including GISE related work to clean up the database. management, communications, PDO: Increase accountability of PFM consensus building, and so on) Cabinet Retreats: Supported two high-level retreats: (a) civil service management and (b) budget execution through computerized system. PDO: Increase efficiency of PFM Economic Adviser: Support an adviser in CREF to donor- funded activities including DPOs. PDO: Increase efficiency of PFM Support to CREF: Provided light renovations, VC, and website. PDO: Increase transparency of PFM Reform Dissemination: Supported a workshop on key reforms in 2015 to communicate reform priorities with stakeholders. PDO: Increase efficiency of PFM Update of PEFA 2016: A PEFA self-evaluation was completed in June 2016. PDO: Increase transparency of PFM Tax Administration Diagnostic Assessment Tool (TADAT) Dissemination: Workshop of dissemination and feedback on TADAT. PDO: Increase transparency of PFM Civil Society Training: Training for civil society organizations to read understand the budget process and the production of a citizen budget. 33 PDO: Increase transparency of PFM National Anticorruption Agency: Financed office and IT equipment, including website. Component 4: Project Management  Training/workshops M&E, PEFA, civil service management, accounting, and procurement training were provided to the PIU staff.  Facilities Procurement of office equipment, IT equipment, and office supplies.  Staffing Salaries of PFM reform manager, SCR manager, M&E specialist, accounting and procurement assistants, midterm review consultant, and procurement consultant.  Incremental operating Costs Rent, fuel, and utilities. Note: DAF = Administrative and Finance Director (Directeur Administratif et Financier). Overview of RRIs45 1. Critical initiatives along two cycles at the level of the four entities were selected, designed, launched, and implemented by multi-stakeholder teams with the close support of four local RRI coaches, and supervision/quality support from two regional RRI coaches in addition to World Bank specialists.  Cycle1/2016: 12 RRIs 46 from mid/end-April to end-July/mid-August 2016 o Focus areas. Of the 12 total RRIs, each of the four entities (three islands plus the Union) implemented three RRIs each, relating to the following topics: (a) GISE: paying salaries of all state officers, (b) SIMBA: integration of all expenditure and revenue data since the beginning of the fiscal year and recording of daily transactions in real time, and (c) publication of budget execution reports for 2015 and the first quarter of 2016. o Results achieved. The average completion rates for the three groups of RRIs were respectively around 70 percent for the payment of state employees with GISE, 76 percent for the operationalization of the budget execution, and 75 percent for the publication of budget execution reports, a critical criteria expressing transparency in the management of public funds. 2. At the end of the first cycle of RRIs, clear progress was made, however, some civil servants continued to be paid through manual statements, data integrity problems persisted, and a number of users had not yet mastered the new systems, because of a lack of training (or the quality of training).  Cycle2/2016: 16 RRIs from October to end-December 2016 45 Source: CL4D Summary of P156344 LLICC CL4D Comoros (Support to Economic Governance Technical Assistance [AF] in the Union of the Comoros) March 2017. 46 Those initiatives contributed to the development of five indicators of the project's Results Framework: Intermediate Results Indicator [IRI] 2.1 (PEFA PI-18); IRI 1.4 (PEFA PI-24); PDO-level Results Indicator [PDO] B. (PEFA PI-10); and PDO C. 34 o Focus areas. Three RRIs were then conducted at the level of each entity (that is, a total of 12 RRIs), focusing on (a) cleaning personnel database, including verifying GISE records with a national ID system, (b) registering all employees and payroll details in GISE, and (c) minimizing manual payments and registering all expenditure/revenue in SIMBA on a daily basis. In addition, three other initiatives were planned at the level of the union to establish some interfaces between the systems (GISE and National ID system to exchange data; GISE and SIMBA for secure transfer of salary payment details; and SIMBA and banks for e-payments and reconciliation). o Results achieved. By end of December 2016, all payments have been processed through the GISE system for the union, Ngazidja and Mwali, while 14 civil servants and one contractor (representing 0.3 percent of the total) were still paid manually—that is, outside the GISE system—in the Island of Ndzuani. Also, the preparation of budget execution reports was controlled and well carried out, leading to the publication of the budget reports for 2015 and 2016. The same applies to the availability of civil status data for state employees. However, progress was limited in setting up interfaces between systems, for various reasons related to insufficient readiness. 35 Annex 3. Economic and Financial Analysis Not applicable. 36 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team Members Responsibility/ Names Title Unit Specialty Lending Otieno Ayany Financial Management Specialist AFTFM Jacques Morisset Lead Economist AFTP1 Catherine Gachukia Consultant AFTP2 Anne Khatimba Program Assistant AFCE2 Aurélien Kruse Sr. Economist AFTP1 Team Leader Betty Maina Consultant AFTP2 Jean Gaspard Ntoutoume Ayi Sr. Financial Management, Consultant AFTP1 Sylvain Rambeloson Sr. Procurement Specialist AFTPC Thomas Jeffrey Ramin Sr. Operations Officer AFTRL Raj Soopramanien Lead Counsel LEGAF Sr. Financial Management Specialist Renaud Seligmann AFTFM (co-TTL) Javier Suarez Sr. Economist AFTFE Roger Sullivan Consultant AFTP2 Johannes Widmann Country Officer AFCKE Evangaline Rose Kumsinda Office Administrator AFTP1 Supervision Anne-Lucie Lefebvre Sr. Public Sector Specialist GGO17 Team Leader Andrianjaka Razafimandimby Consultant GGOPS Rado Benjamina Randrianarivelo Sr. Operations Officer GGELI Cem Dener Lead Governance Specialist GGO19 Hajarivony Andriamarofara Consultant GGO13 Heriniaina Mikaela Public Sector Specialist GGO13 Andrianasy Maharavo Harimandimby Financial Management Specialist GGO26 Ramarotahiantsoa Michael Christopher Jelenic Consultant GGO19 Said Ali Antoissi Operations Analyst AFMMG 37 (b) Staff Time and Cost Staff Time and Cost (World Bank Budget Only) Stage of Project Cycle US$, thousands (including No. of Staff Weeks travel and consultant costs) Lending FY2007 28.63 124,174.44 FY2008 7.65 44,100.49 FY2009 17.66 85,079.01 FY2010 19.42 132,268.20 FY2011 9.77 79,504.67 Total Lending 83.13 465,126.81 Supervision/ICR FY2011 3.76 26,346.01 FY2012 18.16 135,210.03 FY2013 15.05 132,672.89 FY2014 18.72 65,117.53 FY2015 19.27 129,274.41 FY2016 21.87 127,130.11 FY2017 19.45 160,967.39 Total Supervision 116.28 776,718.37 38 Annex 5. Beneficiary Survey Results Not applicable. 39 Annex 6. Stakeholder Workshop Report and Results Not applicable. 40 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR submitted 1. A French version of the draft ICR report was circulated to the client by the World Bank team on May 2, 2017 and comments were received on May 15 2017 from the PIU. Most of the government formatting suggestions and requests for clarification were included in the final ICR. The client did not contest the overall findings, nor the performance rankings. 2. The following is a summary of the evaluation of the project conducted before this ICR in November–December 2016 by an independent evaluator. This evaluation of the ABGE was intended to assess the implementation of the project by identifying strengths and constraints through an analysis of the relevance and coherence, effectiveness, and efficiency of the project. It also aimed to provide insights on project sustainability and ownership. (a) The project has achieved satisfactory results. Indeed, majority of the activities were carried out successfully. Maintaining the performance of the project would require consolidating the gains made in streamlining PFM and supporting civil service support activities initiated by the project. (b) On project implementation. The project aligned with the World Bank's ISN. The project formed an integrated package with the World Bank's operations under the DPO. The design of the project also considered the requirements of the HIPC decision point document under the IMF Enhanced Initiative, which required complementarity between the interventions of the AfDB and the World Bank. The ABGE 1 contributed significantly to reaching completion point in 2012. All the conditions for the implementation of the two ABGE grants were met within an overall reasonable timeframe. The project M&E is considered to be effective and in line with good internationally recognized practices. The borrower has fully complied with its commitments to the donor. (c) On project execution. The mission found that the project management team was in place and operational. All activities under the annual budgeted work plan were carried out in almost all cases. As of November 30, 2016, the commitment rate for the public finance management component was 87 percent for a disbursement rate of 82 percent. The ‘CSR support’ component was executed at a commitment rate of 109 percent. The ‘support to demand’ component had a commitment rate of 62.5 percent. The commitment rate for the project management component was 96.7 percent for a disbursement rate of 95 percent. The activities to be finalized on November 30, 2016 were mainly linked to the audit of the accounts for 2016 and the final evaluation mission. The project was generally efficient in meeting the timing of disbursement, resource use, reallocation of funds, and savings in funds. However, better management of the procurement of goods and services would have allowed to respect the disbursement schedule and favor a better impact of the project on the management of the public finances. (d) On project’s implementation arrangements. The executing unit completed its job well. Overall, it has performed its role as an executing agency effectively. The 41 members of the PIU had the necessary technical experience in the conduct of economic governance projects. The technical assistance missions provided to the members of the project management office were deemed satisfactory insofar as they contributed to the capacity development of the PIU. The perception of the stakeholders of the Steering Committee was generally mixed. Some stakeholders considered that the monitoring carried out by the Steering Committee was insufficient. Others believed that Steering Committee had been effective. In addition to the functioning of these two bodies, the participatory approach to the project had been characterized by a participatory approach involving beneficiaries. This emerged from the actions undertaken by the project, including the organization of seminars to launch the project with the participation of beneficiaries and the organization of workshops for the restitution of certain studies financed by the project. The beneficiary structures confirmed that they were involved in the identification of needs. The Government and the World Bank agreed on the implementation of the project to give particular importance to the issue of leadership and change management. This approach has been implemented through interconnected tools, including the organization of Government seminars, the use of RRIs, and the consideration of socio-anthropological aspects. (e) On relevance and coherence. The project is considered to be globally relevant and coherent. Apart from the fact that the PDO remains, the design of the project was deemed relevant because the project included a ‘demand side’ component. Adjustments along the way of the Results Framework and the conformity of the activities selected with the needs of the beneficiary structures also justified the relevance of the project. The beneficiary structures have a very positive perception of the procedure for selecting activities. Changes introduced to the project along the way are well documented and justified. The ABGE is in full coherence with the guidelines established by the Growth and Poverty Reduction Strategy. The coherence of the ABGE with the SCA2D is justified by the fact that this strategy emphasizes the structural reforms of public management which is the essence of the ABGE. In addition, axis 4 of the SCA2D on strengthening governance and institutional and human resilience capacities is in perfect harmony with the purpose of the ABGE. The 2010–2019 Public Finance Management Reform Strategy Paper (SRGFP) overlaps with the ABGE in several respects. (f) On effectiveness. The multi-criteria evaluation of efficiency shows that overall, the program is efficient. The efficiency of the FM of the project is generally high. The good quality of the M&E of project activities has positively contributed to improving the overall efficiency of its implementation. The unit cost of training per participant was relatively high. The project could have provided training for a larger number of participants. It is therefore judged to be moderately efficient. The average cost of the studies implied that the project was moderately efficient in this respect. For the efficiency of computerization, the implementation of these two systems, GISE and SIMBA, was in line with best practices. With regard to turnaround time, the results of the evaluation concluded that these delays were considered as generally average. The implementation of the enhanced GISE has been carried out over a long period, since 42 it was initiated in 2014 and validated only in 2016. Finally, the costs of computerization actions varied widely. (g) On efficiency. The overall comparative analysis of the evaluations of the ABGE on the various criteria selected made it possible to note that the project is effective. In improving PFM, the project is globally effective. This is mainly due to the achievement of the objectives of improving the PEFA indicator on the composition of budgetary expenditure. On the other hand, with regard to the publication of the budget execution report, the target note was not implemented despite efforts to publish various documents. For the analysis of project effectiveness with regard to improved public service management, the project is considered to be effective. The development of the Public Service Reform Strategy and its budgeted action plan has been completed. On the other hand, the objective of paying 90 percent of civil servants through the automated system and the objective of strengthening the controls on the accuracy of the civil service payroll have not been fully achieved. The project was perfectly effective in carrying out the activities. The project has succeeded in achieving the objectives of the execution of the activities. (h) On impact and sustainability. The multi-criteria assessment of the impacts shows that overall, the project has a reduced impact. While the project has had important results in building the capacity of the recipient administrations, these results are slow to generate positive effects in improving the transparency of public service management and improved accountability and transparency in PFM. The impact of the project on the capacities of civil society remains limited for the time being. On the other hand, the internalization of reforms relating to the role of civil society in budget monitoring and procurement should strengthen the contribution of the project in this direction. The beneficiaries believe that training has had an impact on the organization and development of their activities. The project has significant potential to generate positive effects in the future on the various PEFA indicators. Thus, by focusing on the internalization of legal reforms, computerization, and communication, this project is likely to make a significant contribution to improve the performance of PFM. Although the bulk of the activities have been fully implemented, there are still significant challenges to ownership of the reforms. 43 Annex 8. Comments of Co-financiers and Other Partners/Stakeholders Not applicable. 44 Annex 9. List of Supporting Documents (a) List of budgetary information published on the MOF’s website as of ICR  Source : http://www.finances.gouv.km/v1/index.php/directions/direction- generale-du-budget  Loi des finances 2013 de l'île autonome d'Anjouan  Loi des finances 2015 de l'île autonome d'Anjouan  Rapport d'exécution budgétaire 1er trimestre 2016 (Mohéli)  Rapport d'exécution budgétaire 2e trimestre 2016 (Mohéli)  Rapport d'exécution budgétaire 3e trimestre 2016 (Mohéli)  Loi des finances 2013 de l'île autonome de Ngazidja  Loi des finances 2015 de l'île autonome de Ngazidja  Rapport d'exécution budgétaire 4ème trimestre 2015 (Ngazidja)  Rapport d'exécution budgétaire 3ème trimestre 2015 (Ngazidja)  Rapport d'exécution budgétaire 2ème trimestre 2015 (Ngazidja)  Rapport d'exécution budgétaire 1er trimestre 2015 (Ngazidja)  Rapport d'exécution budgétaire 1er trimestre 2016 (Ngazidja)  Budget 2009  Budget 2010  Budget 2011  Circulaire budgétaire 2011  Budget 2012  Mécanisme de partage budgétaire 2012  Loi des finances 2013  Budget 2013 45  Rapport sur l'exécution budgétaire 2014  Loi des finances 2014  Loi des finances 2014 de l'île autonome d'Anjouan  Annexes de la loi des finances 2014  Rapport sur l'exécution budgétaire du deuxième trimestre 2015  Rapport sur l'exécution budgétaire du premier trimestre 2015  Loi des finances 2015 et annexes  Rapport d'Exécution Budgétaire 2015  Loi des finances 2016 et annexes  Rapport d'exécution budgétaire du 1er trimestre 2016  Circulaire budgétaire 2009 destiné-Présidence-Ministères-Institutions  Manuel de procédure Budgétaire  Préparation Budgétaire  Nomenclature budgétaire  Principe de gestion Budgétaire  Budget Grande Comores  Lois des finances  LOLF  Salaire effectifs des Ministères  Budget Mohéli  Gestion axée sur les résultats  TOFE  Processus budgétaires et comptables en vue de l'informatisation de la chaine de la dépense 46  Arrêté processus budgétaire Union des Comores  Rapports sur l'exécution budgétaire (2013,2012,2011) (b) PEFA INDICATORS 2011 Key Cross-cutting Issues: Comprehensiveness and Transparency PI-10 Public access to key fiscal information Transparency will depend on whether information on fiscal plans, positions, and performance of the Government is easily accessible to the general public or at least the relevant interest groups. The narrative of the assessment should comment on the quality of information made available (for example, understandable language and structure, appropriate layout, summarized for large documents), and the means used to facilitate public access (such as the press, websites, sale of major documents at no more than printing cost and notice boards for mainly locally relevant information). The extent to which the means are appropriate depends on the nature of the documentation and the characteristics of the relevant interest or user groups, such as access to different media. Elements of information to which public access is essential include the following: (a) Annual budget documentation. A complete set of documents can be obtained by the public through appropriate means when it is submitted to the legislature. (b) In-year budget execution reports. The reports are routinely made available to the public through appropriate means within one month of their completion. (c) Year-end financial statements. The statements are made available to the public through appropriate means within six months of the completed audit. (d) External audit reports. All reports on the Central Government consolidated operations are made available to the public through appropriate means within six months of completed audit. (e) Contract awards. Award of all contracts with value above approximately US$100,000 equivalent are published at least quarterly through appropriate means. (f) Resources available to primary service units. Information is publicized through appropriate means at least annually, or available upon request, for primary service units with national coverage in at least two sectors (such as elementary schools or primary health clinics). Dimension to be assessed (Scoring Method M1): 47 (a) Number of the above listed elements of public access to information that is fulfilled (to be counted in the assessment, the full specification of the information benchmark must be met). Budget Cycle Policy-based Budgeting PI 11: Orderliness and Participation in the Annual Budget Process While the MOF is usually the driver of the annual budget formulation process, effective participation in the budget formulation process by other ministries, departments, and agencies, (MDAs) as well as the political leadership, impacts the extent to which the budget will reflect macroeconomic, fiscal, and sector policies. Full participation requires an integrated top-down and bottom-up budgeting process, involving all parties in an orderly and timely manner, in accordance with a pre-determined budget formulation calendar. The calendar should allow for passing of the budget law before the start of the fiscal year, as well as for sufficient time for the other MDAs to meaningfully prepare their detailed budget proposals according to the guidance. Delays in passing the budget may create uncertainty about the level of approved expenditures and delays in some government activities, including major contracts. Clear guidance on the budget process should be provided in the budget circular and budget formulation manual, including indicative budgetary ceilings for administrative units or functional areas. To avoid last minute changes to budget proposals, it is important that the political leadership is actively involved in the setting of aggregate allocations (particularly for sectors or functions) from an early stage of the budget preparation process. This should be initiated through review and approval of the allocation ceilings in the budget circular, either by approving the budget circular or by approving a preceding proposal for aggregate allocations (for example, in a budget outlook paper). Dimensions to be assessed (Scoring method M2): (a) Existence of and adherence to a fixed budget calendar (b) Clarity/comprehensiveness of and political involvement in the guidance on the preparation of budget submissions (budget circular or equivalent) (c) Timely budget approval by the legislature or similarly mandated body (within the last three years) Predictability and Control in Budget Execution PI- 16: Predictability in the Availability of Funds for Commitment of Expenditures Effective execution of the budget, in accordance with the work plans, requires that the spending MDAs receive reliable information on availability of funds within which they can commit 48 expenditure for recurrent and capital inputs. This indicator assesses the extent to which the central MOF provides reliable information on the availability of funds to MDAs, that manage administrative (or program) budget heads (or votes) in the Central Government budget and therefore are the primary recipients of such information from the MOF. The MDAs concerned in this indicator are the same as those concerned in indicator PI-11. In some systems, funds (commitment ceilings, authority to spend or transfers of cash) are released by the MOF in stages within the budget year (monthly, quarterly, and so on). In others, the passing of the annual budget law grants the full authority to spend at the beginning of the year, but the MOF (or other central agency) may, in practice, impose delays on ministries in incurring new commitments (and making related payments), when cash flow problems arise. To be reliable, the amount of funds made available to an entity for a specific period should not be reduced during that period. Predictability for MDAs in the availability of funds is facilitated by effective cash flow planning, monitoring, and management by the treasury, based on regular and reliable forecasts of cash inflows and of major, atypical outflows (such as the cost of holding an election and discrete capital investments) which are linked to the budget implementation and commitment plans for individual MDAs, and incorporate the planned in-year borrowing to ensure adequate liquidity at any time. Governments may need to make in-year adjustments to allocations in the light of unanticipated events affecting revenues and/or expenditures. The impact on predictability and on the integrity of original budget allocations is minimized by specifying, in advance, an adjustment mechanism that relates adjustment to the budget priorities in a systematic and transparent manner (for example, protection of particular votes or budget lines that are declared to be high priority, or ‘poverty related’). In contrast, adjustments can take place without clear rules/guidelines or can be undertaken informally (for example, through imposing delays on new commitments). While many budget adjustments can take place administratively with little implication for the expenditure composition outturn at the more aggregate level of budget classifications, other more significant changes may change the actual composition at fairly aggregate administrative, functional, and economic classification levels. Rules for when the legislature should be involved in such in-year budget amendments are assessed in PI-27 and not covered here. The adherence of MDAs with the ceilings for expenditure commitment and payments is not assessed here, but is covered by indicator PI-20 on internal controls. Dimensions to be assessed (Scoring Method M1): (a) Extent to which cash flows are forecast and monitored (b) Reliability and horizon of periodic in-year information to MDAs on ceilings for expenditure commitment (c) Frequency and transparency of adjustments to budget allocations, which are decided above the level of management of MDAs. 49 PI -18: Effectiveness of Payroll Controls The wage bill is usually one of the biggest items of Government expenditure and susceptible to weak control and corruption. This indicator is concerned with the payroll for public servants only. Wages for casual labor and discretionary allowances that do not form part of the payroll system are included in the assessment of general internal controls (PI-20). However, different segments of the public service may be recorded in different payrolls. All of the more important of such payrolls should be assessed as the basis for scoring this indicator, and mentioned in the narrative. The payroll is underpinned by a personnel database (in some cases called the ‘nominal roll’ and not necessarily computerized), which provides a list of all staff, who should be paid every month and which can be verified against the approved establishment list and the individual personnel records (or staff files). The link between the personnel database and the payroll is a key control. Any amendments required to the personnel database should be processed on time through a change report and should result in an audit trail. Payroll audits should be undertaken regularly to identify ghost workers, fill data gaps, and identify control inadequacies. Dimensions to be assessed (Scoring Method M1): (a) Degree of integration and reconciliation between personnel records and payroll data (a) Timeliness of changes to personnel records and the payroll (b) Internal controls of changes to personnel records and the payroll (c) Existence of payroll audits to identify and control weaknesses and/or ghost workers. Accounting, Recording, and Reporting PI -24: Quality and Timeliness of In-year Budget Reports The ability to ‘bring in’ the budget requires timely and regular information on actual budget performance to be available both to the MOF (and Cabinet), to monitor performance and, if necessary, to identify new actions to get the budget back on track, and to the MDAs for managing the affairs for which they are accountable for. The indicator focuses on the ability to produce comprehensive reports from the accounting system on all aspects of the budget (that is, flash reports on release of funds to MDAs are not sufficient). Coverage of expenditure at both the commitment and the payment stage is important for monitoring of budget implementation and utilization of funds released. Accounting for expenditure made from transfers to deconcentrated units within the Central Government (such as provincial administrations) should be included. The division of responsibility between the MOF and line ministries in the preparation of the reports will depend on the type of accounting and payment system in operation. The role of the MOF may be simply to consolidate reports provided by the line ministries (and where applicable, from deconcentrated units) from their accounting records. In other cases, the MOF may undertake the data entry and accounting for transactions in which case the role of the line ministry is reduced, perhaps to reconciling MOF data with their own records. In yet other cases, the MOF can generate 50 reports out of integrated, computerized accounting systems. The important requirement is that data are sufficiently accurate to be of real use to all parties. Dimensions to be assessed (Scoring Method M1): (a) Scope of reports in coverage and compatibility with budget estimates (a) Timeliness of the issue of reports (b) Quality of information Scoring Methodology Method 1 (M1) is used for all single dimensional indicators and for multidimensional indicators where poor performance on one dimension of the indicator is likely to undermine the impact of good performance on other dimensions of the same indicator (in other words, by the weakest link in the connected dimensions of the indicator). Method 2 (M2) is based on averaging the scores for individual dimensions of an indicator. 51 MAP 52