Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Financial Sector Support TA Project (P121514) Report Number : ICRR0021229 1. Project Data Project ID Project Name P121514 Financial Sector Support TA Project Country Practice Area(Lead) Sierra Leone Finance, Competitiveness and Innovation L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-H6620 31-Dec-2015 2,235,425.93 Bank Approval Date Closing Date (Actual) 07-Apr-2011 30-Nov-2017 IBRD/IDA (USD) Grants (USD) Original Commitment 4,000,000.00 0.00 Revised Commitment 4,000,000.00 0.00 Actual 2,384,323.67 0.00 Prepared by Reviewed by ICR Review Coordinator Group Nestor Ntungwanayo Vibecke Dixon Christopher David Nelson IEGFP (Unit 3) 2. Project Objectives and Components a. Objectives As per the Financial Agreement (FA) on page 5, "the objective of the Project is to strengthen the capacity of the Bank of Sierra Leone (BSL) and contribute to improving efficiency in financial intermediation, safeguarding financial sector stability, reducing transaction costs of money transfer and expanding access to financial services." The statement of the project objectives in the Project Appraisal Document (PAD) on page 7 is identical. Page 1 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Financial Sector Support TA Project (P121514) b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? No PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? Yes d. Components The project had two components as developed below, with details provided on page 9 and 10 of the ICR: Component 1: Enhancing access to financial services (US$1.6 million at approval, with accrual amount of US$0.7 million) Under this component, the project aimed to carry out a program of activities to improve efficiency in financial intermediation, reduce transaction costs of money transfer and expand access to financial services. Planned activities included: • the development of a basic bad debtor database; the set-up of a comprehensive credit information bureau; and the strengthening the Bank of Sierra Leone’s oversight capacity; • the implementation of the payment systems and remittances policy framework through: (a) the preparation of payment systems regulations and policies; (b) the preparation of mobile payment regulations; (c) the preparation of a remittances policy framework; and (d) the strengthening of the Bank of Sierra Leone’s capacity for oversight and supervision; • the provision of technical advisory services for developing micro, small and medium enterprises as well as /value chain finance capacity of commercial banks including: (a) studies on an enabling environment to support micro, small and medium enterprises (MSME) finance and agricultural finance; (b) developing sector policies and supporting dialogue on enhancing access to financial services, (c) preparing secured lending laws and regulations; and (d) strengthening related technical capacities of selected commercial banks. Added activity at restructuring • At project restructuring in October 2016, an activity aimed at conducting a special diagnostic of two state-owned banks was added. Component 2: Building Financial Sector Reform and Oversight Capacity of the Bank of Sierra Leone (US$2.40 million at approval, with accrual amount of US$1.6 million) This component intended to strengthen the technical and institutional capacity of the Bank of Sierra Leone so that it can implement the Financial Sector Development Plan and supervise the financial sector. The project's support was to cover: • the financial sector legal reforms, the implementation of the Financial Sector Legislation, and the strengthening of the capacity of the Bank of Sierra Leone’s legal department; • the implementation of the Bank of Sierra Leone’s Institutional Development Plan, including: (a) preparing a new human resources policy; (b) supporting the staff development program, (c) developing a broad- based training program for the Bank of Sierra Leone’s staff; (d) reengineering both functional and business processes; (e) developing an information technology strategy and implementation plan; (f) implementing Page 2 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Financial Sector Support TA Project (P121514) the Anti-Money Laundering and Countering the Financing of Terrorism (AML/CFT) framework; and (g) designing and implementing efficient supervisory processes; • the strengthening of the technical, institutional, fiduciary, monitoring, evaluation, and project management capacity of the FSDP Secretariat; and • the provision of technical advisory services to support the Bank of Sierra Leone in the preparation of financial sector development strategies. Added activities at restructuring • At project restructuring in October 2016, three activities were added as follows: (i) to support the establishment of a collateral registry; (ii) to set-up a switch infrastructure and make it operational; and (iii) to prepare a financial inclusion strategy and conduct a baseline financial inclusion survey. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The project cost amounted to US$2.2million, or 56 percent of the approved amount. Financing: The project was to be funded by an International Development Association (IDA) grant in the amount of US$4.0 million Borrower Contribution: There was no borrower contribution planned for this project. Dates: The project was approved on April 7, 2011, and became effective on July 20, 2011. The project was restructured twice: (i) on December 28, 2015 to extend the grant closing date from December 31, 2015 to December 31, 2016, and (ii) on October 17, 2016 to add new activities, revise the result framework, and extend the project closing date to November 2017, with a total extension of about two years. 3. Relevance of Objectives Rationale Original project: The 2018 Systematic Country Diagnostic (SCD) is one of the recent World Bank's reports assessing the country's socio-economic context and its action in Sierra Leone. In the financial area, it identifies access to credit as a binding constraint to business development, and notes that economic diversification requires access to credit and other financial services. To bolster financial intermediation, the SCD suggests to: (i) enhance financial sector supervision, and (ii) address impediments to the growth in credit, and the weak performance of the two state-owned banks (RCB and SLCB), which do not convert deposits into credit, and pose stability concerns. The pillars of the 2017 SCD in the financial sector mirror the project’s PDOs. By the project's closure, the stated objectives were in line with the country's priorities in reforming and strengthening the financial sector, but were also overambitious, given the limited implementation capacity on the ground, and the large scope of the identified activities. Revised project: Page 3 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Financial Sector Support TA Project (P121514) In October 2016, the project was restructured by adding four new activities, and revising the results framework. However, recent Implementation Status Reports were reporting delays in executing initial activities. At the project closure, while both the country's priorities, and the World Bank's agenda in the country's financial sector were like those described in the original project, the new activities made the project more difficult to implement. The revised project was overambitious and could not be implemented as desired. Because of the disconnect between the scope of the project, and the local capacity to implement it, the relevance of objectives in the context of the final project objective is rated Modest. Rating Modest 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective (i) To strengthen the capacity of the Bank of Sierra Leone, (about 60 percent of the disbursed amount) Rationale Outputs: The following output targets in the original project were not systematically assessed in the result framework of the ICR: (i) Key milestones of the BSL Institutional Development Plan (IDP) should have been implemented, (ii) the Revised Financial Sector Legal Framework should have been presented to Cabinet, (iii) the Revised BSL HR policy should have been approved by BSL board of directors, (iv) Staff capacity development plan should have been approved by BSL board of directors, and (v) the Sector Dialogue through Regular Financial Sector Consultative Forum (FSCF) and Financial Sector Steering Committee (FSSC) meetings should have been organized by the Financial Sector Development Plan (FSDP) Secretariat However, the following outputs were generated during project implementation and are developed in the ICR: • A unit of payment systems under the Banking Supervision Department in the BSL was created and staffed with dedicated personnel, and several consultancies contributed to the capacity building of this unit; • A second Deputy Governor and Financial Stability Department was created in 2018, including a Risk Management Unit; • An IT strategy was developed and adopted by the Executive Board, and the BSL has begun its implementation, including hiring six new staff and updating their Enterprise Resource Planning (ERP) systems; • The project also supported training and study tours, including in the following areas: Financial inclusion, Project and Program Management, Cash and Treasury Management, Leadership and Strategy Page 4 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Financial Sector Support TA Project (P121514) for Senior Managers, and Development Finance, Micro-insurance & Microfinance for Business Development; • The World Bank supported the implementation of the Institutional Development Plan of the BSL, including banking supervision, the strengthening of the legal environment of the financial sector, the building of BSL capacity to lead implementation of the project activities, and the completion of the HR review and the IT strategy; • The project conducted a comprehensive review of the regulatory framework surrounding payment systems and provided recommendations for the Payment Systems Act (2009), but there was no revised Act. Outcomes: Performance against key outcome indicators were as follows: • Doing Business Depth of Credit Information Index was to be in line with Sub-Sahara Africa average, but the indicator at closure was at 0, against a target of 1.7, and the target was missed; • Compliance with Basel Core Principles (BCPs) for Banking Supervision should have improved for 10 out of 25 BCPs has improved: The ICR indicates that the project did not fund any activity to improve the compliance with BCPs for banking supervision, and the target was missed. • A comprehensive payment system policy and regulatory framework should have been established. The indicator set to assess the establishment of a comprehensive payment system policy and regulatory framework was missed, because the project did not fund a bad debtor base or an information bureau; and the ICR indicates that any observed impact is not attributable to the project; • Additionally, the project supported the BSL's acquisition (under AfDB funding) of the right systems of Real Time Gross Settlement (RTGS), Automated Clearing House (ACH), Security Settlement System (SSS), and core banking application systems by facilitating site visits to financial institutions and telecommunication systems. The project was instrumental in the successful launch and the modernization of a payment system infrastructure. When planned AfDB implementation support was cancelled, the project stepped in to ensure that these large infrastructure investments, and essential building blocks for developing a robust payment ecosystem were successfully implemented. • Key achievements are: (i) the institutional and organizational improvement of the BSL, (ii) the modernization of a payment system infrastructure, and (iii) the implementation of an HR and Information Technology (IT) strategies. However, the regulatory framework of the payments system could not be improved, and the scope of the remaining achievements was limited. However, all three outcome indicators were missed. Rating Modest PHREVDELTBL Page 5 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Financial Sector Support TA Project (P121514) PHINNERREVISEDTBL Objective 1 Revision 1 Revised Objective (i) To strengthen the capacity of the Bank of Sierra Leone (about 40 percent of the disbursed amount) Revised Rationale During the October 2016 restructuring, there was no change in the PDO, but an activity was added to set- up an operational switch infrastructure and facilitate electronic payments and foster digital finance. This activity was to be implemented in parallel with those approved at appraisal. Outputs: At project closing, the switch infrastructure was not implemented, because it was never procured by the BSL. Outcome: There was only one new outcome indicator: A comprehensive payment system policy had to be established, but the regulatory framework of the payments system could not be improved, and with the failure to install the switch infrastructure implies that the target was missed. Overall, there was a disconnect between the new activity and the selected indicators, and the performance toward the revised objective was weak, as both the output and outcome were missed. Revised Rating Negligible PHEFFICACYTBL Objective 2 Objective (ii) To contribute to improving efficiency in financial intermediation, safeguarding financial sector stability, reducing transaction costs of money transfer, and expanding access to financial services (about 60 percent of the disbursed amount). Rationale Outputs: The ICR did not assess systematically achieved performance against the following indicators: (i) Credit Information Infrastructure: Credit Bureau design in private or public/private ownership developed, (ii) Volume of World Bank Funding – Enabling Environment SME: US$400K, (iii) Volume of World Bank Funding – Institutional Development SME- US$300K, and (iv) Draft payment system policy prepared: Limited outputs were generated in three key areas supported by the project as discussed below: • Contribution to improving efficiency in financial intermediation: • The project supported improved efficiency in financial intermediation by helping establish the rules and procedures for the Automated Clearing House (ACH), the Real Time Gross Settlement (RTGS), and the Page 6 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Financial Sector Support TA Project (P121514) Security Settlement System (SSS), and making these systems go live. The above helped reduce settlement risk with the RTGS, facilitated growth in interbank transactions, and provided a platform for the digitization of direct debits and credits; • Toward reducing transaction cost of money transfer, the project supported an assessment of the market for remittances in Sierra Leone, which provided recommendations on how to improve the market for remittances and increase the efficiency, safety, and reliability of these flows, ultimately aiming at reducing the cost of receiving remittances in Sierra Leone. • Contribution to safeguarding financial sector stability: • The project supported the BSL to investigate the non-performing loans (NPLs) developments, and to explore a framework to enhance banking supervision capacity by migrating to risk-based supervision and building up top-down stress testing capacity. Given that high NPLs are the biggest challenge to financial sector stability, the diagnostics helped to understand the scope of the NPLs issues, but critical actions were not triggered and implemented by the project; • Contribution to expanding access to financial services • A geospatial mapping exercise was conducted in a comprehensive manner and will assist the private sector and international development partners in determining areas that lack financial services and use them to inform the design of a World Bank financial sector project, which will have a focus on financial inclusion; • The project funded a baseline survey of financial service access points, but it did not use the proper methodology to have a national representative sample. The indicator to assess performance of the credit information infrastructure was missed, as it was at 0, instead of being at the level of the SSA average. Outcomes: There ICR did not assess the unique PDO indicator: a 25% increase in outstanding MSME loan portfolio of commercial banks supported through capacity development activities of the project and stable portfolio at risk • There is little evidence that financial intermediation has improved in Sierra Leone during project implementation, as interest rate spreads, a standard measure of efficiency in financial intermediation, increased overall; • With regards to ensuring financial sector stability, the project accomplished little in terms of building BSL supervision capacity; • The project had a negligible impact on expanding access to financial services, and credit to the private sector declined during the project implementation. Financial intermediation declined in recent years, while risks accumulated. Between 2010 and 2017, the ratio of credit to the private sector to GDP declined from 7.7 to 5.6. The indicator assessing efficiency in financial intermediation was missed, as the Doing Page 7 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Financial Sector Support TA Project (P121514) Business Depth of Credit Information Index remained at 0, instead of being in line with the Sub-Sahara Africa average of 3. Rating Negligible PHREVDELTBL PHINNERREVISEDTBL Objective 2 Revision 1 Revised Objective (ii) To contribute to improving efficiency in financial intermediation, safeguarding financial sector stability, reducing transaction costs of money transfer, and expanding access to financial services. (about 40 percent of the disbursed amount) Revised Rationale During the October 2016 restructuring, there was no change in the PDO, but three activities were added, which aimed to: (i) conduct a special diagnostic of two state-owned banks, (ii) establish a collateral registry, and (iii) promote financial inclusion. These activities were implemented in parallel with those approved at appraisal, and achievements arising from the additional activities are detailed below: Outputs: Performance against the unique output indicator was as follows: (i) the target of adopting the financial inclusion strategy by cabinet was achieved, but this activity looks like it was accomplished by the United Nations Capital Development Fund (UNCDF), and is therefore not entirely attributable to the project. • The project supported the diagnostics of the two state-owned banks, the Rokel Commercial Bank (RCB) and the Sierra Leone Commercial Bank (SLCB), which allowed the authorities to take decisive measures and “forcefully restructure the two-state owned banks”, and the intervention involved BSL joint management of the institutions. This work entailed independent audits and diagnostics by international firms. The target of completing two reports and submission to the BSL was met. • The project helped to build the capacity of the BSL staff that facilitated the adoption of the credit registry and the collateral registry. The collateral registry has been operational since 2017, but the indicator was missed, because this performance is not entirely attributable to the project, as the borrower provided much of the funding. • Regarding the collateral registry, this activity was dropped from the project, and no output can be attributed to the project. Outcomes Performance against the outcome indicators was as follows: • Doing Business Depth of Credit Information Index was to be in line with Sub-Sahara Africa average In line with SSA Average (2015: 4.5 percent), the indicator at closure was at 0, against a target of 3, and the target was missed; • Effective implementation of the collateral registry supporting MSME finance, (i) the target of security interests registered with the collateral registry (MSME finance) was exceeded, reaching the level of 107, against a target of 5, (ii) the target of number of inquiries to the collateral registry received from financial Page 8 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Financial Sector Support TA Project (P121514) institutions (MSME finance) was also exceeded, reaching the number of 62, against a target of 5, While targets were achieved, performance cannot be attributed to the project . Overall, while there were some achieved outputs (diagnostic of State banks), key outcome indicators were either missed, or not attributable to the project Revised Rating Modest PHOVRLEFFRATTBL Rationale Theory of change The theory of change of the project could have been improved, through a better statement of the PDO, more precise identification of the activities leading to a set of outputs and outcomes, and a better selection of performance indicators. For instance, improvement in the BSL could have been measured by indicators specific to the country, and not necessarily aligned to international standards. As designed, the project needed more clarity and consistency in linking the project activities and the expected outputs and outcomes, and the pursued objectives. On the positive note, the ICR attempted to construct a theory of change (ICR, p-8) post- implementation, based on the PAD information, but it is a difficult task to build an effective results framework after the project's closure. The theory of change in the ICR was overall acceptable, but it could not be translated into practice, because it was underpinned by a weak results framework, and most importantly, it was defeated by the lack of the required implementation capacity on the ground. Regarding the first objective, there were some limited achievements in (i) the institutional and organizational strengthening of the BSL, (ii) the modernization of the payment system infrastructure, and (iii) the implementation of an HR and Information Technology (IT) strategies. However, the regulatory framework of the payments system could not be improved. Toward the second objective, there were no tangible results toward improving financial intermediation and financial sector stability, or in expanding access to financial services. There was only an increased use of the collateral registry, although not entirely attributable to the project. Because of the limited results toward the two objectives, the project’s overall efficacy is rated Modest. Achievements completed under the original project are also attributable to the revised project. Results from the activities approved during the restructuring phase were minimal: (i) toward the first objective, the switch infrastructure was not implemented, and (ii) toward the second objective, the diagnostics of the two state-owned banks was completed. The collateral registry has been operational since 2017, and the preparation of a financial inclusion strategy was also completed, but both achievements are not entirely attributable to the revised project. Overall, the performance of the revised project is also rated as Modest. Page 9 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Financial Sector Support TA Project (P121514) Overall Efficacy Rating Primary reason Modest Low achievement 5. Efficiency As this was a TA project, there was no economic analysis in the PAD, nor in the ICR. However, the 2006 growth diagnostic suggested that alleviating the binding constraint of access to credit would support growth and poverty reduction. During implementation, some project expenditures in critical areas were cost effective (ICR p.21), including the support to the Payment Systems unit, which facilitated the successful implementation of the AfDB-funded payments infrastructure. Other aspects of project implementation were evidently inefficient, including (i) the cost of the Bank diagnostics which was four times higher than budgeted, (ii) the frequent change of Task Team Leaders (TTLs), which the client noted as a problem, (iii) the time- overruns (a total of a 23-month extension) attributed partly to the Ebola crisis, and finally (iv) the ICR did not document the oversight of the project's financial management. In the end, the project produced few results and did not fully utilize available resources (about 36.4 percent was undisbursed at closing). As a result, there is a large unfinished agenda, and the BSL has requested a new project to address needed reforms and investments. On balance, efficiency is rated as Negligible. Efficiency Rating Negligible a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Point value (%) *Coverage/Scope (%) Available? 0 Appraisal 0 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Original project (about 60 percent of the disbursed amount) Rating: Unsatisfactory The relevance of the objectives at project completion was modest. In addition, efficacy was rated modest, and efficiency was rated negligible due to the poor performance of many of the reforms, resulting in an unsatisfactory outcome. Page 10 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Financial Sector Support TA Project (P121514) Revised project: (about 40 percent of the disbursed amount) Rating: Unsatisfactory The relevance of the objectives at project completion was modest. Both efficacy and efficiency were rated modest, resulting in an unsatisfactory rating. a. Outcome Rating Unsatisfactory 7. Risk to Development Outcome The project generated limited intermediate outcomes, which include (i) the institutional and organizational strengthening of the BSL, (ii) the modernization of the payment system infrastructure, and (iii) the implementation of an HR and Information Technology (IT) strategies. Overall, there are no immediate risks that could jeopardize the achieved outcomes, and the ICR did not raise any. On the contrary, the achieved outputs and outcomes might lead to sustained reforms and modernization of the financial sector in the future, especially if there is sufficient support to understand what limited progress in this project and what still needs to occur to embed the reforms. 8. Assessment of Bank Performance a. Quality-at-Entry Project preparation relied on thorough diagnostics of the country's finance sector, and involved the country's key stakeholders and development partners (ICR, p. 23). The key challenge during project preparation was the weakness of local technical capacity in the finance and banking sector, prompting the project appraisal to target many activities addressing key technical issues. The project design ended up being too complex and overambitious, given the retention of several project components in a post-conflict context, rife with risks and technical capacity weaknesses. The Quality Enhancing Review (QER) recommended a reduction in the number of project activities, but this was not followed through. Identification of project risks was partial, with only the implementation risk identified as the key obstacle. As the risks identification and mitigation were incomplete, the project was set to experience implementation obstacles. Moreover, implementing arrangements were not well thought-out, and project coordination mechanisms and leadership never fully materialized, weakening project coordination and implementation capabilities (ICR, p.23). Page 11 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Financial Sector Support TA Project (P121514) Quality-at-Entry Rating Moderately Unsatisfactory b. Quality of supervision Five Task Team Leaders (TTLs) were involved in the project's design and implementation, and the borrower noted that the TTLs rotation presented a challenge for project supervision. TTLs fielded regular missions, maintained frequent contact with the BSL, and carried out extensive due diligence regarding all aspects of project implementation. The project's governance structures never fully materialized, stripping the project of key project coordination and implementation mechanisms, and requiring close supervision by the project team to get results on the ground. In hindsight, the project could have benefited from additional supervision missions and should have engaged the Country Management Unit (CMU) more to get support in addressing issues like delays on procurement. More specifically, the TTLs should have opposed adding new activities during the restructuring phase, as there was no sufficient progress in the implementation of the original activities. Overall, supervision was sustained, but it did not correct design shortcomings, and could not overcome the implementation challenges on the ground. Quality of Supervision Rating Moderately Unsatisfactory Overall Bank Performance Rating Moderately Unsatisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The Annex 1 of the PAD was devoted to the project results framework, but there was no consistent theory of change by which performance indicators accurately captured the progress towards the PDOs. The results framework had the following major shortcomings: (i) some indicators (for instance the compliance with Basel Core Principles (BCPs) and increase in MSME finance) were dependent on activities supported by other donors; (ii) there was no articulated connection between the PDO indicators, the multiple intermediate results, and the project's objective; (iii) the indicators were not adequate to assess the progress on the PDO's achievement; and finally (iv) activities related to strengthening the capacity of BSL were not captured in the Results Framework. Two examples to illustrate the disconnect between the expected outcomes and the results framework: (i) toward strengthening the BSL, having in place an efficient payment system was critical for the Central Bank and the financial sector as a whole, but there was no specific outcome indicator to gauge expected achievement, (ii) similarly, there were many ways to measure the progress in financial intermediation, but none among the three PDO indicators was specific enough to monitor progress to that effect. Page 12 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Financial Sector Support TA Project (P121514) b. M&E Implementation M&E data was collected throughout the project's duration and accurately captured the progress of project implementation, as all 11 ISRs provided the status of performance indicators, despite the weakness and changes of the result framework. A mid-term review took place in 2015, and led to a project restructuring in October 2016. The restructuring brought in new activities supporting the PDO and a new results framework, but the linkages between intermediate results, PDO indicators, and the project objective remained unclear. Additionally, given that the project ultimately did not support the development of a credit registry or bad debtor base nor a collateral registry, these two PDO indicators should have been dropped. c. M&E Utilization The M&E section in the ISRs reported regularly on the obstacles to project implementation, and on the status of performance indicators. The analysis of the ISRs on project implementation informed all changes adopted during the 2016 project restructuring. Overall, the M&E information process was instrumental in supporting project implementation, but the weaknesses in the PIU, the country's fragile context and the multiple rotation of TTLs could not help to turn the project around. M&E Quality Rating Modest 10. Other Issues a. Safeguards The project was classified as a Category C and no safeguards policies were triggered. This was a TA project with no potential social or environmental risks. b. Fiduciary Compliance Financial Management: The ICR provided little information on financial compliance; it only indicated that critical fiduciary challenges delayed payments to project consultants, and that the BSL faced challenges in implementing the project. A request for additional information related to the audits of resources use did not generate any response from the project team. Procurement: The PAD correctly noted that the BSL did not have the required capacity to manage procurement of the project. Procurement remained a challenge during project implementation despite training, Page 13 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Financial Sector Support TA Project (P121514) and significantly failed to attract qualified consultants. More specifically, the selected firm to undertake the financial inclusion survey was unqualified, resulting in the spending of scarce resources without the publication of a usable survey. c. Unintended impacts (Positive or Negative) The ICR did not identify any unintended impact. d. Other The ICR did not identify any other issues related to the project. 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment The ICR rating in the Moderately completion report is also Outcome Unsatisfactory Unsatisfactory Unsatisfactory, effectively indicating no disconnect. Due to significant shortcomings, quality at entry was rated Moderately Moderately Moderately Unsatisfactory. Quality of Bank Performance Satisfactory Unsatisfactory supervision also had significant shortcomings and was rated Moderately Unsatisfactory. Quality of M&E Modest Modest --- Quality of ICR Substantial --- 12. Lessons The ICR identified learned lessons which are summarized and rephrased below: (i) An effective governance structure on the side of the borrower is key to smooth project implementation: The lack of an FSDP governance structure and frequent changes in the BSL leadership affected project implementation. For a project with components across multiple BSL departments and multiple financial sector stakeholders, there need to be a Financial Sector Steering Committee (FSSC) with direct communication to decision makers, who could provide timely direction during project design and Page 14 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Financial Sector Support TA Project (P121514) implementation. The absence of such an oversight structure may severely affect the quality of project implementation. (ii) Prioritization and sequencing of reforms may enhance the potential for effective implementation in fragile environments: The project was ambitious and focused on a wide set of objectives. Despite the country’s vast needs in the financial sector, support should have been focused on priorities for which the World Bank has the relevant expertise and the Government ownership is stronger. (iii) When the PIU is weak, it is risky to count on the capacity building planned in the context of the project itself: The project planned to build capacity within the scope of the project itself and this resulted in limited upgrade of the PIU's ability to execute the project. (iv) Staff working in fragile states require sufficient budget and time to effetively support the supervision of World Bank projects: The implementation of this project illustrates the need for the World Bank and clients to consider a broader set of instruments to accompany an Investment Project Financing (IPF) in accomplishing financial sector reform and modernization. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provides a concise account of the performance of a small project implemented in a post-conflict environment, and is commended to reconstruct a theory of change of the project after the fact. However, the ICR preparation ran into a couple of difficulties: (i) an undue attempt to split the project's PDOs resulted in the assessment of multiple sub-objectives not supported by appropriate indicators to gauge efficacy; (ii) two key annexes supporting project's efficacy- the Results framework and outputs by component- were not comprehensive, and did not provide the needed material to assess project's performance, and (iii) the ICR did not cover the fiduciary oversight in a comprehensive way. However, lessons learned were derived from the project implementation and were adequate. The Annexes 1 and 3 on results framework and outputs by component were updated by the project team and provided upon request. The report made a solid effort to establish a candid overview of the project in a data constrained circumstance. a. Quality of ICR Rating Substantial Page 15 of 15