Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MW-Financial Sector Technical Assistance (P122616) Report Number: ICRR0021704 1. Project Data Project ID Project Name P122616 MW-Financial Sector Technical Assistance Country Practice Area(Lead) Malawi Finance, Competitiveness and Innovation L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-48970 31-Aug-2016 26,280,041.80 Bank Approval Date Closing Date (Actual) 24-Mar-2011 29-Jun-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 28,200,000.00 0.00 Revised Commitment 28,200,000.00 0.00 Actual 26,280,041.80 0.00 Prepared by Reviewed by ICR Review Coordinator Group Nestor Ntungwanayo J. W. van Holst Christopher David Nelson IEGFP (Unit 3) Pellekaan 2. Project Objectives and Components DEVOBJ_TBL a. Objectives As per the Financing Agreement (FA) on page 5, the objective of the Financial Sector Technical Assistance Project ( FSTAP) was "to increase access to finance for the currently unbanked, but bankable, population of the Recipient". The PDO statement in the Project Appraisal Document (PAD) was identical, and was not revised during project implementation. Page 1 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MW-Financial Sector Technical Assistance (P122616) b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? No c. Will a split evaluation be undertaken? No d. Components The project had five components as detailed below. Annex 3 in the ICR showed the costs at appraisal based on the PAD, but the actual costs shown in Annex 3 were the revised costs at restructuring. The Bank task team provided IEG with the data on actual costs. Component 1: Financial Sector Regulation and Supervision (cost at appraisal of US$5.32 million, revised cost of US$5.66 million, and actual cost of about US$5.6 million). Activities identified under this component are detailed below: (a) Carrying out of a program of activities designed to enhance the Reserve Bank of Malawi (RBM) legal and regulatory framework for the financial sector, including, inter alia, carrying out of a diagnostic review of the existing regulations, development of a new regulatory framework for capital markets, carrying out of capacity assessments for the Microfinance Institutions (MFIs) and the Savings and Credit Cooperatives (SACCOS), development of a dispute resolution framework in the insurance industry between the insurer and the insured, and design of a regulatory and supervisory framework for the pension industry, and (b) Carrying out of a program of activities designed to strengthen the supervisory capacity of the RBM, including: (i) design of new or amended prudential reports for the financial sector; (ii) development of off-site analytical tools for banking, insurance, microfinance institutions, savings and credit cooperatives, pension funds and capital markets; (iii) development of macro-prudential analytical tools; and (iv) provision of goods required for the purpose. Revisions to Component 1: In the context of the October 2015 restructuring, new activities were identified under this component (see Medium Term Review (MTR) Aide-memoire Annex 2) using savings from the cancelled procurement of a new supervisory technology system, and the decision to not proceed with establishing the Financial Sector Deepening Trust (FSDT). Final activities were not identified in the restructuring paper, but were to be selected on a case by case basis from a list of eligible activities identified during the November-December 2014 MTR mission. Component 2: Financial Infrastructure (cost at appraisal of US$13.22 million, revised cost of US$13.68 million, and actual cost of about US$11.4 million). Activities identified under this component were as follows: (a) upgrading of the existing real time gross settlement (RTGS) system and acquisition, installation and commissioning of a new RTGS system, (b) design, development and implementation of a central payments system platform under which banks settle their accounts (National Switch), (c) establishment of a microfinance institution transaction processing hub, (d) development of securities settlement and non-bank agency networks, (e) carrying out of a program of activities aimed at enhancing consumer understanding of payment products and services, the functioning of Page 2 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MW-Financial Sector Technical Assistance (P122616) the credit reference bureau and the implications of the National Switch, and (f) provision of goods required for the purpose. Component 3: Financial Consumer Protection and Financial Literacy (cost at appraisal of US$3.02 million, revised cost of US$3.33 million, and actual cost of about US$2.6 million). Activities identified under this component were: (a) carrying out of a program of activities aimed at strengthening the legal and regulatory framework for protection of financial consumers, (b) carrying out of a program of activities designed to enhance institutional arrangements for consumer protection in all financial services, and (c) development of a consumer awareness and financial literacy program. Component 4: Ministry of Finance's Financial Sector Policy and Governance Capacity and Long- term Finance (cost at appraisal of US$3.39 million, revised cost of US$3.09 million, and actual cost of about US$3.2 million). The two categories of activities identified under this component are detailed below: (a) carrying out of a program of activities aimed at enhancing the capacity of the Ministry of Finance for formulation, implementation and monitoring of financial sector policies and regulations, and for overseeing the Recipient’s interventions in the financial sector, including: (i) the development of an ownership policy of state-owned financial institutions and programs; (ii) the publication and dissemination of ensuing policy work; and (iii) the provision of goods required for the purpose, and (b) development of a policy framework and a legal and institutional infrastructure for long-term finance, including, inter alia: (i) preparation of a long-term finance policy and regulatory framework; (ii) carrying out of a cost-benefit study on the maturity period of the Recipient’s debt; (iii) carrying out of an institutional review of the operations and the investment policies of pension funds and life insurance companies; (iv) carrying out of an assessment of the capacity of the capital markets and the Malawi Stock Exchange; (v) establishment of a framework for leasing; (vi) carrying out of a feasibility study for establishing a development bank; and (vii) undertaking of diagnostic studies on sustainability of a long-term finance guarantee scheme. Component 5: Implementation Support (cost at appraisal of US$3.26 million, revised cost of US$2.45 million, and actual cost of about US$2.9 million). Activities under this component were as follows: (a) carrying out a program of activities aimed at improving the capacity of the RBM for Project management, including establishment of the Project Implementation Unit, recruitment of staff for the PIU and provision of goods required for the purpose, and (b) carrying out a program of activities designed to enhance the RBM’s capacity in the administration of the Malawi Financial Sector Deepening Trust, such activities to include the recruitment and maintenance of a procurement specialist, a financial management specialist, a monitoring and evaluation specialist, a project coordinator and support staff. Revisions to Component 5: Following the October 2015 restructuring, the funds initially allocated to establishing the FSDT were reallocated to other activities (MTR Aide-memoire Annex 2) that will help achieving this goal. The final activities to be funded were not identified in the restructuring paper, but were to be selected during implementation from a list of eligible activities identified during the November- December 2014 MTR mission. Page 3 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MW-Financial Sector Technical Assistance (P122616) e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Cost: The appraisal estimate was US$28.21 million, and the actual project cost amounted to US$26.3 million. Financing: The project was funded by an IDA credit, with an appraisal amount of US$28.21 million, and an actual amount of US$26.3 million. Co-financing: Co-financing was expected from the Department of International Development (DFID) of the United Kingdom, but was later canceled because of internal policy changes and general public finance management concerns in Malawi. (ICR, para 64) Borrower Contribution: There was no planned borrower contribution, and none occurred during implementation. Dates and Restructuring: The project was approved on March 24, 2011, and made effective on March 7, 2011. The project was restructured twice: (i) on October 29, 2015 to incorporate changes in the results framework, components and cost, in the reallocation between disbursement categories, in the implementation schedule, and the credit closing date, and (ii) on August 28, 2017 to make additional changes in the results framework, the implementation schedule, the credit closing date, and the legal covenants. However, changes in the results framework were unclear. The project was closed on June 29, 2018, almost two years after the initial closing date. Changes in the components and their cost were mainly triggered by the decision to not proceed with establishing the Financial Sector Deepening Trust (FSDT), as the required co-financing to support the FSDT from the Department of International Development (DFID) was canceled. Restructuring documents did not provide a final description of revised components and their cost. This task was left to the Bank's team that finally selected from a list of eligible activities identified during the November-December 2014 MTR mission. Changes in the results framework aimed to strengthen the M&E system by adding a few new PDO level indicators (three for a total of six), and some intermediate indicators to better measure the project's broader impact beyond access to finance, including stability of the financial system and its financial payments infrastructure, and financial literacy. As the PDO was unchanged, and that the change in the results framework was a refinement aimed at expanding the number of indicators to better capture the project's impact without changing the project's ambition, this review therefore did not consider there was any value added from applying a split assessment of overall outcome. The project's efficacy was assessed on the basis of the revised and strengthened results framework. 3. Relevance of Objectives Rationale Page 4 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MW-Financial Sector Technical Assistance (P122616) The project objective was "to increase access to finance for the currently unbanked, but bankable, population of the Recipient". The relevance of objective is illustrated by its alignment with the Borrower's and World Bank's priorioties as described below. Alignment with the Borrower's priorities: The focus of the Malawi Growth and Development Strategy (MGDS III, 2017-2022) in the area of financial services aims to achieve financial inclusion, to be reflected by effective access by citizens to a range of quality financial services. The main goal of the strategy is to create a vibrant, efficient, stable and inclusive financial sector that will meet the financing needs of all in the country. Key outcomes of the strategy in the financial sector include the following: (i) an enhanced policy and regulatory framework for the financial sector, and (ii) an improved digital payments ecosystem in the financial service provision (MGDS III). All project activities were in support to the goal of the country's strategy, and in contribution to its outcomes. In response to the delicate financial situation of the country created by the 2013 public finance mismanagement (dubbed cash-gate), the project supported the completion of a pilot Government electronic fund transfer (EFT) program for civil servant salaries and pensions compliant with new financial control and reporting standards. Alignment with the World Bank's strategies:The PDO was in line with the latest the 2013 World Bank Country Assistance Strategy (CAS), specifically under its theme aimed at promoting sustainable, diversified, and inclusive growth. A 2017 CAS Performance and Learning Review specifically noted the importance of the project and how it continued “to help Malawi meet its financial inclusion targets by upgrading the national payments financial infrastructure". The project continued to remain relevant and flexible to accommodating the following three emerging priorities: (a) in the banking sector, Basel II implementation support and provision of a resident bank supervision advisor; (b) the move toward the use of electronic funds transfers (EFTs); and (c) transaction support to look at options for Government's divestiture from a number of financial institutions. There was a congruence between the country's priorities in the financial sector and the World Bank's strategy in that country as highlighted in the 2017 CAS Performance and Learning Review. More specifically, both the World Bank's strategy and the Government's priorities aimed to improve financial inclusion targets in Malawi, and this goal was at the core of the project's objective of increasing access to finance for the unbanked, but bankable population. The scope of the project activities was realistic, and most of them were implemented at project closure. Overall, the relevance of project objectives was rated high. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL Page 5 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MW-Financial Sector Technical Assistance (P122616) OBJECTIVE 1 Objective The project objective was "to increase access to finance for the currently unbanked, but bankable, population of the Recipient". Rationale Theory of change: The theory of change depicted in the ICR is sound, but could have been improved. There is a logical link between the set of activities supported by the project, and the identified intermediate outcomes. For instance, there is a strong linkage between the improvement of the country's financial infrastructure, and the completion of (i) modernizing the real time gross settlement system, (ii) designing and implementing the central switch for processing payments, (iii) improving processing efficiency for small value payments, and (iv) reviewing the adequacy of the legal framework. The theory of change had two serious weaknesses: (i) the link between the intermediate outcomes and the PDO was unclear (ICR, Figure 1, page 10), as it was not clear which intermediate outcomes resulted in the PDO and how, and (ii) the identified underlying assumptions were too broad and needed to be disaggregated to pinpoint the more specific conditions that had to be met for the PDO to be achieved. Two assumptions that could have been spelled out were: (i) the Government's commitment to stay the course with the continuation and the coordination of the reforms in the financial sector, and (ii) the importance of sustaining technical capacity both in the public and private sectors to keep up with the ongoing changes in the financial sector, both on the domestic and the international front. Attribution and Counterfactuals: The ICR did not discuss these issues, but this review concluded that the achieved outcomes were attributable to this project and the Government of Malawi. As the country's economy was going through a difficult financial period (including the 2013 massive public finance mismanagement, dubbed cash-gate scandal), the achieved outcomes could not have occurred without the role played by the World Bank through this project, and the Government's commitment and coordination of the modernization of the financial sector and its sector reform program. Increased access to finance revealed by the increase of the population gaining access to formal finance, and the explosion of payments transactions through the banking system could therefore not have occurred without the achievements completed in the context of this project. The results are summarized below according to the themes of the project's assistance. Outputs Toward Strengthening the Financial Sector Regulation and Supervision  The number of agency banking regulations that became operational was exceeded, reaching 468 against a target of 450, and a baseline of 0. An increase in agency banking agreements represents an increase in availability of banking services especially in areas where banking services were not available and, therefore, impacting positively on financial inclusion;  The Parliament passed a new Financial Crimes Bill (February 2017), and in May and June 2017, dissemination workshops on the new law were conducted, and guidelines on regulations of Islamic Finance were developed;  Regulations related to the 10 laws that were passed from March 2009 have either been issued or are at gazetting stage when the project closed; Page 6 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MW-Financial Sector Technical Assistance (P122616)  The RBM reviewed and finalized several directives/regulations on bank supervision for gazetting. As of the project closure date, a number of new directives and regulations had been gazetted or were in the process of being implemented;  The Bank Supervision Application (BSA) was enhanced to include non-bank financial institutions, the enhancement was completed in June 2017, and the financial stability reports are regularly published;  Growth strategies, regulatory and hedging instruments, and an action plan for bond market development were developed;  Transaction support was provided to evaluate options for the Government's divestiture from a number of financial institutions, resulting in the Government selling off the majority stake in all financial institutions it previously controlled;  Ownership regulations for statutory bodies in Malawi, state-owned enterprises (SOE) guidelines on overdraft limits, issuance of parastatal guarantees, indemnities, and letters of consent and comfort under implementation were promulgated;  The Financial Services Appeals Committee (FSAC) was operationalized to hear appeals by financial institutions on decisions made by the Registrar of financial institutions, and rules of procedure for the operations of the committee were also established;  Core functions of the Financial Sector Policy Unit (FSPU) were executed by appropriately trained staff. The FSPU has since been combined with the Government Pensions to form the Pensions and Financial Sector Policy Division headed by a Director;  Policy frameworks for provision of long term financing were developed; including (i) equity and debt/leasing financing, (ii) public private partnership (PPP) for infrastructure, and (iii) development of long term savings and investment institutions;  Detailed sectoral reviews were completed for pensions, credit guarantees and Islamic finance;  Support was provided to third-party bank diagnostic audits for assessing the overall health of the bank sector;  Automated tools for offsite surveillance and macro-prudential analysis were designed, acquired and implemented;  The Reserve Bank of Malawi (RBM) staff were trained in using analytical tools and new regulations, and in particular the enhanced Bank Supervision Application (BSA); moreover, the project procured 30 laptops on which the BSA application was installed to allow access to the BSA by staff at all times; and  Basel II implementation was supported. Toward Strengthening the Financial Payments Infrastructure  A new National Switch was procured, installed, and made operational in January 2015; and interoperability was achieved for automated teller machines (ATMs) and point-of sale (POS), and the MFI Hub was integrated on the national switch;  The Microfinance Transaction Hub was deployed and is operational, and data migration and verification was completed for 16 pilot institutions, with the system software installed, and user acceptance testing (UAT) training conducted at nine MFIs and SACCOs;  All commercial banks in Malawi are integrated into the National Switch (on ATMs), and five banks have been integrated on point of sale. Page 7 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MW-Financial Sector Technical Assistance (P122616)  The EFT roadmap was finalized and its implementation commenced. Contracts for the development of various interfaces to secure government payment streams have been signed and are being implemented;  The Real Time Gross Settlement (RTGS) system was strengthened, and the automated transfer system (ATS/CSD) implementation was completed and the system is now considered stabilized;  The Automated Trading System for the Malawi Stock Exchange (MSE) went live on June 7, 2018;  The integration of mobile money services was approved for the switch and the use cases include Person-to-Person (P2P), Bill Payment, Cash-pull from bank account, and Cash-push to bank account;  The hardware and the application services are ready for use. The data center at Globe Internet was selected for use and the application installed. The service-level agreements were finalized to cover three key relationships: hardware as a service (Haas) with Globe Internet, software as a service (Saas) with Fintech International, and service levels between participating financial institutions and the Hub Company. Seven MFIs and Savings and Credit Cooperatives (SACCOs) have also gone live;  The design of the Centralized National Pensions Database was completed;  Risk assessment was conducted to assess the move by the Government from checks to EFT and compatibility of public finance management system, including Integrated Financial Management Information System (IFMIS) to ATS;  Government payments: all vendors are on the ground with user acceptance testing under way as of the project closure date, and integration implementation commenced and currently in progress. The expected changes relate to funding, payments, sweeping (domestic), rejections and resubmissions (cancellations and reversals), reconciliations, payments security, coding, and reports. Toward Improving the Financial Consumer Protection and Financial Literacy  The Consumer Protection and Financial Literacy (CPFL) drafted the Financial Ombudsman Bill and the Financial Consumer Protection Bill for the Cabinet as a step towards the adoption of a Twin-Peak model to financial consumer protection;  The Alternative Dispute Resolution (ADR) Bill was drafted and stakeholder consultations were completed, and was submitted to the Ministry of Finance (MoF) for processing;  The financial Ombudsman Bill was drafted and reviewed by the stakeholders. Stakeholder consultative workshops on financial consumer protection directives were held. Also, a financial literacy survey was conducted, and the final report was released;  The legal and regulatory framework for consumer protection was prepared and presented to the Cabinet, and the third (final) priority regulations were drafted, including the market conduct risk-based supervisory guidelines; and  The RBM institutional capacity has been enhanced for consumer protection supervision. Except for staff employed in the two years prior to the ICR, 100% of staff in the consumer protection unit was trained. Toward Improving Financial Sector Policy, Governance Capacity, and Long-term Finance  The 2016-2020 National Financial Inclusion Strategy (NFIS) was launched in July 2017. The strategy identified six priority areas and specific approaches to increase financial inclusion: (i) expansion of the reach of digital payments; (ii) expansion of savings, especially through savings groups; (iii) targeted finance for micro, small, and medium enterprises (MSMEs) and farmers; (iv) niche insurance opportunities to reduce vulnerability; (v) effective consumer empowerment and Page 8 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MW-Financial Sector Technical Assistance (P122616) education; and (vi) national coordination of financial inclusion. The Government has included a chapter on access to finance in the MGDS III and since launched the Malawi Financial Sector Development Strategy (MFSDS). Outcomes Toward Increased Access to Finance  The target to increase the adult population that uses formal financial institutions was largely exceeded, reaching 50.4 percent, compared with a target of 40 percent and a baseline of 19 percent;  The target to increase the proportion of women within the adult population that is formally banked was also almost achieved, with an actual value of 38.5 percent, against a target of 40 percent, compared with a baseline of 17 percent;  There was an increased use of mobile payments from about 186,000 transactions in December 2012 to 4.5 million transactions in May 2018 (the month before project closed), and access to finance has significantly improved and people are able to perform financial transactions without going to banks. In particular, the rural population has benefited more from mobile technology. Toward Growth in Payments Transactions  The volume and value of transactions processed in point-of-sale (POS) were largely exceeded reaching 2,556,541 and Malawi Kwacha (MK) 101,907 million, against a revised target of 245,570 and MK 6,286 million, and a baseline of 183,248 and MK3,548 million respectively;  The volume and value of transactions processed through ATMs were exceeded, reaching 136,907,780 and MK 2,417 billion, against a target of 23,073,209 and MK0.302 billion, and a baseline of 17,217,584 and MK 0.170 billion respectively;  The volume and value of transactions processed through MNO-led mobile payments were largely exceeded, reaching 333,395,067 and MK 1,022 billion against a target of 11,717,526 and MK 158.28 billion, and a baseline of 698,419 and MK 1.40 billion respectively;  The volume and value of transactions processed of Bank-led mobile payments were largely exceeded, reaching 75,072,703 and MK 462,439 million against a target of 4,869,216 and MK 5,731 million, and a baseline of 3,633,484 and MK 3,235 million respectively;  Targets for the transactions volume and value processed by the Automated Transfer System (ATS) were exceeded, as the transactions increased by 488% in volume and by 1,465% in value. Volume and value of transactions increased from 4,349,952 and MK 5,011 billion to 21,220,986 and MK 73,415 billion respectively;  The target of the transactions volume processed by the National Switch was virtually met, reaching 5,390,276 against a target of 5,399,428, and the value of transactions reached MK 83.51 billion, with no target for the value of transactions set at approval;  The target for the transactions volume and value processed by the MFI Hub was fully achieved, as five non-banking financial institutions (MFI and SACCOs) are using the hub software at project closure, against target of five, and a baseline of 0. Toward Financial Consumer Protection and Financial Literacy Page 9 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MW-Financial Sector Technical Assistance (P122616)  The awareness of the RBM and its Consumer Protection Unit increased by 10 percentage points (74.2% to 84%) and 3 percentage points (12.5% to 16%) respectively. Despite increased awareness of Financial Consumer Protection institutions, 36.2% (in 2014) of consumers who had conflicts with their financial providers were not aware of the existence of these consumer protection agencies, in comparison to 44% in 2018;  The proportion of financially literate population was partially achieved, reaching an index value of 3.90 against a target of 4.80, and baseline of 0, based on an index that ranges from 0 (illiterate) to 7 (highly literate). The proportion of financially literate female population was also partially achieved, as the achieved index was 3.60, against a target of 4.80, and a baseline of 0. Rating Substantial OVERALL EFF TBL OLD Rationale Overall efficacy: Increased access to finance was significant, and outcome targets were for all practical purposes achieved. The growth in payments transactions exploded due to improvements in the financial sector regulation and supervision, and most particularly in the strengthening of the financial payments infrastructure. In the same vein, financial consumer protection and financial literacy improved significantly from a very low level at project approval. In all, there was a substantial increase in the access to finance by the previously unbanked country's population. Overall Efficacy Rating Substantial 5. Efficiency This was a technical assistance project, and no quantitative economic and financial analysis was undertaken at appraisal, nor in the ICR. Key findings from research that underpinned the project were as follows: (i) the potential of an efficient financial sector to accelerate economic growth and poverty reduction, (ii) a well- functioning financial system provides reliable and inexpensive payments services, makes available remunerative and safe deposit facilities and offers entrepreneurs access to a suitable range of sources for short- and long- term funds, and (iii) financial sector development is associated with beneficial distributional effects (PAD, p.19). To illustrate efficiency of project implementation, the ICR underscored the efficient use of human and financial resources during project implementation, and the timeliness in the implementation of the project's activities. Efficient use of project resources. The ICR indicated that preexisting entities with adequate human resources were allocated responsibilities based on their area of expertise and comparative advantage, and new ones were Page 10 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MW-Financial Sector Technical Assistance (P122616) deliberately created (for example the creation of the Consumer Protection and Financial Literacy under the component 3) and were provided the statutory mandate to execute specific activities of the project. For the most part, the exploitation of existing structures contributed to the project's cost-effectiveness. Financial resources were allocated to the most important activities toward achieving increased financial access for the "currently unbanked, but bankable, population" In Malawi. Credit funds were disbursed at project closure. Timeliness. The considerable training activities were centrally financed by the project, which served to cut overhead costs, and to a large extent the outputs were achieved on time. Digitization, which was one of the core project components was also efficiently implemented, resulting in the overall improvement of speed of transactions to the mutual satisfaction of service providers and consumers. However, because of the challenges facing the Government in handling the overall financial sector reform, the project's closing date was extended by two years, but this did not reflect any shortcomings in the efficiency with which the financial sector reform was being implemented. For these reasons, the project's efficiency is rated substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of objectives was high, based on the congruence between the country's priorities and the World Bank's strategies in promoting financial inclusion through an increased access to finance by a higher number of the country's population. Efficacy was substantial, since the project achieved a strengthened financial payments infrastructure, an increased access to finance, growth in payments transactions, an improved financial consumer protection and financial literacy. The project's implementation efficiency was substantial. The project had minor shortcomings, and therefore its overall outcome is rated satisfactory. Page 11 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MW-Financial Sector Technical Assistance (P122616) a. Outcome Rating Satisfactory 7. Risk to Development Outcome The project achieved significant progress in strengthening the financial payments infrastructure, in scaling up access to finance, and in launching the financial consumer protection and financial literacy. This occurred mainly because of strong commitment displayed the Government and the RBM management. As the regulatory framework of the financial sector is in place and embedded in a well-established institution (the RBM), there is a potential for the sophisticated infrastructure to increase financial institutions capability and to improve financial products and innovate, as well as building the capacity of stakeholders and end- users to encourage financial product uptake. These institutions benefited also from the existing maintenance policy at the RBM for critical infrastructure and information systems. Likewise, the ATS is housed at the RBM with adequate financial resources; its sustainability is therefore ensured. However, the ATS as well as the legal and regulatory framework will need to be upgraded regularly to keep up with market and technological developments. However, as the ICR notes (para 78) there are aspects that need follow up. The oversight of the financial market infrastructure handed to beneficiaries needs to be established. Regarding the achievements of the MFI Hub, the innovation is not likely to be pursued if the unit costs for joining and maintenance are high. Consequently, alternative financing already being considered by the authorities to enable more MFIs and SACCOs to adopt the concept needs to be continued and this should also include commercial financing options (ICR para 78). The integration of mobile money services into the National Switch significantly expands the support base for the sustainability of the platform and confirms the market relevance of the technical solution. The use cases integrated include person-to-person (P2P) transfers, bill payments, cash-pull from bank account, and cash- push to bank account. The National Switch) is likely to function using existing policies and handed over to beneficiaries, who will manage it as owners of the assets and the liabilities of the facilities provided. Finally, the ICR notes that the Government had requested for a successor World Bank project to FSTAP to enhance private enterprise growth and job creation in Malawi by increasing MSME access to financial services and improving their capabilities. A new project under preparation (P168577) aims to pursue the scaling up of financial inclusion and entrepreneurship, and is likely to be approved in FY20. Key activities envisioned include the following: (i) access to finance for entrepreneurs and MSMEs, (ii) entrepreneurship Page 12 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MW-Financial Sector Technical Assistance (P122616) development focusing on youth and women, (iii) financial infrastructure and regulatory environment, and (iv) financial education. This review agrees with the ICR that the risk to development outcome is moderate. 8. Assessment of Bank Performance a. Quality-at-Entry At appraisal, the World Bank team reviewed the relevant aspects of the project, including the macro- financial context, as well as the technical, institutional, and procurement implications of the project activities. Major risk factors and lessons learned from earlier projects were incorporated into the design. The project was well grounded in the realities of Malawi and its problems in the financial sector at appraisal (ICR para 74). The project objective was precisely stated, and the components were realistic, measurable and achievable, and identified implementation arrangements were adequate. However, there were two categories of weaknesses: (i) the results framework had shortcomings that were corrected during two restructuring operations (2015 and 2017), and (ii) the risk of withdrawal of partners was not identified, and when the risk materialized, there were no mitigation measures to continue with identified activities. The overall quality of the World Bank’s performance in ensuring quality at entry was Moderately Satisfactory. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision The ICR noted (para 76) that the Bank Task Team Leader (TTL), technical experts, financial management and procurement specialists, and consultants consistently engaged closely with Government counterparts. The task team responded promptly to the requests of the Government of Malawi. The World Bank’s technical and fiduciary teams provided regular support to focus on maximizing the project’s development impact, which resulted in adjustments, including project restructurings and extensions of the closing dates. Moreover, the task team conducted regular supervision missions, on average twice a year, to take stock of progress, and the implementation status and results reports (ISRs) were candid, detailed, and outlining important events and providing a complete picture of the progress. The supervision teams also produced clear and detailed Aide Memoires. Page 13 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MW-Financial Sector Technical Assistance (P122616) Furthermore, the ICR indicated (para 76) that three TTLs managed the project, including one co-TTL based in the local country office, who followed up on ground implementation progress, and the turnover of project TTLs was within norms of other projects in the portfolio that were approved at the same time (e.g. energy and mining projects). A special emphasis went into having in-country support to the PIU teams and beneficiary institutions. The regular supervision missions, meetings and technical guidance helped a lot to reach the project purpose. Changes in TTLs were undertaken carefully and field supervision was carried out with an adequate skills mix at critical project implementation periods to address problems identified. It is worth noting that the turnover in TTLs caused anxieties for the client even though this did not disrupt implementation. However, the ICR questioned the efficacy of the supervision missions, and argued that the latter should have done more in focusing on advising the PIU teams and beneficiary institutions on assessing the efficiency of innovations carried out by the project and on proactivity in preparing exit strategies for the shared platforms. While proactivity regarding future project efficiency and exit strategies were desirable, these issues were not at the core of the project components and will be addressed by a follow- on project as indicated under Section 7. Overall, quality of supervision was strikingly effective, because the project team, relying on a resident task team leader, succeeded to: (i) oversee the successful delivery of key project activities that generated the expected outcome, (ii) sustain a close dialogue with the Government, and (ii) provide timely support to the implementing agencies, all this in a context of fragility and weak public finance governance. Based on the above performance, the quality of supervision is rated Satisfactory. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The M&E design was anchored by key performance indicators and intermediate outcome indicators, as included in Annex 1 of the PAD. The PIU at the RBM was the responsible implementing agency for conducting M&E activities which regularly collected data and provided detailed reports for the overall M&E system with inputs from the designated focal points in the implementing units. These units were responsible for data collection and reporting for their respective component. In hindsight, while the results framework had moderate shortcomings, it was revised and strengthened by additional indicators during both restructurings to better capture project's theory of change. Page 14 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MW-Financial Sector Technical Assistance (P122616) b. M&E Implementation There were shortcomings in the M&E Framework which were later corrected during restructuring to better measure the PDO through the upgrading of some intermediate outcome indicators and new PDO-level indicators. Additional changes in the results framework were later introduced through project restructuring to make them (see Section 2-e) more realistic and able to capture achieved outcome in modernizing the financial sector and in promoting access to financial services. The Bank's task team regularly collected detailed progress data, updated progress against the baseline, and highlighted issues for the World Bank management and donor’s attention. Information provided included quarterly progress and financial management reports and annual audits of project accounts. The MTR took place during November/December 2014, and provided the needed guidance to (i) restructure the project and revise the results framework, (ii) reallocate resources towards reviewing the Malawi Financial Inclusion Strategy, following the impossibility of establishing the Financial Sector Deepening Trust, due to the cancellation of the DFID support prompted by the United Kingdom's internal policy changes and general public finance management concerns in Malawi (ICR, para 64). c. M&E Utilization The data collected from the progress reports, including the MTR report, were evaluated and used to inform the World Bank management, serving as a basis to provide guidance related to prioritization of activities and reallocation of resources. For example, restructurings and closing date extensions were based on inputs from the M&E and supervision mission findings. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was classified as a Category C, as the civil works described in Component 2 included the establishment of a Micro Finance processing hub and an interoperable switch. These works were minor and localized, and the national and local laws and regulations were sufficient to manage any social and environmental impacts. As a consequence, no environment safeguards were triggered. Environmental and social safeguards compliance was assured as the project did not involve any activities that affected environmental or social safeguards. b. Fiduciary Compliance Page 15 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MW-Financial Sector Technical Assistance (P122616) Financial management. The project management team within the RBM effectively complied with the Financing Agreement covenants. The quarterly interim financial reports (IFRs) and the annual audited financial statements were submitted on time for the World Bank’s consideration and were found acceptable. The FM arrangements facilitated smooth implementation of the project as required, including funds flow and reporting. The FM arrangements were rated Satisfactory throughout the implementation period and all audited financial statements had unqualified audit opinions. Nevertheless, there were disbursement delays in resolving matters related to the IFRs. Procurement. The project followed the procurement procedures stipulated in the Procurement Guidelines, the PAD, the Financing Agreement, and the Project Implementation Manual. There were no recorded cases of ‘mis-procurement’ or unresolved complaints from bidders. However, there was a slow start on implementation due to delays in procurement staffing at the beginning of the project and significant turnover of fiduciary staff. All these delays led to extension of the project closing date. Overall, fiduciary compliance was assured as internal control arrangements were in place and adequate FM, procurement, and disbursement systems were maintained. c. Unintended impacts (Positive or Negative) Because of the increased awareness on financial literacy created by the project, many secondary schools have incorporated the topic of financial literacy in their curriculum and teaching materials. d. Other The ICR did not raise any other issues. 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment As will be mentioned in Section 14 of this review, the ICR incorrectly rated the project's outcome Moderately Moderately Satisfactory. This review rates Outcome Satisfactory Satisfactory the project's outcome Satisfactory on the basis that relevance of objectives was high, and efficacy and efficiency were substantial. Moderately Bank Performance Satisfactory Satisfactory Page 16 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MW-Financial Sector Technical Assistance (P122616) Although there were moderate shortcomings in the design of Quality of M&E Modest Substantial the M&E system, they were corrected at both project restructurings. Quality of ICR --- Substantial 12. Lessons Lessons identified in the ICR are summarized and rephrased below: (i) Realism of activities and readiness for execution are the predictors of a successful project outcome. The high-quality technical studies that underpinned the project activities identified at appraisal, and the advance procurement of the technical consultants were key factors that helped expedite progress toward pursued project outcome and objective. For instance, project design drew heavily from the conclusions of the 2008 World Bank-IMF Financial Sector Assessment Program report. (ii) The PIU can be used to reinforce ownership of project achievements through end- user involvement throughout the project lifecycle. The RBM worked together with the PIU and facilitated stakeholder consultation and engagement on all activities related to financial market infrastructure upgrade, and ensured that stakeholder engagement was infused into the project throughout its lifecycle as an iterative process. The goal was to gather views of key stakeholders and arrive at common positions of the future design of the technical solutions, through a participatory and inclusive process. In this context, a mapping and identification of stakeholders was undertaken in the early stages of the project to obtain a complete picture of key sector players. (iii) Proactive actions by the implementing agency and the PIU teams can help strengthen local technical capacity. This is especially critical when defining the technical specifications of the technological solutions to be acquired as well as the commissioning of the solutions. Special emphasis went to ensuring continuity in participation by key stakeholders and actors throughout the project lifecycle and at critical project implementation periods to act on trends and other signals before they become issues. Retention of adequate skills mixes on the PIU teams and shared platform were key to building capacity and had implications on the implementation and monitoring of innovative activities. (iv) Sustainability of project results can be generated from deliberately investing in ongoing relationship building. When a project involves multiple stakeholders, it is critical that the implementation agency and PIU teams maintain good and productive relationship with all of them. There is a need to have upfront conversations on the nature of the technical solutions and business model and what is required from everyone to make it work and succeed. The development and implementation of shared infrastructures such as a national switch was a complex process requiring the active involvement of all stakeholders and supported by technical working groups that bring together both bank and non-bank financial sector actors to consider all project outputs and proposals for the target architecture of the infrastructure and operating business model. The PIU facilitated the transfer of good practices and sharing of lessons and experiences. Formal and Page 17 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MW-Financial Sector Technical Assistance (P122616) informal pathways were established for consistent communication to continually build the relationship among stakeholders. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provides a comprehensive and candid review of the project design, implementation and discussion of the achieved results. Data and information provided in the ICR and in the annexed Borrower's contribution were sufficient to illustrate what was achieved by the project. However, as acknowledged to IEG by the Bank task team, the split assessment of project's outcome resulted in an incorrect final rating (Moderately Satisfactory) of the project's overall outcome (see Annex 7 of the ICR). There were also shortcomings in Annexes 1 and 3. Annex 1 did not provide the "comments on the achievements compared to targets" for the various indicators, and Annex 3 included incorrect data which the Bank task team corrected for IEG. On the other hand the lessons provided in the ICR were sound, and reflected the experience of project implementation. a. Quality of ICR Rating Substantial Page 18 of 18