Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review APEI Regional DPO (RI)(P146512) Report Number : ICRR0021696 1. Project Data Operation ID Operation Name P146512 APEI Regional DPO (RI) Country Practice Area(Lead) Africa Macroeconomics, Trade and Investment L/C/TF Number(s) Closing Date (Original) Total Financing (USD) IBRD-86030,IBRD-86040,IDA-57850 31-Mar-2017 20,317,392.89 Bank Approval Date Closing Date (Actual) 07-Apr-2016 31-Mar-2017 IBRD/IDA (USD) Co-financing (USD) Original Commitment 29,900,000.00 0.00 Revised Commitment 20,012,953.00 0.00 Actual 20,317,392.89 0.00 Prepared by Reviewed by ICR Review Coordinator Group Hjalte S. A. Sederlof Clay Wescott Malathi S. Jayawickrama IEGEC (Unit 1) 2. Project Objectives and Policy Areas a. Objectives The development objective of the Accelerated Program for Economic Integration (APEI) Regional Development Policy Operation (DPO) was to improve the policy environment for trade in APEI countries by: (i) removing barriers to trade in goods; (ii) promoting trade in services; and (iii) enhancing measures to facilitate trade (Program Document, PD, page 21). Page 1 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review APEI Regional DPO (RI)(P146512) b. Pillars/Policy Areas The operation had three pillars: Pillar 1: removing barriers to trade in goods by identifying non-tariff barriers (NTB), and streamlining or eliminating those NTBs that affected regional trade flows the most; Pillar 2: promoting trade in services by building on existing commitments under other Eastern and Southern Africa trade agreements for gradual services sector liberalization (Common Market for Eastern and Southern Africa, COMESA; and Southern African Development Community, SADC); and by facilitating cross- border movement of persons. Pillar 3: enhancing measures to facilitate trade by improving efficiency of customs and border management. c. Comments on Program Cost, Financing, and Dates The operation was originally a programmatic series of two DPOs, with participation open to all five members of the APEI (Malawi, Mauritius, Mozambique, Seychelles, and Zambia), but with participation conditional on meeting defined readiness criteria, introduced to mitigate macroeconomic risks. The readiness criteria were linked to readiness triggers for development policy lending. They included an adequate medium-term macroeconomic framework, transparency in the budget process, and a budget system that followed sound fiduciary processes. At the time of preparing the first operation, three countries met the readiness criteria – Mauritius, Mozambique, and the Seychelles. However, following approval of the first operation by the Board, Mozambique disclosed previously unreported state-backed loans, calling onto question the country’s macro-economic policy framework. As a result, the IDA Credit to Mozambique under the operation was not declared effective. The second operation was subsequently cancelled, as the only APEI member country to request it and at the same time meet the readiness criteria was Mauritius. Program cost and financing. Commitments under the first operation included two IBRD loans – one to Mauritius in the amount of US$14.9 million equivalent; and one to the Seychelles in the amount of US$5 million. An IDA Credit of US$10 million equivalent to Mozambique was never made effective. Dates. The operation was approved on April 7, 2016, and became effective on June 9, 2016 for Mauritius and Seychelles (ICR, page 14). The operation closed on March 31, 2017. Since Mauritius was the only country that met readiness criteria and chose to participate in the planned second DPO operation, preparation for the second operation was cancelled in February 2018. Page 2 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review APEI Regional DPO (RI)(P146512) 3. Relevance of Objectives & Design a. Relevance of Objectives The APEI’s objective to improve the policy environment for trade in the APEI countries aligns with Pillar II, i.e. Institutional Cooperation for Economic Integration of the Bank Group's 2008 Regional Integration Assistance Strategy for Sub-Saharan Africa (page 37). The operation’s objectives are relevant to APEI country priorities, as its three pillars coincide with three of the four main themes in the APEI reform program (the fourth theme deals with improving the business environment in each participating country). The operation chose to include only the first three themes because of their potential spillover effects into neighboring countries. The willingness of APEI countries to jointly address barriers to trade and learn from each other also aligns with World Bank Group priorities of advancing South-South peer-to-peer exchanges and offers scope for mutual learning. Rating High b. Relevance of Design The Bank had chosen a programmatic DPO to support regional reforms. The regional approach would help promote collective action and coordination among participating countries; similar approaches to address common challenges collectively have been used successfully in the Asia-Pacific region. A programmatic DPO was chosen to drive this effort, as sequencing over time of a heavy agenda of reform was more likely to bring about sustainable change. The first operation was designed to support immediate reforms, while the second one was to focus on policies with effects that were likely to be felt only over the medium term. The selected interventions (pillars) were likely, if successfully implemented, to open up regional trade. The results framework underlying that objective was clear, and in most instances meeting outcome targets that were monitorable and measurable could reasonably be expected to deliver the intended outcomes (see Section 9). Prior actions introduced regulations to reduce non-barriers to trade in goods; included specific commitments , to liberalize trade in services; and required institutional reforms to reduce delays in transit flows. The operation would nevertheless turn out to be overly ambitious. The use of a DPL approach with readiness criteria would severely limit participation in the operation - only two countries (out of five APEI members consistently met the criteria - and ultimately lead to the elimination of the second DPO (only one country, Mauritius, remained eligible and chose not to participate in that DPO. Design would also prove to be overly ambitious as the operation would provide only modest outcomes (Section 4). Page 3 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review APEI Regional DPO (RI)(P146512) During preparation of the operation, the macroeconomic policy framework of all three countries was deemed adequate, as well as sustainable over the medium term. All three had posted positive growth in recent years which was expected to be sustained; all had a positive outlook over the medium term, with inflation projected to slow down or remain stable at relatively low levels. The debt outlook for Mauritius and the Seychelles was benign, and would remain so; while Mozambique was facing moderate risk of debt distress that would dramatically worsen by project start and lead to cancellation of its participation. Rating Modest 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective To remove barriers to trade in goods Rationale The objective was to be achieved by increasing transparency regarding non-tariff barriers (NTB) to trade and streamlining or eliminating those NTBs that were affecting regional trade flows the most. The target was to eliminate a minimum of 5 NTBs. The target was met by Mauritius, where import and export permit requirements were removed on 20 products. However, there is no indication of the effects that had on easing trade. Neither Mozambique nor the Seychelles reported removal of any NTBs. Rating Modest PHREVDELTBL PHEFFICACYTBL Objective 2 Objective To promote trade in services Rationale The objective was to be achieved by supporting regulatory and institutional reforms for the liberalization of trade in selected services between APEI countries, building on existing regional agreements; and by facilitating the movement of professionals across borders by easing regulations on business visas and work Page 4 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review APEI Regional DPO (RI)(P146512) permits. As a result, the number of border crossings by business people was to increase by at least 5 percent. The priority sectors where barriers to trade in services were to be liberalized were communications, financial, transport, and tourism services. Prior actions consisted of offers by all three countries to reduce barriers to services submitted to SADC and COMESA, which were supposed to be followed by implementation through changes in legislation at the country level. In fact, no further action was undertaken beyond the prior actions. A Memorandum of Understanding on the movement of business people across borders was signed by Mauritius and Seychelles as a prior action. Results, as measured by subsequent increases in the number of business visitors from APEI and other countries, are available for those two countries: for Mauritius, there was an increase of 28 percent in the number of business people entering the country, but a 0.8 percent decline in business people entering from APEI countries. The former compares to an original target of a 5 percent increase; there is no information on the results expected for visitors from APEI countries. For Seychelles, there was a decrease of 4 percent in the number of business people entering the country; and a 9 percent increase in entry by business people from APEI countries. The Seychelles also had a target of a 5 percent increase for all business visitors, which was not met; while the target for business visitors from APEI countries was exceeded by some 5 percent. Rating Modest PHREVDELTBL PHEFFICACYTBL Objective 3 Objective To enhance measures to facilitate trade Rationale The objective was to be achieved through regulatory and institutional reforms that reduce clearance and dwell times at borders and clearing depots, notably by increasing transparency of procedures, introducing risk management (risk-based inspections), and harmonizing transport standards. Results were measured in terms of: (i) traders reporting improved access to comprehensive trade related information; and (ii) average clearance time for imports at key intra-APEI border posts. With the introduction of a single-window procedure, Mauritius was able to provide access to comprehensive information on border crossing requirements; and clear imports in 11.3 hours compared to a baseline of 12.4 hours. In the case of Seychelles, their foreign trade portal was not yet fully functional as a single-window operation; and clearance of imports took 97 hours compared to a target of 90 hours and a baseline of 97 hours. Page 5 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review APEI Regional DPO (RI)(P146512) Regarding risk based inspections, adjustments to the functioning of the customs management system in Mauritius, and the introduction of new processes in Seychelles, achieved the following outcomes measured in the reduction of consignments physically inspected by customs. In Mauritius, inspections declined to 10.3 percent from previous levels compared to a target of 10.8 percent and a baseline of 12 percent; in Seychelles, inspections increased to 38 percent, compared to a baseline of 29.4 percent and a target of 26.5 percent. Effective functioning of transit arrangements was relevant only to Mozambique as the major transit route with an extensive port system, as it pertained to improving transit links between Malawi/Zambia and ports in Mozambique. This initiative was part of the dropped second operation which, among other things, was intended to support the development of a consistent transit system, reduce transport costs and make ports in Mozambique more accessible to the landlocked countries. Rating Modest PHREVDELTBL PHREVISEDTBL 5. Outcome The relevance of objectives is rated high, as the operation supports both government priorities in opening up regional trade and Bank priorities in promoting open economies and South-South collaboration. Relevance of design is rated modest, reflecting an overly ambitious design and a programmatic DPO approach that would severely limit project impact. Efficacy is rated modest for all three objectives. The increase in border crossings by business people was only partially met. The target of increasing transparency of procedures and reducing NTBs and inspections was met only in Mauritius. a. Outcome Rating Unsatisfactory 6. Rationale for Risk to Development Outcome Rating According to the ICR, the APEI members remain committed to the organization and its objectives. Indications of this are the effort now underway to strengthen the APEI coordination mechanism; ongoing interactions with the Bank to identify a balanced mix of interventions to advance APEI’s objectives, based on the Annual Meeting discussions referred to in Section 7a. However, currently no follow-up operations are envisaged. Another positive factor could be the African Continental Free Trade Agreement that came into force May 30. All five Page 6 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review APEI Regional DPO (RI)(P146512) APEI countries have signed the agreement, although none have yet ratified it. Even then, countries will need to specify which tariff lines will be liberalized, the nature of rules of origin, how to address non-tariff barriers, and the extent of commitment to liberalize trade in services, a process that could take many years. (World Bank 2018. Supporting Africa's Transformation: Regional Integration and Cooperation Assistance Strategy for the Period FY18-FY23, page 11; and Tralac, 2019 https//www.tralac.org/resources/by-region/cfta.html). There was only modest achievement of development outcomes under the DPO, and the risk to this is judged modest. a. Risk to Development Outcome Rating Modest 7. Assessment of Bank Performance a. Quality-at-Entry The project was strategically relevant in supporting the policy environment for trade, and ultimately growth, sought by the APEI countries. It drew on prior regional studies and country specific work to inform design; and it complemented other Bank operations supporting trade-related reform (a regional transport project; a sub-regional trade project). At the same time, it took an ambitious and not altogether realistic approach to trade reform that ultimately would be only modestly successful as participating countries felt difficult decisions in moving forward with the reduction/elimination of barriers to trade. The instrument selected – a programmatic DPO – was not appropriate: the interventions were complex and the scope ambitious; moreover, the readiness criteria were beyond the reach of most of the potential beneficiaries - they would narrow the reach of the operation and ultimately derail the planned second operation. Implementation arrangements may also not have been commensurate with the demands of project design: the ICR (page 13) notes that implementation arrangements were “lean”, with a coordinating group that met irregularly making it difficult to monitor progress and maintain information flows. There is no evidence in the ICR that the team reviewed alternative modalities for achieving the PDO, nor that the team brought in lessons learned from regional approaches elsewhere and incorporated them in the design. There is no evidence that the Bank had ever before used a DPO successfully for the purpose, which would have called for a much stronger justification for the modality than provided in the PAD (pages 21-31). Quality-at-Entry Rating Unsatisfactory b. Quality of supervision Based on the ICR, supervision of the operation was light - a planned schedule of quarterly supervision missions was not maintained, and may have been unrealistic in view of the potential costs in terms of staff time and travel to the participating countries. Supervision (and more generally, project implementation) may have been further weakened by the irregular work program of the APEI coordinating working group - intended to meet monthly. Considering the complexity and scope of the operation, a more concentrated effort might have been warranted. Page 7 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review APEI Regional DPO (RI)(P146512) Quality of Supervision Rating Unsatisfactory Overall Bank Performance Rating Unsatisfactory 8. Assessment of Borrower Performance a. Government Performance While the participating governments claimed strong commitment to the APEI and its goals, the timetables set for meeting goals were not commensurate with varying country capacities to implement reform, nor are the implementation arrangements themselves sufficient to do so. The challenges would subsequently be recognized by participants: at an APEI member meeting during the 2017 Annual Meetings, participants called for "more flexible support that was not instrument specific (including a mix of technical assistance, investment lending and policy lending support) over a longer time frame" (ICR, page 13). The unsatisfactory outcome of the operation in part reflects this, albeit also a result of the Bank team’s strategic approach to proceed with a programmatic DPO. Government Performance Rating Unsatisfactory b. Implementing Agency Performance Not assessed Implementing Agency Performance Rating Not Rated Overall Borrower Performance Rating Unsatisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design M&E design included six results indicators to measure program performance, all aimed at providing information on changes in behavior or conditions. Three of the indicators had quantitative, measurable, targets linked to the DPO. The target on liberalization of trade in services had not been defined in the PAD. The indicator measuring access to trade-related information is vague, and the indicator Page 8 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review APEI Regional DPO (RI)(P146512) measuring increase in business traveler numbers is subject to other factors, and hard to link to measures supported by the DPO. The policy and results matrix was used as the primary monitoring tool for the operation. Monitoring was expected to draw on existing coordination mechanisms, at the country level national working groups, and for the operation as a whole on the coordination working group, which had been set up to meet on a monthly basis. b. M&E Implementation The monthly meetings of the coordinating working group that had been planned didn't take place, and the frequency of meetings even declined over time. This made it difficult to get updates from countries on progress. Actual monitoring was based on periodic discussions between the Bank team and client stakeholders, albeit timetables do not appear to have been rigorously followed. c. M&E Utilization With the irregular monitoring, the value of the information for adjusting activities appears to have declined, and with no follow-up operation envisaged, utilization became moot. M&E Quality Rating Negligible 10. Other Issues a. Environmental and Social Effects No safeguard policies were noted in the ICR. b. Fiduciary Compliance No fiduciary concerns were noted in the ICR. c. Unintended impacts (Positive or Negative) None Page 9 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review APEI Regional DPO (RI)(P146512) d. Other None 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment IEG notes that the design was Moderately not appropriate for the Outcome Unsatisfactory Unsatisfactory circumstances, resulting in modest outcomes. Risk to Development IEG notes recent initiatives to Substantial Modest Outcome move forward with APEI. IEG notes the design Moderately challenges at entry, and the Bank Performance Unsatisfactory Unsatisfactory light supervision effort during implementation. IEG notes strong commitment by APEI governments, but Moderately Borrower Performance Unsatisfactory weak follow-up in Unsatisfactory implementation, notably in program monitoring. Quality of ICR Modest --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 12. Lessons The following lessons are drawn from the ICR: Alternative design options for regional trade operations should be carefully reviewed. Alternative modalities should be carefully reviewed, and the reasons for the one selected carefully thought through, along with the reasons for rejecting alternatives. In the APEI operation, the conditions relating to country participation, notably their readiness in terms of macroeconomic adequacy, would come to The EU has been facing this challenge for years, and inequality between APEI members, notably both cyclical and structural differences, reduced the scope of the operation, and ultimately led to cancellation of the planned follow-up operation. In the Page 10 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review APEI Regional DPO (RI)(P146512) meantime, many of the potential gains that a strong economy such as Mauritius might have reaped from a one- country DPL, was lost, as noted in the modest achievements of the operation. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provides enough information to allow an assessment of the operation. On certain points, it could have been stronger – especially in weighing design options: a platform to facilitate regional dialogue was and continues to be necessary - but participants did not appear ready; a more thoughtful discussion of options seems justified. It is raised in paragraph 23 and 44 but they seem rather brief considering the largely unsuccessful outcome. Some discussion of the lessons to be drawn from a regional DPO would also have been helpful. a. Quality of ICR Rating Modest Page 11 of 11