Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review First ICT Sector Development Operation(P128013) Report Number : ICRR0020674 1. Project Data Operation ID Operation Name P128013 First ICT Sector Development Operation Country Practice Area(Lead) Marshall Islands Transport & ICT L/C/TF Number(s) Closing Date (Original) Total Financing (USD) IDA-H8380 31-Dec-2013 3,000,000.00 Bank Approval Date Closing Date (Actual) 19-Mar-2013 31-Dec-2013 IBRD/IDA (USD) Co-financing (USD) Original Commitment 3,000,000.00 0.00 Revised Commitment 3,000,000.00 0.00 Actual 3,029,260.00 0.00 Prepared by Reviewed by ICR Review Coordinator Group Peter Nigel Freeman George T. K. Pitman Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Policy Areas a. Objectives The Program was to increase the availability of telecommunications services and enable the more widespread application of ICT services supporting improvements in economic and social development in the Marshall Islands. It was to be implemented through three consecutive operations; only the first operation was completed. The objective of this first operation stated in the Program Document (page iii) was to support the transition from a monopolistic to a liberalized telecommunications market. Page 1 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review First ICT Sector Development Operation(P128013) The objective of the program is not specifically stated in the Financing Agreement and so the development objectives given in the Project Paper are used in this review. b. Were the program objectives/key associated outcome targets revised during implementation of the series? --- c. Pillars/Policy Areas There were three policy areas: The first policy area focused on introducing a pro-competitive ICT sector policy. The new National ICT Sector Policy established the Government’s commitment to introducing a competitive market, while also recognizing that the small market size and challenging business environment in the Republic of the Marshall Islands (RMI) may mean that new entries do not necessarily materialize, and that the National Telecommunications Authority (NTA) may continue as a sole provider to be monitored and supervised by an independent regulator. The second policy area was focused on strengthening the legal and regulatory framework. The existing Marshall Islands National Telecommunications Authority Act 1990 did not (and still does not) allow for the introduction of competition, and reserves certain monopoly rights over telecommunications services in favor of NTA. The third policy area focused on restructuring and liberalizing the ICT Sector. The sustainability of the reforms depended upon restructuring and strengthening NTA to compete effectively in a liberalized market, which included providing the fiscal support needed for the government to carry out a restructuring of NTA’s debt and to finance a recapitalization. d. Comments on Program Cost, Financing, and Dates Program Cost: The program cost was to be $13 million in three operations. The first operation was for $3 million and this was fully disbursed. The second and third operations were cancelled in May 2016. Financing: The lending instrument was a Development Program Loan. Dates: The project was approved on March 19, 2013, became effective on June 3, 2013, and closed as expected on December 31, 2013. 3. Relevance of Objectives & Design a. Relevance of Objectives The Republic of the Marshall Islands (RMI) is one of the least “connected” countries in the world. It has a single service provider, the majority state-owned National Telecommunication Authority (NTA). Survey data Page 2 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review First ICT Sector Development Operation(P128013) from 2013 indicated that only around 35 percent of the population subscribed to ICT services. Mobile phone penetration was around 26 percent of the population. Less than two percent of the population subscribed to an Internet connection. Mobile broadband was not yet available. Total broadband Internet take-up was approximately 520 subscribers, or around one percent penetration of the population. Information flows were poorly coordinated and the stakeholder engagement process was not managed satisfactorily within the Government. Key reasons for the limited and costly service included the high costs of connecting remote and sparsely populated islands, the monopolistic market structure, and the constrained financial position of the Marshalll Islands NTA, which limited access to funds for new infrastructure. However, ICT sector reforms had been implemented in similar countries in the Pacific such as Fiji, Samoa, the Solomon Islands and Vanuatu and demonstrated linkages between market-based reforms to attract new private sector investment and improved economic and social indicators.. NTA presented a significant fiscal risk for the Government. The RMI section of the HANTRU-1 submarine communications cable system was installed at a cost of $21.5 million. It was financed by a loan of $18.5 million from the United States Department of Agriculture’s Rural Utilities Service to NTA, unconditionally guaranteed by the Government of RMI. However, NTA’s financial situation has deteriorated due in part to the burden of loan repayments. The company was generating insufficient cash flow from its operations to meet its repayment obligations without additional financial support. The Government was making annual transfers to NTA of approximately $1.5 million. An analysis of its financial position determined that NTA was unable to undertake the investments in new infrastructure that were required to improve access and services in line with demand. External debt at appraisal was around 67 percent of GDP, mostly in the form of concessional Asian Development Bank loans and Government guaranteed debts in the state owned enterprises sector. Consequently, a new legal and regulatory framework was needed to attract new investment. The nation’s remoteness and small size were recognized as significant challenges to attracting foreign investment, and the Government agreed that it would need to undertake policy and legislative reforms to improve the investment climate and to attract new investment into the ICT sector. The Program’s objectives were aligned with the Bank’s Country Partnership Strategy for the Marshall Islands FY13-16, which remain applicable. The objectives were closely aligned with the Government’s priorities under its Vision 2018 National Development Plan, and also reflected the urgent needs to address the fiscal risks posed by NTA and establish a more efficient ICT sector. Rating Substantial b. Relevance of Design The Program recognized and built upon the extensive experience of the Bank in the field of telecommunications policy, regulatory reform and rural access issues. Program design followed established good practice principles regarding the structuring, sequencing, and implementation of ICT sector reform. All prior actions and triggers were consistent with the development objectives. The Program followed a logical sequence, beginning with the basic policy and regulatory foundations for ICT sector reform, the enactment of Page 3 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review First ICT Sector Development Operation(P128013) key enabling legislation, the establishment of the new sector regulator, the adoption of a plan for restructuring of NTA, the issuing of new licenses and market entry, the commencement of regulatory functions, and the further strengthening of the ICT sector’s enabling environment, with a particular emphasis on facilitating access in the Outer Islands. The identification of the need to strengthen and reposition NTA financially, especially its debt obligations, prior to the liberalization of the market and the introduction of competition, also followed good international practice. Specific measures to prepare NTA for the introduction of competition were deemed important to build a consensus and manage the risk of opposition to the reforms. However, while the design may have been sensible in theory, for a first Bank engagement in the Marshall Islands in a fragile environment and where there was no prior experience, the timeframe to deliver the policy and institutional reforms was overly ambitious. The pace of reform under the Program caused anxiety amongst NTA and other stakeholders according to the ICR (page 13), which ultimately slowed progress as the Government’s attention was diverted towards opposition to the reforms. A program with a less ambitious series of deliverables may have been more appropriate and may also have helped to avoid suggestions that the sector restructuring and liberalization program had been predetermined or stipulated by the Bank. The Program's monitoring and evaluation framework also had serious weaknesses (see section 9). The pairing of the program with a parallel technical assistance (TA) project was a proactive design feature identified during appraisal. With the benfit of hindsight, the program's start might have been postponed until the TA project could be implemented and produce results. Such a deferral, however, would not have been feasible, given the Government's urgent needs for budget support and the pressure to commit IDA funds in FY13 that was the last year of IDA 16. Rating Modest 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective The first DPO policy area was to introduce a pro-competitive ICT policy Rationale The baseline value at appraisal identified no policy outlining the Government’s vision for sector development. However, the Government formally endorsed a new ICT sector policy through a Cabinet Resolution and the publication of this in the official Government journal committed it to liberalizing the ICT sector and to restructuring NTA. This said, the Government never completed the underlying analytical studies needed to provide a foundation for sector reform. Information flows were poorly coordinated and stakeholder concerns were not dealt with satisfactorily leading to a lack of trust in supporting the enabling legislation, which was not passed. Page 4 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review First ICT Sector Development Operation(P128013) Rating Negligible PHREVDELTBL PHEFFICACYTBL Objective 2 Objective The second DPO policy area was to strengthen the legal and regulatory framework. Rationale The baseline value at appraisal identified an inadequate ICT sector legal and regulatory framework that prohibited competition and provided inadequate mechanisms for oversight of service outcomes. Supporting prior action two, the Government prepared a draft Communication Bill consistent with the new ICT policy framework, and the Ministers of Finance and of Transportation and Communications authorized the joint introduction of the Communications Bill 2012 to the Parliament. International expert advisors were to be appointed. However, such advisors were never retained and the Bill never completed its Third Reading. In May 2016, the new Government reconfirmed its commitment to ICT sector reform, but requested a deferral of the Second ICT Sector DPO, at which point the Bank dropped the operation, terminating the Program. Rating Negligible PHREVDELTBL PHEFFICACYTBL Objective 3 Objective The third DPO policy area was to restructure and liberalize the ICT sector. Rationale The DPO Program did not increase the availability of ICT services and enable the more widespread application of such services. The Government did not implement any of the subsequent, more challenging, and substantive policy and institutional actions called for in the Program’s Policy Matrix. The Communications Bill was not enacted. Consequential to the completion of the law reform process, an independent ICT sector regulator was not established. Spectrum allocations for new wireless mobile services were not carried out, and no new operators were licensed. A restructuring plan for NTA was not prepared or approved. The burden and fiscal risk posed by NTA to the government remains unaddressed. No steps were carried out regarding open access to international connectivity. The limited development of the ICT sector due to the failure to implement the reforms is also illustrated by the lack of improvement in sector indicators. Mobile penetration in 2015 was 32 percent of the population, which represents only three Page 5 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review First ICT Sector Development Operation(P128013) percent annual growth since 2013, which is significantly lower than the reported annual growth rate of 12.5 percent for the Pacific region between 2009 and 2014. Mobile broadband is also yet to be introduced. Rating Negligible PHREVDELTBL PHREVISEDTBL 5. Outcome The development objectives were not achieved. The Program's key milestones were not achieved relating to the enactment of the new legislation introducing competition, the establishment of an independent ICT regulator, Government approval of a restructuring plan for NTA, and the offering of new licenses for a second operator. The second and third operations were cancelled in May 2016, when the program was terminated for lack of progress with reform. a. Outcome Rating Unsatisfactory 6. Rationale for Risk to Development Outcome Rating The risks that the development outcomes will not be maintained or realized are rated as High, recognizing that the Program’s policy and institutional actions were not implemented and outcomes were not achieved. Without the Government’s renewed commitment to the objectives under the Program, specifically to examine the fiscal risk posed to the Government by NTA debt and the potential to introduce legal reforms to promote private sector led investments in new infrastructure and improved services, there would have been no basis to continue with the implementation of the ICT TA Project. The new Government realizes that it needs to strengthen the financial performance of NTA, promote its financial sustainability, and reduce the call on Government funds for activities that could be financed commercially. a. Risk to Development Outcome Rating High 7. Assessment of Bank Performance Page 6 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review First ICT Sector Development Operation(P128013) a. Quality-at-Entry At the request of the Government, in 2010/2011 the World Bank provided technical assistance (TA) in relation to formulating development options and a strategy for the ICT sector. This included an assessment of existing network infrastructure, a supply and demand analysis, and an initial review of the current ICT policy and legal environment. The Program team utilized some of the extensive experience of the Bank in the field of telecommunications policy, regulatory reform and rural access issues. The appraisal identified the role that the Bank had performed in providing advice to other governments in the Pacific and globally on ICT issues. The proposed Program was also designed to draw upon lessons learned from the implementation of similar reforms in the Pacific region and in other comparable countries and was consistent with the Country Partnership Strategy for the Marshall Islands. However, it was identified that the capacity of the Ministry of Transport and Telecommunications to lead an engagement on ICT sector policy was highly constrained and would need to be enhanced. A joint Bank-Fund debt sustainability analysis was not available for RMI at the time of appraisal. However, an informal Bank assessment carried out using the joint framework showed RMI exceeding several policy thresholds under baseline and stress-test scenarios. At the time of appraisal, the budget support associated with the proposed program was expected to assist the RMI to move towards long- term fiscal sustainability. The pairing of the Program with this TA Project was a positive design feature that was attuned to the capacity issues in RMI, which were anticipated during appraisal. With the benefit of hindsight, the Program’s start should have been postponed until the TA Project could be implemented and produce results. Such a deferral, however, while ideal in principle, probably would not have been feasible, given the Government's urgent needs for budget support and the pressure to commit IDA funds in FY13 (the last year of IDA16). The First DPO might also have included a prior action specifying Cabinet consideration and endorsement of a report on market restructuring options for NTA, which would have helped to ensure the DPO Program was implemented with the benefit of expert technical assistance. The up-front presence of recipient executed technical advisors to provide independent guidance on policy and legal reforms would also have helped to counter suggestions that the Bank was forcing the reforms. In engaging in ICT sector reform in RMI, the Bank faced an extremely challenging situation and took a substantial risk in supporting the preparation and implementation of the program at the urging of the Borrower. The Bank, however, did not have sufficient understanding of the political and governmental context, which emphasized shared decision-making. A better understanding of the interests supporting the incumbent monopoly stakeholders would have allowed the Bank to design the program in a more realistic way or perhaps not to have undertaken a PDO operation at that time. Quality-at-Entry Rating Moderately Unsatisfactory b. Quality of supervision Implementation was not well managed. The Bank should have done much more in the first year of the DPO Page 7 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review First ICT Sector Development Operation(P128013) Program to support the Government to get the consultant advisers mobilized and have them produce an initial scheme for restructuring NTA. There were discussions to help improve transparency and the possibility of dividing the proposed third DPO into two operations, and to restructure NTS's debt, but these suggestions were not taken up due to the Government's lack of readiness. The Bank also carried out a review of the TA project's financial management arrangements, which had been performing unsatisfactorily due to very poor contract management and lack of oversight. Despite all these endeavors progress was extremely slow. A new Government was formed in early 2016, and the Bank sought guidance regarding the allocation of IDA 17 resources for the DPO Program and its commitment to using the TA Project. In its engagement with the Government, the Bank recommended that the DPO be dropped from delivery under IDA 17 and that these funds be reprogrammed. The Bank also emphasized that the Government could continue to use the TA Project to support work on telecommunications sector reform, but that strong commitment would need to be demonstrated before the Bank would consider an extension of the TA Project’s closing date of August 31, 2016. Quality of Supervision Rating Unsatisfactory Overall Bank Performance Rating Unsatisfactory 8. Assessment of Borrower Performance a. Government Performance The Government initially indicated that the Communications Bill (No. 44), dealing with sector liberalization and regulatory reform, would be introduced to parliament and would receive its first reading in March 2014. However, it adjourned on March 31, 2014 without the Bill having been tabled. An indicative Board date of May 29, 2014, for the Second DPO was therefore dropped due to the lack of progress in satisfying this key prior action. The Bill reached Second Reading stage in February 2015. In March 2015, during an interim Bank supervision mission, the Government also confirmed its intention to expedite the recruitment of international expert advisers to provide essential support for the law making process, including consultation and engagement with stakeholders and NTA as part of the Select Committee processes. These advisors, however, were never retained and the Bill never completed its Third Reading due to opposition in parliament. In May 2016, the new Government reconfirmed its commitment to ICT sector reform but requested a deferral of the Second ICT Sector DPO, at which point the Bank dropped the operation, terminating the Program. The new Government communicated to the Bank its intention to examine options for addressing the fiscal risk posed by its guarantee of NTA debt. It also indicated its willingness to examine new market arrangements to promote private sector led investments in new infrastructure and improved services. It requested the postponement of the DPO Program until the work under the TA Project was further advanced and indicated that it would review IDA 18 programming in light of the strategic options and recommendations for ICT sector reform and development that would be identified under the TA Project, Page 8 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review First ICT Sector Development Operation(P128013) including funding requirements needed to implement any reforms. Government Performance Rating Unsatisfactory b. Implementing Agency Performance The MTC and NTA did not interact in an effective or coordinated way with each other or with the Ministry of Finance (ICR, page 21). The implementing agencies, through the Government, repeatedly requested additional time to overcome severe capacity constraints that impeded the commencement of the analytical work required to support the Program. The agencies attempted unsuccessfully to complete the procurement processes for the selection of advisors, financed under the TA Project, on two separate occasions. In 2013 there was an extended delay in completing the request for proposal stage that led to the procurement process becoming stale. This process was restarted in August 2014, but formally abandoned again in March 2016 until the Government could determine whether to restructure and extend the TA Project. The inability of the Government to complete one international recruitment during three years of implementation of the TA Project, from August 2013 until August 2016, despite extensive technical support from the Bank, including extensive assistance of a procurement specialist deployed by the Bank during 2015 to help build capacity and provide in-country training on procurement procedures and processes, highlights the deep capacity limitations of the implementing agency encountered during the period of this Program. Implementing Agency Performance Rating Unsatisfactory Overall Borrower Performance Rating Unsatisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The Program’s monitoring and evaluation had significant shortcomings of design, implementation, and use. The three end-of-program key indicators were adequate to measure the first key Program outcome of increased availability of ICT services, but only one of these indicators (increased number of licensees) had baselines and target values. b. M&E Implementation The provisions for monitoring and evaluation in the Program Document were vague, referring only in very general terms to monitoring by the Ministry of Finance and the Bank. The Ministry provided no formal reports on implementation progress or results. The Bank did not file Implementation Status and Results Reports during the life of the Program. No restructuring was carried out to address the gaps in the M&E Page 9 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review First ICT Sector Development Operation(P128013) framework. There are no indications that an M&E system for the Program was implemented or used. c. M&E Utilization None. M&E Quality Rating Negligible 10. Other Issues a. Environmental and Social Effects Not applicable to DPOs.http://operationsdashboard.worldbank.org/project/secure/sap/forms/icrr?projId=P128013&stage=com# icrreview-ratings b. Fiduciary Compliance The funds for the First DPO (US$3,029,260.00) were disbursed at effectiveness following completion of the prescribed prior actions. c. Unintended impacts (Positive or Negative) The Government has now established a unit within the Ministry of Finance to manage international development and aid programs. Policy engagement and coordination is also being reformed and enhanced including formal record keeping and project management including procurement. d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Unsatisfactory Unsatisfactory --- Page 10 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review First ICT Sector Development Operation(P128013) Risk to Development High High --- Outcome The Bank could have done Moderately Bank Performance Unsatisfactory more to resolve issues Unsatisfactory regarding the lack of capacity. Borrower Performance Unsatisfactory Unsatisfactory --- Quality of ICR High --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 12. Lessons 1. There was a lack of borrower readiness for a development policy operation in the ICT sector. The delays and political opposition to reforms in the ICT sector highlight the difficulties of such sector reforms and the challenges of opening monopolies to competition. A thorough political economy assessment is an essential tool to support ambitious reform programs. The interests of incumbent monopolies, especially private shareholders, management and workers, capacity issues, and government readiness all play a role. The robustness of the government’s consultation and decision making processes also need to be evaluated carefully, especially to avoid situations where a lack of information, or misinformation, may be used by opponents to undermine the credibility or desirability of reforms. Political economy issues or concerns may be aggravated if the Government’s decision-making processes are weak and consultation processes are ineffective, especially in countries where shared decision- making is favored. The Bank needs to undertake realistic assessments of the political economy situation and ensure that operations include actions to resolve likely areas of opposition or resistance 2 . Thorough risk identification and mitigation is essential in sector reform. The two key risk factors that caused the Program to fail—weak capacity of the public sector and opposition from the incumbent operator to sector reforms—lean against the use of the DPO instrument, with its basis of policy and institutional actions as conditionality. In this context, close implementation support or capacity building by the Bank might have added to misconceptions on the part of opponents that reforms were being forced by the Bank. In countries with high budget deficits and related macroeconomic weaknesses, DPOs can create incentives to commit to unattainable reforms in order to trigger the release of budget support, notwithstanding a lack of readiness or consensus among key stakeholders. 3 . Public sector capacity constraints are especially challenging to deal with in the context of a DPO. Sufficient capacity and sustained commitment are necessary to deliver on reforms. There needs to be strong linkages between the DPO and associated technical assistance. In this case the failure of the TA inter alia led to the failure of the program. Page 11 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review First ICT Sector Development Operation(P128013) 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR was well prepared and written and in accordance with the guidelines applicable to ICRs for DPOs. It was honest and candid about the failure of the Program and the reasons for its poor performance. It brings out some lessons based on a thoughtful analysis of events. a. Quality of ICR Rating High Page 12 of 12