ICRR 13347 Report Number : ICRR13347 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 04/14/2010 PROJ ID : P073479 Appraisal Actual Project Name : Mozambique: US$M ): Project Costs (US$M): 15.40 16.90 Communication Sector Reform Project Country : Mozambique Loan/ US$M): Loan /Credit (US$M): 14.90 15.64 Sector Board : GIC Cofinancing (US$M ): US$M): 0 0 Sector (s): Telecommunications (40%) Aviation (36%) Postal services (24%) Theme (s): Infrastructure services for private sector development (50% - P) Regulation and competition policy (50% - P) L/C Number : C3577 Board Approval Date : 11/27/2001 Partners involved : Closing Date : 06/30/2006 06/30/2009 Evaluator : Panel Reviewer : Group Manager : Group : Keta Ruiz Roy Gilbert IEGSE ICR Reviews IEGSE 2. Project Objectives and Components: a. Objectives: The Project Development Objectives (PDO) are stated differently in the Development Credit Agreement (DCA) and in the Project Appraisal Document (PAD). This ICR Review uses the objective as written in the PAD to assess the project outcome as this is consistent with the project description and logical framework in project documentation (the PAD) and the PDO as stated in the DCA refers mostly to the means to achieve the PDO . The PDO according to the PAD are: "to improve access and quality of efficient and affordable communications services by creating an enabling environment for competition and private participation in key sectors deemed critical to facilitate national and regional market integration, i.e., telecommunications, postal and air transport infrastructure and services ." The PDO according to the DCA are "to assist the Borrower in: (i) reforming and restructuring its telecommunication, postal and air transport sectors to attract private management, know how and capital; (ii) developing an institutional and regulatory framework designed to promote competition and private investment in the telecommunication, postal and air transport sectors; and (iii) develop a strategy for increasing access to communication services in rural areas ." The Implementation Completion and Results Report (ICR) footnotes the discrepancy between the PDO in the DCA and the PAD and highlights that improved access through increased geographic service coverage and lower prices were the main objectives of the project, with the enabling environment and private sector participation being the means to it, as reflected in the Key Performance Indicators (KPIs) from the PAD (please see list of original KPIs below in M&E section). b.Were the project objectives/key associated outcome targets revised during implementation? Yes If yes, did the Board approve the revised objectives /key associated outcome targets? No c. Components (or Key Conditions in the case of DPLs, as appropriate): The following are the project components as listed in the PAD (and almost verbatim in the ICR). The actual component amounts are drawn from information in the ICR : Component 1: Telecommunications Reform (Appraisal : US$9 mill ion; Actual : US$9 US$ 9.70 million; US$ 9.8 million ) 1.1: Development of telecommunication policy, strengthening policy capacity and awarding two new cellular licenses to private operators (Appraisal: US$0.9 million; Actual: US$1.3 million) 1.2: Preparation of a comprehensive telecommunications and postal legal and regulatory framework including strengthening of the Telecommunication and Postal Regulatory Authority (INCM) (Appraisal: US$3.4 million; Actual: US$6.1 million) 1.3: Privatization of the Public Telecommunication Company (TDM) (Appraisal: US$1.2 million; Actual: US$0.7 million) 1.4: Strategy for market based universal access including establishment of a universal service fund and provision of seed money for rural operations (Appraisal: US$3.5 million; Actual: US$1.0 million) 1.5: Information and communications technology enabling legislation (Appraisal: US$0.5 million; Actual: US$0.3 million) 1.6: Development of postal sector policy and modernization plan (Appraisal: US$0.2 million; Actual: US$0.4 million) Component 2: Air transport reform (US$5 US$ 5.7 million ) US$ 5.5 million; Actual : US$5 2.1: Development of air transport policy and regulatory framework, and air transport policy unit strengthening (Appraisal: US$0.6 million; Actual: US$0.5 million) 2.2: Strengthening of the Civil Aviation Authority (INAC) regulatory capacity (Appraisal: US$3.3 million; Actual: US$3.4 million) 2.3: Restructuring and private participation scheme of the National Airline Company (LAM) (Appraisal: US$l.0 million; Actual: US$0.9 million) 2.4: Granting airport concessions (Appraisal: US$0.6 million; Actual: US$1.0 million) Component 3: Project management and implementation (Appraisal : US$0 US$ 1.3 million ) US$ 0.2; Actual : US$1 3.1: Strengthening project management, procurement and financial management capacity,including project audit (Appraisal: US$0.1) 3.2: Communications campaign, information and dissemination (Appraisal: US$0.1). The project components in the DCA are presented organized in a different way but are in essence the same, except that the DCA does not include Component 3: Project management and implementation. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The project went through three extensions in April 2006 (for 18-months), in October 2007 (for 5 months) and in March 2008 (for 13-months), and two disbursement category reallocations . The first extension was to make up for slow implementation and to allow for the completion of several activities that could not be completed before the original closing (only US$8.45 million of the loan was expected to be disbursed by the original closing date). The activities that required an extension to be completed included the implementation of the universal access program in the pilot region, installation of spectrum management and monitoring equipment and enhancement of safety oversight capacity . The second extension was a tentative extension until the results of the tender for the Universal Access, which was relaunched on October 11, 2007, were to be known (by December 2007). At that point, management noted that should the current tender be unsuccessful, the Bank will cancel the US$ 3 million Universal Access component from the project . The tender was partially successful and the project was extended for a third time. At the end, however, due to limited implementation of the Universal Access component, US$2.7 million were cancelled at credit closing . Borrower counterpart funding was increased from estimated US$ 0.5 million at appraisal to US$1.3 million used to fund additional project management as the project was extended for 3 years. 3. Relevance of Objectives & Design: Relevance of objectives (Substantial ): The PDOs ("to improve access and quality of efficient and affordable communications services i.e., telecommunications, postal and air transport infrastructure and services ") and second tier objectives ("of creating an enabling environment for competition and private participation in telecommunications, postal and air transport infrastructure and services ") continue to be relevant to current priorities . Access and quality of telecommunications continue to be important priorities according to the most recent CAS (2007). It mentions that despite accelerated growth in recent years, teledensity is still very low, and needs to continue to increase in order to enable growth, especially in rural areas . The 2007 CAS cites the GoM’s ambitious telecommunications targets which include (i) increasing teledensity to 22 percent by 2009, and (ii) expanding internet access points by 2009. The access and quality of air transport dimensions of the PDO are also in line with the 2007 CAS which states the important role of air transport in activities with high growth and export revenue earnings potential such as business travel, tourism, and high value exports . The second tier objectives continue to be priorities as the stalled privatization plans for LAM and concession plans for Maputo International Airport (MIA) and other airports are also mentioned in 2007 CAS. The objectives related to postal services are not in line with priorities of the 2007 CAS (no mention of postal services), but they seem to be still a priority for improvement . For example, the average number of letter -post items posted per inhabitant in 2007 was only 0.10 (source: Universal Postal Union ) significantly below the average of other African countries with similar income per capita level such as Ethiopia (0.14), Rwanda (0.19), and Togo (0.21). Relevance of design (Modest ): Project design was mostly relevant to achieve the PDOs although there were weaknesses in the results framework to allow proper monitoring of all dimensions of the PDOs, including missing baselines and targets (please see M&E section below). The project design although relevant at the time of appraisal was not restructured regarding its components for increasing access and quality to efficient and affordable services in rural areas as demonstrated by failed efforts or limited results in achieving these . Likewise, the modalities used for introducing private sector participation in the telecommunications and air transport sectors proved limited relevance to achieve the expected outputs (privatization of TDM and LAM and concession of MIA and other airports ) and were not restructured to increase their relevance . 4. Achievement of Objectives (Efficacy): To improve access to efficient and affordable communications services (Modest ): While some aspects of this part of the PDO (such as overall telephone penetration ) show significant results, other aspects were not measured at all (such as efficiency, rural penetration, airline and freight costs ). On the one hand, in the telecommunications sector, market penetration jumped from less than one percent in 2000 to over 20 percent in 2008), coverage increased significantly (from 3 percent of the population at project start to 44 percent today). and prices dropped in 75 percent. The number of internet users per 1000 people increased from 1.64 to 16.00. On the other hand, although the percentage of rural population covered by GSM (mobile telephone technology) increased significantly from 2.7 percent at project start to 44 percent in 2007, no measures of rural penetration or rural telephone density are reported . The latter was a KPI in the PAD and information on this could have informed the impact that the limited success of implementation of the rural access component has had (only an internet pilot was implemented). Furthermore, the PDO included "to improve access and quality of efficient and affordable communications services " (emphasis added), however, no measures of efficiency of telecommunications services were included in the ICR either . The ICR could have included indicators such telecommunications staff as a ratio of number of subscribers . Regarding air transport, the annual average number of air transport passengers increased from 394,671 in 2000 to 589,579 in 2006 (no more recent statistics are presented in the ICR ), a modest increase of 49 percent as GDP increased in 87 percent over the same period . Domestic traffic performed relatively stronger with the number of domestic flights that increased by a factor of 2.5. There is, however, no measure of freight transport performance in the ICR, despite this being one of the original KPIs in the PAD . No measure of affordability of air transport is presented in the ICR either. Regarding airport fees, these remained constant in nominal USD terms during the span of project implementation. No measures of efficiency were included in the ICR, although a small labor retrenchment in LAM is mentioned in the ICR (from 800 staff to 695 in 2009). To improve quality of efficient and affordable communications services (Negligible ): There are no measures of quality nor efficiency in the original KPI nor in the ICR . The ICR could have included quantitative indicators such as telephone faults as a ratio of number of subscribers, percentage of telephone faults cleared by next working day, waiting time to be connected; or qualitative indicators such as customer satisfaction . The second tier objective - to create an enabling environment for competition and private participation in telecommunications, postal and air transport infrastructure and services (Modest ): The project supported the introduction of competition in the mobile phone market, Vodacom was holding about 30 percent of market share in June 2009. New sector policies were adopted for the telecommunication, postal and air transport sectors; they allowed increased private sector participation and led to sector restructuring and growth in telecommunication, and one new entrant in the domestic air transport industry, although the new entrant - Air Corridor - terminated its operations later on . The project supported the gradual liberalization of air services between Maputo and Johannesburg. New legal and regulatory frameworks were adopted and started to be implemented in the telecommunication and the air transport sectors . The project assisted government in the preparation of a new Telecommunications Law, adopted in 2004, and a Civil Aviation Law, adopted in 2009. The Postal law has also been recently completed, and its adoption is forthcoming . The project also helped the government prepare air safety regulations, and regulations for airport and aerodrome certification . These new regulations allow for a level playing field and increased customer protection through better service quality oversight by the regulatory institutions whose capacities have also been reinforced through the project . Capacity building in the new regulatory agencies (INCM and the Instituto da Aviacao Civil de Mocambique - IACM), including training, equipment with IT network, specialized equipment and technical assistance, interconnection study for INCM, Global Navigation Satellite System approaches -GNSS- for IACM), have contributed towards strengthening their effectiveness and their credibility vis -a.-vis sectors' stakeholders . A new spectrum management equipment was acquired (one station in Maputo and two mobile stations outside Maputo ). It has allowed a better allocation of the spectrum, control of bandwidth usage, and dispute resolution in case of users complaints . A major regulatory weakness, however, is the lack of competitively priced backbone access which has constituted a severe barrier for operators to provide rural access (more of this below) and only an internet rural universal access strategy and pilots were implemented in 4 districts. RNA V (Area Navigation)/GNSS equipment (non precision approaches ) for eight airports has increased security and safety in regional airports . ICAO will determine early next year whether MIA meet international safety and security standards . The project failed to support the privatization of TDM and LAM .The launching of a third mobile license did not happen and TDM retains 70 percent of the mobile market. The MIA concession failed but the project supported a study evaluating financing options for the airport infrastructure system in Mozambique, covering Maputo airport along side other provincial airports. This study served as input to the preparation of the government's transport strategy review, and for medium and long-term financial planning of the airport infrastructure system, but the study recommendations have not been implemented yet. The telephony component of the universal access strategy was unsuccessfully tendered and only the internet component tendering was successful after a second attempt due to the lack of sector competition together with the changing policy environment and government policy . Despite the lack of progress in the areas above, the possibility of a formal project restructuring was not brought up by the Bank Team or the Government. 5. Efficiency (not applicable to DPLs): In the PAD, a short, qualitative estimate of costs and benefits shows that the project would bring indirect benefits for example cost savings, additional income from net job creation, increased income through reduced transaction costs for the rest of the economy, and consumer surplus . The fiscal impact was originally meant to be significant with two privatizations, the award of two mobile licenses, and a concession contract . There is no quantification in the ICR of the project benefits nor of the fiscal impact that the continued government ownership of TDM and LAM has . The ICR does not contain an assessment of the economic benefits /costs of the significant investment by the state -owned TDM (US$ 100 million in a fiber optic backbone). The ICR only mentions as quantified benefit the significant increase in revenues to the government from the award of the second mobile license to Vodacom (US$15 million). ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re-estimated value at evaluation : re- Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Based on sections 3-5, outcome is rated moderately satisfactory . a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: In the telecommunication sector risks to the pro -competitive environment introduced through the project are low and the continued engagement by the Bank in the sector through the follow -up project will contribute to reduce risks . In the air transport, the newly approved Civil Aviation Law locks in changes in the sector . The risks to capacity built in IACM are moderate, however, because of financial resources and the large spread between public sector salaries with the private sector. These might reduce the sustainability of improvements in safety and security in this sector . a. Risk to Development Outcome Rating : Negligible to Low 8. Assessment of Bank Performance: at -Entry (Moderately Satisfactory ): The project design was of adequate quality except for a Ensuring Quality -at- weak monitoring and evaluation framework . The PDO was somewhat complex as it spanned three communications sub-sectors (telecommunications, air transport, and postal services ) and although these were under the same Ministry, this complexity made project implementation more complicated . The risks to implementation were, however, underestimated as demonstrated by the failure to privatize TDM and LAM and to concessioning of airports . Quality of Supervision (Moderately Satisfactory ):The ): project went through changes in task management, and although the last change was to strengthen supervision by decentralizing task management to the field, implementation was slow (therefore the project had to be extended three times ) and the Bank only tried to fix the M&E design problems after many years of implementation (in 2008). The mid-term review was carried out late in implementation (May of 2005), only one year before the original closing date . Despite the lack of progress in many areas, the possibility of a formal project restructuring was not brought up by the Bank Team or the Government. More proactive supervision in the TDM and LAM privatization and the concessioning of airports in terms of providing alternatives to privatization and working with the government moving forward private participation in the sectors could have resulted in more competitive and efficient services . Supervision was significantly more pro-active regarding the universal information and telecommunications access strategy, but unfortunately these efforts only resulted in partial success . at -Entry :Moderately Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Moderately Satisfactory c. Overall Bank Performance :Moderately Satisfactory 9. Assessment of Borrower Performance: The change in government in 2004-2005 and consequent changes in government policy had a strong impact on the pace and the orientation of project implementation . The absence of a strong policy unit in the Ministry of Transport and Communications was also a handicap for the implementation of a project . INCM and IACM suffered from changes in management and board members . Lack of coordination and leadership and staff turnover also affected implementation . The capacity of the Project Management Unit (UGPTC) was strengthened by hiring qualified staff; their management of other project, however, seemed to have diluted their implementation effort in this project, in the latter years . a. Government Performance :Moderately Satisfactory b. Implementing Agency Performance :Moderately Satisfactory c. Overall Borrower Performance :Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: M&E desig n (Modest ): The PAD lacked indicators to measure the quality and efficiency dimensions of the PDO . There were no indicators to measure postal services improvements despite being part of the scope of PDO - although the sub-components associated to postal services were relatively small . The KPI did not have baselines; targets were too broad or lacked units of measurement . Despite weaknesses in the government's capacity to monitor performance, the project did not contain any support to improve such capacity . The following are the KPI from the PAD: (i) Telecommunications and Post: • Overall telephone density (fixed and mobile) to at least double from 0.8 (2000) to 1.6 lines per 100 inhabitants by 2006 as increased competition and new licensing regime were expected to accelerate network build -out; • Telephone density in rural areas to double from 0.012 lines per inhabitants (2000) to at least 0.024; number of pay phones and other community access points (cybercafes, telecenters ) to communications services to increase once the rural and universal access strategy developed under the project is implemented; • International tariffs, mobile tariffs, cost of leased lines and internet access costs to decrease to regional standards by 2006 as competition increases throughout the sector . (ii) Air Transport: • Passenger and air freight traffic volume to increase in line with average regional SADC growth rates by 2006; • Airport services at MIA to improve through a reduction in airport fees in line with average aeronautical fees in competing regional airports and an increase in the number of connections at least equal to that of average regional connection increase by 2006; • Cost of international and domestic airline tickets to be reduced in line with published economy and charter fares fro competing destinations and similar distances (such Johannesburg/Mauritius) by 2006; • Freight costs to be reduced in line with regional competing destinations by 2006. M&E implementation (Modest ): Some indicators were never tracked, e .g., until June of 2008 when new KPIs were introduced in the Implementation Status and Results Report (ISR), the only outcome indicator tracked was "Overall telephone density" and only one air transport indicator was tracked at the intermediate outcome level of the ISRs (tender of concession of Maputo International Airport ) while no postal indicators were tracked at any level . Because of their limitations, the original KPIs were revised and a new Results Framework (RF) was adopted in 2008, very late in project implementation and only as the project was being extended for 18 months.The ICR listed the revisions which included: (a) transforming the logframe into a RF, (b) selecting only outcome and intermediate results indicators originally proposed in the logframe for the use in the RF; (c) dropping two indicators measuring change in price of airline tickets; (d) dropping indicators for air freight traffic, (e) adding an indicator at the PDO level measuring increase in air traffic by change in number of air transport passengers .The ICR provides four main justifications for the revisions: (a) the logframe was no longer used by the Association for results monitoring, therefore the current format of Results Framework was adopted; (b) the logframe consisted of a mix of outcomes /results and large number of progress monitoring outputs; (c) the indicators measuring change in price of airline tickets and air freight had not been monitored by the client during the last five years of implementation (in addition, given the fluidity and confidentiality of airline pricing and measurement of freight traffic, historic data is not available ); and (d) the project did not envision any major investment into improvements of freight traffic, making the intermediary indicator measuring increase in freight traffic useless . These revisions, however, were not done through a formal restructuring despite dropping indicators and associated targets to measure access and affordability which are main dimensions of the PDO Non-evaluable ): The ICR provides little information on M&E utilization and rather highlights that the M&E utilization (Non main beneficiary agencies (INAC and INCM) did not collect data after 2007 and that after the project closed in June 2009, the UGPTC was dismantled and it became impossible to seek updated data . a. M&E Quality Rating : Negligible 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): Procurement and financial management were generally adequate during project implementation . One major issue was that UGPTC, based on their interpretation of what was required, did not submit Financial Management Reports (FMR) throughout most of the project life, despite numerous reminders, and the presence of a Bank Financial Management Specialist in the field to follow up . However, the annual audits of the project financial accounts showed adequate management of funds . At the end of the project, the PIU provided the cumulative FMRs . An environmental pre-audit was prepared by the government in 2001. It was the only safeguard policy that applied to the project . It provided three sets of provisions relative to the divestiture of TDM, MIA and LAM . These divestitures did not occur and no specific safeguard issues aroused during the project . 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Satisfactory Satisfactory Risk to Development Negligible to Low Negligible to Low Outcome : Bank Performance : Satisfactory Moderately Based on moderately satisfactory Satisfactory quality at entry and moderately satisfactory supervision (as per the harmonized criteria). Borrower Performance : Moderately Moderately Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: This review concurs with most of the lessons presented in the ICR . The need for a pro-active Bank approach and timely project restructuring to adapt to changes in realities is crucial in any project but especially in this case in which there were significant changes in government priorities . Substantial achievement of project outcomes could have been possible through project restructuring to allow for alternatives to privatization or full concessions, e .g., public-private partnerships or partial concessions . A related lesson is the need for more pro -activity by the Bank, in for example supporting the telecommunications regulator, as the privatization efforts failed and essential transmission and backbone (international and domestic) facilities remained as monopoly priced services while mobile services continue to be provided within a duopoly structure . 14. Assessment Recommended? Yes No Why? The Bank has embarked in a follow -up project so a more in depth analysis of this project will be helpful to inform continued involvement in the sector . 15. Comments on Quality of ICR: The ICR offers a candid assessment of the project achievements and points out design deficiencies and implementation problems. The ICR should have done a better job at assessing efficacy of the missing dimensions of the PDO ("... quality of efficient ... communications services") and of project efficiency. a.Quality of ICR Rating : Satisfactory