ICRR 12558 Report Number : ICRR12558 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 02/09/2007 PROJ ID :P035076 Appraisal Actual Project Name :Zm-Power Rehab SIL (fy98) Project Costs 223.43 364.31 US$M ) (US$M) Country :Zambia Loan/ US$M ) Loan /Credit (US$M) 75.0 75.0 Sector (s):Board: ): US$M ) EMT - Power (96%), Cofinancing (US$M) Health (1%), Roads and highways (1%), Water supply (1%), General agriculture fishing and forestry sector (1%) L/C Number :C3042 FY ) Board Approval (FY) 98 Partners involved : Closing Date 12/31/2002 12/31/2005 Evaluator : Panel Reviewer : Division Manager : Division : Robert Mark Lacey Peter Nigel Freeman Alain A. Barbu IEGSG 2. Project Objectives and Components a. Objectives The project’s main development objective was to enhance the ability of Zambia ’s electricity supply industry to provide power efficiently, at least cost, and in a sustainable manner . This was to be achieved through (i) establishing an institutional and regulatory framework conducive to sustainable efficiency and private sector participation; (ii) improving the efficiency of the national electricity company ZESCO through capacity building measures and a commercialization plan; and (iii) enhancing technical performance, service quality and reliability through rehabilitation of the existing generation, transmission and distribution facilities. An additional objective was to implement a rehabilitation and development program for the Gwembe-Tonga people, a group which had been inappropriately and unsuccessfully resettled following the construction of the Kariba Dam Project, financed by the Bank in the 1950s. b. Components (or Key Conditions in the case of Adjustment Loans ): The PAD cites 18 components and sub-components ranging from physical rehabilitation of capital works to community participation and AIDS awareness. In summary: (a) rehabilitation of the Victoria Falls, Kafue Gorge and Kariba North Bank hydropower stations; (b) transmission system rehabilitation and upgrading; (c) upgrading of the Lusaka distribution system; (d) completion of a national telecommunications and control center; (e) replacement of transformers; (f) verification of the safety of the Kariba Dam; (g) various actions to improve the financial viability of ZESCO including tariff adjustments, strengthening of customer services management, establishing a business information system, training and capacity building; (h) strengthening of ZESCO’s Environmental-Social Affairs Unit; (i) establishment of an institutional and regulatory framework conducive to private sector participation in the power sector, including the privatization of ZESCO (this set of components was also supported by a sequence of adjustment operations and HIPC conditionality); and (j) the Gwembe Tonga Development Project (GTDP), itself involving several sub-components including road construction, rural electrification, improved water supply, food security, and community participation, mobilization and capacity -building. Revisions . New activities were added during implementation : (a) At ZESCO’s insistence, the re-powering of the generation plants at Kariba North Bank and Kafue Gorge, financed by additional loans from the European Investment Bank (EIB and the Development Bank of Southern Africa (DBSA); and (b) 355 kilometers of access roads and an HIV/AIDS awareness program were added to the GTDP. c. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost . The complexity of the project was increased by the considerable number of agencies in involved in its funding. In addition to IDA, parallel financing was provided by DBSA , EIB, the Norwegian Agency for Development, the Swedish International Development Agency, the Finnish International Development Agency, and the Nordic Development Fund. In addition, an agreement with the African Development Bank was concluded during implementation. Total project financing requirements at completion were US$364.31 million, 63 percent higher than the US$223.43 million forecast at appraisal . While the transmission rehabilitation was completed within the appraisal cost estimate, there were over-runs of almost 80 percent on the hydropower component. The higher costs were due to serious under-estimation at appraisal, delays in implementation (see below), and the addition of the re-powering component. The cost of the GTDP was grossly underestimated – outlays of US$172 million had been made by the time the ICR was prepared (and some components remain under-financed) as compared to US$6.94 million predicted at appraisal . This was because no technical evaluation was undertaken, nor was a financial plan prepared. Financing . The ICR provides no detailed breakdown of project financing by source of funds . However, it can be calculated that the IDA contribution amounted to just over 15 percent of total project costs as compared to 25 percent anticipated at appraisal. Due to the overruns, the secured financing for the project was inadequate to cover all components. The rehabilitation cost overruns were met through supplemental loans from EIB and DBSA, and from commercial bank borrowing by ZESCO. The allocations to the GTDP were made after all other components had been funded. The Government undertook to fill the remaining financing gap for the GTDP from its current budget. Dates. Dates The original closing date of the end of 2002 was extended three times to December 31, 2005. The main causes of the delay were: (a) The project took over a year to become effective because of government inaction related to cross-covenants with co-lenders; (b) technical preparation was inadequate; (c) the fact that rehabilitation of existing power plants is often more time consuming than green field projects was not taken into account at appraisal; (d) important technical alternatives were not evaluated; (e) the Kariba re-powering component was subsequently added; and (f) the capacity limitations of the ZESCO PMU were addressed only at a comparatively late stage of implementation . There was a distinct danger that there would still be an unspent balance of the IDA contribution at the final closure . This was avoided by contract swapping with the DBSA so as to utilize fully the IDA credit for works already completed prior to its closure. 3. Relevance of Objectives & Design : There is no discussion in the ICR of the project ’s relevance to either CAS or Borrower objectives . According to the PAD, it supported the goals of both the Borrower and the CAS to promote private sector led development, greater public sector efficiency, and more rapid and inclusive economic growth . There is no doubt that these goals were and remain relevant in Zambia. However, the over-complex project design and a number of other weaknesses during the preparation make the relevance of the project design to the stated objectives less clear. 4. Achievement of Objectives (Efficacy) : The first two project objectives: (i) establishing an institutional and regulatory framework conducive to sustainable efficiency and private sector participation, and (ii) improving the efficiency of the national electricity company ZESCO through capacity building measures and a commercialization plan -- were not achieved during the project's implementation period . The third goal -- improving technical performance, service quality and reliability through rehabilitation of the existing generation, transmission and distribution facilities -- was achieved through the largely successful implementation of the project's physical investment components. However, the failure to put ZESCO on a sound financial footing threatens the long-term sustainability of these improvements . The project failed to promote any significant amelioration to sector institutions or to promote private sector participation during the project's implementation period . Although the Bank had received, in 1997 a Letter of Sector Policy from the Borrower, it was discovered shortly after effectiveness that the Government and ZESCO had no consistent views about the appropriate structure of the power sector . There was no agreement on how to implement the hydropower and transmission strategy outlined in the Government ’s Statement of Sector Policy established at negotiations. No significant progress was made in restructuring the sector or in setting up an office for promoting private investment . There was clearly little government support for such participation. The project’s impact on the effectiveness of the regulatory framework and on tariffs was equally negligible during project implementation. Although the Energy Regulation Board (ERB) was established as a condition of appraisal, it has so far failed to achieve its objectives . Tariffs have not been adjusted to levels consistent with financial viability; and nothing has been done to create an environment conducive to private sector participation. Although legislation in late 2003 has enabled ERB to increase its technical capacity, and an international consultancy firm is assisting with training and with development of a cost-based tariff structure, a rational tariff structure was still not in place by the project's end . During the period 2002-2005, inflation was 220 percent and electricity tariffs – already below cost – were adjusted by only 70 percent. ERB was not allowed to participate in tariff negotiations for the critically important large scale supply contracts for new copper mining projects . (These large users are given disguised subsidies through low utility tariffs.) Although new business information and customer service management systems were introduced in ZESCO, the project failed to achieve its objective of commercializing the company . In particular, the Government did not honor the commitments it made originally to IDA to privatize ZESCO . (These commitments were original conditions for Completion Point attainment under the Enhanced HIPC .) When they were no longer acceptable to the Government, alternative road paths to commercial viability, not involving privatization, were developed . To date, these too have not been implemented and ZESCO remains a vertically integrated monolithic power company in acute financial difficulty . Little was achieved under the GTDP due to poor technical and financial appraisal, inadequate implementation arrangements, and lack of interest on the part of the Authorities . Only in 2003 did a new supervision team refocus the Government’s attention on the component. A detailed and comprehensive program was drawn up for the next eight years to implement the program and to sustain the little that had been achieved. Although the Government made an allocation in its 2006 budget for the road network, there remains a serious question as to the future of the GTDP as a whole since it will be so reliant on public budget financing. 5. Efficiency : At appraisal, a detailed set of ERR calculations was made for each of the six physical investment components of the project. The ICR questions the methodology used at appraisal (the Wien’s Automatic Systems Program or WASP to establish the least cost justification ) and recalculates the ERR at credit closure taking the benefits as the outputs of the rehabilitated power plants (valued at the marginal cost of generation) plus the increased sales of the transmission and distribution systems resulting from improved reliability and reduced power losses . Because the methodologies are different, the results are not directly comparable. However, both exercises show that all the rates of return are well above the estimated opportunity cost of capital (12 percent). The ERRs at appraisal range from 20.1% to 50.1%, and at closure from 22 percent to 57 percent (even though neither take account of the benefits from the re-powering of the hydro plants). The averages (weighted by net present value of benefits) are close to each other. a. ERR Point Value Coverage/Scope Appraisal: Generation 42% 32% Transmission 33% 39% ICR estimate Generation 44% 35% Transmission 33% 19% 6. M&E Design, Implementation, & Utilization: The text of the ICR contains no detailed discussion of monitoring and evaluation . The key performance indicators in Annex 1 compare, in some cases, latest estimates with those in the last PSR, but do not specifically refer back to the outcome and output projections prepared at appraisal . This reflects the fact that the relevant table in the PAD was seriously inadequate: there was no real differentiation between outcomes and outputs and no attempt was made to establish a clear hierarchy or causal chain . Comparison between appraisal estimates and outcomes is thus almost impossible . At IEG's request, the Region subsequently provided data on physical performance indicators -- generation plant availability, system interruptions, distribution losses and percentage of the population with access to power supply -- comparing the situation before rehabilitation in 2000 and after in 2005. These all show improvements and attest to the beneficial impact of the rehabilitation, at least in the short term . 7. Other (Safeguards, Fiduciary, Unintended Impacts--Positive & Negative): As the project's environmental category was B, there is comparatively little discussion of safeguards in the ICR except in the context of the GTDP. Since the PAD states as one of the project's objectives the building of ZESCO’s capacity in environmental management, it would have been useful if the ICR had included a more systematic evaluation of achievements under this rubric . There is substantial discussion of tariffs and ZESCO's financial weaknesses which addresses the financial covenants and other fiduciary issues. 8. Ratings : ICR ICR Review Reason for Disagreement /Comments Outcome : Satisfactory Unsatisfactory Although the project's objective of enhancing technical performance was largely achieved, the other two objectives were not achieved. Furthermore, the persisting financial weakness of ZESCO, seriously undermines the likelihood of the technical achievements being sustained in the long-term. Institutional Dev .: Negligible Negligible Sustainability : Unlikely Unlikely Bank Performance : Satisfactory Unsatisfactory There were serious weaknesses during preparation and appraisal, and quality at entry was unsatisfactory. Initial supervision was also below standard. During the last two years of implementation, supervision improved considerably. However, this could not compensate for earlier shortcomings, and the overall outcome of the project remained unsatisfactory (see above). Overall, in accordance with the recently issued rating guidelines, Bank performance is therefore rated unsatisfactory (on the four-point scale). Borrower Perf .: Unsatisfactory Unsatisfactory Throughout most of the project implementation, the government has failed to take the necessary action to improve ZESCO’s financial situation. On the contrary, it delayed settlement of debts, assumed new arrears when the earlier debts were settled, refused to adjust tariffs, failed to endow the ERB with either the authority or the resources to do its job, and would not countenance any institutional or structural reforms (by no means confined to commercialization) to modernize the sector. Moreover, the Ministry in charge of the energy sector failed to exercise its administrative responsibilities (for example, issuing of bidding documents) in a timely manner. Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness . 9. Lessons: It is critical to establish the political viability of the reforms supported through extensive consultation with actual and potential political leaders . Critical policy issues must be addressed up front prior to effectiveness (prior to Board approval would be preferable). Unless the Authorities respect their undertakings, notably in the area of tariff adjustments, then this will lead, as in the case of ZESCO, to an unsustainable financial situation . This issue is also emphasized by the DBSA in its comments. Much stronger insistence by the Bank on such issues is required, including the threat to suspend disbursements . Large environmental subcomponents, such as the GTDP, require government commitment and adequate financing. It is mistaken to expect a power utility to have the capacity to implement effectively social projects such as agricultural extension schemes and feeder roads . Rehabilitation projects in the power sector tend to be much more complex than green field schemes . Commercialization and fair, soundly based and regulated transfer pricing mechanisms are minimum elements of power sector reform (these elements were not specifically addressed in the PAD ). Governments should not use utilities to provide hidden subsidies to large industrial consumers . Senior managers in utilities need capacity in the financial and commercial, and not just the engineering sides of the business. ZESCO's staff, while displaying technical excellence, was weaker in other critical management areas. Cross conditions of effectiveness with parallel lenders need to be spelled out in the PAD and included in the risk analysis. Trying to address, in one single project, not only investment requirements but also a substantial range of policy issues accumulated during more than 20 years of absence from the sector (including policy dialogue), is likely to lead to an over-complex and unwieldy operation. Arguably this project should have been canceled when it became clear that there was no real Borrower commitment to the agreed reform agenda. However, this would have been difficult given the large number of agencies involved. 10. Assessment Recommended? Yes No Why? To verify ratings and draw lessons. 11. Comments on Quality of ICR: The ICR presents a considerable amount of data and analysis, and presents the reader with a clear picture of project outcomes, although it has a number of shortcomings : The ratings, especially of project outcome, do not fully reflect the discussion in the text . The treatment of the monitoring and evaluation aspects in general, and of performance indicators in particular, is insufficient . Although this reflects weaknesses in the PAD, it would have been useful for the ICR to have included comparative performance data (provided subsequently by the Region). There is no breakdown of project costs by source of financing, despite the considerable number of co-lenders involved. The ICR could have highlighted (as subsequently clarified by the Region) that there was little or no policy dialogue in the sector during the twenty years prior to the project . The ICR is poorly edited and contains a number of spelling mistakes and typographical errors .