ICRR 13328 Report Number : ICRR13328 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 06/03/2010 PROJ ID : P069996 Appraisal Actual Project Name : Ug-energy For Rural US$M ): Project Costs (US$M): 123.3 204.1 Transform (fy02) Country : Uganda Loan/ Loan /Credit (US$M ): US$M): 49.1 56.9 Sector Board : EMT US$M ): Cofinancing (US$M): Sector (s): Power (55%) General energy sector (18%) Telecommunications (11%) Other social services (8%) General agriculture fishing and forestry sector (8%) Theme (s): Education for the knowledge economy (25% - P) Other financial and private sector development (25% - P) Rural services and infrastructure (25% - P) Health system performance (25% - P) L/C Number : C3588 Board Approval Date : 12/13/2001 Partners involved : Closing Date : 08/31/2006 02/28/2009 Evaluator : Panel Reviewer : Group Manager : Group : Robert Mark Lacey Fernando Manibog IEGSE ICR Reviews IEGSE 2. Project Objectives and Components: a. Objectives: The project (ERT1) was the first in a proposed series of three APLs which aims to support the Government of Uganda’s Energy for Rural Transformation Program (ERTP). The development objective of the program, according to the ERT1 PAD, is “to develop Uganda’s rural energy and information and communication technology (ICT) sectors, so that they would make a significant contribution to bringing about rural transformation .� ERT1 supports the first phase of the ERTP . Its specific project development objective, again according to the PAD, was “to put in place, ‘on the ground,’ a functioning conducive environment and related local capacity for commercially oriented, sustainable service delivery of rural /renewable energy and ICTs.� ERT1’s PDO as stated in the Development Credit Agreement (DCA) is substantively similar, but has greater detail and precision: “to assist the Borrower in: (i) putting in place a conducive environment and related capacity for : (A) commercially oriented rural electrification schemes, and (B) small scale, renewable power generation schemes to be carried out by eligible enterprises; (ii) facilitating investment in (A) commercially oriented rural electrification schemes, and (B) the scaled up delivery of electricity in rural communities; and (iii) the acceleration of (A) rural access to basic telephone service, and (B) the spread of the Internet to district capitals and vanguard rural institutions .� This ICR Review bases its evaluation on the ERT 1 PDO as stated in the DCA. Progress towards achievement of wider ERT program goals is assessed against the program objective stated in the PAD . The project included participation by the Global Environment Trust Fund (GEF). The global environmental objective (GEO) was “to build in-country capacity for renewable energy .� b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): 1. Main grid -related power distribution and generation (US$45.6 million at appraisal, US$154.1 million at closure), financing extension of the main electricity grid to un -served rural areas. 2. Independent grid systems (US$26.1 million at appraisal, US$15.0 million at closure), supporting expected private sector implementation. A significant part of the power generation was expected to rely on renewable energy resources and to benefit from GEF funding . PV) systems (US$10.8 million at appraisal, US$4.5 million at closure) for installation in homes, 3. Solar photovoltaic (PV) community institutions, and rural enterprises located in relatively dispersed areas with small loads where grid systems (even when small scale and independent ) would not be viable. 4. Cross -sectoral linkages (US$12.4 million at appraisal, US$7.6 million at closure), financing (a) energy systems (for example, solar refrigerators and water heaters ) in the health sector; (b) up to 90% of the capital costs of energy/ICT packages in selected post -primary schools and related technical assistance to the Ministry of Education and Sports; (c) technical assistance to the Ministry of Water to identify and meet the energy needs of the rural water strategy; and (d) support for private investments in agriculture through subsidizing priority connections and technical assistance for program promotion in the Ministry of Agriculture . 5. Energy sector capacity building (US$15.9 million at appraisal, US$15.5 million at closure), financing support to a variety of public and private sector institutions to help them carry out their respective responsibilities under the project. 6. ICT sector (US$12.5 million at appraisal, US$6.5 million at closure), to (a) provide technical assistance to the Uganda Communication Commission (UCC) for the preparation of competitive tenders to increase access in rural areas; and (b) support the financing, by the UCC, of the subsidy (to be provided on the basis of competitive bidding ) to suppliers of services in commercially unattractive rural areas . GEF funding was to be provided for the following components : component 1, US$4.6 million; component 2 US$1.0 million; component 3 US$1.4 million; component 4 US$0.8 million; and component 5 US$4.3 million. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The project was designed as a demand -driven, private sector led initiative in which private investment, supported by “smart subsidies,� would bear the greater part of the financial burden of the government ’s rural energy and ICT programs. With regard to energy, the private sector response was disappointing . The Government ended up funding 100% of the main grid extension (component 1), hence accounting for its greatly increased cost to the project compared to the appraisal estimate . Much of this was ultimately financed from “foreign� and “bilateral� sources (both commercial and otherwise in the first case ), but which are not specified in the ICR . In addition, in October 2007, nearly US$12 million of IDA and GEF funds were reallocated to accommodate -- with ERT monies -- a substantially larger number of sub-projects (component 2) than foreseen at appraisal, again because of the disappointing private sector response. The increase in the dollar value of the IDA credit reflects the weakening of the dollar against the SDR in which the credit was denominated . Implementation got off to a very slow start because, first, private sector interest failed to materialize; second, there was no detailed implementation plan, consultancy terms of reference, or ready -prepared procurement packages so that many components were not ready after Board approval or even at effectiveness; and third, no project coordinating unit was in place until July, 2004. Only 10% of the IDA credit had been disbursed by the end of October 2004, when the mid-term review (MTR) was held. A two-year extension of the closing date to August 31 2008, was granted to allow the Borrower time to complete project -financed investments, and there was a further extension until February 28, 2009 to permit completion of works and payment of contractual commitments . 3. Relevance of Objectives & Design: Relevance of objectives was and remains high . Infrastructure provision to further rural development is a key element in the Government of Uganda ’s economic and social strategy . Since rural areas contain the majority of Uganda’s poor, and also suffer from very low rates of access to energy and ICT services, the Authorities give a high priority to sector investments in these areas . IDA’s Country Assistance Strategy (CAS), presented to the Board in November 2000, focuses on poverty reduction through sustained growth . The ERTP is highly relevant to, and supports, all four pillars of the CAS : directly increasing the ability of the poor to raise their incomes; directly increasing the quality of life of the poor; creating an enabling environment for economic growth and structural transformation; and ensuring good governance and security (including improved public service delivery ). These priorities continue in subsequent CASs . Relevance of project design is rated as modest . Although it was based on a thorough background analysis of the energy and ICT sectors, and of the quality of service delivery in rural areas, design had two significant shortcomings . First, there was no in-depth investigation of the likely response of private investors to the incentives to be put in place by Uganda, and hence the risk of a lack of interest was seriously underestimated . As the Government observed following the completion of ERT1, “the demand-driven, private sector model was found to be premature for the Ugandan economy, while Government has still a big role to play in extending services to the people .� Hence the design of ERT was, in a fundamental sense, unrealistic . Second, the capacity of Government and sector institutions to be able to carry out such an ambitious program of rural transformation was over -estimated. Although training and technical assistance were incorporated in the project they inevitably took time to impact favorably on severe absorptive capacity constraints . Capacity limitations were aggravated by weak implementation arrangements, notably the lack of any kind of project coordinating unit . Overall relevance of objectives and design is rated as substantial . 4. Achievement of Objectives (Efficacy): ERT1’s PDO was to assist the Borrower in: (i) putting in place a conducive environment and related capacity for : (A) commercially oriented rural electrification schemes, and (B) small scale, renewable power generation schemes to be carried out by eleigible enterprises (sub- sub -objective 1); (ii) facilitating investment in (A) commercially oriented rural electrification schemes, and (B) the scaled up delivery of electricity in rural communities (sub- sub -objective 2); and (iii) the acceleration of (A) rural access to basic telephone service, and (B) the spread to Internet to district capitals and vanguard rural institutions (sub- sub -objective 3). Achievement of the first two sub -objectives is rated as modest . Although sub-objective 3 is rated as high, ICT was not a major focus of the project . Overall efficacy is, therefore, assessed as modest . Sub -objective 1: modest . There were two dimensions to this sub -objective – the regulatory and institutional Sub- framework and the adequacy of financial intermediation arrangements . With regard to the first, some progress was made. The Rural Electrification Agency (REA), which is responsible for technical dimensions including the evaluation of projects, is evolving into a key institution for planning and managing the ERTP . The regulatory agency (Energy Regulatory Authority – ERA), established under the Electricity Act of 1999, the Rural Electrification Fund (REF, to provide investment subsidies ), and the Rural Electrification Board (REB, responsible for subsidy policy, approving projects and managing the REF), are all fully established and have benefited from capacity -building technical assistance. In addition, a renewable energy policy has been put in place to guide investors . However, REA still requires strengthening in certain key technical areas, and it lacks autonomy since it remains directly under the control of the Ministry of Energy (the Chairman of the REA Board is the most senior civil servant in the Ministry ). Lack of operational autonomy also affect the REB and the REF . The former is unable to make independent decisions based on agreed financial and socio -economic criteria – indeed, at present, there are no clear criteria for selecting rural electrification projects. The REF also remains under the Ministry ’s control, and the fears that the allocation of its funds would be subject to political interference have been realized . The ERA enjoys greater nominal independence, but the Electricity Act needs to be reviewed to give greater clarity to its role and that of other sector institutions . A perceived lack of de facto regulatory independence undermines investor confidence in the power sector as a whole . Concerning financial intermediation, progress has also been made in establishing the institutional framework – as well as the REF, a refinancing facility is in place at the Central Bank which did finance some investments during ERT 1. The facility provided critical long term finance without which commercial banks and other private financiers would have been unwilling to participate . The facility will not be extended into the next phase of the ERTP and is being replaced by a credit support facility which is still in its infancy . Sub -objective 2: modest . The target of the satisfactory functioning of two rural electrification operations with Sub- independent grids has not been attained . Of the two schemes financed under the project – a hydropower plant and an independent grid – the former is still under construction and is not expected to be commissioned before the end of 2010, while the latter was commissioned in February 2009, and is connected to about 300 clients, starting from a base of zero. Two other independent grids financed by the Government are in operation . The failure of independent grid systems to make the expected progress is due to financial, sub -contracting and management difficulties within the sponsoring companies. High connection fees also present a constraint . The goal of satisfactory functioning of the energy systems provided to users in the health, education, water and agricultural sectors has been partially achieved . The health and water components are on course, and some agricultural projects have been connected . However, education has been delayed by disagreement on tender documents; the agriculture component has been constrained by inappropriate subsidy criteria and by unclear demarcation between the roles of the REA and other involved institutions; and in general, progress has been slowed by poor technical capacity and project preparation, and lengthy procedural requirements (not least on the part of IDA). Sub -objective 3: high . One public payphone has been provided for every 2,500 inhabitants in the designated areas, Sub- double the target of one per 5,000; 32 Internet access points have been installed in district capitals, and additional dedicated bandwidth has been added between the national capital and rural areas; 20 new smaller and more efficient multi-purpose telecommunication centers are fully operational, replacing the original target of seven large centers; the national postal service has set up an additional 20 centers and connected them to the Internet; and the collection rate of the Telecommunications Universal Service Levy has reached 95% since the end of 2007as against a target of 80% by project closure. Household access to, and use of, ICT facilities is 45% in areas benefiting from ERT1, compared to 29% elsewhere in rural Uganda. GEO building in -country capacity for renewable energy . Much progress was made during Phase I of the ERTP in GEO: commissioning grid-connected power generation from renewable sources, though not all of it was funded by ERT1.Total new capacity of about 55 MW was installed by the end of 2009, and some 25 MW supplied to the grid. Over 1.3 million Watt-peak sales of solar PV systems were supported through matching grants and subsidies, well in excess of the target figure of 320,000. However, this needs to be qualified . Most of the systems purchased were for community-related services benefiting from a secure source of funding . The response from domestic consumers of Solar Home Systems (SHS) remains low and well below the project target . This is despite the fact that the aim of reducing SHS prices by 30% from the year 2000 benchmark was achieved. The disappointing result was due to insufficient marketing and technical capacity on the part of sellers, still high capital outlays, and inadequate financing terms. A long term renewable energy capacity building strategy and action plan was finalized in April, 2007. Renewable energy-related training in sector institutions (for example, REA, ERA, Ministry of Energy) has been completed, and is ongoing in the private sector . A pipeline of renewable energy projects, mostly, mini -hydro, is growing. ERTP programmatic objectve : to develop Uganda ’s rural energy and information and communication technology ICT ) sectors, so that they would make a significant contribution to bringing about rural transformation (ICT) . While the ICT program has made solid progress, and a number of policy and institutional steps have been taken in both the ICT and power sectors, private sector participation in rural energy investments has been much more limited than was hoped and anticipated during preparation . On the positive side, a culture of partnership and collaboration with the private sector is being developed in the sector ’s public institutions; independent grid and rural electrification schemes, while still in their infancy, have started to operate, and suggest the possibility of eventually benefiting a considerable number of consumers; and solar energy panels, especially in community centers, have enhanced rural inhabitants’ access to a higher quantity and quality of other public services . However, as noted above, much remains to be done to create an environment conducive to private investment, and the prospects of a significant increase in private sector provision of rural energy services in the second phase of the ERTP are not promising . 5. Efficiency (not applicable to DPLs): Neither the PAD nor the ICR attempt to calculate an economic rate of return (ERR) on the grounds that ERT1 involved investments in a wide range of sub -projects stretching over two sectors . However, the ICR could and should have estimated ex-post ERRs for at least a sample of the sub -projects financed, as is normally done for this kind of project. Instead, it rates the project as “efficient,� citing as justification a qualitative listing of the direct and indirect benefits to the rural economy from the energy and ICT facilities provided . This is not adequately justified . In the absence of fuller information and analysis, IEG rates efficiency as modest . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: With substantial overall relevance and modest efficacy and efficiency, outcome is rated moderately unsatisfactory . a. Outcome Rating : Moderately Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: ERT1 achievements in the energy sector face a significant risk . This is that the private sector, concerned at the high costs and low returns associated with rural energy investments, and lacking confidence in the ability of the legal, regulatory and institutional frameworks in place to provide sufficient countervailing guarantees and compensation, will continue to display reluctance, thereby undermining the whole approach of the ERTP . This risk is reinforced by the still weak capacity and lack of autonomy of the public agencies involved, by Uganda ’s vulnerability to drought conditions and hence to volatile, internationally determined costs of thermal power, and by the slow progress in developing a market for SHS. a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: Quality at entry (QAE) was moderately unsatisfactory . The main sector issues and strategies to deal with them were analyzed in depth. A quality enhancement review (QER) was conducted in August 2001, to examine several aspects of ERT1’s objectives and design. There were wide consultations with stakeholders, potential beneficiaries and policy makers . The main features of the project were discussed in public meetings and the views of participants reflected in design . The preparation team had a strong technical composition, and a number of alternatives designs were considered . However, as noted in Section 3 above, this good work was undermined by two significant weaknesses – inadequate analysis of the likely private sector response and insufficient consideration of in-country implementation capacity -- the first of which was of sufficient gravity as to threaten the viability of the model adopted for the project . Supervision was satisfactory . A large team (21 members, not including management and administrative support), with the appropriate skill mix was mobilized . The project was intensely supervised with 14 missions in just over six years supplemented by close monitoring by the Bank ’s Kampala office, especially from 2004 onwards. The Office supplied a senior member of the supervision team . Local staff members had a good knowledge of Uganda’s rural sector, linguistic and technical, and they were well backed by an engaged Bank country management team. The supervision team performed well in addressing the project ’s lack of readiness for implementation and was instrumental in, for example, encouraging the Government to fill a number of key posts that were still vacant. The MTR was used to discuss in depth with the Borrower the steps that needed to be taken to revive a faltering operation . Disbursements subsequently picked up, but 50% of them were still concentrated in the (extended) last two years of the project ’s life. The team responded flexibly to Borrower concerns, and even the perennial complaint about slowness in giving non -objections was alleviated by better communications with the Kampala office. The Implementation Status Reports (ISR) were generally reflective of the reality on the ground, with the exception of the last ISR which granted an overall rating of satisfactory for a project which had failed to meet several key targets and closed 30 months behind schedule. In light of the success of the supervision effort in overcoming many design weaknesses and the lack of Borrower preparedness, IEG rates the overall quality of Bank performance as moderately satisfactory . at -Entry :Moderately Unsatisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Moderately Unsatisfactory 9. Assessment of Borrower Performance: Government displayed and sustained a strong commitment to the program and the project from the outset . This commitment was reflected in a number of actions such as amending the 1999 Electricity Act in accordance with Bank recommendations, organization of a stakeholders ’ workshop, a meeting of bilateral donors, presided by the Minister of Finance, to garner their support for the ERTP and ERT 1, and the strong support and participation of sector ministries such as agriculture, health and education . The Authorities also demonstrated their commitment to a policy of encouraging renewable energy . On the negative side, there was some tardiness in filling key posts and a persistent reluctance to grant greater autonomy to sector institutions . Implementing agency . The PCU performed its coordinating role reasonably well . Reporting was complete and informative, though not always timely . It provided valuable feedback on implementation progress and was a useful input to supervision missions . Although the PCU played an important role in bringing together the different ministries and other public entities involved in the project, it focused perhaps too much on the components managed by the Ministry of Energy, and could have provided more coordination and support to ministries and agencies in other sectors . a. Government Performance :Satisfactory b. Implementing Agency Performance :Satisfactory c. Overall Borrower Performance :Satisfactory 10. M&E Design, Implementation, & Utilization: Design . The PAD promises an M&E system for the project to be presented in Annex 2. Unfortunately, that Annex contains no such description . Instead (in Annex 1: “Project Design Summary�), there is a list of some 25 “end of project� performance indicators, some of which are very precise (“Number of rural households connected [to independent grids],� “MWh of renewable energy produced �), but others quite vague (“Economic and social changes, � “Employment/economic gain from SMEs involved in the project �). There are no baseline values so that it would have been difficult, in any event, to use the indicators as a measure of progress towards meeting project objectives . For some project activities, for example, supplying energy to the rural health, water and education sectors, the PAD does not propose any indicators but states that a scheme will be set up (by whom?) to monitor and evaluate the impacts on service delivery. M&E design is rated as negligible . Implementation . Responsibility for M&E was implicitly divided three ways – project level monitoring by the PCU (though that did not happen for two years until the PCU was established ); sector monitoring (health, education, agriculture, water) by the concerned Ministries and agencies; and the project ’s impact on poverty to be monitored by the Ministry of Finance. These arrangements alone complicated the task of developing and maintaining a robust M&E system to measure progress in a complex project involving a variety of sectors and implementing agencies . The difficulties were compounded by the unsuitability of the indicators developed in the PAD . This latter problem was addressed at the MTR where a detailed monitoring system was put in place including baseline values and a number of standard quantifiable indicators for measuring progress in attaining the PDO . These were incorporated in the Data Sheet of the ICR. In addition, the Ministry of Finance conducted baseline surveys to enable assessment of the project’s social impact. The PCU eventually assumed responsibility for bringing together the data gathered by the different agencies and used them to prepare regular quarterly reports . Implementation is rated as substantial . Utilization . Although tardy, a workable M&E system was in operation subsequent to the MTR and establishment of the PCU. It was used for decision making on project activities and the reallocation of funds . The system should prove useful for the subsequent phases of the ERTP . Utilization is assessed as substantial . a. M&E Quality Rating : Substantial 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): There is no discussion in the ICR of compliance with safeguards, beyond a general statement that “there were no significant deviations or waivers from the Bank safeguards and fiduciary policies and procedures during implementation of the project.� Moreover, the Data Sheet does not contain a recognizable classification of the project for the purposes of OP4.01. Such deficiencies are a major gap since the PAD recognizes that the project was expected to have potentially significant effects on the environment of rural Uganda and also involve resettlement and land acquisition. The PAD refers to the fact that the Government had established an Environmental and Social Management Framework which it summarizes in an Annex . The implementation of this Framework should have been examined and assessed in the ICR . There is some discussion of fiduciary matters . The financial management system put in place was adequate and satisfied Bank requirements according to the ICR . Financial management was centralized at the Ministry of Energy and the REA was responsible for coordinating reporting from the other Ministries and agencies involved . Project audits “improved� suggesting that they were initially inadequate, and REA ’s response to auditors’ recommendations was also judged satisfactory . There were procurement difficulties due to complex bidding packages and inexperienced implementing agencies, but no major procurement issues arose according to the ICR . 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately Private investor response was well Unsatisfactory below expectations and the prospects for greater investor interest in subsequent phases of the program are doubtful, thereby undermining the whole approach of the ERTP. A number of rural energy-related targets were not met. The project closed 30 months behind schedule. Risk to Development Significant Significant Outcome : Bank Performance : Satisfactory Moderately QAE weaknesses were sufficiently Unsatisfactory serious as to call into question the demand-driven, private sector led approach of the project. Borrower Performance : Satisfactory Satisfactory Quality of ICR : Unsatisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: IEG considers the following lessons from the preparation and implementation of ERT 1 to be especially noteworthy: • Preparation of a project relying, as does this one, on significant private sector participation should include a thorough assessment of the likely private sector response, both locally and internationally, as well as of the removal of barriers, enhancement of incentives, and guarantees required to enhance such a response . • Preparation should also include an in -depth evaluation of the capacity of the Borrower ’s agencies to implement a complex operation of this kind, and to ensure that key institutions are or will be established, and crucial managerial positions filled, before the project is presented to the Board . Where such capacity is missing, design should take this into account • Proper implementation and coordination arrangements, including a central project management unit, should be put in place at the outset, especially in an operation stretching over a variety of sectors and institutions . The basing of a senior member of the supervision team in the Country Office greatly enhanced communication and collaboration between the Borrower and the IDA team . 14. Assessment Recommended? Yes No Why? To verify the ratings and reassess the impact of the program in the light of private investor response in subsequent phases. 15. Comments on Quality of ICR: While the ICR has some positive elements, it contains shortcomings which, taken together, lead to an unsatisfactory quality rating. First, there is no attempt to calculate economic rates of return for even a sample of sub -projects, which is a critical gap in a project such as this one . Second, there is no analysis of compliance with safeguards policies, though the PAD recognizes that the project was expected to have potentially significant effects on the environment of rural Uganda and also involve resettlement and land acquisition . Furthermore, the ICR should have made a better attempt to relate the evidence on key performance indicators to the PDO . There are also a number of contradictions – for example, a small hydro plant is described as under construction in one place and in operation in another, while the Risk to Development Outcome is rated moderate in the Data Sheet and “substantial� (presumed equivalent to significant) in the text. a.Quality of ICR Rating : Unsatisfactory