IEG Report Number: ICRR14655 ICR Review Independent Evaluation Group 1. Project Data: Date Posted: 05/29/2015 Country: Guinea Project ID: P074288 Appraisal Actual Project Name: Decentralized Rural Project Costs (US$M): 17.00 6.82 Electrification Project L/C Number: C3685 Loan/Credit (US$M): 5.00 5.32 Sector Board: Energy and Mining Cofinancing (US$M): 2.00 0.43 Cofinanciers: GEF Board Approval Date : 07/02/2002 Closing Date: 12/31/2006 06/30/2013 Sector(s): Renewable energy (70%); General finance sector (30%) Theme(s): Rural services and infrastructure (40% - P); Pollution management and environmental health (20% - S); Climate change (20% - S); Other financial and private sector development (20% - S) Prepared by: Reviewed by: ICR Review Group: Coordinator: Fernando Manibog Lauren Kelly Christopher David IEGPS1 Nelson 2. Project Objectives and Components: a. Objectives: According to Schedule 2 of the Development Credit Agreement (DCA) dated July 8, 2002, the Project Development Objective (PDO) is "to assist the Borrower in increasing access to electricity in rural and peri-urban areas, through the establishment of policies, incentive systems, and institutional mechanisms to promote: (i) the facilitation of private investment in energy projects in rural communities; (ii) the establishment of community-based, decentralized and affordable village electrification schemes; (iii) the establishment of independent credit sources and technical support structures; and (iv) the promotion of renewable energy technologies, wherever feasible." The Implementation Completion and Results Report (ICR) is based on a different formulation of the PDO coming from the Project Appraisal Document (PAD) dated April 22, 2002 (page 3), as follows: "The project will test institutional, financial and delivery mechanisms to promote better access to electricity in rural and peri-urban areas. In support of that objective, the Project seeks to: (a) Test institutions, regulations and delivery mechanisms to develop decentralized and affordable village electrification schemes; and (b) Test financial mechanisms to deliver increased access to electricity and mobilize private sector financing for energy projects in rural communities." The project also included a Global Environment Objective (GEO): "to promote the adoption of renewable energy technology by removing barriers and thus mitigate CO2 emissions." Following IEG guidelines, this IEG assessment of the ICR is based on the DCA statement of objectives. b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: The project included the following components: Component 1: Capacity Building (US$2.2 million appraisal estimate; US$1.4 million actual cost) This component provides technical assistance (TA) for:  The implementing agency, the Decentralized Rural Electrification Office (BERD): to define priority areas; evaluate proposals for Decentralized Rural Electrification (DRE); coordinate, supervise and monitor the implementation of DRE projects; and evaluate, disseminate and replicate activities  Private providers: to identify and set up the delivery of electricity services under concession arrangements; and provide training in the installation and maintenance of equipment  Village associations: to encourage them in organizing service delivery through an operator Component 2: Financing Mechanism and Implementation of the Decentralized Rural Electrification (DRE) Program (US$11.3 million appraisal estimate; US$2.21 million actual cost) This component includes the establishment of the Fund for Decentralized Rural Electrification (FERD) intended to address the high up-front costs of rural electrification systems and the lack of long-term credit. the FERD would: (i) fund the operational cost of managing the BERD; and (ii) provide loans and grants to promoters of RE projects. Component 3: Project Coordination and Management (US$1.55 million appraisal estimate; US$2.56 million actual cost) This component is intended to support and strengthen BERD's operational capacity to coordinate, supervise, and monitor project implementation, including the provision of qualified staff, training, vehicles and equipment. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost Including physical and price contingences, the total appraisal cost was US$17 million, of which IDA committed US$5 million, the GEF US$2 million, the Borrower US$1.1 million, and US$8.9 million from local sources. Component 2 on the FERD financing mechanism accounted for 66% of the total cost, or US$11.3 million, of which only 20% was actually disbursed. The actual total project cost was US$6.82 million. Project Financing and Borrower Contribution IDA financing amounted to US$5.32 million (actual). Although the Government committed to provide US$1.1 million in counterpart funds, disbursements were delayed and it only contributed US$0.78 million. Only US$427,000 of the GEF funds were disbursed from the initial amount of US$2 million, given the very low level of demand for RET subprojects due to the terms that were offered, including the high up-front costs (see Section 4 on Efficacy of the GEO). Dates Credit effectiveness was delayed by one year because of the extensive negotiations between BERD and a commercial bank (BICIGUI) to enter into an agreement for channeling IDA and GEF funds to the private operators as loans and grants. The project's closing date was extended three times, for a total of six and one half years from the original closing date of December 31, 2006. The first two extensions were the result of delays in implementation, while the third extension was due to the suspension of Bank operations in 2009 and 2010 after a military coup. The project finally closed on June 30, 2013. 3. Relevance of Objectives & Design: a. Relevance of Objectives: Substantial At the time of appraisal, the project development objectives (PDOs) were substantially relevant and directly aligned with the Government’s power sector strategy, which emphasizes the promotion of decentralized electricity supplies and the mobilization of private sector financing for electricity sector investments. The project's focus on electrification is directly relevant to the development of its high export and tourism potential. However, given the nature of the LIL instrument, the formulation of the PDOs should have made more explicit recognition of the underlying goals of testing and promoting innovations, learning lessons on what works and what does not from the project's activities, and adapting successful interventions. The PDOs remain substantially relevant today, given the continuing energy deficits in Guinea. In May 2013, the government endorsed a third poverty reduction strategy for the period 2013-15 around four main priorities, of which two are directly linked to the project's decentralized electrification goals: (i) service delivery; and (ii) reducing regional disparities through local development and decentralization. The 2014-2017 Country Partnership Strategy (CPS) continues to place strong emphasis on diversified and inclusive growth, regional integration, and the promotion of small and medium enterprises as well as an enabling business environment, all of which require electrification programs. More specifically, the CPS indicates that: "From a sectoral perspective, proposed operations are mainly focused on agriculture and energy." (page 17) b. Relevance of Design: Modest The project is a Learning and Innovation Loan (LIL) operation. The Bank defines a LIL as "loans of US$5 million or less, financing small, experimental, risky and/or time-sensitive projects in order to pilot promising initiatives and build consensus around them, or experiment with an approach in order to develop locally based moels prior to larger-scale interventions." The project adopted a phased approach, which was relevant to: (i) the reality that the project's DRE financing mechanism was still a very new concept in Guinea; and (ii) the perception that DRE investments were risky, especially since there had been no previous examples of sustained profitability. To mitigate these, the project was designed to subsidize capital costs, select only mature and commercially available technology options, provide a regulatory framework, and establish a mixed public-private steering committee. However, the PAD did not include a Results Framework. Consequently, it was difficult to establish and validate the causal chain between the project's activities and the outcomes related to learning, adaptation, innovation and utilization of the project's lessons related to what DRE mechanisms worked and what did not. 4. Achievement of Objectives (Efficacy): Substantial Within this LIL's over-arching, long-term goal of assisting "the Borrower in increasing access to electricity in rural and peri-urban areas", the project aimed to test various institutional, financial and service delivery mechanisms. Based on the lessons that would be derived from these interventions that were designed to generate learning and innovation, the project would then promote electricity access through four possible service delivery mechanisms: (i) private power investments in rural areas; (ii) community-based electrification schemes; (iii) access to independent financial and technical assistance sources; and (iv) the promotion of renewable energy while also reducing CO2 emissions. Designed as a LIL, important lessons were learned across the various service delivery mechanisms, although more information should have been provided about the various institutional and political incentives that either fostered or derailed the aim of increasing electricity access to the rural and peri-urban areas of Guinea. The project’s key innovations and lessons – clustered below as Learning Areas -- were derived from testing the four different service delivery mechanisms along with their underlying policy, institutional, and design choices, with a view to promoting what worked, adapting and improving upon what did not, and eventually scaling-up. Learning Area: Private Investment in Electricity in Rural Area - substantially achieved Innovations: The project tested the promotion of Private Sector Investment in Rural Areas by providing technical assistance and training to private operators in areas such as: financial modeling; financial indicators; small enterprise management in rural areas; specialized training in small hydro investments and on how to conduct site visits for projects, including management aspects; and maintenance and follow-up of production equipment. A key innovation here was the creation of the Fund for Decentralized Rural Electrification (FERD) that was intended to address the high up-front costs of rural electrification systems and the lack of long-term credit. The project provided FERD staff with training on the processing loan application, the development of indicators, supplemented through study tours and workshops. The project also promoted private investment in electricity in rural areas by supporting the legal and regulatory framework, and by establishing and developing the Decentralized Rural Electrification Office (BERD). The aim was to help the BERD learn how to define priority areas; evaluate proposals; coordinate, supervise and monitor the implementation of projects; and evaluate, disseminate and replicate activities. Learning: A mix of regulatory incentives and technical investments and training within a body dedicated to promoting rural electricity access within the Government of Guinea proved partially effective in incentivizing private sector investment in electricity in rural areas. The BERD and its steering committee continue to operate, having demonstrated their capacity to support DRE projects, as evidenced by 26 DRE projects that continue to operate. This signals that the regulations governing them provide the private operators with sufficient assurance that the DRE sub-projects would be financially sustainable. However, there were serious Government delays with respect to the adopting the Rural Electrification Law (which was adopted by the National Assembly only in October 2013), and by project closing, the implementing decrees have not yet been developed. Plans to transform BERD into a permanent rural electrification agency also did not materialize. For learning purposes, the ICR should have provided more information on the institutional and political obstacles to developing a more effective enabling environment for rural electricity. An important area of learning indicated in the ICR is that implementation of RETs should take into account their high capital costs, together with the greater demands and risks on operators. As a result, PV projects would not appear to be well suited to a decentralized approach where small private operators undertake electrification at the village level. Other models, such as the dealer model or the concessioning of services to a large enterprise, may have better prospects of success because more experience and resources can be provided by larger firms, as evidenced by the satisfactory implementation of other Bank-supported projects in poor countries (e.g., Nicaragua and Bolivia). Learning Area: Community Based Decentralized and Affordable Village Electrification Schemes - substantially achieved Innovations: Under this scheme, a potential investor/operator (often an existing business from the region) registers its interest in setting-up an electricity production and distribution facility in a village without power by applying to BERD. BERD provides technical support and training, including feasibility analyses and design. If the project proposal is deemed feasible, the Ministry of Hydraulics and Energy (MHE) awards the investor/operator a concession contract to supply electricity within a defined service area, thereby legalizing its operation. The investor/operator provides his equity contribution, which is deposited in the commercial bank supporting the project; and the investor/operator receives a loan and a subsidy channeled through the commercial bank managing the funds on behalf of the BERD. Learning: By project closure, 35 DRE sub-project applications had been processed, resulting in 32 signed agreements. Of these, 26 micro-grids were in operation while 3 were under construction. This represents a 90% achievement rate. Activities for the remaining 3 signed agreements never materialized. At project closure, the access rate achieved in electrified concessions was 58%, which is lower than the 86% target. In terms of learning, it became clear that there was an over-estimation of the poor people’s ability to pay for electricity and an under-estimation of the cost of electrification in rural areas where houses are spread-out. Moreover, although the sub-projects have resulted in 14,116 available connections, a lower level of 8,248 connections are being used; the balance was to be completed within two years after project closure. The ICR should have provided more information about the obstacles facing the poor, especially for those that had gained access but who were not able to make use of it. An M&E system was established, albeit late in the project implementation period, which may have decreased the learning value of the project's interventions. Learning Area: Independent credit sources and technical support structures - modestly achieved Innovations: The project established a financing mechanism for DRE -- the Fund for Decentralized Rural Electrification (FERD) -- that was intended to subsidize the high up-front costs of rural electrification systems and the lack of long-term credit. The FERD would: (i) fund the operational cost of managing the BERD; and (ii) provide loans and grants to promoters of RE projects. FERD was established and managed by BICIGUI, a commercial bank, while receiving financial support from the Government and the project. Learning: Several lessons were learned about setting up an independent financing mechanism for rural electrification in a fragile state environment, with low institutional and financing capacity. Government commitment, for example, was needed to incentivize and sustain the deal. In this case, the Government supported FERD but its actual contribution was only 71 percent of the level established during the project design stage. The fee basis (4 percent of outstanding loans) that BERD agreed with BICIGUI for that commercial bank to manage the loan and grant funds was flawed, as it was not tied to BICIGUI's performance. Instead, the bank had incentives to lend as much money as possible, without focusing on repayments from the DRE operators. By project closure, loan collection rates were only around 59%. According to the ICR (page 13): "the financial mechanism for recovering the loans to project operators/investors has important weaknesses in its design and cannot sustain a revolving fund for further electrification." A major learning area indicated in the ICR is that projects in which private financial institutions are expected to play a significant role during implementation should include a detailed assessment with potential interested commercial banks about the conditions under which they are expected to perform; on this basis, the incentives of the institutions need to be aligned with those of the project. Global Environmental Objective Negligible Innovation: Introduction of renewable energy technologies (RETs). Learning: The GEO was to promote the adoption of RET by removing barriers and thus mitigate CO2 emissions. In terms of learning, no RET sub-projects went into operational service because the high up-front cost of RETs discouraged local investors. For the only one pico-hydro project (Bofossou) that was supported, construction was not completed and by project closure the developer was looking for alternative sources of finance to complete construction. The one effort to support photovoltaic (PV) projects encountered procurement delays and was terminated before the delivery of any PV equipment. Consequently, no CO2 emissions reduction was achieved under the GEF funding. 5. Efficiency: Modest Administrative and Implementation Efficiency As from effectiveness in mid-2003, about two years lapsed just to set up the software and procedures assessing and designing DRE projects. The process of seeking an agreement with a commercial bank to manage the funds for a fee involved extensive negotiations and delayed the start-up of project implementation considerably. M&E arrangements, which are important for a LIL to maximize learning by doing, was also delayed. The first two extensions were related to implementation delays, which clearly had a direct influence on the low rate of disbursements. This small LIL took 10 years to complete. Economic Rate of Return (ERR) The ICR estimated the project's ERR as 60% compared to the PAD estimate of 7.5%. The large difference is due to the PAD's use of the incremental cost methodology, which was developed by GEF to conduct the economic analysis of renewable energy technology projects and to assess eligibility for GEF funding. The ICR used the Bank's ERR methodology, which captured the consumer surplus as well as the large benefits accruing from the replacement of traditional lighting cased on high-cost kerosene. For example, a kerosene lamp only produces about 10% of the luminance from a compact fluorescent lamp (CFL), yet monthly kerosene charges are three times that of CFLs (ICR page, 16).The possibility of new services, such as television, was reflected in the high willingness to pay. a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal Yes 7.75% 100% ICR estimate Yes 60% 100% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The relevance of the project's objectives is substantial and the relevance of its design is modest. Efficacy is substantial, taking into account the innovations introduced and lessons learning regarding the following (with the level of achievement indicated): (i) facilitating private investment - substantial (ii) promoting decentralized and affordable electrification schemes - substantial (iii) promoting independent credit sources and technical support structures - modest While the ex-post ERR is high, the project was seriously hampered by implementation delays that led directly to very low disbursement rates, thus resulting in modest project efficiency overall. a. Outcome Rating: Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: The ICR (page 17) indicates the following positive factors in assessing risks to development outcome:  BERD has achieved a significant amount of learning about the challenges and pitfalls of rural electrification and how to address them. While this could be lost if political instability returns, Guinea’s prospects in this regard seem to have improved.  The financial mechanism established through the FERD is likely to be more successful as long as reforms to improve its sustainability are put in place, with support from other donors and the Government.  The risk associated with investments in electrification sub-projects is low, given the diversification among multiple small companies, most of which are being managed with private sector criteria and run by trained operators. However, the following factors raises significant risks:  BERD is not a permanent agency.  There is still a need to partially subsidize the high up-front costs and obtaining such subsidies can be difficult.  Implementing decrees for the Rural Electrification Law still need to be promulgated and adopted.  The use of connections is lower than the number available.  BICIGUI's loan collection rates are low.  Removal of barriers to RETs had failed. a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: a. Quality at entry: The project was prepared based upon studies supported by ESMAP regarding the possibilities for rural electrification in several African countries. It also incorporated lessons learned from other Bank projects in Guinea (the Power II Project), Sri Lanka (energy service delivery), Indonesia (solar home systems), and China (renewable energy). The choice of the LIL instrument is appropriate, since there was still a need to test the innovative decentralized approach to increase rural electricity access. Alternatives were adequately considered. The results framework, however, was poorly defined. Moreover, the measures to address the risk of low loan recovery were weak. Quality-at-Entry Rating: Moderately Satisfactory b. Quality of supervision: The Bank conducted 11 supervision missions, or an average of about 1 per year during the 10-year project period. The first mission took place only in October 2004, or about 14 months after effectiveness, at which time an initiating workshop was held with the project staff to determine the strategies and procedures needed to attract operators. Given the complexities of the untested DRE approach, earlier Bank support would have helped project implementation to start more rapidly. This lack of progress of was discussed with the Government only after 2011, when there was little time left to complete the single pico hydro project. The results indicators were not modified even though two closing date extensions were granted. Procurement thresholds remained low, although this has been pointed out as an obstacle to implementation. The Bank’s interventions, such as no-objections, were considerably slow. Although the Bank's reporting was adequate, an M&E system was set up at BERD only late in the project. Quality of Supervision Rating : Moderately Unsatisfactory Overall Bank Performance Rating : Moderately Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: Early on, the Government showed its commitment by adopting a new DRE strategy, creating the BERD and planning for DRE investments, and establishing FERD as the financing mechanism. During the period of political upheaval, the Government continued to support the project and maintained the agency’s staff. However, key legislation--such as the approval of the Rural Electrification Law and the establishment of the Rural Electrification Agency--were seriously delayed. The Government also gave inadequate attention to future scaling-up requirements for this pilot project, by making FERD sustainable and transforming BERD into a permanent Agency for Rural Electrification. Although the Government originally committed to US$1.1 million as its counterpart financing, its actual contribution amounted to only US$0.78 million at project closing. There were also disputes with customs authorities, thus significantly delaying the granting of reduced taxes and import tariffs that were needed to support the project. Government Performance Rating Moderately Unsatisfactory b. Implementing Agency Performance: BERD was strongly committed to the learning objectives of the project and was closely involved in all aspects of implementation. It complied with its fiduciary responsibilities according to the rules set out in the Credit Agreement. BERD's procurement performance helped bring the sub-projects into service. However, the project's exploratory nature of the project led to delays. Although BERD was able to establish the project implementation procedures, it had difficulty with the RET component due to issues that were outside its capacity to resolve. There was a lack of RET experience in rural areas and the capital costs of RET were high, which made the implementation of this component very challenging, In the end, those challenges were not resolved and the GEO objectives were negligibly achieved. M&E procedures were also established late in the project and lacked coverage of social outcomes of electrification, which would have provided valuable data for any future scale-up efforts. Finally, although not entirely within its control, BERD did not succeed in making FERD sustainable. Implementing Agency Performance Rating : Moderately Unsatisfactory Overall Borrower Performance Rating : Moderately Unsatisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The PAD did recognize that M&E is an important tool to enhance the sustainability and replicability of rural electrification projects. As designed, Independent consultants were expected to undertake two performance reviews to evaluate the implementation experience. The areas to be covered included the economic, financial, technical, social, and environmental dimensions. However, although essential for a LIL, which is intended to promote learning, M&E arrangements were seriously delayed. b. M&E Implementation: BERD was responsible for M&E with the support of consultants. However, the actual monitoring activity only took place at a late stage in the implementation of the project with the assignment of a member of the professional staff responsible for executing it. Since the PAD indicators were of poor quality and some were difficult to measure, the indicators were refined informally during supervision, but no formal modifications were made through project restructuring. c. M&E Utilization: Given the delays, there was little or no opportunity to assess multiple consequences of rural electrification, such as such as the impact on households, on community relations, and on village economics. In the end, the learning value expected through a LIL was minimal. M&E Quality Rating: Modest 11. Other Issues a. Safeguards: The project was classified as a Category C and did not trigger any safeguard policies. (ICR, page 9) b. Fiduciary Compliance: Throughout project implementation, the financial management arrangements were implemented adequately throughout the project implementation period . The audits of the implementing agency and project financial statements were submitted regularly, albeit with some delays in a few instances. Project audits did not highlight any major irregularity. (ICR, page 10) c. Unintended Impacts (positive or negative): d. Other: 12. Ratings: ICR IEG Review Reason for Disagreement/Comments Outcome: Moderately Moderately Satisfactory Satisfactory Risk to Development Moderate Significant BERD is not a permanent agency. Outcome: There is still a need to partially subsidize the high up-front costs and obtaining such subsidies can be difficult. Implementing decrees for the Rural Electrification Law still need to be promulgated and adopted. The use of connections is lower than the number available. BICIGUI's loan collection rates are low. Removal of barriers to RETs had failed Bank Performance: Moderately Moderately Satisfactory Satisfactory Borrower Performance : Moderately Moderately Key legislation was delayed. Attention Satisfactory Unsatisfactory to future scaling-up needs was inadequate. Plans to make FERD a permanent agency failed to materialize. There were significant delays, albeit not necessarily within BERD's control. It could not resolve the challenging issues with the RET component. M&E procedures were established too late in the project. Quality of ICR: Satisfactory NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The ICR (pages 20-22) provides lessons specific to LIL operations and lessons pertinent to rural electrification (RE) and renewable energy technology (RET) operations. These are highlighted in summary form below: Lessons for LIL Operations  Adequate allowance should be given to degrees of uncertainty in choice of approach, technology and financing.  A LIL should have a well-designed M&E framework, with regular reporting, in order to internalize the learning experience of the project.  The M&E of an electrification-oriented LIL should not be limited just to the technical aspects of the project, but should also try to assess ultimate outcomes in terms of the social and economic consequences within the beneficiary communities. Lessons specific to RE and RET Projects  Implementation of RETs should take into account their high capital costs, together with the greater demands and risks on operators. As a result, PV projects would not appear to be well suited to a decentralized approach where small private operators undertake electrification at the village level. Other models, such as the dealer model or the concessioning of services to a large enterprise, may have better prospects of success because more experience and resources can be provided by larger firms, as evidenced by the satisfactory implementation of other Bank-supported projects in poor countries (e.g., Nicaragua and Bolivia).  Projects in which private financial institutions are expected to play a significant role during implementation should include a detailed assessment with potential interested commercial banks about the conditions under which they are expected to perform, aligning the incentives of the institutions with those of the project. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR is clearly written, analytically strong and candid. It provides direct and adequate evidence to support its findings and ratings. The lessons are well defined and articulated. The sector background is well analyzed and presented. The ICR is well aligned with the OPCS guidelines for ICR preparation. However, there were some inconsistencies in figures presented. For example, in Annex 1: The actual project cost (a) is shown as US$6.82 million but the actual financing provided (b) is indicated as US$7.13 million. a.Quality of ICR Rating : Satisfactory