ICRR 14484 Report Number : ICRR14484 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 10/14/2014 Country : Indonesia Project ID : P099757 Appraisal Actual Project Name : Geothermal Power US$M ): Project Costs (US$M): 9.00 3.86 Generation Development L/C Number : Loan/ US$M): Loan /Credit (US$M): 4.00 1.45 Sector Board : Energy and Mining Cofinancing (US$M): US$M ): Cofinanciers : Board Approval Date : 05/29/2008 Closing Date : 06/30/2011 06/30/2013 Sector (s): Renewable energy (50%); Power (25%); Central government administration (25%) Theme (s): Climate change (67% - P); Other financial and private sector development (33% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : Mamundi G. Sri-Ram George T. K. Pitman Christopher David IEGPS1 Aiyer Nelson 2. Project Objectives and Components: a. Objectives: The Project Appraisal Document (PAD page 3) states that the development objectives were : "to promote the expansion of economic and environmentally friendly geothermal power in Indonesia and reduce CO2 emissions from the power system". The GEF Grant Agreement (page 63) has an identical statement of objectives . The PAD (page 34) states that the Global Environment objective (GEO) of the project was: “to promote on-grid electricity from geothermal sources, to reduce the need for coal -based generation capacity, and to avoid associated greenhouse gas emissions .� b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: The project had four components : 1. Policy Framework for Scaling up the Development of Geothermal Power . (Cost estimate at appraisal US$ 3.60 million; at completion US$ 0.81 million). Assist GOI in developing and implementing an integrated set of policies that would provide sufficient regulatory certainty, risk mitigation and economic incentives for increased public and private investments toward developing geothermal power in Indonesia. The three key areas of assistance proposed were : 1. Development and implementation of policy to address incremental costs . 2. Development and introduction of upstream risk mitigation measures . 3. Support the implementation of the Geothermal Law . 2. Transactions Management for Mobilizing Investments in Geothermal Power Generation . (Estimate at appraisal US$ 3.35 million; at completion US$ 2.71 million). Assist GOI (especially the Ministry of Energy and Mineral Resources (MEMR)) to develop the capacity for planning and transacting geothermal power developments in an efficient and transparent manner . Two main activities were planned: 1. Expanding power generation in geothermal fields already allocated to investors . 2. Facilitating transactions of new geothermal fields for power development . At the time of restructuring, approved on June 22, 2011, a new activity was added--Data upgrade and validation for geothermal green fields. 3. Geothermal Sector Technical Capacity Building . (Estimate at appraisal US$ 1.65 million; at completion US$ 0). This aimed to address the limited domestic technical capacity for handling most geothermal related activities, and support the sector's long term development prospects . Three activities were envisaged : 1. Training of government officials and technical staff in planning and transaction management . 2. Building awareness among stakeholders . 3. Options for long term cost reduction . 4. Project Management Assistance . (Estimate at appraisal US$ 0.40 million; at completion US$ 0.34 million). Provide the necessary technical consultant support to the Directorate of Geothermal Enterprise Supervision and Ground Water Management, the executive implementation unit, for management and supervision of the project . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Costs : At completion, the total expenditures, including the government contribution for Component 1 were 23% of the appraisal estimate, while Component 2 expenditures were 81% of the appraisal estimate. No funds were spent on Component 3 and for Component 4 the final expenditures were 85% of the appraisal estimate. GEF funds were also reallocated from Components 1 and 3 to Components 2 and 4, as some US$ 654,850 were freed up due to cost savings from contracts under some activities --of this amount, US$ 600,000 were allocated to the new activity to upgrade and validate data for green field geothermal fields, proposed for tender under Sub-component 2.2. The remaining US$ 54,850 was reallocated to Component 4 to improve overall project implementation. The project objectives and expected outcomes of each activity remained unchanged. Financing : The GEF Grant was US$4.00 million. The final amount of the GEF grant disbursed was US$ 1.45 million, or some 36% of the appraisal amount. The undisbursed amount of US$2.55 million was returned to the Trust Fund.. Borrower Contribution : The Government contribution, originally anticipated at US$ 5 million equivalent, was US$ 2.41 million at completion, or 48% of the appraisal amount. Dates : A level-2 restructuring in June 2011 extended the Closing Date of the GEF grant from was extended from June 30, 2011 to June 30, 2013 to compensate for initial delays and allow time for completion of critical activities under the project.. 3. Relevance of Objectives & Design: a. Relevance of Objectives: Substantial Project objectives were relevant to the country ’s needs and global concerns . Indonesia has the world's largest geothermal power development potential, estimated at about 27,000 MW. The Government of Indonesia planned to achieve 6,000 MW of installed geothermal capacity by 2020, up from about 970 MW (4% of the identified potential) in 2007. At appraisal there was nearly 1,000 MW of unexploited geothermal potential under private control, and over 3,000 MW with state owned enterprises. Geothermal power was seen by the government as a suitable base load substitute for coal fired capacity in many areas, with much lower emissions of air pollutants and green housegreenhouse gases . In 2006, GOI initiated a Master Plan Study for Geothermal Power Development funded by JICA, further advancing the knowledge and understanding about the resource base. The main barriers deterring investment in this sector were : (i) lack of an adequate policy framework that incorporates environmental benefits of geothermal energy and provides sufficient economic incentives for investing in the sector; (ii) lack of government planning and management capabilities to efficiently conduct transactions in geothermal power projects; and (iii) lack of domestic technical capabilities to support the sector's long term growth. Benefits of greater utilization of geothermal resources for power include improved local and global environment, enhanced energy security using a non -tradable indigenous resource, and an improved mix to serve as a hedge against volatility in fossil based commodity prices . The Bank was supporting GOI with infrastructure and energy sector reform and was requested to assist with the project. This was an expansion of Bank activity underway in the geothermal sector such as the 2003 Geothermal Law, and with the reform agenda which identified barriers preventing greater investment in the sector . The Bank was also facilitating a carbon finance transaction in the country . The project was consistent with the strategic objectives in the Bank's 2004-2007 Country Assistance Strategy as well as the GEF -4 's climate change strategic objective to scale up geothermal power . focal area. It would supportProject objectives were aligned with the CAS strategy that aimed to of helping to improve : (i) the climate for high quality investments, and (ii) service delivery by the utilization of a more efficient energy resource . At exit, tThe project objectives remains aligned with the Bank's Country Partnership Strategy for FY 2013-15 , which calls for engagement in areas that are Pro-Poor, Pro-Growth and Pro-Green. The project also addresses country needs in the Bank's 2013 Energy Sector Directions Paper, which considers renewable energy including geothermal as a priority area for securing affordable and sustainable energy supply needed to end extreme poverty and promote shared prosperity . The project still remains relevant today, b. Relevance of Design: Modest The components financed a set of activities to develop the policy framework including a pricing system, economic incentives, transactions management, and technical capacity building, all of which were directly linked to the objectives. The results framework (PAD, p 21-22) consisted of two outcome indicators : (i), installed geothermal power capacity resulting from the policy reforms and investment transactions developed through the project;, and (ii) reduction in CO2 emissions by offsetting fossil fuel based power with geothermal power . These two outcome indicators are of the kind that would take much longer than the project life, as the policy reforms will have longer term impacts on installed geothermal power than the project period --investment transactions benefitting from this kind of project take several years to develop, and the same could be said for the reduction of offsetting CO2 emissions. Also, the reduction of CO2 emissions is a direct consequence of the expansion of geothermal power use (the first PDO), rendering the second outcome indicator superfluous . In all, the high level of specificity of the outcome indicators was not appropriate . The intermediate outcomes and indicators for each component, tracked specific outputs resulting from the inputs/activities under each component, and their use . The causal chain between funding and outcomes are clear and well thought for expansion of geothermal power development . The PAD (p 10-11) also contains an assessment of potential risks and mitigation measures . Perhaps optimistically, the overall risk was assessed as moderate. As the ICR rightly points out, it is questionable whether these activities could should have been phased rather than in into a single project, considering the nascent stage of geothermal power development and the lag in achievement in Indonesia relative to original plans made in 1991, albeit affected by the financial crisis . 4. Achievement of Objectives (Efficacy): To promote the expansion of economic and environmentally friendly geothermal power in Indonesia and reduce CO2 emissions from the power system . The statement of objectives contains two objectives and the efficacy of CO2 these are individually discussed below : (a) To promote the expansion of economic and environmentally friendly geothermal power generation in Indonesia . Rating Modest . Outcomes : The results framework in the PAD calls for a target 350 MW of installed geothermal power by Year 3 of the project, starting with a baseline of 0 in Year 1 (05/01/2008). This target was not revised at restructuring . At completion (06/30/2013), the actual value of installed geothermal power remained 0, and thus, the target value was not achieved. Geothermal investments benefitting from this type of project involving public policy and institutional reforms and private sector investment typically take over 5 years to develop. The investment climate, to which the project was to contribute, is viewed by stakeholders as stalled and so far, no investment project has benefitted from project interventions during the life of the project . Even so, the project did open the door for Bank engagement in the geothermal sector in Indonesia, strengthened the working relationship with relevant government institutions in the energy sector and paved the way for the subsequent geothermal investment (one of the first among all MDBs) that the Bank approved in late 2011 to support the development of two geothermal fields with a total capacity of 150 MW, involving US$ 125 million of Carbon Trust Fund (CTF) loans and US$ 175 million in IBRD lending. Outputs : Three key barriers identified in the PAD have been addressed, namely identifying pricing and incentive policy options, establishing a geothermal risk mitigation mechanism, and developing standard documentation for geothermal transactions . The government has issued a Geothermal Law and a number of regulations to support the Geothermal Law ’ s implementation. Institutional capacity and awareness were significantly enhanced, with the creation of a new unit in the Ministry of Energy and Mineral Resources, responsible solely for geothermal development, with a core team of professionals trained under the project . Information dissemination to enhance awareness about sector policies and business opportunities was conducted at 5 to10 stakeholder dialog seminars. A Geothermal Fund to overcome geothermal resource risks was established and capitalized with US$ 300 million budget funding. Model procedures and standard documentation for competitive bidding of geothermal transactions were developed, and improved transaction management skills and knowledge of relevant officials were partially achieved through discussions and stakeholder workshops during the preparation of the model procedures . A pricing mechanism to provide adequate economic incentives was only partly developed, Training through workshops and seminars for government agencies responsible for undertaking geothermal transactions was partly completed . Three planned outputs were not achieved : (i) structured offers to mobilize investments for additional geothermal power in fields that are controlled by existing operators; (ii) implementation of a pilot transaction for one power project in a new geothermal field competitively tendered based on the new Geothermal Law; and (iii) formulation of a strategy for domestic technology development . (b) To reduce CO 2 emissions from the power system . Rating Negligible . Outcomes The target value of 2 million tonnes per year for reduction in CO 2 emissions by offsetting fossil fuel based power generation (i.e., coal) with geothermal power was not achieved, as no geothermal power investments had resulted from the project interventions . Outputs None. Progress towards achievement of the GEO, namely to promote the expansion of economic and environmentally friendly geothermal power generation in Indonesia and reduce CO 2 emissions from the power system are also very limited, due mainly to the unrealistic expectations under the project as to the speed with which they could be achieved. 5. Efficiency: Economic and Financial Efficiency . Because the project only assisted the GOI to reform policies and enhance institutional capacities, and did not directly finance any investments, an economic and financial analysis of the project is not applicable. Administrative Efficiency . The project's key activities were delayed . Many planned activities were not carried out, despite a 2- year extension in the Closing Date --a formal mid term review might have helped resolve some issues. Oonly 36% of the available project funds were utilized . Compared with appraisal estimates, utilization of funds was zero for Technical Capacity Building, 23% for the Policy Framework, and 85% for Project Management Assistance. For activities that were implemented, efficiency is satisfactory, but several could not be implemented because of deficiencies in institutional procedures . The two main consulting contracts were competitively awarded at lower prices than the original appraisal estimates due to the internationally competitive selection process. Overall Efficiency is rated Modest . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The relevance of project's objectives is rated as Substantial and its design as Modest . Efficacy of the first project objective is rated Modest and that of the second objective, Negligible . Efficiency is also rated as Modest . Overall the project outcome is Unsatisfactory . a. Outcome Rating : Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: The government targets for development of geothermal power to substitute for fossil fuel power are ambitious . Coordination among government agencies left key gaps in terms of policy /regulations and institutional capacity and performance. While the outcome indicators for installed geothermal power and CO 2 emission reduction were not very appropriate, and not achieved, projected intermediate outcome indicators were only partially achieved because several planned activities were cancelled . Notwithstanding the Government’s commitment, unless there is a rapid change in the operating environment in several areas, further development of geothermal electric power in Indonesia is likely to be slower than GOI desires . The key risks are as follows : Policy and Institutional: International experience shows that the first prerequisite for a successful renewable energy program is the development of a transparent mechanism to recover incremental costs . The 2012 feed-in tariff introduced by the Ministry of Energy and Mineral Resources --without adequate consultation with the Ministry of Finance--could not be implemented as it conflicted with other existing regulations . Unless this incentive policy issue is properly addressed, it will pose a risk for projects now under development, and to new ones . The Stakeholders Advisory Group , proposed in the PAD as a mitigation measure to reduce the risk of inadequate inter-governmental coordination did not function during project implementation . This risk remains. Financial: Geothermal Law. (i) The tendering process required by the Law has several deficiencies, most importantly that bidders are required to estimate electricity prices typically 7-9 years into the future based on inadequate below-surface data; (ii) Poor pre-qualification standards have led to unqualified bidders setting unrealistically low prices that cannot be achieved . Geothermal Fund. A satisfactory model for use of these resources has yet to be developed, and so no disbursements have been made . Access to finance for private sponsors is highly limited . After the Closing Date, the Ministry of Energy and Mineral Resources discussed continuing Bank support for the unfinished activities. An Action Plan with support from the World Bank and Asian Development Bank was adopted by the Ministry to review the feed -in tariff regulation. Also, IFC expressed interest in supporting the geothermal demonstration transaction after the new tariff scheme is approved . a. Risk to Development Outcome Rating : High 8. Assessment of Bank Performance: a. Quality at entry: Project preparation was based on sound analytic work carried out by the Bank and other donors, and on international experience, and the barriers hindering expansion of geothermal power in Indonesia were correctly identified. The Bank mobilized a strong technical team and additional resources for preparation, and related upstream analytical work. The risk of delays due to government budget procedures was not foreseen or adequately mitigated. As the ICR points out, in hindsight, given the complex government procedures and the nature of policies to be addressed, the activities should have been phased over more than a single project . The outcome indicators as stated are likely to take a few years to materialize, and appraisal was both unrealistic and over-ambitious. at -Entry Rating : Quality -at- Unsatisfactory b. Quality of supervision: Regular Bank supervision missions monitored project progress, to help resolve outstanding issues, except for a gap in 2012 when the Task Team Leader (TTL) changed. Supervision findings were documented and shared with the Borrower in Supervision Reports . The Bank team also mobilized additional resources to help implement some project activities, but could not move forward because of capacity issues in the Ministry of Energy and Mineral Resources. Bank fiduciary specialists regularly provided guidance and support to the Project Management Unit. The Bank undertook a project restructuring, and extended the Closing Date by two years to allow additional time to carry out planned project activities . However, there was no formal mid-term review to address design and implementation issues . Timely attempts to address design flaws and implementation issues to better utilize project resources seem to have been ineffective or absent . The Bank team did not follow up on capacity building for the Project Management Unit, notably after restructuring of the Ministry . Inadequate oversight was also a factor in the undue expiration of two major consultant contracts . The outcome indicators were not revised in 2009 when all projects in the Bank were to systematically review PADs and Legal Agreements and correct inconsistencies and mistakes . This is all particularly noteworthy given the large Resident Mission in Jakarta. According to the TTL, project supervision was conducted by HQ staff until 2012, after which it was taken over by the field office staff in Jakarta . Quality of Supervision Rating : Unsatisfactory Overall Bank Performance Rating : Unsatisfactory 9. Assessment of Borrower Performance: a. Government Performance: The GOI's strong commitment to expansion of geothermal power was evident in its adoption of the second Fast Track Program in 2008, aimed at developing an additional 4,000 MW of geothermal generation capacity; and the Geothermal Fund Facility (GFF) in 2011. But the commitment was not followed up by effective actions during the project period. The restructuring of the Ministry of Energy and Mineral Resources, leading to the establishment of the Geothermal Directorate in the midst of project implementation accompanied by frequent changes in its leadership, resulted in high project counterpart staff turnover and inconsistent high level support for the project. Besides several changes in responsible government departments, coordination and consultation among relevant government institutions was inadequate, hampering policy making and the viability of regulations issued. The budget procedure was complex, rigid and time consuming, making it difficult to effectively utilize available project funds and delaying payments for completed activities . Government Performance Rating Moderately Unsatisfactory b. Implementing Agency Performance: Project Management Unit staff was highly committed despite frequent changes in organizational structure and leadership. Despite a heavy workload, the staff closely reviewed consultants' outputs, pressing for high quality deliverables. Since the project was subject to both Bank procurement procedures and those of the Government, they also learned about Bank fiduciary processes, diligently reconciling differences between the two. In parallel they completed an enormous work program of their parent Ministry, alongside activities planned under the project. However, the lack of strong and consistent high level support and internal coordination hindered their effective performance . Inadequacies among representatives of other related sub-directorates in the Project Management Unit slowed down decision making at mid -level management. Inter-government coordination, which is essential to policy and regulation making was inadequate, affecting the project's outcome. In addition weak financial controls and management created problems throughout implementation. Implementing Agency Performance Rating : Moderately Unsatisfactory Overall Borrower Performance Rating : Moderately Unsatisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The M&E design was anchored by the results based framework, but while the objectives were clear, achievement of the outcome indicators chosen would require far longer than the project implementation period . The intermediate outcomes and indicators for each component tracked specific outputs resulting from the inputs/activities under each component, and their use . The proposed data collection methods and analysis, and their use are also well presented . The Project Management Unit was responsible for collection of qualitative and quantitative data, for assessment of the relevance and effectiveness of proposed interventions on an annual basis. The Project Management Manual required quarterly reports from the Project Management Unit to cover Financial Management, Procurement Management and Project Progress . b. M&E Implementation: The borrower/implementing agency and Bank teams performed the M&E throughout the project period . Monitoring was carried out during supervision missions and supported by reports from the Project Management Unit which included project progress, procurement management and financial management . c. M&E Utilization: The progress reports were used to assess and evaluate project performance . Inputs, outputs and timelines were tracked. However, it seems any corrective actions that resulted did not have much effect on eventual project performance. Delays in procurement processes by the Ministry of Energy and Mineral Resources prompted the Bank team to secure alternative funding to address geothermal resource risk with a Bank -executed grant. M&E Quality Rating : Modest 11. Other Issues a. Safeguards: The project is classified as environmental assessment Category C under OP /BP 4.01. An Environmental Assessment was not required. The project envisaged production of a guidance note on safeguards obligations for geothermal investment transactions involving the public or private sector . This guidance note was not prepared as the pilot transaction was not implemented . b. Fiduciary Compliance: Procurement : The original procurement plan included eleven packages including five large contracts valued at US$ 200,000 to 1.7 million using Quality and Cost Based Selection (QCBS). The Ministry of Energy and Mineral Resources experienced significant procurement delays, taking nearly two years to have the two key consultant contracts signed in April and May 2010 respectively. However, in February and April 2011 these two contracts had expired while the services had not been fully delivered . The Ministry tried to use single source selection to appoint the same two consultants to complete their work, but given strict conditions on single source contracts, it took a further six months for these contracts approved and signed . Slow completion of procurement actions, inadequate quality assurance and monitoring of the procurement process, and weak contract management undermined project performance. By the end of the project period, two key contracts were signed and implemented. In addition, a number of local individual consultancy contracts to support the Project Management Unit were signed and implemented. Financial Management : This element was moderately unsatisfactory . Serious challenges were faced including (a) delays in approval of the annual budget; (b) recurrent delays of financial management reports; (c) weak internal controls reflected by audit findings, in particular related to project monitoring and payment verification for consultancy services; (d) lengthy follow up of audit findings with some remaining unresolved until the Closing Date. According to the TTL, these, these are apparently still being worked upon . While some problems were partially addressed in December 2011, project financial management was weak until the end of the project, with overdue payments until the Closing Date . The Ministry finalized the revised budget and made the payment by end December 2013--the Closing Date was extended by two months to permit disbursement . c. Unintended Impacts (positive or negative): d. Other: 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Unsatisfactory Unsatisfactory Risk to Development High High Outcome : Bank Performance : Unsatisfactory Unsatisfactory Borrower Performance : Moderately Moderately Unsatisfactory Unsatisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The ICR presents four lessons and the most important was : In preparation for public-private partnerships and investments all the necessary regulatory and financial incentive mechanisms should be put in place and discussed with all stakeholders before project preparation even begins. This is necessary when private investors are expected to come forward with financing for the investments expected to result from project interventions in order to reduce risks and cost of entry, as for example in geothermal power . IEG adds another lesson: Before considering support for projects in renewable energy, particularly in geothermal power, it is important to establish that data on below -the-ground conditions are reliable and accepted by the investor community. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR provides good quality evidence, well summarized and with adequate information on details . The approach is analytical, and the conclusions are based on actual information and implementation details . a.Quality of ICR Rating : Satisfactory