Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Coal-Fired Generation Rehabilitation (P100101) Report Number: ICR00004323 1. Project Data Project ID Project Name P100101 IN: Coal-Fired Generation Rehabilitation Country Practice Area(Lead) India Energy & Extractives L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-76870,TF-58018 30-Nov-2014 132,085,965.82 Bank Approval Date Closing Date (Actual) 18-Jun-2009 29-Mar-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 180,000,000.00 0.00 Revised Commitment 104,137,505.07 0.00 Actual 104,137,505.07 0.00 Prepared by Reviewed by ICR Review Coordinator Group Ihsan Kaler Hurcan Victoria Alexeeva Ramachandra Jammi IEGSD (Unit 4) P100531_TBL Project ID Project Name P100531 IN: Coal-Fired Generation Rehabilitation ( P100531 ) Page 1 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Coal-Fired Generation Rehabilitation (P100101) L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF-94676 27948460.75 Bank Approval Date Closing Date (Actual) 18-Jun-2009 IBRD/IDA (USD) Grants (USD) Original Commitment 0.00 45,400,000.00 Revised Commitment 0.00 27,948,460.75 Actual 0.00 27,948,460.75 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Loan Agreement dated December 17,2009 (LA, p.5), the project objective was: "to improve the energy efficiency of selected coal-fired power generation units through renovation and modernization and improved operations and maintenance, and a significant co-benefit of the Project is the reduction of greenhouse gas emissions per kilowatt hour of electricity generated." The PAD (p.7) defines the project objective as “to improve the energy efficiency of selected coal-fired power generation units through renovation and modernization and improved operations and maintenance” and specifies the co-benefit of greenhouse gas emissions reduction as the global environment objective. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 29-Sep-2014 Page 2 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Coal-Fired Generation Rehabilitation (P100101) c. Will a split evaluation be undertaken? No d. Components The project had two components: 1. Energy Efficiency Renovation and Modernization Pilots. (Appraisal cost: US$295.9 million; Actual cost: US$173.54 million) Renovation and modernization (R&M) of 640 MW of old coal-fired power generation capacity of the participating utilities, specifically: (a) 210 MW capacity Unit 5 of Bandel Thermal Power Plant of West Bengal Power Development Corporation Limited (WBPDCL); (b) 210 MW capacity Unit 6 of Koradi Thermal Power Plant of Maharashtra State Power Generation Company Limited (MSPGCL); and (c) Each with 110 MW capacity Units 3 and 4 of Panipat Thermal Power Plant of the Haryana Power Generation Company Limited (HPGCL), to demonstrate energy efficient rehabilitation approaches. 2. Technical Assistance. (Appraisal cost: US$7.5; Actual cost: US$3.67 million) Technical assistance and support to: (a) WBPDCL; (b) MSPGCL; (c) HPGCL; and (d) Central Electricity Authority (CEA), as appropriate, to implement energy efficiency renovation and modernization pilots under Component 1, develop a pipeline of energy efficiency renovation and modernization interventions, addressing barriers to energy efficiency renovation and modernization projects, and strengthening institutional capacities of the participating utilities and the CEA. Revised Components In August 2012, HPGCL decided not to renovate and modernize Units 3 and 4 of Panipat Thermal Power Station because of the unsatisfactory experience the utility had in renovating and modernizing Unit 1 (Restructuring Paper dated April 11, 2014, p.5-6). Until the end of 2013, attempts were not successful to find substitutes for these units. As a result, at the first restructuring in April 2014, Component 1 was revised to exclude these activities and IBRD funds allocated to them were partially cancelled. Balance amount was reallocated to the renovation and modernization of other units at the second restructuring in September 2014. The cancellation of these activities and the revision of the outcome and intermediate indicators were in line with the project conditions set during appraisal (PAD, p.8 and p.19-21) and the Bank guidance; therefore, a split evaluation will not be undertaken, despite the change in project scope. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The total project cost was originally estimated at US$303.4 million including US$74.9 million for physical and price contingencies, interest during construction and front end fee. After the first restructuring, which was triggered by the Page 3 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Coal-Fired Generation Rehabilitation (P100101) HPGCL’s decision not to undertake the renovation and modernization of its units, the revised project cost was estimated at US$226.7 million. In March 2018, the project closed with a total cost of US$177.2 million. Financing: At appraisal, the International Bank for Reconstruction and Development (IBRD) loan was estimated at US$180.0 million. At the first restructuring, US$43.48 million was cancelled decreasing the IBRD funding to US$136.16. At project closing, the IBRD financing stood at US$104.14 million. These funds were used to finance the project activities under Component 1. At appraisal, Global Environment Facility (GEF) committed US$45.4 million as grant. There was no revision to the GEF grant amount at the first project restructuring. At project closing, the GEF financing stood at US$27.95 million, of which US$24.28 million was used to finance the project activities under Component 1 and US$3.67 million for Component 2. Borrower contribution: The borrower contribution was estimated at US$78.0 million as equity. At project closure, the borrower contribution stood at US$45.12 million, which contributed to cover the cost of renovation and modernization of coal-fired power units. 　 Restructurings and Dates: There were four restructurings. First Restructuring (April 21, 2014): As a result of the HPGCL’s decision not to continue with the renovation and modernization of its coal-fired generation units, US$43.84 million from IBRD funds were cancelled. 　 Second Restructuring (September 29, 2014): Balance US$21.84 million from the partial cancellation of IBRD funds at the first restructuring was reallocated to meet the funding gap for the renovation and modernization of Unit 5 of Bandel Thermal Power Station. The funding gap was due to time gap between the preparation of detailed project report in 2008 and award of the main package in February 2012, adoption of new technology, and delayed finalization of the contract (Restructuring Paper dated September 24, 2014, p.8). The outcome and intermediate indicators related to the cancelled renovation and modernization activities for Units 3 and 4 of Panipat Thermal Power Station of HPGCL were deleted and consolidated indicators were revised. Eligible expenditure category was modified from only "goods" to "works and goods including installation and other service" to allow utilities to disburse funds up to 74 percent of project costs. Lastly, the project closing date was extended by two years from November 30, 2014 to November 29, 2016 due to the delays in finalizing the contracts for the renovation and modernization of Unit 5 of Bandel Thermal Power Station and Unit 6 of Koradi Thermal Power Plant. 　 Third Restructuring (November 28, 2016): The project closing date was extended by 12 months from November 29, 2016 to November 29, 2017 due to delays in completion of works at Unit 6 of Koradi Thermal Power Station caused by technical problems and contractual issues" (ICR, p.10). Fourth Restructuring (November 29, 2017): The project closing date was extended by four months from November 29, 2017 to March 29, 2018 to allow additional time only for the completion of all the pending works after synchronization of Unit 6 of Koradi Thermal Power Station (Restructuring Paper, RES30550, p.3). Page 4 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Coal-Fired Generation Rehabilitation (P100101) 3. Relevance of Objectives Rationale The project objective is outcome-oriented; the outcome of the project is a modified condition both in terms of energy efficiency and reduced greenhouse gas emissions. Greenhouse gas emissions were projected to grow in India, and the country needed to improve energy efficiency to keep emissions from increasing at a faster rate and to sustain power for economic development. The Government of India (GOI) and states had already identified power plants with a total installed generation capacity of 27 GW requiring energy efficient renovation and modernization (EE R&M). The implementation of selected pilot investments in coal-fired generation units covered by this project was designated as Phase-I of the National R&M Program. It was expected that the demonstration of successful EE R&M could potentially result in the GOI and states developing the subsequent phases of the wider National R&M Program, targeting the entire 27 GW of eligible plants. In this case, the pipeline of energy efficient R&M pilots supported by the Bank would allow challenges faced by each pilot to be addressed in the next one. At the same time, the pilot nature of interventions also implied that if all pilots failed to demonstrate adequate barrier mitigation approaches (and the energy efficient R&M approaches were seen as unsuccessful), the project would demonstrate that the rehabilitation route for energy efficiency needs to be abandoned in favor of replacement of such units (PAD, pages 3-5). The project objective was consistent with the World Bank Country Partnership Strategy (CPS) for India for FY 2013-17 (when project closed in March 2018, the Country Partnership Framework FY 2018-22 was not approved yet). As a pilot for energy efficiency renovation and modernization (EE R&M) of selected coal-fired power plants, the project sought to address the adverse environmental impact of coal-fired power plants by increasing their efficiency. Therefore, the project objective corresponds to one of the expected outcomes of the strategic engagement area titled "Transformation," which is "reduced greenhouse gas emissions through energy efficiency and renewable energy production" (CPS, p.56). It also corresponds to strengthening India’s electricity supply to increase availability and reliability of power for economic development under the engagement area of “Integration” (CPS, p.52). However, EE R&M of coal-fired plants is not specifically stated in the Bank strategy as an activity that will be supported; this was because of the decision of the Government of India, during project implementation, to shift focus to the construction of supercritical power plants, which was also supported by the experience gained in this pilot project. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) Page 5 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Coal-Fired Generation Rehabilitation (P100101) EFFICACY_TBL OBJECTIVE 1 Objective “to improve the energy efficiency of selected coal-fired power generation units through renovation and modernization and improved operations and maintenance, and a significant co-benefit of the Project is the reduction of greenhouse gas emissions per kilowatt hour of electricity generated.” Rationale Theory of Change The project objective was clearly defined to improve the energy efficiency of selected coal-fired power generation units as pilots, with the co-benefit of a reduction in greenhouse gas emissions. The energy efficiency renovation and modernization activities (PAD, p.43) were common sector practices in the world, but with fewer applications in India. The project activities were to directly lead to the achievement of the renovation and modernization of the selected power plants, through the replacement of old parts and the introduction of new energy efficiency components. The outputs, together with technical assistance support to strengthen the operation and maintenance (O&M) capacities of the utilities, were to be expected to increase the efficiency of the units and decrease greenhouse gas emissions, which are critical to improving electricity security in India and decreasing local and global pollution. The causal pathways from inputs to outcomes are credible and robust. A direct attribution can be established between the project’s intervention and the outcomes. Outputs • The energy efficiency renovation and modernization of Unit 5 of Bandel Thermal Power Station (TPS) was completed and the unit was commissioned in October 2015. The installed capacity and the residual life of the unit increased from 210 MW to 215 MW and from 7 years to 15 years, respectively. • The energy efficiency renovation and modernization of Unit 6 of Koradi TPS was completed and the unit was commissioned in October 2018. The installed capacity and the residual life of the unit increased from 210 MW to 228 MW and from 8 years to 15 years, respectively. • The energy efficiency renovation and modernization of Units 3 and 4 of Panipat TPS was cancelled upon the request of the utility. (Please see Revised Components in section 2.d Components above.) • Under technical assistance support, four energy efficiency renovation and modernization barrier reduction studies were completed as targeted. • In all three participating utilities, implementation of operation and maintenance (O&M) and institutional strengthening programs was completed. • As detailed in Annex 7-Borrower's ICR (ICR, p.49-69) consultancy services in design of energy efficiency renovation and modernization, environmental audit and due diligence assessment study, project implementation support, impact assessment study on corporate social responsibility activity were completed. Page 6 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Coal-Fired Generation Rehabilitation (P100101) Outcomes Unit 5, Bandel TPS • Coal consumption decreased from 0.684 kg/kWh to 0.560 kg/kWh. The target was 0.602 kg/kWh. The decrease was 18.13 percent against the target of 12 percent. • Oil consumption increased from 2.50 ml/kWh to 4.21 ml/kWh. The target was to reduce oil consumption to 2.00 ml/kWh. The increase in the oil consumption was "due to the changes in the design of certain equipment by the contractor to meet contractually agreed parameters, faster than expected wearing-off of certain equipment parts, and frequent trippings caused by failure of raw coal feeder belt" (ICR, p.11). Unit 6, Koradi TPS • Coal consumption increased from 0.703 kg/kWh to 0.730 kg/kWh. The target was to reduce coal consumption to 0.605 kg/kWh. The increase in the coal consumption was due to the decrease in the gross calorific value (GCV) assumption of the coal. The target was set based on GCV of 4,500 kcal/kg; however, the actual GCV after rehabilitation was 3,170 kcal/kg, since the plant could operate on low GCV coal. Assuming GCV to be 4,500 kcal/kg, the coal consumption would decrease to 0.514 kg/kWh, which would be an achievement better than the target of 0.605 kg/kWh. (ICR, p.11). • Oil consumption decreased from 2.27 ml/kWh to 1.22 ml/kWh. The target was 2.00 ml/kWh. Greenhouse Gas Emissions • The target for reduction in CO2 emission intensity for each of the generation units after completion of renovation and modernization was 10 percent. The achievements were 15.39 percent for Unit 5 of Bandel TPS and 17.89 percent for Unit 6 of Koradi TPS. • The target for reduction in CO2 emission was a total of 0.45 TCO2. The achievement was 0.46 TCO2 assuming that Unit 6 of Koradi TPS, which was commissioned in October 2018, operated one full year. O&M • Standard Operating Procedures and Standard Maintenance Procedure were prepared and implemented. • The gaps in O&M practices, such as absence of protocols for checking interlocks and protections, weaknesses in equipment condition monitoring, lack of efficiency testing of systems and equipment, weekly rather than daily O&M reviews, non-practice of system of scheduled equipment changeover, were identified and actions to be taken were identified and implemented. The ICR provided detailed information about outputs and outcomes of TA activities related to O&M for the participating utilities (ICR, p.43-69). Pilot Applications Page 7 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Coal-Fired Generation Rehabilitation (P100101) The project was designed to go beyond the common rehabilitation practice in India—restoring original generation capacity, life- extension, and improving availability—by also replacing equipment and systems to enable the units to operate with higher fuel efficiency. Such experience in India was limited and the project had a pilot aspect, as well, which would provide feedback to the decision makers how to proceed with the rehabilitation of a total of 27,000 MW aging installed capacity. However, after the signing of the Paris Agreement, the Government of India had already shifted its strategy towards the construction of greenfield supercritical coal-fired power plants (ICR, p.14), which are more efficient than the rehabilitated power plants despite a higher total overall cost. The challenges faced during project implementation, such as unexpected structures resulting in stoppage of work for several weeks, mismatches in the design of replaced part and original equipment, lack of institutional capacity to implement complex renovation and modernization projects despite support given through technical assistance, difficulties in the coordination of different contractors, supported the government’s decision to focus on greenfield supercritical investments. As the project team confirmed, lessons learned from project implementation were shared with the Government of India as inputs for policy papers drafted about the future course of thermal power generation in the country including comparison of renovation and modernization with greenfield technology options, such as supercritical thermal power generation systems (Aide Memoire dated September 3-21, 2015, p.4). Rating Substantial OVERALL EFF TBL OLD Rationale The project was partially successful in achieving the energy efficiency targets, while the targets in reducing greenhouse gas reduction were achieved (energy efficiency targets would be achieved if high gross calorific value coal, as foreseen in project design, were used instead of the currently used low gross calorific value coal). Furthermore, as a pilot, the project was successful in showing that due to technical issues which could not be known before the dismantling of the power plant components and much longer shut-down period required, EE R&M of old power plants was not a viable option. The project also provided important data and findings about the issues faced during such EE R&M interventions and important lessons were learned to be benefited from in similar operations in future. Overall, given the project’s achievement in increasing energy efficiency and reducing greenhouse gas emission in selected coal-fired power generation units and, as a pilot project, its contribution to policy making in India regarding the future of thermal power generation, the achievement of the project objective is rated Substantial. Overall Efficacy Rating Substantial Page 8 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Coal-Fired Generation Rehabilitation (P100101) 5. Efficiency Economic Analysis At appraisal, "without project" and "with project" economic analyses were conducted only for Unit 5 of Bandel Thermal Power Station (TPS). Three options were considered under "with project" scenario: (i) efficiency integrated renovation and modernization; (ii) boiler-turbine-generator replacement; and (iii) complete replacement. Underlying assumptions used in cost- benefit analyses for different options at appraisal were typical for such investments. Benefits were taken as reduction in fuel cost and increased generation, whereas costs were taken as project cost and incremental cost. However, the economic analysis at appraisal did not include operation and maintenance (O&M) costs, which were added to the analysis at project closing. The economic analysis at appraisal resulted in an economic internal rate of return (EIRR) of 29.09 percent for the efficiency integrated renovation and modernization option compared to 15 percent for complete replacement for a period of 22 years (PAD, Annex 9). The boiler-turbine-generation replacement option was excluded due to lack of sufficient space at the power station for construction. Net present value was not calculated. At project closing, separate economic analyses were conducted for Unit 5 of Bandel TPS and Unit 6 of Koradi TPS using the same methodology at appraisal. The analysis resulted in an EIRR of 11.94 percent including O&M costs, and 16.73 percent excluding O&M costs for Bandel TPS. These figures are lower than the appraisal EIRR of 29.09 percent. The main reasons for lower EIRR at project closing were (i) the doubling of the coal price; (ii) longer shutdown period (6 months at appraisal compared to 24 months actual); and (iii) increase in oil consumption per kWh. (ICR, p.35 and 38). The economic analysis conducted for Unit 6 of Koradi TPS resulted in an EIRR of 25.62 percent. There was no economic analysis conducted for this unit at appraisal because energy audit studies and detailed project report had not been completed by then (PAD, 21). Financial Analysis The financial benefit of the renovation and modernization of the Unit 5 of Bandel TPS is increased revenue from additional power sales. Due to the uncertainties about the tariff regime, the financial analysis for Unit 5 of Bandel TPS was conducted under two assumptions: existing tariff continues or tariff is calculated on cost plus approach (PAD, 93). The calculations resulted in a financial internal rate of return (FIRR) of 16.6 percent and 10.44 percent, respectively, for Bandel TPS, which were higher than the weighted average cost of capital of 8.52 percent. At project closing, FIRR could not be calculated for either unit because of net incremental cash flows being predominantly negative. The ICR (p.39) states the main reason for this as "the difference in the coal price—the estimated base period coal price considered in the analysis for the ICR is more than double the base period price assumed in the financial analysis for the PAD." Operational and Administrative Efficiency The shift in government’s strategy towards construction of supercritical coal-fired power plants from energy efficiency renovation and modernization (EE R&M) adversely affected the operational efficiency of the project. A dedicated EE R&M team did not exist at either power station (ICR, p.18). Furthermore, the utilities’ ownership of the project was weak because of frequent changes in management and team members. Lack of coordination among contractors contributed to implementation delays. Limited capacity Page 9 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Coal-Fired Generation Rehabilitation (P100101) to handle complex procurement processes and managing contract, and cumbersome internal approval process resulted in delays in procurement. On the Bank side, there were five Task Team Leader changes during the first five years of project implementation. There were persistent long delays in the transfer of funds from local state governments to participating utilities, which slowed the project implementation (ICR, p.18). The project closing date was extended for a total of 40 months, increasing the project implementation period from 5 years 5 months to 8 years 9 months due to the delays in finalizing the contracts for the renovation and modernization of the thermal power units and for the completion of works at Koradi TPS. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re- estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0 　 Not Applicable 0 ICR Estimate 0 　 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The energy efficiency and greenhouse gas emission objectives are substantially relevant to the country strategy and the sector context. The project was partly successful in achieving energy efficiency targets, due to the usage of low gross calorific value coal, rather than high gross calorific value coal as envisaged in the project's original design. The project was successful in achieving greenhouse gas reduction targets, and as a pilot project, it also provided invaluable data and findings through monitoring and evaluation (M&E), which contributed to the Government's decision not to continue with the EE R&M of the remaining 27 GW aging coal-fired power plants. These data and findings about the issues faced during the EE R&M interventions and important lessons learned would be useful should there be similar operations in the future. Therefore, the Achievement of Objectives (Efficacy) is rated Substantial. However, due to the lower than expected economic rate of return for the Bandel Thermal Power Station and significant shortcomings in operational and administrative efficiency, such as implementation delays caused by lack of coordination among coordinators and delayed transfer of funds from local state governments to participating utilities, limited capacity of the utilities to handle complex procurement processes and to manage contracts, and cumbersome internal approval processes, the Efficiency is rated Modest. Overall, the ratings of the Relevance of Objectives, the Efficacy and the Efficiency being Substantial, Substantial and Modest, respectively, the Outcome is rated Moderately Satisfactory. Page 10 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Coal-Fired Generation Rehabilitation (P100101) a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome Potential shortcomings in operation and maintenance (O&M) of the rehabilitated units might pose a risk for the sustainability of the project outcomes. The project realized the shortcomings in O&M, and provided technical assistance (TA) in strengthening the O&M capacity of the utilities (ICR, p.15). Utilities used TA funds to build their O&M procedures. The project also showed that rather than an overhaul of power units after 25 years, rehabilitation of a unit can be broken into smaller works and completed in phases. This increases the importance of O&M furthermore. However, if sufficient funds are not allocated to regular maintenance works or experienced members of staff could not be maintained, O&M of the units might be adversely affected. Project outcomes of energy efficiency and reduced greenhouse gases would not be sustained if the utilities continue using low calorific value coal. One of the outcomes of the energy efficiency renovation and modernization works was that both units can now use low calorific value coal, which was not possible before the project (ICR, p.11). As a result, the coal consumption at Unit 6 of Koradi TPS increased from 0.703 kg/kWh to 0.730 kg/kWh, whereas the target was to reduce it to 0.605 kg/kWh (ICR, Table 3, p.11). Switching to high calorific value coal would not only improve the efficiency of the power units, but also have a longer term positive environmental impact. 8. Assessment of Bank Performance a.Quality-at-Entry The objective was clearly stated and strategically relevant. The theory of change was robust and a direct causal link could be established among project inputs, activities, outputs and outcomes. As a pilot project, technical design of rehabilitation works was sound. Given the difficulty in drafting technical specifications for energy efficiency renovation and modernization works, there can be unforeseen technical issues upon the opening of the power unit components; therefore, upfront studies, such as residual life assessments (RLA), energy audits, detailed project reports and environmental audits were conducted (PAD, p.22) with the assumption that project activities would start without delay—by the time the project implementation started after a four-year delay, RLA studies had already become redundant. The project activities were sequenced with a scaling up approach to support the government’s National Renovation and Modernization Program targeting eligible power units with a total installed capacity of 27 GW. Lessons learned from similar Bank projects implemented in Turkey and China, and projects implemented by other development banks, such as Asian Development Page 11 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Coal-Fired Generation Rehabilitation (P100101) Bank, were incorporated into the project design (PAD, p.10-11; ICR, p.16-17). Implementation arrangements and fiduciary aspects were in place. M&E arrangements—highly important for such pilot projects so that lessons learned during implementation could be utilized in similar works and in policy making—were robust. A detailed list of risks and mitigation measures were prepared during appraisal under three main groups; (i) technical aspects of the project; (ii) level of participation in the bidding process and high prices; and (iii) weak client capacity and inadequate decision-making framework (PAD, 16-18). On the other hand, although it was included as a condition in the loan agreement, the risk related to flow of funds, such as delayed transfers of counterpart funds from local governments to utilities (ICR, p.18-19), was not identified during appraisal. This led to delays in project implementation—30 months from effectiveness for Koradi TPS. Workforce planning was not implemented properly resulting in a shortage of adequate labor force. Economic analysis did not include O&M costs. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision There were nine supervision missions during project implementation and monthly technical visits during the construction period of 2013-2015 for Bandel TPS and 2016-2018 for Koradi TPS (ICR, p.19). The Implementation Status and Result Reports were candid. The project team ensured that technical aspects of the project were reviewed by a panel of technical experts (ICR, p.21). The project team proactively provided support and guidance to the project implementation teams in procurement, especially for smaller value contracts (ICR, p.19). The project team also provided support "in resolving issues, such as co-financing problems, limited bidder interest in procurement, and prolonged negotiations with a single bidder" (ICR, p.21). There was a high turnover of Task Team Leaders (TTL)—five TTLs between 2008 and 2013. Last two co-TTLs, one stationed in India, had overseen the project from 2014 until project closing, which helped project gain pace. The Borrower’s ICR (ICR, Annex 7, p.55-56) acknowledges the overall involvement of the project team as satisfactory. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization Page 12 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Coal-Fired Generation Rehabilitation (P100101) a. M&E Design The project objective was clearly defined and the theory of change—documenting how the key activities and outputs led to the outcomes—was sound and adequately reflected in the results framework. The indicators, i.e., decrease in coal and oil consumption per kWh of power generated and reduction in greenhouse gas emissions fully encompassed the outcomes of the project development objective statement. Indicators were specific, measurable and achievable. For such renovation and modernization projects, the indicators were highly relevant. Baseline data for indicators were based on detailed project reports prepared by qualified consultants (PAD, 15).The M&E design and arrangements were adequately embedded institutionally; data were to be collected by participating generation utilities for both implementation progress and achievement of the project outcomes. Arrangements were in place to monitor and evaluate the implementation of these pilot project activities to draw lessons and disseminate learnings from demonstration projects to relevant authorities at the local and national levels to provide inputs for decisions on national policies and regulations, and to set up benchmark and good practices for replication (PAD, p.14-15). b. M&E Implementation Participating generation utilities and consultants provided detailed data about project implementation and achievement of the outcomes. While financial management reports, annual audits of project accounts and external monitoring reports from consultants were submitted to the project team mostly on time, there were delays in delivering timely progress reports (ICR, p.19-20). Project team closely monitored the implementation of project activities; there were regular monthly technical visits during construction. Indicators included in the results framework were adequately measured and reported. Issues encountered during complex renovation and modernization activities were recorded and reported systematically to the national authorities and the Bank management (ICR, p.19). At the second restructuring in September 2014, the outcome and intermediate indicators related to the cancelled renovation and modernization activities for Units 3 and 4 of Panipat Thermal Power Station of HPGCL were deleted and consolidated indicators were revised. Participating generation utilities also monitored performance indicators, such as unit heat rate, auxiliary consumption, and unit availability, which were shared with the project team. Since M&E functions and processes are embedded in the operation of the rehabilitated power units, they are highly likely to be sustained after project closing. c. M&E Utilization M&E findings, especially those related to the implementation of the renovation and modernization activities, were communicated to various stakeholders adequately. This resulted in finding solutions to technical issues encountered during construction. The lessons learned from the implementation of pilot project activities were shared with the government authorities to provide input for policy decision-making (Aide Memoire dated September 3-21, 2015, p.4). The project team utilized M&E findings with a focus to achieve the project outcomes. As the project team confirmed, the experience gained in this project contributed to the government’s decision to give priority to the construction of supercritical coal-fired power plants, rather than continuing with the energy efficiency renovation and modernization of Page 13 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Coal-Fired Generation Rehabilitation (P100101) aging power plants. Nevertheless, the lessons learned from the implementation of these pilot project activities would provide invaluable input if similar rehabilitation works were to be undertaken in the future. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was classified as Category B under OP/BP 4.01 (Environmental Assessment). Initially, OP/BP 7.50 (Project on International Waterways) was triggered because of the water drawn from River Hooghly for cooling water requirements, which is part of Ganga Basin extending to Bangladesh; however, an exception to external notification was issued since "the project would not adversely affect the quality or quantity of the water flows to other riparian countries" (PAD, p.27-28). Environmental Assessment OP/BP 4.01: Despite the limited expected adverse environmental impact of rehabilitation activities, the project was to produce major positive environmental benefits through increased energy efficiency and reduced greenhouse gas emissions. Both participating generation utilities prepared Environmental Audit Due Diligence Reports (EADD) acceptable to the Bank, including Environmental Management Action Plans for implementation. These documents were disclosed both in India and at the Bank’s Infoshop. During project implementation Rapid Social Assessment (RSA) studies were carried by the participating generation utilities to document adverse impacts of the power stations on agricultural fields and habitations, to screen development issues and to assess the needs of the community. Findings of these studies contributed to the preparation of Corporate Social Responsibility Plans of the utilities. The project team confirmed that the project was in compliance with OP/BP 4.01. b. Fiduciary Compliance Financial Management The financial management arrangements benefited from the country’s finance system. The financial management manual incorporated adequate internal control guidelines. Quarterly interim unaudited financial reports were submitted to the Bank on time. The final audit was carried out by internationally affiliated firm and the report was unqualified. The project was in compliance with legal covenants, except one that is related to the transfer of funds to Koradi TPS; delays in transfer of funds from local state to the utility resulted in a total of 30-month delay in project implementation. The legal condition was to transfer funds within 15 days. At the time of evaluation, all project funds were accounted for. Page 14 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Coal-Fired Generation Rehabilitation (P100101) Procurement All procurement of goods, works and services financed by the project followed the Bank procurement guidelines. Procurement packages were completed according to the procurement plan, but there were extensive delays. The procurement process was adversely affected by two factors: (i) limited procurement capacity within the participating generation utilities to handle complex procurement processes and managing complex contracts; and (ii) cumbersome internal approval process—conflict with state procurement rules or delays in approval of environmental permits (ICR, p.19). Consequently, the Bank project team had to intervene in resolving procurement difficulties, such as providing support and guidance in smaller value contracts and rebidding of major contracts, and concluding prolonged negotiations with a single bidder. c. Unintended impacts (Positive or Negative) None. d. Other None. 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Energy efficiency and reduction of greenhouse gas emissions are substantially aligned with the country strategy and sector context. Efficacy is rated Substantial due to the partial achievement of the energy efficiency Moderately Outcome Moderately Satisfactory targets caused by the usage of low Unsatisfactory calorific value coal contrary to the usage of high calorific one, and the achievement of the greenhouse gas emission targets. Efficiency is rated Modest due to the lower than expected economic rate of return and Page 15 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Coal-Fired Generation Rehabilitation (P100101) significant shortcomings in operational and administrative efficiency. Overall, the pilot aspect of the project taken into account, the outcome of the project is rated Moderately Satisfactory. Moderately Bank Performance Moderately Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons Four lessons are selected from the ICR with some adaptation of the language. Robust monitoring and evaluation (M&E) arrangements and close supervision do not only help monitor the progress of project activities and take timely measures to address implementation issues, but also create an important knowledge base to be used in replication projects or in policy decision-making. Being a pilot in energy efficiency renovation and modernization (EE R&M) of coal-fired power generation units, the project had a robust M&E system embedded in the participating generation utilities operations. The M&E findings and close supervision of the project implementation, especially during construction, helped the project implementing teams and the Bank project team to intervene when implementation issues arose. The project also provided important technical, operational and procurement related information from which future replications could benefit. Furthermore, the M&E findings were communicated to the national authorities to provide input to policy decisions about the future of thermal generation in the country. At project preparation, services of a design consultant for the preparation of detailed rehabilitation scope through residual life assessment (RLA) studies for entire power plant may not be necessary. In this project, the RLA studies were completed with the anticipation that the project activities would start without delay. However, by the time the project activities started after a significant delay, the RLA studies had already been redundant. Furthermore, due to conflict of interests, the design consultant could not continue working for the project during implementation. Therefore, it could be more practical and efficient if the boiler-turbine-generator contractor decided what the scope of replacements and modifications should be. Energy efficiency renovation and modernization (EE R&M) of coal-fired power plants after 25 or 30 years of operation can result in a longer shut-down time than estimated. The shut-down time for both power plants was estimated to be six months. However, due to technical issues, which had not been foreseen before the project activities started, the shut-down periods extended to two years. When EE R&M works are implemented after a very Page 16 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Coal-Fired Generation Rehabilitation (P100101) long operation period, the likelihood of having unexpected technical issues with the mechanical systems in a coal- fired plant increases. Therefore, as the experience of this project shows, it might be advisable to break EE R&M operations into phases consisting of separate smaller works, such as turbine improvements, and boiler area retrofits, which would require shorter shut-down periods. For complex rehabilitation projects, it is critical to have a dedicated project implementing team with high technical and project management capacity for the entire duration of the project. Frequent change of officials in the project implementing teams of the participating generation utilities and the low technical and project implementation capacity of these teams adversely affected project implementation. Some complex contracts had to be tendered twice. Inadequate workforce planning resulted in a shortage of labor force. A program evaluation review technique, detailing out all project activities, was not prepared. Coordination among various contractors was inadequate. The government’s decision to shift strategy in favor of construction of supercritical coal-fired plants rather than energy efficiency renovation and modernization of aging power plants contributed to the weakening of the project implementing teams’ ownership of the project. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR was concise and candid in presenting the project’s shortcomings during implementation. The narrative was internally consistent; there is a logical linking and interrogation of the various parts of the report, and the results are mutually reinforcing. The findings were supported by evidence which were appropriately referenced. The discussion on theory of change was adequate. Economic analysis was detailed and clearly represented. The lessons were clear and based on evidence responding to specific experiences and findings for the project. The ICR was mostly consistent with the guidelines. Annex 1 was detailed and well presented, and it included the agencies engaged in civil work and contract details. On the other hand, given that this was a pilot project, the section on M&E utilization could have benefited from a more detailed discussion of how M&E findings contributed to the Government’s decision on adjustments of national policies and regulations, and setting-up of benchmarks. The section on safeguards did not report whether the project was in compliance with the environmental safeguard policy. The discussion of risk to development outcome was weak. The outcomes of technical activities were explained in detail for one of the participating generation utilities only. a. Quality of ICR Rating Substantial Page 17 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Coal-Fired Generation Rehabilitation (P100101) Page 18 of 18