Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Power Sector Reform DPC(P128258) Report Number : ICRR0021106 1. Project Data Country Practice Area(Lead) Pakistan Energy & Extractives Programmatic DPL Planned Operations: 0 Approved Operations: 0 Operation ID Operation Name P128258 PK: Power Sector Reform DPC L/C/TF Number(s) Closing Date (Original) Total Financing (USD) IDA-54360 30-Jun-2015 603,789,088.00 Bank Approval Date Closing Date (Actual) 01-May-2014 30-Jun-2015 IBRD/IDA (USD) Co-financing (USD) Original Commitment 600,000,000.00 0.00 Revised Commitment 600,000,000.00 0.00 Actual 603,789,088.00 0.00 Prepared by Reviewed by ICR Review Coordinator Group Ranga Rajan Fernando Manibog Christopher David Nelson IEGSD (Unit 4) Krishnamani PHPROJECTDATATBL Operation ID Operation Name P152021 Pakistan: Power Sector Reform DPC 2 ( P152021 ) Page 1 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Power Sector Reform DPC(P128258) L/C/TF Number(s) Closing Date (Original) Total Financing (USD) IDA-54360,IDA-57400 31-Dec-2016 489,390,304.00 Bank Approval Date Closing Date (Actual) 12-Nov-2015 31-Dec-2016 IBRD/IDA (USD) Co-financing (USD) Original Commitment 500,000,000.00 0.00 Revised Commitment 500,000,000.00 0.00 Actual 489,390,304.00 0.00 2. Program Objectives and Policy Areas a. Objectives The First and the Second Power Sector Reform Development Policy credit operations (PSRDPO 1 and 2) comprised a programmatic series of power sector reforms of the Islamic Republic of Pakistan (hereafter referred to as Pakistan). According to the First Program Document (page 13), the operations supported the government's Action Plan for implementing the 2013 National Power Policy. The DPO 1 reflected the prior action taken by the government to meet the pressing needs set out in the broader National Power Policy and DPO 2 was to focus on structural reforms. Combined, the series aimed at restoring the sector's financial viability and thus reducing the public financing for it. The DPO project development objectives were: (1) Reducing subsidies and improving tariff policy (2) Improving sector performance and opening the market to private participation (3) Ensuring accountability and transparency The objectives were unchanged during the operation. However, the indicator targets and results were modified during DPO2 in view of the changing circumstances and evolving policy dialogue between the Bank team and the government (discussed in section 3b and 4). The Legal Agreement of May 2, does not contain a statement of the PDO. b. Pillars/Policy Areas There were three policy areas. First. Reducing subsidies and improving tariff policy. This policy area aimed at improving the financial viability of the energy sector and thereby improving the government's fiscal position by reducing subsidies to Page 2 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Power Sector Reform DPC(P128258) the sector, making the subsidies more transparent and better targeting of subsidies. Measures in this area aimed at limiting subsidies, moving tariffs to levels consistent with reasonable cost recovery, and strengthening the role of the sector's regulator, the National Electric Power Regulatory Authority (NEPRA). Second. Improving sector performance and opening the market to private participation. This area aimed at improving energy sector performance (both with respect to downstream electricity provision and upstream supply of gas for power generation) and providing incentives for private sector participation in the sector. Measures included: (i) reducing the inefficiencies of the Distribution Companies (Discos) by reducing the losses associated with electricity theft and uncollected bills by Discos; (ii) providing incentives for electricity generation through domestic gas; and (iii) moving the sector towards market-oriented operations by separating market and transmission system operations. Third. Ensuring accountability and transparency. This area aimed at improving sector accountability for broader stakeholder support. Measures included monitoring and self-reporting mechanisms for sector entities and oversight by independent experts. c. Comments on Program Cost, Financing, and Dates Program cost. The total cost for the programmatic series was US$1,093.18 million. This included US$603.79 million for DPO1 and US$489.39 million for DPO2. Program financing. The total IDA financing for the DPOs US$1100.00 million (US$600.00 million for DPO 1 and US$500.00 million for DPO 2). Total amount disbursed US$1093.00 million (US$603.79 million for DPO1 and US$489.39 for DPO2). The proceeds of the IDA credit for each operation was withdrawn in a single tranche and provided as budgetary support to the government. There was parallel financing for sector reform technical assistance activities from the Asian Development Bank (ADB), the Japan International Cooperation Agency (JICA), the United States Agency for International Development (USAID) and the United Kingdom Department for International Development (DFID). Borrower contribution. There was no counterpart funding from the Borrower. Dates. The effectiveness date for DPO 1 was May 6, 2014 and for DPO 2, December 12, 2016. DPO1 closed on June 30, 2015 and DPO2 on December 31, 2016. 3. Relevance of Objectives & Design a. Relevance of Objectives Before appraisal, Pakistan's energy sector faced a crisis, with electricity shortages averaging 4,000 - 5,000 Megawatt (MW) and about a quarter of demand unmet. Unscheduled load shedding of eight to nine hours or more daily was common. At the same time, about 5,000 MW of electricity capacity remained unused on account of the acute liquidity crisis of the electricity generators. The sector relied heavily on government direct subsidies, which amounted to about 1.75% of GDP in 2012-2013 (about four times the subsidies provided to health and education sectors). Costs that could not be recovered from the buyers accumulated as debt of Page 3 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Power Sector Reform DPC(P128258) the Distribution Companies (Discos). The Discos in turn failed to pay for the electricity generated by the privately-financed electricity generated by Independent Service Providers (ISPs). The accumulated arrears (commonly called the circular debt), accounted for about 4% GDP in 2012-2013. (Circular debt refers to the amount of cash shortfall within the Central Power Purchasing Agency Guaranteed Limited (CPPA-G), which it cannot pay to power supply companies. This shortfall was due to two factors: (i) the difference between the cost of producing electricity and the revenue realized by Discos from electricity sales and, (ii) insufficient payments by Discos to the CPPA-G out of the revenue realized). As owner of the Discos, the government was liable for the debt and this constituted a further subsidy to the sector. According to preliminary estimates, the poorly performing electricity sector reduced GDP by 2% in the years before appraisal. The objectives were highly relevant to the government’s strategy. In the years before appraisal, the government adopted a strategic plan aimed at unbundling the sector, which had been vertically integrated until then. The government’s National Power Policy articulated in 2013 had these objectives: (1) decreasing the gap between supply and demand for electricity: (2) improving affordability of electricity through decreasing average electricity generation cost: (3) decreasing aggregate technical and commercial transmission and distribution losses: (4) improving collection of unbilled electricity: and (5) improving governance and decreasing the decision making times at ministries, related departments and regulators. The National Power Policy also specified targets for reaching these targets by 2017. The government followed this through by settling the circular debt in July 2013 and putting a cap on circular debt. The objectives were well-aligned with the Bank strategy. The Country Partnership Strategy (CPS) for the 2010-2014 period identified power shortages as a major impediment to the private sector and highlighted the need for supporting policies aimed at financial sustainability, least cost expansion of generation (through increasing hydropower and gas in the generation mix and better efficiency promoted through least cost planning), and improvements in the efficiency of transmission. The current CPS for the 2015-2019 period devotes one pillar exclusively to the energy sector (pillar one) and reiterates the need for a structural transformation in the energy sector through outcomes relating to reductions in load shedding and subsidies to the sector. Rating High b. Relevance of Design The lending instrument used - the DPO dedicated to the energy sector (as opposed to inserting a limited number of prior actions in a multi-sector or general macroeconomic support operation) - was appropriate, in view of the unscheduled load shedding and challenges faced by the energy sector at appraisal. The DPO results framework is logical. The program development objectives "reducing subsidies and improving tariff policy, improving sector performance and opening the market to private participation and ensuring accountability and transparency" is clear and explicit. The three policy areas - reducing subsidies and improving tariff policy, improving sector performance and opening the market to private participation and ensuring accountability and transparency of the sector - represented choice of policies and institutional reforms that would enable realizing the program development objectives. Each of the three policy areas comprised a set of prior actions, triggers and expected end-program outcomes. The prior actions - the set of Page 4 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Power Sector Reform DPC(P128258) policy and institutional actions that are deemed critical to achieving the program objective are well designed and linked to objectives and outcomes. The triggers setting the foundation for the succeeding operations of the programmatic series were appropriate. These triggers became the prior actions of the succeeding operation. Finally, the expected outcomes are clearly specified. The causal linkages from prior actions to expected outcomes, and from the expected outcomes to the program development objectives, are research-based and credible. • The objective of "reducing subsidies and improving tariff policy" was to be realized through policies and institutional reforms aimed at the financial viability of the sector through moving tariffs to levels consistent with cost recovery and reducing energy subsidies. The expected outcomes of these policies aimed at reduction of subsidies (specified in terms of percent of GDP) to the energy sector. • The objective of "improving sector performance and opening the market to private participation" was to be realized through policies aimed at improving downstream energy operations through reducing electricity theft and increasing bill collection by Discos, improving supply of gas for power generation through providing incentives for increasing electricity generation through domestic gas, and improving sector performance through opening the market to private participation (through the separation of market operations and transmission system operations). The expected outcomes of these policies are increases in collection by Discos, increases in domestic gas production, and ensuring that the power generated by Independent Power Providers were traded through an independent Central Power Purchasing Agency (Guarantee) Limited acting on behalf of the Discos. • The objective of " ensuring accountability and transparency" was to be enabled through policies aimed at improving the accountability of the energy sector through better monitoring, self-reporting mechanisms for sector entities and oversight by independent experts. The expected outcome of these policies aimed at broader stakeholder participation through public access to information regarding the performance of the Discos. • There were synergies between the policy areas. Reducing energy subsidies and ensuring that tariffs reflect costs aimed at providing strong signals to Discos and the government, that they need to address issues relating to electricity theft and uncollected bills. Increasing domestic gas supply aimed at reducing the average costs of electricity generated. Commercially-oriented unbundling was expected to provide incentives for private sector participation in the sector. Improving transparency and accountability were aimed at strengthening public ownership of reforms. • The design was flexible and was adapted to changing circumstances. For example, the original target of reducing subsidies was modified given the government's commitment (for political economy considerations) to maintain subsidies for middle class households using 300 Kilowatt-hours (kWh) per month (discussed in section 4). Rating Substantial 4. Achievement of Objectives (Efficacy) Page 5 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Power Sector Reform DPC(P128258) PHEFFICACYTBL Objective 1 Objective Reducing subsidies and improving tariff policy Rationale This policy area aimed at reducing subsidies and improving tariff policy. • Following the 2014 Tariff and Subsidy Policy Guidelines, in June 2013, the Ministry of Water and Power (MWP) notified the revised tariffs determined by the regulator - the National Electric Power Regulatory Agency (NEPRA). According to the revised tariffs, effective from June 2013, the average effective tariffs were to increase by 44% for industries, commercial and bulk consumers and by 32% for households, the agricultural sectors and consumers using more than 200 kWh/month. • In the same year, following the 2014 Tariff and Subsidy Policy Guidelines, the MWP informed NEPRA to incorporate the budgeted subsidy in the tariff determination. These subsidies were to be reduced from 1.8% of the Gross Domestic Product (GDP) at the baseline to 0.4% of GDP by 2014-2015. Also following those Guidelines, the MWP modified the original target of 0.4% of GDP to O.8% of GDP, as the original target was not deemed to be achievable in the face of the government's continued commitment to subsidizing the consumers using up to 300 kWh/month due to political economy considerations. • As a prior action of DPO1, the Ministry of Finance (MoF) settled the power sector circular debt in the amount of Pakistan Rupee (PKR) 480 billion. Following this and in line with the government policy of reducing the circular debt: (i) The NEPRA issued guidelines for Disco tariff determination (covering principles, methodologies, timetable formula and procedures for both annual and multi-year tariffs): (ii) The MoF published in its website a cap for circular debt at PKR 220 billion: and(iii) overdue payables to power generators were not to be below the cap for at least three months. As a prior action for DPO2, the MWP published in its website and set a cap for total overdue payables to power generators not to exceed PKR 314 billion and a plan to reduce the flow of new overdue payables to PKR 39 billion by 2017/2018, with interim targets for the flows of PKR 92 billion in 2015/2016 and to PKR 57 billion in 2016/2-17. The level of circular debt however increased above the level of this cap reaching PKR 321 billion at the end of 2016 and PKR 450 billion in November 2017 due to a combination of factors associated with Discos performance (higher losses and lower collection). • As a prior action for DPO1, the Economic Coordination Committee approved the Tariff and Subsidy Policy Guidelines for: (i) subsidy policy for low-income residential customers: (ii) multi-year tariffs: (iii) tariff equalization mechanism and guidance for tariff setting as envisaged in the NEPRA act, including forward looking fuel price adjustments: and (iv) guidance for circular debt management related to overdue payables to generators by the Central Power Purchasing Agency (Guarantee) Limited. As a prior action for DPO2, the government implemented a tariff setting mechanism through tariff surcharges and tariff rationalization to maintain nationally uniform tariffs. This new measure was introduced in November 2014 in response to falling fuel costs, with the surcharge equalizing tariffs across seven of the ten Discos. This measure meant that part of the (negative) fuel price adjustment was not passed on to household and agricultural customers and the available extra funds were used to help pay the generators, thus reducing the subsidy needs and circular debt. Outcomes Page 6 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Power Sector Reform DPC(P128258) • Subsidies to the Discos by the Federal Government were reduced from 1.8% of GDP in the baseline to 0.7% of GDP in June 2016. This exceeded the revised target of 0.8% of GDP, although below the original target of 0.4% of GDP (ICR, page viii). Subsidies were reduced from 1.8% at the baseline in 2012/2013 to 1.2% of GDP between DPO1 and DPO2, 0.9% of GDP by 2014/2015 and to 0.7% of GDP by the end of 2015/2016). Given that the reduction in subsidies exceeded the revised target ( (although short of the original target in view of political economy considerations), it is reasonable to conclude that the operation made a significant contribution to achieving the desired outcome of reducing subsidies and improving tariff policy. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 2 Objective Improving sector performance and opening the market to private participation Rationale This policy area aimed at improving sector performance (both downstream electricity provision and upstream supply of gas for power generation) and opening the market to private participation. For improving sector performance with respect to downstream electricity provision, the prior actions for improving collection and reducing distribution losses were: (i) the outsourcing of collections of four Discos to the private sector: (ii) all Discos were required to implement a revenue protection program that ensured correct billing, reducing losses in particular through theft and improving collections: (iii) the Council of Common Interests initiates discussion on a mechanism to automatically withhold a proportion of the electricity arrears of provincial government entities: and (iv) the Federal Government establishes a mechanism to withhold budget transfers to federal agencies or entities which have arrears of payment for electricity that exceed 90 days of billing by Discos. The prior action regarding outsourcing of collections was dropped during DPO2 as it was not possible to assess improvements due to outsourcing for lack of publicly available data. None of the Discos implemented revenue protection plans. However, each Disco developed a program to reduce losses, and improve meter reading, billing and collection. Policy areas associated with improving sector performance vis-a-vis upstream electricity generation aimed at providing incentives for building up supply of natural gas. Although domestic gas is a low-cost source for power generation, it had lost its share in generation because of the government's decision to allocate gas to other sectors below current average supply cost (such as to households). This reduced the availability of gas for power generation. The DPO aimed at building up domestic gas supply. The prior actions for DPO1 included: (i) Disclosure by the Ministry of Petroleum and Natural Resources (MPNR) of the 2013 Model Petroleum Concession Agreement on its website: (ii) The MPNR was to announce award of petroleum exploration blocks; and, (iii) The Oil and Gas Regulatory Authority issued at least three pricing notifications to Page 7 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Power Sector Reform DPC(P128258) enable producers to start developing new and incremental gas production with increased prices allowed under the 2012 Petroleum Policy. Following this the MPNR notified rules for enhancing gas production from producing dormant or under producing concessions. As a prior action for DPO2, the MPNR signed supplemental agreements with revised prices for 92 exploration concessions and production leases. Under DPO2, the original trigger was modified to better target some problems in accessing new gas sources. These included: (1) signing of new and profitable agreements for further domestic exploration: and (2) Liquefied Natural Gas was to be made available at full cost recovery for the gas-fired power plants that were willing to offtake gas at such prices. Policy areas associated with opening the market to private participation aimed at separation of market operations and transmission to minimize the conflict of interest that arises by having both the dispatch of the system and market operations in the same organization. Policy areas included establishment of the Central Power Purchasing Agency (Guarantee) Limited (CPPS-G) as an independent agent to purchase electricity on behalf of distribution companies (including Discos) with its own office and having a web-site that provided information to all market participants. Outcomes. • The bills collected by DISCOs increased from 86% in 2013 to 94.6% in 2016. This exceeded both the original and revised targets of 90% and 94%, respectively. The ICR (page 14) notes that the last monitoring report by NERA indicated that trends in reduction losses were not clear. Further the Discos deliberately misreported on load shedding, the System Average Interruption Frequency (SAIFI) and System Average Interruption Duration Index. • Domestic gas supply increased from 3.8 billion standard cubic feet per day (bscfd) to 3.9 billion bscfd in June 2016. This was short of the target of 5 billion bscfd. • All contracted power generated by Independent Power Producers, the Generation Company (Genco), the Water Power and Development Authority (WAPDA) and the Hydroelectric Company (Hydel) were traded through the newly-constituted CPPA-G. The ICR (page 10) notes that while CPPA-G became an independent agency, its mandate is still temporary and the underpinning Commercial Code had not revised when the operation closed. Rating Modest PHREVDELTBL PHEFFICACYTBL Objective 3 Objective Ensuring accountability and transparency. Rationale This policy area aimed at ensuring the accountability and transparency of the sector for broader stakeholder participation. Page 8 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Power Sector Reform DPC(P128258) The prior action for DPO1 were: (I) The National Transmission and Dispatch Council (NTDC) implements web-based open access to operational information and daily payment instructions to generators; (ii) Each Disco includes the subsidy amount in customer’s bills and publishes on its website the monthly billing and collection data aggregated by consumer category: and (iii) The Economic Coordination Committee (ECC) of the Pakistan Cabinet approves the establishment of monitoring units within both the Ministry of Water and Power (MWP) and the Ministry of Petroleum and Natural Resources (MPNR) with responsibilities to monitor the energy sector and report on a quarterly basis. The triggers for DPO1 were: (I) The CPPA-G was to implement web-based access to monthly amounts due and payments by each Disco to CPPA-G; and (ii) MWP implements public web-based access to monthly results of performance contracts signed with Discos, the NTDC and Gencos. The triggers for DPO2 were: (i) CPPA-G publicly discloses on its website the monthly amounts due and payments made by each Disco to CPPA-G to Generators; and (ii) NEPRA discloses the annual Discos performance and evaluation report. Outcomes. •Disco performance reports and the National Electric Power Regulatory Agency (NEPRA) review was published. Rating Substantial PHREVDELTBL PHREVISEDTBL 5. Outcome The relevance of objectives to the government and the Bank strategy is high. The relevance of design is substantial. The efficacy of two objectives – i.e., reducing subsidies and ensuring accountability and transparency -- is substantial. The efficacy of the third objective – i.e., improving sector performance and opening the market to private participation - is modest. The overall outcome is moderately satisfactory. a. Outcome Rating Moderately Satisfactory 6. Rationale for Risk to Development Outcome Rating There are high risks to the sustainability of the development outcome of the PSRDDPO. One, there are risks associated with backsliding of policy reforms, pertaining specifically to subsidies, due to the political economy context and in the face of political pressures or popular resistance. This is particularly so Page 9 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Power Sector Reform DPC(P128258) given that past reforms in the energy sector have floundered in the face of vested interests within the sector and government inefficiencies. There are also risks associated with social and industrial opposition to further reform if service delivery does not improve. Although increases in billing levels have been offset by lower oil prices, there are risks that stakeholder expectations of service quality improvement, especially a reduction in load shedding, will not be fulfilled. Two, although the Central Power Purchasing Agency (CPPA-G) has been set up as an independent agency, its roles and responsibilities are yet to be defined. The future of the privatization of Discos remains in doubt. Three, given Pakistan's vulnerability to external and internal shocks, there is high risk that these shocks could cause macroeconomic instability. While the government has a strategic action plan under the National Power Policy of 2013, delays or ineffective implementation of policy measures could result in the government not being able to maintain subsidy support to the power sector. a. Risk to Development Outcome Rating High 7. Assessment of Bank Performance a. Quality-at-Entry This operation was prepared based on the lessons from previous energy lending operations in Pakistan and the Independent Evaluation Group’s Project Performance Assessment Report No 34101 issued on December 19, 2005 for the Public-Sector Adjustment Loan/Credit (Ln 3645-PK, Cr. 2542 PK), the Structural Adjustment Loan (Ln 4435- PK), and the Structural Adjustment Loan and Structural Adjustment Credit (Cr 3655). Lessons incorporated at design included having a dedicated energy sector operation as opposed to inserting a limited number of prior actions in a multi-sector or general macro-economic support operation and making allowance for continuous assessment of progress and adaptation as the reform evolved through changing indicative triggers and indicators. The DPC program was prepared by a team working in close collaboration with a team preparing another DPO series called "Financial Sustainable and Inclusive Growth", which focused on macroeconomic reform, including revenue collection. The operation was prepared based on close collaboration with the International Monetary Fund (which had an Extended Fund Facility, a substantial focus of which was structural adjustment in the energy sector), the Asian Development Bank and the Japan Agency for International Development. This collaboration aided in providing a harmonized approach to policy-based lending and reduced the government's transactional costs. The sequencing of the design of the operation was coherent, though the focus was on addressing the crisis management in the first phase and addressing sector reforms in the second phase. While this aspect of design was aligned with the government's interests, it is not clear if this was entirely appropriate, given the government's waning on the commitment to sector reform, particularly with regards to power distribution. The team clarified that both crisis management and sector reform were conceived as a coherent package. However, phase one proceeded with great speed and a sense of urgency, since the Development Policy Credit was designed to ensure that the governmental finances would not collapse and thereby cause further civil unrest in a region that was already highly volatile. The team further clarified that the weak Page 10 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Power Sector Reform DPC(P128258) management capacity of the operating companies and the Ministry of Water and Power were clear to the team and the reform actions aimed at addressing precisely these issues. Quality-at-Entry Rating Satisfactory b. Quality of supervision The budget allocated and staff resources were adequate, which aided in the close monitoring of project activities. The supervision team provided technical assistance to the government in (i) analyzing options to introduce changes before the round of tariff increases and (ii) addressing issues associated with separation and operationalization of the Central Power Purchasing Agency (Guarantee) Limited. The team closely monitored the program's progress and worked effectively with Country Management Unit in dialogue with the government. Support provided by the team include Vice President Unit- level assistance, which aided the performance of the operation. The team worked in close collaboration with the IMF, including through participation in the quarterly meetings, which aided the IMF in their program. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 8. Assessment of Borrower Performance a. Government Performance The new government formed following the general election of April 2013 showed strong commitment to energy sector reforms at preparation. The government agreed to an Extended Fund Facility (EFF) with the International Monetary Fund early in its term, a major feature of which was the structural reform of the energy sector (particularly relating to subsidy and tariff reforms, further development of the market and improving transparency and accountability). Other actions by the new government included: (i) settling the circular debt in June 2013 (the government cleared 1.5% of GDP of the circular debt in June 2013 and an additional 0.6% of GDP in July 2014, and put a cap on circular debt); and (ii) articulating the National Power Policy in 2013 and increasing energy tariffs in August and December 2013. Following the windfall gains produced due to the unforeseen fall in oil prices, the government used the gains to bring about structural reform by introducing a mechanism based on tariff surcharges and a Tariff Rationalization fund to maintain national uniform tariffs in Discos. While the government's commitment to the program objective remained strong during DPO1, the commitment waned during DPO2. The team clarified that the government remained fully committed to the Development Policy Credit under the first phase, but the commitment somewhat waned in the area of power distribution. The team further clarified that the commitment however remained strong in the gas sector, which was the most important part of the reform and this contributed substantially to stabilizing the supply of power in Pakistan. Page 11 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Power Sector Reform DPC(P128258) Government Performance Rating Satisfactory b. Implementing Agency Performance --- Implementing Agency Performance Rating --- Overall Borrower Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The Program Document for PSRDPO1 presented a set of indicators and associated targets that were consistent with the outcomes being measured and were sufficient for Monitoring and Evaluation. The three key outcome indicators were measurable and included: (i) reduction in subsidies as a percent of GDP; (ii) the increased collection by Discos (as percent of total billing), and (iii) the increase in gas supply realized during the operation. There were baseline figures for these indicators and specified targets. Because some of the outcomes were redefined by the following operation, the associated indicators and targets had to be revised accordingly. M&E was designed to be reported jointly with the other two parallel donors - the ADB and JICA. b. M&E Implementation The targets for reducing subsidies were reduced in the face of the government's commitment to maintaining subsidies to consumers using up to 300 kWh per month on account of political economy considerations. The quantitative indicators for monitoring were through reported indicators, such as through the government data for monitoring the trend in subsidies. Regular monitoring of sector reforms was through face-to face interaction and more formally, during quarterly IMF program reviews. c. M&E Utilization Data gathered from M&E implementation were used to inform the design of the policy matrices, for decision making on policy work, and for the dialogue with the Government in shaping and improving power sector governance. Page 12 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Power Sector Reform DPC(P128258) M&E Quality Rating Substantial 10. Other Issues a. Environmental and Social Effects Poverty Impact. A dedicated Poverty and Social Impact Analysis (PSIA) was undertaken for the two DPOs. Given that the subsidies for consumers using up to 300 kWh per month were maintained by the government, no adverse effects from the reforms on low-income electricity consumers compared to the status quo were expected. The ICR (page 19) reports that energy subsidies continue to be regressively targeted, given that poor households remain exposed to high bills especially in the summer months. Gender Issues. The program did not explicitly target gender issues. The PSIA conducted for the DPO2 noted that reforms targeting load shedding in Pakistan disproportionately benefits women because women are more affected by load shedding and by the efforts to manage household electricity expenses, since they are on average the main users of electrical appliances and spend more time at home. b. Fiduciary Compliance Procurement. The DPO proceeds were provided as general budget support to the Government and not earmarked for any agency or purpose. Hence there were no special procurement requirements or any related issues. Financial Management. The ICR does not cite any financial audits or financial management issues with the PSRDPO. c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Efficacy of one objective - to Outcome Satisfactory Moderately Satisfactory improve sector performance and open the market to private Page 13 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Power Sector Reform DPC(P128258) participation - is rated as Modest. Risk to Development High High --- Outcome Bank Performance Highly Satisfactory Satisfactory --- Borrower Performance Satisfactory Satisfactory --- Quality of ICR Substantial --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 12. Lessons Five lessons are drawn from the ICR (pages 22-24), with some adaptation of language. 1) A dedicated sector energy reform operation (as opposed to inserting a limited number of prior actions in a multi-sector or general macro-economic support operation) can be more useful in countries facing severe energy sector challenges. The DPOs allowed for the modification of indicators and targets that could better reflect changing circumstances and evolving policy dialogue. For instance, the targets for reducing subsidies were revised in the face of governments' commitment to maintaining subsidies for political economy considerations. Likewise, the government was able introduce, as complementary measures, a mechanism based on tariff surcharges and a Tariff Rationalization Fund to maintain nationally uniform tariffs in the face of windfall gains produced alongside falling oil prices during the operation period. (2). The government's ownership of reform is necessary for the success of a development policy operation. In this operation, the government's ownership and involvement in the reform of the power sector was focused during preparation. The government issued notifications regarding reducing subsidies, increasing tariffs substantially and setting a cap on circular debt. When these priorities were no longer present, the leverage of the Ministry of Finance (MoF) over the Ministry of Water and Power (MWP) diminished and MWP no longer was as committed to sector reforms as it was during preparation. This contributed to the limited efforts at sector reforms during DPO2. (3) A balanced approach among instruments targeting federal and sectoral levels can help bring about more sustained results. Federal DPOs have supported IMF's programs to stabilize the macroeconomy by providing budgetary support for reforms aimed at improving energy sector sustainability and private sector growth. A more strategically balanced mix of budget support with other Bank instruments could help leveraging more sustainable results. (4) Close collaboration with the International Monetary Fund can benefit a DPO program. Closely aligned positions among donors could help in pushing through difficult reforms. Collaboration with the IMF could have played a role in effective completion of the IMF program. (5) In the context of a DPO operation, it can be difficult to implement reforms, even when a requisite prior action has been taken. To help improve the collection rates and reduce distribution losses of Discos, one prior action in DPO 1 had required the outsourcing of collections of Discos. Even though the prior action Page 14 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Power Sector Reform DPC(P128258) was met, only two distribution companies proceeded with the outsourcing of the collection of feeders with losses of 50% or above. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR is well-written and has a concise narrative. It provides a candid discussion of the energy sector in Pakistan and transparently discusses the issues faced during implementation. It is also candid when discussing the waning government commitment after the first DPO1. The ICR draws good lessons from the experience of implementing this operation. The ICR on page 14 states that some Discos were deliberately misreporting on load shedding, without providing sufficient details. The team clarified that the issue of misreporting on load shedding was taken from the monitoring report by the National Electric Power Regulatory Agency's (NEPRA's) website. Similar allegations were made by NEPRA when the ICR team visited NEPRA and that NEPRA did not provide any evidence for these observations beyond the reports. a. Quality of ICR Rating Substantial Page 15 of 15