Document of The World Bank Report No: ICR0000548 IMPLEMENTATION COMPLETION AND RESULTS REPORT (MULT-28200) ON A GRANT FROM THE MULTILATERAL FUND IN THE AMOUNT OF US$13 MILLION TO THE ISLAMIC REPUBLIC OF PAKISTAN FOR A OZONE DEPLETING SUBSTANCES PHASE OUT PROJECT September 21, 2007 Environment and Water Resource Management Unit South Asia Sustainable Development South Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective July 17, 2007) Currency Unit = Pakistan Rupee (PKR) US$ 1.00 = PKR 60.45 FISCAL YEAR January 1 ­ December 31 ABBREVIATIONS AND ACRONYMS CBR Central Board of Revenue MoE Ministry of Environment CFC Chlorofluorocarbon, an ozone MP Montreal Protocol on Substances that depleting substance Deplete the Ozone Layer CP Country Program MLF Multilateral Fund for the Implementation of the Montreal Protocol CPU Country Program Update MT Metric Ton, a unit of measure equivalent to 1,000 kilograms CTC Carbon Tetrachloride, an ozone MTR Mid-Term Review depleting substance DAL Domestic Appliances Ltd. NBP National Bank of Pakistan ExCom Executive Committee of the NDFC National Development Finance Multilateral Fund Corporation FI Financial Intermediary ODP Ozone Depleting Potential FY Fiscal Year ODS Ozone Depleting Substance GEO Global Environmental Objectives OTF Ozone Projects Trust Fund GOP Government of Pakistan PCR Project Completion Report IA Implementing Agency PSR Project Status Report ICR Implementation Completion RMCPL Refrigerators Manufacturing Co. Report Pakistan Limited IOC Incremental Operating Costs SGA Sub-Grant Agreement LC Letter of Credit UNEP United Nations Environment Programme MDI Metered Dose Inhalers UNIDO United Nations Industrial Development Programme MELG&RD Ministry of Environment, Local Government, and Rural Development Vice President: Praful C. Patel Country Director: Yusupha B. Crookes Sector Manager: Karin E. Kemper Project Team Leader: Ernesto Sanchez-Triana ICR Team Leader Ernesto Sanchez-Triana PAKISTAN Montreal Protocol Ozone Depleting Substances Phase Out Project CONTENTS A. Basic Information........................................................................................................ i B. Key Dates.................................................................................................................... i C. Ratings Summary........................................................................................................ i D. Sector and Theme Codes ...........................................................................................ii E. Bank Staff...................................................................................................................ii F. Results Framework Analysis.....................................................................................iii G. Ratings of Project Performance in ISRs ................................................................... iv H. Restructuring (if any)................................................................................................. v I. Disbursement Profile.................................................................................................. v 1. Project Context, Global Environment Objectives and Design ................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 6 3. Assessment of Outcomes.......................................................................................... 11 4. Assessment of Risk to Development Outcome......................................................... 13 5. Assessment of Bank and Borrower Performance ..................................................... 13 6. Lessons Learned ....................................................................................................... 15 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 17 Annex 1. Project Costs and Financing.......................................................................... 18 Annex 2. Outputs by Component ................................................................................. 19 Annex 3. Economic and Financial Analysis................................................................. 20 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 21 Annex 5. Beneficiary Survey Results........................................................................... 23 Annex 6. Stakeholder Workshop Report and Results................................................... 24 Annex 7. Summary of Borrower's ICR and Comments on Draft ICR ......................... 25 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders....................... 32 Annex 9. List of Supporting Documents ...................................................................... 33 Annex 10. Additional Annexes..................................................................................... 34 MAP IBRD 33460 A. Basic Information PHASE OUT OF ODS Country: Pakistan Project Name: PRE Project ID: P034301 L/C/TF Number(s): MULT-28200 ICR Date: 09/21/2007 ICR Type: Core ICR Lending Instrument: SIL Borrower: GOP Original Total USD 13.0M Disbursed Amount: USD 8.6M Commitment: Environmental Category: B Global Focal Area: O Implementing Agencies: Ministry of Environment (Ozone Cell) National Bank of Pakistan Cofinanciers and Other External Partners: B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 10/17/1994 Effectiveness: 08/04/1997 08/04/1997 Appraisal: 07/15/1995 Restructuring(s): Approval: 01/23/1997 Mid-term Review: 09/17/2004 Closing: 06/30/2001 12/31/2006 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Global Environment Outcome Low or Negligible Bank Performance: Satisfactory Borrower Performance: Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Satisfactory Quality of Supervision: Satisfactory Implementing Agency/Agencies: Satisfactory Overall Bank Overall Borrower Performance: Satisfactory Performance: Satisfactory C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Performance Indicators (if any) Rating Potential Problem Project No Quality at Entry None i at any time (Yes/No): (QEA): Problem Project at any Quality of Yes None time (Yes/No): Supervision (QSA): GEO rating before Satisfactory Closing/Inactive status D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 2 2 Other industry 98 98 Theme Code (Primary/Secondary) Pollution management and environmental health Primary Primary E. Bank Staff Positions At ICR At Approval Vice President: Praful C. Patel Joseph Wood Country Director: Yusupha B. Crookes Mieko Nishimizu Sector Manager: Karin Erika Kemper Maritta R. V. B. Koch-Weser Project Team Leader: Ernesto Sanchez-Triana Jitendra J. Shah ICR Team Leader: Ernesto Sanchez-Triana ICR Primary Author: Mary-Ellen Foley ii F. Results Framework Analysis Global Environment Objectives (GEO) and Key Indicators(as approved) The objective of the project is to assist Pakistan's transition into non-CFC technology. The project will help Pakistan reach its objective by: (i) supporting the GOP's proposed program to phase out ozone depleting substances (ODS); (ii) implementing cost effective priority subprojects identified in the Country Program for technical conversions; and (iii) building local capacity to identify, develop and implement ODS phaseout. Revised Global Environment Objectives (as approved by original approving authority) and Key Indicators and reasons/justifications Not applicable (a) GEO Indicator(s) Original Target Formally Actual Value Indicator Baseline Value Values (from Revised Achieved at approval Target Completion or documents) Values Target Years Indicator 1 : Help Pakistan achieve its commitments under the Montreal Protocol for phaseout of CFCs, which are (i) 50% phaseout by 2005, and (ii) 85% phaseout by 2007. Initially 640 ODS MT (calculated on the basis of the 7 17 subprojects were Value Zero. No phaseout had subprojects in the completed (quantitative or been done when the foam and amounting to the Qualitative) project started. refrigeration phaseout of 1,243 sectors identified ODS MT. at the time of approval). Date achieved 01/15/1997 12/31/2006 12/31/2006 Comments 100% of 640 ODS MT from subprojects identified at approval plus an additional (incl. % 603 ODP MT resulting from additional subprojects identified during the achievement) project's life. Indicator 2 : Build local capacity to identify, develop and implement ODS phaseout. Ozone Cell has Ozone Cell will be been enabled to able to sustain sustain ODS ODS Phaseout Phaseout after Value Ozone Cell established Country project closure and (quantitative or during project Programme after complete phaseout Qualitative) preparation. Bank project of CFC by 2010. closes and Capacity of NBP complete phaseout was strengthened to of CFC by 2010. identify, develop and implement ODS phaseout. Date achieved 01/01/1996 12/31/2006 12/31/2006 iii Comments (incl. % achievement) (b) Intermediate Outcome Indicator(s) Original Target Formally Actual Value Indicator Baseline Value Values (from Achieved at approval Revised Completion or documents) Target Values Target Years Indicator 1 : Implement cost effective priority subprojects, identified in the Country Programme for technical conversion to replace use of CFCs. Value Seven subprojects About 10 (quantitative or identified at the time the subprojects within 17 subprojects were Qualitative) project was approved. 48 months of GA completed. signing. Date achieved 01/15/1997 12/31/2006 12/31/2006 Comments (incl. % achievement) G. Ratings of Project Performance in ISRs Actual No. Date ISR Archived GEO IP Disbursements (USD millions) 1 10/04/2000 Satisfactory Unsatisfactory 1.65 2 12/19/2000 Satisfactory Satisfactory 2.12 3 06/17/2001 Satisfactory Satisfactory 2.17 4 12/28/2001 Satisfactory Satisfactory 2.34 5 06/14/2002 Satisfactory Satisfactory 3.62 6 12/03/2002 Satisfactory Satisfactory 4.72 7 06/01/2003 Satisfactory Satisfactory 5.09 8 12/30/2003 Satisfactory Satisfactory 5.20 9 06/29/2004 Satisfactory Satisfactory 5.31 10 12/28/2004 Satisfactory Satisfactory 5.80 11 01/06/2005 Satisfactory Satisfactory 5.80 12 05/13/2005 Satisfactory Satisfactory 6.59 13 07/01/2005 Satisfactory Satisfactory 6.69 14 01/13/2006 Satisfactory Satisfactory 7.89 15 07/20/2006 Satisfactory Satisfactory 8.23 16 06/22/2007 Satisfactory Satisfactory 8.62 iv H. Restructuring (if any) Not Applicable I. Disbursement Profile v 1. Project Context, Global Environment Objectives and Design The Montreal Protocol Ozone Depleting Substances Phase Out Project channeled grant funding from the Multilateral Fund (MLF) for the Implementation of the Montreal Protocol to Pakistan to finance the incremental costs of conversion to ozone friendlier technology for a group of subprojects in two industrial sectors. This contributed to the Government of Pakistan's ability to reduce demand for chlorofluorocarbons (CFC) and thereby, assisted it in meeting its obligations to the Montreal Protocol (MP). The country and sector background and rationale for World Bank assistance is described below. 1.1 Context at Appraisal Country and Sector Background Preceded by two decades of relatively fast economic growth and moderate inflation, in 1993 the Government of Pakistan embarked on a program of macroeconomic adjustment and structural reform that aimed to enhance the country's economic performance. Pakistan's economic growth, along with greater population growth, urbanization, and industrialization posed significant challenges to the country's environment. In response, a National Conservation Strategy was adopted in 1992 followed by an Action Plan for 1993-1998 that aimed to improve natural resource management. Provincial environmental strategies, as well as a new comprehensive environmental law, were developed. Pakistan's commitment to address environmental problems also included the ratification in 1992 of the Vienna Convention, the Montreal Protocol, the London Amendment, and of the Copenhagen Amendment in 1995. These instruments were the response of the international community to control the production and consumption of substances that scientific assessments found to be causing the thinning of the Earth's ozone layer, resulting in more harmful ultraviolet radiation (UV-B) reaching the surface of the Earth. Ozone depleting substances (ODS) controlled by the MP are categorized as Annex A substances (CFC and halon), Annex B substances (including carbon tetrachloride (CTC)), Annex C substances (including hydrochlorofluorocarbons (HCFCs)), and Annex E substances (methyl bromide (MeBr)). Developed countries that are Parties to the MP initially committed to phase out Annex A substances by the year 2000 (which was accelerated to 1996 in a 1993 adjustment to the MP). Recognizing the special situation of developing countries, Article 5 of the MP established that developing countries with an annual consumption level of Annex A substances of less than 0.3 kg per capita could benefit from a ten-year grace period, up to 2010. Moreover, Article 5 countries were automatically eligible to receive technical and financial assistance from the Multilateral Fund (MLF) to meet the agreed incremental costs for complying with the Protocol's control measures. With an estimated annual per capita consumption of 0.018 kg of ODS substances at the time of project appraisal, Pakistan was also eligible for MLF assistance. 1 Government of Pakistan ODS Phaseout Strategy The Ministry of Environment, Local Government, and Rural Development (MELG&RD) led the development and implementation of the national program to comply with Pakistan's obligations under the MP. MELG&RD defined three program components, namely: (i) the development of a phase out strategy; (ii) setting up of institutional arrangements to implement the strategy, and (iii) an investment component comprising technical and financial assistance to enterprises for the adoption of non-ODS technologies. Taking advantage of Pakistan's eligibility to receive MLF assistance, MELG&RD sought support for the development and implementation of these components from the four MLF implementing agencies: the World Bank; United Nations Environment Programme (UNEP); United Nations Development Programme (UNDP); and the United Nations Industrial Development Organization (UNIDO). With UNEP's assistance, Pakistan prepared its strategy, or Country Program (CP) in 1996 which laid out the investment, non-investment, institutional and policy actions that the country would undertake to phase out ODS and comply with its MP obligations. The main sectors targeted by the CP were domestic and commercial refrigeration, as well as foam. Under the CP, the Government of Pakistan's (GOP's) main role consisted of providing information, financial incentives in the form of lower tariffs on equipment and materials required for conversion to non- CFC technology, and instituting a ban on imports of CFC-based equipment. The CP further envisioned that, responding to such stimuli, the industry would decide how and when to convert to non-ODS technology. The following table summarizes Pakistan's obligations under the MP by ozone depleting substance. Consumption is defined in the Protocol as imports plus production (not applicable in the case of Pakistan), minus exports of ODS. BASELINE CONSUMPTION OF ODS IN PAKISTAN AND MP OBLIGATIONS Montreal Protocol (ODP MT) CFC Halon Methyl CTC TCA Bromide Baseline consumption (1995-1997) 1679.4 14.2 14* 412.9** 2.3** Allowed consumption during 1999-2004 1679.4 14.2 14 412.9 2.3 Allowed level of consumption in 2005 839.7 7.1 11.2 62.00 1.61 Allowed level of consumption in 2007 251.91 7.1 11.2 62.00 1.61 Allowed level of consumption in 2010 0 0 11.2 0 0.69 Allowed level of consumption in 2015 0 0 0 0 0 Source: World Bank and Ozone Cell (2002), "The Islamic Republic of Pakistan: Country Programme Update". *1995-1998 baseline **1998-2000 baseline The second component, institutional arrangements for the implementation of the CP, received support from UNDP and included the establishment of an Ozone Cell within MELG&RD to develop and implement all ODS-related activities within Pakistan and to guide the country's ODS phase out program. Activities undertaken by the Ozone Cell included assisting in the preparation of legislation, developing public awareness campaigns, making ODS-related information available, and facilitating the enterprises' coordination with the MLF implementing agencies. The final component was the provision of technical and financial assistance to enterprises for the adoption of non-ODS technologies (the investment component). This component was to be 2 supported by UNIDO and the World Bank. Whereas the former would focus on small and medium-scale subprojects, the original intention in GOP's strategy was that the Bank would work with the larger enterprises in the foam and refrigeration sectors. Country Program Update During the implementation of the CP it was seen that there was a need to reassess the level of ODS consumption on a sector level. First, a number of ODS-consuming firms had not been identified in the CP and some sub-sectors were excluded. Second, exogenous factors led to the restructuring of the composition of relevant markets, with some firms exiting the market and others increasing their market share while overall growth in these economic sectors continued. This meant that the level of reductions of ODS from projects was not necessarily reducing baseline consumption. In this context, Pakistan received funding in 2002 to carry out a Country Programme Update (CPU) with the World Bank's assistance. The CPU's objectives were to provide information on recent ODS use, correct any past data reporting inconsistencies and incorporate into an overall ODS phaseout strategy the sectors and ODS consumption not targeted in the original CP. According to the CPU, total ODS consumption in Pakistan amounted to 2,300.7 ODP MT as of December 2002 (excluding HCFC). This amount indicated that between 1995 and 2002, the level of consumption decreased slightly overall, and increased in two sectors (solvents and halon) despite ongoing MLF projects and in light of the impending 50% MP reductions in Annex A substances by 2005. The following table compares actual ODS consumption by controlled substance in 1995 and 2002 with Pakistan's MP compliance baseline. The phaseout achieved up to 2002 at the subproject level through MLF and implementing agency support (also included in the table) did not translate to significant consumption reductions at the national level. COMPARISON OF ODS CONSUMPTION BY CONTROLLED SUBSTANCE ODS Consumption in Phase out Compliance Remaining CP (ODP MT) achieved 2002 baseline consumption in 1995 thru MLF CPU (ODP MT) subprojects 2002 CFC 2102.7 691.83 1679.4 1646.71 Halons 21 0 14.4 16.95 Methyl 0 0 14 0 Bromide TCA 12.8 0 2.3 0 CTC 614.9 0 412.9 636.9 Source: World Bank and Ozone Cell (2002), "The Islamic Republic of Pakistan: Country Programme Update". The outcomes of CP implementation also underscored the need for more interventions from the GOP. When the CP was developed, one regulation was in place ­ a ban on the trade of ODS with non-Parties to the MP. In the following years, a number of additional regulations were phased in. Key regulations included: i) a licensing system introduced in 1998 by MELG&RD and the Ministry of Commerce (MoC) to allow only registered companies to import ODS; ii) an Import Policy Order (SRO 895(1) 99 Import Policy) to restrict the level of ODS imports; and iii) the 2000 Ozone Depleting Substances policy which among others, introduced a progressive CFC quota reduction schedule starting in 2003. 3 1.2 Original Global Environment Objectives (GEO) and Key Indicators The objective of the project was to assist Pakistan's transition to non-CFC technology. The project aimed to help Pakistan reach its objective by: (i) supporting the GOP's proposed program to phase out ODS; (ii) implementing cost-effective priority subprojects identified in the Country Program for technical conversions; and (iii) building local capacity to identify, develop and implement ODS phaseout. The project utilized an Umbrella Grant Agreement under which the GOP and the Bank agreed on the overall objectives of the project, as well as on the eligibility and operational criteria to assess and support the implementation of technology conversion subprojects. The Agreement was followed by a Memorandum of Agreement signed by the Economic Affairs Division, MELG&RD, and the Financial Intermediary (National Development and Financial Corporation- NDFC). MELG&RD, as lead agency for implementing the country program, was responsible for overseeing the project on behalf of the GOP and for ensuring that subprojects were consistent with the ODS Country Program. MELG&RD had the responsibility for issuing the final clearance of subprojects that were presented for funding, as well as for implementing a policy framework to encourage early phaseout of ODS. As financial intermediary, NDFC's responsibilities included: assisting in promotion of subproject identification, administering funds allocated by the Bank to subprojects endorsed by ELG&RD and approved by the Executive Committee of the MP and by the Bank, evaluating financial viability of enterprises based on the eligibility criteria agreed with the Bank, disbursing grants to subproject beneficiaries, and supervising subproject implementation. MLF's resources from the Bank were channeled through NDFC to ODS users to cover the incremental costs of ozone protecting technologies, understood as the difference between the cost of complying with MP targets and the costs that would be incurred without the MP. After each subproject had been endorsed by MELG&RD and the appraisal reports had been approved by the Bank and the Executive Committee of the MP, sub-grant agreements were signed between NDFC and subproject beneficiaries according to a model sub-grant agreement approved by the Bank. Subprojects also included technical assistance for technology transfer, design, safety, training, and implementation to beneficiaries. The original project indicators used by the Bank to monitor performance of ODS projects were established by the MLF and the Bank's MP Unit. Main indicators included ODP phased out during and after project implementation; disbursement rate; compliance with baseline CFC equipment disposal requirements; time from approval of subproject to signing of the sub-grant agreement; cost effectiveness; and time to project completion. Individual project completion reports (PCRs) for each subproject that were required by the MLF contain further details on these indicators and actual performance. MELG&RD was responsible for monitoring the overall project and achieving the subprojects' ODS phaseout targets, while enterprises remained responsible for implementing their own subprojects. NDFC was required to prepare reports for MELG&RD and the Bank, monitor compliance of enterprises (including equipment disposal agreements), prepare jointly with enterprises subproject completion reports upon final implementation of each subproject, and monitor compliance with environmental and safety standards, among other requirements. 4 1.3 Revised GEO (as approved by original approving authority) and Key Indicators, and reasons/justification The original global environment objectives were not revised. However, they were slightly reoriented to focus more directly on compliance with GOP's Montreal Protocol obligations through the original project approach ­ implementation of subprojects and building local capacity to develop and implement ODS phaseout projects ­ rather than on the CP. As previously mentioned, during the implementation of the project it was found that some sectors using CFC had not been identified in Pakistan's Country Programme. By looking at CFC consumption (import) data, it was seen that CFC use was greater than first assessed in the CP. For example, there was one large sector of thermoware in rigid foam insulation that had not been identified by the CP (which subsequently was included under the ODS Project through two subprojects). ODS consumption was reevaluated in different sectors and enterprises, as well as the total ODS consumption of the country in the Country Program Update. The Bank assisted the Government in preparing the CPU, which also laid out the strategy for the phased reduction and phaseout of controlled substances by the year 2010, as scheduled under the Montreal Protocol (halon, CFC and CTC). It encompassed initiatives to strengthen general and sector specific policies, and deliver technical assistance to ensure compliance with the phase out of ODS. The CPU identified the final project activities required to completely end the use of CFC in manufacturing in Pakistan. The ODS Project objective was therefore slightly recast to better align enterprise-level CFC phaseout with Pakistan's impending Montreal Protocol CFC reductions (50% by 2005, 85% by 2007, and 100% by 2010) and its associated quota system. 1.4 Main Beneficiaries The primary target group as identified at appraisal was CFC-consuming enterprises in the foam and the refrigeration manufacturing sectors. The project anticipated that more firms from the refrigeration and foam sectors than originally identified at the time of appraisal (seven subprojects) would benefit from the financial assistance to convert to non-CFC technologies. By the end of the Project, a total of 21 subprojects had been approved for these sectors by the MLF (including four subprojects that were later cancelled due to beneficiary choice or liquidation). The project also included a technical assistance component to strengthen the Financial Intermediary's (i.e., NDFC) technical capabilities in ODS-related data collection, project implementation review and supervision. This component had a significant impact as it helped NDFC to develop managerial skills and technical understanding of ODS-related issues that were crucial to advance project implementation. Finally, the GOP benefited from the Project in that it helped deliver multilateral financing for incremental costs associated with complying with the MP. In turn, global benefits were accrued insofar that additional emissions of ozone-depleting substances were avoided. 1.5 Original Components (as approved) The two original components of the ODS Phase-out Project were: 1) an investment component which would include approximately 10 subprojects and 2) a Financial Intermediary (FI) Fee, which was used by NDFC, along with additional Bank resources, to fund technical assistance to 5 strengthen NDFC's technical capacity for project implementation. These components were captured as follows by the Grant Agreement: No. Component Finance allocation (US$ million) 1. Subprojects 12.61 2. Financial Intermediary Fee 0.39 Total 13.00 An additional US$2 million was envisioned as counterpart funding to contribute to meeting the incremental costs of converting to non-CFC technology. The design of the Project is directly related to the achievement of objectives in that the FI effectively served as an executing agency for all subprojects in Component 1 and enabled funds to be channeled to the beneficiaries for technology conversion and ODS phaseout while safeguarding MLF funding. Implementation of Component 1 served to strengthen the FI's administrative capacity over the course of the project. 1.6 Revised Components The above components were not revised. In practice, counterpart funding amounted to nearly US$3 million (see Annex 10.1 for a breakdown by subproject). 1.7 Other significant changes The closing date was extended twice primarily because i) some subprojects experienced delays at the appraisal and implementation stages; ii) additional CFC users were identified at a later stage, leading to required revisions of the country strategy to comply with ODS phaseout obligations; and, ii) a disconnect between early GOP policy and investment grants, resulting in reduced incentives for timely technological conversions. (See next section for further discussion). 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry The umbrella project was modeled on other ODS phaseout projects under the Bank in its role as one of four implementing agencies of the MLF. The modality served its purpose well, enabling an overall agreement to be reached with the GOP on ODS phaseout while individual subprojects were approved by an external body (the Executive Committee of the MLF) on a rolling basis based on demand over the duration of the grant agreement. The umbrella ceiling amount of US$13 million was sufficient to absorb the total cost of subprojects and fees approved by the MLF. The design was appropriate for the capacity and needs of the Government of Pakistan. The prominent role in project implementation that was given to the designated Financial Intermediary, NDFC, permitted program continuity in times of flux within the Ministry of Environment (MoE)1. 1The Ministry of Environment, Local Government and Rural Development was bifurcated in pursuance of Cabinet Division notification number SRO.826 (1)/2002 and an independent Ministry of Environment was established on April 22, 2002. 6 Some of the limitations of the design are apparent when looking at the actual duration of the ODS Project in comparison to the planned timeframe. The closing date was extended twice for a total duration of ten years instead of 4.5 years. In retrospect, the planned duration was perhaps overly ambitious due to several reasons described next. In the first place, the duration of the project had been largely based on the expected subproject demand as identified in Pakistan's first Country Program; an ODS consumption assessment and action plan, prepared with the assistance of UNEP (one of the MLF Implementing Agencies). However, during project implementation, two important developments occurred: first, initially identified subprojects did not move forward and were eventually cancelled; and second, additional ODS users were later identified in several sectors. Another reason for the length of implementation was apparent at the subproject design level. During subproject technical appraisal, some projects were significantly modified to account for unforeseen changes. These changes were at times a departure from Executive Committee approval criteria and time was needed to devise a plan that met the needs of the beneficiaries while still fulfilling MLF requirements. These changes were due to the lag in time between preparation and approval of funding, changes in enterprise business and their markets and to some extent, initial assessments in the project preparation phase were rushed. These challenges in subproject implementation are directly linked to the most significant aspect not foreseen in project design ­ the critical role of concurrent government policy to provide added incentive for enterprises to convert to alternative production technologies. Enterprise demand for subprojects was limited to available grant funding and not to changes in the market, which provided more economic gain in the short-term given high growth rates in the sector and low CFC cost. This omission in project design was more common in earlier Bank ODS umbrella projects. In fact, this aspect was later added by the Bank in new umbrella projects and was subsequently adopted by the MLF as a new strategic approach to ODS phaseout assistance: the linking of policy to investment incentives to achieve sustained ODS phaseout. Nonetheless, the dialogue instituted with the GOP through the project (reaching its pinnacle during the CPU preparation) did lay the path for policy development. The Government of Pakistan did implement more comprehensive regulations beginning in 2000 and when these came into effect, the pace of project implementation in the refrigeration sector quickly accelerated. The GOP has sought to complement such regulations with adequate training for officials who are responsible for their enforcement. This is illustrated by the Refrigerant Management Plan (RMP) which is implemented by UNIDO and aims to train senior customs officials, along with other activities, including training technicians and supplying servicing equipment. The regulations enacted by the Government, coupled with careful planning for the implementation of investment projects, have been fundamental to ensure that Pakistan is in compliance with its obligations with the MP. An action plan was prepared in 2003 to eliminate the use of CFCs in the manufacturing sector on a priority basis so as to end the manufacturing of CFC-based refrigerators, while commencing activities in the servicing sector to ensure sufficient CFC supply in the coming years. With the quota system imposed by the Government and CFC in the manufacturing sector completely phased out, Pakistan is in compliance. 7 2.2 Implementation Some of the factors that affected implementation were beyond the control of the GOP and the implementing agency. Technology itself played a less prominent role in the project outcome as might have been expected, however, it was a factor in terms of its appeal to enterprises as compared to the pace of project initiation once a subproject was approved by the MLF. Some enterprises requested changes in equipment and at times in technology, to adapt to new market circumstances. This led to some delays during the technical appraisal. Non-CFC technology was readily adopted by the foam sector, in part because the technology was well-proven and the alternatives were marginally higher in cost than CFC. However, in the refrigeration sector enterprises were reluctant to switch over to non-CFC technology because, among other reasons, more than one technology was available and enterprises were reluctant to take the lead and preferred to "wait and see", especially in the absence of regulatory and market pressures to reduce the high cost differential between CFC-12 and the alternatives. Some resistance was also encountered in regards to cyclopentane, which was the cheapest technology in terms of operating costs but what was perceived to be dangerous and requiring high upfront costs. Moreover, early in implementation, cyclopentane suppliers were difficult to find. Another factor which had an impact on project implementation and outcomes was the role of the MLF and the guidelines and policies governing the use of grant funding. The MLF Executive Committee issued decisions and guidelines throughout ODS Project implementation which were expected to be adhered to by the Implementing Agencies and respective executing agencies and consultants. Although these decisions were aimed at making the most effective use of funding and to foster sustainable outcomes, they were at times difficult to apply in the midst of implementation. Implementation of some of these decisions were perceived to be ambiguous or irrelevant by the beneficiaries and executing agents and led to some minor delays. Additionally, a few external events added to delays in the project. For example, after "9/11" subprojects at the equipment procurement stage suffered some delay because of travel restrictions imposed on technicians from European suppliers who were needed to provide training and complete equipment installation and commissioning. Implementation was also affected by factors subject to government control. The major reason for delay in project implementation was that incentives offered to enterprises to convert to non-CFC technology were not sufficient to compensate for the gains to be had in the market. Regulatory pressure in parallel with the grant funding by the MLF would have created a fairer playing field for the enterprises by establishing conditions conducive towards the introduction of cleaner technology and alternatives to the market. This factor was exemplified in the case of Dawlance and United Refrigeration. These two major projects in the refrigeration sector were cancelled in early 2001 at the request of the enterprises because they had not found it financially feasible to implement projects while CFC prices were low. Their withdrawal caused other enterprises with ODS subprojects to delay implementation because of fears of losing market share to these firms. Once the GOP import quota system came into effect and began to impact CFC prices, the two enterprises reapplied for technical and financial assistance (but received much reduced assistance from the MLF). The critical role of Government policy was apparent in the speed of implementation of the last group of subprojects to be approved by the MLF for Pakistan in the foam and refrigeration sectors in 2003 and 2004. By this time GOP had instituted several important regulations. These 8 subprojects were completed within three years of approval ­ a significant departure from the average duration of subprojects under the umbrella project of six years. Another source of delay which also created an avoidable financial burden to enterprises participating in the ODS Project, was the difficulty in obtaining duty waivers for implemented non-CFC equipment mainly due to lengthy procedure. Because the equipment was being funded to cover incremental costs of converting to non-CFC technology under the Protocol, the MLF requested Governments to exempt equipment of duties. The responsible authority for the issuance of waivers of the customs duty and sales tax was the Central Board of Revenue (CBR). However, due to a high turnover of concerned officials in CBR, the implementation of the policy was never widely adopted and a waiver had to be sought on a case-by-case basis during the life of the project. This resulted in the delay of clearance of machinery imported under the project and the incurrence of demurrage, which at sometimes was higher than the customs duty. Finally, factors subject to the implementing agency's control affected project implementation. NDFC was initially chosen as a Financial Intermediary by the Government of Pakistan. During implementation, NDFC was merged with NBP and project implementation came to a halt. It took some time to streamline procedures and regain momentum. Once NBP assumed the role of Financial Intermediary, it was able to partly overcome this hurdle by contracting the original MP project staff to continue MP project implementation. However, due to changes in upper management, and because of revision of policies post-merger, disbursement approvals continued to take more time than necessary. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization The monitoring and evaluation framework focused on key measurable indicators planned at project inception. These included ODP phased out versus that planned to be phased out; the cost- effectiveness of a specific intervention (the cost divided by the amount of phaseout); and destruction of CFC-based equipment (indicating sustainable phaseout). In addition, the disbursement rate and time elapsed from appraising subprojects to signing of subgrant agreements were indicators of the progress at the overall umbrella project level. Data was collected through several monitoring mechanisms. NBP tracked implementation based on main progress and financial indicators and reported to the Bank on a semi-annual basis. The data was, in turn, reported through the Bank's progress and financial database of subprojects to the MLF Executive Committee on an annual basis for monitoring. Upon completion, a PCR evaluating main results was required and submitted to the MLF for each subproject. In practice, the monitoring and evaluation framework was easily implemented and provided complete information at any given time. This is due primarily to having easily measurable indicators and strong external demand (the MLF) for detailed and frequent reporting on project progress. In fact, a separate mechanism was set up by the MLF to monitor subprojects with implementation delays (determined by the time needed to reach key project milestones). Several subprojects with some delays benefited from the increased surveillance ­ because of the pressure of possible cancellation under the mechanism, beneficiaries were compelled to ensure implementation did not stall completely. The main sustainability indicator, that baseline equipment used to manufacture with CFC was destroyed, was captured through a certificate of destruction passed on to the Ministry of Environment. With project completion, full monitoring responsibility of project beneficiaries will fall on MoE and subproject files will be provided by NBP. (See Annex 10, Table 10.1 for 9 performance of individual subprojects for some of the main project indicators and individual subproject completion reports for detailed evaluation of subproject results). 2.4 Safeguard and Fiduciary Compliance The project was rated as EA Category B (partial assessment) because with the main objective being environmental protection, there was no possibility of major environmental impacts. During preparation, subprojects were evaluated by international experts to ensure the design was in line with accepted environmental and safety norms and that the replacement chemical was one of the approved replacement substances under the MLF. Project beneficiaries were also required to comply with all country environmental and health regulations and standards related to their operations and to receive clearances from local authorities before new operations commenced, as required. In cases of safety issues surrounding some of the replacement substances (i.e., hydrocarbons), safety, training and audit measures were included in the subproject design. The utilization of NBP as a financial intermediary served to ensure fiduciary compliance. It was responsible for ensuring that enterprises and suppliers complied with procurement and financial management/disbursement policies of the Bank. During implementation NBP staff received guidance on Bank policies as well as MLF eligibility criteria to safeguard the MLF grant. This included support from Bank procurement and financial management specialists and training through Bank-organized, annual FI workshops. Procurement and financial management reviews were conducted on a regular basis in accordance with Bank guidelines. 2.5 Post-completion Operation/Next Phase Transition to normal, non-CFC based operations was the end objective of each subproject. After the satisfactory completion of the conversion, the enterprise resumed operations and reported on its project in terms of cost, phaseout, technology choice and sustainability. Some enterprises transitioning to alternatives with higher costs at the time of subproject approval were paid incremental operating costs to sustain operations for a limited amount of time. However, it is assumed that with the new technology and GOP ODS policy, regular operations will continue. The Government of Pakistan is responsible for compliance with MP obligations. As such, enterprises in sectors where CFC was once utilized are under the direct supervision of GOP. GOP is aware of its obligations to the Protocol and its role in ensuring that CFC phaseout is sustainable. A post-impact study might provide additional insight in a few years on how enterprises have sustained their operations and utilized the new equipment. Pakistan is well ahead of the CFC phaseout schedule but periodic monitoring and control of the end-use of CFCs is as important as monitoring the supply to ensure that the country remains in compliance. Pakistan has signaled its strong commitment to continue phasing out the use of ODS, as required by the MP. The GoP has implemented projects in other sectors besides refrigeration and foam. In particular, with UNIDO's assistance, Pakistan has completed 32 projects in the solvent sector out of a total of 34 identified projects. GOP has further imposed a ban on the import of CTC with immediate effect through SRO. (1)/2007 dated 28th May, 2007. Important actions have been also carried out regarding halons. There have been no imports of halons over the last two years and the Halons Banking and Recycling System has been established at Lahore. Finally, in pursuance of an agreement between GOP and the MLF, a notification regarding the imposition of a ban on halons is also in the pipeline. 10 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation (to current country and global priorities, and Bank assistance strategy) The project was consistent with the Government's priorities. Pakistan is committed to comply with its obligations as a party to the Montreal Protocol. GOP has aimed to ensure that meeting such obligations does not affect the competitiveness and economic performance of enterprises and facilities in key sectors. In this context, the objectives of the project responded to Pakistan's interests as it (i) supported the GOP's ODS phaseout program by providing financial and technical assistance to support firms' conversion to non-ODS technologies; (ii) implemented 17 cost-effective priority subprojects including an umbrella project in the foam sector and another in the refrigeration sector; and (iii) built local capacity to identify, develop, and implement ODS phaseout, both in the MoE's Ozone Cell, and in NBP. 3.2 Achievement of Global Environmental Objectives (including brief discussion of causal linkages between outputs and outcomes, with details on outputs in Annex 2) The main objective of the ODS Project was to facilitate the transition to non-CFC use in several sectors. Ten subprojects were expected to be fully committed within 48 months of the Grant Agreement's effectiveness with an expected total phaseout of 640 MT of CFC. The objective was fully achieved as evidenced by the complete cessation of CFC-manufacturing technology in the 17 subprojects that were eventually implemented under the ODS Project (more than the original 10 subprojects target), for a total of 1,243 MT of CFC phased out." Under the main project component, 21 subprojects were approved in the refrigeration and foam sectors only despite the fact that the ODS Project had not excluded possible activities in other sectors. Other Implementing Agencies assisted Pakistan in ODS phaseout in the remaining sectors. Four of the subprojects were umbrella subprojects ­ three in the foam sector and one in the refrigeration sector. Thermoware I and II, two group projects approved by the MLF to assist 26 enterprises, were treated as one subproject in implementation. Conversely, the implementation of the two remaining umbrella projects (one in foam and one in refrigeration) was done on an individual beneficiary level (with a total of 10 separate subgrant agreements). In the end, 81% of the subprojects approved were implemented due to four cancellations. Two of the cancellations were attributed to bankruptcy. Similarly, in the large Thermoware subprojects, several enterprises went out of business. In all these cases, CFC phaseout was achieved. The Dawlance and United Refrigeration projects were cancelled on the request of the enterprises because they did not find it financially feasible to implement projects while CFC prices were low. The firms were subsequently converted in the last refrigeration umbrella subproject by 2006. By the closure of the ODS Program, the total phase out target of 1,243 MT was met at a cost to the MLF of US$6.73/ kilogram. The cost effectiveness, when factoring counterpart funding by the beneficiaries was US$9.11/ kilogram. This includes consumption by Dawlance and United Refrigeration and the closed Thermoware enterprises. The project's main indicator of success is that enterprises in the two manufacturing sectors in Pakistan that were covered by the project (foam and refrigeration) have successfully transitioned to non-CFC technology and are no longer consuming CFCs as of the end of 2006. The gradual 11 reduction in consumption of CFC in the manufacturing sector, a total of 1,243 ODP MT, has allowed Pakistan to stay in compliance and meet its 2005 fifty percent CFC consumption reduction targets. The performance of individual subprojects in regards to the indicators outlined in Section 1.2 is partly captured in Annex 10, Table 10.1. Under the Financial Intermediary fee component, planned objectives were likewise achieved on a longer trajectory than had been first envisioned. NBP's fees were based on the percentage of disbursement made to the enterprises (3% of the amount). However, due to implementation and administrative delays, the cost of business significantly increased for NBP since its fees were capped at the amount of the disbursed grants but project duration more than doubled. Support of a national consultant contracted by the Bank became necessary to assist NBP in some of its requirements under the project, such as with the subproject PCRs. 3.3 Efficiency (Net Present Value/Economic Rate of Return, cost effectiveness, e.g., unit rate norms, least cost, and comparisons; and Financial Rate of Return) ODS conversion projects provide neither economic return nor financial return to the enterprises affected. Out of the allocated umbrella amount of US$12.61 million for subprojects, the MLF approved 21 projects for the Government of Pakistan at a value of US$9.17 million with an overall planned cost effectiveness of US $7.13/kilogram. The actual cost to the MLF was US$8.36 million for the phase out of 1,243 MT in 17 subprojects. The resulting cost effectiveness of US$6.73/kilogram is close to the lowest cost/kg thresholds set by the MLF for the foam and refrigeration sub-sectors (which range from US$6.23 to US$16.86) that were covered by the project. 3.4 Justification of Overall Outcome Rating (combining relevance, achievement of GEOs, and efficiency) Rating: Satisfactory The project's overall outcome rating is satisfactory, as a complete cessation of CFC- manufacturing technology was achieved in the subprojects covered by the ODS Project and a total of 1,243 MT of CFC were phased out. A "highly satisfactory" rating would have been achieved if the project had not incurred delays in implementation, which resulted in additional CFC emissions to the environment. 3.5 Overarching Themes, Other Outcomes and Impacts (if any, where not previously covered or to amplify discussion above) (a) Poverty Impacts, Gender Aspects, and Social Development Not applicable. (b) Institutional Change/Strengthening (particularly with reference to impacts on longer-term capacity and institutional development) The implementation of the ODS Project under the NBP and MoE has resulted in the augmentation of capacity in both institutions. Project implementation came to a halt after NDFC was merged with NBP, and it was only after NBP contracted the original MP Project staff that project implementation resumed. Once project implementation regained momentum, NBP gained 12 a better understanding of procedures surrounding the appraisal and implementation of environmental protection projects. Through the project, it has become a conduit for building awareness among the general public on environmental protection. NBP staff capacities were extended beyond financial appraisal to the ability to manage projects requiring a technical understanding paralleled with that of external MLF policies and guidelines on a myriad of project implementation details. Relatively high turnover in the Ozone Cell in the Ministry of Environment slowed down project implementation. However, overall cooperation with the Government was good throughout the project. Ozone Cell and MoE officers were receptive to NBP and the Bank's efforts in involving GOP in overseeing the progress of the project. This was particularly the case after 2002 during the preparation of the Country Program Update where its success was directly dependent on GOP's lead in the process. (c) Other Unintended Outcomes and Impacts (positive or negative, if any) A positive unintended outcome arising from the project was an increase in broad environmental awareness of stakeholders (NBP and the beneficiaries). In addition, beneficiaries are now in the position to market their products as "ozone-friendly," or "green" which is an added marketing advantage. Another unintended outcome was a slight change in the market due to external factors and the ability of some enterprises with new equipment and production methods to fill resulting market gaps. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops (optional for Core ICR, required for ILI, details in annexes) No survey or workshop was conducted for the Core ICR. 4. Assessment of Risk to Development Outcome Rating: Low or Negligible Project implementation was delayed by a number of factors, ranging from those outside of the control of the government (including market circumstances and changes in MLF policies) to those that GOP could have addressed earlier (such as the issuance of customs waivers and the imposition of comprehensive regulations on the use of CFCs). However, the GOP remained committed to fulfilling its obligations as a party to the MP and therefore maintained strong ownership of the project throughout its design and implementation. 5. Assessment of Bank and Borrower Performance (relating to design, implementation and outcome issues) 5.1 Bank (a) Bank Performance in Ensuring Quality at Entry (i.e., performance through lending phase) Rating: Moderately Satisfactory 13 The performance of the Bank in terms of delivering grant funding for subprojects is rated as moderately satisfactory which is applied to the identification, preparation and obtaining MLF approvals of respective subprojects. The identification of the subprojects was well conceived keeping in mind the capacity of enterprises, technology options and the industrial environment. The Bank assisted the GOP to prepare 21 subprojects (including four umbrella subprojects) comprising of 80 units small, medium and some large. The absence of a project implementation manual and insufficient corresponding training arranged for the NBP staff is a factor contributing to delay in project implementation. However, on average, the Bank performance in project preparation is rated as moderately satisfactory. (b) Quality of Supervision (including of fiduciary and safeguards policies) Rating: Satisfactory Quality of supervision is rated satisfactory as there was strong continuity in the supervision efforts (only three task managers over a ten-year project duration) and there was clear and substantial achievement of target outcomes. On average, more than three to four supervision missions per year were made in the initial years. (c) Justification of Rating for Overall Bank Performance Rating: Satisfactory The overall Bank performance is rated satisfactory, given a successful outcome, in terms of ODS abatement achieved, in terms of helping Pakistan meet its broader Protocol-mandated phaseout deadlines and in terms of the performance management framework that was introduced during the latter part of project implementation. 5.2 Borrower (a) Government Performance Rating: Satisfactory The Ozone Cell in MoE and NBP/NDFC, the FI, were staffed adequately for project implementation. Performance of both these actors was in general satisfactory. NBP kept 2-3 staff assigned full time to the project in the early years. The Ozone Cell provided adequate support from its core staff. Policy support for the project could ideally have consisted in the issuance of rules earlier in the project. Such a measure would have gotten the attention of the industry and made it easier for the financial intermediary (NBP) to accelerate the pace of subproject preparation and processing. The Ozone Cell working under the MOE was helpful during the implementation of the projects and played an active role. There were some quick transfers and postings in the Ozone Cell which resulted in delays in formulating policies. However, this was managed efficiently by the newest staff member who had some prior experience in the Ozone Cell. The non- or delayed issuance of certificates of duty waiver by the Central Board of Revenue (CBR) was also one of the causes of delay in the implementation of the project. 14 (b) Implementing Agency or Agencies Performance Rating: Satisfactory Enterprises visited by the World Bank mission were generally satisfied with the Financial Intermediary. Hence, it is clear that NBP performed creditably and maintained substantial compliance with its reporting obligations under the Grant Agreement. Financial management, disbursement and procurement arrangements were found satisfactory, although there were some shortcomings in the initial period of project implementation. A detailed review of financial management arrangements/risk assessment was not undertaken at the time of appraisal; however, detailed reviews were carried out during implementation review and agreement was reached on a set of actions that were implemented to improve financial management. Shortcomings were overcome through guidance from Bank staff and commitment on the part of NBP financial management staff. Staffing, internal controls, fund flow and financial reporting were largely adequate. Acceptable annual audit reports and quarterly financial management reports of the project were mostly submitted to the Bank on time and there were no audit issues. Continuous follow up by NBP/Bank resulted in most of the subprojects maintaining separate books of account, including fixed assets registers, and having these audited. Financial ratios and cost effectiveness of the subprojects was also worked out after follow up by the Bank. Subprojects contributed their share in the project cost and also created charge (for assets financed) in favor of NBP/NDFC. The project was not subjected to review by the Bank's Quality Assurance Group (QAG). During the initial stage, the project withdrew a small amount of US$ 100,000, as an initial advance into the Special Account. Therefore, the NBP continued to follow the Special Commitment procedure for Letters of Credits, even for small amounts, for import of goods. Regular coaching on withdrawal of funds procedures was made available to NBP staff throughout the life of the project. Based on the Bank's advice, the project increased the initial advance up to the allowed authorization of US$500,000, to avoid submission of applications for issuance of Special Commitment for small amounts. As required, the NBP did not submit Withdrawal Applications on regular monthly intervals. (c) Justification of Rating for Overall Borrower Performance Rating: Satisfactory The borrower's overall performance is rated satisfactory as both the Ministry of Environment and the NBP responded effectively to the challenges that arose during project implementation. Both MoE and NBP faced various difficulties, including significant organizational changes and the need to undergo a learning process to strengthen their capacity to oversee and facilitate the implementation of technically complex subprojects. Although these factors were associated with the lengthening of project duration, both organizations fulfilled their functions and contributed to the achievement of the project's objective. 6. Lessons Learned (both project-specific and of wide general application) During the implementation of the umbrella projects (where there were more than one enterprises involved) it would have been more appropriate that the selection of machinery be based on each individual enterprises' technical and working capability so that the equipment supplied met their 15 working environment. This was not the case in the Thermoware project where one type of equipment was procured for a group of enterprises with different levels of capacity and resources to maintain and use the equipment properly. The key in umbrella projects is to strike a balance between individual enterprise needs and the overall resources available. Finding common denominators does lead to more cost-effective approaches but they should not overshadow any differences between the beneficiaries. The Montreal Protocol provides for a CFC phaseout schedule for Article 5 countries which is supported by MLF assistance to these countries. In most of the subprojects, the enterprises also contributed counterpart funding because the grant was at times insufficient (due to cost- effectiveness thresholds set by the MLF) and could not be used for specific equipment that enterprises selected (because of MLF eligibility rules). In these cases, it was found that enterprises were extremely pro-active in implementation and in making choices on technology options. There were cases, however, where no contribution was made by the enterprise (the grant covered the full incremental costs). It was found that these enterprises had less of a stake in the equipment and consequently were not as careful with using and maintaining the equipment. In projects that involve funding the private sector to transform a market, it is advantageous to have some counterpart contribution to ensure ownership. In cases where subproject implementation was deliberately stalled because of a lack of enterprise commitment, it would have been helpful to the FI to have a "stick" to encourage enterprises to move forward in implementation. The Executive Committee did have a cancellation procedure for projects identified as having delays. Many of the Pakistan subprojects entered the list and remained in the list until completion. However, they escaped cancellation due to steady, albeit slow progress. The early ODS conversion subprojects generally were implemented apart from Government policies and rules to restrict the supply and demand of ODS. This was partly a result of the initial MLF approach to assist countries develop ODS strategies (Country Programs) and to then approve investment activities as stand-alone subprojects in specific sectors without linking them to national or sector ODS consumption levels. Institutional strengthening and capacity building activities were also treated separately. Performance-based, sector or national approaches to CFC phaseout which combine policy and investment measures in one project and provide for a project management unit within the Government have now become the norm in the MLF. The lessons drawn from the Pakistan ODS Phase-out Project support the findings that sustainable phaseout hinges upon Government policy action early on and a project design that promotes Government involvement in implementation through capacity building and other support measures. Although an assessment of economic costs and benefits to the country to convert to non-ODS technology and to implement the MP is beyond the scope of this report, some broad conclusions can be made based on experiences from implementation of the ODS Project. The MLF covered the majority of the conversion costs; however, costs were incurred by the beneficiaries through duties and demurrage on equipment and short-term losses by enterprises that did convert and had to compete with the enterprises that continued using CFC. Some of the benefits were technological upgrades which allowed enterprises to become more competitive and to expand their markets. They could also advertise that their products were ecologically sound. As discussed above, project implementation took longer than originally planned for various reasons. Consequently, supervision efforts were conducted over a timeframe that was twice as long as had been originally envisioned, resulting in above average supervision costs. Delays in project implementation could have been reduced if the GOP had implemented early on some of 16 the policies that it eventually implemented. However, these measures would have not sufficed to address other factors that affected project implementation, such as the events of 9/11. Additionally, the project's supervision costs could have been reduced if the FI's staff had been exhaustively trained in all aspects of project implementation. Still, efforts were made to reduce supervision costs, as supervision missions were reduced from five to a maximum of three, and after the national consultant had been hired, to a maximum of two. Greater interaction of Bank staff and providing guidance to the implementing agency/follow up helped a great deal in improving financial management of the project. Staff continuity also paid off. Finally, one important lesson learned was that a capacity needs assessment would have been helpful to determine what type of training NBP needed to ensure it was able to deliver on its obligations in the earlier stages of the project. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies Minimal changes and additions were provided by the Ministry of Environment of the Government of Pakistan and have been incorporated in ICR (See copy of letter in Annex 7). In addition, the GOP provided a contribution to the ICR which is also found in Annex 7 as a short report. NBP's comments are captured in Annex 8. (b) Cofinanciers Not applicable. (c) Other partners and stakeholders Not applicable. 17 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Appraisal Estimate Actual/Latest Components Percentage of (USD millions) Estimate (USD millions) Appraisal INVESTMENT COMPONENT 12.61 8.34 66.14 TECHNICAL ASSISTANCE COMPONENT (Agency Fee) 0.39 0.25 64.10 ODS Phase-out / Charges 0.0 0.03 103.00 Total Baseline Cost 13.00 8.62 66.31 Physical Contingencies 0.00 0.00 Price Contingencies 0.00 0.00 Total Project Costs 13.00 8.62 66.31 Project Preparation Facility (PPF) 0.00 0.00 Front-end fee IBRD 0.00 0.00 Total Financing Required 13.00 8.62 66.31 (b) Financing Appraisal Actual/Latest Source of Funds Type of Estimate Estimate Percentage of Cofinancing (USD (USD Appraisal millions) millions) Borrower Commercial Credit 2.00 2.96 148.00 Montreal Protocol Investment Fund 13.00 8.62 66.31 18 Annex 2. Outputs by Component The project covered two manufacturing sectors with a total final phaseout of 1,243 ODP MT at a cost of US$8.34 million under the first component of the Project. All subprojects complied with project completion requirements set forth by the Bank and the MLF ­ phaseout of CFCs was achieved; baseline equipment was destroyed (where applicable) and non-CFC manufacturing is underway. With the completion of these projects, the Government of Pakistan has closed CFC- based manufacturing in the country in these two sectors. MLF Funds Planned Cost- Cost- Approved Disbursed ODP Phaseout ODP Phased effectiveness effectiveness Sector Funds (US$) (US$) Approved Out (US$/Kilogram) (US$/Kilogram) Foam 5,933,784 5,470,732 866 866 6.85 6.32 Refrigeration 2,967,851 2,893,252 377 377 7.87 7.67 Total 8,901,635 8,363,984 1,243 1,243 7.16 6.73 The second component consisting of technical assistance and the FI fee for NBP/NDFC had a US$390,000 allocation. Actual disbursement was US$255,594. 19 Annex 3. Economic and Financial Analysis (including assumptions in the analysis) Not applicable. 20 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Names Title Unit Responsibility/ Specialty Lending Jitendra J. Shah Country Sector Coordinator EASRE Task Team Leader (Lending) Supervision/ICR Paul Jonathan Martin Sr. Environmental Specialist AFTEN Task Team Leader (Supervision) Mosuf Ali Consultant SASDN Technical Anwar Ali Bhatti Financial Analyst SACPK Financial Mgmt Mary-Ellen Foley Operations Officer ENVMP Technical (MP) Afzal Mahmood Program Assistant SASDO Task Team Support Hasan Masood Mirza Consultant SACPK Procurement Ernesto Sanchez-Triana Senior Environmental Engineer SASDN Task Team Leader Hasan Saqib Sr Financial Management Specialist SARFM Financial Mgmt Bert Veenendaal International Consultant SARES Process Expert Carla P. Vale de Holguin Research Analyst SASDN Support to ICR Review Cecilia Belita Senior Program Assistant SASSD Task Team Support (ICR) Jack H. Williams E T Temporary WBIVP Task Team Support (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle No. of staff weeks USD Thousands (including travel and consultant costs) Lending FY94 30.25 FY95 99.28 FY96 92.17 FY97 49.45 FY98 0.00 FY99 7.54 FY00 0.04 FY01 0.00 FY02 0.00 FY03 0.00 FY04 0.00 FY05 0.00 21 FY06 0.00 FY07 0.00 Total: 271.15 Supervision/ICR FY94 0.00 FY95 FY96 0.00 FY97 55.91 FY98 78.56 FY99 142.49 FY00 46.94 147.73 FY01 14.22 101.98 FY02 9.84 74.71 FY03 8.81 91.79 FY04 15.88 137.77 FY05 27.49 174.97 FY06 113.18 110.64 FY07 17.79 103.52 Total: 154.15 1220.11 Note: Information on number of staffweeks for FY04-FY99 not available 22 Annex 5. Beneficiary Survey Results Not applicable. 23 Annex 6. Stakeholder Workshop Report and Results Not applicable. 24 Annex 7. Summary of Borrower's ICR and Comments on Draft ICR GOVERNMENT OF PAKISTAN MINISTRY OF ENVIRONMENT OZONE CELL ------ PAKISTAN OZONE DEPLETING SUBSTANCES PHASE OUT PROJECT (FOAM AND REFRIGERATION SECTOR) Implementing Agency World Bank IMPLEMENTATION COMPLETION REPORT 29th June, 2007 25 PAKISTAN OZONE DEPLETING SUBSTANCES PHASE OUT PROJECT (FOAM AND REFRIGERATION SECTOR) COMPLETION REPORT (i) Assessment of the operation's objective, design, implementation, and operational experience OBJECTIVE The objective of the project was to assist Pakistan in the transition from CFC-based technology into non-CFC technology by: (i) Supporting the Government of Pakistan's proposed program to phase out Ozone Depleting Substances (ODS); (ii) Implementing cost-effective subprojects, identified in the Country Program (CP) for technical conversion; and (iii) Building local capacity to identify, develop and implement ODS phase out subprojects. The country strategy considered the possibility of taking advantage of the Multilateral Fund (MLF) funds and assistance available for Article 5 countries of the Protocol. With the MLF's resources and support of the United Nations Environmental Program (UNEP), the Government of Pakistan developed the Country Programme (CP), which defined actions to phase-out ODS. Also, with the support from United Nations Development Program (UNDP), the Government of Pakistan created the Ozone Cell, Ministry of Environment in 1996 to facilitate implementation of the CP. The project with the World Bank also helped to receive MLF's resources for assistance in technical and financial terms to support enterprises that wished to convert to non-ODS technologies. The objective of the project followed the actions identified by the CP to achieve compliance with the Montreal Protocol (MP). DESIGN The design of the project was based on the Country Program and was adequate. The CP contained a first estimation of ODS consumption in Pakistan, that was developed using end-user data for the refrigeration and foam sectors and import data for the Halons and solvents sector. The CP established that refrigeration and foam were priority for conversion for Pakistan. The ODS phase-out strategy in the CP gave responsibility to the Government for increasing public awareness about use of ODS and to the enterprises for converting to the non-ODS technology of their choice. The project had two components - US$ 12.61 million for subprojects for enterprise conversion to non-ODS technologies and US$ 0.39 million for payment of the fees of the National Bank of Pakistan (NBP), which acted as Financial Intermediary (FI), for a total of US$ 13 million. The design of the project also included umbrella sub-projects. The 26 inclusion of umbrella projects strengthened the project's design because it set the basis for cooperation and definition of targets in the reduction of ODS, while allowing flexibility to provide financial support to the enterprises. Identification and preparation of project document was carried out well in accordance with the guidelines of Montreal Protocol and to the utmost satisfaction of the sub-project beneficiaries within ambit of rules. Phase out of project was supported by the MLF assistance, but in some cases the grant funding was not sufficient and the project beneficiaries had to contribute from their own resources for the change in technology. The implementation of the project took more than double time that was originally envisioned. However, it was not because of the design but due to evidence arose during project implementation that the estimates of ODS consumption contained in the CP were inaccurate, as some important enterprises and sub-sectors had not been taken into consideration. Keeping in view the need to include more enterprises and sub-sectors in the country's strategy to comply with MP's obligations, the Government of Pakistan prepared a Country Program Update (CPU) in 2003. The CPU developed a strategy to phase-out controlled substances by 2010, as required by the MP. Based on the CPU, the project's targets were revised to achieve compliance with the reduction of ODS as per schedule envisaged in the MP. IMPLEMENTATION The novelty of non-CFC technologies reduced the interest for conversion of enterprises in some sectors. For example, the availability of more than one technology in the refrigeration sector delayed conversion due to the fact that the enterprises preferred to see the experience of other firms with new technologies, and also to observe the trend of the market for adopting any specific technology. Although the operational cost of the cyclopentane was low, yet enterprises were reluctant to switch over to cyclopentane due to safety concerns and high capital cost. Since in the beginning the prices of CFCs were lower as compared to non-CFCs, the refrigeration industry was hesitant to switch over to non-CFCs. Another reason regarding slow implementation was frequent changes in MLF rules and guidelines for the use of grants. The MLF Executive Committee issued new decisions and guidelines on numerous occasions during project implementation. Several sub-projects were already being implemented and it was at times difficult for these sub-projects to comply with new decisions and guidelines without a delay in their implementation. Besides above, the progress of the projects was hampered on a couple of occasions such as the event of 9/11 in 2001 and merger of Financial Intermediary National Development Finance Corporation (NDFC) with National Bank of Pakistan (NBP) in 2002. 27 OPERATIONAL EXPERIENCE The implementation of the project accelerated as the Government of Pakistan introduced regulatory measures to provide more incentives for conversion to non-ODS technologies. These regulatory measures reflected Pakistan's commitment to comply with the Montreal Protocol. The regulations included the:- Introduction of an import authorization/licensing system in July 1998 to regulate and monitor the imports of ODS in the country. Adoption of the Import Policy Order in 1999 (SRO 895(1)99 Import policy, dated 3rd August, 1999). Putting in place in 2000 the regulation ECC-196/16/2000, dated 27th November, 2000 that modified the custom duty tax in favor of CFC-free components and introduced a CFC quota reduction schedule. Prohibition established through SRO 489(I)2000 to import used compressors, air- conditioners, refrigerators and other second-hand household machines Imposition of ban on import of CFC-based refrigerators and deep freezers in fiscal year 2002 ­ 2003. Ozone Cell, Ministry of Environment in collaboration with the Directorate General of Training & Research (Customs) and the United Nations Industrial Development Organization (UNIDO) also arranged training of 200 Customs officials to enhance their capacity to curb illegal import of ODS. Besides, training of 3000 technicians in the servicing sector under Refrigeration Management Plan (RMP) is also in progress through UNIDO. (ii) Assessment of the outcome of the operation against the agreed objectives; Major target of the implementation of the project was achieved by assisting Pakistan to convert the CFC-based technology into non-ODS technology. Financial and technical assistance provided by MLF accelerated the implementation of the CPU. World Bank implemented 17 cost-effective technical conversion sub-projects, and helped strengthen the capacity of the Ozone Cell and NBP. As per CP, the project was originally expected to help phase-out 640 MT of CFC in four and a half years. However finally, 1,243 MT of CFCs (almost double the initial target) were phased-out over a period of ten years. Out of 21 identified sub-projects, 17 sub-projects were implemented successfully and CFC was phased out. Remaining 4 sub-projects were cancelled, as either enterprises became bankrupt or went out of business. As a result of the implementation of these sub-projects, the refrigeration and foam industries in Pakistan have completely switched over to ozone friendly technology. 28 Pakistan is in strong position to stay in compliance with its 2007 reduction obligations after the technological conversion of these industries coupled with introduction of regulatory measures taken by the Government to reduce the use of ODS. (iii) Evaluation of the borrower's own performance during the preparation and implementation of the operation, with special emphasis on lessons learned that may be helpful in the future; The Ozone Cell, Ministry of Environment was created to facilitate and coordinate with the implementing agencies for the implementation of the CP and CPU to fulfill obligations under the MP. The capacity of the Ozone Cell was enhanced since its creation to identify and manage technological conversion projects. The Cell had a good coordination with the World Bank and NBP and provided adequate support to the NBP. The NBP deputed adequate staff to work full time in the project, particularly in the initial years. This helped the Financial Intermediary to provide services of high quality to the enterprises that participated in the phase out program. It may be noted that NBP consistently complied with its monitoring obligations and conducted disbursement and procurement arrangements as required by the World Bank. NBP and the Ozone Cell played a vital role through their dedicated and devoted efforts in the phase out plan under the MP. (iv) Evaluation of the performance of the Bank, any co-financiers, or of other partners during the preparation and implementation of the operation, including the effectiveness of their relationships, with special emphasis on lessons learned The World Bank helped the Government of Pakistan to identify and prepare a total of 21 subprojects (including 4 umbrella projects) that comprised 80 small, medium and large enterprises. It also helped in delivering resources and grants, and obtaining MLF approval for these subprojects. The design of the subprojects was appropriate considering the characteristics of the enterprises, the available technologies, and the industries' context. The Bank carried out supervision missions regularly to ascertain timely implementation of the projects. This helped to meet the project's ODS abatement goal and support Government of Pakistan to manage project preparation and implementation. (v) Description of the proposed arrangements for future operation of the project The Government of Pakistan is fully committed to comply with its obligations under the MP. Pakistan is ahead of its CFC phase-out schedule and the Government is making all out efforts to remain in compliance. It includes continuing enforcement of regulatory measures to control the end-use and supply of CFCs. Ozone Cell will periodically visit the enterprises that have converted to non-CFC technology to ensure that they are operating with the new ozone friendly technology. 29 SUGGESTIONS: 1. Before closing the projects formally, final visits to the sub-projects may be carried out jointly by the Ozone Cell, Ministry of Environment, National Bank of Pakistan and World Bank to ensure that all the sub-projects are operating with new technology/equipment. 2. It would be appropriate if one of the team members from National Bank of Pakistan is allowed to continue coordination with the Ozone Cell for a period of at least six months for assistance, if any, required in connection with the completed projects. 30 31 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Comments of the Financial Intermediary, the National Bank of Pakistan Received during the ICR Mission Quality Of Technical Appraisal: NBP reported that some enterprises remarked that the technical appraisals performed by the World Bank consultants did not fully provide for actual incremental costs with the result that extra costs had to be borne by the enterprises. Project Implementation Manual: NBP was of the view that project implementation could have been speedier had an operational manual been prepared for the guidance of the financial intermediary for project implementation and the appraisal of the project. Incentives and penalties: NBP felt it did not have sufficient instruments at its disposal to enforce timely implementation by subproject beneficiaries who could not be penalized for delay. NBP would have liked to have the option of reducing, say, the reimbursement entitlement in respect of incremental operating cost or alternatively enforcing a penalty for delay, paid for from enterprise entitlements. In the absence of such wherewithal, NBP is of the view that much of the overall delay in project implementation remained outside of its control. It is felt that this contention would be moot had there been adequate regulatory compulsion. Technical Training for Staff: NBP suggested that there should have been provision for basic technical training of its staff, relative to the technological issues relevant to process conversion in various sub-sectors. Initially NBP faced problems in understanding the technical issues/terms related to conversion. However, after hiring of a National Consultant this problem was resolved. This also helped the World Bank in identifying new eligible projects for conversion. Supervision Frequency: NBP made the comment that frequent Bank supervision missions were critical in supporting implementation. Two per year was considered optimal. Canceled Projects and Compensation: NBP is of the view that all the work they did for a project should have been compensated, even if the project was eventually cancelled. This did not happen in the two cases where the projects were cancelled and NBP received no compensation, although it had undertaken a financial appraisal and other activities. In this regard, NBP expressed its dissatisfaction with the disbursement-fee arrangement. In their opinion, it would have been preferable to anticipate such situations by adding a clause for compensation in such special cases. From the perspective of the World Bank and the MLF, however, it is clear that a disbursement-based fee arrangement is to be preferred, both from the standpoint of simplicity, and because it ensures that fees are tied to performance. It entails some risk for NBP, but it may be noted that they have again accepted a disbursement-tied fee arrangement. Incremental Operating Cost (IOC) Disbursement: Finally, it was observed that IOC disbursements were problematic until the Bank decision to link IOC disbursement to actual expenditures for raw material purchases. 32 Annex 9. List of Supporting Documents 1. The World Bank. Report No. 14990-PAK. Pakistan Montreal Protocol ODS Phase Out Project, in the form of a Memorandum and Recommendation, Country Department I Director to the Regional Vice President (South Asia Region). January 15, 1997. 2. OTF Grant Number TF-028200-PAK. Ozone Projects Trust Fund Grant Agreement between Islamic Republic of Pakistan and International Bank for Reconstruction and Development, Acting as Trustee of the Ozone Projects Trust Fund. February 7, 1997. 3. Amendment 4. Montreal Protocol Ozone Depleting Substances Phase-Out Project: Mid-Term Review. Aide Memoire ­ September 13-17, 2004. 5. The World Bank and the Ozone Cell, Ministry of Environment, Government of Pakistan. The Islamic Republic of Pakistan: Country Programme Update. December 2003. 6. Other documents kept in Project Files and in Iris, including subproject completion reports. 33 Annex 10. Additional Annexes Table 10.1 List of Subprojects and Associated Performance Counter-part Funding Funding Cost Effective- Total Cost Enterprise Name CFC to CFC Phased Date of MLF Date of Funding Approved Disbursed ness (US$ per Effective-ness Phaseout (MT) Out (MT) Approval Completion Disbursed (US$) (US$) kg) (US$ per kg) (US$) 1Refrigerators Manufacturing Company Pakistan Ltd. - - Nov-98 Cancelled Returned - - - - 2United Refrigeration Industries Ltd. - - Jul-98 Cancelled Returned - - - - 3Dawlance P. Ltd. - - Jul-98 Cancelled Returned - - - - 4Domestic Appliances Ltd. (DAL) - - May-96 Cancelled Returned - - - - Pakistan Air-conditioning Engineering Co. P. Ltd., 5(PAECO) 20 20 Dec-00 Aug-05 176,681 176,681 70,619 8.97 12.55 6Mumtaz Engineers 14 14 Dec-00 Nov-05 204,736 182,866 18,800 13.16 14.51 7Diamond Group of Industries 64 64 Nov-97 Feb-05 563,339 558,939 146,113 8.72 11.00 8Master Group 205 205 Jul-95 Feb-05 1,211,000 1,204,000 465,000 5.87 8.14 9Jaguar Industries 40 40 Nov-99 Jun-05 279,280 273,667 55,000 6.84 8.22 10Singer Pakistan Ltd. 18 18 Nov-97 Jul-04 205,893 205,893 238,225 11.57 24.95 11Synthetic Products Enterprises (Pvt) Ltd. (SPEL) 14 14 Nov-97 Jan-02 160,625 136,829 - 10.06 10.06 12Kold Kraft Ltd. 12 12 Nov-97 Dec-02 175,000 171,435 8,500 14.91 15.65 13United Foam Industries 29 29 Nov-98 Dec-01 178,200 178,200 32,500 6.23 7.37 14Saleem Automotive Industries Ltd. 3 3 Nov-98 Mar-99 33,875 31,603 - 12.64 12.64 15Shadman Electronic Industries P. Ltd. 16 16 Jul-98 Nov-02 236,936 236,936 24,450 15.26 16.83 16Razi Sons 60 60 Nov-95 Nov-02 493,262 493,274 59,000 8.22 9.20 17Umbrella project: Thermoware I 240 240 Nov-97 Oct-06 1,600,000 1,390,525 52,000 5.80 6.02 18Terminal umbrella: Thermoware II 106 106 Jul-98 Oct-06 718,900 557,052 - 5.27 5.27 19Cool Industries Ltd. (Waves) 118 118 Nov-97 Dec-06 841,750 841,750 506,000 7.13 11.42 Foam Umbrella: Pakistan Insulation, Simpson Wire, HEPCO, Indus Plastic, Workman, Thermocraft 20Engineering 107 107 Dec-03 Dec-06 658,973 646,643 172,923 6.04 7.66 Refrigeration Umbrella: Dawlance, United 21Refrigeration, Ice Age, 29 small enterprises 181 181 Apr-04 Dec-06 1,126,855 1,077,691 1,110,512 5.95 12.09 TOTAL 1,243 1,243 8,865,305 8,363,984 2,959,642 6.73 9.11 34 Table 10.2 Subproject Approvals and Completion by Year Subprojects 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 Total Approved 2 1 6 7 1 2 -- -- 1 1 -- -- 21 Physical -- -- -- -- 1 -- 1 4 -- 1 5 5 17 and Financial completion Table 10. 3 Subproject size in US$* No. Size Range Count Percentage 1 Less than $100K 1 5.90 2 $100K - $199K 4 23.52 3 $200K - $499K 5 29.41 4 $500K - $1,000K 4 23.52 5 More than $1,000K 3 17.65 TOTAL 17 100.00 *Excludes 4 cancelled subprojects. 35 36