71574 v2 GEF COUNTRY PORTFOLIO STUDY: JAMAICA March 2011 REVIEW OF OUTCOMES TO IMPACTS JAMAICA DEMAND SIDE MANAGEMENT PROJECT (Volume 2) Unedited Jamaica Demand Side Management ROtI Team David Todd: (Team Leader) David Lee: (Environment and Energy Specialist) GEF Evaluation Office Director: Robert D. van den Berg Task Manager: Carlo Carugi Acknowledgements The Jamaica Demand Side Management Project Review of Outcomes to Impacts was conducted in November and December 2010 by David Todd and David Lee. The team is particularly grateful to the ROtI Discussion Group of December 2nd 2010, which consisted of Denise Tulloch of Petroleum Corporation of Jamaica, Hugh Harris of the Ministry of Finance, Government of Jamaica and Raymond Wright, Consultant to the Petroleum Corporation of Jamaica. The ROtI team is responsible for the contents of this report, which do not represent the official position of the GEF Evaluation Office. 3 Volume Two Contents Acronyms 4 1: Introduction to the ROtI Approach ................................................................................................5 2: The Jamaica Demand Side Management Project ...............................................................................................6 2.1: Project Objectives .................................................................................................................................................6 2.2: Project Financing ................................................................................................................................................. 6 2.3: Project Implementation and Components........................................................................................................ 6 2.4: Changes Made During Project Implementation ............................................................................................... 8 2.5: Project Performance at Completion .................................................................................................................. 8 3: The Project's Global Environment Benefits ........................................................................................................9 4: The Jamaica DSMP Outcomes-Impacts Theory of Change ..............................................................................9 5: Assessment of Achievements of the Outcomes to Impacts Pathway ........................................................... 13 5.1: Strategy 1: Reduce Fossil Fuel Use ...................................................................................................................14 5.2: Strategy 2: Strengthen Institutional Capacity for Reduced Fossil Fuel Use ...................................................24 5.3: Strategy 3: Strengthen Public Support for Demand Side Management Measures ........................................31 6: Summary of Project ROtI Ratings .....................................................................................................................41 7: Overall Assessment of the Project's Progress Towards the Intended GEB ....................................................42 Annex 1: References ..................................................................................................................................................43 Annex 2: Participants in the ROtI Process ............................................................................................................. 43 Figures Figure 1. Generic Theory of Change for Outcomes to Impacts Pathways in the Climate Change Sector................. 9 Figure 2. Theory of Change for JDSM Project ........................................................................................................ 11 Figure 3. Theory of Change for Strategy 1 ............................................................................................................... 16 Figure 4. Theory of Change for Strategy 2 ............................................................................................................................ 25 Figure 5. Theory of Change for Strategy 3 ........................................................................................................................... 32 Tables Table 1. Definitions of Theory of Change Elements in the Outcomes to Impacts Pathways......................10 Table 2. Field ROtI Rating System .............................................................................................................13 Table 3. Outcomes to Impacts Assessment Findings for Strategy 1 ...........................................................23 Table 4. Outcomes to Impacts Assessment Findings for Strategy 2 ...........................................................30 Table 5. Outcomes to Impacts Assessment Findings for Strategy 3 ..........................................................40 Table 6. Overall Assessment of Project Progress Towards Global Environment Benefits .......................41 Box 1: ROtI Rating Guideline Interpretations.............................................................................................13 4 Acronyms A (External) Assumption CDs Compact Discs CFL Compact Fluorescent Lightbulbs EE Energy Efficiency EECP Energy Efficiency and Conservation Programme CO2 Carbon Dioxide CSOs Civil Society Organisations GET Global Environment Trust DSMU Demand Side Management Unit GEB Global Environment Benefit GEF Global Environment Facility GEF EO Global Environment Facility Evaluation Office JPS Jamaica Public Service Company GHG Greenhouse Gas IBRD International Bank for Reconstruction ICR Implementation Completion Report ID Impact Driver IDA International Development Association IDB Inter American Development Bank IS Intermediate State JAMCO Jamaica Marketing Company JBS Jamaica Bureau of Standards JDSM Jamaica Demand Side Management Project NGOs Non Governmental Organisations NRCA Natural Resource Conservation Authority PV Photo Voltaic RF Rockefeller Foundation ROtI Review of Outcomes to Impacts SDR Special Drawing Rights SWH Solar Water Heater TOC Theory of Change 5 1. Introduction to the ROtI Approach Terminal project evaluations rarely provide information about impact due to the lack of data available to make such an assessment, and the complexity of environmental processes and the long timeframe needed to generate impact, which may only be realised many years after project completion. The ROtI methodology seeks to overcome these challenges by exploring the underlying logical sequence of conditions and factors (referred to as the theory of change) that will lead to impact, and by assessing the extent to which the theory of change has been realised in practice. The methodology provides a quick and cost effective way of indirectly measuring project impact, or the potential to deliver impact in the future. GEF projects are generally designed and structured according to the logical framework, which in essence is a simplified theory of change, with the following basic means-ends hierarchy: ACTIVITIES OUTPUTS OUTCOMES IMPACTS Strategy Means Ends Most GEF projects and their terminal evaluations mainly focus on the first three steps in the above logframe hierarchy; i.e. the project activities that achieve a set of outputs, which in turn will contribute to achieving outcomes. However, GEF Terminal Evaluations (TEs) do score the likelihood that project results will be sustainable, thereby implicitly assessing the likelihood of impact. But to more explicitly understand the process for delivering eventual impact, the ROtI methodology focuses on the last step in the means-end hierarchy, i.e. developing a detailed theory of change from outcomes to impacts. The ROtI Practitioner’s Handbook (GEF EO, 2009) provides a detailed explanation of the ROtI methodology; the main stages include: � Stage 1: Impact identification. The initial characterisation of the project’s intended long-term environmental impacts, which in the ROtI methodology are referred to as Global Environment Benefits (GEBs). Understanding what the project is ultimately trying to achieve is a vital first step in developing the theory of change for achieving impact. � Stage 2: Project logframe review. The verification of the project’s outcomes, and their appropriateness to achieving the desired impacts. (The Jamaica DSM Project was designed before a Logframe was required). � Stage 3: Outcomes to impacts analysis. The identification and assessment of the logical steps, conditions and associated factors (i.e. theory of change) necessary to overcome the barriers to realising impacts, which provide an indirect measure of impact. 6 2. The Jamaica Demand Side Management Project 2.1 Project Objectives The Global Environment Trust Fund Grant Agreement for the Demand Side Management Demonstration Project, which was dated June 1, 1994 (GET Grant Number TF028695 JA), stated its objectives as follows :  “to demonstrate the potential for electricity savings to reduce fossil fuel consumption in power generation  to develop the basis for expanding demand side management programmes throughout the Recipient’s territory  to build and strengthen the institutional capabilities of JPS and other governmental and private entities to enable them to implement full-scale demand-side management programmes in the power sector�. 2.2 Project Financing The project had a substantial financial package, including national and international co-funding to the GEF inputs. The total intended financing at project commencement was approximately US$12,500,000, made up as follows:  a loan in the amount equivalent to four million Dollars (US$4,000,000) (the IDB Loan) from the Inter-American Development Bank (the IDB) under the terms and conditions set forth in an agreement entered into between the Recipient and IDB on April 8, 1991 (the IDB Loan Agreement)  a grant in an amount equivalent to two hundred thirty-seven thousand Dollars (US$237,000) (the RF Grant) from the Rockefeller Foundation (the RF) under the terms and conditions set forth in an agreement entered into between the Recipient and the RF on May 14, 1990 (the RF Agreement)  a grant, in an amount equivalent to one hundred fifty thousand Dollars (US$150,000) (the Canadian Grant) from IDB, as administrator (the IDB Administrator) of funds provided by Canada (the Canadian Trust Grant Fund) also for the purpose of financing the Project  The Government of Jamaica, through the Jamaica Public Service Company (JPS) pledged US$4,310,000 to the project activities  The Trustee of the GET funds (i.e., the GEF) agreed to make available to the Recipient a GET Grant in an amount in various currencies equivalent to two million eight hundred thousand Special Drawing Rights (SDR 2,800,000), which was valued at US$3,800,000. 2.3 Project Implementation and Components The project was to be implemented by the Demand Side Management Unit (DSMU) of the Jamaica Public Service Company (JPS), the national supplier of electricity; which was then wholly owned by the Government of Jamaica. Other Government Agencies, notably the Natural Resource Conservation Authority (NRCA), and the Jamaica Bureau of Standards (JBS), as well as unspecified NGOs, were also expected to participate. 7 The project was complex and consisted of a number of components, for which modifications would require the prior approval of the grant recipient (Government of Jamaica) and the Trustee (IBRD). According to the Grant Agreement, the intended components were to be as follows: 2.3.1 Commercial Sector Demonstration Testing and demonstrating the viability of energy savings and efficiency in the commercial sector as well as their long term sustainability through:  carrying out of a comprehensive energy audit in about thirteen large commercial buildings  providing design assistance to the construction of seven large commercial buildings and adopting cost-effective measures to improve efficiency in the use of energy in such large commercial buildings, including installation of, or retrofitting with, energy-efficient appliances and equipment  carrying out of a comprehensive energy audit in about ten small commercial buildings; and adopting cost-effective measures to improve efficiency in the use of energy in such commercial buildings, including retrofitting with energy-efficient appliances and equipment, assessing potential savings and market conditions of commercial refrigeration (including air- conditioning)  based on the evaluation of the results of the activities referred to above, develop and implement a programme of energy-efficient measures in the commercial sector  a programme to promote private cogeneration of electric power, in particular in hotels, including: (a) feasibility studies; (b) technical assistance for identification and installation of the appropriate equipment, quality control, monitoring and evaluation; and (c) support for the sale of electric power. 2.3.2 Residential Demonstration Implementation of energy-efficient measures in residences including:  installation of compact fluorescent lamps in about one hundred homes at no cost to the residents and assessment of the results of such activity  assessment of the solar water heater market and monitoring performance of solar water heaters, including those installed in the Oakland Housing Development  assessment of potential savings and market conditions of residential refrigeration (including air- conditioning)  based on the results of these assessments, develop and implement a programme to support the provision of up to 100,000 compact fluorescent lamps to residential customers and schools, strengthening of the solar water heater market and the residential energy-efficient refrigeration market. 2.3.3 Industrial Sector Assessment of potential energy savings in the industrial sector and development and implementation of a strategy to foster such savings. 8 2.3.4 Monitoring and Evaluation  Development and implementation of a data base for the execution of the Project  Development and implementation of detailed monitoring and evaluation plans for the execution of the Project, including evaluation of the long-term sustainability of the demand side management measures tested on a self-financing basis. 2.3.5 Institutional Strengthening, Market Infrastructure and Public Awareness A range of activities was included to raise institutional and public awareness of the potential and importance of demand side management measures, including:  support to the institutional strengthening and development of JPS (in particular of DSMU), JBS, NRCA, NGOs and other entities involved in demand side management programmes in the power sector, in particular for the enhancement of market infrastructure and public awareness  carrying out of public awareness campaigns about energy efficiency and dissemination of information on demand side management programmes in the power sector, including, inter alia, promotion of building codes and labeling programmes for electric appliances. 2.4 Changes Made During Project Implementation The final cost of the project was reduced from the initial estimate of US$12,500,000 to US$9,850,000. In mid- 1994, one of Jamaica’s major power generating plants was severely damaged and had to have emergency repairs. The Government, through JPS, diverted funds from the project to help finance the emergency remedial work. It also gained permission to reallocate US$1,320,000 of its IDB loan to that purpose, reducing its DSM work by that amount. These reductions mainly affected the industrial and commercial elements of the project, reducing both support and results in these areas. Some elements of the project did not utilise their full budgets, notably some of the institutional development components and monitoring and evaluation. In the last year of the project, these funds were used for two additional renewable energy components, namely:  promotion of Solar Water Heaters (SWH) for commercial hotels and residential consumers, to reduce electricity demand  piloting of the use of solar photovoltaic systems (PV) for isolated rural communities deemed unlikely to receive grid electricity in the near future. 2.5 Project Performance at Completion The World Bank’s Implementation Completion Report for the Project was submitted June 1st 2000. The ratings were as follows:  Project Outcome – Satisfactory  Sustainability – Likely  Institutional Development Impact – Substantial  Bank Performance – Satisfactory  Borrower Performance – Satisfactory. 9 3. The Project’s Global Environment Benefits The starting point for the ROtI assessment is to identify the project’s intended environmental impacts, which for GEF projects are the delivery of Global Environment Benefits (GEBs), which are defined in the ROtI Handbook as ‘lasting improvements in the status of an aspect of the global environment that safeguard environmental functioning and integrity as well as benefiting human society’. The GEB for this project is the reduction in Carbon Dioxide (CO2) emissions that contribute towards climate change; which would be achieved as a result of market transformation for energy products, major increases in the energy efficiency of all sectors in Jamaica and a consequent reduction in the use of fossil fuels. 4. The JDSMP Outcomes to Impacts Theory of Change The theory of change for a project is the logical sequence of conditions and factors that are necessary to deliver the ultimate project impact. The basic project theory of change starts with activities and develops through a means-ends hierarchy until finally reaching impact. GEF project terminal evaluations assess the basic theory of change as far as outcomes, but do not usually go far in assessing the crucial last step to impact. The ROtI assessment focuses on this last step and develops and assesses a detailed theory of change between outcomes and impacts, referred to as outcomes to impacts pathways. Each outcome to impacts pathway represents a specific project strategy. Figure 1 below illustrates the key elements and relationships for the detailed theory of change between outcomes and impacts, which would hold for many GEF Climate Change activities. Figure 1. Generic theory of change for outcomes to impacts pathways in Climate Change sector ASSUMPTIONS Impact: OUTCOMES INTERMEDIATE MARKET Impact: Strategy TRANSFORMATION GEB STATES IMPACT DRIVERS The key ingredients in the outcomes to impacts pathways that are examined by the ROtI are intermediate states, impact drivers and assumptions, which are defined in Table 1 below. If the project outcomes are assessed to be successfully delivered and the key ingredients of the theory of change between outcomes and impacts are in place, then it is reasonable to conclude that there is indirect evidence that the barriers and threats to impact have been overcome and that impact has or will be achieved with time. 10 Table 1. Definitions of Theory of Change Elements in the Outcomes to Impacts Pathways TOC terms Definition Intermediate These are the transitional conditions between the project’s outcomes and impacts that States (IS) must be achieved in order to deliver the intended impacts Impact Drivers These are the significant factors that, if present, are expected to contribute to the ultimate (ID) realisation of project impacts and that are within the ability of the project to influence Assumptions These are the significant factors that, if present, are expected to contribute to the ultimate (A) realisation of project impacts, but that are largely beyond the power of the project to influence or address The overall theory of change for the Jamaica DSM Project is presented in Figure 2 below. 11 Figure 2: Theory of Change for Jamaica Demand Side Management Project 12 It can be seen from Figure 2 that the project had three strategies to deliver its intended impacts, namely:  Strategy 1: Reduce fossil fuel use  Strategy 2: Strengthen institutional capacity for reduced fossil fuel use  Strategy 3: Strengthen public support for DSM measures. These strategies will be examined individually to see how they were adopted and with what effects. However, it should be born in mind that the strategies are inter-related, rather than discreet and that they are treated individually for purposes of analytical clarity. Furthermore, underlying the project was a set of assumptions, which needed to hold true across all of the strategies if they were to succeed. During the course of project support, the three strategies were expected to contribute towards the delivery of four broad outcomes:  Outcome 1: Advantages of energy efficient products and services demonstrated  Outcome 2: Awareness, technical and management capacity in energy efficiency raised in all sectors (government, private sector industry and commerce)  Outcome 3: Energy efficiency standards are established  Outcome 4: Public awareness of climate change and energy efficiency raised. For analytical purposes, the broad outcomes were subdivided into more specific deliverables, all of which should be attained in order for the strategy to deliver its expected results. For each strategy, there were also impact drivers, which needed to be active at all stages of the intended cause and effect chain in order to promote and sustain new patterns of behaviour. These impact drivers were part of the processes set in motion by the project, but are not direct project outcomes. Although individual impact drivers may relate particularly closely to one strategy or another, they all need to be in place for any of the strategies to deliver its intended results. For the JDSM, there were five key impact drivers, namely:  ID 1: Energy efficient products and services are competitively priced, so that they make economic sense to users  ID 2: Government fiscal measures, such as taxes and duties, promote energy efficiency  ID 3: Government energy policies and measures promote energy efficiency  ID 4: Effective use made of capacities raised by the project  ID 5: The media and public bodies (such as Civil Society Organisations) would support and promote energy efficiency. Finally, a set of assumptions formed the basis on which the project results could be achieved. The extent to which these held would have an important effect on the extent to which the project could move towards its Global Environment objective. Three key assumptions were identified as underlying the project:  A 1: Energy efficient products and services will add value in the Jamaican situation  A2: Government exerts control over the energy sector  A3: Public opinion can be influenced in favour of Demand Side Management/Energy Efficiency. Between the outcomes and the intended impacts a complex set of intermediate states must be passed through. These states can be seen as forming two levels of change. Firstly, an increased demand for various products, services and types of expertise associated with energy efficiency; followed by predominance of these products and services in the national market. In order to analyse the achievements of the project in the long term, it is necessary to review each of its three strategies in turn, to see to what extent the intended cause and effect chain materialised. This is undertaken in Chapter 5 below. 13 5. Assessment of Achievement of the Outcomes to Impacts Pathways The assessment of achievement of the outcomes to impact pathways was undertaken on the basis of a ROtI group discussion with former senior project officers, individual interviews and a review of project documentation. The assessment is presented according to the three main strategies identified for achieving impact in the project brief. These strategies link with the identified intermediate states considered necessary to deliver impact. The rating system used for the assessment is defined in the ROtI handbook and shown in Table 2 below. It is applied at different levels of the Theory of Change, both at the individual TOC element level (outcomes, impact drivers, assumptions and intermediate states), and at the overall project level. Table 2. Field ROtI Rating System Rating Description 0 Not achieved 1 Poorly achieved 2 Partially achieved 3 Well achieved In Box 1 below, the ratings are elaborated with descriptions of their interpretations. These descriptions provide guidance for the rating; although the complexity of GEF projects means that an aspect of the Theory of Change may not exactly fit at one particular rating level. In this situation, the lower rating level is assigned. Box 1: ROtI Rating Guideline Interpretations Not achieved (0) From a theoretical perspective, the Theory of Change (TOC) aspect is not explicitly or implicitly identified by the project, and/ or from a delivery perspective, very little progress has been made towards achieving the TOC, and the conditions are not in place for future progress. Poorly achieved (1) From a theoretical perspective, there are no appropriate mechanisms set out to achieve the TOC aspect after GEF funding ended, and/ or from a delivery perspective, little progress has been made towards achieving the TOC aspect, but the conditions are in place for future progress. Partially achieved (2) From a theoretical perspective, the TOC aspect is explicitly recognised and the mechanisms set out to achieve it are appropriate but insufficient (e.g. there is no clear allocation of responsibilities for implementing the mechanisms after GEF funding ends). From a delivery perspective moderate and continuing progress is being made towards achieving the TOC aspect, although there is not yet a strong basis for the eventual delivery of the intended Global Environmental Benefits. Well achieved (3) From a theoretical perspective, the TOC aspect is explicitly recognised and appropriate and sufficient mechanisms to achieve it are apparent (e.g. specific allocation of responsibilities after GEF funding ended), and/ or from a delivery perspective substantial progress has been made towards achieving the TOC aspect and a strong basis is in place for eventual delivery of the intended Global Environment Benefits. 14 The reporting for each strategy starts off by providing a justification for why the identified intermediate state and associated factors for the strategy are considered important in delivering ultimate impact. The theory of change for the strategy is then examined through its logical steps, firstly validating the extent to which the outcomes were achieved at project closure, followed by an assessment of the extent to which the impact drivers and assumptions have been realised. Each section concludes with an assessment of achievement of the intermediate state itself. 5.1 Strategy 1: Reduced Fossil Fuel Use 5.1.1 Theory of change overview This strategy focuses on delivering a complex set of intermediate states, which were considered essential to enable attainment of the intended project impact of reduced CO2 emissions (the project GEB). The cause and effect chain is shown in Figure 3 below. The targeted intermediate states would provide essential steps to enable the scaling up and replication of the original project outcomes, leading to market predominance of energy efficient products and services, as well as energy efficient practices being adopted as the norm in government, commercial and industrial sectors. The project outcome that was identified as essential for delivering these intermediate states was Outcome 1 of the Overall Theory of Change, “Advantages of Energy Efficient Products and Services Demonstrated.� This outcome can be disaggregated to enable detailed analysis of the pathways towards the intended GEB. Thus Outcome1 can be seen to have four elements, as follows:  Outcome 1a: advantages of Compact Fluorescent Lamps (CFLs) demonstrated  Outcome 1b: advantages of energy efficiency (EE) demonstrated  Outcome 1c: advantages of Solar Photovoltaic power systems demonstrated  Outcome 1d: advantages of solar water heater demonstrated. Based on ROtI meetings and discussions, two sets of intermediate states (IS) were identified as necessary steps towards the expected impacts. These were as follows (see Figure 3):  IS 1a: Increased demand for CFLs leading to CFLs dominate lighting market  IS1b: Increased demand for EE measures and services leading to EE equipment and materials dominate markets and EE practices are the norm  IS 1c: Increased demand for solar PV, leading to an established market niche  IS 1d: Increased demand for solar water heater, leading to a major market share. Three impact drivers and three external assumptions were identified as necessary to bridge the gap between the project outcomes and the intermediate states, as shown in Figure 3 below. The first impact driver, “Energy Efficient products and services competitively priced and marketed� is a critical factor for scaling up from the initial outcomes demonstrating the advantages of CFLs, Solar PV and Solar Water Heaters and Energy Efficiency, as well as contributing to those outcomes. The second impact driver, “fiscal measures promote EE products and services� is critical to ensure that energy efficiency products and services can be offered at the lowest possible cost on a sustained basis. The third driver “media and public promote Energy Efficiency� will play an important role in ensuring a broad and continuing base of support for Government in promoting energy efficient products and services and for private sector companies, which adopt them. Progress towards the identified intermediate states is built on the three external assumptions that provide a basis for the entire cause and effect chain. The first assumption, “energy efficient products and services add value in the Jamaican situation� has several dimensions. The first concerns the extent to which energy is seen as a major cost to individuals and organisations. This relates to the cost of electricity supply, of products such as light bulbs and of energy management measures compared to the value of the savings they generate. This assumption relates strongly to the second, “Government controls or influences the energy sector;� since issues such as tariffs, 15 subsidies, import duties and of the direct role of government in energy supply will all play a major role in the relative cost-effectiveness of demand side management measures. The third assumption, “public opinion can be influenced in favour of DSM/EE� affects both the scale and the sustainability of support for policies, measures and products promoting decreased energy use. This relates both to direct energy issues and to the potential for growing public concern about Climate Change, which could be expected to increase support for DSM. The next section assesses the extent to which the theory of change relating to Strategy 1 has been realised in design and practice, by examining the achievement of the TOC components, starting with the outcomes and finishing with the intermediate state. Table 3 at the end of this section provides a summary of this analysis. 16 Figure 3: Theory of Change for Strategy 17 5.1.2 TOC assessment Outcomes In the overall Theory of Change for the project, Outcome 1 is that “Advantages of EE products and services are demonstrated.� Performance on this dimension was mixed. According to the ICR: “For the residential sector, the energy savings mobilized indicate that there would be significant potential for additional savings at low cost, so the Project provides an adequate basis for extending the lighting, solar water heating, and solar P.V. programs on a larger scale�. This corresponds to the sub-outcomes 1a, 1c and 1d, which are now reviewed individually. Outcome 1a: advantages of Compact Fluorescent Lamps (CFLs) demonstrated This was assessed by the ICR and by the ROtI panel as the most successful project outcome. By project closure, 32,000 households (nearly 10% of JPS customers) had benefitted and almost 100,000 CFLs had been sold to customers; with an estimated reduction of peak energy demand of 1.7MW. This sub-outcome is therefore assessed as well achieved. Outcome 1c: advantages of Solar Photovoltaic power systems demonstrated This activity aimed to provide energy to relatively isolated rural villages, which were thought to be unlikely to receive grid connection in the near future. Solar PV units were installed in 42 homes in two villages, with supporting training components in operation and maintenance of the equipment. The supplied equipment could power three light bulbs, a small television and a radio at the same time and was supplied under a 10 year repayment plan. The ICR did not assess this part of the Project, which started very late, but it appears that the units installed operated as intended. However, it cannot be said that such a small activity had effectively demonstrated the advantages of solar PV systems; so that this sub-outcome is assessed as poorly achieved. Outcome 1d: advantages of solar water heaters demonstrated. This component also started very late and its outcome was not assessed in the ICR. However, it appears that the intended 300 units were installed in residential units, resulting in an estimated 0.6 MW reduction in peak demand. Units were supplied on the basis of a two year repayment period. A second, commercial, component supplied, installed and maintained units in 12 hotels and in some student residences at the University of the West Indies, with repayment over a three year period. Payments made by beneficiaries under this programme fed into a Revolving Fund intended to ensure that the programme could be sustained. In view of the small scale and duration of this component, it is felt that its intended outcome was poorly achieved. Outcome 1b: advantages of energy efficiency (EE) demonstrated Under this component, the DSM Unit supported by the project in JPS undertook energy audits for 16 large volume commercial consumers, six of which implemented the proposed energy efficiency measures. At that stage, the relatively low take up of the measures, which were intended to reduce operational costs of the businesses, was attributed to the absence of low cost financing (i.e. incentives) to undertake the measures, the weak Jamaican economy and the inability of firms to self-finance such investments. It is therefore assessed that this sub-outcome was poorly achieved. 18 Overall Assessment of Outcome 1 The largest component of this outcome, the residential CFL component, was well achieved. However, the other three components were poorly achieved. The overall assessment for Outcome 1 is therefore that it was partially achieved. Impact drivers In order for the strategy to progress from the outcome stage towards the necessary intermediate states and impacts, three impact drivers needed to be active. The actual situation is assessed as follows. ID 1: Energy Efficient products and services competitively priced and marketed There is evidence that substantial efforts were made by the Government, and particularly by JPS, to make energy efficient products and services affordable. The main factor in this ID was the price of CFLs. Once initial technical concerns about what type of bulbs would work effectively on the 50 Hertz electricity system in Jamaica had been overcome, the project bought bulbs in huge quantities (one million in one order from General Electric in USA), which enabled them to be sold on to consumers cheaply with an effective subsidy. This condition did not obtain for other aspects of the project. Solar water heaters were too expensive to attain effective market penetration and were regarded as a luxury item. As a result of the project, the Government reduced duties on this equipment, with the intention of providing an incentive. However, importers simply held the original price to obtain a bigger profit margin, so that the net cost effect was minimal. Solar PV prices were also unaffordable without low cost financing and were not competitive for the rural communities being targeted. Energy efficiency audits offered price savings for industry and commerce in the long run, but no competitive financing arrangements to enable businesses to meet the initial costs. Overall then, it is clear that this impact driver was partially achieved, since it obtained for the biggest component but not for the other three, which were also important in moving the project towards its impact objective. ID 2: Fiscal measures promote EE products and services As indicated for ID1 above, Government did take fiscal measures to promote EE products and services. Notably, by project completion a range of Energy Saving Devices were zero-rated for General Consumption Tax. These included: compact fluorescent lamps and ballast, fluorescent fixtures and tubes, circular fluorescent lamps, fluorescent ballasts, high intensity discharge fixtures and tubes, fibre glass panels for sky lighting, automated, electronic or computerised lighting control systems, solar panels and tubes for solar water heating systems, solar cells and apparatus or machinery designed to produce motive power, heat, light or electricity through the utilisation of renewable sources of energy including sun, wind and water. In addition, Customs Duties on these items were reduced from 20% to 5%. Overall, it can be seen that this Impact Driver was adopted by Government, but that it had limited success, mainly with CFLs. It is assessed as partially achieved. ID 5: Media and public promote Energy Efficiency The project was active in engaging the media and public in efforts to promote energy efficiency. During and after the project, the press has regularly published articles on energy efficiency and several NGOs have also been active, often from a consumer affairs-oriented approach. CDs were produced and widely circulated and environmental journalism has become more productive. The DSM Unit marketing team had a dynamic programme in schools and communities. Furthermore, by project completion, DSM was well institutionalised in the energy industry, particularly in the publicly-owned sector. 19 The most effective messages have been those related to cost savings, which can be achieved by energy efficiency. Climate Change, although well-disseminated, has been less enthusiastically adopted in view of the small scale of Jamaica’s contribution in comparison with USA, China, India and other large users of conventional energy sources. Overall, this impact driver is assessed as well achieved. Overall Assessment of Impact Drivers 1,2 and 5 Two of the Impact Drivers to move along the causal pathway of this strategy were partially achieved, while the third was well achieved. Overall, it is assessed that the necessary IDs were partially achieved. Assumptions Three assumptions needed to hold true throughout the process leading towards the intended GEB, if progress were to be fully sustained. The same assumptions apply to all three strategies and are assessed below. A 1: Energy efficient products and services will add value in the Jamaican situation This assumption held true for CFLs. Initially, project funds were used to bulk purchase these and sell them at below market price. This succeeded in raising the market to such an extent that private retailers could eventually buy in sufficient numbers to get a good price and pass this on to consumers. The market price to consumers fell from a high of J$800 before the project to around J$300 in 2010. So CFLs now add value to the home lighting market in terms of saved energy costs, long life of bulbs and modest purchase price. It did not hold true with regard to solar water heaters for several reasons. Firstly, cultural factors limited the perceived added value of these units. In low cost housing areas hot water supply from any source is rare and is not seen as a priority. Secondly, the initial purchase price of a SWH was seen as very high. Thirdly, there was no loan and maintenance package to go with the purchase, so it was not affordable to most people; and the possibility of additional maintenance costs added further discouragement, which was exacerbated by the unreliability of early units imported. The cost of the solar PV units for rural areas was determined following a survey of existing energy use and costs. Repayments were scheduled over a five year period and were tailored to generate cost savings for users in that time. However, it appeared that users did not have adequate knowledge of how to best use the systems and that battery replacements were needed more frequently than anticipated, at a prohibitive cost of J$15,000 per unit. This made the Solar PV systems uneconomical, so that they were not seen as adding value, but rather as imposing unaffordable costs in the medium term. Energy efficiency in Government, commercial and industrial premises became understood as a means of saving substantial costs in the medium to long term. JPS itself was retrofitted with energy efficient lighting, air handlers and other systems, which led to significant reductions in operating costs. However, the initial retrofit was intended to be supplemented with additional Government funding, which did not materialise. In the private sector, initial energy audits showed the potential cost savings, but there was no programme of rebates or incentives to enable companies to commit the initial capital expenditure. Overall, the project promoted increased understanding of the potential added value of energy efficiency measures, but did not lead to affordable financing packages to enable this value to be realised on a large scale. Overall, it is assessed that this assumption only held true over time for one of the four outcomes and impact pathways of Strategy 1 and it was therefore partially achieved. 20 A 2: Government exerts control over the energy sector This assumption had several dimensions. The most substantive and critical dimension for the project was that Government held a monopoly on power supply during its planning and implementation. The JPS was therefore able to control market factors, such as the importation of CFLs in bulk, incentives for energy efficiency and repayment packages for energy efficient appliances. Furthermore, improved DSM meant that Government had to spend less on power generation, as well as contributing to its efforts to reduce Jamaica’s contribution to climate change. Within this context, the activities of the DSM Unit in JPS were valued and supported by Government. However, in 2002 JPS was privatised and sold to an overseas company. The new company was primarily interested in expanding its sales volume, so that increased consumption became the goal. Demand Side Management was far from a priority, since it could actually lead to reduced sales; and the only “energy efficiency� measures of interest were those to control losses due to illegal connections, as a means of raising sales. A second strand of Government control in the sector is that of tariffs and duties on energy related products and services. In this arena, support was sustained beyond the project and the fiscal climate is still somewhat favourable to DSM. The third aspect of Government control is the arena of energy policy. Here the Government has been active since project completion and a substantial set of policy documents has been produced, including an Energy Green Paper, which has not yet become a White Paper. Some measures have already become law, including the possibility for additional sources of power supply (with a 15 MW threshold) through co-generation, which can be sold on to the national grid; which has been taken up by a small number of companies. Overall, this assumption is assessed as partially achieved, since Government remains in control and active in policy and tariff structures related to energy efficiency; but not in power supply and distribution. A 3: Public opinion can be influenced in favour of Demand Side Management/Energy Efficiency The project devoted substantial attention to raising public awareness and attempting to create favourable public opinion concerning energy efficiency and demand side management, both in terms of the financial advantages offered and of its climate change aspects. However, despite these efforts, it cannot be said that major influence was achieved, to raise the issues as a major area of effective public interest and activity; so that this assumption is assessed as poorly achieved. Overall Assessment of the Validity of Assumptions On the basis of the above analysis, it is shown that two were partially achieved and one poorly achieved. Overall the assessment is that the assumptions were partially achieved (that is, they partially held true during the project and over the next ten years). Intermediate States As shown in Figure 3, there were two broad stages in the achievement of intermediate states; firstly, an increase in demand and secondly, a predominant market position established on each pathway towards the reduction of fossil fuel use. These are now analysed, focussing particularly on the second intermediate state, since it was possible that this could be reached within the ten years, which have elapsed since project completion. 21 IS 1a: CFLs Dominate the Market Initial problems with the quality of CFLs had been overcome and consumer satisfaction and confidence in the product were well-established by project completion. The possibility of paying for the CFLs as part of the regular JPS electricity bill was an important contributor to the good rate of take up and by 2000, about 10% of JPS customers, totalling 32,000 households had purchased CFLs. At this time, preparations were under way to privatise electricity supply and, as part of this process, several of the JDSMP staff were transferred to the PCJ, which remained a Government entity, to manage a new set of activities, the Energy Efficiency and Conservation Programme (EECP), which commenced in 2000. Under the EECP, sales of CFLs were now made through PCJ outlets (i.e. petrol stations) on a non-profit basis, slightly under-cutting the retail price. In 2001, 80% of the Government’s shares in JPS were sold to Mirant Energy Inc., although the name of JPS was retained under a 20 year licensing agreement. It was not felt that DSM would be a high priority of the new enterprise, which had to focus on renovating the existing power supply system, increasing generation capacity and reducing system losses. The project Outcome 1, of demonstrating the advantages of CFLs, had been delivered by project completion to such an extent that the retail price of CFLs had reached a sustainably lower level, of about one-third of the pre- project cost. Nevertheless, they remained almost twenty times as expensive as incandescent bulbs, although this ratio continued to fall. Sales continued to increase and an Evaluation of the EECP conducted in 2004, estimated that 40% of households were using an average of 2 CFLs. At the time this represented some 184,800 households, a substantial increase of around 600 percent since project completion. It is likely that incandescent bulbs will retain a substantial market share in the short to medium term, for at least two reasons. Firstly, they remain much cheaper in terms of initial costs. Secondly, a substantial proportion of electricity users, particularly in poor urban communities, do not pay for their supply since they have “informal� connections to the grid. They do not therefore stand to gain from using less energy. To the extent that JPS can reduce this body of free users, pressure in favour of CFLs may increase over time, promoting a still higher market share. Although it cannot be said that CFLs dominate the Jamaican market yet, the project achieved as much as could be expected on this dimension. Further market penetration may occur in the long term, as part of broader global trends against incandescent bulbs and if substantial numbers of low income households currently using free electricity later have to pay for it. Overall it is assessed that IS 1a was well achieved. IS 1b: Energy Efficient Equipment and Materials Dominate the Market and Energy Efficient Practices are the Norm The Demand Side Management Unit was not transferred to the new JPS, but its Energy Services Unit took over the role of conducting energy audits for its major clients on a fee paying basis. The follow up to these audits in terms of actual energy saving measures has not been recorded or published. The Petroleum Corporation of Jamaica, which is Government owned, housed a new Energy Efficiency Unit, created in 2003; intended to oversee energy efficiency and conservation programmes at the national level, including the promotion of CFLs and solar PV systems and the conducting of energy audits for government entities. This was to a large extent the successor of the project and many of its staff were transferred across to the new body. In Jamaica’s commercial and industrial sectors, energy costs are rarely seen as major elements of the overall cost of doing business. A handful of prominent companies, such as Grace Kennedy and Pegasus have institutionalised their own energy efficiency systems, but they are the exception. Issues and staff in such areas as operations and maintenance are of low status and do not attract the attention of management. Much energy continues to be wasted through poor maintenance. Similarly, in hotels there is little emphasis on issues such as the use of cards to switch room electricity off when rooms are not in use. There are however some hotel projects to use solar water heating and lighting, particularly among boutique hotels targeting a more environmentally 22 aware international clientele. Government has been much more interested than the private sector in energy efficiency, since fuel bills are a major element of its overall operating budget. It has continued to make major efforts to retrofit hospitals and schools to save energy. These measures have yielded substantial results and Government has saved 20% of its energy costs. The main hindrance to further savings in this sector is the lack of funds to address more buildings. Overall, the project moved demand a small way along the line from the original outcome towards the first step of the IS, since there was a slightly increased demand for energy efficient services, products and practices. However, ten years after project completion, it is still far from the situation in which energy efficiency is the norm. IS1b is therefore assessed as poorly achieved. IS 1c: Solar PV has an Established Niche The solar PV component of the project was something of an afterthought and only commenced in the last year of activities. Although the two villages engaged in the pilot found the systems usable after some initial difficulties, it later emerged that the cost of replacing the battery was prohibitive, at round J$15,000. An end user survey conducted in the villages in 2007 found that none of the units remained in operation. For medium cost housing, a loan of J$1.5 million is available, to purchase and install solar PV. However, take up is low, particularly since the title deed to the house is held as security on the loan. There have been some specific uses of solar PV, such as for lighting streets in tourist areas. The Government-owned Jamaica Marketing Company (JAMCO) had as one of its objectives the provision of solar PV in schools, but progress has been limited. Overall IS 1c has been poorly achieved. IS1d: Solar Water Heaters have a Major Market Share Solar Water Heaters did not take off in a major way as a result of the project. There was a concept for PCJ to promote this technology and encourage a manufacturing facility in the country to provide units to the region at a more reasonable price. Substantial research was undertaken on the feasibility and options for this, but the facility was not established. Although the outcome was delivered, there was little progress towards even the first step of this IS, which would require increased demand to be established for SWH; and still less for the second stage of SWH having a major market share. Overall, IS 1d was poorly achieved. Overall Assessment of the Achievement of Intermediate States 1a to 1d The four Intermediate States, which needed to be reached in order to enable delivery of the intended impacts, had very mixed results. Whilst one was well achieved, the other three were poorly achieved. The overall assessment is that these ISs were partially achieved. 5.1.3 Overview of Progress from Outcomes to Impacts of Strategy 1 On the basis of the evidence and analysis presented above, an overview of the progress made from Outcomes to Impacts over the period 2000 to 2010 is presented in Table 3 below. 23 Table 3: Outcomes to Impacts Assessment Findings for Strategy 1 Theory of Change Assessment Rating Component Outcome 1a: Advantages of By project completion, 32,000 households had started to use 3 CFLs demonstrated CFLs. Outcome 1b: Advantages of Energy audits were undertaken in private sector bodies, but few 1 Energy Efficiency could afford the necessary measures. Government was more demonstrated. active, but limited by finances. Outcome 1c: Advantages of Solar PV units were installed in two villages, but the 1 Solar PV systems demonstration value was very limited. demonstrated. Outcome 1d: Advantages of SWHs were installed in small numbers in residential, 1 Solar Water Heaters commercial and government buildings with a minimal demonstrated. demonstration effect. Impact Driver 1: Energy CFLs were effectively subsidised and marketed by the project, 2 Efficient products and but the other elements were not. services competitively priced and marketed. Impact Driver 2: Fiscal Government instituted favourable tax and duty measures, which 2 measures promote EE were more effective for CFLs than for other products and products and services. services. Impact Driver 5: Media and Considerable efforts were made and some CSOs adopted EE and 3 public promote EE. climate change issues. Assumption 1: Energy CFLs added value, but other products promoted did not on any 2 Efficient products and scale. Energy audits and other services potentially added value, services add value. but few could afford to pursue the measures they proposed. Assumption 2: Government Government continues to exert control through policies, 2 exerts control over the strategies and financial measures. However, it lost its main energy sector. control when it privatised energy supply. Assumption 3: Public Substantial efforts were made, resulting in a moderate shift in 2 opinion can be influenced public opinion in favour of DSM/EE. in favour of DSM/EE. Intermediate State 1a: CFLs Ten years after project completion, the market share was about 3 dominate the market. 40%, from virtually nil beforehand. Intermediate State 1b: The main gain in the market was for CFLs, but demand for EE 1 Energy efficient equipment equipment and materials only rose slightly. The proportion of and materials dominate the energy efficient buildings has increased slightly, but mainly in market and EE practices are the Government sector. the norm. Intermediate State 1c: Solar The demonstration value of solar PV systems was short-lived 1 PV has an established and they were neither sustained nor widely replicated. No niche. Market niche was established. Intermediate State 1d: Solar There was minimal progress on this pathway, since the systems 1 Water Heaters have a were expensive and demand was low. Major Market Share. 24 5.2 Strategy 2: Strengthen Institutional Capacity for Reduced Fossil Fuel Use 5.2.1 Theory of change overview The second strategy focussed on a complex set of outcomes and intermediate states, which would support progress towards the intended GEB of reduced CO2 emissions, as a result of all sectors in the country operating in an energy efficient manner. The cause and effect chain is shown in Figure 4 below. Whilst the intended outcomes would raise capacity, awareness and standards, the intermediate states would build on these achievements to place EE and DSM at the centre of national policies, approaches and practices. This strategy had two intended outcomes, which were seen as essential in order to successfully move towards the intermediates states. These were Outcome 2, “Awareness, Technical and Management Capacity in all Sectors in Energy Efficiency Raised,� which consisted of three sub outcomes; and Outcome 3, “Energy Efficiency Standards Applied.� The overall set of outcomes and sub-outcomes for this strategy is therefore as follows:  Outcome 2a: technical capacity in Energy Efficiency raised  Outcome 2b: commercial/industrial/Government sectors’ awareness raised  Outcome 2c: JPS and energy sector DSM/EE management capacity strengthened  Outcome 3: Energy Efficiency standards established. Based on ROtI meetings, discussions and document analysis the impact pathway was expected to lead through an initial set of five intermediate states to 3 second stage states. These were as follows (see Figure 4):  IS 2a: Demand for technical services for energy efficiency raised, leading towards the second IS of EE practices as the norm  IS 2b: Government and private sector interest sustained and increased and the Government prepares an energy policy, leading towards the inter-related second Intermediate States of EE practices as the norm and Government adopting an energy policy  IS 2c: JPS and the energy sector sustain and increase interest in EE/DSM, leading to DSM as a core approach of the energy sector  IS 3: Energy efficiency standards applied, leading towards EE practices as the norm. Two impact drivers were necessary to enable movement from the outcomes achieved towards the intermediate states, as shown in Figure 4. The first ID was “Government policies promote DSM/EE,� which was held to be critical if awareness of the issues and appropriate responses were to be raised to a sufficient level to ensure consistent action. The second was “Effective use made of capacities raised by the project,� which had to be true for policies and actions to attract significant take up across all sectors, since the “bottom line� would have to be reduced costs of doing business for those organisations, which enacted the measures recommended by energy audits. The status of these impact drivers will be assessed below. Progress towards the intermediate states is built on three external assumptions, which underlie the cause and effect chain. These assumptions are the same across all three strategies and have already been assessed under Strategy 1. They are as follows:  A1: Energy Efficient products and services add value in Jamaican situation  A2: Government controls or influences energy sector  A3: Public opinion can be influenced in favour of DSM/EE. The next section explores the extent to which the Theory of Change relating to Strategy 2 has been realised in the ten years since project completion, by examining each of the components in turn. Table 4 at the end of this section summarises the evidence and assessments. 25 Figure 4: Theory of Change for Strategy 2 26 5.2.2 TOC assessment Outcomes In the overall Theory of Change, Strategy 2 was expected to deliver two outcomes. These were Outcome 2, “Technical Capacity in all Sectors in Energy Efficiency Raised,� which consisted of three sub outcomes; and Outcome 3, “Energy Efficiency Standards Applied.� Performance of the three components of Outcome2 is first analysed and assessed. Outcome 2a: Technical capacity in Energy Efficiency raised The project made substantial efforts to provide training on the technical aspects of energy efficiency. In particular, JPS staff and engineers experienced substantial capacity development, in many cases leading to professional certification. As a result of the training, the DSM Unit was able to conduct Energy Audits and to advise on their contribution to business productivity. Later, the new Energy Efficiency Unit of PCJ was able to channel this expertise through sponsoring courses on EE at University level. This sub outcome is assessed as well achieved. Outcome 2b: Commercial/Industrial/Government sectors’ awareness raised Substantial efforts were made to raise awareness across all sectors of the opportunities and means to reduce costs by enhancing EE. The most progress was made in the Government sector, where the services of JPS managed to make many agencies aware of measures, which they could take; particularly in the health and education sectors. In the industrial and commercial sectors, a small number of companies were energy audited, leading to substantially raised awareness of issues and solutions. However, the spread of companies was relatively small compared to the size of these sectors in Jamaica. This sub outcome is assessed as partially achieved. Outcome 2c: JPS and energy sector DSM/EE capacity for energy management strengthened In addition to the technical skills, it was vital that the project delivered enhanced capacity to conduct and promote energy management within the energy sector. JPS was pivotal in this process and received substantial training in energy management and planning, which was a major resource for further development of government strategies and policies. This sub-outcome is assessed as well achieved. Outcome 3: Energy Efficiency standards established The project assisted with the development of a variety of standards; notably for SWHs, CFLs and energy equipment. The intended Testing Laboratory did not reach the required standards and international companies were contracted to conduct work in this area. The Bureau of Standards Office is located at the port and devices and equipment are tested in situ, with refusal of entry to failed products. Appliance labelling proved difficult due to lack of local capacity. This sub outcome is assessed as partially achieved. 27 Impact drivers Two impact drivers are seen as essential to move from the outcomes specified above towards the long term goal and GEB of the project. Achievement of these is assessed below. ID 3: Government promotes energy efficiency During the course of the project, the DSM Unit in PSJ spearheaded Government’s efforts to promote EE. Campaigns targeted Government, industry and commerce. A major strand of this activity was the promotion of energy efficiency audits. Some Government sectors, notably education and health, as well as PSJ itself took measures to implement a range of energy efficiency measures on the basis of these audits. However, in the private sector, operators were reluctant to put recommendations into practice, since they had to balance substantial immediate costs against long term benefits. It had been anticipated that a programme of Government rebates would be instituted for those taking EE measures, but this was never enacted and many companies decided not to go ahead in view of the uncertain state of the Jamaican economy. With regard to solar water heaters, demand was slight, owing to the high initial cost. The most effective way to boost adoption would be as a requirement for new housing, through compulsory provisions in the Energy Efficient Building Code. This was not enacted and the current Code merely recommends to architects and developers to include SWH in housing units costing more than J$8 million. Overall, this ID is assessed as partially delivered. ID 4: Effective use made of technical capacities raised by the Project Performance on this impact driver was strong during project implementation, particularly in regard to the trained staff in JPS. However, once JPS was privatised, staff were dispersed and the opportunities and impetus to use their new capacities diminished substantially. After a gap of more than two years, an Energy Efficiency Unit was created in PCJ, with some staff from the previous DSM Unit and to a limited extent use was again made of the technical skills learned with project support. Within Government in particular, the practice of energy audit has continued, with substantial follow up in a limited number of sectors. Work on the energy efficiency of hospitals has been continued through support from UNDP. Overall, this ID has been poorly achieved, since progress made during the project was disrupted owing to major changes in the structure of the power sector. Overview of Impact Drivers The first of the two impact drivers was partially achieved, whilst the second was poorly achieved. Overall, it is assessed that the two necessary impact drivers for this strategy were poorly achieved. Assumptions The same three assumptions needed to hold true to support all three strategies of the Project. Performance on these has been assessed in Section 5.1 above. The overview is as follows: A 1: Energy efficient products and services will add value in the Jamaican situation: partially achieved. A 2: Government exerts control over the energy sector: partially achieved. A3: Public opinion can be influenced in favour of DSM/EE: poorly achieved. 28 Overall Assessment of the Validity of Assumptions On the basis of the above analysis, it is shown that two were partially achieved and one poorly achieved. Overall the assessment is that the assumptions were partially achieved (that is, they partially held true during the project and over the next ten years). Intermediate states The pathway through the Intermediate States for this strategy was complex. Five initial ISs would focus down on three final ISs, leading to the intended impact. These processes are now assessed. IS 2a: Demand for EE technical services raised, leading towards EE practices are the norm The Outcome leading towards this IS was well achieved; and capacity to undertake EE audits and recommend energy saving measures was available for all sectors. However, a number of factors reduced the onward progress of this trend. Notably, the privatisation of JPS led to a reduction in interest in DSM/EE from the energy provider, which transmitted itself to energy users. Furthermore, the initial costs of improvements in energy efficiency had proved more than most private sector operators felt they could afford. Finally, the initial DSM Unit staff were dispersed, leading to a loss of impetus. Overall, demand for EE technical services, which rose during the project, retreated after its completion and particularly, after the privatisation of JPS. So the first stage of this IS was poorly achieved as was the final step, “EE practices are the norm�. IS 2b: Government and private sector interest sustained and increased, leading towards EE practices are the norm Progress towards this IS had divergent paths. For Government, the advantages of taking EE measures in terms of recurrent budget were widely accepted and the initial cost was mainly a problem in terms of when the money might be made available. On the other hand, in the private sector, the initial costs were seen as too high in a difficult economic climate, when the benefits would be realised in a slow “trickle� over time. So only a few very well capitalised Companies were able to sustain and increase their interest in EE practices, while the majority postponed making that interest effective until the economy and their profits could take a turn for the better. The first stage of this IS was partially achieved and the final step, “EE practices are the norm� was poorly achieved. IS 3: Energy Efficiency standards applied, leading towards EE practices are the norm. The Government took a major lead in applying EE measures in its own buildings, with savings of 20% of energy costs achieved through the use of improved lighting systems. Many schools and hospital buildings were also retrofitted to operate efficiently. The main restriction on progress was because of limited Government budgets. In the commercial and industrial sectors, only a few big players were able to apply the highest standards. Building Codes did not institute compulsory requirements concerning EE. Standards for the quality of some items such as CFLs and SHW were developed and applied, so that sub-standard products were gradually phased out. Overall, the application of EE standards was partially achieved, but fed into the next phase IS of EE practices as the norm, which was poorly achieved. IS 2c: Government prepares an Energy policy, leading towards Government adopts energy policy Substantial work has been done to prepare an energy policy. It exists as a Green Paper, which effectively indicates that, with possible modifications, it will eventually move on to become a White paper, or full policy document approved by Parliament. In addition to the main policy document, there are several specific support papers exploring specific aspects of energy, including EE. The DSM project is seen as one of the important 29 streams which fed into preparation of this document. Although the energy policy has not yet become government policy, this Intermediate State is assessed as well achieved. IS 2d: JPS and energy sector interest sustained and increased During the project, JPS was the focus of the emerging interest in EE/DSM in the country. However, with the privatisation of the major energy provider, the DSM Unit was disbanded and much of this interest was lost. The JPS focus switched to creating additional power generation capacity and, to a lesser extent, on reducing system losses to enhance profitability. The energy sector changed its nature, with the privatised JPS generating and distributing power and PCJ taking on some responsibility for DSM through its new Energy Efficiency Unit. The coherence and drive of interest in DSM/EE was lost. Overall, this IS was poorly achieved. Overall Assessment of the Intermediate States Out of the five Initial ISs, one was well achieved and the other four were all poorly achieved. Looking at the second phase of these ISs it is assessed that one was well achieved. Government is well on the way to adoption of a national energy policy, with a well-researched set of documents to support it. This provides a firm foundation for improvements in energy management in the country. However, the policy on its own cannot lead to the desired impact and it is assessed that the other two second stage ISs are still far from achieved, ten years after project closure. Energy efficient practices remain in a minority and DSM is far from a core approach in the Sector. Overall, it is assessed that the Intermediate States to which the project should have contributed have been poorly achieved. 5.2.3 Overview of Progress from Outcomes to Impacts of Strategy 2 On the basis of the evidence and analysis presented above, an overview of the progress made from Outcomes to Impacts over the period 2000 to 2010 is presented in Table 4 below. 30 Table 4: Outcomes to Impacts Assessment Findings for Strategy 2 Theory of Change Component Assessment Rating Outcome 2a: Technical capacity in EE raised By project completion, technical capacity had been 3 substantially raised in the country, particularly in JPS. Outcome2b:Commercial/Industrial/Government Whilst Government awareness was substantially raised, the 2 sectors’ awareness raised. other two sectors made slight progress. Outcome 2c: JPS and energy sector DSM/EE Energy management capacity, particularly within the 3 capacity for energy management strengthened Government entity JPS, was substantially raised. Outcome 3: Energy efficiency standards established Progress was made with the development of standards in 2 some areas of EE/DMS. The intended Testing Laboratory was not delivered and standards cannot yet been seen as firmly established. Impact Driver 3: Government promotes energy During the project, the Government made substantial efforts 2 efficiency to promote EE/DSM, but this faded after project completion and the privatisation of the main energy supplier. Impact Driver 4: Effective use made of technical Technical capacities were raised during the project, but the 1 capacities raised by the project. reorganisation of the energy sector led to dispersal of staff with EE/DSM expertise and downgrading of the skill set, in comparison with power generation and system loss reduction. Assumption 1: Energy Efficient products and CFLs added value, but other products promoted did not on 2 services add value. any scale. Energy audits and services potentially added value, but few could afford the measures they proposed. Assumption 2: Government exerts control over the Government continues to exert control through policies, 2 energy sector. strategies and financial measures. However, it lost its main control when it privatised energy supply. Assumption 3: Public opinion can be influenced in Substantial efforts were made, resulting in a moderate shift 2 favour of DSM/EE. in public opinion in favour of DSM/EE. Intermediate State 2a: Demand for EE technical Demand was raised by the project, but dropped off 1 services raised, leading towards EE practices are the afterwards; owing to different priorities of the privatised norm. power provider, perceived high costs of EE measures for the private sector and reduction in availability of technical support. Intermediate State 2b: Government and private Government sustained its interest, but was restrained by 1 sector interest sustained and increased, leading tight budgets; whilst private sector interest declined. towards EE practices are the norm. Intermediate State 2c: Government prepares an Government has made major progress towards adoption of 3 energy policy; leading towards Government adopts an energy policy. energy policy. Intermediate State 2d: JPS and energy sector interest The energy sector substantially changed, with reduced 1 sustained and increased. interest in the privatised JPS and a delayed take over of responsibilities by the new Energy Efficiency Unit in PCJ. Intermediate State 3: Energy Efficiency standards Government applied EE standards within budgetary 1 applied, leading to EE practices are the norm constraints, but few private sector organisations did. Building Codes did not demand Energy Efficient buildings. The application of EE standards stalled. 31 5.3: Strategy 3: Strengthen Public Support for DSM Measures 5.3.1 Theory of Change Overview This strategy aimed to deliver a set of Intermediate States, which were expected to play an important role in leading towards ultimate delivery of the intended impacts. The cause and effect chain is shown in Figure 5 below. These ISs would ensure that Government continued to promote and support EE/DSM measures, leading to reduced fossil fuel use over time. Outcome 4 was expected to lead towards these States. This was “Public awareness of climate change and energy efficiency raised.� As a sub-outcome, special interest was paid by the project to ensuring that “Youth awareness of CC and EE raised,� since this was seen as vital to ensure the sustainability of public interest. Based on ROtI meetings and discussions, the Intermediate States were expected to pass through two stages. Firstly, the Outcome of increased public awareness would lead to “increased public demand for the adoption of EE measures and products.� Secondly, the sub-outcome of “youth awareness of CC and EE raised� would lead to the IS, “young people promote EE.� Together these first stage ISs would lead to the final IS, “Government strengthens support for EE measures and products,� which would contribute towards achievement of the first Impact stage, “increased EE and reduced use of fossil fuel.� Two Impact Drivers would provide impetus along this causal pathway. The first of these was “Media promotes EE� and the second was the parallel requirement that “Government promotes EE.� As a sub-section of the latter, it was required that the “Education system promotes EE,� in order to build sustainability into public support for EE measures. The same three assumptions needed to hold true as for the other strategies. The next section examines how Strategy 3 was delivered in practice, during and after the project, by examining each element of the TOC in turn. 32 Figure 5: Theory of Change for Strategy 3 33 5.3.2 Theory of Change Assessment Outcomes The single Outcome for this Strategy concerns raised public awareness, particularly amongst the nation’s youth. Performance is now assessed. Outcome 3a: Public awareness of climate change and energy efficiency raised The project made substantial efforts to promote public awareness of climate change and the connection between these and energy efficiency. Although effective campaigns were mounted, the issue remains peripheral in a country, which faces many other issues perceived as more pressing. Awareness of the benefits of CFLs, at least in terms of cost effectiveness, was widely received among the general population by project completion, as shown by the substantial rise in their use. Overall, this outcome was partially achieved. Outcome 3b: Youth awareness of CC and EE raised The project had a substantial focus on schools and educational programmes, as a means of raising awareness in the long-term. For example, GEF funding sponsored a schools essay competition on EE and, as part of the prize, winners had their homes retrofitted to conserve energy. Overall, the project generated substantial new interest in these issues among youths and is assessed as well achieved. Overall assessment of Outcome 3 Overall, substantial progress was made in raising awareness among the public in general and the youth in particular. Although this did not place EE and CC in the centre of public concerns, in view of the many economic and political problems facing the country, this outcome is assessed as well achieved. Impact Drivers Three Impact Drivers were seen as necessary to ensure progress along the Theory of Change linkages towards the ultimate impact. These are assessed below. ID 5: Media and public promote EE. As noted in connection with Strategy 1, the project was active in engaging the media and public in efforts to promote energy efficiency. The press frequently publishes articles on energy efficiency and several NGOs have been active, often from a consumer affairs-oriented approach. CDs were produced and widely circulated and environmental journalism has become more productive. The DSM Unit marketing team had a dynamic programme in schools and communities. Furthermore, by project completion, DSM was well institutionalised in the energy industry, particularly in the publicly-owned sector. The most effective messages have been those related to cost savings, which can be achieved by energy efficiency. Climate Change, although well-disseminated, has been less enthusiastically adopted in view of the small scale of Jamaica’s contribution in comparison with USA, China, India and other large users of conventional energy sources. This ID is assessed as well achieved 34 ID 3: Government promotes EE Strategy 2 also relied on this ID and it was assessed that the DSM Unit in PSJ spearheaded Government’s efforts to promote EE. Campaigns targeted Government, industry and commerce. A major strand of this activity was the promotion of energy efficiency audits. Some Government sectors, notably education and health, as well as PSJ itself took measures to implement a range of energy efficiency measures on the basis of these audits. However, in the private sector, operators were reluctant to put recommendations into practice, since they had to balance substantial immediate costs against long term benefits. It had been anticipated that a programme of Government rebates would be instituted for those taking EE measures, but this was never enacted and many companies decided not to go ahead in view of the uncertain state of the Jamaican economy. With regard to solar water heaters, demand was slight, owing to the high initial cost. The most effective way to boost adoption would be as a requirement for new housing, through compulsory provisions in the Energy Efficient Building Code. This was not enacted and the current Code merely recommends architects and developers to include SWH in housing units costing more than J$8 million. This ID is assessed as partially achieved. ID 3a: Education system promotes EE This is a sub-component of ID 3, since the Government is the principal player in the education system. The initial progress at the outcome level has continued and EE issues have even been adopted in University level courses. The range of education system support includes:  Tertiary�level courses and research on energy conservation, efficiency and renewable energy were established at the University of the West Indies, Mona and the University of Technology, Jamaica.  UWI established the Solid State Electronics Research Laboratory, which focuses on the development and utilization of alternate energy sources through photovoltaic cells. The Physics Department offers undergraduate courses in solar power, Wind & Hydro power, and Integrating Alternative Energy. A new post graduate course in Energy Efficiency and Renewable Energy Technologies in the faculty of Pure and Applied Sciences was introduced.  At the University of Technology, a course in mechanical engineering entitled Energy Production Systems is offered and about 50% of it covers the sub�topic entitled Renewable Energy Resources. Energy conservation is included under this sub�topic. In electrical engineering, modules on energy conservation and renewable energy are offered to undergraduate degree students.  The Ministry of Education has incorporated energy issues, including conservation and renewable energy in the Revised Primary Curriculum (Grades 1�6) and the Reform of Secondary Education (ROSE) (Grades 7�9) Science syllabus.  The Caribbean Examinations Council offers a course in Environmental Science for the Caribbean Advanced Proficiency Examination (CAPE) level; this course includes examination of energy issues.  The Joint Board of Teacher Education developed two teacher education courses on Environmental Education for teachers in early childhood / primary and secondary programmes; these courses include discussion of energy issues.  The Management Institute for National Development offers the “Holistic Governance: Sustainable Development in Action� Programme that includes courses in Environmental Management and Stewardship (which include topics related to energy) targeted at all levels of the public sector  Numerous non�formal environmental education initiatives are conducted in schools by NGOs and government agencies such as PCJ and NEPA, which raise awareness about energy issues including pollution, conservation and renewable energy. In light of the significant developments highlighted above, this ID is assessed as well achieved. 35 Overview of Impact Drivers The first ID was well achieved; whilst the second was partially achieved overall, but with an important sub component well achieved. Overall the IDs for this Strategy are assessed as well achieved. Assumptions The assumptions necessary to enable success of this strategy are again the same and their assessment is: A 1: Energy efficient products and services will add value in the Jamaican situation: partially achieved. A 2: Government exerts control over the energy sector: partially achieved. A3: Public opinion can be influenced in favour of DSM/EE: poorly achieved. Intermediate States The pathway through the Intermediate States for this Strategy was less complex that for the other two strategies, with only two initial ISs and one final IS, expected to lead directly towards the intended impacts. IS 3a: Increased public demand for adoption of EE measures and products The heightened awareness among the public and youth in particular, has not led to any major increase in public demand for EE measures and products. Apart from CFLs, which have secured a strong market position, aspects of EE and DSM have not yet “taken off� in public requirements of the Government, partly because there is a high degree of public disillusionment with national Governance procedures. Indeed, even EE has not escaped this problem. In 2006, the Cuban government donated four million CFLs free of charge to the Jamaican Government for distribution to the population. It later emerged that Government contracts for more than J$114 million had been awarded to distribute the bulbs. The matter went to Parliament and later to the Courts. Although perhaps raising awareness of CFLs still more, this did not help the cause of strengthening public demand for interventions in the area of EE. Overall, this IS has been poorly achieved. IS 3b: Young people promote EE Substantial activities within the educational sector have continued and the PCJ Energy Efficiency Unit has targeted inner city schools. This has led to raised awareness among children and youths concerning Climate Change and Energy Efficiency. There is no systematic evidence on the results of this, but in view of the high degree of attention given to this area of activity it is assessed that this IS has been well achieved. IS 3: Government strengthens support for EE measures and products The two initial ISs were expected to lead to this final IS. This IS has been subject to divergent forces. On the one hand, Government sold its controlling interest in the power supply and distribution facility, PSJ and thereby greatly reduced its opportunity to effectively support DSM. As discussed earlier, the priorities of the new private sector company were not on DSM. Furthermore, Government did not make Energy Efficient equipment, such as Solar Water Heaters compulsory on new buildings with hot water capacity. On the other hand Government has made substantial efforts to incorporate DSM into its new energy policies. Within the National Energy Policy, which awaits Parliamentary approval, three goals address EE/DSM issues:  Goal 1: Jamaicans use energy wisely and aggressively pursue opportunities for conservation and efficiency 36  Goal 6: Government Ministries and agencies are a model/leader in energy conservation and environmental stewardship in Jamaica  Goal 7: Jamaica’s industry structures embrace eco-efficiency for advancing international competitiveness and move towards a green economy. A further goal has implications for DSM issues, namely:  Goal 5: Jamaica has well-defined and established governance, institutional, legal and regulatory framework for the energy sector that facilitates stakeholder involvement and engagement. As a supporting element of the Jamaica National Energy Policy 2009-2030, the Ministry of Energy and Mining has released (October 2010) a draft National Energy Conservation and Efficiency Policy. This Policy document, if approved, would provide major support and renewed drive to the issues of DSM/EE originally promoted by the Project, which have to some extent declined since its completion. Many of its key strategies and actions can be seen as following up on the DSM Project. Some of these are listed below. Strategies and actions related to Information, Education and Training, and Demonstration  Develop and implement programmes to influence market behaviour toward and promote efficient use of energy including the use of energy-efficient appliances, equipment, and building designs; setting and enforcing standards for public sector organizations  Provide opportunities for access to clear and consistent information on energy efficient products and services.  Development of awareness and training initiatives that will ensure that all persons, including householders, business persons, professionals, informal traders, farmers, drivers, public transport operators and students are sensitized so that better choices and more efficient behavioral changes can be made.  Develop and Implement awareness and communication campaigns will also be carried out to ensure stakeholder endorsement of the intervention programme and consumer shift towards more rational energy use.  Incorporate international best practices and findings of market surveys and consumer focus groups in design and implementation of demand side management programmes  Review, evaluate and improve previous and existing demand-side energy management programmes for performance, strengths, weaknesses and lessons learned  Development of programmes to facilitate the infusion of ECE across the curricula in all levels of the educational system.  Create forums to showcase and promote energy efficiency technologies and energy conservation measures for society at large – possibly though the use of community and government buildings and educational facilities to showcase new technologies.  Identify energy efficiency skills requirements across the economy and associated training, accreditation and higher education needs.  Establish networks and partnerships with government, private sector and academia as well as other key research institutions to promote the development of energy efficient technologies.  Develop the capacity to collect and manage energy data while being cognizant of the need to preserve the data and respect confidentiality.  Development of an energy information clearing-house, using information and communication technologies that will enable information to be easily accessible and available in a user-friendly format to relevant stakeholders.  Develop institutional capacity to implement demand-side energy management programmes  Ensure an adequate supply of energy efficient products, goods and services- energy management cadre, promotion of ESCOs, Standards and Labeling 37  Launch a National Demand-Side Management (DSM) Initiative Designed to Reduce Residential Energy Consumption Strategies related to Private Sector and Industry  Support and Assist in the Establishment of Energy Service Companies (ESCOs) that derive their income by generating energy savings for their clients.  Facilitate the development of a national approach to encourage companies to develop internal systems to assess and prioritize energy efficiency opportunities.  Support companies to identify and implement high energy saving energy efficiency opportunities including through whole of supply chain assessments and systems optimization of priority industrial technologies.  Promote best practice and innovation within energy-using corporations and the energy services sector (through case studies).  Develop programmes that would support industry to identify and implement high energy saving energy efficiency opportunities including through whole of supply chain assessments and systems optimization of priority industrial technologies.  Encourage greater energy efficiency and lower energy costs in the bauxite and alumina industry and the manufacturing sector  Facilitate sourcing of low cost development funds for productive enterprises for energy technology projects  Promote best practices in design of new production facilities and retro-fitting of existing facilities to maximize energy efficiency  Adopt Cleaner Production Mechanism (CPM) through promotion of incentives (Carbon Credits) and capital financing available  Promote the development and implementation of environmental management systems in the productive sectors (ISO 14001) Strategies related to Energy Generation  Maximize the potential for the application of co-generation and other distributed generation technologies that increase energy efficiency.  Encourage broader use of cogeneration output of energy by manufacturers  Encourage integrated energy industrial parks with cogeneration facilities  Implement demand-side management programmes relating to load control  Facilitate the introduction of energy-saving devices  Employ energy-saving approaches in building design and construction  Facilitate the introduction of energy-saving devices  Employ energy-saving approaches in building design and construction Key strategies and actions related to the enabling environment  Create relevant legislation to support required investments in energy efficiency  Provide incentives for the use of innovative/clean technologies in power generation, mining and manufacturing to improve energy efficiencies  Design and introduce appropriate financing mechanisms to facilitate the  spread of energy efficiency and renewable energy technologies  Development of a framework to capitalize on the opportunities offered by the carbon market, under the Clean Development Mechanism (CDM) for efficiency and conservation projects. 38  Accelerate and expand the current energy labeling program.  Establish a system to identify and replace old and inefficient units/plants with more fuel efficient and cost efficient technologies and plants  Retire the old generation plants and replace them with modern plants through a competitive basis to improve the conversion efficiency  Conduct periodic review and update of building code  Strengthen capacity of local authorities to enforce building code on an ongoing basis  Update, apply and enforce the Energy Efficiency Building Code to support efficient use of energy in buildings  Develop and implement demand side initiatives including general demand reductions from energy efficiency, peak load shifting, cost-reflective pricing, and measures to address asymmetry of information.  Develop and implement appropriate tax and pricing structure for road users that reflect environmental costs and other externalities  Review related policies for other sectors including transport, mining, agriculture, tourism and industrial policy, and make recommendations to harmonize with the national energy policy and sub-policies under the national energy policy  Provide incentives for the usage of renewable energy in productive sectors  Provide incentives where applicable to encourage employment of high levels of capital to increase use of cleaner technologies  Provide incentives/disincentives for the use of innovative/clean technologies in key energy-intensive sectors including mining and manufacturing to improve energy efficiencies Key strategies and actions related to Government energy conservation and efficiency  Ensure that Ministries and Agencies develop and implement environmental stewardship action plans, with special emphasis on energy and fleet management  Develop and implement a specific programme of energy management for the National Water Commission, the single largest consumer of energy in the public sector  Fast track the implementation of energy efficiency programmes (the recommendations of the energy audits undertaken) in hospitals and other areas of the public sector, based on the findings of various earlier studies and energy audits  Establish energy conservation and efficiency (ECE) protocols for the operation of public sector facilities and entities including the appointment of an energy coordinator for each facility  Expand the role of the Energy Efficiency Unit (EEU) within the Petroleum Corporation of Jamaica (PCJ) to provide technical assistance for ECE initiatives in the public and private sectors  Align energy conservation and efficiency initiatives with Government procurement requirements.  Promote and accelerate the use of energy efficient equipment (including information-communications technology, refrigerators, etc) in government operations, and investigate the adoption of mandatory energy efficiency requirements, taking into account life cycle costing.  Increase the energy efficiency of street lighting.  Collect and make available to street lighting service providers and local governments nation-wide information on energy efficient street lighting.  Provide information to street lighting service providers and local government authorities on energy efficient street lighting.  Develop specific energy conservation and efficiency programme for the National Water Commission to address issues such as pumping efficiency and distributed storage programmes to facilitate better management of pumping operations. 39 Overview of Government support for EE measures and products Although Government lost its direct control over the energy sector when it privatised the major supply company, this can be seen as an unavoidable consequence of a measure, which is part of a global trend towards Government retreat from implementing economic activities. It could be said that more contractual obligations could have been written into the sale agreement to protect DSM, but these would in any case have been difficult to enforce. Against this negative factor, as shown above, Government policies, strategies and planned actions show strong commitment to the areas of EE and DSM. Many of the policy elements can be seen as directly following paths opened by the JDSM project, which has therefore made an important contribution to establishing strong Government support in this area. Overall, this Intermediate State is assessed as well achieved. Overall Assessment of Strategy Three An overall assessment of Strategy Three is provided in Table 5 below. 40 Table 5: Outcomes to Impacts Assessment Findings for Strategy 3 Theory of Change Component Assessment Rating Outcome 3a: Public awareness Although effective campaigns were launched, the profile 2 of Climate Change and EE of these issues was not greatly raised, owing to more raised pressing public concerns. Outcome 3b: Youth awareness This component was strongly pursued and established 3 of CC and EE raised these issues as a significant concern among children and youth. Impact Driver 3: Government Government made substantial efforts, which were most 2 promotes energy efficiency effective in its own sector of control. There were insufficient incentives to generate much take up in the private sector. Impact Driver 3a: Education Government implemented or supported numerous 3 system promotes EE initiatives in this area, which promoted strong education sector resources for EE and more broadly CC. Assumption 1: Energy Efficient CFLs added value, but other products promoted did not 2 products and services add value. on any scale. Energy audits and services potentially added value, but few could afford the measures they proposed. Assumption 2: Government Government continues to exert control through policies, 2 exerts control over the energy strategies and financial measures. However, it lost its sector. main control when it privatised energy supply. Assumption 3: Public opinion Substantial efforts were made, resulting in a moderate 2 can be influenced in favour of shift in public opinion in favour of DSM/EE. DSM/EE. Intermediate State 3a: Increased Despite substantial awareness raising efforts, there is no 1 public demand for adoption of evidence of a major increase of public demand in favour EE measures and products of EE. Intermediate State 3b: Young There has been substantial take up of these issues 3 people promote EE throughout the education system, providing a firm platform for young people to promote EE. Intermediate State 3c: Although Government lost direct control of the energy 3 Government strengthens support sector through privatisation, it has strongly pursued the for EE measures and products development of policies in favour of EE and DSM. 41 6: Summary of Project ROtI Ratings The final consolidated rating of the project’s progress along the Theory of Change elements towards its intended Global Environment Benefit is shown in Table 6 below. Theory of Change Element Rating Strategy 1: Reduced Fossil Fuel Use Outcome 1a: Advantages of CFLs demonstrated 3 Outcome 1b: Advantages of Energy Efficiency demonstrated 1 Outcome 1c: Advantages of home solar PV systems demonstrated 1 Outcome 1d: Advantages of solar water heaters demonstrated 1 Impact Driver 1: Energy Efficient products and services competitively priced and marketed 2 Impact Driver 2: Fiscal measures promote EE products and services 2 Impact Driver 5: Media and public promote EE 3 Assumption 1: EE products and services will add value in the Jamaican situation 2 Assumption 2: Government exerts control over the energy sector 2 Assumption 3: Public opinion can be influenced in favour of DSM/EE 2 IS 1a: CFLs dominate the market 3 IS 1b: EE equipment and materials dominate the market and EE practices are the norm 1 IS 1c: Solar PV has an established niche 1 IS 1d Solar Water heaters have a major market share 1 Overall Assessment of Strategy 1: One Intermediate State of this strategy, transformation of the market for CFLs 2 so that they dominated the home lighting sector, was well (but not fully) achieved. The other ISs were poorly achieved. The Impact Drivers and Assumptions were mostly partially achieved. Overall, this Strategy is assessed as partially achieved. Strategy 2:Strengthen Institutional Capacity for Reduced Fossil Fuel Use Outcome 2a: Technical capacity in EE raised 3 Outcome 2b: Commercial/Industrial/Government sectors’ awareness raised 2 Outcome 2c: JPS and energy sector DSM/EE capacity for energy management strengthened 3 Outcome 3: Energy efficiency standards established 2 Impact Driver 3: Government promotes EE 2 Impact Driver 4: Effective use made of technical capacities raised by the project 1 Assumption 1: EE products and services will add value in the Jamaican situation 2 Assumption 2: Government exerts control over the energy sector 2 Assumption 3: Public opinion can be influenced in favour of DSM/EE 2 IS 2a: Demand for EE technical services raised, leading towards EE practices are the norm 1 IS 2b: Government and private sector interest sustained and increased, leading towards EE practices are the norm 1 IS 2c: Government prepares an Energy policy, leading towards Government adopts energy policy 3 IS 2d: JPS and energy sector interest sustained and increased 1 IS 3: EE standards applied, leading towards EE practices are the norm 1 Overall Assessment of Strategy 2: One IS of this Strategy, Government progress towards adoption of an Energy 2 Policy, was well achieved. The others were poorly achieved, largely because of the low absorption of EE concerns in sectors other than Government. Strategy 3: Strengthen Public Support for DSM/EE Measures Outcome 3a: Public awareness of CC and EE raised 2 Outcome 3b: Youth awareness of CC and EE raised 3 ID 3: Government promotes EE 2 ID 3a: Education system promotes EE 3 ID 5: Media and public promote EE 3 Assumption 1: EE products and services will add value in the Jamaican situation 2 Assumption 2: Government exerts control over the energy sector 2 Assumption 3: Public opinion can be influenced in favour of DSM/EE 2 IS 3a: Increased public demand for adoption of EE measures and products 2 IS 3b: Young people promote EE 3 IS 3c: Government strengthens support for EE measures and products 3 Overall Assessment of Strategy 3: Two of the three ISs for this Strategy were well achieved and the third was 2 partially achieved. The main reasons why public demand for EE measures and products did not rise significantly were because of high costs of such products as Solar PV and Solar Water Heaters. Overall Assessment of the Project: Ten years after project completion, the project has partially achieved its 2 intended progress towards the final impact. 42 7: Overall Assessment of the Project’s Progress Towards the Intended Global Environment Benefit Ten years after project completion, the project has partially achieved its intended progress towards impact and the expected Global Environment Benefit. After a post-completion period in which it appeared that the project’s contribution might fade away (apart from the market transformation for CFLs), as a result of the privatisation of the main national energy supplier, there was a return of interest focussed around Government’s development of an Energy Policy. This has a component sub-policy on Energy Conservation and Efficiency that explicitly refers to the DSM project as part of its antecedents. The issues around Climate Change, Energy Efficiency and Demand Side Management have also, over time, been well incorporated at all levels of the national educational system, including at University level, which is a regional facility. An additional aspect of the project’s contribution towards impacts concerns the regional dimension. As well as the development of related courses and research at the University of the West Indies, there have also been some “spin offs� in other countries. A senior officer of JPS, instrumental in project implementation, moved to the CARICOM Energy programme, which supported Energy Efficiency activities and Energy Policy development throughout the region. Training activities on EE were conducted at the Institute of Engineers in Trinidad. Although the results of this regional catalytic role cannot be specified, they form an additional dimension of the project’s contribution to its intended impacts. Furthermore, energy efficiency approaches have recently been supported by UNDP, both in terms of specific projects in the health sector and of support to the development of the Government’s Energy Policy. The Inter American Development Bank is also providing substantial support in this area. Thus it appears likely that the progress of DSM/EE will continue and even expand in future. The DSM Project has clearly made an important contribution towards these trends, although it would not be possible to say exactly how much contribution. Whereas the earlier World Bank-GEF Post-Implementation Impact Assessment, five years after Project Completion, focussed mainly on attempting to quantify the contribution of reduced GHG emissions (and reduced energy costs), the ROtI after ten years has found that a “second wave� of results, further along the Outcomes to Impacts pathways, has become evident. These results focus in the field of enhanced knowledge and expertise concerning Energy Efficiency/Demand Side Management promoted through all levels of the education system; and in the development of a detailed and coherent Government approach towards Energy and Climate Change, embodied in a new national Energy Policy, intended to focus national approaches until the year 2030. 43 Annex 1: References Energy Efficiency Campaign Evaluation, Market Research Services Ltd., for Petroleum Corporation of Jamaica, 2005. Energy Efficiency and Conservation Program Evaluation – Final Report, Petroleum Corporation of Jamaica, 2004. Implementation Completion Report on a Global Environmental Facility Grant for a Demand-Side Management Demonstration Project to the Government of Jamaica. World Bank Report No. 20245, 2000. Jamaica Demand Side Management Demonstration Project – World Bank GEF Post-Implementation Impact Assessment. World Bank, 2006. Jamaica’s National Energy Policy 2009 – 2030. Ministry of Energy and Mining, 2009. National Energy Conservation and Efficiency Policy. Ministry of Energy and Mining, 2010. Annex 2: Participants in the ROtI process ROtI Discussion Group Denise Tulloch, Senior Research Officer, Centre of Excellence for Renewable Energy, Petroleum Corporation of Jamaica. Hugh Harris, Ministry of Finance, Government of Jamaica. Raymond Wright, Consultant to Petroleum Corporation of Jamaica. Individual Interviews Althea Spence, World Bank Jamaica, Operations Analyst Chris Corbin, United Nations Environment Programme, Regional Coordination Unit, Programme Officer. Claon Rowe, Petroleum Corporation of Jamaica, Senior Project Engineer. Clifford Mahlung, Meteorological Service of Jamaica, Section Head, Data Processing, Climate Branch and Climate Change Focal Point. Earl Green, Petroleum Corporation of Jamaica, Group Technical Director. Fitzroy Vidal, Ministry of Energy and Mining, Senior Director, Energy Division. Jeffery Spooner, Meteorological Service of Jamaica, Climate Branch Head. Leonie Barnaby, Office of the Prime Minister, Environmental Management Division, Senior Director and GEF Focal Point. Nigel Logan, Petroleum Corporation of Jamaica, Acting Group Managing Director and Chief Finance Officer. Rohan Longmore, World Bank Jamaica, Economist Winsome Townsend, National Environment and Planning Agency, Senior Director, Strategic Planning. Yvonne Barrett-Edwards, Ministry of Energy and Mining, Director, Energy Division.