ICRR 12869 Report Number : ICRR12869 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 03/03/2008 PROJ ID : P058476 Appraisal Actual Project Name : Liepaja Region Solid US$M ): Project Costs (US$M): 16.8 15.3 Waste Management Project Country : Latvia Loan/ US$M): Loan /Credit (US$M): 2.2 2.2 Sector Board : ENV Cofinancing (US$M): US$M ): 8.0 8.2 Sector (s): Solid waste management (88%) Sub-national government administration (6%) Other industry (6%) Theme (s): Administrative and civil service reform (25% - P) Pollution management and environmental health (25% - P) Climate change (25% - P) Municipal governance and institution building (25% - P) L/C Number : L7033 Board Approval Date : 09/14/2000 Partners involved : European Closing Date : 06/30/2007 06/30/2007 Commission, Nordic Investment Bank, SIDA Evaluator : Panel Reviewer : Group Manager : Group : Roy Gilbert Ridley Nelson Soniya Carvalho IEGSG 2. Project Objectives and Components: a. Objectives: To demonstrate self-sustaining modern management of municipal solid waste through maximum collection and utilization of landfill gas (LFG) in the city and district of Liepaja . Other project objectives include : a) demonstrating modem sanitary landfill techniques on a regional basis; b) strengthening institutional capacity at the local /regional levels on issues related to municipal solid waste management; c) arresting on-going contamination of groundwater; d) reducing environmental disamenities for neighbors of existing disposals sites that would be closed; e) facilitating the separation of recyclable material; and f) reducing greenhouse gas emissions through an emission reduction agreement with the Prototype Carbon Fund (PCF). b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): A: Investment Component (appraisal cost US$8.9 million; actual cost US$13.5 million), incl: (i) remediation and closure of selected existing dump sites through separation of surface water coverage and /or re-vegetation; (ii) operational improvements such as reception station for separation /recycling, storage for recyclable /hazardous materials, leachate treatment plant, automated transport between cells, and sludge treatment equipment; (iii) shredding equipment, energy cell installation and landfill gas collection system; and (iv) power generators at Grobina (1 mw) and Skede (0.3 mw) using landfill gas; (v) waste collection points in each municipality in Liepaja Region . B: Technical Assistance Component (appraisal cost US$2.2 million; actual cost US$1.9 million), incl: (i) provision of training to waste management utilities in municipal governments, municipal authorities and private sector companies involved in the implementation of the National Solid Waste Management Strategy; ; (ii) capacity building of Liepjas RAS Ltd (L-RAS); and (iii) detailed designs for bidding documents . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Start up was delayed owing to considerable processing delays in obtaining EU funding for the Instrument for Structural Policies for pre-Accession (ISPA). The pace of implementation later picked up, however, so that the project was completed on time. Overall, actual costs were close to those estimated at appraisal, but there were considerable differences among sub-components. There was a nearly ten times cost overrun for the installation of energy cells and landfill gas collection system (actual costs of US$8.5 million, versus an appraisal estimate of US$ 0.88 million). On the other hand the project incurred much lower costs for the remediation of existing sites and for operational improvements (actual costs of US$3.6 million, versus an appraisal estimate of U$ 6.8 million). Local governments' contributed U$4.5 million counterpart (up from an appraisal estimate of US$3.9 million). 3. Relevance of Objectives & Design: Objectives were highly relevant to Latvia's need to meet EU levels of environmental management required for EU accession. They remain highly relevant to several priorities laid out in the latest (2002) CAS, including a focus upon development outside the capital Riga and making better services available throughout the country . Given Latvia's few natural resources and need to import all natural gas and oil products, the country's National Energy Strategy, supported by a new Energy Law, makes the diversification of energy sources --such as promised by using landfill gas--a high priority. Furthermore greenhouse gas emissions from inadequate landfill operations, that the project sought to correct, are an even more serious environmental concern today . 4. Achievement of Objectives (Efficacy): Overall objective--Modest efficacy . The operation did demonstrate self -sustaining modern management of municipal solid waste in Liepaja, but with shortfalls in the collection and utilization of landfill gas --central to the project design. a) demonstrating modem sanitary landfill techniques --modest efficacy . Modern features included installations to collect landfill gas and to both flare the gas and use it for electricity generation, but they were not used as intended . The intended (on-site) biological treatment of leachate was not implemented, at least until a reverse osmosis facility began to operate in 2006, near implementation completion. Landfill gas collection remains below projections --also exact data on volumes in not provided by the ICR --because the installed gas wells are too large and too few, according to the project team. Nevertheless, the project helped Latvia reach some of the environment standards required for EU accession. b) strengthening institutional capacity at the local /regional levels on issues related to municipal solid waste management--modest efficacy . Although project TA for strategy did not go ahead as planned, the project gave L-RAS a management information system. The ICR describes L-RAS' financial performance as "solid", even without the additional revenues expected from the sale of electricity that would have been generated by landfill gas had more been collected. Elsewhere (p. 16), the ICR reports L-RAS as loss making, but likely to achieve positive cash flows in the future, if the landfill gas production problem is resolved . L-RAS also now has well trained staff on its payroll and regularly exchanges experiences and information with other agencies . Also on the institutional side, the country's Energy Law was amended allowing higher sales price for landfill gas . c) arresting the on-going contamination of groundwater --substantial efficacy . Closure of insanitary dumpsites helped achieve this. The ICR reports that L-RAS voluntarily financed some laboratory analyzes of ground water quality, but does not provide the baseline and outcome measurements obtained . d) reducing environmental disamenities for neighbors of existing disposals sites that would be closed --substantial . The project helped close and remediate (i.e. rehabilitate) 26 insanitary dumpsites, although the ICR does not report their scale. e) facilitating the separation of recyclable material modest efficacy . The ICR reports that good results have been achieved, but only at the Grobina site . f) reducing greenhouse gas emissions through an emission reduction agreement with the Prototype Carbon Fund (PCF)--negligible efficacy . Not achieved, because landfill gas was not collected in the amount expected . The methane content of gas that is flared produces carbon dioxide, another greenhouse gas . 5. Efficiency (not applicable to DPLs): When international demand (via carbon finance) for environmental improvements resulting from the project are computed, high ERRs reported below point to this being a worthwhile operation . Taking into account only local demand, estimated ERRs are lower: 10.1% at appraisal and 8.9% at completion. In reviewing these results, it should be recognized that this kind of project involving solid waste disposal rarely leads to a high ERR based upon local demand alone. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal Yes 21.2% 86% ICR estimate Yes 20.1% 96% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The project was successful in achieving its highly relevant objectives, albeit with moderate shortcomings, especially in the collection and use of landfill gas that was at the heart of the design . a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: IEG considers that the likelihood that project achievements thus far will be undermined in the future is relatively small. The ICR reports that L-RAS is a strong agency, tied into long term participation agreements that will ensure its ongoing contributions to solid waste operations in its region . IEG's "moderate" risk rating is based upon this project-wide assessment. The ICR's "high" risk rating appears to be based more narrowly upon the likelihood of achieving the agreed 2012 targets of the Emission Reduction Purchase Agreement (ERPA) only a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: Project preparation was very careful . It required, for instance, up-front participation agreements to be signed by signed by all municipalities. It also included the evaluation of five different locations for the regional waste management facility. Bank team took advantage to apply what they had learned from the earlier Riga project, especially the lesson of the importance of achieving political consensus around solid waste projects . Supervision went smoothly. at -Entry :Highly Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Satisfactory 9. Assessment of Borrower Performance: Government quickly set up the necessary arrangements for project implementation, setting up working groups as needed and coordinated across interested ministries . As implementing agency, L-RAS promptly took the necessary actions to ensure project implementation within the time foreseen at appraisal, handled procurement efficiently and enjoyed excellent collaboration by municipalities . a. Government Performance :Satisfactory b. Implementing Agency Performance :Satisfactory c. Overall Borrower Performance :Satisfactory 10. M&E Design, Implementation, & Utilization: In the initial M&E design, six relevant sets of indicators were identified to monitor the achievement of outcomes . They were: (i) efficiency of waste management; (ii) local environmental pollution; (iii) financial sustainability; (iv) local government capacity; (v) landfill gas capture and energy production; and (vi) greenhouse gas abatement. Shortly after start up, the M&E design was revised to focus upon just three sets of indicators : (a) reduction of greenhouse gas emissions; (b) reduced environmental disamenities for neighbors through closure of dumpsites; and (c) facilitating the separation of recyclable material . This narrower scope of the M&E scope left out some important project objectives, however, such as institutional strengthening and the avoidance of groundwater, pollution without the cover of performance indicators . Nevertheless, the revised M&E was effectively implemented since fewer indicators simplified reporting, and could be compared with baseline measures taken at the outset . The monitoring of key aspects of the solid waste management system, such as amounts of waste delivered for disposal, ground water quality at disposal sites, and leachate amounts, was systematically followed . M&E was well utilized. It was M&E that drew attention to the declining production of landfill gas, for example, leading to remedial actions to try to improve output. a. M&E Quality Rating : Modest 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately Shortfalls of landfill gas collection and Satisfactory use--central to the design--are moderate shortcomings in an otherwise successful project. Risk to Development High Moderate IEG's rating considers the risk faced by Outcome : project achievements as a whole . The ICR rating appears to focus only upon the ERPA. Bank Performance : Highly Satisfactory Satisfactory Preparation and design work was highly satisfactory. Supervision appears to have been good, but the ICR does not give any clear reason (with evidence) why it should be considered as better than fully satisfactory. Borrower Performance : Highly Satisfactory Satisfactory Again, both government and implementing agency performed well, but the ICR provides little convincing evidence of highly satisfactory performance by them that would warrant the top rating. Furthermore, the ICR faults the placement of insufficient gas wells for the shortfall in landfill gas production, implying a shortcoming on the Borrower's side. Quality of ICR : Satisfactory NOTES NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: An external stimulus, such as EU accession and related harmonization of local legislation to EU standards, requires extra effort by a project (beyond the achievement of its formal objectives ) and may impact project results. A project implementation unit (PIU) should be integrated into the management structure of the ongoing project rather than operate as a separate unit . In this way its competence will be retained within the project and not lost at the end of the contract period . A complex business plan with long term company strategies should only be applied to well established companies. It can prove too complicated for a new start -up operator like L-RAS to take on. Getting project ideas across requires good communications and knowledge of the languages used by project stakeholders. Intensive language training, in English for local managers and in the local language for Bank staff can help. 14. Assessment Recommended? Yes No Why? To learn of post-ICR progress in finding solutions to the landfill gas shortfalls that affected this project (a problem that has also been encountered by similar operations in other countries ). 15. Comments on Quality of ICR: On the whole, the ICR provides a good account of the project experience and results, although there are some shortcomings. The ICR's analysis of the financial performance of the project and of L -RAS is not always easy to follow as the evidence presented is not always consistent with the reported findings . This is the case, for example, of the "solid" financial position reported of a, nevertheless, loss making L -RAS. It is also the case of the (optimistic) assumption that L-RAS will find a solution to the landfill gas problem in the future, even though, despite intensive best efforts, none has been found thus far . The ICR would have been more readable if acronym use had been restricted to the names of agencies, programs and technical expressions . Although referred to, some project data is not provided in the report. Thus the ICR does not provide evaluation ratings during implementation, even though the text refers to "realistically rated achievements " (p. 19). a.Quality of ICR Rating : Satisfactory