Document of The World Bank Report No:ICR-000039 IMPLEMENTATION COMPLETION AND RESULTS REPORT ( IDA-27510 IDA-27511 ) ON A LOAN / CREDIT IN THE AMOUNT OF US$ MILLION 65.99 (SDR 48.47 MILLION CREDIT) TO THE REPUBLIC OF AZERBAIJAN FOR GREATER BAKU WATER SUPPLY REHABILITATION PROJECT August 30, 2006 Sustainable Development Department South Caucasus Country Unit EUROPE AND CENTRAL ASIA REGION CURRENCY EQUIVALENTS (Exchange Rate Effective 04/18/2006) Currency Unit = Manat Manat 1.00 = US$ 1.09325 US$ 1.00 = Manat 0.9147 Fiscal Year January 1 to December 31 ABBREVIATIONS AND ACRONYMS ARWC Apsheron Regional Water Company CAS Country Assistance Strategy CDP Corporate Development Partner CDU Corporate Development Unit DPL Development Policy Loan EBRD European Bank for Reconstruction and Development IBRD International Bank for Reconstruction and Development IBTA Institution Building Technical Assistance ICR Implementation Completion Report IDA International Development Association IDP Internally Displaced Persons MDP Medium Term Development Program O&M Operations and Maintenance PPF Project Preparation Facility SAC Structural Adjustment Credit SAR Staff Appraisal Report SECO State Secretariat for Economic Affairs of the Government of Switzerland UFW Unaccounted for water WHO World Health Organization WSS Water Supply and Sanitation Vice President: Shigeo Katsu Country Director: D-M Dowsett-Coirolo Sector Manager: Sumter Lee Travers Project Team Leader: Andreas Rohde REPUBLIC OF AZERBAIJAN GREATER BAKU WATER SUPPLY REHABILITATION PROJECT CONTENTS Page No. 1. Basic Information 1 2. Key Dates 1 3. Ratings Summary 1 4. Sector and Theme Codes 2 5. Bank Staff 2 6. Project Context, Development Objectives and Design 2 7. Key Factors Affecting Implementation and Outcomes 9 8. Assessment of Outcomes 17 9. Assessment of Risk to Development Outcome 23 10. Assessment of Bank and Borrower Performance 23 11. Lessons Learned 26 12. Comments on Issues Raised by Borrower/Implementing Agencies/Partners 27 Annex 1. Results Framework Analysis 30 Annex 2: Restructuring 33 Annex 3. Project Costs and Financing 34 Annex 4. Outputs by Component 36 Annex 5. Economic and Financial Analysis 39 Annex 6. Bank Lending and Implementation Support/Supervision Processes 40 Annex 7. Detailed Ratings of Bank and Borrower Performance 43 Annex 8. Beneficiary Survey Result 43 Annex 9. Stakeholder Workshop Report and Results 43 Annex 10. Summary of Borrower's ICR and/or Comments on Draft ICR 44 Annex 11. Comments of Cofinanciers and Other Partners/Stakeholders 46 Annex 12. List of Supporting Documents 47 1. Basic Information GREATER BAKU Country: WATER SUPPLY Azerbaijan Project Name: REHABILITATION PROJECT Project ID: P008288 L/C/TF Number(s): IDA-27510,IDA-27511 ICR Date: 07/26/2006 ICR Type: Core ICR Lending Instrument: GOVT OF SIL Borrower: AZERBAIJAN Original Total Commitment: XDR 38.8M Disbursed Amount: XDR 48.5M Environmental Category:B Implementing Agencies AzerSu Joint-Stock Company Cofinanciers and Other External Partners European Bank for Reconstruction and Development (EBRD) Swiss Agency for Development Cooperation 2. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 03/09/1994 Effectiveness: 02/14/1996 02/14/1996 Appraisal: 02/08/1995 Restructuring(s): 12/10/2002 Approval: 06/28/1995 Mid-term Review: 12/01/1998 Closing: 12/31/2001 01/31/2006 3. Ratings Summary 3.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory 3.2 Quality at Entry and Implementation Performance Indicators Implementation Performance Indicators QAG Assessments (if any) Rating: Potential Problem Project at any No Quality at Entry (QEA): None time (Yes/No): Problem Project at any time Yes Quality of Supervision (QSA): None 1 (Yes/No): DO rating before Closing/Inactive Satisfactory status: 4. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Water supply 94 94 Sewerage 6 6 Original Priority Actual Priority Theme Code (Primary/Secondary) Access to urban services and housing Primary Primary 5. Bank Staff Positions At ICR At Approval Vice President: Shigeo Katsu Johannes F. Linn D-M Dowsett- Country Director: Yukon Huang Coirolo Sector Manager: Sumter Lee Travers Jonathan C. Brown Project Team Leader: Andreas Rohde Robert Wildeman ICR Team Leader: Andreas Rohde ICR Primary Author: Sanyu Lutalo 6. Project Context, Development Objectives and Design (this section is descriptive, taken from other documents, e.g., PAD/ISR, not evaluative): 6.1 Context at Appraisal (brief summary of country and sector backgrounds, rationale for Bank assistance): Country Background: The Republic of Azerbaijan is located on the western coast of the Caspian Sea. It covers an area of about 86,000 square kilometers, and is endowed with various natural resources including agricultural resources and minerals such as oil, which accounts for almost one third of GDP. The country is also endowed with a diversified industrial structure, a relatively educated labor force, and rapidly expanding private sector. Baku, the capital city, is located on the southern side of the narrow Apsheron Peninsula along the coast of the Caspian Sea. The Greater Baku area, which covers the city and its environs is not just the political capital, but also the economic, industrial and cultural center of Azerbaijan. By 1993 more than two thirds of the urban population, or about 2.5 million people, were concentrated in this area. Immediately after independence from the FSU in 1991 the country underwent a period of dramatic economic decline, and by 1993 Gross Domestic Product (GDP) had dropped to about 60 percent of its 1988 level. The Government initiated reform measures to liberalize the economy and ensure transition to a market economy. 2 Sectoral Background: Azerbaijan inherited from the Former Soviet Union (FSU) a highly centralized top-down system of managing utility services, including water supply and sewerage. Sector management was also highly fragmented at the central level, and at the time of project appraisal, responsibility for the sector was divided between five different agencies and ministries. On a day to day basis, water supply services were run by the Baku Water Agency (BWA), a centrally managed Government department with little or no autonomy, which was responsible for production and bulk supply of water to towns and villages along the transmission lines. A subsidiary to BWA, the Baku Water Department (BWD) was responsible for water distribution and operation of water services in the city of Baku. The high level of management fragmentation impeded the efficient development of integrated planning, programming, and budgeting initiatives for the sector, and resulted in a piecemeal response to sectoral needs. Sectoral investments focused on the installation of large scale physical infrastructure facilities, which were not integrated into an adequate asset maintenance regime at the planning and implementation stages, resulting in serious deterioration in their condition over time. Thus although as a result of the Soviet legacy about 95 percent of households were connected to the piped water network in Baku, the quality of services had declined significantly at the time the project was appraised. A Social Assessment carried out during project preparation revealed that virtually 100 percent of households in all neighborhoods and income groups had unreliable, intermittent, and insufficient water service. On average water was available to individual families for only 22 days per month, for a maximum of 11 hours a day. In some areas households were limited to less than 6 hours of piped water a day, and domestic water rationing was not uncommon. The poor quality of services was exacerbated by the lack of equipment, tools and spare parts, and the poor organizational and operating methods that were in place at the time. The water treatment plants serving the city were in serious need of repair and rehabilitation, and the distribution pipe network was badly corroded and in poor structural condition. In 1993 alone, almost 3500 leaks and bursts were attended to on the 2500 km long network in the city, and unaccounted for water was estimated at between 65-70 percent. Sewerage service levels were even lower than those for water supply, with only about a third of the population having access to a sewerage connection. Existing facilities were also in poor condition and most sewage was discharged untreated into the environment. Rationale for Bank Involvement: During a reconnaissance mission in January 1994, the Government requested the Bank to consider providing a credit to finance a water supply and sewerage project that would address the most urgent needs in the sector. Additional financing was mobilized from the EBRD. The Project was the Bank's first infrastructure operation in Azerbaijan. In view of the magnitude of resources that would be required to address the investment needs as a single lending operation, and the weak performance capacity of the designated institutions given their unfamiliarity with Bank operations, it was agreed that a longer term approach divided into two or more phases would be taken. It was also agreed that the first phase would focus on the most urgent needs in the production and distribution of potable water through major institutional reforms. 3 The project was consistent with the World Bank Country Assistance Strategy (CAS), which was discussed by the Board on April 20, 1995. The CAS aimed to support Azerbaijan in its transition to a market economy, and its main focus included support for economic transformation through assistance for institutional strengthening and related financial support to the economy. At the sectoral level, the CAS expected to focus on projects in the energy, agricultural and social infrastructure sectors. The project was also in line with the Government Medium-term Development Program (MDP). Under the MDP the Government sought to channel its increasing revenues from the petroleum and agricultural sectors towards expenditures and investments that would: (i) contribute to sustainable and broad based development; (ii) strengthen its transition to a market economy; and (iii) provide adequate infrastructure. 6.2 Original Project Development Objectives (PDO) and Key Indicators (as approved): The primary objectives of the Project were: (a) to make emergency short term improvements in the water supply system to restore the water supply to Baku, in particular to the poorer elements of the population; (b) to improve the water supply system as a whole; and (c) to provide the basis for longer term planning and recovery. Although the Government later requested IDA assistance through a Supplemental Credit to the ongoing credit to permit financing of the cost of repair and reconstruction of Apsheron Regional Water Company (ARWC) facilities damaged by the November 25, 2000 earthquake, the objectives remained unchanged. The performance indicators with respective to the above objectives were: (i) Improvement in hours of service from the pre-project average of 6 hours/day to an average of 12 hours/day by the end of the project; (ii) Reduction in unaccounted for water (UFW) from the pre-project level of 66 percent to less than 45 percent by the end of the project; (iii) Improvement in water quality and water pressure by the end of the project; (iv) Adjustment of residential and industrial water tariffs as necessary to cover operating costs; (v) Change in the billing system from flat to metered rate; (vi) Reduction in the ratio of employees per thousand service connection or households; and (vii) Reduction in the Working Ratio from the pre-project level of 0. 75 to less than 0.60 by the end of the project. 6.3 Revised PDO and Key Indicators (as approved by original approving authority), and reasons/justification: The project objectives were not revised. 4 6.4 Main Beneficiaries, original and revised (briefly describe the "the primary target group" identified in the PAD and as captured in the PDO, as well as any other individuals and organizations expected to benefit from the project): The main beneficiaries of the project were the population of the Greater Baku area, including the poor, who would all receive improved water supply services as a result of the improvements made under the operation. This would result in significant benefits in terms of convenience and savings on the costs of coping measures such as investment in storage tanks and water purification measures to deal with the poor services. The Social Assessment carried out during appraisal revealed that households in the Greater Baku area spent about 17 times more on coping measures for the poor quality of water supply services that they were receiving at the time than on the cost of their monthly water bills. The poor, who spent an average of about 7 percent of their income on water compared to the non poor who spent an average of 2 percent, were particularly vulnerable. Other beneficiaries of the improved services from the operation included the businesses, commercial enterprises, and industry that were dependent on the Greater Baku water supply system. 6.5 Original Components (as approved): The project comprised five major components: (A) Water Demand Management; (B) Operations and Maintenance Improvements; (C) Supply Improvements; (D) Institutional Capacity Building; and (E) Studies. A: Water Demand Management (US$ 6.7 million or 7.1 percent of project costs). This component focused on improving demand management of water at the consumer level, and was designed to reduce the losses and wastage at the household level, which were largely caused by the lack of awareness and/or lack of incentives to conserve water. It was divided into three sub-components: (i) a metering and billing sub-component which comprised implementation of a metering program through which about 15,000 domestic water meters would be installed. The billing system was also to be gradually transferred from flat rate to metered billing; (ii) a consumer education sub- component which comprised implementation of a consumer awareness program aimed at reducing water losses and optimizing water use; and (iii) a household leakage sub-component which comprised reduction of household leakage through installation of about 180,000 toilet cisterns. B: Operation and Maintenance Improvements (US$ 28.7 million or 30.3 percent of project costs). This component was designed to provide the utility with proper equipment and materials to improve the effectiveness of pipe main repairs which were extremely frequent in Baku at the time (an average of about 3500 repairs were made every year). It comprised procurement of materials and equipment to strengthen ARWC's operations and maintenance capacity, and included: (i) provisions for vehicles, workshops and equipment; and (ii) procurement of at least one year's supply of pipes, valves, and fittings. Provision of these supplies and equipment were expected to facilitate improvements in ARWC's O&M capacity, thus improving the water supply system as a whole. 5 C: Supply Improvements (US$ 39.5 million or 41.7 percent of project costs). This component was designed to improve the reliability and quality of water to consumers by providing a sustained supply of water from the treatment works to the end user. It comprised three main activities: (i) rehabilitation of the Jeiranbatan and Kura river water treatment works; (ii) rehabilitation of pumping stations; and (iii) supply and installation of master meters. These activities were together expected to contribute significantly to improving the quality and reliability of water supplied to the city. D: Institutional Capacity Building (US$ 8.3 million or 8.7 percent of project costs). This component involved the introduction of a reform strategy for ARWC to enable them to operate the water supply system in an efficient manner providing acceptable minimum levels of service. It comprised two main activities: (i) technical assistance provided through a twinning arrangement with an international water Operator experienced in the sector; and (ii) office consolidation, including provision of relevant equipment. These activities sought to address the key institutional constraints that the water utility was facing at the time, which included: a corporate culture lacking management and financial autonomy; fragmentation of operations with administrative, financial and operational units dispersed widely across the city; major operational constraints due to lack of tools and equipment, and inadequate organizational and operating methods; limited budgeting and planning capacity; and lack of transparency in the accounting and internal control system. E: Studies and Construction Supervision (US$ 6.3 million or 6.6 percent of project costs). This component comprised activities geared towards longer term planning and development of the sector that would form the basis for a second phase intervention for Greater Baku, including: (i) preparation and implementation of a National Policy Framework; (ii) preparation of a regional Water and Sewerage Masterplan for the Apsheron Peninsula, which would address all current sector issues by taking into account possible future developments and options looking at a 20-year design horizon; and (iii) consultant services for construction supervision. In addition to the five major components outlined above, the project cost estimates included: F: Interest during construction (US$ 4.4 million or 4.6 percent of project costs); and G: Repayment of the Project Preparation Facility (PPF) Advance (US$1.0 million or 1.0 percent of project costs). Overall, the project components were in line with the Government's development priorities at the time, and the scope of activities with respect to each one of them was adequate and appropriate for meeting the intended objectives. The design of the investment components combined activities geared at system optimization in order to improve efficiency, with activities geared at improving demand management, a lesson taken from other Bank operations. The anticipated outcome of that effort, substantially increased loans of service and improved water quality, responded to an acutely felt need of the population. The institutional component was designed to improve ARWC's operational efficiency and sustainability by ensuring that the company was restructured and: a) 6 became a market oriented joint stock company; b) established a twinning arrangement with a modern market oriented agency from a developed country; and c) consolidated its operations under one management structure in one location. The above steps, combined with other measures such as improvements in accounting, billing and collection, financial management, information management and operations management, aimed at strengthening ARWC's management systems, would in principle set the stage for achieving the desired efficiency improvements. The scope of Studies included in the project was quite comprehensive, covering not only physical investment planning, but also seeking to address key policy reform issues, that would together provide a basis for longer term planning and recovery of the sector. The capacity to translate policy recommendations from the relevant studies into actions that would achieve intended outcomes was however beyond the project's jurisdiction, and was subject to other factors such as political commitment and Government support for the recommendations. Similarly achievement of the intended reform outcomes was also subject to the presence of several conditions including a supportive political environment, a readiness for change of the implementing agency, and availability of adequate capacity at various levels. Policy dialogue carried out by the Bank under a series of Development Policy Loans (DPLs) in parallel to the project, included measures on increasing tariffs, improving the financial viability of Azersu, which contributed to the positive steps achieved in this regard. 6.6 Revised Components: The original project components were not revised. Their scope was however expanded as a result of the Supplemental Credit in the amount of SDR 10.3 million (US$ 12.92 million) approved by the Bank's Board in November 2002 to finance emergency works aimed at repairing damage done to water supply facilities by an earthquake that occurred on November 25, 2000. The original credit funds including all contingent funds were already fully committed by the time the earthquake occurred. The main activities financed under the Supplemental Credit were: (i) Repair of components of the Kura and Jeiranbatan water treatment facilities and an operational warehouse; (ii) Repair of sections of the water distribution network and related structures; and (iii) Consulting services for construction supervision and project implementation. Project implementation arrangements, which were being carried out effectively through the CDU, remained under the Supplemental Credit. The breakdown of these activities in terms of the project components was as follows: (a) Operation and Maintenance Improvements (US$ 7.5 million or 58 percent of the credit). This included: provision of earth-moving equipment, bulldozers and tractors; procurement of pipes, valves and fittings; and provision of warehouse facilities. (b) Supply Improvements (US$ 4.63 million or 35.8 percent of the credit). This included: rehabilitation of Jeiranbatan and Kura river water treatment works including chlorination building; rehabilitation of pumping stations; supply and installation of master meters; and rehabilitation of water distribution networks. 7 (c) Institutional Capacity Building (US$ 0.25 million or 1.9 percent of the credit): This basically comprised provision of office equipment. (d) Studies and Construction Supervision (US$ 0.4 million or 3.1 percent of the credit): This comprised consultancy services for construction supervision. 6.7 Other significant changes (in design, scope and scale, implementation arrangements and schedule, and funding allocations): The main changes in project scope and schedule, and funding allocations were directly related to implementation of the Supplemental Credit. As a direct result the project costs increased by about US$ 12 million, and the implementation timeframe was extended by more than three years, including extensions made to accommodate delays in achieving effectiveness. In addition to these direct changes related to the Supplemental Credit, a few other project activities were cancelled or scaled down due to higher than anticipated costs for the rehabilitation of the water treatment plants and the depreciation of the SDR against the US dollar. The actual costs of the works for the water treatment plants amounted to US$26.3 million instead of the originally estimated cost of US$21.9 million (an increase of about 20 percent). Activities that were cancelled included the development of a Management Information System (MIS) and the Household Leakage Reduction Sub-component which was to have involved installation of about 180, 000 toilet cisterns. There were no major changes in implementation arrangements over the course of the project. 8 7. Key Factors Affecting Implementation and Outcomes 7.1 Project Preparation, Design and Quality at Entry (including whether lessons of earlier operations were taken into account, risks and their mitigations identified, and adequacy of participatory processes, as applicable): a) Soundness of the background analysis: Background analysis carried out by the Bank during project identification and preparation was sound. In this regard the project was prepared with due diligence based on a sound analysis of the issues facing the sector at the time. Relevant economic, financial, technical, and institutional factors were taken into consideration in the design of specific components. The Bank's Safeguard Policies were also taken into consideration, and environmental studies were carried out consistent with the Bank and the Government's procedures. Although the Bank had no implementation experience in Azerbaijan, relevant lessons were identified and design of the project was informed by lessons from similar operations in other countries. The design of components was thus informed by several lessons from similar operations. From a supply perspective, the key lesson was the need for paying rigorous attention to water conservation and system optimization so that households can be provided with a particular level of service at the lowest possible cost. Difficulties in achieving institutional and financial viability through project design were also recognized from experiences elsewhere, from which it was noted that full commitment from the Borrower and substantial resources for Technical Assistance and training were essential to achieving improved outcomes in these respects. The choice to focus on water supply alone given the resource limitations also helped to concentrate the available resources on activities that would be able to have a demonstrable impact on development outcomes. b) Assessment of the Project's design: The project's design was generally sound. Project objectives, as stated in the SAR, focused on addressing the most critical challenges facing the water sector in the Greater Baku area at the time, which can be summarized as: (i) inefficiencies in the production and distribution of potable water. On average water was only supplied 6 hours a day with some areas receiving even less, and low water quality was a severe health risk for the population; and (ii) weak financial and performance capacity of the institutions responsible for managing service delivery. They were responsive to the Borrower's development priority of improving water supply service delivery in Greater Baku, and consistent with the Bank's Country Assistance Strategy for Azerbaijan. In prioritizing implementation of investments to meet the most urgent needs while planning for longer term recovery of the sector, the objectives supported the long term phased development strategy taken by the Government and the co-financiers. While the decision to focus on water supply alone without sewerage is not ideal, it was justified in this case given the limited funding available and in view of the significant needs normally involved in sewerage operations. The sewerage aspects were thus included in the longer term planning. During project preparation the team documented the relatively greater burden of coping costs borne by the poorer elements of the population. As in other post-Soviet cities, Baku did not have a settlement pattern segregating the poor, rather they were inter-mixed with wealthier elements of the 9 population. The PDO assertion that the poor would particularly benefit from the project was based on the logic of relative coping costs, not on targeted interventions for the poor. While logically sound, the project lacks a baseline or poverty based monitoring mechanism that would allow evaluation of this assertion. The project components were generally in line with the objectives, and most of the activities with respect to each one of them were realistic and appropriate for achieving the desired development outcomes. Physical investments which were mostly of a rehabilitative nature were generally not complex in design. However, the demand management component included only the installation of 15,000 water meters, sufficient to pilot a program but much below what is needed to make an impact. The institutional component which was designed to improve ARWC's efficiency and sustainability of operations by ensuring that the company was restructured and became a market oriented joint stock company, as well as involve other activities such as improvements in accounting, billing and collection, financial management, information management and operations management, aimed at strengthening ARWC's management systems, was in principle adequate to set the stage for achieving the desired efficiency improvements. Actual achievement of the intended reform outcomes was however subject to several risks and dependent on the presence of a set of conditions including a supportive political environment, a readiness for change of the implementing agency, and availability of adequate capacity at various levels. One risk that was not foreseen during the project design was the transformation of ARWC into Azersu. By expanding the service area to include most of the rayons in the country, an additional financial and operational burden was put on Azersu and makes the evaluation of the changes in performance of the original company difficult. In hindsight, the reading of the political environment and readiness of the Government to implement consistent tariff increases and enforce tariff collection was over-optimistic. Similarly, although studies designed to provide a basis for longer term planning and recovery of the sector were included, the Government moved much more slowly than expected to translate the policy recommendations from the relevant studies into actions that would achieve intended outcomes The Bank expanded policy dialogue on these issues beyond the project itself through DPLs including the Second Structural Adjustment Credit (SAC) II and the Second Institution Building Technical Assistance Credit (IBTA II), which included measures addressing key issues such as the need for Government support in improving utility's such as ARWC/Azersu's financial situation, and development of social assistance programs to alleviate the impact of utility sector reform on the poor, respectively. The level of cost recovery expected under the project was high, because at the time of project design it was not foreseeable that Government would be able to provide significant investment funds. The project team did not anticipate that oil revenues would increase to a level which allows significant Government contribution to investment and therefore provides for sustainability at a lower cost recovery level. c) Adequacy of Government commitment: Government's overall commitment and support to the project was a key factor in ensuring that it was generally implemented in a satisfactory manner despite the existing challenges. The following section (7.2) however highlights those aspects subject to Government control that in some way hindered or failed to enhance implementation progress. 10 d) Assessment of risks: Potential risks to the project were identified during appraisal. In this regard, inherent risks expected for a first time investment operation for a new Bank borrower, such as: (i) delays in implementation due to limited experience of ARWC with this type of activity; (ii) reluctance of the Government to face up to the challenges of required reforms; (iii) under-estimation of construction costs, and of the effort and commitment required to achieve financial sustainability; and (iv) unrealistic expectations that the water supply crisis could be resolved in a short period of time; were identified up-front. Measures to mitigate the risks, such as establishment of the twinning arrangement to support implementation, adequate supervision from the Bank, application of appropriate physical and price contingencies, public education, and continuing dialogue between the Bank and Borrower, were identified and incorporated in project design. While it would seem in that these measures would in principle address most of the risks, their effectiveness was subject to additional factors such as political will to make required changes, and the level of engagement and effectiveness of the dialogue. As a result they were not able to adequately mitigate some of the risks such as the ability to achieve financial outcomes through adequate tariff adjustments. In response, the Bank broadened the dialogue through the SAC II DPL (May 2002 to December 2003) to advocate required reforms. The conditions for withdrawals from this Credit after the aggregate of the proceeds withdrawn from the Credit Account exceeded the stipulated SDR 24.2 million, included satisfactory progress of the Borrower with respect to the following: (i) implicit subsidies in the water sector reduced by at least 30%, through a combination of better cash collection, tariff adjustments and/or increased efficiency; (ii) budgetary institutions current with utility payments for at least six months; (iii) cumulative payments to Azerenergy by ARWC equal 20 percent of the cost of the electricity supplied to ARWC in 2002; (iv) six months compliance with Decree prohibiting water bill payment with netting operations involving budgetary institutions and/or state-owned enterprises; (v) approval of draft regulations to ensure adequate private sector participation in the water sector; and (vi) appropriate economic prices for electricity, gas, water, wastewater and solid waste, approve and a published time-bound plan to move tariffs to economic prices, and implementation of appropriate country-wide tariff changes in 2002. Implicit subsidies in the water sector were reduced by 23 percent in 2002. However, losses continued and the 2005 tariff increase, while sufficient if matched with a high collection rate, resulted in a drop in collections, hence no immediate improvement in revenue. 7.2 Implementation (including any project changes/restructuring, mid-term review, Project at Risk status, and actions taken, as applicable): (a) Effects of the Earthquake: Project implementation was affected by the earthquake that occurred in Baku on November 25, 2000. It caused major damage to various water supply and wastewater facilities in the city, including water treatment structures and sections of the distribution network. As a result, some achievements such as reductions in water losses were set back significantly due to a sharp escalation in the occurrence of bursts in the distribution network. The Supplemental Credit was approved by the Bank primarily to finance major repairs and rehabilitation to the most damaged facilities. The Supplemental Credit was negotiated in May 2002 and approved by the Bank's Board 11 in December 2002. However it did not become effective until October 2003 - almost a year and a half after negotiations were held and ten months after approval. At negotiations, the project completion date was expected to be at the end of June 2004, with the credit closing at the end of December the same year. However, another eighteen months of the planned two-year project had elapsed before any disbursements were possible. The combination of these delays cost the project significant time. While implementation progress picked up considerably in 2004 and 2005, the closing date had to be rescheduled to January 31, 2006, to recover time lost in delayed effectiveness and allow for completion of the works. In the end the project implementation period was ten years, double the original implementation period of five years. (b) Macroeconomic Environment: During the course of project implementation, Azerbaijan's economy went through some difficult suffering from the economic crisis of 1998 and relatively low oil prices, during a period when the country was still making the adjustment from a command economy to a market economy. Related exchange rate fluctuations, Government budget stress and the earthquake of 2000 contributed to difficulties in project implementation. It was at this point that the Management Information System and Household Leakage Reduction sub-component were cancelled. The upswing in oil prices in recent years, coupled with increasing Azeri production, have had the opposite effect. Growing budget resources and personal incomes convinced Government to increase transfers to the water sector, and to allow substantial tariff increases. Government has now turned its attention to environmental problems and has declared its commitment to implementing wastewater collection and treatment plans for the Apsheron Penninsula, offering the prospect of a delayed pay-off to the earlier planning effort under this project. (c) Limited Experience with Bank Operations: Being the first Bank investment operation in the water sector in Azerbaijan, the Borrower and implementing agency lacked adequate implementation experience with Bank operations. Although the project design provided for technical assistance through the CDU to support the Borrower in several aspects of implementation, the Borrower's decision-making process was sometimes slow. ARWC, the Implementing Agency was newly created and still undergoing a learning process. The challenges of managing the institutional transitions during the restructuring exercises while implementing the project concurrently, had some impact on implementation progress, particularly due to factors such as time lost in finding and appointing suitable candidates to manage key positions in the utility, and building their capacity for decision-making. (d) Policy Oversight over the Water Utility: Policy oversight over ARWC, and later Azersu, was not strong enough to push through some of the required reforms in a timely manner. The risk of Government being reluctant to face up to the challenges of implementing the required sector and institutional reforms had been identified during appraisal, and according to the project design it had been agreed that in order to create a macro environment conducive to reform, the Government's role would need to be changed from that of a direct provider of services to that of a regulatory body with a policy and regulatory function. A study to recommend required policy changes, develop the regulatory framework, and propose a development strategy and action plan for implementing it was to have been prepared by June 30, 1996 and the results to have been discussed with IDA and 12 implemented by December 31, 1996. While most relevant actions including the institutional restructuring, were successfully completed, the establishment of a regulatory framework for the sector was not completed. (e) Approval of tariff adjustments: It was agreed at appraisal that periodic tariff adjustments would be made over the course of implementation, in order to improve ARWC's financial standing and put it in a position to make contributions to its investment programs from internally generated funds. The Government in principle supported adjustment of tariff structures to meet operational costs, and adjustments were made several times, albeit not to the levels required to reach full self financing. With the 2005 tariff adjustment, the increase was substantial enough to cover full operational costs. Nevertheless, while Azersu is now expected to slowly improve its revenues, the company's debt level remains substantial and debt service continues to rely on Government support. One of the issues being discussed with Government through the new Sector Strategy is the need for debt restructuring, which will require Government intervention and financing. Successful restructuring would permit sustained improvement in the company's financial standing. In the interim, Government has contributed investment subsidies that allow continued physical improvements in the system but do not respond to the underlying financial problems. It has also showed renewed commitment to increase metering which should help on collections. While not optimal, given the large financial resources available to Azerbaijan, there is a strong probability that services will be sustained and improved even if the need for a Government subsidy continues. (f) Loss of experienced staff: The CDU lost several trained staff to better paying jobs and further studies during the course of implementation. One of the reasons for this turnover was the large salary difference between the CDU and ARWC staff which reduced the long-term attractiveness of the company to the former. This was a major hindrance to the long-term capacity benefits of Azersu from the CDU who were expected to eventually merge with the Utility staff after the project. (g) Inadequate Metering: While Azersu made significant progress in terms of installation of bulk water meters, metering levels at the individual/domestic consumer level remained low (only 6.2 percent by the end of the project), and Azersu was not able to make the transfer to a metered billing system by the end of the project. This had consequences on the project's achievement of the envisaged demand management and billing improvement outcomes, and contributed to weaknesses in the project monitoring system. More recently, the Government has shown renewed interest in metering and has begun its own ambitious metering program which aims for universal metering and is currently installing about 300 water meters per day. The project was a `problem project' for a total period of about fifteen months on two different occasions over the project's life (February 1996-September 1996) and (May 2004 to April 2005) due to unsatisfactory progress towards meeting its development objectives. The problems were mainly with respect to the utility's poor financial performance and its implications on financial sustainability of the operation. The latter downgrade, made in response to persistently poor financial performance 13 was maintained until 2005 when the substantial tariff increase to cover O&M costs was made. The Bank team continued to hold dialogue with the Government and to recommend measures to try to address the financial situation through the project and parallel policy dialogue. 7.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization: (a) Monitoring and Evaluation (M&E) Design: According to the project design, implementation progress was measured against a list of physical and financial targets. Agreement was obtained during negotiations that actual performance would be compared against the appraisal estimates and reported to IDA on a semi-annual basis. The M&E targets were linked to achievement of different project outputs over given timeframes. A list of seven performance indicators relating to various operational aspects including service quality improvements (hours of service, reduction in unaccounted for water, water quality, and pressure), financial efficiency (tariff adjustments to cover operating costs, and reduction in Working Ratio), and measures to improve operational efficiency such as reduction in staff per thousand connections was also monitored to measure achievement of project outcomes. It should be noted that robust baseline data with respect to some operational indicators such as physical losses, total unaccounted for water, and system pressures were not available during project appraisal (in the first two cases due to the lack of metering), and were based on estimates. M&E was thus affected by the overall lack of reliable baseline data which stemmed from the inadequate monitoring, reporting, and information management system from which ARWC was starting at the onset of the project. (b) M&E Implementation: While the CDU with the help of the consultants carried out overall monitoring of project outputs and reported these on a quarterly basis to the Bank, outcome indicators were not monitored regularly. In addition to weaknesses in ARWC's operational monitoring system, and accountability to its management in this regard, there were other methodological limitations. It was for instance difficult to carry out accurate assessment of indicators requiring the measurement of water volumes (UFW, total losses, etc) given the weak baseline data and the fact that metering levels were very low and full system metering was not completed by the end of the project. Other limitations are linked to the failure of the project to implement a sub-component that would improve the internal financial and accounting reporting systems. In this regard, although the project design had required a Management Information System (MIS) to be put in place, this sub-components was cancelled due to insufficient funds in the IDA credit resulting from a drop in the exchange rate between the SDR and the US dollar. Improvement of the monitoring and benchmarking system for the sector is one of the issues that is addressed under the new sector strategy. 7.4 Safeguard and Fiduciary Compliance (focusing on issues and their resolution, as applicable): Safeguards: The main Safeguards issue under the project was Environmental Assessment. Project preparation with respect to environmental assessment was carried out in accordance with the World Bank Operational Directive 4.01 Environmental Assessment and environmental procedures of the Republic of Azerbaijan. The Project was a Category B project given that the rehabilitation works for 14 the water supply facilities had by their general nature potential environmental impacts on human populations or environmentally important areas, which were site-specific. The analysis accordingly examined the project's potential negative and positive environmental impacts and recommended measures needed to prevent, minimize, mitigate, or compensate for adverse impacts and improve environmental performance. There were no major problems that arose due to environmental assessment during project implementation. Fiduciary Aspects Financial Management: Financial Management of the project was carried out in accordance with the arrangements agreed upon in the legal agreements. A formal audit for both the Project and the entity was carried out on an annual basis, stating the adequacy of the accounting system and internal controls, the reliability of the Statements of Expenditure (SOEs) as a basis for credit disbursements, and compliance with covenants of the credit agreement. While the audits were carried out in accordance with International Standards of Auditing each year, there were delays in achieving the agreed upon deadlines for report submission on several occasions. Procurement: Given the limited capacity within the implementing agency, responsibility for procurement of goods, works and services under the project was by design carried by the CDU with the assistance of consultants who prepared final designs, prepared bidding documents, and carried out construction supervision. An independent Procurement Review commissioned by the Bank in 2005 confirmed that the function was generally carried out in a satisfactory manner in accordance with relevant World Bank procurement procedures, with minor discrepancies on a few contracts. The preparation of tender documents and procurement proceeded smoothly without any major problems, and the record keeping and filing systems were good. The CDU reported procurement information to the World Bank and EBRD on a quarterly basis, and issues arising were addressed with support from the Bank Procurement Staff. Change orders were agreed with contractors and ARWC and were subject to the Bank's 'no objection'. On balance there were a few delays in the procurement process, some of which occurred due to other factors such as counterpart funding not being readily available. 7.5 Post-completion Operation/Next Phase (including transition arrangement to post- completion operation of investments financed by present operation, Operation & Maintenance arrangements, sustaining reforms and institutional capacity, and next phase/follow-up operation, if applicable): Investments financed under the project are already being successfully operated and maintained under Azersu's regular operations, which have contributed to improved overall performance in terms of service delivery. The World Bank has worked with the Government to develop a Strategy for the Water Sector in Azerbaijan, which among other things identifies operations for development of the sector over the next decade. Key components of the Sector Strategy seek to address not only the required investment needs for improving sector performance and achieving Government investment goals, but also address financial and institutional strengthening needs. 15 Investments for the Greater Baku area are largely based on the Masterplan that was prepared as a part of the project and Government goals. The Government has requested for funding from the Bank for further development of water supply and sewerage services in the Greater Baku area in order to provide reliable services to the city. It has already allocated funding in its budget over the next three years to provide counterpart funding for investments in the water and sewerage sector in Baku. Other activities have also been initiated in parallel to improve sector performance. For instance, a Government decree instituted in 2004 requiring universal metering by 2008 also presents an opportunity for propelling the metering program forward beyond the project. In response to this decree, by 2006 about 50,000 new meters will have been installed in Baku outside of the project, the Government budget has estimated funds to complete the work, and all new buildings have metered connections. A Financial Restructuring and Recovery Plan is one of the key components of the Sector Strategy that is expected to contribute to improving Azersu's financial situation in order to ensure sustainable service delivery. 16 8. Assessment of Outcomes 8.1 Relevance of Objectives, Design and Implementation (to current country and global priorities, and Bank assistance strategy): Relevance of the project objectives is rated high. The project objectives, as stated in the SAR, focused on addressing the most critical challenges facing the water sector in the Greater Baku area at the time, which were: (i) inefficiencies in the production and distribution of potable water. Baku had inadequate water treatment facilities resulting in water that failed to meet bacteriological standards, placing the population under a severe health risk, and water was supplied on average only six hours a day; and (ii) weak financial and performance capacity of the institutions responsible for managing service delivery. They were responsive to the Borrower's development priority of improving water supply service delivery in Greater Baku, and consistent with the Bank's Country Assistance Strategy for Azerbaijan. Although the project was able to make some improvements, several issues in the sector still remain, and are a priority for both Government and development partners including the Bank. The project objectives are thus still relevant at the time of the ICR, and remain consistent with Azerbaijan's development priorities. Improving infrastructure is one of the intermediate objectives of the Government's current Poverty Reduction Strategy - the State Program for Poverty Reduction and Economic Development (SPPRED). It is also linked to the other SPPRED objectives which include: (i) creating an enabling environment for growth of income generating opportunities, (ii) maintaining macroeconomic stability, and (iii) improving living conditions of Internally Displaced People (IDPs) of the Nagorno-Karabakh conflict. Infrastructure service delivery has also been a central theme in each of the past and on-going CASs during the life of the project. One of the objectives outlined in the current World Bank Country Partnership Strategy with Azerbaijan (dated April 2003) seeks to improve access to services by reversing the decline in social services and infrastructure, such as water supply. The link between improving water supply and water quality to health issues is also important. As a result, continued improvements in water supply in Baku, and extending this improvement to the rest of the country is expected to feature prominently in the next CPS. 8.2 Achievement of Project Development Objectives (including brief discussion of causal linkages between outputs and outcomes, with details on outputs in Annex 4): The project outcome is rated moderately unsatisfactory based on its having achieved satisfactory outcomes with respect to physical investments, substantial quantifiable improvements in water access and quality, corporatization of the sector, increased tariffs and better planning, albeit with shortcomings. The doubling of the hours of water supply from 6 to 13, and the compliance with WHO water quality standards, represent a significant achievement. Indeed, this level of quality improvement, coupled with reduced water losses, is rare among water projects in ECA. Outcomes related to financial sustainability were rated unsatisfactory because despite tariff increases Azersu continues to rely on budgetary support which the project anticipated ending, not having foreseen the capability of Government to contribute to capital investments. The working ratio deteriorated substantially over the project life, rather than improving. It should be noted that the Government has 17 shown renewed interest this year in improving metering and self sustainability of Azersu on its own. Also, while not the optimal approach, given the growing financial wealth of Azerbaijan and commitment to improving water supply, it is likely that even if the need for budgetary support continues, the improved outcomes in water supply will be sustainable. Achievement of individual objectives and their associated outcome indicators based on the performance indicators identified at appraisal are outlined briefly below. (a) Achievement of Objective a). To make emergency short term improvements to the water supply system to restore the water supply to Baku, particularly to the poorer elements of the population: Achievement of this objective was satisfactory. Works in respect of urgent short term physical improvements to the water supply system including leakage reduction and other facility repairs were completed, resulting in improvements in the reliability of water supply to domestic consumers in Baku. Water production also increased through the rehabilitation of the Jeiranbatan and Kura River Water Treatment Plants, and physical losses were reduced through the rehabilitation of the distribution network. Although water supply was still intermittent in some parts of the city by the end of the project, the average number of hours of service had increased from the pre-project 6 hours a day to an average of about 13 hours a day, exceeding the targeted average 12 hours of service. This improvement benefited all areas of the city, and the poor together with the non-poor. The reduced coping costs and higher network water quality were likely more important to the poor, but monitoring was not in place to evaluate this result. (b) Achievement of Objective b) To improve the water supply system as a whole: This objective has been rated satisfactory on the basis of the impact of water supply investments in terms of improvements in service quality and reliability, and reduction in physical losses. Official estimates from Azersu in 2004 put UFW in the area at about 35 percent, which is better than the targeted 45 percent envisaged at appraisal, and physical losses in the distribution network at the ICR stage were reported to be as low as 20 percent. It is however difficult to accurately assess the extent of the reduction in UFW given the lack of metering at the start and the low level of metering at the end of the project. It could also be argued that the overall almost five year extension of the project due to the Supplemental Credit and the two other Closing Date extensions meant that achieved targets reached before the revised project closing date, appeared to have been over-achieved by the end of the project when measured against the original performance indicators. On balance however, achievement of targets directly related to the facilities affected by the earthquake were set back by the damage caused them. Planned physical investments including rehabilitation of the two main water treatment plants serving the Greater Baku area were substantially completed, and 30 chlorination stations for disinfection of water were installed. Consequently, the quality of water treated at the rehabilitated facilities is fully compliant with the relevant standards of the World Health Organization (WHO). As a result of the successful implementation of the leak detection and distribution system repair program the number of bursts in the city reduced, contributing to the ultimate reduction in water losses. 18 (c) Achievement of the Objective c) To provide the basis for longer term planning and recovery: Achievement of this objective is rated moderately unsatisfactory given the mixed outcomes with respect to two main aspects. First, it is assessed against satisfactory completion of the studies and analytical work geared towards the longer term planning and recovery of ARWC and the sector. This aimed at: (i) strengthening ARWC's technical, operational, financial, and managerial capacity; (ii) defining ARWC's long term objectives and development of a corporate development strategy; and (iii) laying the groundwork for ARWC's corporate development. Second, it is assessed against the extent of achievement of the desired outcomes of the reforms. While the first assessment comprised satisfactory results in view of the successful completion of relevant studies, analytical work, and other activities relating to strengthening ARWC, achievement of most of the intended outcomes was only moderately satisfactory even though the relevant activities were carried out: the extent to which they translated into required actions varied. A key achievement of the project was the corporatization of ARWC. The company was successfully restructured and provided with necessary facilitation to improve its overall capacity to manage services, including transfer to a newly refurbished building and provision of equipment required to run the systems administration and financial functions. Another major restructuring took place in 2004, when ARWC was formed into a new company Azersu Joint Stock Company (Azersu) which was based on similar commercial principles. There is evidence of improvements in overall operational and managerial capacity of the entity. For example, there was a reduction in staffing over the project implementation period. The staff ratio was reduced from about 12.3 employees per 1,000 service connections in the base year to about 7.8 per 1,000 connections at the end of the project. A maximum of 5 employees per 1,000 connections average is advised for international good practice. According to the Corporate Development Plan for 2005, manpower levels are to be further reduced by 100 employees a year up to the end of 2009. Improvements in Azersu's financial situation on the other hand have been marginal, and are rated unsatisfactory in view of their implications for the overall financial sustainability of the investments. In spite of putting in place several measures to improve the company's financial viability, these have not yet resulted in the company's financial viability in the absence of significant Government financing. After several significant increases, tariffs were at a level to cover the company's O&M costs by 2005, the end of the project. However, these have not yet been enough to ensure self sustainability because of collection problems. At the time of preparing this ICR, Azersu's end of project Working Ratio based on audited financial accounts was not yet available, and figures for 2004 have been used. The company's audited accounts for 2004 indicate a Working Ratio of about 1.1 which is significantly higher than the 0.6 ratio targeted, and even higher than the pre-project level of 0.75. Audited figures for total liabilities for the Greater Baku area by the end of the project, though not yet available, are not expected to have changed significantly in spite of the tariff increase due to an initial drop in revenue collection levels experienced after the increase. Moreover, even if revenues are expected to slowly improve, the company's financial situation still remains precarious due to the high debt burden it has, unless the Government is able to intervene in restructuring it. The mitigating factor in this outcome is the on-going Government allocation of budget resources to sector investments. Unusually for a company in this revenue situation, Azersu has been able to improve 19 and sustain system quality measured in hours of service, service pressure, and the quality of delivered water. The budget has supported sustainable service and allocations continue to increase. Given high probability budget scenarios, based on the rapidly growing oil wealth of the country, this behavior is itself sustainable. However it does not accord with the project vision of a self-financing service provider. Key commercial objectives such as improvement of billing methods through metering were also not achieved. Consequently, the billing system could not be changed from a flat rate to a volumetric system by the end of the project, due to the low level of metering achieved (only about 5 percent for domestic consumers). However in 2005, as noted above the Government made a renewed commitment to introduce universal metering with initial very positive results, and has made budgetary allocations towards meeting this objective over the next three years. 8.3 Efficiency (Net Present Value/Economic Rate of Return, cost effectiveness, e.g., unit rate norms, least cost, and comparisons; and Financial Rate of Return): A detailed economic analysis of the project was carried out at appraisal. The EIRR on the water supply investments was estimated at 27 percent on the basis of the quantified benefits, which included (a) willingness to pay for an improved water supply service; (b) public health benefits through a reduction in waterborne and water-related diseases; and (c) cost savings through a reduction in the annual recurrent costs of operating the water supply system, indicating that the project was a very robust investment. Sensitivity tests carried out at the time also indicated that the project was generally robust to variations in cost and benefit parameters. The net present value and the EIRR were recalculated at ICR, on the basis of assumptions outlined in Annex 5. Because of lack of data, public health benefits have been assigned a zero value despite the significant improvements in water quality, and no reduction coping cost was taken into account despite the doubling of time water is supplied now. The costs involved in achieving the project benefits are deemed to have been reasonable, given the positive net present value of the project (US$ 10.3 million) and the 19 percent EIRR, demonstrating the satisfactory economic performance of the operation. 8.4 Justification of Overall Outcome Rating (combining relevance, achievement of PDOs, and efficiency): Rating: Moderately Unsatisfactory The overall outcome of the project is rated moderately unsatisfactory on the basis of the foregoing arguments on its relevance (Section 8.1), the level of achievement of the objectives (Section 8.2), and efficiency (Section 8.3). In this regard, improvement of water supply services in the Greater Baku area, which formed the basis for the project, was significant: Hours of water supply have increased on average from 6 to 13 hours and water quality is in compliance with WHO standards. The project was able to achieve most of its development objectives in terms of service and quality improvements, tariff increases, and the corporatization of the sector, with moderate shortcomings. At the same time, the shortcomings in respect to its financial accounts are concern, although given the growing wealth 20 of the country this is not expected to affect the overall sustainability of its outcomes. Fortunately, Government has committed to improving services, has continued to back this up with needed budget allocations, and has launched a long delayed effort to install consumption meters with its own funds. Building upon the achievements so far and the lessons learnt, the project sets the stage for future operations that will enhance outcomes in the Greater Baku area and beyond. 8.5 Overarching Themes, Other Outcomes and Impacts (if any, where not previously covered or to amplify discussion above): (a) Poverty Impacts, Gender Aspects, and Social Development: The impact of the project on the urban poor and other vulnerable groups such as the elderly and pensioners on fixed incomes was taken into consideration at appraisal. The project was expected to assist the population of Greater Baku including the poor to gain improved access to an essential service. The Social Assessment at appraisal revealed that services for all income levels were severely deficient in Greater Baku, and the poor spent a far larger proportion of their income on coping measures to address deficiencies in water supply services (as much as 7 percent compared to 2 percent on average for the non poor). The Social Assessment also indicated that vulnerable people such as women, the elderly and those on fixed incomes appeared to bear the greatest burden of the water supply crisis, and by inference would benefit most from the project. A formal assessment of the project's poverty impact has not been carried out at the ICR stage. However, the poor benefited from the increase in hours of supply from 6 to 13 hours (on average), with some areas moving up to 24 hours of supply. The poor also benefited from the improvements in water quality which met WHO standards at the end of the project. The Government is committed to ensuring that affordable services are provided to all its population, including the poor. The Bank is assisting the Government, through its ongoing policy dialogue and investment projects, to develop a social assistance program to mitigate the impact of sector reforms, including measures such as possible tariff increases on the poor. The Bank has also prepared an extensive review of the water supply sector and the remaining challenges this year to take stock of lessons learned and establish a foundation for further work in this sector. (b) Institutional Change/Strengthening (particularly with reference to impacts on longer-term capacity and institutional development): The project's institutional development impact, which is defined as the extent to which it has improved the agency or country's ability to make effective use of its human and financial resources, has been rated moderately satisfactory due to the mixed level of achievement with respect to different aspects. While considerable improvements have been gained with regard to use of human resources, the financial resource management still needs to be strengthened. As a result of the institutional reforms introduced under the project, including the institutional restructuring to form the more commercially oriented ARWC (later Azersu), and the support provided through the twinning/Corporate Partner and CDU, there is evidence that the sector is better run than it was at the 21 onset of the project and the project was implemented successfully. Staff productivity improved significantly over the implementation period, and the Productivity Ratio reduced from about 12.3 to 7.8 staff per 1,000 connections. The CDU staff, some of whom became part of Azersu, acquired capacity for project implementation, including procurement related functions, and can be counted upon in undertaking this function in future projects financed by the Bank. In addition staff capacity was built with regard to several operational aspects such as O&M capacity and customer relations, and recruiting of an experienced person as Vice President of Azersu's investment department. A special Dispatcher Communications and Customer Relations department was established within Azersu in 2004, specifically to address customer complaints, and Azersu's responsiveness to customer complaints had improved. The Corporate Development Plan prepared by the CDU on an annual basis provided strategic direction for improvement of operational performance. However, the extent to which the Plans were able to actually translate into concrete actions was hindered by factors such as a lack of commitment on the part of management to fully implement all the recommendations. In terms of financial management capacity, the plan at appraisal was for the restructuring to include explicit performance objectives and well defined budgets for the sector in order to improve corporate governance and increase financial discipline. An important tool towards this end was supposed to be the installation of a Management Information System linked with both operational budget planning and strategic planning, including inter-alia, a general ledger module with provision for departmental accounting; an automated fixed asset module to provide all pertinent information on depreciation and an easily retrievable means of handling revaluations; a general accounts receivable module for fully automating the billing and collection process; and a cost accounting module to separately account for wages, materials, and other expenditures in the central and district workshops, as well as in the treatment plants and pumping stations. This tool (the MIS sub-component) was cancelled during project implementation due to budgetary constraints. While the Bank's policy dialogue through its policy and technical assistance operations is working on addressing some of the key financial issues, including improved regulation, accounting and transparency, the intended results of the dialogue have not yet been fully achieved. At the same time, the Government recently initiated some actions to improve corporate governance, including adopting a new Accounting Law requiring state owned utilities to adopt International Financial Reporting Standards (IFRS) not later than January 1, 2008. (c) Other Unintended Outcomes and Impacts (positive or negative, if any) There were no major unintentional impacts or outcomes of the project. 8.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops (optional for Core ICR, required for ILI, details in annexes): A beneficiary assessment and Stakeholder workshop were not carried out at project completion. 22 9. Assessment of Risk to Development Outcome Rating: Moderate The level of risk to development outcome is rated as moderate on the basis of the existence of both factors that would enhance the technical sustainability of the investments financed under the project, and those that may hinder financial sustainability. Technical sustainability of physical investments in water supply made under the project is likely given Azersu's improved capacity to operate and maintain the facilities in a sustainable manner, which has been enhanced through the project. The sustainability of the investments is also enhanced by the general improvement in institutional capacity to manage service delivery achieved through the project. The increase in tariffs and the renewed focus on metering will also help. There is concern related to Azersu's financial outlook and its potential impact on the investments and service delivery. Azersu as the system operator should optimally receive sufficient revenues from user fees and government transfers to cover the costs of operation and maintenance as well as to finance investment costs and debt service. While tariffs and collection rates have increased in recent years, and Azersu is slowly making positive steps towards self sufficiency, the company remains dependent on Government support for investment financing for new systems or replacing dilapidated systems in the short to medium term. Mitigation of the risk to development outcome will require further improvement in Azersu's financial viability through increased revenue collection, resulting from a combination of improvements in operational efficiency and in the billing and revenue collection systems, supported by adequate policy dialogue geared at achieving the desired results. That said, net Government revenues are increasing rapidly as oil flows ramp up and the President and Government have repeatedly committed to high quality water and sanitation services in Baku to date, backing that commitment with the needed budget allocations. As a result sustainability of achieved outcomes is expected to continue, even if continued budget support for investments is required. 10. Assessment of Bank and Borrower Performance (relating to design, implementation and outcome issues) 10.1 Bank: (a) Bank Performance in Ensuring Quality at Entry (i.e., performance through lending phase): Rating: Moderately Satisfactory Bank performance during identification, preparation, and appraisal of the project was satisfactory. The Bank team consisted of two core members and seven additional specialists who included a sanitary engineer, an institutional and training specialist, an environmental specialist, social specialists, and a finance and economics specialist. Four specialists from EBRD also participated in the appraisal mission. Potential risks were identified, and the Bank Safeguard policies were adequately addressed. Fiduciary capacity of the Borrower was also assessed, and adequate arrangements put in place to build required capacity through the twinning arrangement with the corporate partner and implementation support through the CDU. However, the country readiness for 23 sector reform was overestimated, and the financial performance target was overambitious. The project design rightly stressed the importance of demand management, but this is not reflected in the proposed investment, which foresaw the installation of only 15,000 consumption meters. Quality at entry is therefore rated moderately satisfactory. (b) Quality of Supervision (including of fiduciary and safeguards policies): Rating: Moderately Satisfactory The quality of project supervision carried out by the Bank is rated moderately satisfactory because although the quality of supervision was satisfactory in most respects, there were some shortcomings in ensuring timely project implementation, which contributed to the overall increase in the project implementation period by at least three years, excluding the additional two year timeframe directly resulting from the Supplemental Credit. The Bank Team carried out 23 supervision missions in the field over the ten year implementation period and responded to implementation issues in a timely manner as they arose. Given the nature of the operation and associated activities, the skill mix usually included at least one Sanitary engineer/Water and Sanitation Specialist and a Financial Specialist, who were able to adequately handle supervision of most of the technical and fiduciary issues arising on the project. The internationally recruited supervision team was joined by a local Bank Operations officer who provided close continuous coordination and liaison with the client. Constant communication was also maintained with the Borrower through other means such as electronic mail. Over the course of implementation the project had five different Task Team leaders as a result of the normal reassignment process within the Bank. Nonetheless continuity in adequate project supervision was ensured. Project Supervision reports were generally well written and highlighted the key issues. Monitoring and evaluation however appears to have been rather weak, mainly as a result of the lack of adequate baseline data and/or ability to accurately measure indicators for some of the key performance indicators such as UFW, system pressures, and hours of service. Project supervision also seems to have focused more on implementation of the tangible outputs such as physical investments and activities, and not enough on the outcomes of corporate development activities. The final ISRs rated the project moderately satisfactory, anticipating that the 2005 tariff increase would be matched by high collection rates and therefore significantly improve ARWC's financial performance. As that has not yet happened, this ICR has chosen a moderately unsatisfactory rating, creating a disconnect. The moderately satisfactory rating is based on the above shortcomings. (c) Justification of Rating for Overall Bank Performance: Rating: Moderately Satisfactory The rating for overall Bank performance is based on the satisfactory quality at entry and moderately satisfactory quality of supervision, which together contributed to a moderately satisfactory overall outcome for the project. 24 10.2 Borrower: (a) Government Performance: Rating: Moderately Satisfactory Government performance is rated moderately satisfactory on the basis of its role during project preparation and implementation. Its performance during project preparation was satisfactory, especially given that the project was the first operation of its kind in Azerbaijan. There was close coordination and dialogue between relevant Government officials and the project financiers, the Bank and EBRD. The Government demonstrated its commitment to achieving the development objectives, by providing support for the institutional reforms made under the project. As a result the Baku Water Agency was successfully restructured to form the Apsheron Regional Water Company (ARWC), and Government provided adequate support for putting in place the required legislative framework. Allocation of counterpart funding was generally adequate throughout project implementation, with some delays experienced in availing the funds on a few occasions. The Government was able to respond to implementation issues that required its attention, such as the need for obtaining the Supplemental credit from the Bank after the damage caused by the earthquake. On balance however, the moderately satisfactory rating is justified by the significant delay experienced in achieving results on the ground in response to what was an emergency situation. It was almost three years after the earthquake before remediation works were initiated. Another shortcoming in Government performance relates to the fact that it was not able to push through tariff increases at the rate needed to ensure the utilities sustainability through tariff income early in project implementation. This was only partially achieved at the end of the project. However, the Government deliberately changed its strategy during project implementation. Instead of the overambitious goal to recover initial investment costs from company revenues, it instead has continued a reliance on fiscal transfers and quasi-fiscal measures. This contributed to the inadequate achievement of the financial outcome indicators selected during project design. (b) Implementing Agency or Agencies Performance: Rating: Moderately Satisfactory Implementing Agency AzerSu Joint-Stock Company Implementing Agency performance is rated moderately satisfactory because, while most aspects were satisfactory, there were some moderate shortcomings in performance. The Agency (ARWC and later Azersu) was committed to achieving the project development objectives. Recognizing the relatively low institutional capacity to handle project implementation hence the lack of the Agency's readiness for implementation, the design provided for support to it through the international Corporate Development Partner under the twinning arrangement and the formation of the Corporate Development Unit. As a result the project was well managed, and implementation issues were 25 responded to adequately and fiduciary responsibilities with respect to both procurement and financial management carried out in a satisfactory manner. Relationships between the different stakeholders including the Government and project financiers were smooth, and the implementing Agency was able to play a coordinating role between the different stakeholders, as required. Transition arrangements for activities supported by the project into regular operations were also smoothly carried out, and the newly rehabilitated water treatment and distribution facilities were effectively utilized to improve service delivery. Public relations efforts initiated under the project to improve demand management were mainstreamed into the Agency operations, with the recruitment of a public relations officer. Like the Government however, a shortcoming in the implementing agencies performance relates to the significant delay in responding to the emergency caused by the earthquake, which took at least three years before remedial works could begin. (c) Justification of Rating for Overall Borrower Performance: Rating: Moderately Satisfactory Overall Borrower performance takes into consideration both the Government and the Implementing Agency's performance during project preparation and implementation. On the basis of the justification provided above, Borrower performance is rated moderately satisfactory. 11. Lessons Learned (both project-specific and of wide general application) The main lessons learned from the design and implementation of the project are the following: (i) Assumptions used in projecting future performance need to be as realistic as possible, and to take due consideration of prevailing political, economic, and social risks. Some of the assumptions considered during project appraisal were too optimistic, and raised very high expectations of the project's outcome. The assumption for instance that within three years of implementation, ARWC would be able to generate revenues equivalent to more than the sum of total operating expenses, debt service requirements and provision for non-cash items including depreciation and replacement costs was way too ambitious given the existing circumstances, and identified risks such as underestimation of the extraordinary effort and long-term commitment required to achieve financial viability. During project implementation increasing Government revenues allowed them to revise the strategy for sustainable water services to a more realistic level. Azersu had begun to make positive steps towards self sufficiency by the end of the project, but it still dependent on budget support to achieve sustainability. (ii) Development outcomes of investment projects can be enhanced when done in parallel with a strong overall policy dialogue, including through separate DPLs and other interventions that address key issues in the sector. The project is one piece of an integrated country program, but it is important to recognize during preparation and implementation, its limitations in achieving certain outcomes. The Bank has been engaging in a strong policy dialogue with the Government through a 26 series of DPLs, investment projects and poicy dialogue which included both concrete policy measures on increasing tariffs, improving the financial viability of Azersu, and establishing a regulatory framework. This dialogue also includes establishing International Accounting Principles and targeting social assistance specifically to alleive the impact of utility sector reform on the poor. This dialogue has contributed to the positive, albeit slow, progress towards achievement of these aspects. (iii) Technical Assistance for implementation of reform components should be packaged separately from consultancy services for investment operations. Technical Assistance involving provision of support for both corporate development and investment implementation by the same company appears to have resulted in differing levels of the company's effort towards achieving the outcomes. It is recommended that responsibility for investment and reform components be divided into separate contracts in order to enhance the accountability for each area by the relevant vendors and optimize their outcomes. In this case the combination of the role of foreign specialists to implement the investments as well as influence corporate development of ARWC gave unbalanced results as greater attention appears to have been given to the implementation of investment activities which produced more visible results. (iv) Technical Assistance alone cannot bring about reforms unless it is combined with strong political support and the commitment of all stakeholders, including utility management. Ultimate responsibility for reform lies with the Government and implementing agency/utility, even where TA may be provided to facilitate required actions. All stakeholders need to fully appreciate the principles behind the reforms rather than just be informed of the associated activities. During the course of implementation the structure of the sector underwent two major restructurings, and even though most of the TA activities/outputs were implemented as discussed in Section 4.2, translation into the required reform outcomes was not fully achieved. (v) The objectives and design of project sub-components should be consistent with the resources allocated to them. In the case of the metering component, the project originally only provided for financing 15,000 meters (only 5,000 were installed during the project), yet set a performance target of transfer from a normative to a metered billing system by the end of the project. (vi) The focus on M&E during project implementation/supervision needs to be improved: ARWC experienced both capacity and systematic issues with respect to M&E, which although noted during appraisal, were not adequately focused on during project supervision. 12. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies: Azersu's comments were generally consistent with the ICR's assessment of the project with regard to the outcomes relating to water supply improvements due to the physical investments and the institutional reforms made. It however does not provide comments on issues relating to the financial 27 sustainability of the project. A summary of Azersu's comments is provided in Annex 10, and a detailed report is also attached. (b) Cofinanciers: EBRD's independent assessment of the project's performance, prepared in 2001, assigned a `partly successful' overall performance rating, based on the institution's internal rating methodology which took into account; (i) transition impact; (ii) achievement of objectives; (iii) environmental impact; and (iv) company and project performance. Their evaluation of the project's performance with respect to each of these aspects is summarized below: · Overall transition impact was rated marginal, based on a lack of readiness of the institutional and business environmental, the absence of autonomy of the corporatized ARWC, the difficulty to disseminate the experience of a company operating under urban conditions to rural conditions, and its very weak performance in financial terms. It was however noted that the project enabled the Government to set up an emergency investment programme at a time when it would have been difficult to procure any other financing. · The water quality improvements had not materialized by 2001, due to delays in completion of the rehabilitation of the water treatment plants funded under the IDA Credit. It was noted however water was safer due to the successful installation of chlorination facilities. · ARWC's institutional achievements fell far short of the rather over ambitious goals defined during project preparation. The impact of the project on the organization of ARWC was assessed as limited as a consequence of the limited degree of autonomy granted the central authorities. The evaluation further stated that the success of the cooperation with the Corporate Development Partner and the CDU was limited, making the transfer of institutional knowledge slow and marginal. The limited application of the Corporate Development Plan prepared for ARWC on an annual basis by the CDU in accordance with their Terms of reference was seen as an illustration of these shortcomings. EBRD's project evaluation was carried out in 2001, immediately after the EBRD loan closed, and not after the final project completion date in January 2006, almost five years later. Moreover at the time the EBRD loan closed, the Supplemental IDA Credit which financed repairs to facilities damaged by the earthquake that occurred in November 2000 had not yet been approved. Disconnects between the EBRD assessment and the ICR assessment are thus inevitable. It should also be noted that EBRD's evaluation, particularly with regard to investments, focused largely on outcomes related to the investments financed by EBRD, which comprised the Turnkey contract for rehabilitation of the Water Treatment Plants. On the institutional front, there were also major changes that occurred after the closing of the EBRD loan, the most notable of which was the formation of Azersu. After the contract with the Corporate Partner ended and the Supplemental Credit was approved, the project continued with the CDU taking the lead in supervising implementation, which was done in a generally satisfactory manner. 28 (c) Other partners and stakeholders: (e.g. NGOs/private sector/civil society): The completion report prepared by the consultants to the Swiss Secretariat for Economic Affairs (SECO) focused mainly on the investments financed by the organization, rather than on the entire project. Like the EBRD report, it was produced in 2002, about three years before the project closed. The goal of the SECO financed investments which included rehabilitation of 460 booster pumping stations through a single contract between 2001 and 2002, was to improve the availability of drinking water to a population of 350, 000 people. Out of the original 460 stations identified, 194 were actually rehabilitated. In a number of cases, the residents resisted the taking over of the pumping stations by ARWC, while in others ARWC made a decision not to finance those stations whose owners who included oil, shipping, and railroad companies were deemed able to finance them themselves. Overall the project goals were met, and water was able to be delivered under reasonable pressure to 340,000 customers, 97% of the target population by the end of 2002. 29 Annex 1. Results Framework Analysis Project Development Objectives (from Project Appraisal Document): The primary objectives of the Project were to: (a) make emergency short term improvements in the water supply system to restore the water supply to Baku, in particular to the poorer elements of the population; (b) improve the water supply system as a whole; and (c) provide the basis for longer term planning and recovery. Although the Government later requested IDA assistance through a Supplemental Credit to the ongoing credit to permit financing of the cost of repair and reconstruction of Apsheron Regional Water Company (ARWC) facilities damaged by the November 25, 2000 earthquake, the objectives remained unchanged. The performance indicators with respective to the above objectives were: (i) Improvement in hours of service from the pre-project average of 6 hours/day to an average of 12 hours/day by the end of the project; (ii) Reduction in unaccounted for water (UFW) from the pre-project level of 66 percent to less than 45 percent by the end of the project; (iii) Improvement in water quality and water pressure by the end of the project; (iv) Adjustment of residential and industrial water tariffs as necessary to cover operating costs; (v) Change in the billing system from flat to metered rate; (vi) Reduction in the ratio of employees per thousand service connection or households; and (vii) Reduction in the Working Ratio from the pre-project level of 0. 75 to less than 0.60 by the end of the project. Revised Project Development Objectives (as approved by original approving authority): The project objectives were not revised. (a) PDO Indicator(s) - from Project Appraisal Document Original Target Formally Indicator Baseline Values (from Actual Value Achieved at Value approval Revised Completion or Target Years documents) Target Values Reduction of unaccounted for water from the pre-project level of 66% to less than Indicator 1 : 45% by end of project Value Values were (quantitative 66% 45% not formally 35% or revised. Qualitative) Date achieved 03/01/1996 01/31/2006 December 2004 Physical losses and UFW at both appraisal and ICR are estimates, given the low level Comments of metering. Azersu estimates actual physical losses at 20% at project completion, (incl. % possibly indicating existence of administrative losses of about 25%, hence the achievement) differential with UFW. Improvements in the hours of service from the pre-project average of 6 hours/day to Indicator 2 : 12 hours/day by end of project 30 Value Values were (quantitative Supply at ICR was estimated at 6 hours/day 12 hours/day not formally or about 13.3 hours/day. revised. Qualitative) Date achieved 03/01/1996 01/31/2006 01/31/2006 December 2004 Comments While target was exceeded overall, supply varied from area to area in Baku with some (incl. % receiving less than 12 hrs/day and some receiving 24hr/day. Actual target achieved achievement) before project completion. Indicator 3 : Improvements in water quality and water pressure by end of project Value Water quality achieved WHO (quantitative standards, and Azersu confirmed n/a n/a n/a or improvements in system pressures Qualitative) due to the repaired booster pumps. Date achieved 03/01/1996 01/31/2006 December 2004 Comments (incl. % There were no targets provided at appraisal, hence no measurable targets at ICR. achievement) Adjustment of residential and industrial water tariffs as necessary to cover operating Indicator 4 : costs. Value Values were (quantitative Tariffs cover US$0.08/m3 not formally US$0.15/m3 or operating costs. revised. Qualitative) Date achieved 03/01/1996 01/31/2006 01/01/2005 Comments The current tariff level made in January 2005 is sufficient to cover basic O&M costs (incl. % without depreciation and debt service. achievement) Indicator 5 : Change of the billing system from flat rate to metered billing Value (quantitative Flat rate Transfer to metered billing was not Metered billing n/a or billing achieved. Qualitative) Date achieved 03/01/1996 01/31/2006 N/A Comments The level of metering achieved under the project was low (less than 10%), hence it (incl. % was impossible to transfer to metered billing. Meters being installed under new achievement) Government initiative. Reduction of the ratio of employees per thousand service connections or household Indicator 6 : served Value 12.3 Not quantified Reduced to about 7.8 staff per 31 (quantitative thousand connections. or Qualitative) Date achieved 03/01/1996 01/31/2006 Comments Although the target value was not quantified at appraisal, there was a sustained (incl. % reduction in staff per thousand connections. achievement) Reduction of the working ratio from the pre-project 0.75 to less than 0.6 by the end of Indicator 7 : project Value Target values (quantitative 0.75 0.6 were not 1.1 or revised. Qualitative) Date achieved 03/01/1996 01/31/2006 June 2004 Comments (incl. % The targeted working ratio was not achieved. achievement) (b) Intermediate Outcome Indicator(s) - from Project Appraisal Document Original Target Values Formally Actual Value Achieved at Indicator Baseline Value (from approval Revised Target Completion or Target documents) Values Years Indicator 1 : The project has no intermediate outcome indicators. Value (quantitative or Qualitative) Date achieved Comments (incl. % achievement) Comments No intermediate outcome indicators were identified at appraisal. 32 Annex 2. Restructuring (if any) Restructuring Board ISR Ratings at Amount Disbursed at Reason for Date(s) Approved Restructuring Restructuring in USD Restructuring & Key PDO Change IP DO M Changes Made 12/10/2002 N S S 51.50 n/a 33 Annex 3. Project Costs and Financing1 (a) Project Cost by Component (in USD Million equivalent) Appraisal Components Estimate (USD M) Actual/Latest Percentage of (US$ million) Estimate (USD M) Appraisal INSTITUTIONAL CAPACITY 6.9 6.8 98.6 BUILDING OPERATIONS AND 23.0 21.2 92.2 MAINTENANCE IMPROVEMENTS SUPPLY IMPROVEMENTS 18.6 19.8 106.5 STUDIES AND CONSTRUCTION 4.0 5.6 140.0 SUPERVISION WATER DEMAND 5.0 5.0 100.0 MANAGEMENT OPERATIONS AND 10.3 7.5 72.8 MAINTENANCE--SUPPL. SUPPLY IMPROVEMENTS-- 1.6 4.6 287.5 SUPPL. INSTITUTIONAL CAPACITY 0.3 0.3 100.0 BUILDING--SUPPL. PROJECT SUPERVISION--SUPPL. 0.4 0.4 100.0 Total Baseline Cost 70.1 71.2 Physical Contingencies 0.0 0.0 0.0 Price Contingencies 0.0 0.0 0.0 Total Project Costs 70.1 71.2 Project Preparation Facility (PPF) 1.0 1.0 100.0 Front-end fee IBRD 0.0 0.0 0.0 Total Financing Required 71.1 72.2 1There are some discrepancies in the costs due changes in the exchange rates between the SDR and the US dollar over the project lifetime. 34 (c) Disbursement Profile 35 Annex 4. Outputs by Component Part A. Water Demand Management: Desired outputs related to this component, which covered metering, billing, consumer education, and household leakage, were realized to varying extents under the project. (i) Metering and transfer of billing system: By the loan closing date the level of metering in Baku was only about 6.2 percent for domestic consumers, with industrial and commercial customers at about 77 percent. Consequently, the metering program was not able to achieve the level of coverage required to transfer from a flat rate to a metered billing system. About 5,000 meters were purchased under the project and installed in households in one part of Baku under a pilot program in 46 apartment buildings in poor sections of the city. Under this program external plumbing systems within the building were also rehabilitated. Although the pilot metering program was successfully implemented, the amount of funding allocated to metering under the project was not enough to achieve the level of metering that would be required for the transfer to metered billing. A Government decree instituted in 2004 requiring universal metering by 2008 presents an opportunity for propelling the metering program forward beyond the project, assuming necessary funding and support is provided. By 2006 about 50,0000 new meters had been installed in Baku outside of the project. (ii) Conservation of water at the household level was promoted through the consumer awareness campaign implemented by NGOs under the direction of ARWC's Public Information Officer. Video clips were produced on installation of water meters and water conservation, and presented on television channels in Azerbaijan. In addition, other promotional materials, including water conservation manuals for primary schools and company brochures were produced and distributed across the city. The impact of these public awareness campaigns has not been independently assessed as part of this ICR. (iii) The household leakage sub-component identified at appraisal, which was supposed to comprise household leakage reduction through installation of about 180, 000 toilet cisterns, was not carried out. Part B: Operation and Maintenance Improvements. This component primarily sought to improve operations and maintenance through implementation of a program of leak detection and distribution network repair, workshop repair, and equipment acquisition. Respective activities were completed in a satisfactory manner and achieved desired outcomes. 36 (i) Tools and equipment for leak detection and repair were procured under the project, and ARWC's O&M staff were trained in their use. This resulted in improved O&M capacity at the utility level. As a result of the leak detection and repair activities, the response time for leak repairs in Baku reduced considerably, from an average of about 60 hours to 6 hours. (ii) About 150 km of deteriorated pipeline were replaced, resulting in a significant reduction in physical water losses. It was not possible to accurately measure levels of UFW at the start and end of the project given the low metering level. Average hours of service in Baku also increased from 6 hours at appraisal to 13.3 hours, which was higher than the targeted 12 hours. Part C: Supply Improvements. Physical investments with respect to this component, which sought to improve the quality and quantity of water through a program of selective rehabilitation of water treatment plants and pumping stations, were completed in a satisfactory manner. This included repair of the two main water treatment plants (Jeiranbatan and Kura WTPs) and replacement of 30 chlorination stations. Water supply service improvements were realized in terms of: (i) improvements in water production, safety and quality, through the rehabilitated WTPs and installation of new chlorination equipment; (ii) improvements in the water transmission system, through the rehabilitated pumping plants; and (iii) an increased control of water flows in the system, through bulk water metering. Part D: Institutional Capacity Building. Activities under this component were achieved in a satisfactory manner. The utility entered into a contract with an internationally experienced utility operator as a Corporatization Partner (CP) under a Twinning arrangement. The CP was responsible for providing support for institutional strengthening and for project implementation through direct support to the CDU. This was provided through a team of short and long term international experts in key disciplines relating to water supply utility management. The CP and the CDU had the responsibility of assisting in developing and managing both the corporate development and the investment component of the project. The CDU's primary objectives were: (i) to facilitate the establishment and development of ARWC; (ii) to facilitate the transition to a new corporate structure and improve operational and financial performance; and (iii) to manage all aspects of the implementation of the improvement program. The CDU with the support of the CP was able to facilitate delivery of most of the required outputs with respect to the above Institutional Capacity Building activities, in particular those with respect to formation of ARWC, as well as the management of the investment programme in a satisfactory manner. With respect to the institutional restructuring, in May 1995, then Baku Water Agency (BWA), the original entity responsible for water supply services in the Greater Baku area, was transformed into a corporatized company, ARWC, which was supposed to operate in accordance with principles of commercially oriented water utilities. Its offices were consolidated under one headquarters building, refurbished under the project. The number of employees was subsequently reduced from about 4000 to 3400 in order to improve productivity. ARWC was headed by a president, who was also the chairman of its Board of Directors. In 2004 the Government carried out further restructuring of ARWC, and formed yet another company Azersu Joint Stock Company 37 (Azersu) that would be responsible for all water supply and wastewater municipal service providers across the country. The production units of medium, small towns, and rural utilities were dissolved, and their functions, assets, and subsidiary organizations transferred to Azersu. Baku Wastewater Department, which was originally in charge of wastewater management in the Greater Baku area, was also dissolved and transferred to Azersu. While the structural changes in sector organization were successfully achieved, desired outcomes with regard to improvements in operational and financial performance of the sector, and were much slower even with the Twinning arrangement with the CP. The level of achievement of the benefits of the Twinning arrangement with the CP was moderately satisfactory because although most of the activities were carried out, the reform outcomes were moderately satisfactory. The CP was able to provide on the job training to partners in the CDU, resulting in transfer of relevant operational skills. With the support of the CP the CDU prepared a Corporate Development Plan (CDP) on an annual basis, outlining key development activities required to make the utility a modern efficient, cost effective, and customer oriented water company, and constituting the focus for the technical assistance to be given under the CP. The CDPs addressed various operational issues, including technical, financial, commercial and customer service aspects, most notably support with implementation of a billing program. In spite of the CDPs being prepared on an annual basis under the project, it appears that the Plans as an output in themselves did not always translate into actual implementation on the ground. Full achievement of the benefits depended on other factors beyond the control of the partners, such as inadequacy of tariff levels, and the need to change perceptions, which takes time. Ultimately, responsibility for the benefits of reform lies with the Government and implementing agency/utility, even where TA may be provided to facilitate required actions. Part E: Studies and Construction Supervision. In addition to construction supervision, the following studies were carried out under this component: (i) a Masterplan for the water supply and wastewater services for the Apsheron Penninsula; (ii) a National Water and Wastewater Policy Study; (iii) a Water Law was prepared and approved by parliament; (iv) a Water Tariff Study and a Billing and Collection Review. Delivery of the outputs in terms of relevant studies and reports was completed. These documents have indeed provided the basis for development of the sector, albeit to varying extents. The Masterplan for water supply and wastewater services by 2015 was prepared, and is being used in development of the strategy for the Water Sector which is being done by the Bank and the Government. The National Water and Wastewater Study and the Water Tariff and Billing and Collection Reviews were also carried out. A legal and institutional assessment provided the basis for the preparation of a Water Law that has provided the grounds for reform of the sector. The Water Law prepared under the project and approved by Parliament in 2000 is one of the key legal documents now regulating the Water sector in Azerbaijan. 38 Annex 5. Economic and Financial Analysis (including assumptions in the analysis) At Appraisal: A detailed economic analysis of the project was carried out at appraisal. The EIRR on the water supply investments was estimated at 27 percent on the basis of the quantified benefits, which included (a) willingness to pay for an improved water supply service; (b) public health benefits through a reduction in waterborne and water-related diseases; and (c) costs savings through a reduction in the annual recurrent costs of operating the water supply system, indicating that the project was a very robust investment. Sensitivity tests carried out at the time also indicated that the project was generally robust to variations in cost and benefit parameters. The costs were based on; a) capital costs due to the project and replacement capital costs in 2005, 2010 and 2015, and b) on incremental O&M costs, which were attributable to salaries and wages, increased materials and supplies plus materials to ensure that existing assets are maintained, and on higher administrative costs due to the project. Quantification of health benefits was based on the estimation of economic loss related to the incidence of waterborne and water related diseases as a result of value of lost production time due to illness; economic loss due to premature death and economic loss due to medical treatment. The limitations of the approach were recognized, including; (a) the inaccuracy of health statistics; (b) unreported cases of disease. It is important to note that there are several externalities that can contribute to improved or reduced health outcomes and several unrelated diseases that can have symptoms such as diarrhea similar to water related diseases, which further affects the accuracy of analysis. Incremental savings in operating costs were assumed to accrue through a substantial reduction in electricity consumption resulting from improved system operation, higher pumping efficiency, and non-revenue water and through reduced chemical requirements as a result of the decline in non revenue water. At ICR: The net present value and the EIRR were recalculated at the ICR stage. In addition to those made at appraisal, several assumptions were made during the analysis carried out at ICR because of lack of reliable data on several of the criteria used to estimate costs and benefits, including on the health benefits directly attributable to the project and on the willingness to pay for services at the ICR stage. In the absence of reliable health data and despite the fact that water treatment plant rehabilitation and chlorination station operation have resulted in significant water quality improvements so that water is meeting WHO quality standards now, the public health benefits have been assigned a zero value. Other assumptions were: (i) that there were no benefits directly attributable to the project until 1997 when substantive investments had been made; and (ii) the reduction in UFW from 70 percent to a conservative 35 percent was achieved steadily over the period between 1996 and 2002. Relevant data on the actual costs and benefits, where available, was collected from Azersu and project files in order to estimate the incremental costs and benefits due to the project. The net present value and the EIRR as recalculated are a positive net present value of the project (US$ 10.3 million) and a 19 percent EIRR, demonstrating the satisfactory economic performance of the operation. 39 Annex 6. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Names Title Unit Responsibility/Specialty Lending Ayse Kudat Consultant ECSSD Social Specialist Sr Water & Sanitation Jan Drozdz EASUR Sanitary Engineer Spec. Piotr Krzyzanowski Consultant ECSSD Environmental Specialist William V. Mayville Consultant ECSPE Institutional Specialist Supervision/ICR Brian Steven Smith Sr Financial Spec. EWDEA Financial Analyst - TTL Ede Jorge Ijjasz- Manager EWDWP Water Engineer - TTL Vasquez Farid Alexander Operations Officer ECSIE Operations Officer Mammadov Sr Financial Ida N. Muhoho ECSPS Financial Specialist Management Specia Sr Water & Sanitation Jan Drozdz EASUR Sanitary Engineer -TTL Spec. Lars A. V. Rasmusson Consultant ECSIE Sanitary Engineer Paul Kriss Senior Economist AFTU2 Economist - TTL Plamen Stoyanov Kirov Procurement Spec. ECSPS Procurement Specialist Sanyu Sarah Senkatuka Operations Analyst ECSIE Operations Analyst Lutalo Solvita Klapare E T Consultant ECSSD Consultant 40 (b) Ratings of Project Performance in ISRs No. Date ISR Actual Disbursements (USD Archived IP DO M) 1 07/06/1995 Satisfactory Satisfactory 0.00 2 08/31/1995 Satisfactory Satisfactory 0.00 3 11/17/1995 Satisfactory Unsatisfactory 0.00 4 02/29/1996 Satisfactory Unsatisfactory 0.00 5 04/08/1996 Satisfactory Unsatisfactory 1.00 6 06/10/1996 Satisfactory Unsatisfactory 1.41 7 09/20/1996 Satisfactory Satisfactory 1.71 8 01/22/1997 Satisfactory Satisfactory 1.90 9 06/03/1997 Satisfactory Satisfactory 3.61 10 07/08/1997 Satisfactory Satisfactory 3.67 11 01/09/1998 Satisfactory Satisfactory 13.83 12 03/24/1998 Satisfactory Satisfactory 18.44 13 02/12/1999 Satisfactory Satisfactory 25.01 14 07/02/1999 Satisfactory Satisfactory 26.93 15 11/05/1999 Satisfactory Satisfactory 27.59 16 02/29/2000 Satisfactory Satisfactory 28.15 17 06/02/2000 Satisfactory Satisfactory 32.42 18 12/11/2000 Satisfactory Satisfactory 38.18 19 06/27/2001 Satisfactory Satisfactory 43.80 20 12/26/2001 Satisfactory Satisfactory 47.04 21 02/13/2002 Satisfactory Satisfactory 48.11 22 10/17/2002 Satisfactory Satisfactory 51.50 23 12/17/2002 Satisfactory Satisfactory 51.70 24 04/02/2003 Satisfactory Satisfactory 51.72 25 05/07/2003 Satisfactory Satisfactory 51.73 26 12/09/2003 Satisfactory Satisfactory 51.72 27 05/28/2004 Satisfactory Unsatisfactory 52.17 28 11/30/2004 Satisfactory Unsatisfactory 56.62 29 04/10/2005 Satisfactory Satisfactory 58.79 Moderately Moderately 30 02/15/2006 65.53 Satisfactory Satisfactory 41 (c) Staff Time and Cost (from SAP) Stage of Project Cycle Staff Time and Cost (Bank Budget Only) Fiscal Year USD Thousands (including travel and consultant costs) Lending FY94 126.4 FY95 355.7 FY98 0 FY00 -0.6 FY03 0 FY05 0 Total: 481.5 Supervision/ICR FY95 8.4 FY96 76.6 FY97 105.7 FY98 67.2 FY99 105.6 FY00 88.6 FY01 73.7 FY02 44.0 FY03 45.2 FY04 46.7 FY05 49.1 FY06 17.4 Total: 728.20 42 Annex 7. Detailed Ratings of Bank and Borrower Performance Bank Ratings Borrower Ratings Ensuring Quality at Moderately Satisfactory Government: Moderately Satisfactory Entry: Implementing Quality of Supervision: Moderately Satisfactory Moderately Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Moderately Satisfactory Performance: Performance: Annex 8. Beneficiary Survey Results (if any) A beneficiary survey was not carried out at the ICR stage. Annex 9. Stakeholder Workshop Report and Results (if any) A stakeholder workshop was not carried out at the ICR stage. 43 Annex 10. Summary of Borrower's ICR and/or Comments on Draft ICR Azersu Comments Achievement of Operational and Technical Objectives: The activities implemented under the project were aimed to the rehabilitation of existing infrastructure and by that had allowed cop with the challenges Baku water supply system had been facing to. After the reconstruction works in the water treatment plants and regional pumping stations, the quantity and quality parameters of water supplied to the city has significantly improved. Particular achievements with respect to specific technical aspects of the project are outlined below. (i) Quality: As a result of implemented works, the quality of water treated at the rehabilitated facilities (Kura and Jeyranbatan water treatment plants) is in full compliance with the relevant standards of the World Health Organization. (ii) Reliability: There had been no more major interruptions in operations of the Kura water treatment plants once the rehabilitation was completed. In addition it became doable to discontinue frequent distortions and ensure a stable flow in the distribution network through the reconstruction works carried out at the 25 main pumping stations, (iii) Physical loss reduction: The establishment of leakage detection teams and the network rehabilitation works implemented within the project Azersu significantly decrease daily water losses and steady the water supply in the city. Procurement of specialist plants and other equipment allowed the company to overcome breakages and accidents the network in a shorter time and with adequate reliability. (iv) The rehabilitation of booster pumping stations substantially improved water supply to the multi-storey apartment buildings and in result water supply services to more than 350,000 inhabitants were improved. (v) Construction of production warehouses enabled Azersu to establish proper maintenance service in each administrative district of the city and improve working conditions for the staff. With all above mentioned accomplishments the project objectives are to be considered fully achieved and the project operational and technical performance is rated as successful. Achievement of Institutional Objectives: The corporatization of Baku water utilities set up as a condition for the IDA Credit effectiveness was one of major achievements of the project institutional objectives. The twining arrangements established through the corporate partnership contract with international experience utility operators (Severn Trent and Berlinerwasser Betriebe) had significantly contributed to the institutional capacity building of former Absheron Regional Joint Stock Water Company 44 (current Azersu Open Joint Stock Company). The corporate partnership started though the support to the project implementation allowed to develop the organizational structure of the company, donated to the establishment of billing and collection system that was a new activity for utilities at that time, provided for training of more than 700 staffs abroad (Turkey, England, France, Germany and Switzerland) and in country. The staff was also involved in the work of international supervision consultants (Brown end Rut, CES, DAR) and in result construction and rehabilitation works supervision skills were developed in house. Given the complexity of the project activities and challenges faced during the implementation the Borrowers own performance is rated as successful. The Bank staff input and support was one of the crucial factors for the objectives achievements and project implementation. The quality of advice and recommendations, the understanding of local specifics and flexibility of approach in changing circumstances without scarifying the strategic objectives were notable demonstrated by the Bank through the course of implementation. As a Borrower we evaluate the Bank and other partners performance as successful. One of the important lessons to be drawn from the completed project is that implementation arrangements and availability of implementation capacity are critical for a successful project. Those should include not only fiduciary activities like procurement, disbursement, project financial reporting but also a capacity to perform the institutional reforms. Such capacity should be evaluated during project preparation and building of such a capacity should be included in the scope of the project. 45 Annex 11. Comments of Cofinanciers and Other Partners/Stakeholders Summary of Overall EBRD Assessment: EBRD's Operation Performance Evaluation Review (OPER) of the project rated its overall performance as 'partly successful', based on the EBRD Project Evaluation Department (PED)'s internal rating methodology, which took into account, transition impact achievement of objectives, environmental impact, company and project performance. The evaluation verified the Bank's additionality in enabling the Government of Azerbaijan to set up an emergency investment programme at a time when it would have been very difficult to procure any other type of financing. The overall transition impact was rated 'marginal', mainly due to the lack of readiness of the institutional and business environment, the absence of autonomy of the corporatised ARWC, and the weak financial performance, among other things. On balance however it was noted that the operation could still have been justified given the emergency nature of the operation, and the improvements in aspects such as billing and collection. With regard to specific objectives, the general view is that the objectives were ambitious and were marginally achieved. For some objectives such as cost recovery and tariff, the evaluators felt that the Bank fixed the target too high. Objectives with respect to improvements in service quality (quality and reliability) were generally rated 'partly achieved'. Part of the reason for the poor assessment was due to the fact that the water treatment facilities had not been completed at the time due to delays experienced in implementation, and effects of the earthquake. While the review noted that the objective of corporatising ARWC was achieved, and that some achievements had been made by the company in behaving as a corporation, its performance was still hindered due to a lack of financial autonomy. The team felt that the skills and experience developed within the trained CDU staff were only partly utilized for the core organization of ARWC. The assessment of the objective with respect to improving financial and operational performance was also rated as not achieved, due not only to ARWC's failure to carry out necessary measures, but also failure of MOF to allow them to make necessary tariff decisions. 46 Annex 12. List of Supporting Documents List of Documents Staff Appraisal Report Project and Credit Agreements for IDA 2751 Project Supervision Reports Implementation Status Reports Monthly Progress Reports Quarterly Progress Reports Project Files (Correspondence) EBRD Operation Performance Evaluation Review SECO Final Report Project Audit Reports Independent Procurement Review for Azerbaijan ARWC/ Azersu Financial Statements Project Financial Statements 47