ICRR 13929 Report Number : ICRR13929 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 01/21/2014 Country : Brazil Project ID : P087711 Appraisal Actual Project Name : Br Espirito Santo US$M ): Project Costs (US$M): 62.50 249.60 Water & Coastal Pollution Management L/C Number : L7248; L7655 Loan/ US$M ): Loan /Credit (US$M): 36.0 107.50 Sector Board : Water US$M): Cofinancing (US$M ): Cofinanciers : Board Approval Date : 07/01/2004 Closing Date : 09/30/2008 09/30/2011 Sector (s): Sewerage (75%); Water supply (15%); Sub-national government administration (10%) Theme (s): Pollution management and environmental health (33% - P); Urban services and housing for the poor (33% - P); Environmental policies and institutions (17% - S); Other urban development (17% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : Kavita Mathur Alain A. Barbu Soniya Carvalho IEGPS1 2. Project Objectives and Components: a. Objectives: This project was a follow up to a previous project (Loan 3767, approved in 1994 and closed in 2003) which could only be partially completed due to implementation issues, and aimed to fulfill that project's objectives while adding to them. As such the project's objectives were to: (i) strengthen SEAMA’s and IEMA’s environmental management capacity. (ii) Improve the efficiency of CESAN’s water supply and sewerage systems; (iii) increase the coverage level of CESAN’s water supply and basic sanitation services; and (iv) provide appropriate water and sanitation infrastructure in low-income urban areas. The Project Loan Agreement stated " the Borrower confirms its commitment to the objectives of the Original Project, and, to this end, shall: (i) through SEAMA and IEMA, carry out Part A of the Project; and (ii) without limitation or restriction upon any of its other obligations under this Agreement, cause CESAN to perform all its obligations set forth in the Project Agreement, all with due diligence and efficiency and in conformity with appropriate administrative, environmental, financial, technical and engineering practices, and shall provide or cause to be provided, promptly as needed, the funds, facilities, services and other resources required for the Project." For the purposes of this review, the original project objectives from Loan 3767 will be used. b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: The Technical Annex of the MOP for Loan 7248 identifies four (4) components (MOP p. 9-11). The cost of these components also includes revised estimates based on the appraisal of the additional loan (Loan 7655). (a)Institutional Strengthening SEAMA/IEMA (b) Strengthening CESAN and the State's Sanitation Sector (c) Water Supply Investments (d) Sewerage 1. Institutional Strengthening SEAMA/IEMA (appraisal US$2.0 million; revised US$ 3.80; actual: US$ 1.83) The program would comprise the following activities (MOP p.9-10) (a) an institutional and strategic planning exercise; (b) design and installation of an integrated environmental information system; (c) development of a monitoring network for the environment in general and water resource management in particular; (d) development of IEMA’s environmental licensing capacity; (e) development of a state-wide water resource management system; and (f) training and environmental education activities. 2. Institutional Strengthening CESAN and the State's Sanitation Sector (appraisal US$2.85 million; Revised: US$ 5.99 million Actual: US$ 3.47 million). Activities would include (MOP p. 10): (a) development of a performance benchmarking system to track both CESAN’s service quality as perceived by users, and its operational and financial performance; (b) design and implementation of actions to improve CESAN’s corporate governance; (c) training of CESAN’s staff; (d) non-revenue water reduction; (e) environmental management and education; (f) support for new household sewage connections and operational improvements; and (g) strengthening of the Borrower’s sector organization and the legal and regulatory instruments for water and sanitation service provision. 3. Water Supply investments (appraisal US$8.56 million revised: 26.35 million; actual US$ 38.56 million) An investment. program including: (a) expansion and upgrading of the Carapina and Caçaroca water treatment plants; (b) upgrading of the Ibes-Boa Vista transmission main;(c) expansion and upgrading of the Santa Clara, Pedreras/Santo Antonio, Garoto, Araças, and Morro do Pico storage tanks; and (d) upgrading of the Guarapari raw water transmission and treatment systems. 4. Sewerage Investments (appraisal US$39.29 million revised: 121.63 million; actual US$ 185.37 million) An investment program including: (a) a sanitation master plan for the Vitoria Metropolitan Region (particularly for Vitória, Vila Velha, Cariacica, Serra, Viana and Guarapari) as well as development of a water quality model; (b) expansion of sewage collection systems in the cities of Vitoria, Vila Velha, Guarapari and Cariacica; and (c) construction of new sewage collection and treatment systems in the districts of Nova Almeida and Praia Grande (in the city of Serra), in the district of Manguinhos (in the city of Serra), and in the city of Viana. Revised Components (as described in the ICR p. 3) A restructuring of the project was approved in 2008 in conjunction with additional financing. The original bidding documents for contracting the sewerage works were based on a 2005 updated engineering design, but given the cancellation of the bidding process in 2006, the engineering design of these works was revisited in 2007. Studies concluded that a single stage of treatment works for an incremental capacity of 360 l/s (implemented as an extension to the Mulembá WWTP) would more efficiently satisfy the demand in the GVMR for the design horizon. This redesign also required increasing the capacity of the pumping stations and the outfall, as well as updating the cost estimate of completing the sewerage works and reviewing and re-launching the tendering process. In parallel to re-estimating the project costs, the State Government also revisited the financing plan. This implied removing some of the original activities from the Bank-financed program, namely : (i) the new sewerage collection network works in Serra (94.7 km of pipes); (ii) the new sewerage networks in Praia do Canto, Praia da Costa, Praia do Morro and Cariacica (251.4 km of pipes); and (iii) the new sewerage systems in Jucutuquara and Vitória Centro, including Ilha do Boi and Ilha do Frade (144.8 km of pipes).18. As a result of this exercise, the State Government transferred some of the works to a cluster of works to be financed by the Federal Government under the Growth Acceleration Program (PAC - Programa de Aceleração do Crescimento ), and financed the remainder with its own resources, with the exception of the sewerage collection networks in Serra for which CESAN is considering the implementation of a public-private partnership (ICR p. 4). d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The loan was approved on July 1, 2004 with effectiveness on January 11, 2005 and a projected closing date of September 30, 2008. The original IBRD contribution was US$36.00 million. The original borrower contribution was US$26.32 million. Due to: (i) the devaluation of the US Dollar in relation to the Brazilian Real and a corresponding increase of about 39% in project costs: (ii) domestic inflation; and (iii) additional costs of building the water and sewerage works, additional financing in the amount of $71.5 million was approved in 2008. The original closing date was extended and the project closed on September 30, 2011. The additional financing brought the IBRD's total contribution to US$107.50 million or 62% of total financing as compared to 57% of initial funding. The borrower contribution rose to US$ 64.82 million or $38% of total funding as compared to 43% of initial funding. I 3. Relevance of Objectives & Design: a. Relevance of Objectives: High. High . The objectives of the project were clearly defined and were highly relevant to the State ’s and the Country’s social and economic development at the time of appraisal and continue to have a high degree of relevance at project closing. The project objectives were designed to enable the beneficiaries in the project area to benefit from improved quality of and access to water supply and sewerage services through improving the efficiency of CESAN ’s water supply and sewerage systems, as well as strengthening the environmental management capacity of SEAMA and IEMA. (ICR p.9). The objectives are consistent with the Country Partnership Strategy (CPS) developed in 2008, which continued to support the four pillars put forth in the original CAS (2003-2007): equity, sustainability, competitiveness, and sound macro-economic management. The project’s development objectives were also in line with the water sector strategy for Brazil which recognized water as an important element to promote sustainable growth and a more equitable and inclusive society, and highlighted the importance of improving inst itutional capacity and efficiency . The project' objectives are also consistent with the current CPS (2012-2015) which identified four strategic objectives within the Partnership Strategy. Two of these objectives are directly relevant : (i) Increasing the Efficiency of Public and Private Investments and (ii) Improving the Provision of Public Services for Low Income Households . b. Relevance of Design: Substantial The four project components were very relevant to the project's objectives . The project's design took into account the experiences and progress made under the original project (Loan 3767) insofar as it provided the resources necessary to complete the water supply and sewerage investments along with adding additional focus to the need for institutional strengthening of CESAN and SEAMA . That design was based to a large extent on the utility's 1986 master plan. Engineering design updates were undertaken in 2005 and 2007. These studies resulted in a number of technical changes including increasing the capacity of the pumping stations and the outfall along with price increases. Additionally, the design changes led to an updating of the cost estimate for completing the sewerage works, a revision of the financing plan and the review and re -launching of the tendering process . (MOP p 6-7). The re-design reflected the need establish an independent project management unit, UGP, which would report to the general coordinator of the Project Steering Committee . By establishing the UGP, this avoided overtaxing CESAN's limited implementation capacity which was a key factor affecting implementation under the previous project (MOP p. 16). At the same time, there were weaknesses in the project's design . The sewerage component was redesigned . The results chain did not discuss targeting of low income household to achieve the fourth objective, and additional financing was needed for water and sewerage works . The appraisal loan amount was US$ 36 million. Additional financing of US$71.5 million was needed in 2008, around the time of the original closing date . 4. Achievement of Objectives (Efficacy): The original Project Objectives from the original loan ( Loan 3767, 1994) were to: (i) strengthen SEAMA’s and IEMA’s environmental management capacity. (ii) Improve the efficiency of CESAN’s water supply and sewerage systems; (iii) increase the coverage level of CESAN’s water supply and basic sanitation services; and (iv) provide appropriate water and sanitation infrastructure in low-income urban areas. 1. Strengthen SEAMA’s and IEMA’s Environmental Management Capacity ( Substantially Achieved) Outputs The project helped facilitate the development of environmental management capacity at SEAMA/IEMA. Key project outputs (summarized from ICR Annex 2) included: Completion of two public service exam processes to hire new staff in 2004 and 2007 and training of technical staff, and annual planning exercises to select a portfolio of projects. Acquisition of 6 network servers, software and computers. Acquisition of geographic information systems (GIS) mapping software Development and implementation of the Environmental Licensing Information Management System (Sistema de Informação para oLicenciamento Ambiental ). Basic information on the environmental licensing processes is now available online. Elaboration of a new cartographical base of the State of Espírito Santo (1:10,000 scale) replacing the set developed in the 1970’s. Reconfiguration and modernization of the hydro-meteorological network including the acquisition of hydrological and meteorological stations with the installation of telemetric systems. Acquisition of 2 boats to support water resources monitoring and control in the coastal areas and of 3 vehicles to support environmental control and enforcement in remote areas. increase the number of permits granted each year between 2005 and 2011, reaching a total of over 11,000 permits during this period. Establishment of a cadastre of main water users in 12 basins; a detailed cadaster of users in a pilot area, as well as creation and streamlining of the processes for licensing related to the use of state water resources Creation of water basin committees and elaboration of demand studies and guidelines Outcomes The following achievements are reported at the outcome level. The institutional strengthening measures implemented under the project successfully enabled SEAMA/IEMA to develop its environmental information systems, improving its licensing procedures and capacity (by simplifying the process of environmental licensing and reducing the time to grant permits) needed to accompany the rapid industrialization of Espírito Santo in a sustainable manner. These measures were reinforced and made sustainable by the Espírito Santo state government’s decision to recruit a significant number of qualified staff in 2004 and 2007 combined with an effort to provide adequate technical capacity building to its operational staff, demonstrating again its commitment to these institutions and their sustainability (ICR p. 15). In the recent strategic planning exercise, SEAMA/IEMA prepared a plan that builds on the project achievements and continue to prioritize future activities and projects with well-defined targets, schedule and scope. As a result of this exercise, SEAMA/IEMA will undertake a full review and re-engineering of its organization and processes, with a view towards improving their efficiency and performance in all aspects of the State’s environmental management. 2. Improve the Efficiency of CESAN’s Water Supply and Sewerage Systems (Substantially Achieved) Outputs Development of a performance benchmarking system to track both CESAN’s service quality as perceived by users and its operational and financial performance. Implementation of an Environmental Management System based on ISO 14,000 as a pilot in the WWTP of Guarapari and the water treatment plant in Carcaroca. Outcomes CESAN has 41 operational performance and quality indicators including physical, social, operational, environmental, financial, and service quality indicators. indicator measurements since 2004 indicate steady improvement in most areas with some coming close to or meeting targets. CESAN increased its treated water storage capacity by 56 percent compared with 2002, reaching 108,970 m3 in 2011, reducing intermittency and providing services with greater operational security and more efficient system management. The water supply network was also increased by 28 percent from 2002, reaching 6,924 km in 2011 and the project allowed production capacity to increase by 22 percent. Water losses (liters/connection/day) were reduced substantially to 460 liters which exceeds the target of 511 (ICR p. 11). Additionally, other targets have been exceeded including number of active water connections, ratio of active to non-active connections, coverage level for water supply, average water tariff, EBITA, and return on capital employed (%). 3. Increase the Coverage Level of CESAN’s Water Supply and Basic Sanitation Services (Modestly Achieved) Outputs (water) Construction of the 8,000 m3 Morro do Pico water storage tank. This tank will benefit an estimated 120,000 people in Cariacica and Viana. These works included the expansion of the São Francisco treated water pumping station and installation of 2.8 km of water mains. Expansion of the Santa Maria water production system. CESAN increased to 3,600 l/s the capacity of a raw water pumping station by installing two new pumps. This will benefit 495,000 people in Serra and the northern part of Vitória. Construction of the Ibes / Boa Vista water main. Expansion of the Caçaroca water production system. Construction of the 10,000 m3 Garoto water storage tank and installation of 3.3 km of water mains. Construction of the 6,500 m3 Araçás water storage tank and installation of 2.6 km of water mains. Outcomes (water) CESAN increased its treated water storage capacity by 56 percent compared with 2002. The water supply network was also increased by 28 percent from 2002. The project allowed production capacity to increase by 22 percent. Coverage levels exceeded the original targets and unaccounted-for water was reduced from 45.8 percent at appraisal to 35.6 percent at project closure (surpassing the target at appraisal by 8 percent). The impact of the original 1994 project on reducing unaccounted-for water had been deemed negligible and the investments during the 2004-2011 period helped optimize the existing delivery systems and sustain operational efficiency by installing additional storage infrastructure in the problematic areas to counter rationing and intermittency. (ICR p 26) Outputs (sewerage) The construction of 2 raw sewage pumping stations and the installation of 3.2 km of sewage collection pipes, 170 new connections in Castelo Branco and Jardim de Alah. The construction of 2 raw sewage pumping stations and the installation of 20.5 km of sewage collection pipes and 2,064 new connections in Praia do Morro. The construction of 4 raw sewage pumping stations and of the Manguinhos WWTP (capacity of 111 l/s), with 100% counterpart funds. The construction of 6 raw sewage pumping stations and of the Nova Almeida/Praia Grande WWTP (capacity of 120 l/s), with 100% counterpart funds. The installation of 5.8 km of sewage collection pipes and 350 new connections in Praia da Costa (basin B13). The construction of the Mulembá II WWTP in Jucutuquara to expand the existing Mulembá WWTP’ s capacity by 360 l/s. The construction of 8 raw sewage pumping stations and the installation of 95.3 km of sewage collection pipes and 8,630 new connections in Jucutuquara (basin B5). The installation of 16.1 km of sewage collection pipes and 1,810 new connections in Praia do Canto (basin B4). The construction of 4 raw sewage pumping stations and the installation of 37.1 km of sewage collection pipes and 7,800 new connections in Vitória Center (basins B1 and B2). Outcomes (sewerage) These works contributed to providing sewerage coverage to 1,033,350 people, an increase of 679,350 since 2003 (194 percent of appraisal target). The finalization of works in Vitoria was being financed with the client's own resources and would account for 153,000 of the 679,350 additional beneficiaries. With these results, the State and CESAN now provide universal access to the utility’s sewerage systems in Vitória, and 60 percent sewerage coverage in the GVMR, representing an increase of 192 percent in the population served compared with 2003 (ICR pp 26-27). 4. Providing Appropriate Water and Sanitation Infrastructure in Low-income Urban Areas (Substantially Achieved) Outputs The construction of 4 raw sewage pumping stations and the installation of 37.1 km of sewage collection pipes and 7,800 new connections in Vitória Center (basins B1 and B2). Subsidized program to connect low-income families to sewerage network. Improved water supply services to over 600,000 people Outcomes Upon full completion of the sewerage works, 30 percent (203,805) of the 679,350 additional people served by CESAN, were considered to be of low income (family incomes of less than 3 minimum salaries), compared with an estimate at appraisal of 109,000 beneficiaries of low income. The project thus achieved 187 percent of that target. For water supply, there was reduced rationing in GVMR neighborhoods. In addition, CESAN launched a program to eliminate the 7 percent of the population which remained with intermittent and deficient service provision in the GVMR. Low income municipalities such as Cariacica had a large share of its residents affected by this issue. This program improved the reliability of supply and the rate of intermittency of supply was reduced to 1.5 percent of the population in the GVMR (ICR p. 11). 5. Efficiency: Cost-benefit analysis were conducted at appraisal for the original loan. A 10% opportunity cost of capital was used. Economic analysis was conducted to measure the willingness to pay for the water supply component and to quantify the direct benefits of the project arising from this component. A separate economic analysis was conducted to measure the willingness to pay for the sewage collection and treatment component and to quantify the direct benefits arising from this component. Health benefits were determined using a proportion of the hospital expenditures faced by Brazil's National Health System (INAMPS) that originates from the State of Espfrito Santol. Due to data limitations, fishing benefits were not measured, even though there is a clear indication that they are important for the sustainability of the fishing sector in the state. (SAR p.101) The benefits of the project are quantified using contingency valuation (CV) and opportunity cost (OC) methodologies. A travel cost (TC) methodology is also applied to measure the water recreation benefits in order to check the robustness of the results from the contingency valuation methodology. (SAR p.102). The estimated internal economic and financial rates of return for the original loan were 33% and 15%, respectively. (SAR p viii). As part of preparation of the new loan (7248), the project’s original Cost-Benefit analysis was reviewed and updated. Overall, the activities proposed are expected to have a financial internal rate of return (FIRR) of 28% (51% for water supply investments and 16% for sanitation investments). When discounted at 10% over 30 years, cash flows from additional investments are expected to generate a net benefit of about US$93 million. Given the high positive externalities for investments of this type, the economic rate of return (ERR) of proposed water supply and sanitation activities is even higher: 44% overall (72% for water supply and 24% for sewage collection and treatment). The overall net economic surplus is almost twice the financial benefit, or about US$169 million over 30 years. (MOP p.16) As part of the preparation of the additional financing (Loan 7655) a revised and updated cost-benefit analysis was undertaken. Due to the increase in costs and the delay in implementing the works, the ERR was re-estimated to 22%, one-half of the 2004 estimate for the new loan (7248) and 11% lower that the original project estimate of 33% (Loan 3767). ICR Cost-Benefit Analysis: The ICR cost-benefit analysis was carried out including all components of the investment except the institutional strengthening of SEAMA/IEMA. For the evaluation of the water and sewerage components, the costs of institutional strengthening of CESAN, and project management and supervision were added in proportional form to each of the components. As the costs associated with the institutional strengthening of CESAN along with project management were added proportionally to each component, coverage is estimated at 100%. Actual investment costs were 48 percent higher than the estimated costs at appraisal. Separate cost-benefit analysis were undertaken for the water and sewerage components. For the sewerage component two scenarios were evaluated: (i) benefits attained by current customers at the time of ICR preparation; and (ii) potential customers based on CESAN's projections for 2012--2016. C-B Water (ICR p. 29) : The water component is profitable in financial as well as in economical terms, with returns of 25 percent and 159 percent respectively and profits of US$26 million and US$285 million. Results show similar financial results than expected at appraisal. Economic results surpassed significantly the ones foreseen at appraisal. C-B Sewerage (Scenario I) (ICR p. 29): The sewerage component is not profitable. It loses about US$40 million in financial terms and US$31 million in economic terms. C-B Sewerage (Scenario II) (ICR p. 30): The sewerage component becomes profitable in both financial and economic terms, with results significantly higher than projected at appraisal. The financial profit attained in sewerage is US$67 million and the rate of return is 16 percent, while the economic benefit is US$236 million and the rate of return 21 percent. Efficiency is rated Substantial ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal Yes 44% 100% ICR estimate Yes 30% 100% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The project's outcome is rated satisfactory. The Relevance of objectives is high and the relevance of the design is substantial. The project's design built on the experience of the previous investment (Loan 3767) and addressed the previous projects weaknesses. Efficiency is rated substantial with high financial and economic rates of return for the water component. Despite the current lack of positive financial returns for the sewerage component (under Scenario 1), profits from the water component provide positive overall economic returns of about 30%. Under Scenario 2 the sewerage component becomes profitable in both financial and economic terms with returns of 16% and 35% respectively. a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: Over the life of the investment, CESAN improved its performance not only through improved efficiency gains, but also in its capacity to financially sustain itself. This improved financial sustainability along with an increased focus on client responsiveness (including through the internalization and monitoring of service quality indicators) which provides pressure to maintain service levels, are important factors in securing more sustainable water and sewerage services in the State of Espírito Santo (ICR p. 16). CESAN's organizational development over the last decade coupled with support from the State government has put it in a position where it has the capability of adequately maintaining the systems it has built. Additional investments in the amount of R$1 billion are planned through 2014. The only minor risk is that CESAN's clients (both residential and commercial) may in the future find it difficult to pay for service price increases that will almost certainly come about. There is no evidence in the ICR to suggest that additional risks could be present that might be detrimental to the achievement of the development outcome. a. Risk to Development Outcome Rating : Negligible to Low 8. Assessment of Bank Performance: a. Quality at entry: The prior project's implementation (Loan 3767) had been hampered by poor performance by both CESAN and the State Secretariat for the Environment and Water Resources (SEAMA)) along with a lack of counterpart funds (mainly those owed by the State Government) during much of the original 5-year implementation period. The funding shortfall arose primarily from the State Government’s inability to secure a parallel loan for project implementation from the EIB. By the original closing date (December 3 1, 1999), a large number of sewage collection systems in the Vitoria Metropolitan Region (which includes the municipalities of Vitoria, Vila Velha, Cariacica, Viana, Serra, Fundgo and Guarapari) remained unfinished and unconnected to sewage transmission, treatment and disposal infrastructure. Accordingly, the design of the subject project established adequate mechanisms to mitigate the identified implementation risks including: a. Establishment of an independent Project Management Unit (Unidade de Gerenciamento de Projeto –UGP) that would closely coordinate project implementation activities with CESAN and SEAMA/IEMA. b. Establishment of an escrow account with in advance deposits of the necessary counterpart funds for each phase (equivalent to 3 months’ cash flow). The State Government and the Bank viewed this as a mechanism both to ensure the timely completion of project activities and to instill confidence in contractors and consultants that invoices would be paid without undue delay (ICR p.4). In addition, the project developed 2 components targeted at institutional strengthening of CESAN, SEAMA and IEMA. Under the project, CESAN, would develop a performance benchmarking system to track both CESAN’s service quality as perceived by users, and its operational and financial performance along with improved corporate governance and staff training. For SEAMA and IEMA, key activities included: preparation of a strategic plan, design and installation of an integrated environmental information system, development of a monitoring network for the environment in general and water resource management in particular and development of IEMA’s environmental licensing capacity. The main document used for the project was the Technical Annex to the MOP. This report was well prepared and covered most pertinent subject related to components, implementation, sustainability, environment, benefits and risks. However, it did not provide any specifics on monitoring and evaluation or output and outcome indicators. The project adopted most of the indicators from the previous project and added additional ones during implementation. In response to IEG, the project team confirmed that an effective M&E framework was put in place for the project as part of its design through an exchange of letters in 2004 and 2008. at-Entry Rating : Quality -at- Satisfactory b. Quality of supervision: The project was adequately supervised. Eighty four staff weeks were devoted to supervision over a 7 year period. The Bank supervision team included experts on procurement, engineering and utility management. On average 2 ISRs were prepared each year. The project received a satisfactory rating throughout the period of implementation. In 2006, the project was subject of a QAG review whose timing coincided with the failed bidding process for the large sewerage works. They considered this outcome a threat to the achievements of the PDOs. The panel also was not convinced that the project was achieving its institutional development objectives. The QAG review rated project supervision as moderately satisfactory. Following the QAG review, the supervision team provided timely and high level technical assistance to CESAN in reviewing its bidding documents and cost estimates, which was critical to the success of the re-launched bidding process for the large sewerage works. In addition, the team continuously monitored the project’s indicators as well as CESAN’s physical, operational, environmental and financial indicators, and also adequately supported institutional strengthening efforts. (ICR p 17) The ICR ( p. 6) indicates that there were no deviations or waivers from Bank safeguards during the implementation of the project, and compliance with applicable safeguard policies was rated satisfactory throughout the project. Given that the planned works took place in the public right-of-way and in unoccupied areas that were already owned by CESAN, OP 4.12 was not triggered and did not apply to the project. Quality of Supervision Rating : Satisfactory Overall Bank Performance Rating : Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: Over the life of the project , the State of Espirito Santo along with the key institutional players (CESAN< SEAMA/IEMA exhibited a strong political commitment to project implementation and in in prioritizing the water and sanitation sector. When faced with cost increases both institutions mobilized the necessary resources to undertake planned project activities. The State Government also emphasized the importance of improved environmental management and this emphasis was maintained throughout the project (ICR p. 17). Government Performance Rating Satisfactory b. Implementing Agency Performance: Overall project implementation showed significant improvement when compared to the previous project. Establishing an independent Project Management Unit (Unidade de Gerenciamento de Projeto – UGP) staffed by experienced management/engineering professionals was a key factor in the project's implementation. The UGP closely coordinated project implementation activities with CESAN and SEAMA/IEMA. The contracting strategy adopted for the major water supply and sewerage works proved to be efficient, with the use of turnkey contracts as its center piece. Turnkey contracts simplified the evaluation process allowing for faster payments and faster execution, as the contractor has an incentive to quickly resolve obstacles.(ICR p. 5) The bidding process of the first sewerage works was unsuccessful. After discussions with the Bank, CESAN decided to cancel the bidding process, which resulted in a delay in contracting these works, and a review of its approach to the sewerage turnkey contracts. The engineering design of these works was revisited and a decision was reached to undertake single stage of treatment works for an incremental capacity of 360 l/s would more efficiently satisfy the demand. The revised bidding process took place in 2007 with the contract signed in early 2008. Over the life of the project a strong partnership was developed between SEAMA/IEMA and CESAN. Of particular note was the improvement in the clients data collection and monitoring of indicators. As CESAN's financial situation improved throughout the project, financial risks were gradually mitigated and they could devote more attention to project oversight functions and executing water supply and sewerage activities. (ICR p. 18) Implementing Agency Performance Rating : Satisfactory Overall Borrower Performance Rating : Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: Twenty-five Indicators to monitor progress toward achieving the project's objectives were agreed to in an exchange of letters between the State and the Bank and were similar to the ones used in the original 1994 loan. Care was taken to ensure that the indicators had technical, operational, environmental and financial meaning for the utility (and not only the project) (ICR p. 6). Among the indicators identified were: Water treatment capacity, water connections as % of total sewerage treatment capacity, and sewerage connections (physical): Water coverage levels, water connections with meters, unaccounted for water (operational); efficiency of sewerage treatment plants, number of water quality monitoring stations, and the pollution load control for existing industries (environmental); and average water tariff, average sewerage tariff, working ration and rate of return (financial). These indicators were appropriate for this type of investment. Baseline data was collected on over 40 indicators prior to project initiation in 2005 and covered a broad range of performance measures including: (i) physical; (ii) social; (iii) operational; (iv) environmental; (v) financial; and (vi) service quality and operational performance (GVMR) indicators. The ICR provided detailed reporting on performance relative to the baseline data over the 2005-2011 time frame and included specific "target" values. One aspect of M&E design which could have been further developed relates to the the project objective of providing water and sanitation infrastructure to low-income urban areas. It is difficult to determine which outputs and outcomes pertained to these areas. At the time of the appraisal of the previous loan, the Bank had not adopted the Results Framework as a tool for designing and implementing M&E. b. M&E Implementation: During implementation, an additional 16 indicators were added following the completion of CESAN’s benchmarking. These indicators were used to monitor project targets and long-term benefits. The collection and monitoring of the data was coordinated by CESAN and IEMA. CESAN was responsible for reporting every six months on the indicators related to the company’s operational performance and on the quality of service provision, while IEMA monitored water quality and coastal pollution, on a weekly basis, through a water quality sampling network in the main hydrographic basins of the state and a coastal pollution sampling network on 46 beaches along the state coast (totaling 71 sampling locations). Results of this analysis are posted on the internet on a monthly basis in a friendly format to inform the population about water quality on each beach, as measured by the coliform index. (ICR p. 6) c. M&E Utilization: An important element of M&E utilization during the project was the fact that CESAN internalized and refined the larger set of indicators described above for its own purposes, improving the utility’s decision-making and resource allocation processes. The internalization and appropriation of these indicators ensure the sustainability of these M&E arrangements beyond the project’s implementation period. Furthermore, CESAN published its results state-wide twice a year to enhance accountability vis-à-vis the public. The Bank’s project team followed up during supervision on M&E utilization. M&E Quality Rating : Substantial 11. Other Issues a. Safeguards: The previous project was classified as Category B under the Bank’s environmental guidelines and an Environmental Impact Assessment (EIA) was carried out at the time of appraisal. For the activities to be financed under the proposed additional loan, an updated Program Environmental Assessment and an updated Environmental Management Plan were prepared and discussed in public consultations in Espirito Santo (MOPTA p15). The subject project (7248) was reviewed by by Bank environmental and social safeguards specialists and was classified as Environmental Category B. In accordance with the OP 4.01, Environmental Assessments were carried out in 2004 and 2008 as a result of the modification in the design of the sewerage component. An Environmental Management Plan was also prepared which included the preparation of an environmental construction manual. This manual was included in the bidding process for all construction works, and the adoption of the adequate environmental practices established in the manual was deemed a contractual obligation to be fulfilled by contractors (ICR p. 7). The US$71.5 million additional financing activities (7655)were reviewed by the Bank Safeguard Advisory Team, which confirmed the Environmental Category B classification for this project. During the environmental licensing process of the original Mulembá WWTP, an agreement was signed between the State Government and the Paneleiras Association guaranteeing that the Paneleiras would have access to the site, and that their traditional activities would not be impacted or affected by the construction. This 2001 agreement already included the area for expansion of the WWTP, and among other points, established that the clay material existing in the area of construction of the WWTP would be removed, stored and put at the disposal of the Paneleiras (ICR p.7). b. Fiduciary Compliance: Financial management arrangements were put in place to ensure proper use and accounting of funds. Financial Monitoring Reports were submitted on a quarterly basis, and audits for both the Project and CESAN were carried out on an annual basis and timely submitted to the Bank for review and comment. All audits were unqualified. All financial covenants have been complied with. c. Unintended Impacts (positive or negative): Particularly noteworthy are CESAN's financial, commercial and institutional results, because of their positive impact on the company’s long-term sustainability. Significant institutional reforms implemented in parallel to the project include (i) the distribution since 2006 of dividends to the utility's shareholders; (ii) the definition and implementation of a Code of Ethics and a performance-based bonus program to complement salaries based on the achievement of results that are established and monitored using the performance benchmarking system; and (iii) the universal application of the simultaneous reading and billing of customers. CESAN also made tremendous institutional progress with regards to transparent human resource practices, as its top level management positions (including Board members) are appointed on technical rather than political grounds. It is also to be noted that, on the institutional side, the Espírito Santo State water and sanitation policy evolved during the project with the approval of Law 477 in 2008, which created the State’s regulatory agency for water and sanitation (ARSI). d. Other: 12. Ratings : 12. ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Risk to Development Negligible to Low Negligible to Low Outcome : Bank Performance : Satisfactory Satisfactory Borrower Performance : Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The ICR lists 5 lessons learned of which the first two lessons, Integration in Implementation Arrangements and Environmental and Social Sustainability are not lessons but findings. The third lesson " Turnkey Contract Approach" suggests this approach is superior to using individual contracts. The allocation of a larger part of the responsibilities onto the contracted firm along with a certain technical autonomy can be attractive to firms of quality with proven capacity to tolerate risks, to furnish performance securities as well as to comply with the established targets; and this type of contract can also lead to faster execution of works, particularly in terms of resolving implementation problems: the contract model provides an incentive to the firm to resolve issues rapidly, and fewer contract amendments are necessary. (ICR p 19) The fourth lesson " Connecting households to the sewerage network" suggests that connections to sewerage networks should be included in the works contracts that are building the network, and related social and communication activities should be put in place before and during construction, as well as when collection and treatment works are initiated. The lack of up-front buy-in from communities increases the chances of a low rate of connection to the sewerage network. The fifth lesson "Creating a sustainable M&E system" suggests that Project financial and monitoring indicators should have meaning not only for Bank reporting purposes, but for the utility and implementing agencies themselves. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR is generally well-written and comprehensive. The ICR covered with adequate detail all sections of the template. The quality of the evidence was good as was the analysis. The ICR covered all the pertinent issues regarding the preparation of Loan 7248 and its relationship to the previous Loan 3767. This was important give the fact that the new loan adopted the original project's objectives. The ICR adequately analyzed the weaknesses in both design and implementation of the original loan. The new loan was approved during a time when additional emphasis was being placed on environment and water resources management and project activities reflected that need. The ICR also covered the challenges faced by the Sewerage Component and revisions to that component (including additional financing under Loan 7655) which came about as a result of the changes in the engineering design. The ICR addressed the progress made with the rate of connection to the sewerage network but also highlighted the remaining works which needed to be completed in order to achieve the projects connection goals. The ICR provided a detailed assessment of both project outputs and outcomes. It covered in detail the Post- Completion Operations highlighting the important institutional development of CESAN, SEAMA and IEMA and the roles each would play in the States water and sewerage management. It noted that CESAN is planning investments of about R$1 billion which is an indicator of how far they have come since the original loan was approved. Poverty and gender issues were briefly covered including specifying outcomes in low-income neighborhoods. However, details on specific outputs in low-income areas was difficult to determine as evidence seemed to be tied to components which didn't explain to what extent the outputs were in these areas. Finally, the ICR noted that water quality is deteriorating adding an increased burden on treatment plants which could provide a future opportunity for Bank collaboration. a.Quality of ICR Rating : Satisfactory