Document of The World Bank Report No: ICR0000719 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-45320) ON A LOAN IN THE AMOUNT OF US$ 30.3 MILLION TO THE FEDERATIVE REPUBLIC OF BRAZIL FOR THE LOW INCOME SANITATION TECHNICAL ASSISTANCE PROJECT PROSANEAR-TAL June 9, 2008 Sustainable Development Department Brazil Country Management Unit Latin America and the Caribbean Region i CURRENCY EQUIVALENTS (Exchange Rate Effective 05/15/2008) Currency Unit = Brazilian Real (R$) BR$ 1.00 = US$ 0.60 US$ 1.00 = BR$1.6663 FISCAL YEAR January 1 - December 31 ABBREVIATIONS AND ACRONYMS CAS Country Assistance Strategy CCT Conditional Cash Transfer CEF Caixa Econômica Federal (federal investment and development bank) CPS Country Partnership Strategy EIA Environmental Impact Assessment FGTS Fundo de Garantia de Tempo de Serviço (federal employee indemnity fund) FM Financial management GoB Government of Brazil IBRD International Bank for Reconstruction and Development ICR Implementation Completion Report M&E Monitoring and Evaluation MDGs Millennium Development Goals MoC Ministry of Cities (Ministério das Cidades) MTR Mid Term Review NGO Non-governmental Organization O&M Operation and Maintenance PAC Programa para Aceleração do Crescimento (Program for the Acceleration of Economic Growth) PAD Project Appraisal Document PDLI Plano de Desenvolvimento Local Integrado (integrated local development plan) PDO Project Development Objective PMF Project Management Firm PMU Project Management Unit PRODIN Projetos de Desenvolvimento Institucional Municipal (municipal institutional development projects) PROSANEAR Programa de Saneamento para Populações em Áreas de Baixa Renda (WSS Program for Low Income Areas) PROSANEAR-TAL Low income sanitation technical assistance project PSI Projeto de Saneamento Integrado (integrated water supply and sanitation engineering design) RAP Resettlement Action Plan ii SEDU/PR Secretaria Especial de Desenvolvimento Urbano da Presidência da República (Special Secretariat for Urban Development of the President's Office) SIL Sector Investment Loan SNIS Sistema Nacional de Indicadores de Saneamento (National System for Water Supply and Sanitation Indicators) SNIU Sistema Nacional de Indicadores Urbanos (National System of Urban Indicators) SNPU Secretaria Nacional de Programas Urbanas (National Secretariat for Urban Programs) SNSA Secretaria Nacional de Saneamento Ambiental (National Secretariat for Water Supply and Sanitation) TA Technical Assistance TAL Technical Assistance Loan TOR Terms of Reference WSS Water Supply and Sanitation Vice President: Pamela Cox Country Director: John Briscoe Sector Manager: Guang Zhe Chen Project Team Leader Martin Gambrill ICR Team Leader: Caroline van den Berg iii CONTENTS Page A. Basic Information............................................................................................................ i B. Key Dates........................................................................................................................ i C. Ratings Summary............................................................................................................ i D. Sector and Theme Codes................................................................................................ii E. Bank Staff.......................................................................................................................ii F. Results Framework Analysis..........................................................................................ii G. Ratings of Project Performance in ISRs ....................................................................... vi H. Restructuring (if any).................................................................................................... vi I. Disbursement Profile.....................................................................................................vii 1. Project Context, Development Objectives and Design................................................... 1 2. Key Factors Affecting Implementation and Outcomes .................................................. 6 3. Assessment of Outcomes.............................................................................................. 11 4. Assessment of Risk to Development Outcome............................................................. 22 5. Assessment of Bank and Borrower Performance ......................................................... 23 6. Lessons Learned............................................................................................................ 26 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners............... 30 Annex 1. Project Costs and Financing.............................................................................. 39 Annex 2. Outputs by Component...................................................................................... 40 Annex 3. Bank Lending and Implementation Support/Supervision Processes................. 45 Annex 4. Beneficiary Survey Results............................................................................... 47 Annex 5. Institutional or Stakeholder Survey Results...................................................... 57 Annex 6. Comments of Cofinanciers and Other Partners/Stakeholders........................... 60 Annex 7. List of Supporting Documents .......................................................................... 61 MAP.................................................................................................................................. 62 iv A. Basic Information Low Income Sanitation Technical Assistance Country: Brazil Project Name: Project - PROSANEAR-TAL Project ID: P039199 L/C/TF Number(s): IBRD-45320 ICR Date: 06/08/2008 ICR Type: Core ICR Lending Instrument: TAL Borrower: GOVT OF BRAZIL Original Total USD 30.3M Disbursed Amount: USD 16.7M Commitment: Environmental Category: B Implementing Agencies: Ministry of Cities Beneficiary municipalities Cofinanciers and Other External Partners: none B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 03/05/1998 Effectiveness: 12/19/2000 12/19/2000 Appraisal: 05/19/1998 Restructuring(s): Approval: 01/06/2000 Mid-term Review: 05/07/2001 06/01/2004 Closing: 12/31/2004 12/15/2007 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Moderately Unsatisfactory Government: Unsatisfactory Quality of Supervision: Moderately SatisfactoryImplementing Agency/Agencies: Moderately Satisfactory Overall Bank Overall Borrower Performance: Moderately SatisfactoryPerformance: Moderately Satisfactory i C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Performance Indicators (if any) Rating Potential Problem Project No Quality at Entry None at any time (Yes/No): (QEA): Problem Project at any Quality of Yes None time (Yes/No): Supervision (QSA): DO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 5 19 General water, sanitation and flood protection sector 18 16 Other social services 3 Sub-national government administration 74 65 Theme Code (Primary/Secondary) Access to urban services and housing Primary Primary Municipal governance and institution building Primary Primary Participation and civic engagement Secondary Secondary Pollution management and environmental health Primary Secondary E. Bank Staff Positions At ICR At Approval Vice President: Pamela Cox Shahid Javed Burki Country Director: John Briscoe Gobind T. Nankani Sector Manager: Guang Zhe Chen Abel Mejia Project Team Leader: Martin Gambrill Yoko Katakura ICR Team Leader: Caroline van den Berg ICR Primary Author: Caroline van den Berg F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) To achieve integrated and demand-driven water supply and sanitation (WSS) service delivery within participating local government agencies under the framework of the PROSANEAR national low-income sanitation program. ii Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s) Original Target Formally Actual Value Indicator Baseline Value Values (from Revised Achieved at approval Target Completion or documents) Values Target Years Indicator 1: Number of participating local government agencies that adopt and implement a participatory approach to WSS service delivery Value No target (quantitative or 0 established in 30 qualitative) PAD Date achieved 12/09/1999 12/09/1999 04/15/2008 Comments April 15, 2008, was the last day on which disbursements could be made under (incl. % the project and is the date used for all of the 'Actual Value Achieved at achievement) Completion' measurements Indicator 2: Number of participating local government agencies which undertake integrated urban planning as part of WSS service delivery Value No target (quantitative or 0 established in 30 qualitative) PAD Date achieved 12/09/1999 12/09/1999 04/15/2008 Comments (incl. % achievement) Indicator 3: Number of participating local government agencies that implement a clear and transparent cost recovery / subsidy policy for WSS service delivery Value No target (quantitative or 0 established in Not achieved qualitative) PAD Date achieved 12/09/1999 12/09/1999 04/15/2008 Comments In two state water companies ­ those in Bahia and Pernambuco where numerous (incl. % subproject activities were undertaken ­ clear and transparent cost recovery / achievement) subsidy policies were adopted during the project implementation phase. It is not possible, however, to attribute these changes to the PROSANEAR-TAL Indicator 4: Number of participating local government agencies that establish a multi- disciplinary team to implement low income WSS projects Value No target (quantitative or 0 established in 30 qualitative) PAD Date achieved 12/09/1999 12/09/1999 04/15/2008 Comments (incl. % achievement) iii Indicator 5: Per beneficiary investment costs of less that US$100 for water (US$70 if only distribution and storage) and US$120 for sewerage (US$80 for collection only) of TAL-supported investment programs Value (quantitative or 0 See per capita Substantially qualitative) costs above achieved Date achieved 12/09/1999 12/09/1999 04/15/2008 Comments The target per capita costs of a large sample of subprojects were achieved for (incl. % water supply and for wastewater treatment; the per capita target costs were not achievement) achieved, however, in the case of wastewater collection Indicator 6: Level of awareness and satisfaction among municipal officials, community groups, federal and state officials regarding the PROSANEAR approach Value No target (quantitative or 0 established in 95% qualitative) PAD Date achieved 12/09/1999 12/09/1999 04/15/2008 A stakeholder satisfaction survey showed 95 percent of participating Comments municipalities interested in taking part in the preparation of participatory, (incl. % integrated WSS/urban upgrading projects with federal TA support similar to that achievement) provided under the PROSANEAR-TAL. The survey also registered a satisfaction rate of over 50 percent for all of the main actors involved in the project's implementation. (b) Intermediate Outcome Indicator(s) Original Target Formally Actual Value Indicator Baseline Value Values (from Achieved at approval Revised Completion or documents) Target Values Target Years Indicator 1: Number of grant agreements signed with local governments Value (quantitative or 0 30 33 qualitative) Date achieved 12/09/1999 12/09/1999 04/15/2008 Comments (incl. % The Municipality of São Paulo/SP signed 5 grant agreements; the Municipality achievement) of Olinda/PE signed two grant agreements Indicator 2: Number of key national studies undertaken Value (quantitative or 0 4 5 qualitative) Date achieved 12/09/1999 12/09/1999 04/15/2008 Comments (incl. % achievement) iv Indicator 3: Number of national/international seminars undertaken Value (quantitative or 0 5 5 qualitative) Date achieved 12/09/1999 12/09/1999 04/15/2008 Comments (incl. % achievement) Indicator 4: Number of PDLIs (participatory, integrated WSS plans) completed Value (quantitative or 0 30 34 qualitative) Date achieved 12/09/1999 12/09/1999 04/15/2008 Comments (incl. % achievement) Indicator 5: Number of PSIs (participatory, integrated WSS engineering designs) completed Value (quantitative or 0 30 33 qualitative) Date achieved 12/09/1999 12/09/1999 04/15/2008 Comments (incl. % achievement) Indicator 6: Training courses undertaken for local (municipal) PMUs and CEF offices Value (quantitative or 0 64 120 109 qualitative) Date achieved 12/09/1999 12/09/1999 06/01/2004 04/15/2008 Comments (incl. % achievement) Indicator 7: Demonstration civil works interventions concluded Value (quantitative or 0 0 2 1 qualitative) Date achieved 12/09/1999 12/09/1999 06/01/2004 04/15/2008 Comments By project close, the civil works intervention in São José dos Campos was (incl. % concluded; the CW intervention in Guarulhos was substantially advanced but, achievement) given its complexity and size, required additional time to be complete ­ for which PAC funds was used v G. Ratings of Project Performance in ISRs Actual No. Date ISR Archived DO IP Disbursements (USD millions) 1 04/13/2000 Satisfactory Satisfactory 0.00 2 06/29/2000 Satisfactory Satisfactory 0.00 3 12/19/2000 Satisfactory Satisfactory 0.00 4 06/28/2001 Satisfactory Satisfactory 0.30 5 11/24/2001 Satisfactory Satisfactory 0.70 6 05/30/2002 Satisfactory Satisfactory 0.70 7 08/21/2002 Satisfactory Satisfactory 0.70 8 09/05/2002 Satisfactory Satisfactory 0.70 9 03/13/2003 Satisfactory Satisfactory 0.70 10 11/10/2003 Unsatisfactory Unsatisfactory 1.09 11 06/04/2004 Unsatisfactory Satisfactory 1.61 12 06/16/2004 Unsatisfactory Unsatisfactory 1.61 13 09/24/2004 Satisfactory Satisfactory 1.61 14 05/09/2005 Satisfactory Satisfactory 1.61 15 06/27/2005 Moderately Moderately Unsatisfactory Unsatisfactory 1.61 16 11/18/2005 Satisfactory Satisfactory 5.62 17 06/05/2006 Satisfactory Satisfactory 7.25 18 12/22/2006 Satisfactory Satisfactory 10.28 19 06/25/2007 Satisfactory Moderately Unsatisfactory 11.40 20 12/06/2007 Satisfactory Moderately Satisfactory 13.15 H. Restructuring (if any) ISR Ratings at Amount Restructuring Board Restructuring Disbursed at Reason for Restructuring & Date(s) Approved Restructuring PDO Change Key Changes Made DO IP in USD millions The project was not formally restructured. A demonstration 10/26/2004 N S S 1.61 civil works component was introduced at the Mid Term Review, but the PDO was not changed vi I. Disbursement Profile vii 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal At the time of appraisal, the main issues facing the water supply and sanitation (WSS) sector in Brazil were grouped into three main challenges: (i) a weak institutional and regulatory framework; (ii) a lack of incentives, mechanisms and capacity to serve the poor; and (iii) ineffective water pollution control policies. All three were seen as contributing to the dismal state of service coverage to the urban poor at that time. At appraisal, over 19 million urban poor earning less than three minimum salaries per family per month (then equivalent to about US$330) lived without safe water supply and over 34 million urban poor without adequate sanitation. More specific sector concerns within these three areas were as described below: Weak institutional and regulatory framework. In the absence of competition and regulation, and nurtured by old labor practices and frequent political intervention, Brazilian public water utilities had, at the time of appraisal, generally become inefficient, stagnant and possessed limited financial capacity. Although the 1995 Concession Law had initiated country-wide efforts to seek alternative sector institutional and regulatory models, sector reform was still at a very early stage, with the urban poor then being among the main victims of the mediocre sector performance. Lack of incentives, mechanisms and sector capacity to serve the poor. Extension of services to the poor was often the last priority of cash-strapped water utilities. Prevailing tariff structures and non-transparent cost recovery and subsidy policies provided little incentives to investing in poor neighborhoods. Furthermore, the existing concession contracts contained no incentive mechanism or clear obligations to extend services to the poor. In addition to the lack of institutional incentives to serve the poor, there was also a lack of mechanisms and capacity to extend such services. The PAD recognized that the extension of services to poor urban neighborhoods required non-conventional approaches, including the use of low-cost appropriate technology and active community participation. Experience showed that investments in poor neighborhoods carried out in a supply-driven manner were not sustainable. With no participation in the decision-making process and little consideration on cost recovery, communities got little sense of ownership, and the WSS infrastructure would quickly deteriorate. Conventional WSS systems were seen as often not being suitable in disorganized and often difficult to access favela situations. Nonetheless, only a limited number of utilities and government agencies were seen as having the capacity and knowledge to prepare and effectively implement WSS services for the urban poor. Municipal governments were also seen as lacking the capacity to plan, finance and coordinate the delivery of complementary services that are critical to the sustainability of WSS services in low income areas. Ineffective pollution control policies. Practically all major and medium sized metropolitan areas in Brazil were facing increasing water pollution problems, with domestic and industrial wastewater being dumped into rivers, lakes and the sea with little control. The urban poor were seen as being among the main victims of this practice, suffering the most from poor health and related high costs associated with increasing water pollution. 1 The strategy of the Government of Brazil (GoB) to address these issues was summarized at the time of appraisal in the National Water and Sanitation Policy developed by the Ministry of Planning's Secretariat of Urban Development. The strategy was designed to address the three issues in parallel, without undermining the overarching objective of sector reform. While striving to improve the overall performance of the sector through institutional and regulatory reform in the medium-term, GoB decided to simultaneously address the issue of WSS services for the urban poor. The federally funded PROSANEAR program (Programa de Saneamento para Populações em Áreas de Baixa Renda) was created in 1995 and used subsidized credit from the federal employee indemnity fund (FGTS) to invest in WSS services in peri-urban areas. The performance of the program at the time of appraisal had been disappointing, due to: (i) a lack of institutional capacity in preparing and implementing low-income WSS projects; (ii) a lack of awareness and knowledge about GoB's PROSANEAR program; (iii) unclear incentive mechanisms provided to participating water companies; (iv) weak financial and institutional capacity of municipalities in obtaining credits and undertaking complementary investments such as drainage and solid waste management; and (v) the lessons from the Bank-financed PROSANEAR-I had not been fully incorporated into the national investment program. In addition, with the devolution of authority to municipalities under the 1988 Constitution, the respective roles of municipalities, states and of the federal government in urban policy and urban development became unclear. State and federal urban interventions were seen as ad hoc in nature, often with limited participation and ownership by the municipalities and with little coordination with other municipal programs and services. GoB requested the Bank's assistance to revitalize the national program through the PROSANEAR-TAL, which was thus conceived to: (i) provide technical assistance (TA) to prepare a stock of subprojects to be financed by the PROSANEAR national program; (ii) undertake training programs to strengthen the capacity of project executing agencies in preparing and implementing such subprojects; (iii) disseminate initiatives to raise awareness of PROSANEAR; (iv) develop action plans to strengthen institutional and financial capacity of municipalities to make PROSANEAR more sustainable; and (v) provide TA to develop federal level urban strategies. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The project aimed to achieve integrated and demand-driven water supply and sanitation service delivery within participating local government agencies under the framework of the PROSANEAR national low-income sanitation program. The key performance indicators presented in the PAD were the following: · Number of participating local government agencies that adopt and implement a participatory approach to WSS service delivery; · Number of participating local government agencies which undertake integrated urban planning as part of WSS service delivery; · Number of participating local government agencies that implement a clear and transparent cost-recovery/subsidy policy for WSS service delivery; 2 · Number of participating local government agencies that establish a multi- disciplinary team to implement low income sanitation projects; · Per beneficiary investment costs of less than US$100 for water (US$70 if only for distribution and storage) and US$120 for sewerage (US$80 if for collection only) of TAL-supported investment programs; and · Level of awareness and satisfaction among municipal officials, community groups, federal and state officials regarding the PROSANEAR project approach. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification. Neither the PDO nor its key indicators were revised during project implementation. 1.4 Main Beneficiaries The project was expected to benefit two key target groups: the first group directly through the TAL with institutional development, and the second group indirectly resulting from the TAL through the extension of basic services. Institutional development. The TAL was designed to benefit the agencies responsible for policy formulation, planning, implementing and monitoring WSS projects for the urban poor, including: (i) the Special Secretariat of Urban Development of GoB's Presidency of the Republic (SEDU/PR) which set the policy framework for extending basic urban services to the poor and was the overall coordinator and interlocutor with the Bank regarding the PROSANEAR-TAL1; (ii) Caixa Econômica Federal (CEF), the federal investment and development bank, which acted as the financial and technical agency responsible for subproject approval and supervision and for the making of payments under PROSANEAR-TAL; (iii) local governments and water agencies responsible for subproject preparation and implementation; and (iv) non-governmental organizations (NGOs), consulting firms and constructors which implement subprojects at the community level. Extension of WSS services. The target beneficiaries for the subprojects that were funded under the TAL were the low income populations which lived in densely populated urban and peri-urban poverty pockets situated in municipalities with populations of more than 75,000 in metropolitan areas, and which earned on average less than three minimum salaries per family per month. The focus on metropolitan areas and the population size limit was intended to maximize the development impact of PROSANEAR-TAL since: (i) many of the poor lived in large low-income settlements; (ii) the TAL was intended to have a demonstration impact, aimed at creating a `ripple effect' in neighboring municipalities; (iii) the TAL would create institutional capacity to allow the subsequent undertaking of other subprojects; and (iv) small municipalities were then covered by other federal, poverty-targeted programs. 1Following the election of Luiz Inácio Lula de Silva as President of the Republic in November 2002, and his taking office in January 2003, his administration dissolved SEDU/PR and replaced it with a new ministry, the Ministry of Cities, which includes, amongst others, a Secretariat for Water Supply and Sanitation and a Secretariat for Urban Programs. 3 The two main groups described above were successfully targeted by the project's interventions. The project's impact on these two target groups was analyzed using (i) stakeholder surveys with a sample of the municipal governments directly involved in the PROSANEAR-TAL implementation and (ii) beneficiary assessments in two of the communities which benefited from the demonstration works interventions ­ see Annex 4 and 5 for more details on these assessments. 1.5 Original Components (as approved) The project originally consisted of four components: Component 1 ­ Project Management, Promotion and Studies. The TA provided under this component was designed to include: (i) consultants and monitoring and evaluation system for project management, knowledge management and administrative/logistics support; (ii) promotion and dissemination of best-practice approaches to providing sustainable WSS services to the peri-urban poor, including seminars and workshops to promote broad participation of stakeholders; and (iii) studies on key issues related to the challenges of providing sustainable WSS services to the peri-urban poor. Component 2 ­ Pre-Investment. The main objective of this component was to develop the institutional capacity of the local government agencies in identifying and preparing subprojects to extend services to the urban poor, by supporting the preparation phase of investment subprojects to be financed under the PROSANEAR national program. Specific activities intended to be financed under this component included: (i) socio- economic baseline surveys of beneficiary communities; (ii) elaboration of integrated urban local development plans (PDLIs) to facilitate PROSANEAR's coordination with other urban services; (iii) preliminary/final engineering designs of integrated WSS systems (PSIs), to be developed with community participation; (iv) community participation plans (PTSs) to be implemented before, during and after investment subproject execution; and (v) social tariff and subsidy policy review study (upon request by the executing agencies). Component 3 ­ Training. This component was designed to include training activities for: (i) water utilities and other executing agencies on the preparation and implementation of subprojects; (ii) Instâncias Colegiadas (state level committees which were originally used to select PROSANEAR subprojects based on guidelines set by SEDU/PR) on the selection of subprojects; (iii) local governments on the design of cost recovery policies (subsidy and social tariff policies), elaboration of PDLIs , and identification and selection of subproject areas; (iv) CEF's regional offices on the evaluation of all subproject documents submitted by the executing agencies; and (v) NGOs, consultants and constructors on the preparation and implementation of PROSANEAR subprojects. Component 4 ­ Urban Development Policy. The TA under this component was designed to strengthen the policy formulation capacity at the federal and local government levels, including issues related to ­ but not limited to ­ PROSANEAR, through: (i) local (municipal) capacity building activities; and (ii) the development of a national urban indicator system and development policy. 4 1.6 Revised Components Following the mid term review (MTR), and at the request of the Borrower, the project was reformulated in October 2004. The reformulation included: (i) the cancellation of US$6.4 million (effective January 2004, and requested by GoB given the valuation of the dollar and the protracted disbursement delays that the project had demonstrated to that point); (ii) an extension to the closing date (one year initially and a second year contingent on the attainment of physical and financial targets established during the MTR); and (iii) the inclusion of a pilot investment civil works component to allow the execution of demonstrative integrated WSS/urban upgrading improvements in a small, selected number of strategic, poor, slum areas. While the project did not originally include civil works, the pilot works component was designed to allow the project to finalize the pre-investment subproject cycle in up to four beneficiary municipalities, and thus permit the project to fully accompany ­ and provide TA to ­ these municipalities throughout the integrated and participatory approach to both planning and executing the resultant works. The new component was thus intended to permit the project to provide these municipalities with additional TA support during the works interventions, and afterwards when measuring the subproject's post-works impact and outcome indicators. President Lula's newly created Ministry of Cities (MoC), on initiating its mandate in January 2003, was adamant on the inclusion of the demonstration works component ­ both to be able to demonstrate more directly the PROSANEAR-TAL approach (at a time when federal funding for such investments was scarce), and to offset the increased value of the dollar against the local currency at that time (fewer dollars were needed to undertake the pre-investment TA and so there was an excess of loan proceeds). Following the creation of MoC and the establishment of its National Secretariat for Urban Programs (SNPU), the activities foreseen under Component 4 were removed from the project scope given that the same activities were priorities of the new secretariat, which subsequently undertook them with MoC resources. At the request of MoC, a small amount of resources was left under this component following the MTR to allow for the undertaking of possible activities related to urban development policy should these have subsequently arisen. 1.7 Other significant changes The PROSANEAR national low-income sanitation program, which was explicitly referred to in the project's PDO, was conceived to provide subsidized financing in water supply, wastewater, solid waste and drainage interventions in informal settlements through CEF/FGTS subsidized loans. PROSANEAR/FGTS was, however, replaced in 2003 by the national `Saneamento Integrado' program which had similar rules but a broader scope of eligible expenditures (including resettlement costs, for example). Saneamento Integrado itself became one of the modalities of the Lula administration's Program for the Acceleration of Economic Growth (PAC) ­ providing funding for integrated WSS/urban upgrading interventions through either FGTS loans or federal budget grant allocations. Consequently, PROSANEAR-TAL's subproject designs migrated over time from their original intention of leveraging PROSANEAR/FGTS 5 financing, to their current role of leveraging millions of dollars of investments under the PAC's Saneamento Integrado subsidized lending and grant program. The project's federal counterparts in the entity responsible for WSS ­ formerly SEDU/PR and now the National Secretariat of Environmental Water Supply and Sanitation (SNSA) of MoC ­ changed four times over the implementation period. Each change in counterparts bringing challenges to the level of prioritization and the rhythm of implementation Another important change to the project occurred in the final months of 2006 when MoC decided to demobilize the project management firm (PMF) which, up to that date and since 2002, had been provided technical support to MoC's project management unit (PMU). The PMF had been contracted in 2002 following Bank procurement guidelines, including the establishment of a short-list with international representation given the estimated size of the contract. The contract was subsequently extended in 2004, but with no change to the original value of the contract. However, when the next time-extension of the contract was requested in July 2006, MoC's recently appointed legal advisers within the Minister's office decided that the PMF's contract and modus operandi, and the underlying contract between MoC and CEF (through which the PMF had been contracted), were not acceptable. Considerable effort was subsequently made by SNSA and the Bank to reverse this decision, but to no avail. The sudden removal of the PMF, and of the TA functions that it had previously been providing, caused considerable disruption to the project's implementation progress during the all-important final year of implementation ­ especially given how well project execution had been proceeding since the MTR. The beneficiary municipalities suffered most from this change and lamented the departure of the PMF as it withdrew a solid base of technical and procedural support that MoC was unprepared to provide once the firm departed. Given that MoC was newly-created and that competent in-house staff was scarce, and given that GoB's newly-announced PAC was simultaneously demanding considerable attention from SNSA, MoC was unable to maintain the physical or financial implementation rhythm, nor to deliver on all of the planned PROSANEAR products, during the last year of implementation. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry Lessons learned. A thorough evaluation of the first Bank-financed PROSANEAR investment operation was undertaken as part of the preparation of the PROSANEAR- TAL follow-up project. A number of the lessons learned from the first PROSANEAR were incorporated in the project design, including: (i) a stronger federal government function in management, promotion and training; (ii) decentralized subproject execution, fully utilizing CEF's regional network; (iii) establishment of clearer guidelines to prepare, implement and monitor subprojects; and (iv) greater emphasis on institutional incentives and sustainability of investments. Other lessons referred to in the PAD included the importance of: (i) having transparent cost recovery and subsidy policies for WSS services in peri-urban, informal areas; (ii) integrating WSS service provision with local 6 governments' urban development plans; and (iii) having clear operation and maintenance (O&M) arrangements after systems are implemented. Risks and mitigations. The risks presented in the PAD and their corresponding mitigation measures were reasonably well identified and proved to be relevant. In addition to those identified, the consideration of the following risks could have further improved project design: (i) the weakness of CEF's capacity to undertake Bank procurement and to provide technical support and a review function of subproject products at a number of its decentralized offices; (ii) the implications of the complicated, cumbersome implementation arrangements and their impact on implementation and disbursement schedules; (iii) the great variability of the skill profiles of consulting firms around the country to effectively bring together the technical and social aspects of such participatory, integrated engineering designs; (iv) the stop/start nature of federal budget allocations (with regular freezing of budget payments by MoC) on the pace of implementation, especially as it negatively impacts the ability of consulting firms to stay mobilized for weeks or months on end without receiving payment; and (v) the ability of the project to influence social tariffs/subsidy policies of the WSS service providers. Participation process. Civil society/community participation processes were built in as a central element of the participatory approach to the pre-investment designs ­ in the PAD, in the Operational Manual and in the standard terms of reference (TOR) for the participatory preparation of the integrated WSS/urban upgrading subprojects. 2.2 Implementation Project implementation followed a bumpy course. GoB brought about delays in declaring the project's effectiveness, MoC in mobilizing the PMF to support the PMU, and even in undertaking the MTR (which, at GoB's request, was carried out during three separate missions staggered over a number of months). As mentioned above, implementation effectiveness was also directly affected by the numerous political and administrative changes in the federal counterpart entity and in the municipal executing agencies over the project life. The embracing of the project by the Lula administration, the mobilization of the PMF, and the ensuing MTR, came together to represent a watershed in the project's implementation. As a result of the MTR, the project was reformulated, including a partial cancellation, the establishment of an action plan with physical and financial targets to be used to make decisions on closing date extensions; and the reconfiguration of the project components, including reallocation of funds amongst them and the inclusion of a pilot civil works investment component. The impacts of this turnaround can be seen in the resulting sharp increase in the disbursement profile during the second half of project implementation. Much time was lost during implementation by having the beneficiary municipal governments undertake the pre-investment procurement, in a decentralized fashion, as recommended by one of the PROSANEAR I lessons learned: the municipalities' inexperience with Bank procurement guidelines required much hand-holding by MoC's PMU. Considerable implementation momentum was also lost due to the perennial freezing (`contingenciamento') of the MoC budget, thereby impeding payments to be made by MoC, via CEF, to consulting firms, and by the fact that municipalities were 7 often ineligible to receive federal transfers (including PROSANEAR-TAL payments) whenever they were in breach of any single element of the extensive requirements of the Fiscal Responsibility Law ­ no matter how small or unrelated the issue might be. These factors contributed to a stop/start rhythm of implementation which, on average, drew out to a considerable degree the overall execution period of the respective pre-investment consultancy assignments. Once the project had finally achieved an important implementation momentum, political changes within MoC resulted in the demobilization of the PMF with over a year of execution still remaining. This had a significant negative impact on implementation progress in the crucial final months before project closing, as well as on the ability of MoC, with limited in-house resources, to finalize in a timely fashion a number of the project's key products, to prepare properly for the final evaluation report, and to successfully disburse from the project account for all of the activities executed before the closing date. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization M&E Design. The project's monitoring and evaluation (M&E) design was undertaken before the Bank initiated a more robust approach to results monitoring. For this reason, applying current standards for M&E to the project ex-post is likely to expose a large number of inadequacies in the project's original M&E design, including the following: o The PDO was poorly defined and the overall intention of the project seems to be beyond the scope of what a TA operation can realistically deliver on. o The outcomes of the project could only be inferred from the key performance indicators which measure changes in: (i) several types of behaviors by local governments; (ii) a reduction of per capita WSS investment costs and improved tariff and subsidy policies of service providers; and (iii) a change in awareness and satisfaction from various sector stakeholders. o The actual outcome indicators could only be measured at the end of the project as they require perception surveys and detailed assessments of the costs of the pre-investments. Given the difficulties in implementation, with the PMF demobilized one year before project closing, these surveys and detailed assessments were prepared with great difficulty, finally being delivered only in April 2008. o The design also included a set of indicators to measure the impact of the works to be undertaken as a result of the project's TA ­ the PAD does not, however, describe how, when or by whom these impact indicators would be measured, given that the investment works themselves were not originally intended to be funded under the project. o The PAD's result framework did not include target values for the outcome indicators. o The results framework did not specify any M&E arrangements ­ neither at the project level (not detailing, for example, the roles and responsibilities in M&E, nor the basic reporting mechanisms), nor at the national level (how this 8 information would be used in broader national contexts and how it could support a federal urban monitoring information system). The project was reconfigured at the MTR to better fit within the shifting political currents that emerged after President Lula took office. The reconfiguration impacted the project components, as demonstration civil works interventions were included at GOB's request. MoC and Bank teams included additional monitoring indicators (demonstration civil works concluded) and physical and financial targets to allow closer tracking of project progress over time. The reformulation was not used, however, as an opportunity to revise the results framework more comprehensively in order to address its deficiencies. For example, some indicators related to activities that were no longer included in the reconfigured project design (such as the number of municipalities surveyed for the National Urban Indicators Information System (SNIU), or the number of Municipal Institutional Development Projects (PRODINs) undertaken) were not eliminated from the M&E system during the reformulation. M&E Implementation. Data was collected during project implementation on a regular basis using appropriate methods. As tends to be the approach with projects more widely, more emphasis was given under PROSANEAR-TAL to monitoring and less to evaluation. There are many reasons for this lack of evaluation, but it was not due to a lack of data: baseline surveys were a standard component of the PDLIs, the PSIs and the social participation plans (PTSs) pre-investment methodology undertaken in the municipalities under the project. Such basic data and information were collected using detailed questionnaires that generated many data observations. Due to the project's complex implementation arrangements and the absence of a basic information system accessible to all stakeholders, much of the baseline information, however, remained scattered among the many different stakeholders, buried in reports, and hence not easy to access and analyze. As discussed later, integrated WSS/urban upgrading projects have many intended and unintended outcomes and impacts. Building better information systems that can store the information that is collected to ensure more in-depth analyses of the data is a lesson to be taken from this project. Such information systems would have to be developed up-front to ensure that M&E arrangements are set up in a timely manner, and responsibilities for M&E are clearly defined. M&E Utilization. As a consequence, although detailed data were collected they were not necessarily systematically evaluated. The fact that the PMF was demobilized a year in advance of project closing further complicated MoC's ability to undertake any serious M&E of project indicators towards the end of the implementation period. 2.4 Safeguard and Fiduciary Compliance The project received a Category B safeguards classification, presumably because it was a TAL. Nevertheless, significant potential environmental impacts and resettlement issues arose from the subproject designs prepared under the TAL. Once the demonstration civil works interventions were included in the project scope, the Bank supervision team looked very carefully at the issue of safeguard compliance for those cases where the project would fund the corresponding works. The ensuing intensive interaction with the federal and municipal counterparts was extremely useful to identify weaknesses in the standard PROSANER-TAL TOR and the nature of the environmental assessment and resettlement 9 plan products that were being generated. The municipal government of Guarulhos reported that, although intensive and time consuming, it found this process so helpful that they intend to adopt the Bank's resettlement approach for all municipal urban upgrading interventions. The revised standard PROSANEAR-TAL TOR and manual will take full account of the lessons learned with regard to the adequacy of the safeguard procedures being followed and products being generated. MoC's ability to guide the executing municipalities in undertaking Bank procurement processes improved considerably as project implementation progressed. The PMF was extremely important in improving the quality and consistency with which the municipalities followed the Bank procurement guidelines. The efficiency with which the beneficiary municipalities undertook the procurement of the pre-investment consultancy assignments was greatly helped by: (i) the preparation by the MoC PMU of a standard request for proposal and TOR package; and (ii) the role of the PMF in providing TA to the municipalities on the Bank procurement cycle and in acting as an intermediary between the municipalities and the Bank. In addition, the high level of prior review of these consultancies carried out by the Bank (close to 100 percent), was also an important factor in guaranteeing compliance with the Bank procurement procedures. Initial difficulties with the project's financial management (FM) procedures and systems were corrected with intensive supervision from a Brasília-based Bank FM specialist. When the PMF was demobilized, however, the FM controls deteriorated somewhat since MoC's in-house staff were unprepared to take over this role adequately. Nevertheless, since the project FM system was embedded in GoB's budget system and software, although FM faltered in the wake of the removal of the PMF, the project's FM controls were generally deemed satisfactory. Nevertheless, due to administrative errors by the MoC staff during the last year of implementation, in the absence of PFM support, some US$2.7 million of eligible payments were erroneously not disbursed from the project account ­ had these disbursements (representing over 11 percent of the reformulated loan amount) been made during the last months of implementation/grace period, then the final disbursement figure would have been over 80 percent of the final loan amount. 2.5 Post-completion Operation/Next Phase The PROSANEAR-TAL project had an important impact on GoB's PAC program ­ providing guaranteed investments for the PAC's first generation of interventions and, through PROSANEAR-TAL's standard pre-investment TOR and manual, forming the model on which was based the PAC's own TOR and manual for the preparation of subsequent generations of integrated WSS/urban upgrading subprojects. This leverage and mainstreaming of the project's major products/contributions at the federal level bode well for its legacy. Similarly, the municipal beneficiary governments overwhelming endorsed the PROSANEAR-TAL approach and, in the majority of cases, ascertained that they would mainstream its methodologies within the municipal government for subsequent projects of a similar nature. 10 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Following its approval, the project remained consistent with Bank priorities through subsequent country assistance strategies (CASs). The last CAS, covering 2004-2007 and the final implementation years of the project, focused on supporting long-term actions that would promote poverty reduction and sustainable growth, through activities designed to invest in people (health, education and improved public services and transfers), build social inclusion (empowering people and improving targeting mechanisms), manage natural resources, improve productivity, and stabilize the economy. Improving living conditions ­ especially for the poor ­ and managing urban environmental degradation are key pillars of the PROSANEAR-TAL. The follow-up Country Partnership Strategy (CPS) for 2008-2011 presents several pillars to accelerate economic growth by tackling one of the major causes of slow growth, namely the low quality and quantity of infrastructure. According to the CPS, the Bank's program will continue its support for programmatic and `holistic' urban development programs, while focusing on cities that are central to broader economic growth. The PROSANEAR-TAL project design demonstrates that, despite the delays resulting from the major political and administrative changes that occurred during project implementation, the development goals it prioritized have remained highly relevant. Although the project was prepared in advance of the declaration of the Millennium Development Goals (MDGs), the project is extremely well aligned with global priorities and targets set forth in the MDGs, most notably the targets with regard to improving the access to safe WSS services and to reducing the number of slum dwellers. In 2007, there were still an estimated 18.5 million families living in substandard housing in informal settlements in Brazil. The fight against poverty and inequality, and the inclusion of the socially excluded, through economic development and basic services provision, remain fundamental elements of the key policies of the Lula administration. To this end, GoB has initiated and/or improved in recent years a large number of initiatives, such as the Bolsa Família conditional cash transfer (CCT) program and the PAC, aimed at improving the living conditions of the poor. The PAC is a four-year government investment program designed to stimulate economic growth mainly through an injection of GoB spending in infrastructure projects throughout the country. Two key areas of PAC investment, administered by MoC, are programs for integrated WSS service provision for peri-urban populations and urban/slum upgrading interventions. MoC consequently used the 33 participatory integrated subproject designs undertaken by the PROSANEAR-TAL to leverage some R$ 1 billion of PAC and related funding for investments in integrated WSS and urban upgrading interventions in slum areas throughout the country. 11 3.2 Achievement of Project Development Objectives ICR Rating: Overall achievement of Project Development Objectives is Moderately Satisfactory The Project Development Objective is to achieve integrated and demand-driven water supply and sanitation service delivery to the urban poor within participating local government agencies under the framework of the PROSANEAR national low-income sanitation program. The PDO of the project was poorly framed as it suggested that the project would deliver integrated and demand driven WSS to the urban poor. It is, however, not possible to attribute outcomes such as the use of safe, sustainable water supply or sanitation services, nor physical outputs such as access to water supply or sanitation services, to a TAL intervention. Reconstructing the PDO from the complementary information provided in the PAD suggests that the project in fact aimed to: · Revitalize the national PROSANEAR civil works investment program by addressing the major gaps that existed in the program. To that end, the operation aimed to: (i) provide TA for the preparation o a stock of subprojects, with the corresponding works to be funded through the national program; (ii) undertake training programs to strengthen the capacity of project executing agencies in preparing and implementing such subprojects; and (iii) undertake dissemination activities to raise awareness about the PROSANEAR investment program. · Strengthen the policy formulation, planning and coordination capacity of GoB including: (i) formulation of a national framework for WSS cost recovery and subsidy policies in low-income areas; (ii) introduction of better targeting mechanisms for subsidized investments; (iii) establishment of mechanisms to coordinate complimentary urban infrastructure investments; (iv) establishment of multi-disciplinary teams to implement low-income WSS projects; and (v) increase in community participation. With this reconstructed description of what the project intended to achieve, the PAD's key performance indicators provide a reasonable set of outcome indicators that focus on strengthening municipal capacity to undertake integrated WSS projects and strengthening related policy formulation. With the establishment of this framework, it is possible to infer the outcomes of the project from the PAD's key performance indicators: 1. Number of participating local government agencies that adopt and implement a participatory approach to WSS service delivery In the stakeholder satisfaction survey2 ­ carried out at the end of project implementation with a response rate of 16 municipal government stakeholders of the total of 30 municipalities that participated in PROSANEAR-TAL ­ 95 percent of the respondents stated that they had already adopted, or were planning to adopt, the participatory methods of PROSANEAR-TAL in the preparation of other integrated WSS/urban upgrading projects in poor urban areas. Furthermore, 91 percent of the municipalities surveyed 2See Annex 5 for more details on the stakeholder satisfaction survey. 12 stated that they had already incorporated the methodologies and experiences of PROSANEAR-TAL in the preparation of new integrated WSS/urban upgrading projects in their municipalities. 2. Number of participating local government agencies which undertake integrated urban planning as part of WSS service delivery The stakeholder satisfaction survey found that all the municipalities surveyed had adopted, or intended to adopt, the participatory, integrated WSS and urban planning methodology in the preparation of integrated WSS/urban upgrading projects in low- income urban areas. Follow-up questions in the survey showed that 81 percent of the municipalities were already preparing additional integrated WSS/urban upgrading projects to be funded under the government's new PAC program. 3. Number of participating local government agencies that implement a clear and transparent cost recovery and subsidy policy for WSS service delivery During project implementation it became evident that setting up clear and transparent cost recovery and subsidy policies for WSS service delivery is, in the majority of cases, an issue far beyond the remit of the municipal governments who were the beneficiaries and main interlocutors under the project. Bringing about changes to WSS tariff and subsidy structures in Brazil is a complicated issue, which requires far more political leverage than a single TA subproject can provide by intervening in a small area of a municipality. In most of the informal areas where subprojects were prepared, state water companies or, on occasion, municipal utilities directly managed the WSS services ­ in both cases, the utilities already had existing tariff structures, subsidy policies, and commercial, operational and customer outreach policies. The WSS services provided to the low-income areas targeted under the TAL are not separately built systems but extensions to existing systems. As such, it was not viable to develop separate tariff and subsidy structures for such small, discrete areas of a city since changing a state or municipal utility's tariff or subsidy structure has implications for the whole service area of the utility. The assumption that a small TAL would have sufficient leverage to influence such a politically-sensitive issue as the tariff and subsidy policy of the service provider turned out to be far too optimistic. The sector has, nevertheless, made progress on cost recovery and subsidy policies during the project implementation period. Many utilities have been focusing during the past decade on gaining financial sustainability as demonstrated by the following examples3. o Tariff levels have been increasing rapidly in Brazil in the past decade; the average revenue per cubic meter of water supplied and wastewater collected was US$0.82 in 2000 compared to US$1.17 in 2006. o Operating cost coverage ratios (measuring to what degree revenues cover the O&M costs) increased from 0.99 in 2000 to 1.43 in 2006 ­ representing an average improvement throughout the country of almost 45 percent. For the state water companies (which were the service providers for the majority of the TAL subprojects) 3Based on data from the National WSS Information System (SNIS) compiled each year by MoC; the latest year for which SNIS data is available is 2006. 13 the increase was more pronounced (53 percent), demonstrating that utilities have generally improved their financial sustainability. o Water and sewerage bills in Brazil are generally calculated based on actual consumption; the level of metering increased, albeit slowly, from 87 percent to 89 percent between 2000 and 2006. o All state water companies currently have some sort of social tariff, as do most municipal utilities, though the efficiency of the targeting of these programs varies widely. A number of utilities are already adopting innovative approaches to providing subsidies to their poorest customers: the state water companies of Paraná (SANEPAR), Pernambuco (COMPESA) and Bahia (EMBASA)4 each practice a structured cross-subsidy arrangement between their richer and poorer customers. These utilities use the existing Bolsa Família cadastres in the state to identify which families are eligible for the subsidized social tariff, thus reaching an estimated 10 percent of the companies' respective customers in this way. 4. Number of participating local government agencies that establish a multi- disciplinary team to implement low-income sanitation projects The satisfaction survey showed that 86 percent of the municipalities intended to adopt, or had already adopted and institutionalized, multi-disciplinary teams for the implementation of integrated WSS/ urban upgrading projects in low-income areas. In addition, 76 percent of the municipal technical teams which were responsible for supervising the preparation of PROSANEAR-TAL integrated WSS/urban upgrading subprojects have been, or shall be, maintained for their subsequent implementation and post-implementation phases. 5. Per beneficiary investment costs of less than US$100 for water (US$70 if only for distribution and storage) and US$120 for sewerage (US$80 if collection only) of TAL-supported investment programs The investment ceilings were devised during project preparation as a way of encouraging the development of low-cost subprojects. The per capita ceilings should, however, be re- evaluated for different `typologies' of interventions based on the actual costs gathered from the PROSANEAR-TAL subproject designs. In a detailed cost analysis that covered 24 of the 33 PDLIs and PSIs prepared under PROSANEAR-TAL (see following table), the average per beneficiary cost for different scenarios of water supply and wastewater were calculated and adjusted for the impact of inflation, in order to compare them with the threshold values presented in the 1999 PAD. 4Both the state water companies of Pernambuco and Bahia were represented in the PROSANEAR-TAL project with a number of subproject pre-investment interventions within their area of service. 14 Table 1: Investment per capita thresholds set out in the PAD 1999 value 1999 threshold 2007 value threshold Service provision threshold adjusted for inflation based on PDLI/PSI (US$) to 2007 levels5 designs (US$) (US$) Water supply (distribution and 70 115 104 storage only) Water supply (intake, 100 165 104 treatment, storage and distribution) Wastewater collection 80 132 242 (condominium system) Wastewater collection 120-150 198-247 242 (conventional system)* Wastewater transfer and 40 65 41 treatment Community participation 22 36 30 Source: Detailed analysis of 24 PDLIs and PSIs *Data not given in PAD For the water supply, wastewater treatment and community participation costs of the subprojects, PROSANEAR-TAL stayed within the thresholds identified in the PAD6. For wastewater collection, however, the average costs of the investments surpassed the threshold by a wide margin. This may be largely explained by the assumption in the PAD that a simplified `condominial' approach to wastewater collection would be used, whereas a number of the interventions adopted `conventional' sewerage in difficult, densely-occupied peri-urban settings. When the costs of conventional wastewater collection technologies are considered (included in the above table but not originally identified in the PAD), the difference between the costs actually incurred and the PAD thresholds is more consistent. Furthermore, investment cost estimates for new/rehabilitated/expanded water supply systems (intake, treatment, storage, distribution) and wastewater systems (collection, transfer, treatment, disposal), undertaken using the average cost and expansion data from GoB's SNIS annual series between 2001 and 2006, provide the following per capita costs: R$428 for water (US$214) and R$797 for wastewater (US$398) ­ all in prices adjusted to 2006 ­ which imply that the PROSANEAR-TAL per capita WSS costs are, in fact, reasonable. 6. Level of awareness and satisfaction among municipal officials, community groups, federal and state officials regarding the PROSANEAR project approach It is clear from both the beneficiary surveys (in two of the 30 municipalities ­ see Annex 4) and the satisfaction survey (in 16 of the 30 municipalities ­ see Annex 5) that awareness and satisfaction levels with the PROSANEAR-TAL project approach were 5The per capita thresholds are calculated in Brazilian Reais (R$), adjusted for local inflation and then re- calculated back to US Dollars. 6Although this was achieved on average for the group of subprojects taken together, there were large variations in the costs between subprojects. 15 very high. Over 95 percent of the municipalities surveyed were interested in participating in the preparation of integrated WSS/urban upgrading projects with TA similar to that provided by PROSANEAR-TAL. Most participants (81 percent) thought the roles and responsibilities of the different participating organizations in PROSANEAR-TAL (MoC, CEF, municipalities, states and the Bank) were clear. The respondents generally thought that the anticipated average preparation time of 10 months for undertaking the participatory integrated WSS/urban upgrading designs was insufficient, mostly due to procurement, contracting and payment issues.7 The beneficiary surveys in Guarulhos and São José dos Campos municipalities found that 73 percent of the households surveyed were satisfied with the demonstration works funded under PROSANEAR-TAL. There were, however, large differences in levels of satisfaction between respondents in Guarulhos and Sao José dos Campos ­ with respondents in the former municipality showing a satisfaction rate of 92 percent compared to 54 percent in the latter. The reasons for this difference is believed to be, inter alia, the result of the different baseline conditions in the two project areas resulting in a different assessment of the project benefits (for more details see Annex 4). Summary In summary, the evidence supports the assertion that four of the key performance indicators were fully met, and the PDO indicator on unit investment costs was substantially accomplished. The indicator regarding cost recovery and subsidy policies was, however, deemed to have not been met, given that ­ as discussed above ­ leverage of the TAL to initiate such changes turned out to be too small. Even though two of state water utilities involved in the TAL (EMBASA and COMPESA, covering 9 of the 34 subprojects prepared under the project ­ or over one quarter) implemented clear cost recovery and subsidy policies for their poor customers, it is not possible to attribute this outcome to the PROSANEAR-TAL project. 3.3 Efficiency Efficiency. An ex-ante cost-benefit analysis was not undertaken during preparation given that it was a TA project and many of its benefits were judged intangibles that were intended to be measured in terms of levels of awareness and the adoption of new methodologies for the preparation of integrated WSS/urban upgrading subprojects ­ all aimed at indirectly improving WSS and related services for the urban poor. Nevertheless, the efficiency of the TAL can be measured in terms of how it performed with regard to (i) 7Although the project anticipated a 10 month period for the preparation of the pre-investment subproject designs, not a single municipality was able to complete the consultancies in that timeframe. In some cases, the subproject design preparation lasted several years for a multitude of reasons including, but not limited to: political and administrative changes at the municipal and federal levels; staff turnover in the municipalities; long and cumbersome procurement and disbursement processes inherent in the project design; municipalities having to follow Bank procurement procedures for the first time in most cases; demobilization of the PMF in MoC; the erratic stop-start cycle of federal payments due to perennial budget freezing and the consequent demobilization and remobilization of the consulting firms; delays in technical and financial approval of products by CEF and MoC; etc). 16 its actual unit costs, (ii) compliance with its eligibility criteria, and (iii) its poverty targeting. Unit cost efficiency. The unit costs of preparing the pre-investment subprojects were much lower than expected. In the PAD, it was assumed that the pre-investment cost would make up 12 percent of total investment costs, whereas benchmark values in 2008 for WSS projects vary from 1.5 to 3 percent (MoC figures for water supply, wastewater or drainage subprojects) to over 5 percent (consulting industry parameter for integrated WSS/urban upgrading subprojects). As the actual pre-investment costs added up to about 2 percent of the total investment generated by the pre-investment designs, this shows that project resources were used efficiently. The project also generated 33 PSI subproject engineering designs (10 percent more than was originally targeted in the PAD). As a result, the actual cost of pre-investment per PSI prepared was only 30 percent of the original cost estimated in the PAD. It should be noted, however, that part of these savings is related to exchange rate fluctuations ­ as the Brazilian Real lost on average 40 percent between project appraisal and project closing, and that the low percentages may also be due to a higher level of cost of the corresponding works than was foreseen at appraisal. Table 2: Unit Cost Efficiency Indicator Estimated Value at Actual value at End of Appraisal Project Total project cost for pre-investment 36 million 11.43 million (US$) Total investment generated by pre- 300 million 600 million investment component (US$) Total WSS investment generated by 300 million 93 million pre-investment component (US$) Pre-investment cost as percentage of 12 1.9 total investment (%) Pre-investment cost as percentage of 12 12.2 total WSS investment (%) Number of integrated WSS 30 33 engineering designs prepared (PSIs) Cost per integrated WSS pre- 1,200,000 346,363 investment engineering designs (US$) Source: PAD and Project monitoring data Compliance with project eligibility criteria. PROSANEAR-TAL defined a large number of eligibility criteria for final beneficiaries, for subprojects and for executing agencies, as well as a set of prioritization criteria. In general, compliance with project eligibility criteria was high. The exception to the rule was the requirement of beneficiary municipalities to identify the sources of funds to be used to finance the investments resulting from the PROSANEAR-TAL subproject designs. In the stakeholder survey it was found that 10 percent of the municipalities were able to use their own resources to fund the corresponding investments. Another 45 percent of the municipalities said they were able to access other WSS/urban upgrading programs to fund the investments; an additional 10 percent claimed that they would have been able to have used a combination of own resources and funding through other programs. However, 35 percent of municipalities stated that they would have had no known sources of funding available to 17 implement the corresponding works if GoB's PAC program had not been initiated. In other words, the eligibility criterion related to having proven funds for undertaking the corresponding investments was enforced much more leniently by MoC (and/or interpreted more leniently by the municipalities) than were the other criteria. With the arrival of the PAC, however, the investment works for 24 of the 33 subprojects generated by PROSANEAR-TAL were guaranteed PAC funding. Poverty Targeting. The project was efficient in targeting its interventions to low-income areas and their residents. In the subset of 24 subprojects that were available for evaluation at project close, the average monthly income of the targeted households was less than 3 minimum salaries ­ the poverty threshold used in the PAD. In the more detailed analysis of the two demonstration works interventions (see Annex 4), more than 95 percent were classified as poor, of which 46 percent were earning less than one minimum salary per month. It is therefore concluded that the project's poverty targeting eligibility criteria were properly chosen and applied. Efficacy. The project's efficacy was measured by determining its leverage. The project leveraged more funds than originally estimated during preparation. At appraisal, it was assumed that the pre-investment subprojects would generate US$300 million of investments. However, the PROSANEAR-TAL subprojects actually generated an estimated US$600 million of investments; of which about US$100 million is in water supply and sanitation, and another US$80 million is in drainage and solid waste. This higher leverage is partly because more pre-investment subprojects were prepared (i.e., 33 instead of the 30 envisaged at appraisal). In addition, the average investment subproject was also significantly larger in value than expected ­ with a wider range of urban services being included in the interventions (including significant resettlement solutions in many cases) in addition to the WSS investments. Depending on the municipality and the specifics of the subproject, these investments included, amongst others, housing, access ways, recreational facilities and public spaces, schools, day-care and other social outreach facilities8. This enlarged scope and number of specific investments included in the pre-investment subprojects raises the issue concerning the extent to which each additional service provided additional benefits to the integrated WSS/urban upgrading subprojects. GoB's PAC program will fund the majority of the investments generated by the PROSANEAR-TAL subprojects, since there is a very good fit between the PAC's `Saneamento Integrado' program and the nature of the investments prepared under the TAL's subproject designs. In fact, the TAL's subprojects represent the major part of the PAC's Saneamento Integrado investments for the first two years of PAC implementation ­ because the same issue that hampered implementation of the original PROSANEAR national program was also an issue for the PAC, namely there were very few well- prepared integrated WSS/urban upgrading engineering designs ready to leverage PAC investment at the time of the program's launch. 8Based on a subset of the 33 PSIs prepared under the project, representing about 75 percent of all of the subprojects, 16 percent of the total investments generated was assigned to WSS interventions, 12 percent to solid waste and drainage, 45 percent to housing, and the remaining 27 percent to other urban infrastructure. 18 The leverage of PROSANEAR-TAL was not limited to the funding of integrated WSS/urban upgrading subprojects, but also included the leveraging of an approach to preparing participatory, integrated WSS/urban upgrading projects. The PROSANEAR- TAL methodology, as captured in the extensive standard TOR for the preparation of the PDLIs, PSIs and social participation plans (PTSs, have also been used as a basis for elaborating the TOR and the manual for the preparation of a second and third generation of PAC interventions in integrated WSS/urban upgrading, and have been used as the basis for other similar projects/programs around the country. Relevance. Following its approval, the project remained consistent with Bank priorities through subsequent country assistance strategies (CASs) as described in section 3.1. The project is also well aligned with global priorities and targets set for the MDGs, most notably the targets with regard to improving the access to safe WSS services and to reducing the number of slum dwellers. The fight against poverty and inequality, and the inclusion of the socially excluded, through economic development and basic services provision, remained fundamental elements of the key policies of the GoB despite the many political and administrative transitions that occurred during the project implementation period. To this end, GoB has initiated in recent years a large number of initiatives, such as the Bolsa Família CCT program and the PAC, aimed at improving the living conditions of the poor and, specifically, the very poor. Integrated WSS service provision for peri-urban populations and urban/slum upgrading interventions remains high on the agenda, and a focus on integrated WSS service provision (namely the inclusion of drainage and solid waste management) has been explicitly incorporated in the new federal WSS Law that was approved in early 2007. 3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory The project was highly consistent with the previous two CASs and remains relevant in the context of the new CPS, each of which emphasize support for infrastructure for development and poverty reduction. It fits extremely well with GoB's priorities ­ especially given the advent of the PAC ­ while it also fits and foreshadowed global priorities for WSS and slum upgrading as represented by the MDGs. As such, the project has remained highly relevant given that municipalities, state governments and the federal government are interested in continuing to implement and prioritize the types of intervention championed by PROSANEAR-TAL. The project was able to accomplish its project outcome as measured by the attainment of its key performance indicators, and it did so in such a way that was effective in targeting its interventions to predominantly poor areas, while being cost-effective and leveraging substantially more investment funds than originally intended. Nevertheless, in view of the poorly framed PDO (that presents an outcome that would be unlikely to be met by a TAL), of the missed opportunity to redefine the results framework during the reformulation of the project at the MTR, of the implementation delays, and of the under spending of funds, the overall achievement of the PDO is rated as `moderately satisfactory'. 19 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development The project was reformulated following the MTR to include, at the explicit request of the new Lula administration, a civil works demonstration component to allow the project to follow through in a select number of cases on the implementation of the integrated WSS/urban upgrading subprojects throughout the investment cycle. Although no formal impact evaluation was undertaken, the beneficiary surveys of the two municipalities (Guarulhos e São José dos Campos) for which pilot/demonstration works were (partially, in the former case, and wholly, in the latter case) completed under the project, presented an interesting assessment of subproject impacts. In the two municipalities, there were effects on access to water supply and sanitation service delivery, but also effects on income levels and on income distribution in the informal areas that benefited from the integrated WSS/urban upgrading interventions. As shown in Figure 1, the income distribution changed significantly before and after the subproject works were implemented in the two municipalities. The figures show that the percentage of households without any income reduced significantly, while more households obtained access to higher incomes. As no rigorous impact evaluation has been conducted, it is not possible to attribute all of these income impacts directly to the PROSANEAR-TAL project, especially since GoB introduced and/or improved other poverty-focused initiatives during the project implementation period, such as the Bolsa Família CCT program. However, given the following survey results, it is possible that some of these income effects were indeed due to the subproject interventions: o There was a marked improvement in public security: 47 percent of the respondents mentioned a clear improvement (a finding that municipal representatives during the ICR beneficiary roundtable confirmed). The improvements in public/common land- use, public lighting, public access ways and better recreational facilities as part of the urban upgrading interventions may all have helped improve public security. The impact differed between the two areas, however, with improvements in public security proving to be much less pronounced in São José dos Campos where, inter alia, a local drugs gang continued to be active. o The improvements in public security may have had a positive impact on income growth, while the significant improvements in access to schools (49 percent) child care centers (44 percent), health posts (11 percent ­ but almost three times higher for those that are considered extremely poor), transport (55 percent) and shops (59 percent) may have helped residents to provide themselves with more time and opportunities to get access to work and hence improve their income. o For about 69 percent of the surveyed households, titling of land and property had positive impacts ­ since prior negative discrimination associated with having an address in a `slum' abated, which may explain the changes in income, work opportunities and possible higher real estate values. 20 Figure 1: Impact on income distribution before and after PROSANEAR-TAL project in two pilot areas Income distribution in São José dos Campos Income distribution in Garulhos 50% 40% 50% 40% 30% 30% 20% 20% 10% 10% 0% 0% Without Up to 1 MS1-2 MS 2-3 MS > 3 MS Without Up to 1 1-2 SM 2-3 SM > 3 SM income income SM 2004 2008 2004 2008 Source: Beneficiary surveys (b) Institutional Change/Strengthening The project provided, with a relatively small amount of resources, many training opportunities for the numerous stakeholders working in the sector. During implementation, the project conducted 109 training events, and 5 workshops. It provided this support to 29 municipal governments, to one state government, to 15 CEF teams operating in 12 states, to 20 consulting companies, and to federal government staff in MoC and in CEF. Although more than 90 percent of the respondents of the stakeholder survey were satisfied with the quality of the professionals (both public and private sector) involved in subproject preparation, only 60 percent of respondents felt that the number of professionals available in the country for implementing the subprojects was sufficient. The latter fact implies that there is an unmet demand for qualified sector staff to undertake integrated WSS/urban upgrading projects in Brazil at all levels (in federal, state and municipal governments, in consulting firms and NGOs ­ and for both the technical and the social aspects of the work). (c) Other Unintended Outcomes and Impacts (positive or negative) An intangible effect of PROSANEAR-TAL was its role as a forum for promoting and discussing an integrated approach to WSS, urban services and urban upgrading in poor areas that contributed to a number of results: (i) inclusion of the concept of `integrated WSS services' (namely water supply, sanitation, drainage and solid waste) in the 2007 Federal WSS Law; (ii) inclusion of a credit line and an approach for integrated WSS ('saneamento integrado') in the PAC program; and (iii) discussion on the design and implementation of WSS tariffs and subsidies to the poor. The project's role in functioning as a forum for such issues has fostered discussion and debate that served as an input to a larger process in reforming the sector. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops While this is a core ICR it was deemed important to provide more details and lessons learned from the valuable and varied experiences of the PROSANEAR-TAL project. The municipal stakeholder surveys and the beneficiary survey results are presented in Annex 4 and Annex 5, respectively. In addition, a day-long stakeholder roundtable meeting was held on April 4, 2008, with over 40 representatives of municipal beneficiaries, MoC, and 21 regional and federal CEF offices, with the objective of fully discussing and analyzing the positive and negative aspects of the implementation of PROSANEAR-TAL ­ the results of which are presented in a report in Project files and which contributed to the elaboration of the Borrower's final report and to this ICR. 4. Assessment of Risk to Development Outcome Rating: Moderate As is clearly ascertained from the stakeholder surveys (Annex 5), the risks are small that the local governments will not maintain the knowledge and capacity provided by PROSANEAR-TAL to supervise the design and implementation of investment subprojects in low-income neighborhoods. The stakeholder surveys show that 95 percent of municipal governments had adopted, or planned to adopt (depending on the status of the pre-investment projects), the participatory, integrated approach of PROSANEAR- TAL in the preparation of other such projects in poor urban areas. The survey also showed that 81 percent of the municipalities were already preparing additional new integrated WSS/urban upgrading projects to be funded under GoB's PAC program, while another 91 percent of the municipalities surveyed were already incorporating the methodologies and experiences of PROSANEAR-TAL in the preparation of new integrated WSS/urban upgrading projects in their municipalities. Moreover, the large majority of municipalities (86 percent) also intended to adopt, or have already adopted and institutionalized, the multi-disciplinary teams for the implementation of such projects in poor urban areas. In addition, the current GoB PAC program has taken up PROSANEAR-TAL's methodology for the participatory, integrated preparation of subprojects. Even though political and administrative changes can shift priorities, the overall relevance of integrated WSS/urban upgrading for the urban poor throughout the country makes it unlikely that this capacity will not be sustained in the medium-to-long-term. This is especially so since the concept of `integrated WSS/urban services' is explicitly described and promoted in the new federal WSS law. There nevertheless appears to be an unmet demand to provide more training and professional development opportunities in order to supplement the cadre of professionals that are capable of preparing, implementing and supervising the highly complex participatory, integrated WSS/urban upgrading subprojects in low-income neighborhoods in Brazil. Even though it is highly likely that the project benefits will be retained in the participating municipalities, the universe of low-income neighborhoods in the country extends far beyond those that participated in PROSANEAR-TAL. Adequate short- and medium-term transition arrangements for the project have been made: GoB's PAC program will fund the majority of the investments generated by the PROSANEAR-TAL subprojects, while the PROSANEAR-TAL methodology and TOR have been used as the basis for preparing the PAC Saneamento Integrado's own manual and TOR which, in turn, will be employed to prepare a second and third generation of PAC subprojects. Nevertheless, in view of the political and administrative changes that are likely to continue to occur in the medium- to long-term in Brazil, the overall risk to the development outcome is rated `moderate'. 22 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory Overall, the project is judged to have been of sufficient strategic relevance but the quality at entry of the project design as less satisfactory. The project nevertheless remained highly relevant its implementation. It generated a number of tools for looking at WSS in slum areas in a much more holistic manner ­ stressing the interface of WSS services with other urban services, although not always fully understanding the (economic, financial and related) consequences of this interface (mainly due to a lack of timely information about the important aspects of this interface)9. The project also provided other tools, such as the standardized TOR for the participatory preparation of integrated pre-investment subprojects, which brought a consistent framework to the preparation of such subprojects in vastly differing situations. In addition, the project design provided a large degree of flexibility that ensured that it could cope with the many different changes that took place during implementation in the institutional context in which it was set. Nevertheless, the project preparation team made some overly optimistic assumptions at appraisal, and neglected certain issues, including: (i) The use of `condominial' or low-cost technologies in small, discrete project areas located in municipalities that are already served by water supply and sewerage companies which may have limited interest in using different technologies within their existing networks ­ although condominial sewers are not necessarily the optimal choice in all settings (depending on demographic, social, density of occupation, service provider and other parameters), changing the mindsets of engineers and other decision-makers in those situations were condominial or other lower-cost approaches are appropriate is not always easy; (ii) The degree of influence that a relatively small TA project has in changing tariff and subsidy policies in state and municipal water utilities; (iii) The use of CEF's regional network for decentralized project supervision, despite the limited capacity of many of these CEF offices to analyze technical products and to assist with procurement, disbursement, contract management, etc; (iv) The capacity of municipal governments to undertake Bank procurement was limited and the carrying out of this procurement by the municipalities of limited, or negative, value-added; 9The PAD's proposed approach to the economic analysis of subprojects was, for instance, based on a stand-alone assessment of WSS interventions that did not take into account the interface with other urban services. 23 (v) The guidance provided in the PAD, the Operational Manual and the standard pre-investment TOR was very weak with regard to how to produce environmental impact assessments (EIAs) and resettlement action plans (RAPs) to satisfy Bank or other stakeholder requirements, and thus needed to be reviewed and developed during implementation; (vi) The results framework and M&E arrangements were of limited quality; (vii) The lessons learned of the previous PROSANEAR investment loan were transferred to the TAL without due regard for how the change in lending instrument would affect leverage and implementation arrangements. Given the above design flaws, the Bank performance at quality at entry is rated `moderately unsatisfactory'. (b) Quality of Supervision (including of fiduciary and safeguards policies) Rating: Moderately Satisfactory Supervision missions were consistently carried out at least twice a year, and in four out of the eight years of implementation three supervision missions were undertaken annually. Supervision missions were generally well-staffed and managed to maintain an important degree of continuity during the lifetime of the project. This was important as the project had three task team leaders during its implementation, which is a likely cause of disruption each time such a change is made. The expertise of the team members was high and the teams averaged several members per mission including a balance of international and national expertise. From FY2005 to the close of the project, the task team leader was based in Brasília which helped maintain a much closer level of supervision. Even though it was not captured in the official number of supervision missions, informal supervision was taking place on a much more regular basis. The team retained a steady focus on the implementation progress and the development impact of the project, despite the interferences caused by the numerous political and administrative changes that affected the project over its lifetime. The fiduciary aspects of the project were duly supervised on a regular basis, also with Bank staff from the Brasília office, and addressed as necessary. Good working relationships were generally formed between MoC's implementation team and the Bank supervision team. The demonstration works interventions showed that the standard PROSANEAR-TAL TOR did not include for the generation of reports containing information on the resettlement and environmental aspects in a format which facilitated review by, and interaction with, the Bank team on these issues. This led to intensive work by the Bank's safeguards staff and consultants to ensure compliance with Bank safeguard policy. Similarly, alternative approaches to the economic and financial analyses of the demonstration works were adopted, which differed from the approach presented in the TOR and the PAD. These experiences during the intensive supervision of those of the pre-investment subprojects that were related to the demonstration works provided valuable guidance as to how the standard TOR, and the accompanying guidance manual, should be adapted to improve their effectiveness in these respects The Bank team also showed sufficient candor in its supervision reporting: the project was downgraded on a number of occasions, at times 24 when implementation was considered critically at risk of not being able to achieve its objectives. The MTR was undertaken over a series of missions, and resulted in the reformulation of the project, included a partial cancellation and reconfiguration of the components, as well as the establishment of physical and financial targets which were effectively used to make decisions on subsequent project extensions. The MTR did not, however, use the opportunity to adjust and improve the results framework on which the project was based, nor to improve the M&E arrangements more broadly. Adequate transition arrangements for the project have been made. GoB's PAC program will fund the majority of the investments generated by the PROSANEAR-TAL subprojects, since there is a seamless fit between the PAC's `Saneamento Integrado' program and the nature of the investments prepared under the TAL's subproject designs. In fact, the TAL subprojects represent the major part of the PAC's Saneamento Integrado investments for the first two years of PAC implementation ­ there were very few well- prepared integrated WSS/urban upgrading engineering designs ready to leverage PAC investment at the time of the program's launch. Also the PROSANEAR-TAL methodology and TOR have been used as the basis for preparing the PAC Saneamento Integrado's own manual and TOR which, in turn, will be employed to prepare the second and third generation of PAC subprojects. Given the overall acceptable performance of the Bank during implementation, some failings notwithstanding, the Bank's quality of supervision is rated `moderately satisfactory'. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory The overall Bank performance rating is considered `moderately satisfactory' as, despite some serious design and implementation limitations, the key performance indicators ­ which were taken as proxies for the project's outcomes ­ were mostly achieved. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Unsatisfactory The federal government generally, and the Ministry of Cities in particular, fell short on a number of the criteria used to judge satisfactory project implementation: (i) its ownership and commitment to achieving the development objectives wavered considerably throughout project implementation, being affected significantly by the changes in the project's political and administrative counterparts in the federal government and the degree of priority that they accorded to the project; (ii) the readiness for implementation, the implementation arrangements and capacity, and related issues, proved inadequate for rapid and efficient project implementation; (iii) implementation issues were not resolved in a timely fashion ­ this was true at project initiation, when the mobilizing of the project management firm took over two years, as it was towards project closing when the same firm was suddenly removed and no adequate alternative arrangements were found by MoC to ensure the project's continued smooth implementation and proper closure; 25 (iv) fiduciary issues, budget provision on a timely basis, and the compliance with the loan covenant concerning the adequate staffing of the PMU all proved problematic at times during implementation; (v) the project's M&E system was generally inadequate to allow the following of implementation in an agile manner; and (vi) some US$ 2.7 million of eligible payments were not presented to the Bank during the last year of implementation and/or the grace period due to administrative errors/misunderstandings in MoC. On the positive side, beneficiary and stakeholder consultation and involvement in the project was a central part of the subproject cycle and thus of the project's approach and philosophy; MoC's relationships with CEF and the beneficiary municipalities and other stakeholders was generally good; and the adequacy of the transition arrangements, as discussed above, are very promising. Given the above shortfalls in a number of the key criteria for judging adequate implementation, the Government's overall performance is rated `moderately unsatisfactory'. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory The beneficiary municipal governments were the main implementing agencies under the project and, given its decentralized nature, they made satisfactory efforts to implement the subprojects in an efficient manner ­ although the degree of efficiency varied from case to case. Given that, despite the numerous challenges posed during project implementation, the municipalities were able to conclude the pre-investment participatory, integrated engineering designs, albeit with considerable delays in most cases, then the performance of the implementing agencies is considered `moderately satisfactory'. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory Given the variability encountered on the Borrower's performance throughout the implementation period, and between the different stakeholders involved in project execution, and given that ­ despite the numerous challenges ­ the Borrower managed to conclude the project in a satisfactory manner, the Borrower's overall performance is rated as `moderately satisfactory'. 6. Lessons Learned The complexity of decentralization has a major effect on the successful design and implementation of a project. Although the Bank is often confronted with significant shifts in political realities on the ground, this experience is not usually very well captured in project design and implementation. The PROSANEAR-TAL project was approved at the end of 2000, but went through a large number of political and administrative changes at the federal, state and municipal levels that affected both project design and implementation. Each change in counterparts at different levels of government brought challenges to the level of prioritization and the rhythm of implementation. In such circumstances of shifting political priorities, the task team has the option of either being client responsive and adjusting the project accordingly, or cancelling the project. Both approaches have numerous repercussions, to which scant attention is often given. 26 Continuing implementation while simultaneously redesigning a project can have significant impacts on the cost of supervision, can delay implementation, and can impact the attainment and measurability of project results during implementation. A lesson learned of this project is that doing business in increasingly decentralized and politically complex environments will require flexibility and adaptability if Bank projects are to remain relevant to the borrower and implementers, but such flexibility will also result in additional time and resource costs to the Bank. The role of the Bank in countries with access to their own investment funds. In such countries, the Bank's comparative advantage is not in providing investment funds but in delivering knowledge that can significantly leverage the effectiveness of these funds. Project preparation teams must bear in mind that different operational tools have greatly differing potential to affect reform ­ TALs, for example, lack the financial leverage implicit in a sector investment loan (SIL). When embarking on the preparation of TAL projects it is important to understand whether the incentives they provide are sufficient to ensure that the anticipated leverage can take place. When designing a TAL project preparation teams should specifically consider: (i) whether lessons learned from related SILs can be directly transferred to TALs; (ii) the results frameworks (including a set of appropriate outcome and output indicators) of such operations should be adjusted to the limitations of what a TAL can actually achieve; (iii) carefully crafting implementation arrangements to ensure sufficient incentives are inbuilt to encourage the required performance, given that a TAL does not automatically have the ability to leverage a large investment budget. Trade-offs between complexity and inclusiveness: reducing the transaction cost of doing business. A centrally coordinated and controlled project, which is implemented at a decentralized level, as was the case of PROSANEAR-TAL, necessarily involves a large number of actors and tradeoffs with regard to the agility of implementation. To be inclusive in such cases means adding to complexity by involving more stakeholders in project implementation. This has resulted in complex institutional arrangements in this project. The Ministry of Cities and its PMU, CEF (both at the federal and municipal/state level), municipal governments and their PMUs, community organizations, service providers of the different public services, the consulting engineering firms and social outreach specialists, and the Bank all participated in and contributed to the project's implementation. Furthermore, the degree of community participation throughout the subproject design and implementation cycles needs careful analysis to identify the optimal level of such participation in order to maximize ownership but not unduly affect the time or cost of the subproject cycle. The many different relationships, which often come with their own ways of doing business, may result in misunderstandings, variations in implementation modalities, and a need for extensive learning processes (for instance if local government are required to learn Bank procurement processes) which, in turn, may result in high transaction costs and result in implementation delays. For complex projects of a similar nature, the Bank should determine upfront what the possible implications of these transaction processes are regarding time, cost and real risk, in order to ensure that sufficient consideration is given to them in terms of the impacts on a project's implementation schedule, disbursement profiles, training and other TA needs to stakeholders, and supervision costs. A related specific project lesson is that, to maximize ownership, capacity building and continuity, the municipality PMUs should be formally 27 created with the municipal government and should consist of municipal staff in the key engineering, urban and social areas. Lack of capacity to undertake participatory, integrated WSS/urban upgrading subprojects. Despite Brazil's relatively long and broad experience in undertaking such subprojects, there is nevertheless still a surprising lack of capacity among the various stakeholders involved in the preparation and implementation of these subprojects. Given the country's size and regional disparities, this capacity varies considerably, with some municipalities and local CEF offices demonstrating ample ability to efficiently implement such subprojects. Many municipal governments and local CEF offices, however, revealed significant weaknesses in being able to procure and execute the subprojects, and both required much assistance throughout the `subproject cycle' in order to bring the activities to closure. Furthermore, it was also revealing to see the discrepancy in the ability of the national consulting engineering industry to undertake subproject designs which marry the technical and the social ­ even though Brazil has pioneered such approaches in WSS and urban upgrading over recent years. The project was able to identify these weaknesses in the different groups of stakeholders, and provided training events and TA visits through MoC's PMU in order to correct implementation issues as necessary. For such centrally coordinated projects executed at a decentralized level, ample TA and training should be provided on an as-needed basis to the different stakeholders. Having the municipalities undertake Bank procurement should be questioned as it differs from national procurement procedures, creates considerable confusion, and is often the only time a municipal government will ever need to use IBRD procurement guidelines. Although such an approach helped build ownership at the local level, it also created much frustration and delay ­ procurement could be undertaken centrally to bring efficiencies to the process, with the municipal governments and CEF local offices invited to participate in the process in order to understand it and to build ownership. Ceilings for investment for different typologies of urban upgrading The PAD provided per capita investment ceilings for water supply and for sanitation interventions, but not for the complementary infrastructure (access ways, drainage, solid waste management, housing improvements, resettlement, public spaces, community facilities, etc), that are an integral part of a broader urban upgrading intervention. In order to counter `designing up' to a high level of intervention, investment ceilings should be developed for the different typologies of urban upgrading intervention. These ceilings should be based on `typologies' of intervention which identify the minimal level of intervention needed in different demographic and/or regional settings. The central project management unit and/or the standard TOR/manual should guide municipalities in identifying the low-income areas to be prioritized for intervention, with a view to maximizing the number or families to be benefited at the lowest possible cost ­ this function under PROSANEAR-TAL was left to the municipalities and did not always result in the most cost efficient interventions. Also, because of a lack of understanding of the multiple effects of integrated WSS/urban upgrading projects, the question arises as to what combination of interventions has the most positive cost/benefit impact in urban upgrading ­ a definitive answer to this issue is unlikely to be easily identified and will require more, and thorough, evaluations of such projects. Another lesson is the need to build better M&E information systems that can effectively store and organize the 28 collected data from subprojects and allow for their in-depth analysis. Such systems should be developed up-front so that M&E arrangements are set up in a timely manner, and responsibilities for M&E are clearly defined. Complex terms of reference The TOR used to undertake participatory, integrated WSS/urban upgrading subprojects were complex in nature and involved the production of seven reports over the estimated ten-month execution period. Much was learned from the utilization of the TOR in municipalities of differing capacity throughout the country and, based on this experience, revised, simplified TOR are being produced by MoC. One recommendation is that the products that the TOR generates be more closely linked to outputs that the beneficiary municipality needs to produce for other reasons, such as the EIA and the RAP. Specific elements of the TOR that were identified for revision included (i) the comprehensiveness and the quality of the EIA and RAP products being produced; (ii) the economic and financial analyses need to be suitable for complex, integrated urban services and urban upgrading projects rather than only for stand-alone WSS interventions, and (iii) clearer guidance needs to be provided as to the scope and modality of social participation during subproject preparation. The Borrower has presented a number of specific recommendations for improving the standard TOR, as summarized in its comments in Section 7 below. The Borrower also strongly recommended that the Bank consider the inclusion of budget ceilings in the request for proposal package, instead of the staff-hour ceilings normally provided, in order to encourage bidders to be as close as possible to the market reference costs. Role of the project management firm. The technical assistance provided to the PMU through the project management firm was crucial for the efficiency of project implementation. Even for a central ministry in a middle-income country such as Brazil, the incentive framework to implement a Bank project is not always conducive to effective and efficient execution. Even when Bank financing is designed to support a slice of the government's overall budget funding, its implementation is often still considered a parallel activity within the ministry given that sufficient staff are often not available to implement such projects with their different/additional requirements. This was amply demonstrated in the case of PROSANEAR-TAL where project implementation, and subsequent disbursements, improved considerably once a project management firm was mobilized to support the PMU, and the opposite was true during the last 18 months of implementation once the firm had been summarily demobilized by MoC. Analyses should be undertaken, on a case-by-case basis, of the advantages and disadvantages of including such management firms and/or specialized individual consultants in the implementation arrangements of complex projects, especially where the executing agencies' bureaucratic processes are cumbersome. 29 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies (the following text is a direct, unofficial translation of the comments received in Portuguese from the Ministry of Cities) It is normal practice for the International Bank for Reconstruction and Development (IBRD) to produce an Implementation Completion and Results Report (ICR) on completion of projects financed with World Bank funds. The purpose of the IBRD Report is to assess the results of a project and to draw attention to factors that determine whether a given project has been effective or not, as well as to evaluate the performance of all the players involved in its implementation from the planning stage through to final execution. PROSANEAR-TAL, completed on 15/12/2007, was the subject of an ICR which provided an overall appraisal of the results achieved by the project. The ICR report drew attention in a precise, objective and impartial manner to all the different aspects associated with the effectiveness of the project as well as to the lessons learned as a result of its execution. Although the ICR awarded only a "moderately satisfactory" rating to the project, PROSANEAR-TAL nevertheless achieved its main goals, which were designed to provide direct benefits to two main groups: (i) the institutions that participated in executing the project benefited from institutional development and training; and (ii) the populations which will derive benefit from the expanded availability of essential services, thereby contributing to improved quality of life. PROSANEAR-TAL involved the provision of institutional training for 30 municipal governments in three key areas: improved integrated urban planning of cities, the proper preparation of integrated WSS/urban upgrading engineering designs, and the adoption of a participatory methodology for poor urban areas. This difficult-to-measure capacity building is capable of generating widespread benefits by providing opportunities for the expertise acquired by the target beneficiaries to be replicated in a broad range of other areas of a given municipality. Consulting firms, despite having been contracted on the basis of their professed experience, also benefited from exposure to subprojects involving high levels of social participation, focused on integrated solutions. Furthermore, 15 teams from CEF, the agency responsible for the operational and financial aspects of the project in 12 states, also benefited from training. PROSANEAR-TAL also had a major impact on the PAC by ensuring that the works to be undertaken during the first year under the PAC's Integrated Sanitation program were in fact contracted. This will result in an investment of around one billion Reais in civil works arising from the 30 projects prepared by PROSANEAR-TAL. These PAC investments will lead to better living conditions for over 45,000 families living in socially and environmentally vulnerable areas. 30 It is worth drawing attention to the complementarity and synergy between the two government actions. By incorporating the portfolio of PROSANEAR-TAL subprojects as an investment priority for the PAC, GoB will henceforth be in a position to guarantee that the PAC works will be executed on the basis of well-prepared subprojects which have benefited from continuous follow-up by GoB's own agents responsible for project implementation, especially the CEF teams. Moreover this approach will ensure that the projects prepared by PROSANEAR-TAL will be of direct benefit to the population located in the intervention areas. With regard to the appraisal of the various players involved (Federal Government, Bank, the municipalities and CEF) in the conception, design and implementation of the PROSANEAR-TAL, the ICR assessed their performance generally as "moderately satisfactory". At the same time the ICR considered that the performance of the Federal Government and the Bank with regard to project conception was generally "moderately unsatisfactory". The poor performance of the Federal Government, especially that of the Ministry of the Cities, was justified mainly by the absence of a permanent management team during the entire period of project implementation, as well as by the lack of continuity in Federal budget allocations, evidenced by the successive cutbacks to the project's annual budget. The rating awarded to the Bank (moderately unsatisfactory) with regard to project conception can be attributed mainly to the fact that the expected results of the project were overestimated. This is apparent from the indicators designed to gauge the effectiveness of the project (more concerned with measuring the effects of completed works than those of a Technical Assistance Project) and from the fact that unachievable objectives were established in the first place, such as the need for beneficiaries to adopt clear tariff and subsidy policies for WSS services. This approach failed to take into account the inherent characteristics of WSS service providers in Brazil. It should nevertheless be emphasized that the performance of the Bank team was extremely positive during project implementation. The team responded flexibly and proactively to all the requests and requirements for PROSANEAR-TAL restructuring which were judged necessary in order to bring the project into line with the various economic changes and shifting priorities that occurred during project execution. The following outline description of PROSANEAR-TAL project emphasizes the key aspects of its performance. PROSANEAR-TAL was the outcome of a partnership framework agreement between the Brazilian Government and the International Bank for Reconstruction and Development (IBRD) in accordance with Loan Agreement 4532-BR, signed on 19/09/2000 and concluded on 15/12/2007. The initial project implementation timescale was four years, with conclusion planned for 15/12/2004. The PROSANEAR-TAL cost was initially estimated at US$ 49.3 million, 31 with US$ 30.3 million to be provided by the IBRD and US$ 19 million consisting of non- financial counterpart contributions by Brazil's Federal Government, States and Municipalities. The PROSANEAR-TAL project was conceived with the basic goal of resolving in a sustainable manner the problems of environmental WSS in densely populated urban areas occupied by low-income families lacking social infrastructure in cities of over 75.000 inhabitants. Three main activities were targeted: the development of integrated WSS engineering designs for poor urban areas; the preparation and consolidation of standard approaches to integrated WSS and urban upgrading in precarious areas; and the institutional development of the actors involved in execution of the project. The project was initially structured on the basis of four components: management, promotion and studies; pre-investment; training; and urban development. The project was later reformulated to incorporate an `investment' component. At the planning stage of the project, the goal of PROSANEAR-TAL was to increase the stock of integrated WSS engineering designs/subprojects for low-income population groups. The associated civil works were to be financed by GoB's PROSANEAR program, with funding through FGTS loans. However, the three years following signature of the loan were affected negatively by the total restriction on credit for investment in WSS on account of the fiscal adjustment imposed by GoB aimed at reducing public debt. The project commenced activities under the Presidency of the Republic's Special Secretariat for Urban Development (SEDU). Subsequently, with the change in government in 2003 PROSANEAR-TAL was incorporated into the newly-created Ministry of the Cities, specifically its National Secretariat for Environmental Water Supply and Sanitation (SNSA). In view of the new government structure and the low performance of the project, the SNSA directors decided to reformulate PROSANEAR-TAL. A new structure for the project was tabled during the Mid-Term Review in the second half of 2003, which took effect in 2004. The reformulated structure involved (i) cancellation of US$ 6.4 million; (ii) an extension of the project by two years, subject to performance; and (iii) the inclusion of an investment component for the execution of demonstrative works. The purpose of the cancellation of US$ 6.4 million was to compensate for increases in the value of the US dollar, which meant that more local currency (reais) was available for the project. In the circumstances it was judged that given the progress and future disbursement profile of the project not all the available resources would be used ­ hence the cancelation of part of the funds. The idea of extending the project deadline was to enable the newly-agreed targets to be met in full and for the civil works under the investment component to proceed. 32 The reason for including the investment component, in addition to testing the methodology for developing the PROSANEAR-TAL subprojects and for enabling demonstrative works to go ahead with integral follow-up of the project-work cycle, was basically to meet SNSA´s requirement that the subprojects prepared by PROSANEAR- TAL should be effectively executed and be of real benefit for the target population. With the restructuring, PROSANEAR-TAL received a new boost, which generated a strong increase in disbursement in 2005, but which subsequently lost momentum, principally with the demobilization of the project management firm. Consequently, only US$16.7 million of the reprogrammed US$ 23.0 million were disbursed, even though the implementation period was extended from four to seven years. Nevertheless, it is important to highlight that, although the Loan Agreement was signed to be executed in four years, the goals presented in the Project Appraisal Document (PAD) that synthesize the major aspects of the project, initially foresaw the execution of PROSANEAR-TAL in five years. With regard to the extensions to the deadline, PROSANEAR-TAL was strongly influenced during its implementation by a number of diametrically opposed economic issues that had been present from project conception onwards, such as the restrictions imposed on investment credit and a trend towards privatization of the WSS sector. Credit restrictions and the low indebtedness capacity of the municipalities during the first years of the PROSANEAR-TAL, together with the contractual obligation established under the pre-investment component that the works should be executed by the beneficiaries themselves, resulted in few municipalities showing interest in participating in the project. Up to 2003, PROSANEAR-TAL had only the State Government of Pernambuco as a beneficiary, since it had already secured an IBRD loan for the execution of the works under the auspices of the PRÓ-METROPOLE project. In 2004, with the efforts of the project management firm PROSANEAR-TAL significantly increased its number of beneficiary entities. A number of other factors also contributed to delays in project implementation, such as the requirement of the consultancies to be contracted according to IBRD procurement rules and the failure of the Ministry of the Cities to release funds on a regular basis. While the IBRD procurement processes are more straightforward, time-saving and less susceptible to the drawbacks occasioned by Brazil's public administration and justice systems, the local municipal governments ­ principally the smaller ones which had never previously participated in international financing arrangements ­ experienced major difficulties in organizing the Bank procurement procedures as there was resistance from the municipal attorneys and because of the institutional difficulty of undertaking procurement process that varied from those they were used to, even with the support provided by the federal PMU. 33 The need for example to publish announcements in the Development Business (DB) database targeting the international and local markets, as well as the requirement to comply with the minimum, though generous, deadlines vital for proceeding with the subsequent stages of the selection processes, meant that the selection procedures were generally lengthy, time-consuming exercises. It is also worth mentioning that the publication of the Request for Expression of Interest in the DB, in addition to delaying the selection processes, also proved to be ineffective given the type of subprojects to be contracted. The fact is that these subprojects require knowledge of the local context, given the existence of specific social circumstances, particularly the high levels of crime encountered in the majority of the areas targeted by the subprojects. Virtually all the foreign consulting firms that showed interest lacked the three essential qualifications required for participation in the PROSANEAR-TAL (urban planning in precarious settlements, integrated WSS engineering designs, and community participation). As a result, few foreign firms reached the shortlist and not one of them was subsequently selected for an assignment. Regarding the disbursement profile, the project failed to achieve the initially forecast targets. According to the ICR, even excluding the cancellation of US$ 6.4 million in 2004, US$ 7.2 million of the loan funds were still unspent by 15/04/2008. A considerable part of the unspent resources (around US$ 6 million) had been earmarked for executing civil works under the investment component. Initially five works were planned to be financed by the PROSANEAR-TAL. However, of these only three received funding from the project: the municipalities of São José dos Campos and Guarulhos in the state of São Paulo and the city of Natal in the state of Rio Grande do Norte. Of these three, only the works in São José dos Campos were fully completed during the execution phase of PROSANEAR-TAL. Given that the public works in Guarulhos were extremely complex, an implementation period of 36 months was set, which effectively exceeded the PROSANEAR-TAL deadline. In December 2007 the work in Guarulhos was nevertheless at an advanced stage and on schedule. The intervention in Natal encountered a series of delays in the procurement processes and did not commence until the beginning of 2007. These two interventions will receive complementary resources from the PAC. In the other two works that were initially planned for the municipalities of Caxias (Maranhão) and Nova Iguaçu (Rio de Janeiro) the Bank procurement processes were in the event cancelled since these two areas were awarded PAC funds. Other factors also restricted the disbursement levels. A series of errors were identified in the processing of some of the disbursement applications, for example the non- presentation of official invoices for the payment of certain eligible activities which should have been reimbursed from the loan account to the tune of R$ 2.6 million. 34 Moreover, a failure by the PMU ­ as a the result of a misunderstanding ­ to comply with the final deadline for the payment of invoices within the grace period for disbursement applications, meant that around R$1 million, paid by GoB by 15/04/08, were not withdrawn from the loan account. In addition, a number of payments billed after 15/12/2007 were not accounted for and will be paid from PAC resources. If all of the above factors had been properly presented and accounted for at the correct time, an additional amount of around US$ 2.7 million would have been withdrawn from the loan account. A further factor which undermined both the project execution period and the disbursement levels was the absence of a stable, permanent management firm with sufficient technical and operational capacity to meet the demands of the project throughout its duration. This problem was particularly serious at two points during the execution of PROSANEAR-TAL: in the early years of project implementation and towards the end, when the management firm supporting the national PMU was demobilized only one year before the project was due to close. Before August 2000 the UGP had no management firm and project performance suffered as a result: the project had only two pre-investment contracts signed and only 0.2% of the total loan had been disbursed. Once the management firm was contracted the information required for preparing contracts with the beneficiaries and for drafting the Terms of Reference for contracting the subproject designs began to be systematized. The new management firm's activities aimed to provide the PMU with the necessary information and tools for disseminating the project, as well as for attracting new players interested in participating in it. This development gave fresh impetus to PROSANEAR-TAL and, by 2004, a total of 25 municipalities had formally become participants. The effect on disbursements was apparent a year later when the subprojects began to be contracted. As a result disbursements increased threefold in one year ­ from US$ 1.5 million in 2004 to US $4.5 million by the end of 2005. The disbandment of the management firm in October 2006 caused a significant impact on the performance of the project at a crucial time, particularly with regard to subproject activity follow-up, levels of disbursement, and the preparation of strategic documents. The PMU and the Bank teams made strenuous efforts to retain the incumbent project management firm but the Ministry stuck by its decision to disband it, accepting the view of its legal advisers that since the activities undertaken by the firm were the responsibility of the Ministry of the Cities they should be undertaken only by staff in the direct employ of the Ministry. In the event the latter was unable to substitute the technical staff of the management firm with professionals from its own staff and the PMU remained seriously understaffed; in addition, in 2007, SNSA directed its attention to the formulation and execution of the PAC. 35 Despite the setbacks and delays during the implementation of the project, the planned project objectives can be considered to have been fulfilled: 33 integrated WSS projects and 34 local development plans were concluded from a total of 35 contracted; the civil work in São José dos Campos were finished and the works in Guarulhos are at an advanced stage. The works currently underway in Natal will be concluded with PAC resources. The following are a number of suggestions arising from the implementation of this project: · The selection process for consulting firms could have been undertaken in a centralized manner. It is not an objective of the project to build capacity in the municipal governments in IBRD procurement process, principally those that don't foresee entering into a future direct loan agreement with the IBRD. The municipalities could, instead, participate in the analysis of the technical proposals and in the accompaniment of the elaboration of the products, where capacity building of the municipal teams would be undertaken. · Selection of the intervention areas should be done directly by the national PMU and, wherever possible, by the Bank, in order to ensure that priority is given to areas of key social interest or areas likely to have a powerful impact on other sectors for which the municipal authorities are responsible (e.g. public security, transport and drainage); · The Local PMUs should possess an institutional framework consisting ideally of technical personnel (engineering, urbanism and social work) recruited from the own staff of the beneficiary entities (state governments, the federal district or municipal governments); · It was observed that the Local PMUs working in a designated secretariat of the beneficiary entities (e.g. Housing) were in a position to pursue their activities in a more coordinated manner, as well as possessing greater authority to respond to the demands of the subproject; · It is essential that a single database designed to act as a central point for project impact evaluation be utilized; · The need exists to establish a financial ceiling for subproject contracts, given that the IBRD procurement process does not establish a maximum amount to be paid by the government, only indicating the number of hours needed for subproject execution; · The need also exists to define average parameters for the preparation of subproject budgets. These should take into account regional costs differences and the specific characteristics of each area, which need to be addressed on a case by case basis; 36 · The economic-financial assessment required by the PROSANEAR-TAL was confined to water and sewerage services and failed to address all the factors related to an integrated urban intervention project; · For this type of intervention it makes no sense to carry out an economic-financial evaluation, given the fact that the analysis will not be taken into account in terms of restricting or prioritizing works in the selected areas; · Instead of preparing a cost-benefit analysis (which proved to be wasteful and unable of producing effective results), it would be more useful to ensure that the proposed solutions are cost-effective and genuinely able to achieve the proposed objectives at lowest cost; · The assessments relating to the cost recovery/subsidies policies of the WSS service providers planned under the PAD produced no practical results given the ways in which WSS services are delivered in Brazil ­ where service providers, mainly the state water companies, possess no clear policies regarding cost recovery and subsidies. With regard to the project's standard terms of reference for integrated WSS/urban upgrading of low-income areas, we have the following comments: · The TOR should define cost parameters for the planned works, indicating actual amounts or reference tables (e.g. CEF's National Research System for Costs and Indices for Civil Construction, SINAPE); · The TOR should clearly establish which normative regimes are to be followed (e.g. IBRD safeguards and/or Brazilian legislation), depending on the funding sources for the works, in particular regarding the resettlement and environmental aspects of the subprojects; · The TOR should clearly define the `area of influence' of the intervention (PDLI) and the actual `area of intervention' (PSI) in order to justify the level of influence applying to the respective areas. For example, the PDLI area should be restricted to the area of influence of the corresponding macro-infrastructure (macro- drainage, street /road access system, etc); · The TOR should make a distinction between the level of community mobilization to be undertaken in the PSI and the PDLI areas of influence, in order to avoid creating expectations in areas that are unlikely to be directly benefited by works, especially in peri-urban areas of cities; · The social/community mobilization work should be confined to social that necessary for preparing the subproject and for identifying basic social needs (which should be brought to the attention of the public authorities responsible for meeting such needs); 37 · The social development item (community capacity building and income generation) should not form part of the TOR for execution by a private consultancy firm since the social inclusion process should be coordinated and executed by the public authorities themselves. Consultancy firms should be restricted to identifying the scope, and demand, for capacity building/training; · The products of the TOR should not be analyzed by different bodies. In the event of this not being possible an effort should be made to analyze the TOR products jointly or, at least, in a standardized way. · It proved necessary to reduce the number of products given the amount of administrative work needed for the approval and payment of the same; · The products should not be partly or wholly repeated in subsequent products required under the TOR, thus reducing an excess of subdivisions which makes the approval of intermediate products difficult. (b) Cofinanciers (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society) 38 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Appraisal Estimate Actual/Latest Components Percentage of (USD millions) Estimate (USD millions) Appraisal Project management, promotion, and studies 5.40 3.00 -44.44% Pre-investment 20.00 9.30 -53.50% Training 1.60 0.50 -68.75% Urban development policy 3.00 0.10 -96.67% Demonstration works 0.00 10.70 Total Baseline Cost 0.00 23.60 -21.33% Physical Contingencies 0.00 0.00 no change Price Contingencies 0.00 0.00 no change Total Project Costs 30.00 23.60 Front-end fee PPF 0.00 0.00 Front-end fee IBRD 0.30 0.30 no change Total Financing Required 30.30 23.90 -21.12% (b) Financing Appraisal Actual/Latest Source of Funds Type of Estimate Estimate Percentage of Cofinancing (USD (USD Appraisal millions) millions) Borrower 19.00 5.00 -68% International Bank for Reconstruction and Development 30.30 23.90 -21.12% Total Project Costs 49.30 28.90 -41.38% It should be noted, that the project was essentially much less expensive than originally estimated at appraisal. Several factors contributed to the much lower than anticipated appraisal project estimates: (i) efficiency savings as several appraisal cost estimates turned out too high; and (ii) exchange rate fluctuations. 39 Annex 2. Outputs by Component Project Management, Promotion and Studies. The project management component supported the project management unit, the contracting of the project management firm, and a set of promotion activities and studies. The number of studies undertaken fell one short of appraisal estimates (four studies were realized instead of the estimated five at project appraisal ­ although the target was revised to four at the mid-term review), whereas the number of workshops undertaken was five in line with appraisal estimates. Three major studies were undertaken by the PMF under the auspices of this component: (i) a comparative analysis of best-practice approaches in Brazil to integrated WSS/urban upgrading subprojects for low-income areas; (ii) a review of the indicators and the impacts of integrated WSS/urban upgrading subprojects for low-income areas in Brazil; and (iii) policies and strategies in Brazil for providing WSS services to low-income areas. Although it completed the first two studies, the demobilization of the PMF during the last year of project implementation negatively impacted its ability to complete the last of the three studies by project close. Pre-Investment. The targets for this component were the preparation of the integrated local development plans (PDLIs) and the integrated WSS engineering designs (PSIs). During project appraisal, it was estimated that 30 such plans would be prepared. Despite the implementation delays, by April 2008 the project was able to prepare 34 PDLIs and 33 PSIs.10 The project complied with the PAD investment thresholds for water supply and wastewater treatment, but not for wastewater collection. This may be largely explained by the assumption in the PAD that a simplified `condominial' approach to wastewater collection would be used, whereas a number of the interventions adopted `conventional' sewerage in difficult, densely-occupied peri-urban settings. When the costs of conventional wastewater collection technologies are considered (included in the table but not originally identified in the PAD), the difference between the costs actually incurred and the PAD thresholds is more consistent. Furthermore, investment cost estimates for new/rehabilitated/expanded water supply systems (intake, treatment, storage, distribution) and wastewater systems (collection, transfer, treatment, disposal), undertaken using the average cost and expansion data from GoB's SNIS annual series between 2001 and 2006, provide the following per capita costs: R$428 for water (US$214) and R$797 for wastewater (US$398) ­ all in prices adjusted to 2006 ­ which imply that the PROSANEAR-TAL per capita WSS costs are, in fact, reasonable. 10On December 2007, at the official closing date of the project, the actual numbers were slightly lower at 33 PDLIs and 27 PSIs. 40 Table A2.1: Investment per capita thresholds set out in Project Appraisal Document Threshold 1999 value 1999 threshold 2007 value threshold threshold adjusted for inflation based on PSI designs (US$) to 2007 levels11 (US$) (US$) Water supply (storage and 70 115 104 distribution only) Water supply (intake, 100 165 104 treatment, storage and distribution) Sewage collection 80 132 242 (condominial system) Sewage collection 120-150 198-247 242 (conventional system)* Sewage treatment 40 65 41 Community participation 22 36 30 Source: Detailed analysis of a sample of 24 PDLIs and PSIs *Data not given in PAD The amount of financial resources leveraged for contracting the civil works interventions designed under the subprojects prepared by PROSANEAR-TAL was just over RS$ 1 billion (or some US$ 600 million equivalent). Most of these financial resources, namely R$ 802 million, will be funded through the PAC, whereas the remainder will be funded through other federal and state government grant/credit programs such as PROMETROPOLE, FNHIS, and BNDES. The increase in leveraged funding above the levels estimated in the PAD is deemed to be due to the complex nature of the interventions identified in the pre-investment designs ­ that are integrated in nature and include numerous urban services and related interventions to complement the WSS services. In addition, the number of participating municipalities that conducted PDLIs and PSIs increased from an estimated 30 during appraisal to 33 and 34, respectively by project close. The leveraging of funds was not, however, included in the PAD's project effectiveness our outcome indicators. Training. The number of capacity building/training courses and TA events realized was significantly higher than that planned during appraisal. At the end of the project, 109 training events had been undertaken compared to the 64 envisaged in the PAD ­ demonstrating strong demand from the various stakeholders in receiving capacity building/training activities in the undertaking of integrated WSS/urban upgrading subprojects. This demand was confirmed by the respondents to the stakeholder survey (see Annex 4 for more details). The project provided training to 29 municipal governments and one state government, to 15 CAIXA teams in 12 states, and also to consultant firms when it became clear that the undertaking of PDLIs and PSIs was something that many consultant firms were not well prepared for. The training courses and TA focused on improving the capacity of municipalities and consultants to undertake 11The per capita thresholds are calculated in Brazilian Reais, adjusted for local inflation and then re- calculated back to US dollars. 41 urban planning, on reviewing the environmental impacts and resettlement implications of the interventions, and on the integration of community development and infrastructure interventions to improve overall sustainability. Urban Development Policy and Planning Support. This component foresaw the inclusion of TA to municipalities, and studies, including the development of an urban development policy. Although specific TA and training events were provided to the municipalities, as described above, the specific studies outlined in the outputs for this component were not undertaken. The taking office of a new administration, and the subsequent replacement of the Special Secretariat for Urban Development by the Ministry of Cities, resulted in a number of changes in government priorities. The elaboration of a national urban development policy and of a national system of urban indicators was deemed of high priority by MoC and was carried out by the Special Secretariat for Urban Programs using MoC's own budget. 42 Monitoring Indicators Targets Components Sub-Components Indicators At appraisal At reformulation At project end (April 2008) Pre-investment Integrated local # of PDLIs elaborated / 30/30 30/30 34/30 development plans # of selected areas (PDLIs) Integrated WSS # of engineering designs 30/30 30/30 33/30 engineering designs prepared / (PSIs) # of selected areas Investment # of civil works completed 0 2 1 Studies # of studies realized / 5/5 4/4 4/4 # of studies programmed Training # of courses realized / 64/64 120/120 109/120 # of courses programmed Promotion # of workshops realized / 5/5 5/5 5/5 # of workshops programmed Urban Support to Urban # of municipalities surveyed 80/80 Not included Not included Development Development Politics for SNIU / # of planned Support to PRODIN # of PRODIN realized / 8/8 target 43 Project Effectiveness Indicators Components Indicators Targets At appraisal At reformulation At project end (April 2008) PDLIs # of actions implemented in the NA NA NA intervention area / # of PDLI actions implemented # of PDLI implemented / # of PDLI elaborated 30/30 Not included 33/34 PSIs # of PSI implemented / # of PSI 30/30 Not included 2/33 (demonstration elaborated projects) Social Works # of community participation plans 30/30 Not included 33 elaborated # of beneficiary agreements obtained / # 100% of families in area Investment costs Cost per capita foreseen in preliminary <100% Not included per capita investment ceiling Water distribution < 100% Water distribution and storage < 100% Wastewater collection >100% W/water collection and treatment >100% Mobilization of resources Amount of financial resources contracted NA Not included US$600 million for subprojects prepared (in PAD text: US$300 million) Cost recovery # of service providers that have adopted 30/30 Not included None cost recovery and subsidies / # of TAL beneficiaries Local Project Management # of beneficiaries that have created a local 30/30 30/30 100% (preliminary Units multi-disciplinary PMU data from municipal stakeholders survey; 16 respondents) 44 Annex 3. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Names Title Unit Responsibility/ Specialty Lending Yoko Katakura Financial Analyst / TTL LCSFW Abel Mejia Sector Manager LCSFW Alexander Bakalian Engineer LCSFW Caroline van den Berg Economist LCSFW Paula Pini Community Participation / Resettlement Specialist Braz Menezes Urban Development Specialist Cecile Ramsay Urban Development Coordinator for Brazil Jose Baigorria Senior Procurement Specialist Geise Santos Team Assistant Supervision/ICR Martin Gambrill Senior Water Engineer, TTL LCSUW Juliana Garrido Operations Analyst LCSUW Jose Janeiro Senior Finance Officer LOAFC Karina de Souza Marcelino Team Assistant LCC5C Alexandre Fortes Consultant - safeguards LCSUW Paula Pini Senior Social Development Spec LCSUW Adriana Weisman Operations Officer OPCCE Luciano Wuerzius Procurement Specialist LCSPT João Vicente Consultant ­ financial management LCSFM Rosa Bellido Language Program Assistant LCSUW 45 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle No. of staff weeks USD Thousands (including travel and consultant costs) Lending FY95 13.50 FY96 24.62 FY97 53.62 FY98 126.45 FY99 46.43 FY00 6 17.06 FY01 0.05 FY02 0.00 FY03 0.00 FY04 0.00 FY05 0.00 FY06 0.00 FY07 0.00 FY08 0.00 Total: 6 281.73 Supervision/ICR FY95 0.00 FY96 0.00 FY97 0.00 FY98 0.00 FY99 0.00 FY00 3 10.37 FY01 14 59.15 FY02 17 73.07 FY03 17 80.79 FY04 28 95.47 FY05 20 86.34 FY06 15 84.28 FY07 15 82.12 FY08 6 25.05 Total: 135 596.64 46 Annex 4. Beneficiary Survey Results Introduction Surveys of the direct beneficiaries were conducted in two of the municipalities where the corresponding civil works resulting from the pre-investment subprojects were implemented. The major objective of these beneficiary surveys was to understand to what degree the project's underlying assumptions proved to be valid, and what the impact of the subprojects has been to inform the different stakeholders as to how much the subprojects are able to generate economic growth and reduce poverty. The results of these surveys give a first impression as to on what the two demonstration civil works interventions achieved. Survey set-up The surveys were conducted in the municipalities of Guarulhos and São José dos Campos, both located in the state of São Paulo. The reason for selecting these two interventions was that they represented the only two municipalities where civil works were initiated and (in the case of São José dos Campos, wholly, and, in the case of Guarulhos, partially) completed with PROSANEAR-TAL support. As such, they were the only two interventions where `before' and `after' analyses could actually be carried out. Table A4.1 Basic characteristics of the two municipalities Characteristics Guarulhos São José dos Campos Population of municipality 1,200,000 600,000 Name of area of intervention Vila Nova Cumbica Santa Cruz Area of PSI intervention (m2) 78,580 19,867 Population density in PSI area (m2/person) 14.6 53.6 Actual duration (months) 18 16 Cost of pre-investment (R$) 588,001 427,196 Actual investment cost generated (R$) 24.0 million 2.7 million Population in area of intervention 5,368 370* Average household size (people per household) 3.3 4.0 Estimated cost of investment per capita (R$) 4,479 7,429 Note: the small beneficiary population size of the area of intervention in São José dos Campos is explained by the fact that the city has already upgraded most of its slum areas, and only two small poverty pockets remain (of which Santa Cruz is one). In each pilot area of intervention a statistically significant number of households were randomly surveyed on issues related to: (i) their experience and impressions concerning the PROSANEAR-TAL project; (ii) access to public services after the pilot subproject interventions; (iii) their assessment of the impacts of the subproject interventions; and (iv) their overall satisfaction with the subproject interventions. The surveys took place in March 2008 and, to the extent possible, was compared to the baseline survey data collected in 2004 as part of the pre-investment process. Key findings Community Participation Community participation was an important feature of the PROSANEAR-TAL methodology and is fully integrated into the subproject preparation process from its inception. Name recognition of the PROSANEAR-TAL project was high: more than 80 percent of the households in both localities were aware of its existence. The respondents stated that the most effective form of communication between project actors and the community was through the periodic community meetings carried out during subproject preparation and civil works implementation. 47 In Guarulhos, 89 percent of the households participated in project preparation, of which more than 80 percent of respondents stated that they effectively participated, and the remainder did so more sporadically. Interest in the actual implementation of the civil works garnered a similar interest, with 88 percent of the households participating in this subsequent phase of subproject implementation, with almost two thirds of the households participating on a regular basis. In São José dos Campos community participation rates were lower ­ both in actual participation rates and in the intensity of the participation. In the project area, 74 percent of the households participated in the subproject pre-investment preparation, of which about 55 percent did so effectively and 45 percent more sporadically. The implementation of the works garnered more interest, with 80 percent of the households participating in this phase of the process, with almost two-thirds of the households participating on a regular basis. WSS Services As can be seen from figures A4.1 and A4.2, the `before project' conditions with regard to WSS services differed markedly in the two pilot areas. In Guarulhos, only 29 percent of the households had access to the formal water supply network in 2004, while 54 percent had such access in São José dos Campos. Similar trends were detected for wastewater collection, where only four percent of the households in Guarulhos had access to the utility's network compared to 46 percent in São José dos Campos. When those households with illegal connections are included as being part of the utility's network, however, the access rates in 2004 were 33 percent in Guarulhos and 89 percent in São José dos Campos. As can be seen in Figure A4.1 and A4.2, access became universal once the pilot subprojects were implemented. Not only did access improve, so did the quality and/or convenience of this access. In 2004, two percent of the households in Guarulhos did not have access to a sanitation facility, while 14 percent had a sanitation facility but outside of the house. In São José dos Campos, eight percent of the households had no access to a sanitation facility in their house; by 2008, all households surveyed had a sanitation facility in their home. With the improvements in access, households also had to start paying for these and other services. In 2004 only eight percent of the households in Guarulhos and 54 percent in São José dos Campos paid for their water and sewerage services; with universal access to the utility's water and sewer network, more than 98 percent of households are now paying for these services. Access to other services The pilot subprojects also provided other urban services as part of the integrated urban infrastructure upgrading package. These services included housing, solid waste collection, drainage (only in Guarulhos), as well as housing/land title regularization. Drainage In 2004, 45 percent of the households in Guarulhos mentioned they had experienced flood problems; in 2008, only four percent of households mentioned their experiences with difficulties related to flooding. Solid waste collection In 2004, all households in São José dos Campos used informal collective solid waste collection arrangements. In Guarulhos, a small number of households (16 percent) had access to formal solid waste collection systems, and the 48 majority of the households depended (83 percent) depended on informal arrangements. By 2008, formal systems of solid waste collection were close to universal in both localities. Figure A4.1 Before and After Access to Water Supply in Guarulhos and São José dos Campos Access to Water Supply in Guarulhos 100% 80% 60% 40% 20% 0% Utility Netw ork Neighbors Well Illegal connections 2004 2008 Access to Water Supply in São José dos Campos 100% 80% 60% 40% 20% 0% Utility Neighbors Well Illegal Network connections 2004 2008 Housing Housing was one of the most important interventions in the pilot areas (and also tends to be the most costly intervention). The housing improvements were measured in terms of space (number of rooms) and quality of construction (building materials). In Guarulhos, the proportion of households that had access to four rooms or more in their home increased from 12 percent in 2004 to 100 percent in 2008. In São José dos Campos, it increased from 36 percent in 2004 to 87 percent in 2008. The quality of housing also improved as almost all houses surveyed in the two localities now mention that their houses are made from bricks. 49 Figure A4.2 Before and After Access to Wastewater Collection in Guarulhos and São José dos Campos Access to Wastewater Collection in Guarulhos 100% 80% 60% 40% 20% 0% Utility Netw ork Netw ork by community No collection 2004 2008 Access to Wastewater Collection in São José dos Campos 100% 80% 60% 40% 20% 0% Utility Network by Pit latrine No collection Network community 2004 2008 Impacts of PROSANEAR-TAL pilots The households in the two pilot areas were also asked as to how they perceived the quality of their access to public services. As shown in Figure A4.3, the surveyed households perceived their access to public services (ranging from water supply, sanitation, drainage and solid waste collection to postal services, public telephones and street lighting) had undergone an overall improvement. The improvements were most significant for those services that the PROSANEAR-TAL project set out to address, namely water supply, sewerage, solid waste collection and drainage. An interesting development is that households also perceived improvements in public security: 47 percent of the respondents mentioned a clear improvement (which municipal representatives during the ICR beneficiary roundtable confirmed). The improvements were larger in Guarulhos, where 57 percent of the households mentioned a (significant) improvement compared to only 30 percent in São José dos Campos. Not only were there more households in Guarulhos who experienced an improvement, the level of this improvement was also higher as one third of those that mentioned an improvement 50 thought the improvement to be significant, compared to only one in nine in São José dos Campos. The lower levels of improvements in public security in São José dos Campos may be due, inter alia, to the fact that a drugs gang had been active in the area prior to the intervention and continued to be afterwards. The improvements in public/common land- use, public lighting, public access ways (a major point of contention in São José dos Campos with the drugs gang) and better recreational facilities most probably all helped to improve public security. Table A4.2 Perception concerning improvements in the living environment in Guarulhos and São José dos Campos Variable Guarulhos São José dos Campos Percentage of households that perceived a 57 30 (significant) improvement in public security Percentage of households that perceived a 94 28 (significant) improvement in public illumination Percentage of households that perceived a 91 42 (significant) improvement in recreational facilities Percentage of households that perceived a 89 79 (significant) improvement in "estética da área" Percentage of households that perceived a 92 34 (significant) improvement in available space at home Apart from the changes in access to public services, there have also been major changes in access to other services not directly related to water supply and sanitation. As can be seen in Figures A4.3 and A4.4, households also perceived (significant) improvements in access to shops, schools and day care facilities, health posts and transport. The changes in access were especially notable in Guarulhos. Figure A4.3 Access to public services not directly related to WSS services in Guarulhos Perception on Service Levels in Guarulhos Public security Street lighting Public phones Mail services Energy Supply Flood control Solid w aste Wastew ater collection Water Supply 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Deteriorated Constant Improved Much Improved 51 Figure A4.4 Access to public services not directly related to WSS services in São José dos Campos Perception on Service Levels in São José dos Campos Public security Street lighting Public phones Mail services Energy Supply Flood control Solid waste Wastewater collection Water Supply 0% 20% 40% 60% 80% 100% Deteriorated Constant Improved Much Improved The households surveyed perceived that the pilot interventions also had a series of positive economic and social impacts. The level of community organization and interpersonal relationships with neighbors showed positive improvements ­the interactions thus having possibly built social capital in these neighborhoods. This, in turn, may have had a positive impact on public security perceptions. These effects were stronger in Guarulhos than in São José dos Campos. As for economic effects, households were also asked how they perceived improvements in their economic prospects during subproject implementation. Households in São José dos Campos stated that overall their economic prospects had improved, however households in Guarulhos mentioned their prospects had deteriorated. In Guarulhos, households perceived an overall decline in work and income opportunities (as defined by the difference between the percentage of household that experienced an improvement in either income or job opportunities and those that experienced deterioration). The perceptions in Guarulhos may have been associated with the fact that the households ­ because of space constraints ­ had to be relocated a significant distance to another area to ensure that sufficient space in the original area could be created for the provision of new housing and services for the remaining population. For those that had to relocate, income and job opportunities may have been affected as some of those surveyed could have felt that they were starting a new life in a different place, some eight kilometers from the where they used to live. 52 Figure A4.5 Socio-economic impacts before and after subproject implementation in Guarulhos Perception on Socio-Economic Impacts in Guarulhos Neighborhod relations Community organization Recreation Employment Income Education 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Deteriorated Constant Improved Much Improved Figure A4.6 Socio-economic impacts before and after subproject implementation in São José dos Campos Perception on Socio-Economic Impacts in São José dos Campos Neighborhood relations Community organization Recreation Employment Income Education 0% 20% 40% 60% 80% 100% Deteriorated Constant Improved Much Improved 53 Figure A4.7 Impact on income distribution before and after PROSANEAR-TAL project Income distribution in São José dos Campos Income distribution in Garulhos 50% 50% 40% 40% 30% 30% 20% 20% 10% 10% 0% 0% Without Up to 1 MS1-2 MS 2-3 MS > 3 MS Without Up to 1 1-2 SM 2-3 SM > 3 SM income income SM 2004 2008 2004 2008 Source: Beneficiary surveys ­ the monthly minimum salary was R$380 at the time of the survey in 2008, and R$260 in 2004 (period April 1, 2004 to March 31, 2005). Interestingly, the perception of survey respondents of their employment and income opportunities remaining constant or deteriorating (as was also registered in São José dos Campos) is not necessarily consistent with income trends in the two localities between 2004 and 2008, as is shown in Figure A4.7. As can be seen, the income distribution changed significantly before and after the subproject works were implemented in Guarulhos and in São José dos Campos. The figures show that the percentage of households without any income reduced significantly, while more households obtained access to higher incomes. At the same time, the value of a minimum salary increased above the rate of inflation ­ resulting in real income increases. As no rigorous impact evaluation has been conducted, it is not possible to attribute these income impacts directly to the PROSANEAR-TAL project interventions, especially since GoB introduced and/or improved other poverty-focused initiatives during the project implementation period such as the Bolsa Família CCT. However, the CCT program was implemented in 2003 and may have at least partially been absorbed in the 2004 income survey results. Unfortunately, the survey could not detect the effect of Bolsa Família because no detailed income questions were included in the survey. Table A4.3 Percentage of households that perceive overall improvements (deterioration) in income and job opportunities Variable Guarulhos São José dos Campos Perceptions regarding overall improvement (decline) -13 12 in income (%) Perceptions regarding overall improvement (decline) -25 5 in job opportunities (%) Perceptions regarding percentage of households who 88 73 perceive improvement in real estate values in area of interventions (%) It is nevertheless possible that at least some of these income effects were due to the pilot subproject interventions, given the following specific results of the surveys: o There was a marked improvement in public security: 47 percent of the respondents mentioned a (significant) improvement (which municipal representatives during the ICR beneficiary roundtable confirmed); 54 o The improvements in public security may have had a positive impact on income growth, while the significant improvements in access to schools (49 percent) child care centers (44 percent), health posts (11 percent ­ but almost three times higher for the extremely poor), transport (55 percent) and shops (59 percent) may have provided residents with more time and opportunities to obtain access to work and hence to income (household perceptions in São José dos Campos suggest such a hypothesis to hold, although the data from Guarulhos does not ­ possibly because of the aforementioned relocation effect); o For about 69 percent of the surveyed households, titling of land and property, or at least the formal provision of an address, had positive impacts ­ since prior negative discrimination associated with having an address in a `slum' abated, which may explain the changes in income, work opportunities and perceived higher real estate values. Overall satisfaction In Guarulhos, 92 percent of the households in the areas of project intervention were satisfied. In São José dos Campos the percentage of households was much lower ­ with only 54 percent of the households being satisfied with the results of the interventions. There may be many different reasons for this particular result, but the different conditions at the start of the respective subproject interventions may have played a significant role in explaining this variance. In São José dos Campos, access to water, sewerage and solid waste services was significantly higher when the intervention took place. As a result, the benefits in terms of improved access may have been considered less, while many more households started to pay for services that they did not necessarily used to pay for in 2004 (due to the high number of illegal connections or to the use of neighbors' services). In addition, the level of community participation in subproject preparation and subsequent works implementation in São José dos Campos was lower than in Guarulhos. Conclusions Survey respondents mentioned many positive changes once the civil works interventions corresponding to the subprojects were implemented, ranging from better access to different types of public services, improvements in public security, reduced discrimination because of legalization of the neighborhood, resulting in better work opportunities and consequently increases in income (although the data are not very consistent possibly because of the effect of relocation in the Guarulhos case). These outcomes and impacts are not necessarily linear in nature but can be seen as forming a cascade of positive loops which are difficult to disentangle (since improvements in public security, for example, may result in increased incomes, which in turn may reinforce improvements in public security). The above analysis cannot disentangle these real impacts as it only measures conditions 'before' and `after' the respective subproject intervention not `with' and `without' the subproject interventions. As GoB has embarked on a number of initiatives in recent years which may positively impact socio-economic indicators, it is hard to determine to what extent the changes that have been observed are the result of PROSANEAR-TAL and what is caused by other factors. The results suggest that more in-depth analyses of the effects and impacts of integrated WSS/urban upgrading programs would be useful to further improve policy and/or program design and implementation, especially as the evaluation team had little time to comprehensively 55 analyze the survey data at project close given the demobilization of the project management firm by the Ministry of Cities. 56 Annex 5. Institutional or Stakeholder Survey Results Introduction The institutional or stakeholder survey that was undertaken in March 2008, was essentially a perception survey to better understand the levels of awareness and satisfaction with the PROSANEAR-TAL project. It was also carried out in order to provide a platform of discussion for the beneficiary stakeholder workshop that was held in April 2008 to further review the project's impacts and strengths and weaknesses. The survey was sent to the 30 municipalities that participated in the PROSANEAR-TAL project of which 16 completed the questionnaire , representing a response rate of over 50 percent. Results of the Stakeholder Survey The survey revealed that most municipalities had been working prior to the PROSANEAR-TAL project with urban projects that employed community participation approaches. Most municipalities (94 percent) stated that they were planning to adopt, or had already adopted, integrated WSS/urban upgrading approaches to planning service delivery in low-income neighborhoods. A similar proportion of municipalities stated that they were planning to adopt, or had already adopted, participatory methodologies for the preparation of future WSS projects in poor urban neighborhoods. More than 80 percent of the municipalities that responded to the questionnaire are planning to institutionalize the multi-disciplinary teams for the implementation of the works that resulted from the PROSANEAR-TAL subprojects. Overall, satisfaction with the municipal teams that supervised the preparation of the pre- investment subprojects was high and the municipalities were either maintaining their teams, or were planning to maintain them, for the implementation and post- implementation phases of the subprojects. The questionnaires revealed that the municipalities thought that the most important reasons behind delays in the implementation of the subproject were the delays in disbursement by MoC, and a lack of preparation/skills of the consultant firms that undertook the participatory, integrated WSS/urban upgrading subproject engineering designs. The latter fact resulted in an adjustment of MoC's training program under the project to include specific training and TA for the consulting firms to improve their capacity to prepare the subprojects in a timely manner. Legalization of land tenure was seen by the municipalities as on of the major challenges to be addressed during subproject preparation. Close behind this issue was the challenge of resettlement identified in the subproject designs. A sub-sample of subproject designs provided very detailed data on the engineering designs and showed that ­ given the very high population densities in many of the subproject intervention areas ­ resettlement (within the intervention area) and relocation (outside the intervention area) often had significant impacts on the complexity and the cost estimates of the subproject interventions. The sub-sample of designs revealed plans to resettle 6,368 households and to relocate another 3,341 under the corresponding interventions. These numbers translate into more than 6 percent of the households within the areas of subproject interventions were identified for resettlement or reallocation. The economic-financial evaluation of the subprojects also posed difficulties as the integrated character of many of the interventions made it impossible to apply the standard WSS analysis provided in the PAD/standard 57 TOR, and made it difficult to disentangle the different benefits and to understand the precise benefits of such integrated approaches. Figure A5.1 Factors that contributed to project delays Factors which contributed to delays in subproject preinvestment preparation (05) 1. Lack of commitment from the consulting firm 4.18 2. Requirement to undertake procurement following 2.21 Bank guidelines 3. Lack of clarity in the terms of reference 2.33 4. Lack of preparedness/capacity on the part of the 1.43 municipal team 5. Inability of municipality to receive payment 1.85 transfers given its periodic fiscal ineligibility 6. Delays in payment transfers by MoC 3.67 7. Inadequate capacity of the consulting firm's 2.57 technical team Note: rating from 0 to 5 with 0 the least important and 5 the most important contributors to delays Figure A5.2: Key technical issues in project preparation Issues that presented most difficulty during the preparation of subprojects (05) 2.73 1. Monitoring indicators 2.38 2. Economicfinancial analysis 2.95 3. Resettlement 3.14 4. Land tenure 2.76 5. Environmental aspects 6. Lack of clarity regarding the definition of the 1.43 physical and technical scope of the PDLI and PSI Note: rating from 0 to 5 with 0 the least important and 5 the most important issues. 58 Figure A5.3: Evaluation on Performance of different Stakeholders Evaluation of the project actors during preparation of subproject Ministry of Cities World Bank Caixa Econômica Federal Project management firm Municipal government Water companies 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% Project Caixa Water Municipal Ministry of managemen Econômica World Bank companies government Cities t firm Federal Satisfactory 47.6% 71.4% 75.0% 52.4% 57.1% 52.4% Average 47.6% 28.6% 5.0% 42.9% 28.6% 42.9% Unsatisfactory 4.8% 0.0% 20.0% 4.8% 14.3% 4.8% Overall, the municipalities were quite satisfied with the performance of the different stakeholders. The PROSANEAR-TAL project management firm was awarded the highest satisfaction ratings; the lowest satisfaction ratings were given to the WSS service providers, CEF and MoC. All those municipalities that responded stated that their municipal administrations have incorporated the methodologies and experiences of the PROSANEAR-TAL project in the preparation of new projects of a similar nature. Whereas a similarly high number of municipalities were interested in receiving technical assistance to prepare subprojects similar to those of PROSANEAR-TAL. 59 Annex 6. Comments of Cofinanciers and Other Partners/Stakeholders 60 Annex 7. List of Supporting Documents The following documents, which were used for the preparation of this ICR, can be found on file: Project Documents: · Project Appraisal Report; · Legal Agreements; · Project Progress Reports; · Auditing Reports; · PSRs and ISRs; · Aide- Memoires. Other Background Information: · Report on Inputs for the Elaboration of the Implementation Completion Report of PROSANEAR-TAL (Relatório: Subsídios para Elaboração do Relatório de Conclusão de Implementação do PROSANEAR-TAL) including: (i) the Beneficiary Assessment applied in the São José dos Campos and Guarulhos areas of intervention; (ii) the results of the municipal questionnaires applied to all project beneficiary municipalities; (iii) a summary of the beneficiary roundtable carried out with participation of the main federal and municipal stakeholders and beneficiaries held in April, 2008; and (iv) presentations prepared by the Borrower and presented during the beneficiary roundtable. · Federal Government's Program for Acceleration of Economic Growth - http://www.brasil.gov.br/pac/ · Study on the Comparative Analysis of Sub-Project Designs for Integrated Environmental Sanitation and Urban Upgrading Interventions in Poor Areas (Análise Comparativa de Projetos de Saneamento Ambiental Integrados e Urbanização de Áreas Precárias); · Study on Impacts of Implementation of Integrated Environmental Sanitation and Urban Upgrading Interventions in Poor Areas (Impactos da Implantação dos Projetos de Saneamento Ambiental e Urbanização de Assentamentos Subnormais) · Draft of the Study on Policies for Serving Low-Income Population with Water Supply and Sanitation Systems (Políticas de Atendimento à População de Baixa Renda com Sistemas de Água e Esgoto) · Report on the National Seminar - PROSANEAR-TAL ­ "Best practices in Integrated Environmental Sanitation Projects" held in 2006 (Seminário Nacional PROSANEAR- TAL 2006 ­ Melhores Práticas de Projetos Integrados em Saneamento Ambiental). 61 IBRD 36210 70°W R.B. DE 60°W 50°W 40°W 30°W BRAZIL VENEZUELA GUYANA French LOW INCOME SANITATION Guiana Orinoco SURINAME (Fr.) AT L A N T I C TECHNICAL ASSISTANCE PROJECT C O L O M B I A Boa Vista (PROSANEAR) RORAIMA AMAPÁ O C E A N PROJECT MUNICIPALITIES: 0° Macapá PRE-INVESTMENT CARRIED OUT 0° Negro UNDER THE PROJECT NORTH Amazon Belém PRE-INVESTMENT AND SUBSEQUENT CIVIL WORKS CARRIED OUT UNDER THE PROJECT Manaus São Luís Parnaíba Amazon Fortaleza A M A Z O N A S Madeira Caxias Tapajós RIO GRANDE PA R Á CEARÁ MARANHÃO Teresina DO NORTE STATE CAPITALS Natal NATIONAL CAPITAL Purus Xingu PARAÍBA arA guaia PIAUI STATE BOUNDARIES João Pessoa PERNAMBUCOJaboatão JaboatJaboatão dos ão dos Paulista Paulista REGION BOUNDARIES Porto P E R U ACRE RecifeRecife INTERNATIONAL BOUNDARIES Velho Maceió Rio Branco Tocantins Guarapes Guarapes Olinda Palma NORTHEAST 10°S RONDÔNIA ALAGOAS TOCANTINS Aracaju 10°S leT SERGIPE Juruena Pseires B A H I A Feira de Santana Lauro de Freitas MATO GROSSO 0 200 400 600 Kilometers Salvador Salvador CENTER-WEST Valença 0 200 400 Miles F.D. Ilhéus Cuiabá BRASÍLIA Goiânia B O L I V I A G O I A S M I NoA S G E R A I S This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and any other information shown on this map do not imply, on the part of The World Bank MATO GROSSO arFncisc Group, any judgment on the legal status of any territory, or any Belo ESPÍRITO endorsement or acceptance of such boundaries. DO SUL Grande São Horizonte Ipatinga SANTO Campo 20°S Paragu Betim Grande SOUTHEAST VitóriaVitória ya São José Juiz de Fora 20°S Paraná SÃO PAULO dos Campos RIO DE JANEIRO CHILE PARAGUAY Guarulhos Nova Iguaçu São Paulo São Paulo Rio de Taboão da Serra aboão da Serra aboão Mogi das Cruzes Mauá See inset Janeiro PARANÁ AT L A N T I C São Bernardo PACIFIC do Campo Santo André Curitaba São José dos Campos BRAZIL STA. CATARINA Guarulhos OCEAN Florianópolis São Paulo O C E A N Taboão Mogi das SOUTH Palhoça da Serra Mauá Cruzes São Bernado Santo Andre do Campo RIO GRANDE São Vicente Santos Guaruja DO SUL Porto Alegre 0 50 km. 30°S ARGENTINA 30°S URUGUAY 80°W 70°W 60°W 50°W 40°W 30°W JUNE 2008