BRAZIL Bahia Poor Urban Areas Integrated Development Project: Viver Melhor II Report No. 127250 JUNE 29, 2018 © 2018 International Bank for Reconstruction This work is a product of the staff of The World RIGHTS AND PERMISSIONS and Development / The World Bank Bank with external contributions. The findings, The material in this work is subject to copyright. 1818 H Street NW interpretations, and conclusions expressed in Because The World Bank encourages Washington DC 20433 this work do not necessarily reflect the views of dissemination of its knowledge, this work may be Telephone: 202-473-1000 The World Bank, its Board of Executive reproduced, in whole or in part, for Internet: www.worldbank.org Directors, or the governments they represent. noncommercial purposes as long as full attribution to this work is given. Attribution—Please cite the work as follows: The World Bank does not guarantee the World Bank. 2018. Country— Bahia Poor Urban accuracy of the data included in this work. The Any queries on rights and licenses, including Areas Integrated Development Project: boundaries, colors, denominations, and other subsidiary rights, should be addressed to Viver Melhor II. Independent Evaluation Group, information shown on any map in this work do World Bank Publications, The World Bank Project Performance Assessment Report not imply any judgment on the part of The Group, 1818 H Street NW, Washington, DC 127250. Washington, DC: World Bank. World Bank concerning the legal status of any 20433, USA; fax: 202-522-2625; e-mail: territory or the endorsement or acceptance of pubrights@worldbank.org. such boundaries. Report No.: 127250 PROJECT PERFORMANCE ASSESSMENT REPORT BRAZIL BAHIA POOR URBAN AREAS INTEGRATED DEVELOPMENT PROJECT – VIVER MELHOR II IBRD-73440 June 29, 2018 Financial, Private Sector, and Sustainable Development Independent Evaluation Group ii Currency Equivalents (annual average) Currency Unit = Brazilian Real (R$) 2005 $1.00 R$2.33 2006 $1.00 R$2.15 2007 $1.00 R$1.76 2008 $1.00 R$2.35 2009 $1.00 R$1.74 2010 $1.00 R$1.66 2011 $1.00 R$1.89 2012 $1.00 R$2.05 2013 $1.00 R$2.36 Abbreviations AVSI Associazione Volontari per il Servizio Internazionale (Association of Volunteers for International Service) CAMA Centro de Arte e Meio Ambiente (Center for Art and Environment) CBO community-based organization CONDER Companhia de Desenvolvimento Urbano da Bahia (Bahia State Company for Urban Development) ERR economic rate of return FY Fiscal Year IBRD International Bank for Reconstruction and Development (of the World Bank Group) ICR Implementation Completion and Results Report ICRR Implementation Completion and Results Report Review IEG Independent Evaluation Group MCMV Programa Minha Casa Minha Vida (My House, My Life) M&E monitoring and evaluation NGO nongovernmental organization PAC Programa de Aceleração do Crescimento (Growth Acceleration Program) PIF poligonal de intervenção física (physical intervention area) PIS poligonal de intervenção social (social intervention area) PLANEHAB plano estadual de habitação (state housing plan) SEDUR Secretaria de Desenvolvimento Urbano (Secretariat for Urban Development) SENAC Serviço Nacional de Aprendizagem Comercial (National Service for Commercial Apprenticeship) SENAI Serviço Nacional de Aprendizagem Industrial (National Service for Industrial Apprenticeship) UGP Unidade de Gestão do Projeto (Project Management Unit) UTP Unidade Técnica do Projeto (Project Technical Unit) VMII Bahia Poor Urban Areas Integrated Development Project All dollar amounts are U.S. dollars unless otherwise indicated. Director-General, Independent Evaluation : Ms. Caroline Heider Director, IEG Financial, Private Sector Sustainable Development : Mr. José Carbajo Martínez Manager, IEG Sustainable Development : Ms. Midori Makino Task Manager : Ms. Maria-Elena Pinglo iii Contents Abbreviations ...................................................................................................................... ii Principal Ratings ................................................................................................................. v Key Staff Responsible......................................................................................................... v Preface............................................................................................................................... vii Methodology ................................................................................................................. vii Summary ............................................................................................................................. 1 Ratings ............................................................................................................................ 2 Lessons............................................................................................................................ 5 1. Background and Context................................................................................................. 7 2. Objective, Design, and their Relevance .......................................................................... 8 Objective ......................................................................................................................... 8 Components and Costs.................................................................................................... 8 Relevance of Objective ................................................................................................... 9 Relevance of Design ..................................................................................................... 10 Overall Objective ...................................................................................................... 10 Targeting ................................................................................................................... 11 Complementarity....................................................................................................... 11 3. Implementation ............................................................................................................. 12 Planned versus Actual Expenditure by Component...................................................... 12 Safeguards Compliance ................................................................................................ 13 Financial Management and Procurement...................................................................... 15 4. Achievement of the Objective ...................................................................................... 15 Objective 1: Access to basic services and improved housing....................................... 16 Outputs ...................................................................................................................... 16 Outcomes .................................................................................................................. 17 Objective 2: Improved access to social programs......................................................... 19 Outputs ...................................................................................................................... 19 Outcomes .................................................................................................................. 20 Other Outcomes ........................................................................................................ 23 5. Efficiency ...................................................................................................................... 23 6. Ratings .......................................................................................................................... 24 This report was prepared by Maria Elena Pinglo, who assessed the project in April 2018. The report was peer reviewed by Victor Vergara and panel reviewed by April Connelly. Romayne Pereira provided administrative support. iv Outcome ........................................................................................................................ 24 Risk to Development Outcome ..................................................................................... 25 Bank Performance ......................................................................................................... 25 Quality at Entry ......................................................................................................... 25 Quality of Supervision .............................................................................................. 26 Borrower Performance .................................................................................................. 26 Government Performance ......................................................................................... 26 IMPLEMENTING AGENCY PERFORMANCE .................................................... 27 Monitoring and Evaluation ....................................................................................... 27 7. Lessons.......................................................................................................................... 28 References ......................................................................................................................... 30 Tables Table 3.1. Planned versus Actual Expenditure, by Component ....................................... 13 Table 3.2. Resettlements Options as of March 2018 ........................................................ 14 Table 3.3. Situation of Resettlements as of March 2018 .................................................. 15 Table 4.1. Access to Basic Services and Improved Housing............................................ 17 Table 4.2. Job and Income-Generation Activities by Social Intervention Area ............... 20 Appendixes Appendix A. Basic Data Sheet.......................................................................................... 32 Appendix B. Vulnerability Map ....................................................................................... 36 Appendix C. List of Persons Met during the Independent Evaluation Group Mission .... 37 Appendix D. Visual Assessment of Project Areas before and after project completion .. 39 Appendix E. Borrower Comments .................................................................................... 47 Appendix F. Methodology ................................................................................................ 50 v Principal Ratings ICR* ICR Review* PPAR Outcome Moderately Moderately Moderately unsatisfactory unsatisfactory unsatisfactory Risk to Substantial Substantial Substantial Development Outcome Bank Performance Moderately Moderately Moderately unsatisfactory unsatisfactory unsatisfactory Borrower Moderately Moderately Moderately Performance Unsatisfactory Unsatisfactory Unsatisfactory * The Implementation Completion and Results Report (ICR) is a self-evaluation by the responsible World Bank department. The ICR Review is an intermediate Independent Evaluation Group product that seeks to independently verify the findings of the ICR. Key Staff Responsible Project Task Manager or Leader Practice Director Country Director Appraisal Ivo Imparato John Henry Stein John Briscoe Completion Alessandra Campanaro Anna Wellenstein Deborah L Wetzel vi IEG Mission: Improving World Bank Group development results through excellence in independent evaluation. About this Report The Independent Evaluation Group (IEG) assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the World Bank’s self-evaluation process and to verify that the World Bank’s work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, IEG annually assesses 20–25 percent of the World Bank’s lending operations through fieldwork. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which executive directors or World Bank management have requested assessments; and those that are likely to generate important lessons. To prepare a Project Performance Assessment Report (PPAR), IEG staff examine project files and other documents, visit the borrowing country to discuss the operation with the government and other in-country stakeholders, interview World Bank staff and other donor agency staff both at headquarters and in local offices as appropriate, and apply other evaluative methods as needed. Each PPAR is subject to technical peer review, internal IEG panel review, and management approval. Once cleared internally, the PPAR is commented on by the responsible World Bank country management unit. The PPAR is also sent to the borrower for review. IEG incorporates both World Bank and borrower comments as appropriate, and the borrowers’ comments are attached to the document that is sent to the World Bank’s Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public. About the IEG Rating System for Public Sector Evaluations IEG’s use of multiple evaluation methods offers both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (additional information is available on the IEG website: http://ieg.worldbankgroup.org). Outcome: The extent to which the operation’s major relevant objectives were achieved, or are expected to be achieved, efficiently. The rating has three dimensions: relevance, efficacy, and efficiency. Relevance includes relevance of objectives and relevance of design. Relevance of objectives is the extent to which the project’s objectives are consistent with the country’s current development priorities and with current World Bank country and sectoral assistance strategies and corporate goals (expressed in poverty reduction strategy papers, country assistance strategies, sector strategy papers, and operational policies). Relevance of design is the extent to which the project’s design is consistent with the stated objectives. Efficacy is the extent to which the project’s objectives were achieved, or are expected to be achieved, taking into account their relative importance. Efficiency is the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared with alternatives. The efficiency dimension is not applied to development policy operations, which provide general budget support. Possible ratings for outcome: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, and highly unsatisfactory. Risk to Development Outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized). Possible ratings for risk to development outcome: high, significant, moderate, negligible to low, and not evaluable. Bank Performance: The extent to which services provided by the World Bank ensured quality at entry of the operation and supported effective implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of supported activities after loan or credit closing, toward the achievement of development outcomes. The rating has two dimensions: quality at entry and quality of supervision. Possible ratings for Bank performance: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, and highly unsatisfactory. Borrower Performance: The extent to which the borrower (including the government and implementing agency or agencies) ensured quality of preparation and implementation, and complied with covenants and agreements, toward the achievement of development outcomes. The rating has two dimensions: government performance and implementing agency(ies) performance. Possible ratings for borrower performance: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, and highly unsatisfactory. vii Preface This is a project performance review of the Bahia Poor Urban Areas Integrated Development Project financed by the International Bank for Reconstruction and Development (IBRD) and implemented between 2005 and 2013 in two cities in the state of Bahia: Salvador and Feira de Santana. Original financing was anticipated to be $78.6 million, including a $49.3 million IBRD credit and $32.9 million borrower contribution. Actual costs were $137.2 million because of two additional financings and an additional government contribution of $56.3 million after project closure. The project sought to reduce urban poverty sustainably in the poorest and most vulnerable sections of Salvador by providing access to basic services and improved housing and social support services. It was designed when Bahia had the highest quantitative housing deficit in absolute numbers and the highest number of people living in slums in the country. The project follows a series of previous World Bank–financed urban development operations in Bahia that focused on integrating physical infrastructure and social services delivery in low-income communities. This project represents the World Bank’s first large-scale investment supporting urban upgrading at a state level. Methodology This assessment was based on a review of World Bank project documentation, supplemented by several sources of primary and secondary data collected during a field mission to Bahia and conducted between April 2 and April 13, 2018. Secondary data collected includes the original management information system data, geographic information system data, and all project-related information. Primary data collection gathered the perceptions of the affected stakeholders, including the government of Bahia, municipalities, project implementation units, academics, resettled beneficiaries, beneficiaries of trainings, and the cooperatives that received grants for private productive assets. Specifically, the assessment conducted six group interviews, and five group interviews with cooperatives, nongovernmental organizations, and community-based organizations. Appendix F describes the field methodology. This project was selected for an in-depth Project Performance Assessment Report (PPAR) for three reasons. First, the PPAR explores the degree of integration and cohesiveness in operations to provide insights into the value of an integrated approach to slum upgrading for poverty mitigation. Second, this PPAR will provide input to the Independent Evaluation Group’s (IEG) forthcoming major evaluation on urban resilience as part of a diverse set of project-level performance assessments. Third, it will provide insights into the longer-term impacts, sustainability, and replicability of urban upgrading interventions. IEG gratefully acknowledges the contributions of all respondents, including World Bank staff in Washington, DC and staff from the local office in Brazil. The mission is grateful with the project implementation unit and its technical team for their support during the mission. The mission was supported by Francesco de Villarosa a local senior consultant who led the group interviews and provided valuable sector knowledge and research support. viii Following standard IEG procedures, a copy of the draft report was sent to the relevant government officials and agencies for their review and feedback. Comments received have been incorporated into the report. 1 Summary Slum upgrading in Brazil has evolved slowly from slum eradication and relocation of its dwellers to underserved housing settlements in the outskirts of cities, to occasional, partial, municipal upgrades. In the mid-1990s and into the 2000s, internationally financed projects brought integration, participation, social outreach, and institutional development to the slum- upgrading approach in Brazil. The national ProMoradia program launched in 1995 to help families in situations of social risk to get better housing and a better quality of life. The ministry of cities was created in 2003 to elaborate, coordinate, regulate, and supervise urban development policy at the national level. There were two important, concurrent national programs for slum upgrading and housing during program implementation. The Growth Acceleration Program (Programa de Aceleração do Crescimento) was launched in 2007. For the first time, slum upgrading was operationalized as a national policy in Brazil, targeting 1.8 million families of slum dwellers. In 2009, the Ministry of Cities launched My House, My Life (Programa Minha Casa Minha Vida; MCMV)—one of the largest social housing programs in the world—as a housing policy instrument and as a countercyclical stimulus to the economy (given the 2008 global financial crisis). Comprehensive state interventions in slum upgrading have also been taking place in Bahia and, more specifically, in the Metropolitan Region of Salvador (the Bahia state capital city). The ProMoradia program, implemented between 1995 and 2007, provided housing improvement for 90,000 families in Bahia, 76,000 of whom reside in Salvador. However, ProMoradia ceased when the Growth Acceleration Program and then the MCMV program were initiated. Some of these interventions were concentrated in a region with more than 3,000 stilt houses in Salvador. Initially, an integrated and participatory approach was developed as a pilot in Alagados settlement, and later the same approach was applied in the Ribeira Azul program to a broader urban region. This program resulted from strong collaboration between different actors such as the Bahia state government, the World Bank, Cities Alliance, the Italian Development Cooperation, and the Association of Volunteers for International Service, and it had a decisive influence on the design of the Bahia Poor Urban Areas Integrated Development Project (VMII). The project’s theory of change indicates that integrated physical interventions (such as infrastructure, housing, equipment), and social interventions that strengthen civil society, improve social services to local communities, and provide job training and income- generation activities will generate stronger physical, human, and social capital, which are necessary (though not always sufficient) conditions for reducing or mitigating urban poverty. The theory suggests that such an approach can ultimately reduce urban poverty in the poorest and most vulnerable sections of the area of influence. The project was designed to reduce urban poverty in a sustainable manner in the poorest and most vulnerable sections of Salvador and certain other cities in the borrower’s territory by providing access to basic services and improved housing and social support services. Without an overarching indicator to validate or assess this main aim, this assessment uses the theory of change to assess the contribution of the subobjectives: access to basic services and 2 improved housing, and improved access to social services—both of which have the overarching aim of reducing urban poverty in a sustainable manner. Ratings The relevance of the project objectives is rated substantial. The project objectives remain relevant to country conditions. Although the quantitative housing deficit and poverty have declined over time, Bahia continues to have one of the largest housing deficits in Brazil. 1 The objectives were highly relevant to national and state strategies. Both, Bahia’s 2003–20 state strategic plan and the state housing plan for 2010–13 aimed at improving access to land with adequate basic infrastructure and promoting sustainability through social capital strengthening and implementation of productive inclusion programs. The project objectives were and still are relevant to World Bank Group strategies. At approval, the fiscal year (FY)03–07 country assistance strategy aimed to reduce extreme poverty, vulnerability, and social exclusion in Brazil under the strategic goal of a more equitable, sustainable, and competitiveness pillar. Similarly, the FY12–15 country partnership strategy stressed the need to improve the quality of public services for low-income households as its second strategic objective. The relevance of the design is rated modest. The underlying project logic adequately reflects links between project components to achieve the overarching objective of reducing urban poverty in the targeted area. However, given the size of the investment and the needs of the state, the project’s development objective was ambitious, and the likelihood of reducing urban poverty was overestimated. The integrated approach concept, which relied partly on the successful provision of both physical and social components, was considered critical for the project’s sustainability. Although the selection criteria of the target population were adequate, the social components’ areas of influence were much broader than those of the infrastructure components. Consequently, the project components often did not target the same beneficiaries geographically. Moreover, the concept of using a multisector approach to reduce urban poverty had no clear institutional arrangements or legal provisions to ensure that synergies between sectors are realized. The project’s achievement of its first objective—improving access to basic services and housing—is rated substantial. Three major outcomes show the achievement of this objective. First, a significant amount of infrastructure works were unfinished at project completion. However, based on information provided to the Independent Evaluation Group (IEG) team, all works the Bahia government committed to complete at project closure were executed successfully by the time of the project assessment mission. A year after completion, 2,138 households were improved, surpassing the target of 2,000. In addition, the number of households resettled from high-risk areas or unhealthy conditions increased to 1,448. Infrastructure works that supported the construction of paved roads and water and sanitation infrastructure works have resulted in an increase in access to those services. The project exceeded its goals for the number of people with secure ownership after project closure. Second, interventions focused on neighborhood improvement rather than housing, infrastructure, landscape, or environmental conservation. The project promoted a comprehensive and multidimensional (urban, social, and environmental) requalification of 3 neighborhoods based on a broad menu of housing solutions (versus unidimensional solutions, such as in the MCMV program). Housing interventions are integrated well to the community. Investments in infrastructure, house improvements, and equipment benefited both old and newly built areas. The advantage of retaining beneficiaries in the same territory was that dwellers could keep using their familiar local services, thus avoiding a rupture in the local communities’ social fabric. Beneficiary group participants agreed that having appropriate consultations with beneficiaries was crucial for the design and quality of houses. There were changes in the design and size of the houses based on needs the communities expressed. Third, the quality of housing improved notably compared with the poor housing conditions in which most resettled families had lived, such as stilt houses, semirural dwellings, and shacks. General habitability improved through elimination of risks (especially floods, high tide or fire risks, and exposure to pollution sources from open sewage), the provision of basic services such as water and sanitation, easing accessibility through paved roads, and garbage collection. The project’s achievement of its second objective—improving access to social programs—is rated modest. Three mixed outcomes show the modest achievement related to this objective. First, outputs related to improving access to social programs were delivered, but the integration of physical and social actions was somewhat limited. Only 41 percent of beneficiaries of social actions were selected from the infrastructure component because works were under way, and beneficiaries were usually scattered in rental homes. Moreover, different teams for physical and social components that rarely coordinated the execution of the activities. Second, the strengthening of civil society organizations and social capital has been central to the approach. However, information on actual results and impact is scattered and anecdotal. Structured interviews with community-based organizations (CBOs) revealed that professional management of the CBOs experienced comprehensive capacity building and growth, but the number of civil society organizations for which this materialized into concrete results was limited. Third, the project sought to increase the ability of the targeted population to enhance its income and job-generation opportunities, but the project did not have income or employment-generation targets. To address this information gap, IEG conducted structured interviews to draw lessons about what worked and what did not work to help beneficiaries find a job and enhance their income generation. Groups of beneficiaries of vocational training expressed their overall satisfaction with the quality of the training, but they rarely found a job afterward. There was consensus that further support was needed in strengthening workforce intermediation mechanisms to facilitate access to the labor market. The project efficiency is rated modest. Although the ex post economic rate of return was 16 percent (which is higher than the benchmark of 12 percent), the project efficiency is rated modest because of significant implementation delays caused by institutional and administrative inefficiencies. The delays were mostly attributable to changed political priorities and high staff turnover. The project had two implementation units, which increased 4 the interinstitutional transaction costs throughout the implementation period. Moreover, the government finalized and funded the resettlement activities a year after project closure. The project’s outcome is rated moderately unsatisfactory based on the substantial relevance of objectives and the modest relevance of design, the substantial contribution to improving access to basic services and improved housing, the modest contribution to improving access to social programs, and the project’s modest efficiency. The risks to development outcome are rated substantial. The risk that several community centers will not have adequate operation and management arrangements, and therefore will not be operated or maintained effectively, is significant. There is also a moderate risk that land titling (considered a key element of social inclusion) will not be resolved and might raise discontent among communities. Furthermore, the risk is significant that the current shift in the national political agenda could negatively affect the sustainability of the infrastructure built under the project, as well as the vision of integrated urban upgrading in Brazil. Bank performance is rated moderately unsatisfactory. The project’s challenges were an overambitious project development objective and an inadequate preparation period that complicated interinstitutional arrangements. More than four years lapsed between the initial concept note and project approval, mainly because the project activities were complex and multidimensional. To some extent, the project design underestimated the complexity and uncertainties inherent to a highly participatory slum-upgrading project, which which must allow time to correct unexpected problems. Project supervision suffered from a slow start and significant turnover in task team leadership during the later years of project implementation. However, the Brasilia-based co-task manager, who stayed in the position for most of the period from 2010 onward, partially mitigated this turnover. The World Bank’s role in helping the project move forward after it stalled for two years was critical. The World Bank focused its efforts on helping Bahia counterparts rebuild the implementation capacity that had been impaired during 2007–08. There were regular and extensive supervision missions to solve procurement issues. The World Bank team supervised the project closely, and the aide memoires did not identify any financial management or procurement irregularities. The implementation of civil works was delayed, and there were significant cost overruns. The World Bank carried out regular supervision on safeguards. The borrower’s performance is rated moderately unsatisfactory. Ownership of the project was high during the preparation phase. The change in state leadership after the gubernatorial elections in October 2006 and a change in all municipal department heads caused the project to stall for two years until late 2008. High staff turnover disrupted the continuity of project implementation, and the new government did not consider the project a priority. The state government was late in providing counterpart funding, which slowed project implementation and required an extension. Ownership improved after the restructuring and postproject closure as evidenced by the postclosure agreements and later by provision of the necessary counterpart funding to finish pending works. Having two implementation agencies caused interinstitutional transaction costs throughout the implementation period. The Project Management Unit developed new resettlement practices based on project experience, which 5 were adopted as standard practices throughout the state (monitored indemnification). These practices positively influenced resettlement and beneficiaries’ effective use of cash benefits. Resettlement, environmental monitoring, and financial management procedures were high quality. Lessons Adaptability to local needs, retaining beneficiaries in the territory, and integration between different types of interventions are key features in slum upgrading and social housing. VMII provided a menu of comprehensive and multidimensional interventions (a housing system versus the monodimensionality of MCMV) that can be extended and replicated. Highly participatory processes allowed changes in design and size according to the needs the communities expressed. Retaining beneficiaries in the same territory avoided rupturing the social fabric in local communities and allowed those communities to keep using familiar local services. Housing interventions are integrated well to the remaining areas, both through contiguity and through investments in equipment, infrastructure, and house improvements, thus benefiting both old and newly built areas. Weakness in social service delivery can jeopardize expected outcomes and sustainability. Flexible designs and adequate sequencing of activities might help ensure results. Looseness and low sustainability are common features of social interventions, in which capacity building is difficult to materialize into concrete outcomes, even when resources for social activities are abundant (as in VMII). Lessons learned from VMII can include the following: (i) flexibility at project design and successive prioritization (from participatory diagnostics) to avoid establishing too many fronts at the outset (with gradual enlargement based on results and growing institutional capacity); and (ii) better sequencing of activities, such as the inclusion of slum- upgrading beneficiaries in local social policies when slum-upgrading social work is completed, which implies supporting municipalities to build up capacity to attend them. It is crucial in slum upgrading projects to guarantee delivery of all outputs in the social activities cycle because even small failures can jeopardize the expected outcomes. Social activities such as CBO strengthening suffered from lack of delivery of productive equipment or lack of physical improvement. Additionally, the income-generation activities lacked workforce intermediation after vocational training. Consequently, stricter supervision on delivery of social activities outputs becomes critical if low institutional capacity or insufficient resources to deliver all social outputs are identified during implementation. Adoption of multiphase projects or a programmatic long-term approach in slum-upgrading projects might be appropriate when social capital strengthening is considered necessary to achieve long-term results. Social interventions in one target area, social intervention area (poligonal de intervenção social) Ribeira, outperformed all others, specifically regarding CBO participation in project activities. This is mainly the result of a long-standing presence of state and international institutions—for example, the Bahia State Company for Urban Development (Companhia de Desenvolvimento Urbano da Bahia) and the Association of Volunteers for International Service (Associazione Volontari per il Servizio Internazionale)—through different, sequential projects in this area, which built trust and partnership with local civil society to strengthen its organizations effectively. Stronger social capital also fosters continuity and sustainability (for example, service provision or dialogue with government), which are critical especially in situations of shifting political agendas and lower availability of resources. 6 Appropriate timing in the preparation of slum-upgrading projects is crucial. Studies and diagnostics require sufficient time (though not too long) to avoid frustrating local expectations or hampering integration. VMII preparation took two years, and most works were concluded eight years after project start. Although slum-upgrading interventions are complex and imply a high degree of uncertainty (including in resettlement), a 10-year project completion period is too long because diagnostics become outdated, local expectations are frustrated, project credibility is lost, and a high number of potential beneficiaries vanishes, especially when they are temporarily allocated to social rent arrangements. One consequence of this long project implementation in VMII was the difficulty in attracting housing beneficiaries to take part in social activities, thus jeopardizing the integration of physical and social interventions. Project experience shows that project preparation might develop a clear, realistic indication of target areas, which should not be the subject of political negotiation later. Participatory planning could then be conducted in these areas with beneficiaries to gain their support and consensus, including for resettlement. Physical and social activities should be planned and initiated jointly so that they can support the same beneficiaries and make integration effective. Continuity in project management is a success factor in complex institutional development and social capital–strengthening projects, allowing familiarity with the local context and building mutual trust with the affected communities. This is partly related to the need for long-standing support and a programmatic, multiphase approach in institutional development and social capital strengthening, as discussed previously. In these kinds of projects, both familiarity with the local context and mutual trust with local stakeholders are important and might take some time to build. Frequent turnover in project management implies the need to rebuild this capital with every change of task team leader. Therefore, it is recommended to limit these changes as much as possible or, if they cannot be avoided, include project management transition periods during which the outgoing and incoming task team leaders work together so that the former can support and legitimize the latter for local stakeholders. The Brasilia-based co-task manager, who stayed in the position for most of the period from 2010 onward, partially mitigated the project’s high turnover. It is important for World Bank project management to ensure that slum-upgrading projects in urban areas are consistent with existing city plans and are integrated seamlessly into World Bank Group sectoral and thematic operations. The design of this project included relevant multisector activities to reduce urban poverty. It envisioned active coordination with World Bank investments in sectors such as education and health. However, no clear institutional arrangements or legal provisions were made to ensure synergies between education and health, which affected the achievement of project components. Although the project followed a series of urban-upgrading projects and promoted a systemic approach that was successful in bringing multidimensional neighborhoods together, it had limitations caused by the project interventions’ lack of adherence to the relevant municipality’s master plan. José Carbajo Martínez Director, Financial, Private Sector, and Sustainable Development Department 1 Quantity deficit refers to demand for new houses and people who spend more than 30 percent of family income for rent. 7 1. Background and Context 1.1 Slum upgrading in Brazil has evolved slowly from slum eradication and relocation of its dwellers to underserved housing settlements in the outskirts of cities, to occasional, partial, municipal upgrades. In the mid-1990s and into the 2000s, internationally financed projects brought integration, participation, social outreach, and institutional development to the slum-upgrading approach in Brazil. The national ProMoradia program launched in 1995 to help families in situations of social risk to get better housing and a better quality of life. The Ministry of Cities was created in 2003 to elaborate, coordinate, regulate, and supervise urban development policy at the national level. 1.2 There were two important, concurrent national programs during program implementation. The Growth Acceleration Program (Programa de Aceleração do Crescimento; PAC) was launched in 2007. For the first time, slum upgrading was operationalized as a national policy in Brazil, targeting 1.8 million families of slum dwellers. In 2009, the Ministry of Cities launched My House, My Life (Programa Minha Casa Minha Vida; MCMV)—one of the largest social housing programs in the world—as a countercyclical stimulus to the economy (given the 2008 global financial crisis) and as a housing policy instrument. Given its significance, there is recognition of the MCMV program’s contribution to reducing the housing deficit and the provision of good standards housing. However, the program faced challenges, such as the lack of integration of peripheral housing interventions to the city fabric. This is attributed to the scarce supply of urban and social services compared with the growing, geographically concentrated demand associated with social housing; governance problems for large housing developments; poor maintenance; social disruption for removed families; and increasing crime and violence. Currently, the discussion is ongoing, and the government is searching for solutions and new strategies. 1.3 Comprehensive state interventions in slum upgrading have also been taking place in Bahia and, more specifically, in the Metropolitan Region of Salvador. The ProMoradia program, implemented between 1995 and 2007, provided housing improvement for 90,000 families in Bahia, 76,000 of whom reside in Salvador (the Bahia State capital city). However, ProMoradia ceased when PAC and then the MCMV program were initiated. Some of these interventions were concentrated in a region with more than 3,000 stilt houses in Salvador. Initially, an integrated, and participatory approach was developed as a pilot in Alagados settlement, and later the same approach was applied in the Ribeira Azul program to a broader urban region. This program resulted from strong collaboration between different actors such as the Bahia state government, the World Bank, Cities Alliance, the Italian Development Cooperation, and the Association of Volunteers for International Service, and it had a decisive influence on the design of the Bahia Poor Urban Areas Integrated Development Project (VMII). 8 2. Objective, Design, and their Relevance Objective 2.1 The project development objective, as state in the loan agreement (schedule 2 page 21) is to “reduce urban poverty in a sustainable manner in the poorest and most vulnerable sections of Salvador and certain other cities in the borrower’s territory by providing access to basic services and improved housing and social support services.” ii Components and Costs 2.2 The project had three components: urban infrastructure delivery, social services delivery, and institutional strengthening and project management. 2.3 The urban infrastructure delivery component was to finance urban upgrading of informal settlements (appraisal: $51.40 million; actual: $60.66 million). Specific activities included the following: • Infrastructure investments, such as urban drainage, street paving, water supply and sewerage, solid waste management, home improvement, and replacement housing • Infrastructure works, such as road links to enhance citywide mobility, macro-drainage works, and establishment of green areas and parks • Development of service land delivery and low-cost housing alternatives for the poor • Preparation of the engineering designs and technical assistance • Mobility access (roads and stairways) and sanitation network availability 2.4 The social services delivery component was to finance social services and physical facilities (appraisal: $19.45 million; actual: $8.01 million). Specific activities included the following: • Strengthening local community-based organizations providing services to residents, development of income and employment-generation activities, provision of health and environmental education services, direct support to families’ survival strategies, support to sport and cultural activities, and development of crime and violence prevention initiatives • Refurbishment or construction of facilities and buildings, such as day-care centers, kindergartens, buildings for primary and secondary education classes and health care, and sport and professional training facilities 2.5 The institutional strengthening and project management component was added to project design because of its complexity and relevance of social dimensions (appraisal: $7.75 million; actual: $12.2 million). Specific activities included the following: • Studies • Strengthening of institutions active in slum upgrading and urban poverty reduction at the local and state government levels 9 • Project management, including monitoring and evaluation (M&E) 2.6 Project dates and costs: The project was implemented between 2005 and 2013. Original financing was anticipated to be $82.2 million, including a $49.3 million International Bank for Reconstruction and Development credit and a $32.9 million borrower contribution. Actual costs at completion were $100.85 million. The reasons for cost increases included high inflation in the construction sector, higher housing construction standards, and high fluctuation of the Brazilian real against the U.S. dollar (29 percent of real value lost). At project completion, there were uncompleted infrastructure and resettlement activities that were finalized in October 2014. The state of Bahia increased its contribution to $87.7 million, resulting in a total project cost of $137.1 million. The project sought three level-two restructurings to extend the closing date for a total of 36 months. The first extension responded to significant changes in the geographic scope and results framework caused by delays from a new political environment, government priorities, and adjustments to the scope. The second and third extensions allowed for completion of project activities related to urban upgrading and household resettlement. Relevance of Objective 2.7 The overarching objective of reducing urban poverty by providing access to basic services, improved housing, and social support services was and still is substantially relevant. 2.8 Bahia’s quantitative housing deficit and poverty have declined over time, but the state continues to have one of the largest housing deficits in Brazil. iii At appraisal, Bahia had the highest quantitative housing deficit in absolute numbers. In 2015, the state’s housing deficit was the fourth largest quantitative housing deficit in the country (SEDUR 2015). The qualitative deficit is estimated at more than 2.2 million units, of which 54 percent of existing housing units are in Bahia, and almost 20 percent are concentrated in the Salvador Metropolitan Region. iv A high proportion of qualitative housing deficit is located in rural areas, but it is stagnating because of demographic trends in the countryside. 2.9 Poverty correlates strongly with informal housing and slums. Although stagnant in absolute values with about the same number of urban dwellers (about 3.3 million), there was a significant decrease in relative terms in the percentage of people living below the poverty line in Bahia—from 35.7 percent to 22 percent of Bahian households between 2001 to 2017. 2.10 In the fiscal year FY03–07 country assistance strategy, reducing extreme poverty, vulnerability, and social exclusion were key outcomes under the strategic goal of a more equitable, sustainable, and competitive Brazil. Project objectives remained relevant at project closure as stated in the FY12–15 country partnership strategy, which stressed in its second strategic objective the need to improve the quality of public services for low-income households. The objectives continue to be relevant in FY15–18 under its third focus area on inclusive and sustainable development by seeking to support the provision of more inclusive and sustainable urban services. With this area of focus, the World Bank Group emphasizes an integrated approach for promoting economic, environmental, and social sustainability. 10 2.11 During appraisal, the program was broadly in line with the general objectives of the national and state initiatives. The objective was also highly relevant in the context of Bahia’s statewide development strategy, Plano Estratégico da Bahia, 2003–20, which was aligned with the general objectives of PAC, launched during implementation in 2007. PAC operationalized slum upgrading in Brazil for the first time, targeting slums’ dwellers and allocating 55 percent of project costs to social and urban sectors. Although differences in their approach exist, the program was also broadly in line with the MCMV national program, which began in 2009 and targeted the poor by providing a 90 percent subsidy to beneficiaries in the lowest income bracket. The objective is also in line with the state housing plan for 2010–13, which had the objectives of improving access to land with adequate basic infrastructure and promoting sustainability through social capital strengthening and implementation of productive inclusion programs. Since 2016, there has been a shift in the national political agenda and a significant reduction in domestic resources, both of which have contributed to slowing the implementation of slum upgrading and social housing programs countrywide. Despite this slowdown, the objective of reducing urban poverty by providing access to basic services and improved housing and social support services was and still is relevant to Bahia’s needs. 2.12 The relevance of the objective is rated substantial. Relevance of Design OVERALL OBJECTIVE 2.13 The underlying project logic adequately reflects links to achieve the overarching objective of reducing urban poverty in the targeted area. However, given the size of the investment and the needs of the state, the project’s main development objective was ambitious, and the likelihood of reducing urban poverty was overestimated. 2.14 The project’s theory of change indicates that integrated physical interventions (such as infrastructure, housing, equipment, and social interventions that strengthen civil society, improve social services to local communities, and provides job training and income- generation activities to the communities) will generate stronger physical, human, and social capital, which are necessary (though not always sufficient) conditions for reducing or mitigating urban poverty. The theory suggests that this approach can ultimately reduce urban poverty in the poorest and most vulnerable sections of the area of influence. This theory of change is well anchored in literature (Imparato and Ruster 2003, Magalhães and di Villarosa 2012) and in its predecessor’s pilot projects. v However, it assumes that the type of physical and social intervention is delivered at a scale needed to make a difference. A lesson learned from the beneficiary group interviewes is that although social interventions are important to promoting continuity and sustainability of the support, they are often delivered as capacity building. Their design must be carefully considered and formed, and links to outcomes strengthened. Otherwise, missing aspects can jeopardize expected outcomes. For example, failures such as a lack of delivery of productive equipment, lack of community-based organizations’ (CBOs) physical improvement, lack of workforce intermediation after vocational training, and lack of long-term support to social capital strengthening can significantly hamper the impact of extensive capacity building. These missing elements show 11 that integration is a very complex process that needs solid design and arrangements so that failures in implementation can be avoided. TARGETING 2.15 Eight cities were selected initially, but the project had to be restructured to respond to incoming government priorities and to adjust the scope to the shorter implementation time frame. The number of cities was reduced to two, selected for their concentration of population. The revised design became more focused geographically and added more beneficiaries, which increased design relevance, but weaknesses in the results framework were not addressed sufficiently. The project’s completion report showed that the project indicators do not document progress adequately. 2.16 The project targeted poor regions based on a clear, transparent rationale for selection with verifiable methodology that this assessment identified. Targeting was a two-stage process. First, eight cities were selected that showed the highest growth in Bahia, and thus an increasing number of slums. Second, within each of these cities, vulnerability geoprocessed maps were prepared based on 2010 census data at the census track level, according to a methodology already used in São Paulo (appendix B, map B.1). vi 2.17 Urban regions were selected for comprehensive intervention rather that punctual areas. These target areas were divided into two types: social intervention areas (poligonal de intervenção social; PIS) and physical intervention areas (poligonal de intervenção física; PIF). PISs were meant to receive social outreach actions only whereas PIFs had to benefit from both social and physical interventions. Because physical interventions are more expensive, they had to be targeted strategically. By contrast, social activities could be more extensive, considering that local CBOs (to be supported) and social equipment (to be constructed) would support areas that are larger than PIFs, or that issues like job and income generation or security have broader comprehensiveness than PIFs. In addition, social actions benefitting both slums and the surrounding region would contribute to the integration of slums to the surrounding region (the city). COMPLEMENTARITY 2.18 Project design was based on the theory that the provision of integrated physical and social components is critical for the project’s sustainability. In this regard, the information from field interviews found that the delivery of these two components was managed separately and lacked synergy. Although the selection criteria were similar, the PIS’ areas of influence were much broader than the PIFs; they had different implementation modalities and beneficiary selection criteria, and thus often did not target the same beneficiaries geographically. The design also included relevant multisector activities to reduce urban poverty. It envisioned active coordination with World Bank investments in sectors such as education and health. However, no clear institutional arrangements or legal provisions were made to ensure synergies with education and health. Both the project completion report (Implementation Completion and Results Report [ICR], page 37) and information gathered from field interviews confirmed that the design included complex institutional and implementation arrangements, which affected implementation. 12 2.19 The relevance of the design is rated modest because of weaknesses in the project’s design. The ambitious overarching objective, the need for better complementarity between the assignment of physical and social components, the lack of clear institutional arrangements for multisectoral investments, and the weak results framework were only modestly relevant to reducing urban poverty on a large scale. 3. Implementation 3.1 The project’s implementation phase was severely disrupted from the outset. The country and the state of Bahia held elections in 2006 (soon after approval), and the opposition party won the election. In 2007, the newly elected government appointed a new director to the project management unit in the Bahia State Secretariat for Urban Development (Secretaria de Desenvolvimento Urbano; SEDUR), and this director had very limited experience and capacity in the implementation of internationally financed projects. The new management began a process of project realignment with political priorities. This process was disruptive and caused two years of delays in disbursement of the World Bank loan. The project started to move forward again (though slowly) only after the appointment of new directors of the two implementing units: the Project Management Unit (Unidade de Gestão do Projeto; UGP) from SEDUR, and the Project Technical Unit (Unidade Técnica do Projeto; UTP) from Bahia State Company for Urban Development (Companhia de Desenvolvimento Urbano da Bahia; CONDER). Given the new political landscape and the significant time delay, the need to adapt to changing priorities and redirect activities with a realistic time frame became evident. The project was extended for 36 months and underwent two level-two restructurings and one conditional restructuring. Technical assistance through the Association of Volunteers for International Service (Associazione Volontari per il Servizio Internazionale; AVSI) supported project implementation during the renegotiating period by contracting and allocating staff to UGP and UTP and covering the cost of the project headquarters’ rent. High staff turnover also marked the project implementation. The first task team leader was in the position from the design stage until the first three years of operation. However, after that, the project had six different task team leaders. Planned versus Actual Expenditure by Component 3.2 Project expenditures exceeded the appraisal estimates by 75 percent. As shown in table 3.1, the aggregated totals of actual or latest estimates of costs for both components exceeded the appraisal estimates. The reasons for cost increases included high inflation in the construction sector, higher housing construction standards, and high fluctuation of the Brazilian real against the U.S. dollar (29 percent of real value lost). Lengthy project implementation magnified these factors’ effects. From 2006 to 2014 (when most project works were completed), construction costs rose 63.71 percentage points; between 2006 and 2017 (when the last works in São Bartolomeu community were concluded), costs rose 81.54 percentage points. Postcompletion, the government contributed $56.3 million to deliver pending commitments. The social delivery component 2 expected 25 percent of total costs at project completion, but at the end, only 6 percent materialized. This was due to reallocation of resources to components 1 and 3 . 13 Table 3.1. Planned versus Actual Expenditure, by Component Appraisal Actual or Latest Estimate Estimate Actual as percent of Component ($, millions) ($, millions) Appraisal Estimate Component 1: Urban 51.40 60.66 118 infrastructure delivery Component 2: Social 19.45 8.01 41 services delivery Component 3: Institutional strengthening 7.75 12.20 157 and project management Additional costs n.a. 56.30 a n.a. postproject completion Totals 78.60 137.17 175 Source: Implementation Completion and Results Report; CONDER. a. This amount includes additional government funds delivered after project closure, mostly to finish resettlement activities. Safeguards Compliance 3.3 The project was classified as category A. It triggered three safeguards policies: environmental assessment, cultural property, and involuntary resettlement. The government of Bahia had previous experience with World Bank projects and established steps and procedures regarding environment and resettlement. It also had previous experience with cultural property through the restoration of the historic center, and that project triggered four safeguard policies. Consequently, issues to address these safeguard triggers were related to enhance existing implementing agency guidelines capacity. 3.4 The project’s environmental assessment safeguard policy was triggered because of the project’s potential impacts associated with provision of water supply and sanitation, urban drainage, street paving, solid waste collection and disposal, stabilization of slopes and embankments, parks and recreation areas, and other basic amenities and services at the community level. Existing environmental guidelines for construction activities and environmental site screening criteria were enhanced. For example, the Bahia State Environmental Agency required CONDER to form an environmental group to oversee related issues. The agency also trained members in environmental analysis. Certain areas triggered cultural property issues arising from the need to ensure that sacred sites were not disturbed in one of the macro-areas identified for intervention. More specifically, Ribeira Azul/Cobre in Salvador, including the Cobre river basin and São Bartolomeu Park, is an area of religious significance for African-Brazilian religions. 3.5 The need to relocate families occupying high-risk areas triggered involuntary resettlement safeguards, and the development of an enhanced resettlement plan addressed this. The guide for resettlement was the goal of moving households out of high-risk areas to new housing located at a maximum distance of 1 kilometer from the original homes to lessen the impact on the community and residents’ lives. Households were presented with three 14 options: (i) resettlement to newly constructed houses; (ii) cash compensation (or indemnification); or (iii) assisted purchase (monitored indemnification) of replacement housing. A resettlement action plan was developed in a highly consulted and participatory process. It was incorporated into the borrower’s overall resettlement procedures during implementation and successively institutionalized in the CONDER resettlement policy(table 3.2). The low number of cases in court suggests that the value of compensation is fair. The practice of monitored indemnification, an innovative practice at the time, was chosen in about 13 percent of the cases. Table 3.2. Resettlements Options as of March 2018 FEIRA DE MUNICIPIO SALVADOR SANTANA PIS COBRE PAU DA LIMA RIBEIRA MANGABEIRA São Encosta TOTAL São SusuaranaIII/ Mangueira Avenida Bartolomeu de Bartolomeu Baixa Paz III Anchieta PIF Park Pirajá Resettlement 256 215/c 10 121 132 358 1092 Cash Compensation 155 148 16 18 17 18 372 Assisted purchase of houses /a 120 111 2 0 0 0 233 Judicial 13 40 0 0 0 0 53 Total solutions /b 544 514 28 139 149 376 1750 Total households 503 500 28 132 140 320 1483 New Housing units 256 65/d 160 121 181* 358 1.141 Source: CONDER. a Monitored Indemnification b Some households had more that one family so the project has more solutions than households. c Ressetled in Encosta de Pirajá d Corresponds to units built in Mirante da Enseada for families from Parque São Bartolomeu, Alagados III outside of Mangueira III but in PIS Ribeira. 3.6 There were still 832 pending cases of resettlement at project closing. All of these pending cases and assisted-purchase housing were delivered by October 2014. Evidence of all six resettlement areas show that resettlement has been effective or indemnification finalized in 97 percent of the cases (table 3.3). 3.7 The project completion report included as a resettlement challenge the influx of new families into high-risk or unsanitary areas. To mitigate this impact, project interventions included design features to prevent the encroachment of new, unplanned settlements within the project areas. The field visits completed for this assessment confirmed that formerly occupied, at-risk areas are mostly kept unoccupied because of the construction of roads in the previously occupied area (PIF Achieta, PIF Sussuarana, Mangueira III) or the installation fenced areas in São Bartolomeu Park (see photos in appendix D). However, despite the fence in São Bartolomeu Park, there is still pressure for new informal settlements, and some 15 scattered cases occurred (the number is unknown). CONDER requested government officials to intervene and remove the new occupants. Table 3.3. Situation of Resettlements as of March 2018 São Encosta São SusuaranaIII/ Mangueira Avenida Bartolomeu de TOTAL Bartolomeu Baixa Paz III Anchieta PIF Park Pirajá Effectivea 531 477 28 139 149 376 1.701 In processb 0 0 0 0 0 0 0 Situation undefinedc 0 3 0 0 0 0 3 Judicial 13 34 0 0 0 0 47 TOTAL 544 514 28 139 149 376 1.750 Source: CONDER. a. Indemnification or resettled. b. Units to be resettled or indemnification in process of payment. c. Resettlement remitted to social rent and indemnity in payment process. Financial Management and Procurement 3.8 The procurement documents were found to be of good quality during project implementation. However, according to stakeholder interviews conducted by the Independent Evaluation Group’s assessment mission, the approached entailed excessive bureaucracy in dealing with project procurement matters. 3.9 Financial management was also found to be of good quality, and it had a more integrated approach. The project’s financial statements were submitted quarterly, and overall information was reported to be accurate. All loan proceeds were disbursed and documented properly by the closing date. During the last mission, the WB team followed the steps with the state government on their commitment to project implementation. These included guarantees that the flow of counterpart funds will not be blocked for the payment of remaining contracts. Counterpart funds were successfully delivered and remaining activities completed. The project completion report showed that audits were unqualified. 3.10 The project completion report notes that financial management and procurement were applied to project execution with different underlying modalities. Financial management had a more holistic, comprehensive approach that focuses on ex post remedies (funds ineligibility) as deemed necessary. Procurement has more of intensive approach, focused on step-by-step processing and contracting. 4. Achievement of the Objective 4.1 Development objective. The project was designed to reduce urban poverty in a sustainable manner in the poorest and most vulnerable sections of Salvador and certain other cities in the borrower’s territory by providing access to basic services and improved housing and social support services. Without an overarching indicator to validate or assess this main aim, this assessment uses the theory of change (paragraph 2.14) to assess the contribution of 16 subobjectives to the objective. The aim of institutional strengthening of state and local government capacity, though highly important for the sustainability of interventions, is not evaluated as a separate objective, but rather its results are imbedded into the rating of the infrastructure and social objectives. Objective 1: Access to basic services and improved housing 4.2 This assessment considers the access indicators that the project used, but does not limit its analysis of achievements to those indicators. The project defined access to housing improvements as the number of households resettled from areas of high risk or unhealthy conditions, and the number and percentage of households obtaining secure ownership. Field assessments and group interviews in targeted areas were used to assess service delivery quality. OUTPUTS 4.3 A significant amount of infrastructure works incomplete at project completion. Works in PIF Anchieta were the only ones concluded before project closure. The Bahia government agreed to complete all works in other PIFs at project closure, and these were executed successfully. As of October 2014, the project improved 2,138 households, surpassing the target of 2,000 households. Additionally, the number of households resettled from high-risk areas or unhealthy conditions increased to 1,448. Infrastructure works supporting access to paved roads and access to sanitation were completed as planned (table 4.1). 4.4 The project shows achievement in the number of people with secure ownership (in that they could benefit from land regularization) after project closure. Dwellers have tenure security (meaning that occupancy and rights to housing is secure), but many have not yet obtained full title, and this limits the selling potential. Full titles were issued only in PIS Ribeira and PIS Cobre. Titling is precarious in Sussuarana/Baixa da Paz and, as of April 2018, was absent in Avenida Anchieta. vii This situation might be partly because the overall strategy of SEDUR and CONDER for VMII (and other) interventions has been to provide titling to newly built houses and consolidated houses—that is, the remaining parts of benefited communities and surrounding areas. PIF Sussuarana/Baixa da Paz is located in an area near state government headquarters and has gone through a complicated expropriation process, meaning that newly built houses received a temporary title, but consolidated houses did not. In PIF Avenida Anchieta, titles are ready for the first group of houses that were planned, but not for those added at the later stages of project implementation. This generates dissatisfaction among resettled families in these PIFs. Additionally, if families intend to sell their housing units, the price is usually below the market price and even below construction costs because of its informal status. 17 Table 4.1. Access to Basic Services and Improved Housing After Completion Completion Baseline (October 2013) Target (October 2014) Number of households with house improvementsa 0 1,009 2,000 2,138 Number of households resettled from high- risk areas or unhealthy conditions 0 1,332 983 1,448 Percentage of households resettled from high-risk areas or unhealthy conditions 0 81% 60% 95% Number of households with secure ownership of their home (able to benefit with land regularization) 0 419 2,500 3,100 Increase in the percentage of households with access to sanitation networks 74.8% 20% +35% 90% Increase in the percentage of households with 250 meter of paved roads 92.5% 100% 100% 100% Number of households benefitted with urban infrastructure. 0 1,093 6,332 7,329 Source: Implementation Completion and Results Report; CONDER as of October 2014. a. House improvements referred to households that received new housing units, improved housing, and monitored indemnification. OUTCOMES 4.5 The quality of housing improved dramatically compared with the poor housing conditions in which most resettled families lived. This was notable, especially in stilt houses in Mangueira III and semirural dwellings, and in shacks within São Bartolomeu Park (see photos in appendix D). General habitability also improved through the following: (i) the elimination of risks, especially floods (São Bartolomeu community, Sussuarana/Baixa da Paz, Anchieta), high tide or fire (Mangueira III), and exposure to pollution sources from open sewage (Sussuarana/Baixa da Paz); (ii) the provision of basic services by providing water and sanitation (carrying water in buckets was common in Mangueira III), lighting and basic services, easing accessibility (paved roads), and garbage collection. Infrastructure works supported the construction of paved roads, and sanitation networks has resulted in an increase of access to those services. Access to other basic services did not change because families were resettled where they already lived. Transportation is considered good because of the close location of the settlements in three target areas: São Bartolomeu community, Mangueira III, and Anchieta (though in Anchieta, walking access to the city is difficult because there are no crossing paths between the Ayrton Senna avenue and the new houses). However, it is unsatisfactory in two areas located in the bottom of the valley (Encostas de Pirajá, which is situated on a slope, and Sussuarana/Baixa da Paz), where residents have to use a staircase to the reach the avenue above them. 4.6 Retaining beneficiaries in the same territory avoided rupturing the social fabric in local communities and allowed those communities to keep using familiar local services. Housing interventions are integrated well to the remaining areas, both through contiguity and 18 through investments in equipment, infrastructure (pavement, drainage, and roads), and house improvements (São Bartolomeu community), thus benefitting both old and newly built areas. 4.7 There was consensus among beneficiary group participants that listening to the community was crucial for ensuring the quality of houses. There were changes in the design and size of houses according to the needs the communities expressed. For example, houses in Mangueira III, Avenida Anchieta, and part of São Bartolomeu community allow expansion through self-construction, which residents appreciated because this allows them to accommodate family members more comfortably, grow their families, or rent rooms to complement income. 4.8 The project promoted a systemic approach that was successful in bringing multidimensional neighborhoods together, but it had limitations because of the need to prioritize interventions and the project interventions’ lack of adherence to the master plan, which included a comprehensive and multidimensional (urban, social, and environmental) requalification of neighborhoods based on a broad menu of housing solutions (versus unidimensional solutions such as in MCMV). The focus of interventions was neighborhood improvement rather than housing, infrastructure, landscape, or environmental conservation. Target areas received interventions that integrated the urbanistic, environmental, and landscape dimensions. The most relevant interventions in this respect are those in the São Bartolomeu Park region, which make up half of all interventions. They integrated environmental, housing, infrastructure, social, and cultural facilities as championed by the former Secretary of Urban Development, which led to revitalization of the park and now represents an asset for both the local community and the city as a whole. However, different factors limited such a systemic approach. One factor was the size of slums in Bahia. Prioritization was obviously necessary, but dissatisfaction is manifest in Sussuarana/Baixa da Paz, where other highly at-risk areas were not supported. A second factor was the project interventions’ lack of adherence to the city master plan. Without a reference framework, project interventions lost urbanistic coherence and ended up being negotiated among different political groups within the government (between CONDER and SEDUR, for example). viii 4.9 The targeted areas and the city as a whole benefit from the only community park in the area with high-quality infrastructure, but use and full access to the park is still limited because of lack of security. The facilities of São Bartolomeu Park host a variety of activities, and different partners manage events. In Pirajá, the Center of Citizenship and Culture and the Bahia State Secretariat of Justice, Human Rights, and Social Development run the Vida Melhor program for microentrepreneurs (the Neojibá symphonic orchestra social program for underprivileged children) and a culture program that encourages reading and using a library. In addition, facilities in the park’s two entrances are open to events and cultural activities, such as courses in computing, dance, and sport activities. Persistent crime and violence limit full fruition of and access through the park’s internal area. Although this is said to have decreased, local residents avoid crossing through it, even though this would be the shorter path (from Pirajá to Suburbana) where transport is concentrated. 4.10 Three kinds of common problems emerged among the resettled dwellers who were interviewed. The first concerns electricity or water bills for common areas (staircases, 19 gardens, and the like), which are addressed to individual residents of each block because these housing developments are not registered as a condominium, and they do not have an association representing residents. This situation generates tension among residents who are suspicious of one individual handling money from neighbors. An exception is Sussuarana/Baixa da Paz (discussed in paragraph 4.22). Another problem is the rising costs of electricity and water, which resettled families did not pay previously because most had illegal connections. These costs in some cases have a highly negative impact on family budgets and are one of the reasons that dwellers sell or rent rooms. However, stakeholder interviews revealed that most of the resettled households pay the lowest social tariffs in the city. The third problem, which is present in all PIFs, is the municipality’s delay in updating the housing development address, resulting in postal codes not assigned and undelivered mail. 4.11 The project showed successful achievement of all outputs after project closure, which mostly led to the achievement of improved access to basic services and improved housing. Therefore, objective 1 is rated substantial. Objective 2: Improved access to social programs 4.12 The program’s success hinges on the ability of local community-based organizations, local government, and nongovernmental organizations (NGOs) to provide the necessary local social services so that communities can enhance income and employment generation, sport and cultural education, and crime and violence prevention activities. OUTPUTS 4.13 According to information in the ICR and information gathered during the mission, all expected trainings and capacity building were delivered and surpassed output targets. However, this assessment found that several activities were incomplete, even though they were delivered. 4.14 Two public bids to finance CBO subprojects were launched to enhance their organizational strength. All benefited and eligible CBOs projects were expected to benefit from technical assistance and training and included physical improvements and purchase of equipment. However, physical improvements were implemented in only one CBO, and equipment was not delivered at all. 4.15 The project delivered 1,887 income-generation activities. These included vocational training provided by Serviço Nacional de Aprendizagem Industrial (National Service for Industrial Apprenticeship) and Serviço Nacional de Aprendizagem Comercial (National Service for Commercial Apprenticeship), and a young apprenticeship and economic viability study for microentrepreneurs. Registration was easy, and beneficiaries received free transportation and food during the two or three months of the training. Courses provided basic activities from mason, painter, domestic service, computing, and hairdresser programs. Economic viability studies for microentrepreneurs is an approach that the Catholic University of Salvador developed based on agents selected from local communities and supported by contracted NGOs that interact with local, informal microentrepreneurs. This is done in a 20 pedagogic manner and resulted in 8- to 12-week studies with recommendations for improving the microentrepreneurs’ economic viability. PIS Ribeira had the highest number of participants in job and income-generation activities (table 4.2). This is related to both its inherent characteristics (earlier occupation and urbanization, old industrial area where unions were active, and so on) and the long-standing presence of CONDER and AVSI and their projects in the Ribeira area (30 years). Social capital strengthening and social development in general are lengthy processes that need continuous support. By contrast, residents of PIF Anchieta, where local social capital is weak, state that the local association ceased functioning after CONDER concluded social work. The municipality has little presence to strengthen social capital in the area. Therefore, the residents felt abandoned. Table 4.2. Job and Income-Generation Activities by Social Intervention Area PIS PIS PIS Pau da Activities Cobre Ribeira Lima PIS Mangabeira Total SENAI vocational training 62 92 33 31 218 SENAC vocational training 180 228 163 162 733 Young apprenticeship 106 105 6 0 217 Economic viability study for microentrepreneurs 183 177 186 173 719 Total 531 602 388 366 1,887 Source: CONDER. Note: PIS = poligonal de intervenção social (social intervention area); SENAC = Serviço Nacional de Aprendizagem Comercial (National Service for Commercial Apprenticeship); SENAI = Serviço Nacional de Aprendizagem Industrial (National Service for Industrial Apprenticeship). OUTCOMES 4.16 Although outputs were delivered, integration of physical and social actions was somewhat limited. Only 41 percent of beneficiaries of social actions were selected from PIFs because works were under way, and beneficiaries were usually scattered in social rent accommodations (temporary housing where the housing recipients were moved while houses were built). Targeting was more effective for informal microentrepreneurs because they could be easily localized in their workplace. Moreover, different teams that hardly coordinated executed the physical and social activities. . In addition, despite the envisioned high value of the social component at project design, these funds were not available because of a delay in the conclusion of works and resettlement. This means that resettled families benefited from social work in the same way as in all slum-upgrading interventions (without the additional funds provided by VMII), and physical and social actions were not fully integrated. Integration of projects had to materialize into intersectoral actions, but this did not happen. There was no project link with the health sector, and VMII social actions did not have continuity after project closure because they were not replaced with routine, local social policies and programs. 4.17 Strengthening of civil society organizations and social capital has been central to the approach. However, information on actual results and impact is scattered and anecdotal. There was comprehensive capacity building and growth in the professional management of CBOs, but the number of civil society organizations for which this materialized into concrete 21 results (for example, new funding and new projects or better services to the population) is unclear. Interviews with two civil society organizations attribute some positive institutional strengthening results. For example, CBOs like the Center for Art and Environment (Centro de Arte e Meio Ambiente; CAMA) routinely take part in public bids, and the Union for popular housing (União pela Moradia Popular) gained professionalism in its work of self- managed housing construction. However, according to CAMA’s final report on technical assistance to CBOs, only 10 out of 24 benefited solidarity economy CBOs have established a stable production process, and the others depend on fluctuating demand (CAMA 2013). The lack of access to credit and equipment is mentioned as the main barrier to deliver services sustainably. The failure in equipment delivery and facilities renovation is also a factor that limited the achievement of concrete results. The Beneficient and Democratic Association of Alagados of Itapagipe, the only CBO whose headquarters were restructured, states, “This changed the way the community looked at the association; its headquarters became a reference, something to rely on, within the community.” 4.18 Social operators are civil society organizations that were contracted under the project as brokers between government and civil society. Evidence from field interviews with social operators and NGOs suggest that both state and civil society consider their experiences to be positive. It was a new approach that had to be constructed collectively to help strengthen the relationship between social operators and civil society organizations. Social operators allowed civil society to be heard effectively. Participatory local development plans were produced under the supervision of all PIS social operators and presented to different government sectors. However, their actions were not implemented because of a lack of intersectoral arrangement and coordination. 4.19 Vocational training hardly materialized into tangible results for beneficiaries because the state and municipal agencies that deal with income and job generation (including through workforce intermediation) did not participate in these activities. SEDUR alone took responsibility for them, even though this was clearly outside its institutional mission and competence. Although focus group beneficiaries of vocational training expressed overall satisfaction with the quality of the training, they rarely found a job afterward. There was consensus that further workforce intermediation mechanisms would be necessary to ease access to the labor market. In fact, in 2011 and 2012, the Young Apprenticeship Program (which includes a stage period), reported that all 217 participants were employed in 43 companies, primarily in civil construction. ix 4.20 Economic viability support to microentrepreneurs helped them in the initial stage, but hardly produced concrete results. Early viability studies and capacity building were conducted successfully, but later stages of the process had a much lower performance, partly because of administrative discontinuity of the activity. The later stages depend on the active involvement of other government sectors and consist of concrete support to microentrepreneurs through microcredit, training, provision of productive equipment, and so on. Nevertheless, the pilot under VMII evolved into the state program Vida Melhor and supported approximately 12,000 families of microentrepreneurs (di Villarosa and Curraleiro 2016). 22 4.21 Investment in equipment for social activities has been extensive and integrated into upgrading, with high-quality facilities in place, though underused in the São Bartolomeu Park area. Moreover, the facilities of São Bartolomeu Park host a variety of activities and events that different partners manage. x SEDUR contracted a consultant (a specialist in urban parks) to design the management arrangement of São Bartolomeu Park, in which park administration was to be shared among the state government, the municipal government, and organizations from civil society. The operation and maintenance was not implemented because of a lack of political alignment between the state and the municipality. This resulted in lack of ownership of the facilities by other state departments, the municipality, and CBOs. People must be stimulated to use the park equipment, which includes 16 small shops for petty trade that were built in the Praça de Eventos, but most are unused. By contrast, the child education center Heroínas do Lar relocated and built by the project has had positive results. Demand for vacancies in its classes highly exceeds supply, and the quality of teaching is supported pedagogically by a respected school in Goiânia. 4.22 According to CONDER rules, management of the local equipment in community centers is the responsibility of local residents’ associations. As in most cases, this arrangement was either not created, or was created but is not working. At the time of this assessment, the community center in Mangueira III has been vandalized and is unusable, whereas in Sussuarana/Baixa da Paz—where the residents’ association is being created and the general residents association of the Pau da Lima neighborhood management oversees the management—the community center is clean, intact, and used for several activities, including martial arts and education complementary to school. 4.23 The situation of violence is aggravating in most Brazilian cities and specifically in poor urban settlements, and project PIS and PIF are no exception. An increase in crime and violence was reported during the interviews with beneficiaries in all of these areas. Crossing the São Bartolomeu Park between Encosta de Pirajá and São Bartolomeu Community is considered too dangerous, and the people interviewed feel safe only when walking to the Oxum fall close to the park gateway. Residents of São Bartolomeu Community are scared of the police, but they seem to coexist with drug dealers, who are community members. Violence also increased in Mangueira III and seems to isolate families from one another, as in Anchieta. The only partial exception is Sussuarana/Baixa da Paz, where violence was reduced to some extent. Previously, the slum was used as a path for armed gangs to move across because of a lack of public lighting and overall precariousness. Urbanization eliminated such ostensive forms of violence. However, the whole area and its surroundings, which includes a detention center, are dangerous because a transient population of people related to the prisoners lives near the center. 4.24 Beneficiary interviews and evidence gathered for this assessment showed limited evidence that the project has improved access to social services. As a result, objective 2 is rated modest. 4.25 This assessment finds that the project contributed significantly to reducing urban poverty in a sustainable manner by providing access to basic services and improved housing, but only modestly to improving access to social programs. Overall efficacy is rated modest. 23 OTHER OUTCOMES 4.26 The project assessment found that new tools that were created in VMII were later internalized at CONDER. The adoption of local offices for integrated actions, instead of offices for follow-up works only, has been proposed to be institutionalized in all CONDER projects. In addition, questionnaires for social work diagnostics and procedures and juridical devices for monitored indemnification in resettlement that the project developed were institutionalized at CONDER. At project start, there was no state policy for involuntary resettlement in Bahia, and CONDER senior managers did not want to address resettlement in VMII differently from other slum upgrading or housing projects there. However, the World Bank persuaded CONDER to follow WB guidelines for involuntary ressetelment in Bahia. Conder became responsible for solving the juridical problems involved in monitored indemnification. The World Bank and VMII influenced resettlement policy in Bahia. 4.27 VMII programs were used to inform slum-upgrading policy at the federal level. When project implementation began, project technical assistance through the Cities Alliance and AVSI shifted partly to systematization of the experience and methodology transfer to the Ministry of Cities. A new Ordinance on Social Work in Slum Upgrading and Social Housing was issued by the Housing Secretariat of the Ministry of Cities in January 2014 (Ordinance 21/2014). 4.28 VMII influenced other programs. From 2009 to 2015, the Cities Alliance supported a tripartite cooperation among the Brazilian Cooperation Agency, the Italian Development Cooperation, and the municipality of Maputo for a slum-upgrading pilot program in the neighborhood of Chamanculo C, inspired largely by the methodology developed in Bahia and with the participation of AVSI in its execution. The Chamanculo C pilot program was meant to provide inputs to a global strategy of informal settlements upgrading in the Municipality of Maputo, supported by the World Bank. This strategy was materialized by the ProMaputo Municipal Developent Program xi, in which AVSI was contracted to replicate the diagnostics and integrate planning tools and the civil society strengthening strategies of VMII. 5. Efficiency 5.1 Economic rate of return (ERR). A cost-benefit analysis was used to estimate the ERR both at appraisal and at project closure. A hedonic prices approach was used at appraisal, but because of a lack of information regarding increase in prices on the properties that the project caused, the same approach was not used at closure. xii Instead, avoided-cost, travel time, and willingness-to-pay approaches were applied based on gains obtained with the project. To avoid exchange rate fluctuations and inflation rate, actual costs and benefits were transformed to 2005 prices. This is very important because the Brazilian real appreciated during the period, and construction prices also increased. Results of the analysis of the loan proceeds show that the real cost of the works was 46 percent lower than what was expected. In reality, the overall impact on costs at project completion was absorbed by the counterpart funding, which increased by 36 percent of total project cost. At the time of this assessment, the counterpart funding increased significantly to 64 percent of total project costs mainly to finalize housing and resettlement activities pending at project closure. 24 5.2 Most of the expected benefits included in the assessment have materialized, but there are some that have not been considered. The project estimated an overall ERR of 16 percent, which is less than appraisal, but higher than the benchmark of 12 percent. Benefits materialized include the reduction of losses due to flooding, and improvements in access to the childcare facility and São Bartolomeu Park, which has provided environmental and social benefits (though offset somewhat by the facilities’ lower-than-expected use in number of expected visitors and the use of meeting rooms and auditoriums). The housing benefits have also materialized concerning risk reduction by moving from shacks and stilt houses to proper housing. This assessment found that other possible benefits have not been considered. For example, many of the units that allowed for expansion have already made changes to increase the living area, which presents additional benefits for the resettled families that have not been included in the ERR estimation. Also, the project did not measure the effectiveness of social services, particularly income-generation activities, including vocational training, youth apprenticeship, and capacity building for entrepreneurs. Therefore, a cost-benefit analysis of the $6.2 million spent in social delivery services is unobtainable. 5.3 A possible matter of concern is that at the time of this assessment, the units were being sold at about 25 percent lower than costs. This cost, as noted in the ICR, was already higher on average than international benchmarks ($11,482 versus $3,000). The housing price is usually below the market price and even below construction costs because of its informal status. 5.4 There were institutional and administrative inefficiencies. The project underwent two restructurings, and its completion was delayed for two years and nine months. Moreover, the government finalized and funded resettlement activities a year after project closure. Causes for the delays are mostly attributed to the shifting political landscape and staff turnover. The political landscape negatively affected the implementing agencies’ strategic focus. This created a need to adapt the project scope and time frame to the new government’s priorities. High staff turnover during the last four years affected institutional memory and contributed to further delays. The project had two implementation units, which increased the interinstitutional transaction costs throughout the implementation period. 5.5 This PPAR assigns a modest efficiency rating because of the significant implementation delays caused by institutional and administrative inefficiencies, the lack of inclusion of possible benefits, and the lower percentage of total project costs that the ERR covered (36 percent). 6. Ratings Outcome 6.1 The overall outcome of the project is rated moderately unsatisfactory. The project development objective of reducing poverty in the poorest and most vulnerable areas was and still is substantially relevant. Project design is rated modest because of its ambitious overarching objective, the need for better complementarity between the assignment of physical and social interventions, the lack of clear institutional arrangements for multisectoral investments, and the weak results framework. Although poverty has declined 25 significantly in general in the state of Bahia, there is no evidence that results should be attributable to project results. The provision of access to basic services and housing has improved in the targeted areas, but evidence gathered through this assessment suggests that materialization of outcomes of social service delivery is limited. Efficiency is modest, mostly because of the institutional and administrative inefficiencies. Risk to Development Outcome 6.2 Operation and maintenance risks. There is significant risk that several community centers do not have adequate operation and maintenance arrangements in place. São Bartolomeu Park is potentially the most problematic for its significant cost—park administration was to be shared among the state government, the municipal government, and civil society organizations. However, because of a lack of political alignment between the state and the municipality, it was not implemented. CONDER has been managing the park since the start of operations with a small budget that is unsustainable. CONDER, the State Secretariat of Environment, and SEDUR established a group to develop a shared operation and maintenance management model in which part of the costs would be shared with other actors who use the park equipment (for example, universities, civil society, and other governmental institutions), but such a model is not yet in place. 6.3 Social inclusion risks. Land titling is considered a key element of social inclusion. The project shows different stages of provision of titling. Although the process has been long and tedious, there is a modest risk that full landownership will not be achieved. Titles have been given to two of the four targeted areas, but full and free titling in the other two areas are still an issue. 6.4 Political risk. Since 2016, a shift in the national political agenda and a significant reduction in domestic resources have contributed to a slowdown in slum upgrading and social housing program implementation countrywide. Fiscal contingency has also limited access to internationally financed projects. This scenario could negatively affect the sustainability of the infrastructure built under the project and the vision of integrated urban upgrading in Brazil. 6.5 The risk to development outcome is rated substantial. Bank Performance QUALITY AT ENTRY 6.6 Quality at entry is rated unsatisfactory because of the project’s challenges of an overambitious project development objective and an inadequate preparation period that complicated interinstitutional arrangements. The likelihood of achieving the overarching objective was too low from the start and was even lower after reducing the project’s scope (but not the objective). More than four years lapsed between the initial concept note and project approval, mainly because the activities’ were complex and multidimensional. The preparation efforts were also limited as stated on page 6 in the ICR. They did not emphasize the development of technical inputs that would have facilitated implementation. The lack of 26 these inputs contributed partially to the underestimation of project costs and to the delay in starting resettlement negotiations. To some extent, the project design underestimated the complexity and uncertainties that inherent to a highly participatory slum-upgrading project, which must allow time to correct unexpected problems. 6.7 The project’s initial results framework was designed to monitor and report progress against the higher-level objective of reducing urban poverty in targeted areas, which was designed to be measured through an impact evaluation that never materialized. QUALITY OF SUPERVISION 6.8 Project supervision is rated moderately satisfactory because of its implementation delays, which suffered from a slow start and significant turnover in task team leadership, particularly during the later years of implementation. However, the Brasilia-based, co-task manager, who stayed in the position for most of the period from 2010 onwards, partially mitigated this turnover. According to implementing agencies, the World Bank’s role was critical in helping the project move forward after it stalled for two years. In particular, the World Bank focused its efforts on helping Bahia counterparts rebuild the implementation capacity that was impaired during 2007–08. There were regular and extensive supervision missions to solve procurement issues. The World Bank carried out regular supervision on safeguards. 6.9 As stated on page 6 in the ICR, the World Bank team conducted two restructurings and extended the project closing date, but the team missed the opportunity to restructure the ambitiously defined objective and introduce a realistic M&E design to document progress. 6.10 Overall bank performance is rated moderately unsatisfactory. Borrower Performance GOVERNMENT PERFORMANCE 6.11 The state of Bahia was the borrower. Ownership of the project was high during the preparation phase. The change in state leadership after the gubernatorial elections in October 2006 and a change in all municipal department heads caused the project to stall for two years until late 2008. High staff turnover disrupted the continuity of implementation, and the new government did not consider the project a priority. The state government was late in providing counterpart funding which slowed project implementation and required an extension. Payments were delayed. As a result, the project had to be extended three times, and activities were still ongoing at project closure. According to interviews with Bahia state officials, offices were well equipped and had sufficient human resources available. However, they had to rebuild internal capacity because department heads changed. Ownership improved after the restructuring and postproject closure as evidenced by full satisfaction with the postclosure agreements and later by provision of the necessary counterpart funding to finish pending works. 6.12 Government performance is rated moderately unsatisfactory. 27 IMPLEMENTING AGENCY PERFORMANCE 6.13 There were two implementation agencies for this project. SEDUR, created in 2003, was responsible for planning, implementation, supervision, and M&E. CONDER, created in 2002, had the lead role in project implementation and reports to SEDUR. There were interinstitutional transaction costs throughout the implementation period because of this dual arrangement. In addition, elections were held four months after project approval (October 2006). The newly elected government decided to renegotiate the project to align it to its priorities. The government appointed a Project Management Unit director with limited experience and capacity in implementation of internationally financed projects, which generated a clash between the political perspective of SEDUR/UGP and the technical approach of CONDER/UTP. According to interviewees, the relationship between SEDUR and CONDER was difficult until new managers and staff were appointed at UTP and UGP. Coordination was weak until 2011. It took some time to make the relationship between SEDUR and CONDER work smoothly. The Project Management Unit developed new resettlement practices based on project experience, which have been adopted as standard practices throughout the state (monitored indemnification). These practices positively influenced resettlement and beneficiaries’ effective use of cash benefits. Resettlement, environmental monitoring, and financial management procedures were high quality. 6.14 Implementing agency performance is rated moderately satisfactory. 6.15 Together, these ratings lead to an overall borrower performance rating of moderately unsatisfactory. MONITORING AND EVALUATION 6.16 Design. The M&E design included a comprehensive plan to track progress and included impact evaluations to be conducted by the World Bank’s development impact evaluation unit. 6.17 Implementation. CONDER was to use its management information system to conduct household surveys to produce quarterly reports. The M&E design was modified, restating outcomes and intermediate outcome indicators. However, the revised design remained limited and did not capture the possible project achievements accurately. Baseline and target values were not updated at restructuring and were not consistent with reality. Several indicators included outputs for social access with no attempt to measure quality or other outcomes. The design did not capture the depth of some social activities, including those related to resettlement. The expected impact evaluation was canceled. 6.18 Utilization. The borrower produced satisfactory quarterly reports based on reliable sources. However, the use of data is limited because indicators do not measure the relevant activities. The impact evaluation that was envisioned at project design was canceled, and M&E activities were limited to basically periodic reports of activities and results framework update. Alternative data collections, such as subsample analysis, had to be pursued to address these gaps. 28 6.19 M&E is rated modest. 7. Lessons Adaptability to local needs, retaining beneficiaries in the territory, and integration between different types of interventions are key features in slum upgrading and social housing. VMII provided a menu of comprehensive and multidimensional interventions (a housing system versus the monodimensionality of MCMV) that can be extended and replicated. Highly participatory processes allowed changes in design and size according to the needs the communities expressed. Retaining beneficiaries in the same territory avoided rupturing the social fabric in local communities and allowed those communities to keep using familiar local services. Housing interventions are integrated well to the remaining areas, both through contiguity and through investments in equipment, infrastructure, and house improvements, thus benefiting both old and newly built areas. Weakness in social service delivery can jeopardize expected outcomes and sustainability. Flexible designs and adequate sequencing of activities might help ensure results. Looseness and low sustainability are common features of social interventions, in which capacity building is difficult to materialize into concrete outcomes, even when resources for social activities are abundant (as in VMII). Lessons learned from VMII can include the following: (i) flexibility at project design and successive prioritization (from participatory diagnostics) to avoid establishing too many fronts at the outset (with gradual enlargement based on results and growing institutional capacity); and (ii) better sequencing of activities, such as the inclusion of slum-upgrading beneficiaries in local social policies when slum-upgrading social work is completed, which implies supporting municipalities to build up capacity to attend them. It is crucial in slum upgrading projects to guarantee delivery of all outputs in the social activities cycle because even small failures can jeopardize the expected outcomes. Social activities such as CBO strengthening suffered from lack of delivery of productive equipment or lack of physical improvement. Additionally, the income-generation activities lacked workforce intermediation after vocational training. Consequently, stricter supervision on delivery of social activities outputs becomes critical if low institutional capacity or insufficient resources to deliver all social outputs are identified during implementation. Adoption of multiphase projects or a programmatic long-term approach in slum- upgrading projects might be appropriate when social capital strengthening is considered necessary to achieve long-term results. Social interventions in one target area, social intervention area (poligonal de intervenção social) Ribeira, outperformed all others, specifically regarding CBO participation in project activities. This is mainly the result of a long-standing presence of state and international institutions—for example, the Bahia State Company for Urban Development (Companhia de Desenvolvimento Urbano da Bahia) and the Association of Volunteers for International Service (Associazione Volontari per il Servizio Internazionale)—through different, sequential projects in this area, which built trust and partnership with local civil society to strengthen its organizations effectively. Stronger social capital also fosters continuity and sustainability (for example, service provision or 29 dialogue with government), which are critical especially in situations of shifting political agendas and lower availability of resources. Appropriate timing in the preparation of slum-upgrading projects is crucial. Studies and diagnostics require sufficient time (though not too long) to avoid frustrating local expectations or hampering integration. VMII preparation took two years, and most works were concluded eight years after project start. Although slum-upgrading interventions are complex and imply a high degree of uncertainty (including in resettlement), a 10-year project completion period is too long because diagnostics become outdated, local expectations are frustrated, project credibility is lost, and a high number of potential beneficiaries vanishes, especially when they are temporarily allocated to social rent arrangements. One consequence of this long project implementation in VMII was the difficulty in attracting housing beneficiaries to take part in social activities, thus jeopardizing the integration of physical and social interventions. Project experience shows that project preparation might develop a clear, realistic indication of target areas, which should not be the subject of political negotiation later. Participatory planning could then be conducted in these areas with beneficiaries to gain their support and consensus, including for resettlement. Physical and social activities should be planned and initiated jointly so that they can support the same beneficiaries and make integration effective. Continuity in project management is a success factor in complex institutional development and social capital–strengthening projects, allowing familiarity with the local context and building mutual trust with the affected communities. This is partly related to the need for long-standing support and a programmatic, multiphase approach in institutional development and social capital strengthening, as discussed previously. In these kinds of projects, both familiarity with the local context and mutual trust with local stakeholders are important and might take some time to build. Frequent turnover in project management implies the need to rebuild this capital with every change of task team leader. Therefore, it is recommended to limit these changes as much as possible or, if they cannot be avoided, include project management transition periods during which the outgoing and incoming task team leaders work together so that the former can support and legitimize the latter for local stakeholders. The Brasilia-based co-task manager, who stayed in the position for most of the period from 2010 onward, partially mitigated the project’s high turnover. It is important for World Bank project management to ensure that slum-upgrading projects in urban areas are consistent with existing city plans and are integrated seamlessly into World Bank Group sectoral and thematic operations. The design of this project included relevant multisector activities to reduce urban poverty. It envisioned active coordination with World Bank investments in sectors such as education and health. However, no clear institutional arrangements or legal provisions were made to ensure synergies between education and health, which affected the achievement of project components. Although the project followed a series of urban-upgrading projects and promoted a systemic approach that was successful in bringing multidimensional neighborhoods together, it had limitations caused by the project interventions’ lack of adherence to the relevant municipality’s master plan. 30 References CAMA (Centro de Arte e Meio Ambiente). 2013. “Relatório Final—Assistência Extensiva. Relatório de Atividades do Projeto—Fortalecimento Institucional dos Empreendimentos Econômicos Solidários e Organizações Sociais.” CAMA, Salvador-Bahia. Cities Alliance. 2008. Alagados: The Story of Integrated Slum Upgrading in Salvador (Bahia), Brazil. Cities Alliance, Washington, DC. Cities Alliance and Ministry of Cities of Brazil. 2009. “Distance Learning Course on Integrated Slum Upgrading.” Cities Alliance and Ministry of Cities of Brazil, São Paulo-Brasília. Cortes, Lícia Quadros. 2012. “Relatório Final da Consultoria de Qualificação Socioprofissional.” Salvador- Bahia. di Villarosa, Notarbartolo Francesco. 2016. Impact of the Cities Alliance in Brazil: A Critical Analysis. Cities Alliance. di Villarosa, Francesco Notarbartolo, and Cláudia Baddini Curraleiro. 2016. How to Strengthen Urban Microentrepreneurs: Bahia´s “Vida Melhor” Program. WWP (World without Poverty) Series Study Case. WWP, Brasília. Imparato, Ivo, and Jeff Ruster. 2003. Slum Upgrading and Participation: Lessons from Latin America. World Bank, Washington, DC. Magalhães, Fernanda, and Francesco di Villarosa, eds. 2012. Slum Upgrading:Lessons Learned from Brazil. Inter-American Development Bank, Washington, DC. Ministério das Cidades. 2014. Portaria 21 de 22 de Janeiro 2014 que Aprova o Manual de Instruções do Trabalho Social nos Programas e Ações do Ministério das Cidades.Ministério das Cidades, Brasília- DF. SEDUR (Secretaria de Desenvolvimento Urbano). 2015. Relatório Síntese do PLANEHAB 2010-2013, SEDUR, Salvador-Bahia. World Bank. 2005. “Poor Urban Areas Integrated Development Project-Viver Melhor II.” Project Appraisal Document. World Bank, Washington, DC. ————. 2010. “Bahia Poor Urban Areas Integrated Development Project-Viver Melhor II”. Project Restructuring 58588-BR. World Bank, Washington DC. ————. 2014. “Bahia Poor Urban Areas Integrated Development Project-Viver Melhor II.” Implementation Completion Report ICR2963. World Bank, Washington DC. ii The objective in the project appraisal document (page 4) has slightly different wording from that in the loan agreement, but the meaning is the same. iii Quantity deficit refers to demand for new houses and people that spend more than 30 percent of family income for renting. iv Qualitative deficit is related to houses that are below decent standards (e.g.. low structural quality, no access to water and sanitation). v Cities Alliance, Distance Learning Course on Integrated Slum Upgrading, Cities Alliance-Ministry of Cities of Brazil, São Paulo-Brasília 2009. Cities Alliance, Alagados - The Story of Integrated Slum Upgrading in Salvador (Bahia), Brazil, 2008. vi Vulnerability is determined by the following criteria: (i) cities of the state’s basic urban network (40 cities representing 58 percent of Bahia’s urban population); (ii) municipalities (from the previous list) concentrating 80 percent of the state’s economic activity (23 municipalities representing 53 percent of Bahia’s urban population); (iii) those with population greater than 80,000; and (iv) those located along one of the state’s main 31 transport routes. The result was 13 municipalities which concentrate 44 percent of Bahia’s urban population. In all of these cities, studies undertaken by the borrower using year 2000 census data and geographic information system applications led to the identification and preparation of socioeconomic profiles of poor and vulnerable sections, according to an adaptation of the Map of Social Exclusion methodology developed in São Paulo. Adjoining areas thus identified have been grouped together in macro-areas for intervention. vii A precarious title is a juridical device that guarantees an individual the use of the land and rights to housing, but the individual is not the owner. Precarious title is usually an intermediary step toward full ownership. viii Urbanistic coherence means overall functionality of interventions (from the sum of synergies between them). ix The stage period, according to the Young Apprenticeship Law, lasts between six and 24 months with six hours daily working, and including both theorical and practical training. x In Pirajá Center of Citizenship and Culture, the Bahia State Secretariat of Justice, Human Rights, and Social Development runs the Vida Melhor program for microentrepreneurs and the Neojibá symphonic orchestra social program made up of children from underprivileged communities. The Bahia State Secretariat of Culture runs a program that encourages reading and using a library (40 children are attending). The State Secretariat of Technology offers a computing course. In addition, facilities in the two entrances of the park are open to community and church events, school parties. xi http://projects.worldbank.org/P096332/promaputo-maputo-municipal-development-program?lang=en xii Hedonic pricing is a model that identifies price factors according to the premise that price is determined both by internal characteristics of the good being sold and by external factors affecting it. 32 Appendix A. Basic Data Sheet BAHIA POOR URBAN AREAS INTEGRATED DEVELOPMENT (P081436) Table A.1. Key Project Data ($, millions) Appraisal Actual or Actual as Percentage of Category Estimate Current Estimate Appraisal Estimate Total project costs 82.20 100.85 122.68 Loan amount 49.30 49.30 100.00 Cofinancing 32.9 51.55 57% Cancellation Table A.2. Cumulative Estimated and Actual Disbursements Category FY06 FY07 FY08 FY09 FY10 FY11 FY12 FY13 Appraisal estimate 6.68 23.56 43.58 49.29 49.29 49.29 49.29 49.29 ($, millions) Actual ($, millions) 3.95 6.63 6.63 10.33 18.83 30.93 48.63 49.29 Actual as percent of 59.13 28.14 15.21 20.95 38.20 62.75 98.66 100.00 appraisal Date of final disbursement: 07/2013 Table A.3. Project Dates Category Original Actual Initiating memorandum Negotiations Board approval 12/06/2005 Signing 04/07/2006 Effectiveness 06/20/2006 Closing date 01/31/2011 10/31/2013 33 Table A.4. Staff Time and Cost Staff Time and Cost (World Bank budget only) Staff weeks $, thousands (including Stage of Project Cycle (number) travel and consultant costs) Lending FY04 24.21 140.99 FY05 40.31 236.23 FY06 8.28 53.96 Total 72.80 431.18 Supervision/ICR FY06 12.21 72.40 FY07 14.56 99.31 FY08 13.92 89.84 FY09 17.83 139.83 FY10 21.53 172.23 FY11 6.86 79.37 FY12 12.23 104.99 FY13 16.31 86.38 FY14 7.60 46.28 Total 123.05 890.63 Table A.5. Task Team Members Title Responsibility Names Unit or Specialty Lending Susana Amaral Financial Management Specialist LCSFM Financial Management Judy L. Baker Lead Economist EASIN Economist Fernando Andres Blanco Senior Economist AFTP4 Economist Cossio Jose Augusto Carvalho Consultant LCSPT Eleoterio Codato Sector Manager AFMAO Tulio Henrique Lima Correa Financial Management Specialist LCSFM Financial Management Maria Emilia Freire Consultant World Bank IUR Daniel R. Gross Consultant AFTCS 34 Gerard Martin La Forgia Lead Health Specialist EASHH Health Kathy A. Lindert Sector Leader LCSHD Human Development Rogerio F. Pinto Consultant AFTHE Alberto Rodriguez Sector Manager, Education ECSH2 Education Maria Angelica Sotomayor Sector Leader LCSSD Sustainable Araujo Development Bernice K. Van Bronkhorst Sector Manager SASDC Disaster Risk Luciano Wuerzius Procurement Specialist LCSPT Procurement Supervision/ICR Sinue Aliram Procurement Specialist LCSPT Procurement Susana Amaral Financial Management Specialist LCSFM Financial Management Judy L. Baker Lead Economist EASIN Etel Patricia Bereslawski Senior Procurement Specialist LCSPT Procurement Aberboj Fernando Andres Blanco Senior Economist AFTP4 Economist Cossio Cidalia Brocca Finance Analyst CTRLN Finance Marcus Vinicius Ferreira Da Consultant LCSUW Engineer Silva -HIS Nicolas Drossos Consultant EASNS Maria Emilia Freire Consultant World Bank IUR America Teresa Genta Fons Lead Counsel LEGLA- Legal HIS Ivo Imparato Senior Urban Development TWIWP Urban (Task Specialist Team Leader) Daniel R. Gross Consultant AFTCS Social Specialist Jose C. Janeiro Senior Finance Officer CTRLA Finance Jorge Kamine Counsel LEGLA- Legal HIS Marta Elena Molares-Halberg Lead Counsel LEGES Legal Catarina Isabel Portelo Senior Counsel LEGLE Legal Luis R. Prada Villalobos Senior Procurement Specialist MNAPC Procurement Maria Angelica Sotomayor Sector Leader LCSSD Sustainable Araujo Development 35 Bernice K. Van Bronkhorst Sector Manager SASDC Disaster Risk (Task Team Leader) Sameh Naguib Wahba Sector Manager, Urban UDRUR Urban Anchor (Task Team Leader) Luciano Wuerzius Procurement Specialist LCSPT Procurement Taimur Samad Task Team Leader/Senior Urban EASIS Urban (Task Economist Team Leader) Tatiana Cristina O. de Abreu Finance Analyst CTRLN Finance Souza Controller Francesco Notarbartolo di Consultant UDRUR Social Specialist Villarosa Alessandra Campanaro Senior Infrastructure Finance LCSDU Infrastructure Specialist Finance (Task Team Leader) Emanuela Monteiro Urban Specialist LCSDU Urban (Co-Task Team Leader) Alberto Costa Senior Social Development LCSSO Social Specialist Development Clarisse Dall Acqua Senior Environmental Specialist LCSEN Environment Danilo Pereira de Carvalho Procurement Specialist LCSPT Procurement Paulo Fantini Consultant LCSDU Infrastructure Klas Ringskog ICR Consultant LCSDU Infrastructure Luz MarÍa Gonzalez Economic Analysis Specialist LCSWS Water Supply and Sanitation Fabiana Meacham Consultant LCSDU Urban 36 Appendix B. Vulnerability Map Map B.1. Vulnerability Map of Salvador Macro-Areas Source: CONDER and Census data 2010. Note: This map shows a high concentration of vulnerability in macro-area 3 (Ribeira Azul/Cobre, which make up two corresponding poligonal de intervenção social (PIS) and macro-areas 2 and 6 (Castelo Branco/Jaguaribe and Pituaçu/Boca do Rio—PIS Pau da Lima is located on the border between them). Possibly, only macro-area 1 shows a similar degree of vulnerability. 37 Appendix C. List of Persons Met during the Independent Evaluation Group Mission Deusdete Fagundes de Brito, Housing and Urban Integration Director , CONDER Regina Lúcia Pereira de Assis Luz, Project Coordinator – COPRO/DIHAB (former Project Technical Unit manager), CONDER Adriana Luz, Housing Coordinator, CONDER Tanísia Vieira, Housing Coordinator, CONDER Maria do Socorro Martins, Social Coordinator, CONDER Ana Paula Santana, Land Titling Coordinator, CONDER InfoRMS (geographic information system and data management sector), CONDER Cosme Miranda, São Bartolomeu Park administrator, CONDER Gabriel Nunes, Housing Director, Secretaria de Desenvolvimento Urbano Adalva Tonhã, Land Titling Manager, Secretaria de Desenvolvimento Urbano Raimilton Conceição de Carvalho, Social Operator Movimento de Cultura Popular do Subúrbio André Santana da Silva, former project consultant for job and income-generation subcomponent, Coordinator of Casa Civil of the state government of Bahia Ângela Gordilho, professor at the Faculty of Architecture and Urbanism, Federal University of Bahia Maria de Lourdes Conceição Nascimento, of local community-based organizations Rede Campi and ABDAE and Ssocial Operator CAMA Sérgio Bulcão, Social Operator União Moradia Popular Ana Carolina Valverde, professor at the Faculty of Architecture and Urbanism at UNIFACS (a private university in Salvador) Inês da Silva Magalhães, former Secretary of Housing at the Ministry of Cities, Federal Government of Brazil. Fabrizio Pellicelli, General Manager, the Italian Development Cooperation, and Association of Volunteers for International Service 38 Emanuela Monteiro, Urban Specialist, World Bank Sameh Naguib Wahba, Director of Social,Urban, Rural and Resilience Global Practice, World Bank 39 Appendix D. Visual Assessment of Project Areas before and after project completion Photo D.1. Sussuarana/Baixa da Paz, Ex Ante Photo D.2. Sussuarana/Baixa da Paz, Ex Post 40 Photo D.3. Sussuarana/Baixa da Paz Ex Ante, Durante, Ex Post 41 Photo D.4 Feira de Santana Anchieta Avenue Ex Ante Photo D.5 Feira de Santana Anchieta Avenue Ex Post 42 Photo D.6 São Bartolomeu Ex Ante Photo D.7 São Bartolomeu Ex Post 43 Photo D.8 São Bartolomeu Ex Ante Photo D.9 São Bartolomeu Relocated in Encostas de Piraja Ex Post 44 Photo D.10 São Bartolomeu Park Oxum Square Ex Ante Photo D.11 São Bartolomeu Park Oxum Square Ex Post 45 Photo D.12 Mangueira III Ex Ante Photo D.13 Mangueira III Ex Post 46 Photo D.14 Mangueira III-Ring road Ex Ante Photo D.15 Mangueira III-Ring road Ex Post 47 Appendix E. Borrower Comments APRESENTAÇÃO Esta Nota Técnica está sendo emitida após o recebimento do Relatório Preliminar de Avaliação de Desempenho do Projeto Viver Melhor II promovido pelo Grupo de Auditores Independentes do Banco Mundial. Esta Nota Técnica contém as revisões e os comentários ao Relatório Preliminar que deverão ser encaminhados via e mail até o dia 28 de junho de 2018 para serem refletidos no Relatório Final de Avaliação. DESTAQUE 1. Cabe destacar a oportunidade de uma avaliação em profundidade dos resultados do programa de Desenvolvimento Integrado de Áreas Urbanas Carentes no Estado da Bahia cinco anos após o fechamento do projeto em 31 de outubro de 2018. Verificou-se nesta avaliação que houve por parte do Governo do Estado da Bahia o cumprimento dos compromissos acordados com o Banco Mundial efetivando todos os reassentamentos pendentes com desembolso adicional de recursos de US$ 56,3. 2. Embora este fato tenha sido destacado neste Relatório Preliminar de Avaliação de Desempenho não está refletido na avaliação do Desempenho do Tomador do Empréstimo (Pag. xi, Pag. 21) que não sofreu nenhuma modificação no período do Relatório de Fechamento do Projeto em 2013, na revisão do Relatório de Fechamento em 2015 e neste Relatório de Avaliação de Desempenho de 2018. Há por parte do Governo do Estado a clara expectativa, refletida desde o Relatório de 2015, de passar seu desempenho de moderadamente insatisfatória para moderadamente satisfatória. 3. É fato que a economia do Brasil entrou em crise evidenciada pelo comportamento do PIB a partir do ano de 2014 com leve arrefecimento em 2017. Esta situação de profunda recessão atingiu a renda da população em especial da população mais pobre e a disponibilidade de postos de trabalho com demissões ao invés de abertura de novos empregos. Este fato repercutiu muito no teor das entrevistas efetuadas para a avaliação do Programa Viver Melhor II em especial no que se refere ao desempenho dos serviços sociais. 48 PONTOS PRINCIPAIS A SEREM REVISADOS NO RELATÓRIO PRELIMINAR 4. Na Pagina 8 do Relatório Preliminar apresenta-se a Tabela 3.1 – Opções de Reassentamento que foi revisada pela CONDER e apresenta os seguintes números: FEIRA DE MUNICIPIO SALVADOR SANTANA PIS COBRE PAU DA LIMA RIBEIRA MANGABEIRA São Encosta TOTAL São SusuaranaIII/ Mangueira Avenida Bartolomeu de Bartolomeu Baixa Paz III Anchieta PIF Park Pirajá Reassentamento 256 215** 10 121 132 358 1092 Indenização Extrajudicial( Monetária) 155 148 16 18 17 18 372 Indenização Monitorada 120 111 2 0 0 0 233 Judicial 13 40 0 0 0 0 53 Total de Soluções 544 514 28 139 149 376 1750 Total de Famílias Seladas 503 500 28 132 140 320 1483 Unidades Novas Produzidas 256 65* 160 121 181* 358 1.141 *Correspondem a unidades produzidas em Mirante da Enseada para abrigar famílias moradoras do Parque São Bartolomeu e Unidades Produzidas em Alagados III fora de Mangueira III, mas na PIS Ribeira. ** Reassentado na Encosta de Pirajá 5. Na pagina 9 do Relatório apresenta-se a tabela 3.2 que foi revisada pela CONDER e que apresenta os seguintes números: Situation as of March 2018 São Encosta TOTAL São SusuaranaIII/ Mangueira Avenida Bartolomeu de Bartolomeu Baixa Paz III Anchieta PIF Park Pirajá Effective 1/ 531 477 28 139 149 376 1.701 In Process 2/ 0 0 0 0 0 0 0 Situation defined 3/ 0 3 0 0 0 0 3 Judicial 13 34 0 0 0 0 47 TOTAL 544 514 28 139 149 376 1.750 1/ indemnification and resettled 2/ resettled and indemnification in process of payment 3/resettlement remitted to social rent and indemnity in payment process CONCLUSÕES 49 6. O Governo do Estado da Bahia concorda em termos gerais com o Relatório de Avaliação de Desempenho que foi promovido pelo Grupo de Avaliação Independente do Banco Mundial com a participação efetiva da equipe Técnica da CONDER. 7. As lições aprendidas serão amplamente divulgadas junto ao corpo técnico da empresa para que sejam incorporados nos procedimentos da empresa. 8. O Governo do Estado da Bahia através da CONDER está ciente dos riscos de insustentabilidade a mais longo prazo da operação e manutenção dos equipamentos e dispositivos do Parque São Bartolomeu e está envidando esforços para promover um modelo sustentável e compartilhado de gestão do Parque. Regina Lúcia Pereira de Assis Luz Arquiteta Urbanista CAU –A102513-9 50 Appendix F. Methodology 1. The methodology used in this Project Performance Assessment Report includes the following methods: (i) review of project documents (project appraisal document, Implementation Completion and Results Report, and Implementation Completion and Results Report Review, Back to office reports, aide memoires, Implementation Status and Results Reports, Bahia government plans, and project reports); (ii) World Bank, Cities Alliance, and Federal University of Bahia studies on subjects related to the project; (iii) the Bahia State Company for Urban Development (CONDER) geoprocessed maps; (iv) field visits to project target areas; (v) in-depth interviews with project stakeholders; and (vi) group interviews with project beneficiaries from resettled families, job and income-generation activities, and community-based organizations (CBOs). 13 Information has been triangulated whenever possible, for example, between stakeholder interviews and beneficiary group interviews or between reports and field visits. 2. Project effects on beneficiaries have been assessed through groups interviews—that is, using qualitative methods only. Specifically, measurement of urban poverty reduction (the project’s development objective) through quantitative data was not possible for two reasons. First, geographically disaggregated data at the census track level (the only one that would allow isolating project target areas) are available only for the last 2010 census; in addition, relevant data are collected yearly in the National Domicile Sample Survey (Pesquisa Nacional de Amostra Domiciliar), but they are not statistically representative below the municipal level. Second, no data internal to the project are available on the socioeconomic characteristics of beneficiaries over time. 3. Overall, this assessment has been guided by main evaluation questions and key investigation issues. The Bahia Poor Urban Areas Integrated Development project was a complex project with a wide range of actions. Therefore, the investigation has been focused on a limited and manageable number of issues (from all three project components) that are strategic to assess the project’s performance and the sustainability of its results after project closure, and to extract meaningful lessons. 4. Table F1.1 lists the main evaluation questions together with the corresponding key investigation issues and their justification. Table F1.1. Evaluation Questions Main Evaluation Questions Key Investigation Issues 1. To what extent has the project been successful in improving the quality of life and asset base for the urban poor? a) How successful has the project been in Targeting tools used in project preparation targeting poor and vulnerable people and (vulnerability geoprocessed maps) and their areas? What the key advantages of the use. References: several World Bank works targeting methodology that was adopted? on geographic targeting b. What has happened to the land that was (i) Evidence on the use of land that was vacated because it was high risk? Did they vacated (still vacant or reoccupied? By manage to keep it vacant, or has it been whom? What are the tools CONDER uses to repopulated with squatters? Did they trace control land use and avoid new occupations?) 51 what has happened to the households that (ii) If and what pending works were were relocated? completed (iii) Resettlement issues: different types of resettlement solutions (indemnification, monitored indemnification, new housing production, and so on) and their physical and social implications. Focus will be on resettled families to new houses as part of the urbanization process of PIF and because of their traceability. Questions also to CONDER if and how beneficiaries of indemnification are followed up (iv) Land tenure regularization issues c. Based on focus groups, identify the Physical and social effects on beneficiaries of tangible effects for project beneficiaries from resettlement to new houses and integrated improved infrastructure, housing production, urbanization. Possible synergies between and resettlement. What are the advantages housing production (in situ) and slum and challenges of such an integrated slum- urbanization (drainage, paving, and so on), upgrading approach? for example, better integration between new houses development (conjuntos) and their surrounding areas d. What is the integration, if any, of physical Linked to item 2a (below). Focus on the interventions supported by the project and technical integration between works under broader slum-upgrading interventions VMII and works financed through other financed by the state of Bahia and the federal funds, and their systemic effects on the government? What are the implications territory, including in integration of poor regarding upscaling and sustainability? areas to the urban fabric 2. To what extent has the project been successful in improving the access of the target population to social programs in different areas (for example, health, education, day care, vocational training, income and employment generation, crime and violence prevention, social capital strengthening, and other social and cultural equipment)? a. What additional social services, New social equipment provided by the equipment, and programs were provided by project. Improved access to existing the project, and to how many beneficiaries? equipment provided by the project, including both public and nongovernmental equipment and services. Focus on productive inclusion subcomponent of project component 2 (among all social subcomponents this was the most structured and later evolved into a state program.) b. What are the advantages and challenges of Focus on component B subcomponent of a participatory strategy whereby social CBO strengthening through public bids services are provided by civil society whereby local CBOs from all project PISs organizations to complement provision by were selected on a competitive basis to the state? How effective is service delivery, receive technical assistance and small funds. and how sustainable is such a strategy after What are the performance and effects of this project closure? subcomponent and what is its sustainability? c. What are the synergies, if any, between the Focus on the degree of integration between physical interventions and the social PIF and PIS interventions. To what extent interventions? What are the advantages (for have beneficiaries received both types of example, in operation and maintenance of the interventions, and what are the effects? What 52 physical investments and post-occupation in are the effects of stronger-than-average social general) versus the challenges (for example, support on resettled families? What state in efforts for coordination and integration)? institutions have been involved in managing different equipment and providing different services, and how are they coordinated? 3. To what extent has the project been successful in strengthening slum-upgrading policy design and implementation capacity at the state and federal level? What are the key lessons that have been extracted and successfully transferred from the project? a. What specific policies and plans have been Institutional development of SEDUR produced in Bahia under the project or under (housing and land tenure regularization its influence, and what are their effects? policy and plans) and CONDER. What additional resources and investments Technical projects produced under VMII and have been mobilized that can be associated to used to access PAC and Minha Casa Minha project studies and capacity building? Vida resources b. What approaches, tools, and Social work ordinance 21/2014 and methodologies from the project have nationwide distance learning course, informed the national slum upgrading and methodological kits for CBO strengthening, social housing policy, and what are the income and employment generation, results? participatory planning c. What is the impact of the project on a Transfer of methodologies to Mozambique global scale? under a Cities Alliance–government of Italy– ProMaputo Municipal Developent Program – WB financed project; influence on the Cities Alliance global strategy. Source: IEG. Note: CBO = community-based organization; CONDER = Companhia de Desenvolvimento Urbano da Bahia (Bahia State Company for Urban Development; PAC = Programa de Aceleração do Crescimento (Growth Acceleration Program); PIF = poligonal de intervenção física (physical intervention area); PIS = poligonal de intervenção social (social intervention area); SEDUR = Secretaria de Desenvolvimento Urbano (Secretariat for Urban Development); VMII = Bahia Poor Urban Areas Integrated Development Project. 13 The participation of beneficiaries of job and income-generation activities and (partly) of benefited community- based organizations in group interviews was limited. It was difficult to find and reach them because of the long time since project closure and beneficiaries’ frequent change of address and mobile number.