Operation Clean Neighborhood: working with communities for flood risk mitigation in Senegal Impact Evaluation Final Report March 2018 1 Contents I. Acknowledgements............................................................................................................................... 4 II. Executive summary ............................................................................................................................... 5 III. Context and Introduction.................................................................................................................. 8 IV. Intervention .................................................................................................................................... 12 1. Theoretical Rationale ...................................................................................................................... 12 2. Geographic scope............................................................................................................................ 13 3. OQP Description .............................................................................................................................. 16 4. Implementation Challenges ............................................................................................................ 20 V. Impact Evaluation Questions and Design ........................................................................................... 22 1. Impact evaluation questions ........................................................................................................... 22 2. Impact evaluation study design ...................................................................................................... 22 VI. Data Sources ................................................................................................................................... 25 1. Household Survey ........................................................................................................................... 25 2. CBO Survey ...................................................................................................................................... 25 3. Practical Challenges in Survey Implementation.............................................................................. 26 VII. OQP Impact Evaluation Results....................................................................................................... 27 1. Baseline characteristics ................................................................................................................... 27 2. Impact on CBO behavior ................................................................................................................. 32 3. Impact on household behavior and attitudes................................................................................. 34 4. Impact on household outcomes related to flooding ...................................................................... 41 VIII. Conclusion ....................................................................................................................................... 53 Appendix 1: Additional results tables ......................................................................................................... 54 Tables Table 1: Commune distribution .................................................................................................................. 15 Table 2: Social Facilitator Grading Scale ..................................................................................................... 17 Table 3: Communes, groups and neighborhoods in the study area ........................................................... 23 Table 4: Treatment and control quartiers allocation across the PROGEP intervention area ..................... 24 Table 5 : Summary of household questionnaire content ........................................................................... 25 Table 6: Surveyed households and CBO ..................................................................................................... 27 Table 7: Baseline household and individual characteristics ....................................................................... 29 Table 8: Baseline CBO characteristics ......................................................................................................... 31 Table 9: CBOs that list cleaning activities among top 3 activities ............................................................... 33 Table 10: Household Awareness of OQP and opinion of impact ................................................................ 37 Table 11: Training and impression of cleanliness of neighborhood ........................................................... 40 Table 12: Vulnerability to Flooding Most Recent Year ............................................................................... 44 Table 13: Vulnerability to Flooding Previous Year ...................................................................................... 45 Table 14: Flooding index ............................................................................................................................. 48 Table 15: Prevalence of illness due to Flooding.......................................................................................... 50 2 Table 16: Impact on Work........................................................................................................................... 52 Table 17: Prevalence of Illness .................................................................................................................... 54 Table 18: Impact on education outcomes .................................................................................................. 55 Figures Figure 1 : OQP Theory of Change ................................................................................................................ 13 Figure 2 : PROGEP Phases 1 and 2 (quartiers participating in the OQP impact evaluation are highlighted) .................................................................................................................................................................... 15 Figure 3: OQP Timeline ............................................................................................................................... 20 Figure 4: Distribution of CBOs by type ........................................................................................................ 21 Figure 5: CBOs that list cleaning activities among top 3 activities ............................................................. 32 Figure 6: Household Awareness of OQP ..................................................................................................... 34 Figure 7: Household Awareness of OQP by phase ...................................................................................... 35 Figure 8: Households stating that they believed OQP would improve general cleanliness ....................... 36 Figure 9: Households stating that they believed OQP would improve general cleanliness, by Phase ....... 36 Figure 10: Households that received training in last 12 months ................................................................ 38 Figure 11: Household perception of quartier cleanliness ........................................................................... 39 Figure 12: Neighborhood is responsible for lack of cleanliness ................................................................. 40 Figure 13: Probability of being a flood victim in the last 12 months .......................................................... 41 Figure 14: Probability of one’s house being flooded in the last 12 months ............................................... 42 Figure 15: Impact of treatment on probability of being a flood victim by phase ....................................... 43 Figure 16: Impact of treatment on probability of one’s house being flooded by phase ............................ 43 Figure 17: Flooding index in year prior to survey ....................................................................................... 46 Figure 18: Flooding index in year prior to endline survey by phase ........................................................... 47 Figure 19: Flooding index at endline over both years, by phase ................................................................ 47 Figure 20: Illness due to flooding in the second-to-last rainy season ........................................................ 49 Figure 21: Income lost due to flooding in the second-to-last rainy season................................................ 50 Figure 22: Income lost due to flooding in the second-to-last rainy season by phase ................................ 51 Images Image 1: Badly maintained public spaces in the city of Pikine ..................................................................... 9 3 I. Acknowledgements The impact evaluation (IE) described in this report is the result of a collaboration between the Dakar Municipal Development Agency (Agence de Développement Municipal), the World Bank (Development Impact Evaluation, DIME; and the Social, Urban, Rural, and Resilience Global Practice), and Trinity College Dublin. This work would not have been possible without the dedication and support of many individuals throughout the process of the conception, design, and implementation of the work presented here. Any list will be incomplete, and so the acknowledgements here are not intended to be exhaustive. With apologies to those not explicitly mentioned here, we would especially like to acknowledge the critical role played by the following individuals in the PROGEP IE. From the Agence du Developpement Municipale, Cheikh Issa Sall (ADM Director), Pierre Coly (PROGEP Component C Coordinator) and Ba Mouamadou (OQP Coordinator). From the World Bank, Denis Jordy (former PROGEP TTL), Isabelle Kane (PROGEP TTL), Sokhna Ba (PROGEP Disaster Risk Management Specialist), Fatim Seck (PROGEP ACS) and Gabriele Rechbauer (Former PROGEP Consultant). From Enda Ecopop, Mamadou Mansour Diagne and Mangane Djibril. From MSA, Cheikh Sarr, Pape Diouf, Pathe Ndiaye and Ibrahima Sagna. This work has benefitted from the generous support of the Bank Netherlands Partnership Program (BNPP), the Impact Evaluation to Development Impact (i2i) Program, and the Senegal Country Management Unit. Data collection for the impact evaluation was funded through PROGEP, i2i, and a grant from the Nordic Development Fund. This report was produced by Chloë Fernandez (Research Analyst, DIME, World Bank), Marcus Holmlund (Economist, DIME, World Bank), Tara Mitchell (Assistant Professor, Trinity College Dublin), and Carol Newman (Professor, Trinity College Dublin). Additional persons that have contributed to this work as members of the IE research team include Molly Offer-Westort, Violaine Pierre, Qiao Wang, Michell Dong, Felipe Dunsch, Ababacar Gueye, and Margaryta Klymak. For additional information, please contact Marcus Holmlund (mholmlund@worldbank.org) and Chloë Fernandez (cfernandez2@worldbank.org). 4 II. Executive summary This report presents results from an impact evaluation study of the Operation Clean Neighborhood (Opération Quartier Propre; OQP) intervention, which was carried out in Senegal under the World Bank- assisted Stormwater Management and Climate Change Adaptation Project (Projet de Gestion Environnemental et d’Adaptation au Changement Climatique; PROGEP). Phase 1 of this US$112.3 million project was implemented from 2012-2017 by the Municipal Development Agency (Agence de Développement Municipal; ADM), while phase 2 (additional financing) is set to end in 2019. The project is co-funded by the Government of Senegal, the World Bank and the Nordic Development Fund. The work presented in this report is the result of a multi-year collaboration between ADM, the World Bank, and Trinity College Dublin. PROGEP targeted the population of the peripheral area of the Dakar region in the cities of Pikine and Guédiawaye. These two low-income municipalities represent 12% of the national population and are highly prone to flooding. This is the result of rapid and unplanned urbanization, increased rainfall after a period of relative drought (during which areas which are now highly flood-prone were populated), rising groundwater, and inadequate drainage infrastructure. This has important impacts on health, livelihoods, and assets, and disproportionately affects more vulnerable population groups. PROGEP aimed to contribute in a sustainable and participatory way to the reduction of flood risks in the peri-urban areas of Dakar. The project had three operational components: institutional strengthening and management capacity for stormwater drainage and urban planning (Component A); development of priority primary drainage infrastructure (Component B, which accounts for approximately 73% of project investment); and community engagement in urban flood-risk reduction and management (Component C). The major investment under PROGEP is the construction of new drainage infrastructure in flood-prone areas (Component B). While technical approaches, such as those financed under Component B, exist to alleviate the problem of wastewater management in the short term, maintenance and sustainability of infrastructure investments require both ongoing national and local government operational and financial support, and individual and collective behavior of community members conducive to the functioning and sustainability of these investments. Frequently, such investments suffer from the so-called “tragedy of the commons” – individual community members have the incentive to abuse these common resources as they reap the benefits, whereas the costs are shared by the community as a whole. In the aggregate, this reduces the overall effectiveness and return on these investments. However, we know little about how to effectively engage urban populations for the sustainable operation and maintenance of community resources. For this reason, the impact evaluation study presented in this report focuses on “Operation Clean Neighborhood” (Opération Quartier Propre; OQP), a Component C intervention, which engages local Community Based Organizations (CBOs) to work with their immediate communities to improve and maintain the cleanliness of public spaces and drainage infrastructure. Partner CBOs receive an initial 5 endowment consisting of cleaning material and tools to support them in their task, CBOs are then eligible for non-monetary incentives, including public recognition, based on a basic external assessment of their communities’ cleanliness. The intervention was launched in September 2015 and lasted one year. The objective of the PROGEP impact evaluation is to build evidence on the effectiveness of OQP, over and above all other investments and actions under PROGEP and by others in project areas. To do so, the impact evaluation uses a cluster randomized controlled trial study design, which allows us to isolate the specific impacts of OQP. Of the 391 quartiers (or neighborhoods) in PROGEP areas, 160 were randomly selected to participate in the OQP impact evaluation. Through a public lottery, 80 quartiers were then randomly assigned to participate in the intervention while the other 80 were assigned to a control group. Data collection included a quantitative baseline and endline survey which collected panel data from households and CBOs. In total, 28,010 individuals, 2,400 households, and 160 CBOs were surveyed in Pikine and Guédiawaye in both rounds. The OQP impact evaluation addresses two primary questions: 1. What is the additional impact – beyond all other PROGEP activities – of OQP on community engagement and cooperation to maintain clean public spaces? 2. What is the additional impact – beyond all other PROGEP activities – of OQP on attitudes and behaviors, losses and damage to property, and quality of life (including health, income, and education)? An important step in establishing the impact of the OQP on the outcomes of interest is to first consider the effectiveness of the implementation and the extent to which the intervention was well-received by households. The results from our end-line survey show that CBOs in treatment quartiers were more likely to undertake cleaning events compared with CBOs in control quartiers. Moreover, households in treated quartiers were more likely to have heard of the OQP. These survey results are correlated with the external quartiers grading done by the implementing agencies. These findings suggest that the intervention was implemented effectively. In relation to our first core research question we find that households in the treatment area have a better perception of the cleanliness of their area compared with the control group. These findings suggest that the OQP had a positive impact on community engagement in the maintenance and cleanliness of public spaces. In relation to our second core research question we find that the increased community engagement through the OQP also impacted on some indicators of quality of life. In particular, households in treated areas were less affected by flooding in the past rainy season, and reported reduced levels of both illness and income lost due to flooding. PROGEP was rolled out in two phases corresponding to two different geographical zones: Dalifort- 6 Thiouorour (Phase 1) and Yeumbeul-Mbeubeuss (Phase 2). PROGEP’s infrastructure component was first implemented in the Phase 1 area. Given that, at the time of OQP implementation, Phase 2 quartiers had not yet received the infrastructure component we might expect differential effects of the OQP on treated households compared to control households depending on what Phase of PROGEP they are in. To explore this possibility, we consider interactive effects of being in a treated area in Phase 1 compared with Phase 2. We find that while households in Phase 1 areas are generally more likely to have heard of the program, the effect of the OQP treatment is more pronounced for households in Phase 2. Treated households in Phase 2 areas are more likely than control areas to report that OQP will result in the area being cleaner while there is no difference between treatment and control households in this outcome in Phase 1 areas. The differences in the impact of treatment between Phase 1 and Phase 2 are also evident in flooding related outcomes. We find that for treatment households in Phase 2 areas, OQP reduced the probability of experiencing flooding relative to control households while there is no statistically significant difference between treatment and control households on the incidence of flooding in Phase 1. The results of this IE provide evidence that a program targeting established community-based groups with a non-monetary incentives-linked “social contract” can be effective at improving community engagement and potentially overcoming the tragedy of the commons problem by facilitating collective action. This is particularly the case in areas where infrastructure is not yet well developed and so in addition to being a complement to large infrastructural investments, an OQP-type intervention could be considered as an interim substitute while waiting for such investments to catch up with the needs of communities living in peri-urban areas at risk of flooding. Finally, the results of this IE highlight that local waste management could be improved with little investment and that light touch interventions targeting community engagement and a shift in behavioral norms could have a substantive positive impact on the cleanliness of public spaces and on quality of life. 7 III. Context and Introduction Recurrent stormwater flooding is perhaps the most serious natural hazard faced by Senegal over the last three decades. Between 400,000 and 600,000 people per year were affected by floods between 1980 and 2008, causing damage to infrastructure, public equipment and household economic losses. In 2009, serious flooding caused an estimated USD 104 million of damage/loss and directly affected 360,000 people, with the low-income peri-urban areas of Dakar most seriously affected. In August 2012, historic rainfall again caused devastating floods, leading to the death of 21 people and affecting 264,000 people in total. Located in the peripheral area of the Dakar region, the cities of Pikine and Guédiawaye are large agglomerations which, according to the latest estimates of the National Agency for Statistics and Demography (ANSD), have a population close to 1.3 million inhabitants – two-thirds of them in Pikine – which is 12% of the national population. These two low-income municipalities are highly prone to flooding, and approximately half of residents live in flood-prone areas. This is the result of rapid and unplanned urbanization; increased rainfall after a period of relative drought in the 1970s to late 1980s during which informal communities settled in areas that are now highly flood-prone; rising groundwater; and inadequate drainage infrastructure. This has important impacts on health, livelihoods, and assets, and disproportionately affects more vulnerable population groups. Emergency solutions have been implemented in the peri-urban areas of Dakar to facilitate the drainage and storage of stormwater through the construction and rehabilitation of retention basins. However, historically, investments to decrease the risk and impact of flooding in peri-urban Dakar have been limited and uncoordinated, leading in particular to poor provisions for the maintenance of infrastructure. There has been little engagement of local populations in the planning and construction of drainage and water storage infrastructure, and consequently very little sense of ownership of these resources by their host communities. Provisions for operations and maintenance have been inadequate, and oftentimes the infrastructure has been inappropriately used to dump waste and also subject to vandalism. What was intended as a community good was therefore often converted to a source of disease and insecurity. With this in mind, in 2012 the Government of Senegal launched the World Bank-assisted Stormwater Management and Climate Change Adaptation Project (Projet de Gestion Environnemental et d’Adaptation au Changement Climatique; PROGEP) to contribute in a sustainable and participatory way to the reduction of flood risks in the peri-urban areas of Dakar. This US$112.3 million project was implemented by the Dakar Municipal Development Agency (Agence du Developpement Municipal; ADM). PROGEP has three operational components: institutional strengthening and management capacity for stormwater drainage and urban planning (Component A); development of priority primary drainage infrastructure (Component B, which accounts for approximately 73% of project investment); and community engagement in urban flood-risk reduction and management (Component C). The major investment under PROGEP is the construction of new drainage infrastructure in flood-prone 8 areas (Component B). Such investments, however, suffer from the “tragedy of the commons” – individuals have the incentive to abuse shared resources as they personally incur the benefits while the costs are shared by the whole community (Ostrom, 1990). In the absence of collective action or adequate institutions, individual users acting rationally and independently in their own self-interest do not fully internalize the costs of their use of the public resource and will therefore tend to over-utilize (or incorrectly utilize) it. Ultimately, this leads to depletion of the common resource to the point at which it is of no use to anyone and may actually convert from a net “good” to a net “bad”. In the context of PROGEP, i.e. in Pikine and Guediawaye, the tragedy is that public spaces are often used to dispose of individual/household waste, clogging stormwater drainage systems and reducing their effectiveness, sustainability, and the return on this major investment. Image 1: Badly maintained public spaces in the city of Pikine While technical approaches, such as those financed by PROGEP, exist to solve the problem of wastewater management in the short term, maintenance and sustainability of infrastructure investments require both ongoing national and local government operational and financial support, and individual and collective behavior of community members conducive to the functioning and sustainability of these investments. Component A of PROGEP aims to strengthen the capacities of national agencies and the municipalities of Pikine and Guédiawaye to address flood risk management and urban climate change adaptation through improved urban planning, institutional strengthening, and the development of an integrated urban flood risk and stormwater management program in the peri-urban areas of Dakar. The Code de l’Assainissement (Sanitation Law, 2009) indicates that municipalities are responsible for stormwater infrastructure investments, operations and maintenance, and in 2012 the Municipal Development Agency signed a five- 9 year contract with the municipalities of Pikine and Guédiawaye (“Contrat de ville”), where they list actions that will be taken to empower municipalities, and what is expected from them. In particular, two years after the construction of canals and water ponds, municipalities will be fully in charge of the operation and maintenance of this infrastructure. However, there is recognition that national and local government actions alone will not be sufficient for the long-term sustainability of the stormwater drainage system. In particular, active engagement of target beneficiaries will be needed to maximize returns on infrastructure investments and to assure their sustainability. A major challenge for PROGEP, therefore, was to effectively engage these target beneficiaries to improve the full functionality and sustainability of infrastructure investments (including open and closed canals and water storage basins). This was addressed through Component C, implemented by two “Social Facilitators” (local NGOs contracted under PROGEP). Component C includes awareness-raising measures (public information campaigns; community meetings), and three community-based interventions: A. Community Investment Projects (Projets d’Investissements Communautaires; PICs), B. Local Rainwater and Climate Change Management Committees (Comites Locaux d’Initiatives pour la Gestion des Eaux Pluviales et Changement Climatique; COLIGEP). C. Operation Clean Neighborhood (Operation Quartier Propre; OQPs) PICs are investments of US$10-70k implemented by CBOs, with a high-level of support and supervision from ADM and its implementing partners. PICs focus on adding value to the areas surrounding water retention ponds (e.g. walk paths, fishing areas, picnic tables, sports infrastructure, etc.), securing unbuildable lands, improving waste management around drainage infrastructure, and decreasing possible negative impacts of drainage infrastructure, such as malaria transmission. COLIGEPs, on the other hand, are new coordination structures, which bring together representatives of the different types of local actors (elected, associative, administrative). Their implementation in each commune by the PROGEP Social Facilitator is intended to promote community resilience. Through their unifying character and their mission to strengthen community resilience, COLIGEPs can initiate and promote the implementation of PICs, but above all they are intended to ensure the monitoring, maintenance, and sustainability of drainage infrastructure and PICs in their area. The OQP intervention seeks to further promote active engagement by residents in the cleanliness of their neighborhoods, or quartiers. To do so, the project engages existing Community Based Organizations (CBOs) to work actively towards maintaining the cleanliness of public spaces in their neighborhood. CBOs are provided with an initial endowment of cleaning materials, and are offered a non-monetary award conditional on the future cleanliness of their neighborhood. If the intervention is successful, this should reduce the misuse of public spaces (including drainage infrastructure) for inappropriate uses such as waste dumping, which will improve the functioning and sustainability of PROGEP (and other) drainage infrastructure and consequently improve socioeconomic, health, and other outcomes for residents. 10 OQP is a novel intervention designed in response to a specific problem. To build evidence on the effectiveness of such approaches, an impact evaluation (IE) of OQP was conducted to test its causal impact on outcomes of interest, over and above all other investments and actions under PROGEP and by others in project areas. OQP and its impact evaluation were conceived, designed, and implemented through a partnership between ADM, the World Bank, and Trinity College Dublin. The results are presented in this report.1 Box 1: What is Impact Evaluation? Impact evaluation is the application of a scientific method to resolve development challenges and inform effective policy. The scientific method includes the identification and precise definition of a problem, the development of hypotheses to address the problem based on available evidence, the testing of these hypotheses using an appropriate research design and data, and communication of results. Impact evaluation is based on statistically valid counterfactual analysis. The counterfactual represents that which would have happened to program recipients had the program not taken place, or had a different program design been chosen. This allows us to directly attribute changes in intermediate and final outcomes to specific interventions. Thus, IE complements traditional performance monitoring and tracking of outcomes. When undertaken prospectively and integrated into the operational planning and implementation of a project or policy, IEs are a powerful research and development tool to identify the most effective and cost-effective design options. For this reason, IEs are a management tool for results- based policy planning. At the same time, they are a key instrument for improving aid effectiveness and increasing fiscal accountability. While there are various methods that can be used to construct a counterfactual, the most rigorous approach is an experimental design where treatment (intervention) and control (no intervention) status are assigned to individuals or groups on a random basis. Random assignment ensures that all units have an equal chance of being in the treatment or control groups, and satisfies the conditions of a valid counterfactual comparison. These are that: (i) all pre-intervention characteristics, both observable (e.g. wealth) and unobservable (e.g. willingness to work together) are equal on average across the treatment and control groups; and (ii) the only difference in observed outcomes is due to the intervention and not to any other observed or unobserved factors. 1 The concept for this intervention and its impact evaluation (IE) was first developed at a World Bank DIME-led Impact Evaluation workshop in Naivasha, Kenya, in April 2012. The design was then further developed by ADM, the World Bank, and Trinity College Dublin. The IE received formal approval from the World Bank’s Senegal Country Management Unit following a concept note review held in November 2013. Detailed design and operational planning for both implementation of OQP and IE data collection was then conducted prior to the start of OQP implementation in September 2015. 11 IV. Intervention 1. Theoretical Rationale OQP engages local CBOs to work with their community to improve and maintain the cleanliness of public spaces and drainage infrastructure. Not much is known about how to shift behavioral norms, particularly in densely populated urban settings, but we hypothesize that any lasting social change must be rooted in existing community structures. For this reason, OQP is targeted at existing, established CBOs, which have strong social connections and influence within the community. We hypothesize that some form of external stimulus will be needed to move away from the present equilibrium, and investigate whether or not OQP can provide this stimulus. We therefore assume that the combination of extrinsic motivation provided directly through OQP and the intrinsic motivation it leverages will be sufficient to overcome individual and group internal and external impediments to civic participation, and that CBOs will have the capacity to promote and manage activities targeting the preservation of public spaces. The success of this approach will be measured in terms of changes in outcomes at the CBO, community, and individual/household level. In particular, we assume that increased CBO knowledge, interest, motivation, and capacity will translate into increased CBO engagement, which will lead to improved cleanliness and functioning of community and infrastructure. At the same time, improved knowledge, attitudes, and behavior relating to flood prevention and risk mitigation at the individual/household level will likewise contribute to positive community outcomes. These, in turn, will translate into improvements in socioeconomic and health outcomes in targeted areas. This theory of change is summarized in Figure 1 below. In the longer run, we expect that these improvements, if maintained over time, could contribute to sustained individual and CBO engagement in the preservation of public spaces, improved infrastructure sustainability and return on infrastructure investments, and increased community resilience to flooding. These things combined will further improve individual and household-level socioeconomic and health outcomes. This report does not, however, address the longer-term sustainability of outcomes as we are limited to one single follow-up survey taking place at the conclusion of the OQP intervention. 12 Figure 1 : OQP Theory of Change 2. Geographic scope OQP implementation took place in PROGEP-covered areas in the peri-urban municipalities of Pikine and Guédiawaye (additionally, PROGEP intervened in a small portion of the borough of Hann Bel Air, formally part of Dakar). Pikine and Guédiawaye are divided, respectively, into sixteen and five communes de plein exercice or communes, each of which comprises a number of quartiers, or neighborhoods. The project covers eight of these communes which include a total of 398 quartiers. PROGEP was rolled-out into two phases. Phase 1 covers Dalifort-Thiouorour (communes of Wakhinane Nimzatt, Djeddah Thiaroye Kao, Dalifort-Foirail, Hann Bel Air and the Western halves of Yeumbeul Nord and Yeumbeul Sud) and Phase 2 covers Yeumbeul-Mbeubeuss (Keur Massar, Malika and the Eastern halves of Yeumbeul Nord and Yeumbleu Sud). PROGEP’s infrastructure component (rehabilitation of old water basins, implementation of new drainage pipelines etc.) was implemented in Phase 1 areas between 2014 and 2016 and in Phase 2 areas between 2017 and 2019. 13 Table 1: Commune distribution summarizes the areas included in Phase 1 and Phase 2, and their geographic distribution is illustrated in Figure 2 : PROGEP Phases 1 and 2 (quartiers participating in the OQP impact evaluation are highlighted. 14 Table 1: Commune distribution Phase Cities Communes Number of quartiers Guédiawaye Wakhinane Nimzatt 47 Dakar Hann Bel Air 2 Djeddah Thiaroye Kao 43 1 (ENDA-ECOPOP) Dalifort-foirail 21 Pikine Yeumbeul Nord 17 Yeumbeul Sud 23 Malika 27 Keur Massar 122 2 (MSA) Pikine Yeumbeul Nord 60 Yeumbeul Sud 36 TOTAL Phase 1 153 TOTAL Phase 2 245 TOTAL 398 Figure 2 : PROGEP Phases 1 and 2 (quartiers participating in the OQP impact evaluation are highlighted) 15 3. OQP Description OQP was implemented in three phases: preparation, intervention, and post-intervention. The preparation phase included the following activities: i. Mapping of quartiers and listing of eligible CBOs in all PROGEP intervention areas by the project’s Social Facilitators (January 2014 to May 2015). ii. Selection of focal CBO in each quartier. The Social Facilitators (SFs), NGOs hired by PROGEP to conduct community engagement activities under component C, identified one focal CBO per neighborhood on the basis of capacity to implement OQP and influence within the community (of the 398 quartiers identified in PROGEP areas, seven were excluded due to the lack of a CBO). As several active CBOs can exist within one quartier, the social facilitators placed them into three categories as follows: (i) CBO is representative and well structured (functional structuring and transparent governance, existence of own resources, high level of participation and involvement of members, etc.), (ii) moderately structured CBO (relatively moderate structuring and governance, weaker finances, lack of shared vision, etc.), and (iii) unstructured and non-functional CBO (non-existent structuring and governance, lack of resources and means of operation, lack of vision, etc.). Priority was given to category 1 CBOs; if a quartier did not have a category 1 CBO then a category 2 CBO was designated as the focal point. If there was a CBO specifically focusing on flood management within the quartier, it was automatically selected as the focal CBO. This was the case for 3% of quartiers. iii. OQP pilot in 5 randomly selected quartiers. As phase 2 benefitted from drainage infrastructure construction and rehabilitation and PICs after phase 1 did, the phase 2 facilitators had more time to dedicate to OQP preparation. In light of this, a pilot of the OQP intervention was rolled-out in 5 randomly selected quartiers of phase 2, which were then excluded from the actual OQP intervention (and impact evaluation). This enabled the teams to test the operational aspects of the intervention and to receive feedback from the pilot CBOs. iv. Pre-selection of 160 quartiers to serve as the basis for OQP and its IE. This figure was based on the available budget for the intervention. Specifically, funding was available to implement the intervention in 80 quartiers, and the other 80 were to serve as a control group for the impact evaluation. The 160 quartiers were selected randomly by the IE team within different groups of quartiers/CBOs, where groups were created by the SFs based on geographic proximity and social ties. See section V: Impact Evaluation Questions and Design for further details. v. Public lottery amongst CBOs in the 160 pre-selected quartiers to determine who would be offered the chance to participate in OQP. This lottery was carried out within the different 16 pre-defined groups of quartiers/CBOs, to ensure an even distribution of treatment and control quartiers/CBOs across the PROGEP area. See section V on IE study design for further details. The intervention phase lasted 12 months and included the following activities: vi. The SFs assisted the intervention group CBOs in creating an action plan for OQP implementation (the SF in PROGEP Phase 1 areas was Enda-Ecopop; in Phase 2 areas, it was MSA). SF support was very limited, with the action plans being primarily developed by the CBOs themselves. vii. Launch of the OQP intervention. The CBOs first signed a Letter of Engagement with their respective communal mayors during an inauguration ceremony. After this, the participating CBOs received a package of cleaning materials2. SFs also assisted CBOs in developing a community cleanliness action plan. It is important to point out that the effective start date of the OQP process was the signing of the Letter of Engagement. The acquisition of cleaning equipment was a complement to the community dynamics and not the focus of the process. The emphasis of the OQP intervention was on community engagement and collective action to improve the local population’s living environments. viii. Six months into the intervention, the SFs carried out an initial evaluation of the cleanliness of quartiers. Each SF rated their counterpart’s phase in order to avoid self-evaluation. The grading criteria are described in Table 2. Quartiers that were rated above a certain threshold (60/100) were publically recognized. 35 quartiers received this award, 11 in Phase 1 areas and 24 in Phase 2 areas. Table 2: Social Facilitator Grading Scale CRITERIA RATIONALE WEIGHT 1. Lack of dumping This criterion relates to whether populations are dumping household waste grounds in the 20 % in public spaces. It also speaks to the community’s collective conscience. streets The cleanliness of social and collective spaces is a fairly strong indicator of 2. Cleanliness of the level of community engagement in sanitation. The score on this criterion 10 % public spaces is based on the cleanliness of markets, schools, and other public places. 3. Cleanliness of Abandoned households in project areas are often transformed into garbage abandoned houses dumping grounds. The state of abandoned houses is therefore indicative of 20 % and lands the collective consciousness of community members. 2 The package included brooms, boots, wheelbarrows, etc. The final version of this report will contain an annex with a full list of material received by CBOs. 17 4. Lack of water on This criterion assesses the extent to which households use public spaces to streets and in front dispose of domestic wastewater, a risk factor for disease. It is also indicative 10 % of houses of collective consciousness. 5. Non-obstruction of natural drainage Localities are often flooded due to waterway obstruction from solid waste. 30 % structures and waterways 6. Innovative This criterion focuses directly on the actions of focal CBOs in OQP quartiers. initiatives taken by 10 % the CBO The post-intervention phase entailed the evaluation of quartier cleanliness and the rewarding of partner CBOs based on improvements in the cleanliness of their quartiers since the 6-month evaluation. ix. At the conclusion of the intervention, the Social Facilitators repeated their assessment of quartiers’ cleanliness. The grading process and scale were the same as for the mid-term evaluation (Table 2). The final reward consisted of t-shirts, chairs, and cooking utensils which were distributed to the focal CBO and to other quartier residents. A final ceremony took place in order to celebrate the quartiers publicly and clean neighborhood banners were provided for display in the quartiers meeting assessment criteria. 50 quartiers received the final award, 22 in Phase 1 areas and 29 in Phase 2 areas. OQP implementation is summarized in 18 Figure 3: OQP Timeline. 19 Figure 3: OQP Timeline Initial Discussion (April Endline Data Collection Final Evaluation by SF 2012) (November 2016) (December 2016) Follow-up survey design Baseline and OQP design Public Recognition Final Endowment Delivery Concept Note (November Ceremony and Award Ceremony 2013) (February 2016) (May 2017) Internal Randomization - Selection of 160 quartiers to Mid-Term Evaluation by SF be surveyed at baseline (January 2016) (April 2015) Public Lottery - selection of Baseline Data Collection 80 OQP quartier (July 2015) (August 2015) Several stakeholders in Senegal contributed to the implementation of OQP. These include: (i) the municipalities of Pikine and Guédiawaye and the borough of Hann Bel Air in Dakar; (ii) the Municipal Development Agency (Agence du Developpement Municipal; ADM), particularly the Department of Institutional Support; and (iii) Enda-Ecopop and MSA, PROGEP’s SFs. 4. Implementation Challenges Three primary challenges were observed in implementing OQP. 1. Identification of quartiers Whereas communes are formal administrative units with established and mapped geographic boundaries and known local representatives, quartiers are a mix of informal, semi-formal, and formal agglomerations. In fact, quartiers tend to be largely informal in nature. The SFs therefore spent over a year (January 2014 to May 2015) identifying and mapping the quartiers in PROGEP areas, in an iterative process which also involved ADM and the impact evaluation team. Not only are quartiers primarily informal groupings; in some cases, they are flexible constructs with varying geographic boundaries. For example, quartiers are led by a chef de quartier, and if such a quartier chief passes, the area may be absorbed by a neighboring quartier. Also, two chiefs can exist within the same quartier, potentially leading to internal conflict and the splitting of quartiers. 20 2. Selection and retention of CBOs CBOs were in charge of implementing OQP within their own quartier. Many CBOs can exist within the same quartier, including women’s groups, religious groups, etc. (see Figure 4). Just like quartiers, CBOs are dynamic entities that can be dissolved or absorbed by one another. Eight CBOs originally targeted as OQP participants and who were surveyed at baseline, prior to the launch of the intervention, were not present for the start of OQP. They were therefore replaced by the next most-suitable CBO in the quartier. No OQP-participating CBOs were, however, dissolved during implementation. It was possible that a CBO chose to work with another quartier CBO as part of the intervention. So long as activities remained confined to the quartier, this remained in line with the intervention design. Figure 4: Distribution of CBOs by type CBO TYPE YOUNG MOVEMENT SPORTS AND CULTURE ASSOCIATION, 5% ASSOCIATION(ASC), 13% ECONOMIC INTEREST GROUPING(GIE), 13% DEVELOPMENT ASSOCIATION , 46% WOMEN'S INVOLVEMENT GROUP(GPF), 18% FLOOD PREVENTION RELIGIOUS GROUP , 4% COMMITTEE , 5% 3. Local politics PROGEP areas are highly politicized, and any activity coinciding with an election could be perceived as explicitly political. For this reason, OQP preparation activities were put on hold prior to local elections in June 2014. No adverse consequences for the intervention were observed. 21 V. Impact Evaluation Questions and Design 1. Impact evaluation questions The PROGEP IE is designed to test the effectiveness of the OQP intervention, over and above the infrastructural and other investments of the project. For example, in addition to infrastructure investments under Component B in Phase 1 areas, all areas covered by the PROGEP IE were exposed to an information campaign entitled “Live better in our environment.” This campaign provides general information about responsible civic behavior in relation to flood prevention and aims to encourage people to take into account the effects that their individual actions have on the community as a whole. In addition, all communities included in the IE were given the opportunity to submit proposals for community micro-projects (Projets d’Investissement Communautaire, or PICs). Such projects – ranging in budget from USD 10,000 to 70,000 – were intended to reinforce community capacity and increase engagement in the reduction of risks linked to flooding and climate change. And all PROGEP areas were covered by a COLIGEP. See Section III: Context and Introduction for a summary of PROGEP activities. The PROGEP impact evaluation takes infrastructure (or lack of infrastructure), the information campaign, PICs, and all other project activities as given and focuses on OQP. The objective is to isolate the impact of this specific intervention, over and above all other project activities. OQP was designed as a complement to other community engagement activities under PROGEP’s Component C, in order to involve local populations directly in the preservation of public spaces including streets, water reservoirs, drainage structures, and flood risk zones. This leads to the first IE question: 1. What is the additional impact – beyond all other PROGEP activities – of OQP on community engagement and cooperation to maintain clean public spaces? While achieving greater levels of community cooperation and cleanliness is important, it is an intermediate outcome. While we may care about these things in their own right, our interest is primarily in the effect they have on the quality of life of residents living in PROGEP areas. By increasing the overall level of cleanliness in the community, OQP should help the drainage infrastructure to work more effectively, in turn leading to a reduction in the incidence of flooding and to an improvement in various socio-economic and health-related outcomes. Health outcomes include, for example, respiratory infections, malaria, diarrhea, and child mortality. Improvements in these outcomes could, in turn, impact on productivity, education attainment, business activities, and general well-being. This leads to the second IE question: 2. What is the additional impact – beyond all other PROGEP activities – of OQP on attitudes and behaviors, losses and damage to property, and quality of life (including health, income, and education)? 2. Impact evaluation study design 22 The PROGEP IE uses a randomized controlled trial (RCT) study design. Given a list of areas, such a study design assigns the focal intervention to some areas completely at random, while other areas remain without the intervention. With a sufficiently large number of areas over which to randomize, such a procedure ensures that, on average, those areas assigned to receive the intervention (the “treatment” group) are statistically equivalent to those areas not assigned to receive the intervention (the “control” group). In the absence of the intervention, we would expect that outcomes in the treatment and control groups would remain identical, as they are both exposed to the same influences on average. With the intervention, however, we introduce a new factor which only affects the treatment group. We can therefore confidently attribute any differences in outcomes following the treatment to the intervention alone. OQP is a quartier-level intervention implemented by CBOs within each quartier. A three step procedure was used to assign a subset of quartiers/CBOs in PROGEP areas to either the treatment or control group. First, quartiers were grouped according to geographic proximity. This grouping was done by PROGEP’s SFs. The objective of this grouping was to ensure a relatively uniform distribution of treatment and control quartiers across PROGEP areas. The resulting distribution of communes, groups, and quartiers is summarized in Table 3. Table 3: Communes, groups and neighborhoods in the study area City Communes Groups Quartiers Pikine 6 42 349 Guédiawaye 1 5 47 Dakar3 1 1 2 Total 8 48 398 Second, the PROGEP IE research team randomly selected 160 quartiers from the total population of quartiers, stratified by group. The number 160 was informed by budget constraints, which set the number of intervention quartiers at 80. This random selection was done using the statistical analysis software Stata. Within each of the 160 selected neighborhoods, PROGEP’s SFs identified a partner CBO.4 Third, random assignment into treatment and control quartiers was carried out through a public lottery held with representatives of all study area CBOs, stratified by grouping, in which the representatives themselves drew their CBOs treatment status. A public lottery was selected for maximum transparency and to avoid allegations of corruption, collusion, or clientelism to the largest extent possible. While not ideal from a research design perspective (since, for example, control group CBOs would instantly know their status and might alter their behavior in response to this new knowledge), operationally a public 3 One small commune covered by PROGEP is officially in the city of Dakar. 4 CBOs selection was conducted on the basis of capacity to implement OQP and influence within the community. If there was a CBO specifically focusing on flood management within the quartier, it was automatically selected as the focal CBO. This was the case for 3% quartiers. For more information on CBO selection, see Section IV: Intervention Description. 23 lottery was considered essential, especially as the proximity of quartiers and the population density make it highly unlikely that control quartiers would not eventually find out about the intervention regardless (no negative reactions were reported during the lottery). CBO representatives were informed that this was a pilot intervention as a justification for allocation by lottery. The resulting allocation of treatment and control quartiers is illustrated in Table 4. Table 4: Treatment and control quartiers allocation across the PROGEP intervention area Number of Number of control Phase Cities Communes treatment quartiers quartiers Guédiawaye Wakhinane Nimzatt 7 7 Dakar Hann Bel Air 1 1 1 (ENDA- Djeddah Thiaroye Kao 9 9 ECOPOP) Dalifort-foirail 5 5 Pikine Yeumbeul Nord 4 4 Yeumbeul Sud 6 6 Malika 5 5 Keur Massar 25 25 2 (MSA) Pikine Yeumbeul Nord 12 12 Yeumbeul Sud 6 6 TOTAL Phase 1 32 32 TOTAL Phase 2 48 48 TOTAL 80 80 The justification for the process of randomly assigning the OQP intervention to some quartiers and not to others is to create a control group of quartiers that are statistically identical to the treatment group quartiers prior to the intervention, which provides a valid basis for estimating the effects of the intervention. Importantly, this should be true for both the CBOs and for households in the different groups. Balance tests on key variables collected through the baseline survey show that the groups are statistically identical on average, meaning that most observable differences are not distinguishable from zero. See Section VII: OQP Impact Evaluation Results for further information on the balance tests. The experimental study design allows us to estimate the impact of OQP by looking at differences in outcomes in the treatment and control group after the intervention, controlling for other observable variables which are related to the outcomes of interest. This is further discussed in Section VII. 24 VI. Data Sources Data for the PROGEP IE were collected through a household survey and a CBO survey. Baseline data were collected in July 2015, prior to the OQP roll-out, and endline data were collected in November 2016 at the conclusion of the intervention. For both the household and CBO surveys, panel data were collected meaning that the same household or CBO was visited at both baseline and follow-up. 1. Household Survey OQP is designed to improve cleanliness, reduce flooding, and, through these, to improve the quality of life of persons living in PROGEP areas. Quality of life is measured in terms of socioeconomic (e.g. education attendance, employment, income) and health-related outcomes (e.g., respiratory infections, malaria, diarrhea, child mortality). Data on additional related indicators at the individual and household-levels were also captured through the household survey. 15 households per quartier were randomly sampled, providing a total sample of 2,400 households across the 160 study quartiers. The household questionnaire collected information on (i) household demographics, (ii) livelihoods and income sources, (iii) socioeconomic characteristics, (iv) health outcomes, (v) exposure to flooding, (vi) knowledge of flood risk mitigation methods, and (vii) attitudes towards community participation and one’s general responsibilities vis-à-vis the community (and vice-versa), including a “decision activity” section designed to measure willingness to contribute to a public good. Enumerators also recorded their direct observations of the general cleanliness of the immediate area around the household. At endline, an additional set of questions were asked about the OQP to measure household awareness and perception of the intervention. As the endline was collected after the rainy season of 2016, questions related to flooding were also added for 2015 and 2016. Table 5 summarizes the sections included in the baseline and endline surveys. Table 5 : Summary of household questionnaire content Baseline Endline Section 1. Household demographics √ √ 2. Livelihoods and income source √ √ 3. Socioeconomic characteristics √ √ 4. Health outcomes √ √ 5. Exposure to flooding √ √ 6. Knowledge of risk mitigation methods √ √ 7. Attitudes towards community participation √ √ 8. Decision Activity √ √ 9. Enumerator cleanliness appreciation √ √ 10. OQP awareness and perception √ 11. Physical investments √ 12. Community Investment Projects √ 2. CBO Survey 25 A CBO level questionnaire was also administered, both to “treatment” CBOs participating in the social contract intervention and to “control” CBOs. This survey collected data on basic group characteristics, CBO motivations for participating in OQP (treatment group only), attitudes towards civic participation, and the nature of CBO activities. The endline survey differed slightly for treatment and control CBOs, as it aimed to collect data on awareness of control CBOs about OQP and whether or not this influenced their activities, in order for us to get a measure of potential treatment “spillovers”. Additionally, PROGEP’s SFs collected monitoring data on the quartiers’/CBOs mid-term and final evaluation grades as part of the intervention. These data, however, were not collected in a standardized manner across all quartiers, because each SF was responsible for a different area. While these data were critical to inform the implementation of the intervention, the CBO data used in the analysis presented in this report comes from the CBO survey.5 3. Practical Challenges in Survey Implementation A first challenge in data collection resulted from the fluid nature of urban communities in the study area. As mentioned in Section IV (OQP Intervention), quartiers and CBOs are sometimes dynamic units. At the time of the baseline survey, a small subset of pre-identified quartiers (4) and CBOs (16), identified earlier in the study design process, could not be located. Missing quartiers and CBOs were replaced. This issue did not reemerge during the follow-up, suggesting that the “missing” quartiers and CBOs were anomalies. While it is perhaps not so unusual for CBOs to disband, the disappearance of a geographic unit such as a quartier warrants further attention. It should be noted, however, that the list of quartiers was created by PROGEP’s SFs, and so the four missing quartiers are likely due to errors in that process. A second challenge related to locating households and CBOs for the follow-up (or endline) survey. The household and CBO surveys are both panel surveys, which means that the same households and CBOs were surveyed at baseline and follow-up. While identifying CBOs at follow-up was relatively straightforward – particularly for those in the treatment group – identifying households in the baseline sample was more complicated given the absence of a formal system of street names and addresses. At baseline, GPS coordinates for each household were taken in addition to other identifying information. These coordinates were to be the primary means of re-identifying households. These data, however, turned out to be of little use due to the density of the cities of Pikine and Guédiawaye and the lack of precision in GPS measurement (e.g. a precision of 100 meters can include the totality of houses in one quartier). To overcome this challenge, enumerators relied on neighbor networks and baseline “addresses” to locate the correct households. If a household could not be initially located, the enumerator followed a replacement protocol. These measures allowed us to achieve a low attrition rate: out of the 2,400 households surveyed at baseline, only 115 could not be included in the 5 For robustness purposes, the CBO and household cleanliness data was correlated to the external SF one. Results suggest that the survey data is consistent with the grades that were given at the end of the program. Quartiers that were designated as “clean” and that received the final endowments were al so those with the highest cleanliness outcomes at endline. 26 endline survey, an attrition rate of less than 5%. Table 6 summarizes the target and achieved sample sizes for the household and CBO surveys at baseline and endline. Table 6: Surveyed households and CBO Survey Target sample size Achieved at baseline Achieved at follow-up Household 2,400 2,400 2,400* CBO 160 160 160 *Of which 115 are replacement households VII. OQP Impact Evaluation Results Because our treatment was randomly assigned, a direct comparison of outcomes between our treatment and control groups should give us a causal estimate of the impact of the program on those outcomes. Where available, we also control for the baseline value of the outcome of interest. In all of our specifications, we include a dummy for Phase 1 quartiers in order to determine if outcomes in these quartiers are different from outcomes in Phase 2 quartiers6. We also include an interaction term between the Phase 1 dummy and the Treatment dummy to investigate whether or not the impact of treatment was different in the two areas. We only report the specifications with the interaction term where it has a statistically significant effect. 1. Baseline characteristics We begin by describing the characteristics of the households and individuals in our sample at baseline and, in particular, examining the extent to which balance was achieved across treatment and control groups. 6 Phase 1 quartiers benefited from PROGEP drainage infrastructure before OQP roll-out. Phase 2 had yet to receive infrastructure at the end of the intervention. We hence expect there may be differences between the two areas. 27 Table 7 presents the average values for a range of baseline household and individual characteristics for the treatment group and the control group, along with the difference between groups and the p- value of a t-test of whether this difference is statistically significant (p-value < 0.10; this is identifiable by the presence of one or more stars next to the difference indicator). The first set of variables relates to the characteristics of the household head. Approximately 70% of household heads are male and the vast majority (96%) are Muslim. Most are married (around 80%), either in monogamous or polygamous marriages. The average age of household heads is 55 and the average household size is 10 people. For household heads for whom data are available on education and salary, the average education level achieved is grade 5 and the average daily salary in FCFA is around 11,000 (approximately USD 19). There are no statistically significant differences between the mean head of household characteristics across treatment and control households at baseline. The second set of variables relates to the perceptions of households in terms of the cleanliness of the neighborhood, whether they were victims of flooding, and whether they received any training in relation to flood prevention. Around 30% of households rate their neighborhood as clean7, 50% have been a victim of flooding in the past and around 14% have received training on flood prevention. There are no statistically significant differences between the mean of these variables across treatment and control households at baseline. The third set of variables relates to the individuals within the households. There is almost the same amount of males as females in the sample and the average age is 28. The average years of education for those individuals where information is available is 4.5. Of those attending school, the average number of days missed is around a half of a day while for those that work the number of working days missed in the last 30 days is on average 0.29. Of the adults, 14% are members of a CBO. Average salary is approximately 6,100 FCFA per day (approximately USD 10). The only variable where there is a statistically significant difference between treatment and control groups is the number of days of school missed in the last 30 days, which is statistically significant at the 10% level. All of the results presented in what follows are robust to the inclusion of controls for baseline characteristics. 7 Households were asked to rate the cleanliness of their neighborhood on a 5-point scale ranging from ‘Very Clean’ to ‘Very Dirty’. Households were considered ‘Clean’ if they rated their neighborhood as ‘Clean’ or ‘Very Clean’ (i.e., for the analysis, a ‘Clean’ dummy was created which is equal to 1 for these households). 28 Table 7: Baseline household and individual characteristics n Control Treatment Difference p-value Characteristics of household head Male 2,285 0.70 0.68 0.02 0.40 Muslim 2,285 0.96 0.95 0.01 0.44 Married 2,285 0.80 0.78 0.03 0.17 Age 2,285 55.45 55.31 0.15 0.83 Highest level of 1,310 5.02 5.10 -0.08 0.59 education Salary (FCFA per 1,062 12,656.05 9,139.47 3,516.59 0.21 day) Household size 2,285 10.14 10.21 -0.07 0.84 Cleaning and flooding related characteristics Cleanliness of 2,285 0.29 0.30 -0.01 0.70 neighborhood Ever a flood victim 2,285 0.50 0.51 -0.01 0.87 Received training 2,285 0.16 0.13 0.03 0.25 about flood prevention Characteristics of individual household members Male 23,227 0.49 0.49 -0.01 0.42 Age 23,227 28.01 28.21 -0.20 0.43 Highest level of 13,224 4.60 4.52 0.08 0.34 education School missed in 6,129 0.60 0.40 0.20* 0.10 last 30 days Salary (FCFA per 4,864 6,582.08 5,743.04 839.04 0.36 day) Member is part of a 14,896 0.14 0.14 -0.00 0.82 CBO Illness in last 30 22,749 0.22 0.21 0.01 0.45 days Last winter had no 5,489 0.71 0.73 -0.02 0.55 impact on work Number of working 23,227 0.26 0.31 -0.05 0.27 days missed over the last 30 days Standard errors clustered at the quartier level for tests of difference in means. *** p<0.01, ** p<0.05, * p<0.1 29 Table 8 presents descriptive statistics on the characteristics of the CBOs at baseline focusing again on the balance between treatment and control groups. The first set of characteristics relates to the flood prevention and cleaning activities of the CBO. A number of CBOs (approximately 70%) undertook some action in the fight against flooding in the 12 months prior to the baseline survey. Many are also engaged in activities to counteract flooding including raising awareness and providing financial assistance for victims of flooding. Around 19% of CBOs have as their objective raising awareness on flood-related issues or organizing cleaning events. The average number of days per year spent on flood communication or cleaning activities is 12 and approximately 49 members engage in these activities. We achieve balance on almost all of these measures with the exception of whether the CBO engaged in cleaning activities where we find a marginal statistically significant difference in the proportion of CBOs engaged in cleaning activities at baseline between treatment and control. All of the results presented in later sections are robust to the inclusion of these baseline CBO controls. We also present baseline statistics for other characteristics of the CBO. The average number of members is around 177 with around 39 members present at the previous meeting. On average, around 94 of their members engage in the activities of the CBO. On average members engage in CBO activities for 12 hours per week. This suggests that individuals in our study area are indeed very actively engaged with CBOs. The majority of members of CBOs are women (around 60%). The proportion of members that are youths is small at around 5%. Around two thirds of CBOs vote in their leaders by election. There are some differences across the treatment and control groups in these characteristics at baseline. In particular, CBOs in the control group report more active engagement of their members. We also find that the CBOs in treatment quartiers are less likely to vote in their leaders by election. All of our results presented in subsequent sections are robust to the inclusion of baseline controls. The final set of CBO characteristics we consider relates to the engagement of CBOs with CBOs in other quartiers. This is important given the close physical distance between quartiers in our sample and the likelihood that the activities of CBOs in the treatment area might affect the activities of CBOs in the control areas, potentially contaminating our findings. We find that while there is some degree of collaboration between CBOs within the quartier and with CBOs in neighboring quartiers, there is no statistically significant difference between the treatment and the control groups. 30 Table 8: Baseline CBO characteristics Control Treatment Difference p-value CBO Flood prevention and cleaning activities CBO has taken action in relation 0.72 0.66 0.06 0.42 to flooding in last year Flood activities: raising 0.30 0.30 0.00 1.00 awareness Flood activities: financial 0.28 0.20 0.08 0.27 assistance for victims Flood activities: cleaning of 0.11 0.11 0.00 1.00 canals and lakes Flood activities: surveillance of 0.04 0.05 -0.01 0.70 canals and lakes Flood activities: cooperation 0.25 0.28 -0.03 0.72 with other stakeholders Flood activities: small works to 0.15 0.14 0.01 0.82 avoid flood Objective of CBO: raise 0.19 0.19 0.00 1.00 awareness on flood-related issues/organize cleaning activities Number of days spent on flood 10.32 14.30 -3.98 0.38 communication or cleaning activities Number of members engaged 59.38 38.76 20.61 0.17 in flood communication or cleaning activities CBO engaged in flood 0.24 0.20 0.04 0.57 communication campaign CBO engaged in cleaning 0.64 0.50 0.14* 0.08 activities CBO Characteristics Number of members 200.01 155.22 44.79 0.54 Number of members present at 38.48 39.67 -1.20 0.83 the last meeting Number of members engaged 117.55 71.36 46.19* 0.07 in activities of CBO Average hours a week members 11.93 12.56 -0.64 0.83 engage in CBO activities Proportion of members that are 0.59 0.62 -0.02 0.63 women Proportion of members that are 0.06 0.04 0.01 0.46 young CBO is the head quarters 0.85 0.95 -0.10** 0.04 CBO votes in leaders by election 0.69 0.53 0.16** 0.04 Most important benefit of 0.14 0.07 0.06 0.20 membership of CBO are benefits for the community Collaboration between different CBOs CBO only intervenes in this 0.65 0.65 0.00 1.00 neighborhood 31 CBO collaborates with other 0.65 0.71 -0.06 0.40 CBOs in the quartier CBO collaborates with other 0.69 0.63 0.06 0.41 CBOs in other quartiers N=160 *** p<0.01, ** p<0.05, * p<0.1 2. Impact on CBO behavior Since the OQP program was implemented via the CBOs, we start by looking at the impact that treatment had on their behavior. The key outcome that we are interested in is whether or not the CBO carried out cleaning events as part of its activities. The results for this outcome are presented in Figure 5. We can see that CBOs in treatment quartiers are 72% (26 percentage points) more likely to undertake cleaning events as part of their activities at endline compared with control quartiers. The results for this regression are presented in Table 3. Comparing to the baseline in Figure 5 we can see that the difference at endline is in part due to an increase in the proportion of CBOs in treatment quartiers that engage in cleaning activities but is also due to a large decline in the proportion of CBOs in control quartiers engaged in cleaning events. We do not find a significant difference in treatment effect between Phase 1 and Phase 2 areas (these results are not presented). Figure 5: CBOs that list cleaning activities among top 3 activities 70% 60% 50% 40% 30% *** 20% 10% 0% Baseline Endline Control Treatment Impact Note: *** p<0.01, ** p<0.05, * p<0.1 Controls for full range of CBO characteristics described in 32 Table 8. None of the other outcomes measured at the CBO level showed a statistically significant difference between treatment and control quartiers and so these results are not presented but are available from the authors on request. Table 9: CBOs that list cleaning activities among top 3 activities VARIABLES Cleaning events Treatment 0.263*** (0.0836) Phase 1 0.00261 (0.0880) Constant 0.507** (0.206) Observations 159 R-squared 0.296 Standard errors clustered at the CBO level in parentheses *** p<0.01, ** p<0.05, * p<0.1 Includes controls for full range of CBO characteristics described in 33 Table 8. 3. Impact on household behavior and attitudes In the next set of figures and tables we consider the impact of treatment on household-level variables. The only controls included in these regressions are baseline values of closely related variables. The results are robust to the inclusion of other CBO and household level controls. In all cases, tests of statistical significance are based on standard errors that are clustered at the quartier level. Figure 6 presents the results for a regression investigating the impact of the OQP program on whether or not households have heard of the OQP. The quartiers selected for OQP were chosen via public lottery so we would expect that some households in control areas would have also heard of the OQP. On the other hand, since not all households are members of CBOs, we can also expect some households in treatment quartiers to not be aware of OQP. However, we would be concerned if there was no difference between control and treatment areas in terms of their awareness of the program as this would suggest either that it was not implemented effectively or that spillovers were very high. We do find a statistically significant difference in the awareness of the program between treated and control households. Treated households were almost 50% (17 percentage points) more likely to have heard of OQP than control households. Figure 6: Household Awareness of OQP 60% 50% 40% 30% 20% *** 10% 0% Control Treatment Impact In Figure 7, we consider the difference between Phase 1 and Phase 2 quartiers and their knowledge of the OQP. The results for Phase 1 are presented on the left and those for Phase 2 on the right. We find that, overall, households in Phase 1 areas are generally more likely to have heard of the program. The interaction effect between whether or not the household is in Phase 1 or Phase 2 demonstrates, 34 however, that the treatment had a different effect on household awareness in quartiers in the two different phases. The effect for Phase 2 quartiers is stronger as treated households are almost 80% (23 percentage points) more likely to have heard of OQP than control households. The effect of the treatment was smaller in Phase 1 areas and is not statistically significant, although it is still the case that in general households in Phase 1 areas are more likely to be aware of OQP. Figure 7: Household Awareness of OQP by phase 60% 60% 50% 50% 40% 40% 30% 30% ** 20% 20% 10% 10% 0% 0% Phase 1 Phase 2 Control Treatment Impact Control Treatment Impact Households who reported that they were aware of OQP were then asked what they thought would be the most likely effect of the program. The results without the interaction term are presented in Figure 8 while the results including the interaction term are presented in Figure 9. Households in the treatment area were more likely to say that they believed OQP would lead to areas being cleaner. Once again, when the interaction term is included we can see that these results are being driven by Phase 2 areas. Treated households in Phase 2 areas are 52% (16 percentage points) more likely than control areas to report that OQP will result in the area being cleaner. There was no effect of the treatment on this outcome in Phase 1 areas. The detailed results of these regressions are also presented in Table 10. Column 1 presents the average effect of treatment on whether or not households had heard of OQP and Column 2 introduces the interaction term to investigate the heterogeneous effect of treatment in the two phases. The results related to households’ opinion of the effect of OQP are presented in columns 3-6. It does not seem to be the case that treated households were more likely than control households to report that the program would reduce flooding. When the interaction term is included in column 6, the coefficient on the treatment indicator is positive but it is not statistically significant. However, just over 20% of households in both treatment and control areas believed that the program would reduce flooding. 35 Figure 8: Households stating that they believed OQP would improve general cleanliness 45% 40% 35% 30% 25% 20% 15% 10% * 5% 0% Control Treatment Impact Figure 9: Households stating that they believed OQP would improve general cleanliness, by Phase 50% 50% 45% 45% 40% 40% 35% 35% 30% 30% 25% 25% 20% 20% 15% 15% * 10% 10% 5% 5% 0% 0% Phase 1 Phase 2 Control Treatment Impact Control Treatment Impact 36 Table 10: Household Awareness of OQP and opinion of impact (1) (2) (3) (4) (5) (6) Heard of Heard of VARIABLES OQP OQP AREA CLEANER AREA CLEANER REDUCED FLOODING REDUCED FLOODING Treatment 0.168*** 0.233*** 0.0892** 0.159** -0.00185 0.0482 (0.0394) (0.0461) (0.0446) (0.0640) (0.0409) (0.0508) Phase 1 0.0878** 0.170*** -0.0421 0.0458 0.109** 0.172*** (0.0419) (0.0538) (0.0446) (0.0629) (0.0425) (0.0565) Phase 1*Treatment -0.165** -0.150* -0.107 (0.0826) (0.0870) (0.0825) Constant 0.319*** 0.286*** 0.351*** 0.306*** 0.217*** 0.184*** (0.0276) (0.0279) (0.0403) (0.0465) (0.0342) (0.0380) Observations 2,400 2,400 1,051 1,051 1,051 1,051 R-squared 0.036 0.043 0.011 0.016 0.015 0.019 Robust standard errors clustered at the quartier level in parentheses *** p<0.01, ** p<0.05, * p<0.1 37 Figure 10 shows the estimated impact of being in a treatment area on the probability of receiving training related to managing the risks of flooding. There is a positive and significant effect of the treatment. Treated households are 40% (4 percentage points) more likely to have received training than control households. Figure 10: Households that received training in last 12 months 18% 16% 14% 12% 10% 8% 6% 4% ** 2% 0% Baseline Endline Control Treatment Impact Figure 11 presents the results related to household’s perception of the cleanliness of their neighborhood. Households were asked to rate the cleanliness of their neighborhood on a 5-point scale ranging from ‘Very Clean’ to ‘Very Dirty’. A ‘Clean’ dummy was created which is equal to 1 if the household rated their neighborhood as ‘Clean’ or ‘Very Clean’. Households in treatment areas are 19% (7 percentage points) more likely give their neighborhood one of the higher cleanliness ratings than households in control areas and this difference is statistically significant. 38 Figure 11: Household perception of quartier cleanliness 50% 45% 40% 35% 30% 25% 20% 15% 10% ** 5% 0% Baseline Endline Control Treatment Impact Households who responded that they believed their areas were not clean were then asked who they believed was responsible for this lack of cleanliness. A dummy was created equal to 1 if their response indicated a group or individual from within the neighborhood rather than an external body. The results for this outcome variable are reported in Figure 12. Households in the treatment area are 10% (7 percentage points) less likely to say that groups from within the neighborhood are responsible for the lack of cleanliness than households in treatment areas. The latter may suggest that households that are more aware of their own responsibility in ensuring their community’s cleanliness, and are therefore also perhaps more engaged in the upkeep of public spaces, and therefore recognize the limits of their own actions. It could, however, also mean that households that are more sensitized to the importance of maintaining public cleanliness are more likely to blame others when this is not the case. There was no differential effect of treatment between Phase 1 and Phase 2 areas for any of these outcomes. Table 11 provides additional detail of the regression analysis on household behaviors and attitudes. 39 Figure 12: Neighborhood is responsible for lack of cleanliness 80% 70% 60% 50% 40% 30% 20% 10% 0% -10% ** -20% Baseline Endline Control Treatment Impact Table 11: Training and impression of cleanliness of neighborhood (1) (2) (3) VARIABLES Training Clean Responsible Treatment 0.0421** 0.0729** -0.0727** (0.0189) (0.0321) (0.0357) Phase 1 0.0541** -0.0689** -0.0497 (0.0213) (0.0323) (0.0357) Benefited from training in baseline 0.0361 (0.0219) Clean Baseline 0.156*** -0.00501 (0.0296) (0.0331) Constant 0.0717*** 0.340*** 0.729*** (0.0127) (0.0261) (0.0270) Observations 2,285 2,285 1,381 R-squared 0.012 0.034 0.009 Robust standard errors clustered at the quartier level in parentheses *** p<0.01, ** p<0.05, * p<0.1 40 4. Impact on household outcomes related to flooding Households were asked questions about their experience of flooding in the most recent rainy season (June to September 2016) and in the previous season (June to September 2015). Both of these seasons were during the period of time covered by the OQP intervention, although we might expect any effects to be stronger in the most recent season as the OQP had been running for longer. The results for the impact of flooding in the most recent year are presented in Figure 13 to Figure 16. In Figure 13 we observe that households in the treated quartiers are 20% (2 percentage points) less likely to be victims of flooding. Similarly, in Figure 14 we also observe that households in treatment quartiers are less likely to report that their house was flooded in the previous 12 months. This difference, however, is not statistically significant and is quite small in magnitude. Figure 13: Probability of being a flood victim in the last 12 months 25% 20% 15% 10% 5% 0% -5% Baseline Endline Control Treatment Impact 41 Figure 14: Probability of one’s house being flooded in the last 12 months 16% 14% 12% 10% 8% 6% 4% 2% 0% -2% Baseline Endline Control Treatment Impact Figure 15 considers an interaction term between the Phase 1 indicator and treatment for the probability of being a flood victim in the last rainy season. When the interaction term is included the magnitude of the coefficient on the treatment dummy increases and becomes statistically significant. This suggests that for treatment households in Phase 2 areas, OQP reduced the probability of experiencing flooding in the most recent year by 38% (3.4 percentage points) In contrast, there is no statistically significant impact of treatment on the incidence of flooding in Phase 1. The results for the probability of one’s house being flooded broken down by phase are presented in Figure 16 and display a similar pattern. OQP reduced the probability of one’s house being flooded by 50% (2 percentage points) in Phase 2 areas. The fact that households in the OQP treated quartiers in Phase 2 areas experienced a reduction in flooding is notable, particularly given the possibility of flooding ‘spillovers’ between quartiers. Given that most quartiers are in close proximity to each other, it is possible that flooding in an untreated quartier where there was no OQP could lead to flooding in a treated quartier with an OQP that engaged in flood prevention activities. 42 Figure 15: Impact of treatment on probability of being a flood victim by phase 15% 15% 10% 10% 5% 5% 0% 0% -5% -5% * Phase 1 Phase 2 Control Treatment Impact Control Treatment Impact Figure 16: Impact of treatment on probability of one’s house being flooded by phase 8% 8% 6% 6% 4% 4% 2% 2% 0% 0% -2% -2% * -4% -4% Phase 1 Phase 2 Control Treatment Impact Control Treatment Impact Table 12 provides additional detail of the regression analysis on households’ experience of flooding. This table also includes results for regressions relating to whether the household experienced damage to goods due to flooding. These results are not statistically significant even with the inclusion of the Phase 1 interaction term, although they are negative and of similar magnitude to the other results. The results for the same analysis for flooding in the previous year follow a similar pattern. They are presented in Table 13. 43 Table 12: Vulnerability to Flooding Most Recent Year (1) (2) (3) (4) (5) (6) VARIABLES Flood victim Flood victim Damaged goods Damaged goods House under water House under water Treatment -0.0161 -0.0335* -0.00602 -0.0133 -0.00719 -0.0194* (0.0156) (0.0179) (0.00823) (0.0101) (0.00972) (0.00989) Phase 1 0.0393** 0.0178 0.00836 -0.000590 0.0233** 0.00817 (0.0163) (0.0257) (0.00839) (0.0127) (0.0106) (0.0140) Phase 1*Treatment 0.0431 0.0179 0.0302 (0.0327) (0.0169) (0.0213) Flood victim baseline 0.0258 0.0261 0.0173 0.0174 -0.00292 -0.00272 (0.0206) (0.0204) (0.0119) (0.0119) (0.0121) (0.0120) Flood victim last year baseline 0.0216 0.0209 0.00232 0.00205 0.0442*** 0.0437*** (0.0218) (0.0217) (0.0138) (0.0138) (0.0143) (0.0142) Flood victim present year baseline 0.0666*** 0.0676*** 0.0211 0.0215 0.0297* 0.0303** (0.0224) (0.0222) (0.0141) (0.0141) (0.0152) (0.0151) Constant 0.0448*** 0.0534*** 0.0151** 0.0187** 0.0114 0.0174** (0.0132) (0.0141) (0.00666) (0.00747) (0.00834) (0.00858) Observations 2,285 2,285 2,285 2,285 2,285 2,285 R-squared 0.030 0.032 0.010 0.011 0.026 0.027 Robust standard errors clustered at the quartier level in parentheses *** p<0.01, ** p<0.05, * p<0.1 44 Table 13: Vulnerability to Flooding Previous Year (1) (2) (3) (4) (5) (6) VARIABLES Flood victim Flood victim Damaged goods Damaged goods House under water House under water Treatment -0.0192 -0.0213 -0.0198 -0.0263 -0.00823 -0.0256 (0.0208) (0.0232) (0.0163) (0.0168) (0.0158) (0.0175) Phase 1 0.102*** 0.0995*** 0.0611*** 0.0531** 0.0557*** 0.0341 (0.0225) (0.0338) (0.0179) (0.0245) (0.0171) (0.0250) Phase 1*Treatment 0.00518 0.0160 0.0431 (0.0445) (0.0355) (0.0336) Flood victim baseline 0.0761*** 0.0762*** 0.0349 0.0350 0.0404* 0.0407* (0.0283) (0.0283) (0.0223) (0.0223) (0.0227) (0.0226) Flood victim last year baseline 0.0700** 0.0699** 0.0597*** 0.0594*** 0.0771*** 0.0764*** (0.0270) (0.0270) (0.0222) (0.0221) (0.0222) (0.0221) Flood victim present year baseline 0.0834*** 0.0835*** 0.0580** 0.0584** 0.0386 0.0395 (0.0309) (0.0308) (0.0262) (0.0261) (0.0260) (0.0259) Constant 0.0714*** 0.0724*** 0.0399*** 0.0431*** 0.0245** 0.0332** (0.0157) (0.0167) (0.0120) (0.0123) (0.0123) (0.0138) Observations 2,285 2,285 2,285 2,285 2,285 2,285 R-squared 0.077 0.077 0.050 0.051 0.057 0.058 Robust standard errors clustered at the quartier level in parentheses *** p<0.01, ** p<0.05, * p<0.1 45 Since we have a number of different measures for vulnerability to flooding, indices were created to combine these outcomes and get one overall measure of the experience of flooding. The index is constructed by averaging across the measures already presented (including also damage to household assets, where we do not find impacts when looking at this indicator alone), giving each measure equal weight. This should give an indication of the intensity of the household’s experience of flooding as it will have a higher value if their house was under water and goods were damaged due to flooding than if they just experienced one of these events. A separate index was created for each year and then a combined index was created that accounts for the experience of flooding in both years. The results for these indices are presented in Figure 17 to Figure 19. These results support the findings in the previous Figures. OQP reduced the vulnerability of flooding for households in treatment quartiers located in the Phase 2 area but it does not seem to have had a similar effect in the Phase 1 quartiers. The results also suggest that quartiers in the Phase 1 area are in general more vulnerable to flooding. Figure 17: Flooding index in year prior to survey 0.25 0.20 0.15 0.10 0.05 0.00 -0.05 Baseline Endline Control Treatment Impact 46 Figure 18: Flooding index in year prior to endline survey by phase 0.09 0.09 0.07 0.07 0.05 0.05 0.03 0.03 0.01 0.01 -0.01 -0.01 -0.03 -0.03 * Phase 1 Phase 2 Control Treatment Impact Control Treatment Impact Figure 19: Flooding index at endline over both years, by phase 0.14 0.14 0.12 0.12 0.10 0.10 0.08 0.08 0.06 0.06 0.04 0.04 0.02 0.02 0.00 0.00 -0.02 -0.02 * -0.04 -0.04 Phase 1 Phase 2 Control Treatment Impact Control Treatment Impact Table 14 provides additional detail of the regression analysis on the flooding indices. 47 Table 14: Flooding index (1) (2) (3) (4) (5) (6) Flooding index Flooding index Flooding index Flooding index Flooding index Flooding index VARIABLES This year This year Last year Last year Both years Both years Treatment -0.00977 -0.0221* -0.0158 -0.0244 -0.0128 -0.0232* (0.00984) (0.0115) (0.0159) (0.0173) (0.0119) (0.0134) Phase 1 0.0237** 0.00846 0.0729*** 0.0622** 0.0483*** 0.0353* (0.0102) (0.0153) (0.0173) (0.0250) (0.0126) (0.0188) Phase 1*Treatment 0.0304 0.0214 0.0259 (0.0206) (0.0341) (0.0250) Flood victim baseline 0.0134 0.0136 0.0505** 0.0506** 0.0319** 0.0321** (0.0119) (0.0118) (0.0220) (0.0220) (0.0147) (0.0146) Flood victim last year baseline 0.0227* 0.0222* 0.0689*** 0.0686*** 0.0458*** 0.0454*** (0.0131) (0.0130) (0.0210) (0.0210) (0.0147) (0.0146) Flood victim present year baseline 0.0391** 0.0398*** 0.0600** 0.0605** 0.0496*** 0.0501*** (0.0152) (0.0152) (0.0258) (0.0257) (0.0184) (0.0183) Constant 0.0238*** 0.0298*** 0.0453*** 0.0496*** 0.0345*** 0.0397*** (0.00840) (0.00907) (0.0121) (0.0132) (0.00957) (0.0105) Observations 2,285 2,285 2,285 2,285 2,285 2,285 R-squared 0.030 0.032 0.077 0.077 0.071 0.072 Robust standard errors clustered at the quartier level in parentheses *** p<0.01, ** p<0.05, * p<0.1 48 5. Impact on individual outcomes related to flooding We also consider the impact of OQP on individual level outcomes. One outcome of particular relevance is exposure to illness. It is expected that increased cleanliness and reduced flooding should lead to improved health. We do not find a statistically significant impact of the OQP treatment on individuals’ experience of illness in general. While all of the estimates of the impact of treatment are negative, they are not statistically significant and are all close to zero. From these results, we cannot say that OQP had any impact on the incidence of illness of individuals in treatment quartiers. These results are presented in Table 17 of the Appendix. However, when individuals are specifically asked about whether or not their illness was related to flooding, as can be seen in Figure 20, individuals in treatment areas are 11% (5 percentage points) less likely to say that their illness in the second-to-last rainy season (June to September 2015) was due to flooding than individuals in control areas. It should be noted, however, that there was no statistically significant impact on reported illness due to flooding in the most recent rainy season (June to September 2016). Figure 20: Illness due to flooding in the second-to-last rainy season 50% 40% 30% 20% 10% 0% * -10% Control Treatment Impact Table 15 provides additional detail of the regression analysis related to illness due to flooding. 49 Table 15: Prevalence of illness due to Flooding (1) (2) Illness due to Illness due to flooding flooding VARIABLES this year last year Treatment -0.0184 -0.0557* (0.0224) (0.0314) Phase 1 -0.0500** -0.0236 (0.0225) (0.0316) Illness baseline -0.0726*** -0.139*** (0.0159) (0.0221) Constant 0.333*** 0.496*** (0.0215) (0.0277) Observations 5,369 3,458 R-squared 0.010 0.021 Robust standard errors clustered at the quartier level in parentheses *** p<0.01, ** p<0.05, * p<0.1 The impact of the OQP treatment on the effect of flooding on work related outcomes are presented in Figure 21. The average result across both phases is negative but is not statistically significant. However, as shown in Figure 22, the results once again suggest that OQP had an impact on treatment quartiers located in the Phase 2 area but not in Phase 1. OQP reduced the amount of income lost due to flooding in Phase 2 areas by 47%. The estimate for the impact of OQP is also negative in Phase 1 areas but it is not statistically significant. Figure 21: Income lost due to flooding in the second-to-last rainy season 5000 4000 3000 2000 1000 0 -1000 -2000 -3000 Control Treatment Impact 50 Figure 22: Income lost due to flooding in the second-to-last rainy season by phase 7000 7000 6000 6000 5000 5000 4000 4000 3000 3000 2000 2000 1000 1000 0 0 -1000 -1000 -2000 -2000 -3000 -3000 * -4000 -4000 Phase 1 Phase 2 Control Treatment Impact Control Treatment Impact Table 16 provides additional detail of the regression analysis for outcomes related to work. The outcome reported on in columns 1 and 2 is the number of work days missed in the previous 30 days. While the coefficients for the treatment dummy are negative, they are small and not statistically significant. This is perhaps not surprising since we would only expect the treatment to affect this variable through the impact on flooding and the 30 days prior to the survey coincided with the end of the rainy season when rains were at most moderate. Columns 3 to 6 present results for income lost due to flooding in the most recent year and the previous year. The coefficients on the treatment dummy are negative in all regressions and again become stronger once the interaction term is included, suggesting a reduction in the amount of income lost due to flooding. The result is only statistically significant for the previous year, although the estimate for the most recent year is of similar magnitude. It is important to note that most respondents (97.5%) reported no income lost for these periods and so these results are being driven by a small proportion of individuals and further analysis is ongoing. However, the results are interesting and consistent with the results related to vulnerability of flooding.8 8 We also consider the impact of treatment on education outcomes but the effects were very small in magnitude and were not statistically significant. This is perhaps not surprising as most flooding happens outside of the school year. The results are presented in Table 18 of the Appendix. 51 Table 16: Impact on Work (1) (2) (3) (4) (5) (6) Work days missed Work days missed Income lost Income lost Income lost Income lost VARIABLES last 30 days last 30 days this year this year last year last year Treatment -0.0534 -0.590 -1,676 -2,108 -2,098 -2,680* (0.316) (0.455) (1,388) (1,588) (1,414) (1,568) Phase 1 0.233 -0.368 335.6 -208.7 1,753 1,021 (0.312) (0.470) (1,508) (2,781) (1,552) (2,715) Phase 1*Treatment 1.217** 1,099 1,478 (0.616) (2,997) (3,089) Work days missed baseline 0.0514 0.0528 (0.0351) (0.0346) Constant 2.426*** 2.688*** 3,682*** 3,900*** 3,386** 3,678** (0.294) (0.343) (1,287) (1,471) (1,319) (1,506) Observations 5,296 5,296 4,905 4,905 4,951 4,951 R-squared 0.001 0.003 0.000 0.000 0.001 0.001 Robust standard errors clustered at the quartier level in parentheses *** p<0.01, ** p<0.05, * p<0.1 52 VIII. Conclusion The OQP impact evaluation was designed to test whether this specific intervention, over and above all other actions by PROGEP and others in PROGEP areas, could increase community engagement in the general upkeep of public areas and thus improve the quality of life of residents in OQP neighborhoods. This was a light-touch, bottom up intervention: OQP engaged existing community- based organizations and empowered them to use their local knowledge and networks to work towards improved community cleanliness, while providing minimal guidance and only basic materials. Importantly, however assessment of neighborhoods to see if they would qualify for the “Clean Neighborhood” designation following the intervention, and the related recognition and incentives, was carried out by external parties on the basis of pre-defined criteria. This was an important element of transparency, to guard as much as possible against the allegations by non-qualifying quartiers of favoritism or corruption. Results from the IE show that OQP, which engaged established community-based groups with a non- monetary incentives-linked “social contract”, can be effective at improving community engagement for the upkeep of public spaces. Furthermore, this increase in community engagement led to changes in the quality of life of residents living in OQP areas, even after just one year of intervention. In particular, households in OQP areas were found to be less affected by flooding relative to households in control areas. They also reported reduced levels of illness due to flooding and of income lost due to flooding (although the latter result is driven by a small number of study participants, and so should be interpreted with caution pending further analysis). We find, furthermore, that the impact of OQP on quality of life is more pronounced in PROGEP Phase 2 areas, where the infrastructural component of the project had not yet been implemented. This suggests that an OQP-type intervention could be considered as an interim substitute while waiting for such investments to catch up with the needs of communities living in peri-urban areas that are at risk of flooding. It should be noted, however, that Phase 1 areas were found to be more prone to flooding overall, according to results from the household survey. Thus, OQP can be effective in preventing flooding, even in the absence of infrastructure, but perhaps more so in areas that are relatively less prone to flooding to begin with. This is logical: as flood-risk increases, drainage infrastructure becomes increasingly important. The clear lesson that emerges, however, is that flood risk mitigation should go hand-in-hand with waste management. More broadly, the OQP impact evaluation provides lessons for other projects and initiatives aiming to engage communities in the upkeep or provision of public goods. First, the intervention relies almost wholly on local knowledge, and gives participating CBOs full autonomy in determining the types of activities to implement. Second, CBO rewards are not based exclusively on the activities they conduct (this is only one of six evaluation criteria), but primarily on an external assessment of the cleanliness of their neighborhood. This is therefore an example of a “results-based” intervention at the very local level. Third, such light-touch, non-interventionist approaches can be effective at shifting community- level behaviors, which is fundamental to achieving sustainability and returns on other types of community investments (drainage infrastructure in the PROGEP case, but perhaps also roads, schools, health facilities, parks, etc.). The challenge, now, is to understand how to maintain the engagement and positive results achieved after the first year of OQP. This is a question for future studies. 53 Appendix 1: Additional results tables Table 17: Prevalence of Illness (1) (2) (3) Illness Illness Illness This rainy Last rainy VARIABLES Last 30 days season season Treatment -0.00970 -0.0120 -5.53e-05 (0.0165) (0.0203) (0.0228) Phase 1 0.0339** 0.0325 0.0348 (0.0169) (0.0211) (0.0240) Constant 0.196*** 0.235*** 0.143*** (0.0146) (0.0173) (0.0177) Observations 20,526 20,536 20,391 R-squared 0.019 0.019 0.010 Robust standard errors clustered at the quartier level in parentheses *** p<0.01, ** p<0.05, * p<0.1 Includes controls for reporting having an illness at baseline. 54 Table 18: Impact on education outcomes (1) (2) (3) (4) School year School year School days School days VARIABLES begin on time begin on time missed missed Treatment -0.00171 0.00248 0.0219 -0.0149 (0.00550) (0.00502) (0.0953) (0.134) Phase 1 -0.0210*** -0.0160* -0.0715 -0.116 (0.00620) (0.00881) (0.0928) (0.122) Phase 1*Treatment -0.0103 0.0901 (0.0124) (0.187) Constant 0.990*** 0.988*** 0.360*** 0.377*** (0.00399) (0.00402) (0.0805) (0.0915) Observations 7,240 7,240 4,214 4,214 R-squared 0.006 0.006 0.001 0.001 Robust standard errors clustered at the quartier level in parentheses *** p<0.01, ** p<0.05, * p<0.1 Includes controls for outcomes at baseline. 55