79903 Indonesia: Evaluation of the Urban Community Driven Development Program Program Nasional Pemberdayaan Masyarakat Mandiri Perkotaan (PNPM-Urban) POLICY NOTE JANUARY 2013 POLICY NOTE Indonesia: Evaluation of the Urban Community Driven Development Program Program Nasional Pemberdayaan Masyarakat Mandiri Perkotaan (PNPM-Urban) January 2013 Abbreviations and Acronyms Abbreviations Bahasa Indonesia English ASKESKIN Asuransi Kesehatan Masyarakat Miskin Health insurance for the poor Bappenas Badan Perencanaan Pembangunan Nasional National Development Planning Body BIA Analisa Insiden Manfaat Benefit incidence analysis BLM Bantuan Langsung Masyarakat Community Block Grants BLT Bantuan Langsung Tunai Unconditional cash transfer BOP Biaya Operasional Operational Funds BPS Badan Pusat Statistik Statistics Indonesia BSM Beasiswa untuk Siswa Miskin Scholarships for the poor CCT Bantuan Sosial Tunai Bersyarat Conditional Cash Transfer CDP Program Jangka Menengah Program Community Development Plan Penanggulangan Kemiskinan (PJM Pronangkis) GDP Produk Domestik Bruto (PDB) Gross Domestic Product GFC Krisis Keuangan Dunia (mulai musim gugur 2008) Global Financial Crisis (starting Fall 2008) GOI Pemerintah Republik Indonesia Government of Indonesia Jamkesda Jaminan Kesehatan Daerah Local level health insurance for the poor Jamkesmas Jaminan Kesehatan Masyarakat Health Insurance Scheme for the Population JPS Jaring Pengaman Sosial Social Safety Net JSLU Jaminan Sosial Lanjut Usia Social cash transfer for the elderly JSPACA Jaminan Social Penyandang Cacat Berat Social cash transfer for the disabled Kemendiknas Kementerian Pendidikan Nasional Ministry of National Education, MONE Kemenag Kementerian Agama Ministry of Religious Affairs, MORA Kemenkes Kementerian Kesehatan Ministry of Finance, MOF Kemensos Kementerian Sosial Ministry of Social Affairs, MOSA ND Pembangunan Lingkungan Permukiman Berbasis Neighborhood Development Komunitas (PLBK) NTS Sistem Penargetan Nasional National Targeting System OPK Operasi Pasar Khusus Program for sale of subsidized rice for the poor PAPG Program Penanggulangan Kemiskinan Terpadu Poverty Alleviation Partnership Grant (PAPG) (PAKET) atau Program Penanggulangan Kemiskinan berbasis Kemitraan PK Permukiman Kelurahan Kelurahan Settlement P2KP Proyek Penanggulangan Kemiskinan di Perkotaan Urban Poverty Project (UPP) PKH Program Keluarga Harapan Hopeful Family Program PL Garis Kemiskinan Poverty Line PNPM-Mandiri Program Nasional Pemberdayaan Masyarakat National Community Empowerment Mandiri Program PPP Keseimbangan Daya Beli Purchasing power parity ii Policy Note Abbreviations Bahasa Indonesia English Raskin Beras Miskin Program for sale of subsidized rice for the poor Rp Rupiah Indonesian Rupiah SA Bantuan Sosial Social Assistance SD Sekolah Dasar Elementary School SMP Sekolah Menengah Pertama Junior Secondary School SP Perlindungan Sosial Social Protection SSN Jaringan Pengaman Sosial Social Safety Net SUSENAS Survei Sosio-Ekonomi Nasional National Socio-Economic Survey TKPKD Tim Koordinasi Penanggulangan Kemiskinan Regional Poverty Reduction Coordinating Team Daerah TNP2K Tim Nasional Percepatan Penanggulangan National team for accelerating poverty reduction Kemiskinan UCT Transfer Uang Tunai tak Bersyarat Unconditional Cash Transfer UPP Proyek Penanggulangan Kemiskinan Perkotaan/ Urban Poverty Project (P2KP) P2KP Indonesia: Evaluation of the Urban Community Driven Development Program iii Table of Contents Abbreviations and Acronyms..................................................................................................................................................................ii Acknowledgments.........................................................................................................................................................................................v Executive Summary.......................................................................................................................................................................................vi Section 1: Introduction and Approach.......................................................................................................................................1 1.1 Introduction............................................................................................................................................................................................1 1.2 Background and Context...............................................................................................................................................................2 1.3 Study Approach...................................................................................................................................................................................4 Section 2: Main Evaluation Findings............................................................................................................................................9 2.1 Assessing the CDD Approach. .....................................................................................................................................................9 2.2 Links with Local Government......................................................................................................................................................12 2.3 Capacity Building and Effectiveness of Facilitators........................................................................................................13 2.4 Control Mechanisms and Governance..................................................................................................................................14 2.5 Quality of infrastructure..................................................................................................................................................................15 2.6 The Adequacy of Block Grants and the PAPG/Neighborhood Development Scheme Approach..............................................................................................................................................................................16 Section 3: Discussion and recommendations.......................................................................................................................19 ...........................................................................................................................................................................................................23 References. iv Policy Note Acknowledgments This study was carried out by a team led by Judy L. Baker, Lead Economist, World Bank. The policy note is based on two commissioned studies: i) “A Qualitative Study of the PNPM Urban in Indonesia” prepared by a team at the RAND Corporation including Nicholas Burger, Peter Glick, Francisco Perez-Arce, Lila Rabinovich and Joanne Yoong. The management of the field work was carried out by SurveyMeter, notably Bondan Sikoki, Wayan Suriastini and Dani Alfah; and ii) “Rapid Appraisal of PNPM Neighborhood Development (and Poverty Alleviation Partnership Grant Mechanism” prepared by Nina Schuler and Risye Dwiyani. The work benefitted greatly from inputs by Evi Hermissari, Kumala Sari, Fatima Shah (co-TTL through Sep- tember, 2011), George Soraya, Zoe Trohanis, and Jan Weetjens. The team is grateful to peer reviewers, Daniel Hoornweg, Jon Jellema, Victoria Beard, and Suahasil Nazara, as well as helpful inputs and discussions at vari- ous stages of the work from Vivi Alatas, Ana Maria Arriagada, Amri Ilmma, Yulia Immajati, Victor Vergara, Matt Waipoi, and Michael Woolcock. The work was carried out under the overall direction of Franz Drees-Gross and Stefan Koeberle. Participants at two workshops held in Jakarta, one in March, 2011 and a second in November 2011, includ- ing key members of GOI, Indonesian researchers, NGOs, and donors provided invaluable input on the study design and on early findings. Funding for this study was made possible through the PNPM Support Facility (PSF).     Indonesia: Evaluation of the Urban Community Driven Development Program v Executive Summary The PNPM-Urban, a community driven development program which delivers block grants to the urban poor for small scale infrastructure and to a lesser extent, small projects for social and economic development, cur- rently serves all urban areas in Indonesia with the aims of fostering community participation, improving local governance, and delivering basic needs at the community level. The program is a core part of the country’s poverty reduction strategy as well as national urban strategy which is currently being prepared. This policy note draws on two qualitative field studies that were carried out to assess how specific aspects of the program are working, particularly for the infrastructure component, and to document good practice, distill lessons learned and identify options for program reform.1 A second policy note also prepared by the World Bank, Indonesia Urban Poverty and Program Review, provides analysis of urban poverty and a broader look at the country’s poverty reduction programs of which PNPM is one. Overall the findings of the evaluation of the PNPM-Urban program are quite positive. Feedback from ben- eficiaries generally indicates that the program is an effective approach for community participation and for addressing basic infrastructure at the community level. Independent assessments of infrastructure quality show it to be high, and community organizations are perceived to be working relatively well. The evaluation also identified a number of areas where the program could be improved to further enhance ef- ficiency and impact. These areas include efforts to strengthen project activities for social and economic needs which may have a greater impact on individual and household welfare, ensuring more participation by women and the poorest in communities, more systematic program alignment with the local government budgeting processes, further capacity building of facilitators, and improvements to the MIS. The assessment of the pilot ND program (which incorporates spatial planning and substantially larger investments for communities) identified some implementation challenges related to capacity and processes for the larger projects which would need to be addressed. Many of these recommendations are, at the time of writing, being addressed under the prepara- tion of the PNPM-IV urban program and some may be further mainstreamed over time. Broader issues related to the country’s medium to longer term needs given rapid urbanization and projected increases in urban poverty, and the potential role that the PNPM-Urban Program could play are also explored in the study. Recommendations for Current Program Design In response to the feedback from beneficiaries, it is recommended that the PNPM-Urban program increase ef- forts related to social and economic activities to meet needs at the household level. This will first require some assessment of those needs, and subsequently the design and implementation of some new pilot activities. Over time a more substantial design shift may be required. One model with much potential is the “Generasi” design that is currently being implemented under the PNPM-rural. The Generasi program targets poverty, health and education goals. Each village receives a block grant and with the assistance of facilitators and service delivery workers, villagers undertake a social mapping and participatory planning exercise to decide how best to use the block grant funds to reach 12 education and health targets related to maternal and child health behavior and education behavior. A performance bonus is given to communities based on the villages’ performance on each of the targeted indicators. While the design in urban areas might differ somewhat based on the identification of priority targets, the approach of identifying the poorest and setting specific targets to be achieved (with built in incentives) has much potential for better addressing social and economic needs. 1 See Rand, 2011, A Qualitative Study of the Program Nasional Pemberdayaan Masyarakat (PNPM)-Urba/n in Indonesia, and Schuler, N., 2012, Rapid Ap- praisal of PNPM Neighborhood Development (and Poverty Alleviation Partnership Grant Mechanism). vi Policy Note Executive Summary To address issues of community participation, particularly by women, it is recommended that concerted efforts be made on outreach to encourage female participation in community meetings. Some approach- es which will be explored include holding female-only meetings, providing childcare arrangements at the meetings, or possibly allocating special funds for proposals from women. Reaching out to strengthen relations between the PNPM-Urban and local governments will also be impor- tant for maximizing impact on the ground. In the communities where links to local government work best, respondents in the study most frequently cited the active use of the musrenbang (annual local government planning meeting process) and the commitment of the Department of Public Works and the head of the village to effectively communicate government activities. Recommended actions include working together more closely on planning, using incentives to ensure collaboration, and identifying approaches to institu- tionalize the PNPM-Urban Program into the musrenbang process. Given the critical role of facilitators in the program, efforts to rationalize some of the current responsibilities on reporting would free up time for facilitators to focus more, for example, on socialization activities in the communities. The creation of an incentive based rewards system would also help to motivate performance ultimately improving results on the ground. Options for the Longer Term Role of the PNPM-Urban As Indonesia urbanizes, the demand for land, basic infrastructure, housing, transport and social services at the community level will grow enormously. The study Indonesia: Urban Poverty and Program Review, 2012 iden- tifies key issues for the urban poor, as well as gaps in current policy and programs. Among these issues are targeting inefficiencies with social programs, the growing expansion of informal slums in marginal areas, and gaps in service provision. The PNPM-Urban, a program which operates in all urban areas in Indonesia, also has much potential for expanding its role to further address urban poverty needs, aligned with the GOI poverty reduction strategy. As mentioned, the PNPM-Urban could be used as a platform to improve targeting of social programs within poor communities, work on spatial planning with communities could be expanded, or to extend basic ser- vices to informal settlements which currently do not benefit from Government programs. The continued in- troduction and evaluation of new programs and approaches in the PNPM will allow for the experimentation of changes to meet the evolving needs of the urban poor. Community member presented their plan in Takalar Photo: PNPM-Urban Indonesia: Evaluation of the Urban Community Driven Development Program vii In complex, highly dense, urban areas where needs are particularly high, comprehensive slum upgrading programs may be the most appropriate approach, particularly if a range of infrastructure needs exist or if resettlement is necessary. Slum upgrading programs incorporate spatial planning and community partici- pation, but are typically focused on providing a set of pre-defined basic services, often linked to city-wide infrastructure planning, and are implemented through local governments. In some countries such programs also include social components that are designed to address problems of employment, crime and violence, youth, and health care. Policies to address informal settlements will also be necessary given that many of the growing settlements do not have legal status, complicating the role of government in investing in basic infrastructure. Sharing experiences with other countries through a program of knowledge exchange will also encourage new and innovative thinking for the future. Indonesia has the opportunity to learn from some of the mistakes of other countries that have failed to plan for new urban residents by taking proactive steps now to accom- modate and prepare for its urban transformation. viii Policy Note Section 1: Introduction and Approach 1 . 1 Introduction Indonesia’s Program Nasional Pemberdayaan Masyarakat (PNPM) is the largest Community Driven Develop- ment (CDD) program in the world covering all urban wards (PNPM-Urban) and rural villages (PNPM-Rural) in Indonesia. The program allocates small grants to communities for local decision making on development priorities. The urban program has been operational since 1999. Over time, a number of evaluation studies have been carried out though most have focused on a specific aspect of the program. This policy note summarizes a comprehensive process evaluation of the PNPM-Urban program which has been carried by the RAND Cor- poration in collaboration with SurveyMeter, as well as a rapid appraisal of two pilot subcomponents of the PNPM-Urban program—the Neighborhood Development Scheme (ND), and Poverty Alleviation Partnership Grant Mechanism (PAPG) carried out by a small team of consultants. The RAND evaluation was designed to look at how specific aspects of the program are working, document good practice, distill lessons learned, and identify options for program reform based mainly on a qualitative field study in a sample of kelurahans. The Rapid Appraisal was designed to review the progress and lessons learned from the pilots. The main find- ings of the evaluation and rapid appraisal are summarized below in Section 2, followed by a discussion and recommendations for program design in Section 3. The full studies including detailed findings and data are available as companions to this policy note. Indonesia: Evaluation of the Urban Community Driven Development Program 1 Poor living condition in Semarang Photo: World Bank 1 . 2 Background and Context The PNPM-Urban Program is part of the GOI’s poverty reduction strategy comprised of 3 clusters. Cluster 1 focuses on stabilizing incomes through targeted poverty and social protection programs at the household level, Cluster 2 promotes community level development and empowerment namely through the PNPM pro- gram, and Cluster 3 includes programs that target micro-finance and support to SMEs. The program also fits into the GOI National Urban Policy and Strategy (Kebijakan dan Strategi Perkotaan Nasional or KSPN) which is currently being prepared. The KSPN is a 15-year strategy that outlines the poli- cies and strategies required for cities in Indonesia to develop as "socio-spatial entities.” The PNPM-Urban Program is aligned with the KSPN, especially its goals of: (a) increasing the role of cities as drivers of eco- nomic growth in local, regional and national contexts as well as improving people's welfare and living conditions, and (b) improving the quality of urban governance to be more transparent, accountable, and participatory. The PNPM-Urban program has evolved substantially since its inception more than a decade ago (formerly called the Urban Poverty Project). The Program was designed as a response by the central government to the economic collapse and as a means of injecting resources directly to the poor in urban communities. A similar program, the KDP, was also set up in rural areas. Both programs came at the end of many years of authoritarian rule which had stifled local level decision making as well as local community initiatives. Both the urban and rural programs were originally designed to improve local governance, increase the participation of communities in decision making, and provide increased economic opportunities at the local level. In 2006, the Government of Indonesia launched the umbrella National Program for Community Empowerment (PNPM) to bring together all community driven poverty related initiatives in Indonesia including urban and rural areas. PNPM-Urban is designed on the premise that while many urban issues require larger infrastructure solutions (urban public transport, utility-supplied water, piped sewerage and storm drainage, urban roads), communi- ty-level infrastructure will better respond to community needs and do so at lower investment cost when it is planned and constructed by communities themselves. The program provides direct financial and technical support to poor communities to improve basic infrastructure and social services. Its focus is on empowering communities to make decisions about their investment needs and priorities. 2 Policy Note Introduction and Approach The program now has nationwide coverage; in urban areas, coverage extends to all 11,000 urban wards and 22.3 million beneficiaries (MIS). Since UPP-1, the program has financed over 31,100 km of small roads, 8,800 km of drainage, rehabilitation of 126,800 houses of the poorest, 164,800 units of solid waste and sanitation facilities, and 9,450 health facilities through World Bank loans.2 The overall development goal of the program is aimed at improving living conditions within an environment of improved local governance. This is achieved through three main components: community empower- ment, local government support and service provision, and local capacity building. The project components are summarized in Table 1. TABLE 1: CORE COMPONENTS AND ACTIVITIES OF PNPM-URBAN Component Activities - Community awareness and socialization increased by Community and local government capacity building facilitators through social intermediation activities and training - Local government staff training Kelurahan Block Grants - Infrastructure: Small scale infrastructure investments - Social: Assistance to the extreme poor, intended mainly for the beenifit of individuals. - Economic: Microcredit via revolving loan funds (this is being phased out) Implementation Support - Technical support for project implementation provided by professional consultants - Facilitators aid in accounting and monitoring practices. The PNPM-Urban approach relies on volunteer community based organizations which have elected leaders (BKM). The BKM formulates a community development plan (CDP) to guide activities, and a small team for project administration and subproject implementation. BKM receives funding in three tranches every year for activities designed to meet the goals of community development plans. The grants range from US$15,000 to US$30,000 based on population size per kelurahan and may be disbursed to community self help groups (KSMs) for individual project proposals that are consistent with the CDP, used for social assistance (including grants and training programs), and tertiary infrastructure investments identified by the community in the CDP. Facilitators assist the BKMs and KSMs to prepare financial reports on grant and sub-grant implementation. At the end of the three year cycle, the BKM steps down and a new cycle of election and planning begins. After receiving grants three times under the PNPM-Urban, a kelurahan is expected to graduate to other “channeling” programs and is eligible for coordination funding to do so. The project is executed at the national level by a Project Management Unit in the Ministry of Public Works, and the overall program has a steering committee chaired by the Coordinating Ministry of People’s Welfare, which also chairs the National Poverty Reduction Committee. Members of the steering committee include representatives from the Ministries of Public Works, Finance, Home Affairs, People’s Welfare, Cooperation and Small Medium Enterprises and Industry and Trade. Lastly, two important pilot initiatives have been added to the program in recent years—the Poverty Allevia- tion Partnership Grant (PAPG) and the Neighborhood Development (ND) program. Under PAPG, for the 100 or so local governments selected, in years three to five of the project, a Regional Poverty Alleviation Strategy was created and additional matching funds for activities jointly proposed/cost-shared by the local govern- 2 Based on the MIS as of August 31, 2011. Indonesia: Evaluation of the Urban Community Driven Development Program 3 ment and the community are awarded to local governments. The program aims to make formal a consulta- tive relationship between the local government and community and to finance activities that lie beyond the typical scope of PNPM-Urban, i.e., larger than the typical BLM allows, requiring local government contribu- tions, or covering more than one kelurahan. The Neighborhood Development (ND) Pilot goes further than the PAPG by including spatial planning and larger investments targeted to the poorest communities. Selected sites receive 1 billion rupiah (US$108,000) of which 30 percent is intended for spatial planning, and 70% for implementation and training. The ND was initially piloted in 18 sites and has since been expanded to 255 more sites. As with PAPG, ND is designed to produce one project that is larger than those under PNPM-Urban or produce a series of integrated projects. The core difference between ND and the rest of PNPM-Urban is the element of spatial planning. The ND mis- sion is focused on integrating the planning and management of community settlements into official devel- opment planning. Communities develop a “community spatial plan” (CSP) which is intended as a 5 year plan that donors, local governments, and communities can work towards. To promote collaboration between local government and community-lead initiatives to develop a participatory and sustainable program for settlement planning, the program expects that the investments will be made in coordination with (and with substantial buy-in from) local government, other government, and private sources. 1 . 3 Study Approach The two commissioned field studies (hereafter referred to as i) RAND and ii) the Rapid Appraisal) drew mainly on qualitative methods. The methodology was developed in a consultative way and drew on findings from an impact evaluation of the UPP2 program carried out by the Research Group at the World Bank covering the period 2004-2007 (Box 1).3 The impact evaluation raised a number of questions about impact as well as pro- cesses though a deeper understanding of these issues was not possible through the quantitative household surveys that were administrated. Accordingly, the study carried out by RAND was designed to go into more depth on a number of key questions which were further defined through a consultative process involving main stakeholders; officials from the Ministry of Public Works, Government of Indonesia, the World Bank, members of the PSF and select researchers, and were evaluated in the context of whether the program suc- cessfully delivered key program outputs and its intended/unintended effects on individuals and institutions. The questions focused broadly on governance issues and the infrastructure component of the program as outlined below: 1. Assessing the role of CDD: The CDD approach has been attributed to a number of objectives including building social capital and improving living conditions. a. To what extent is the CDD model realized by PNPM-Urban, in terms of planning, voting and contribu- tion to program activities? 2. Links with local government: a. How does PNPM-Urban link with local government and with sectoral programs (details on types of programs, interactions, etc.)? b. Is there much collaboration / duplication? c. In communities where collaboration works well, what are the contributing factors? Similarly, where it doesn‘t work well, why is that? 3. Capacity building and effectiveness of the facilitators: a. How effective are the current facilitators and do they adequately meet the needs of the communities? b. What characteristics contribute to strong facilitators versus weak? 4. Governance and Control Mechanisms: a. How effective are the current governance structure/ control mechanisms that are in place? Are they adequate or could they be improved? If so, how? 3 Pradhan, Rao, Rosemberg, 2010, The Impact of the Community level activities of the Second Urban Poverty Project. 4 Policy Note Introduction and Approach 5. Quality of infrastructure projects: a. Overall how is the quality of the infrastructure investments? b. What factors contribute to high- and low-quality works? c. Are current standards adequate? d. How cost effective are infrastructure investments? e. Are they adequate or could they be improved? If so, how? 6. PAPG/ND and the adequacy of block grants: a. Currently communities receive relatively small grants. Under the Neighborhood Development (ND) scheme being piloted, the amount of those grants is substantially higher. How do experiences and outcomes vary under PAPG/ND? What are the differential impacts? The analytical approach for the RAND study incorporated several methods, primarily using qualitative tools to gain depth of understanding in a relatively smaller number of kelurahans. This approach was chosen in part as a follow up to the larger scale quantitative impact evaluation of the UPP2 carried out from 2004-2007 that raised a number of questions about program design.4 Primary data was collected in 16 kelurahan across Indonesia (including three participating in the pilot PAPG/Neighborhood Development Program) (May-June, 2011). The field work was carried out together with a qualitative assessment of perceptions of poverty and of Government programs in the same communities. Data gathering included: i) key informant interviews; ii) focus groups; iii) in-depth interviews with poor households and beneficiaries; iv) rapid surveys of households and PNPM-Urban community group leaders; and v) infrastructure inspections by engineers. Secondary data from the MIS on community participation, budgeting and project information and a database of complaints was also analyzed to the extent possible. The criteria for site selection included broad geographical representation, variation in the level of poverty and experience with larger versus smaller grants, and Neighborhood Development projects. Thirteen of the 16 kelurahan were randomly chosen from the sample of urban communities in the Indonesia Family Life Survey. A full discussion of the sampling methodology is discussed in the background report. The Study sites include: TABLE 2: STUDY SITES (NON-ND LOCATIONS*) Kelurahan Kabupaten/Kota Province Wealth index Triharjo Kulon Progo D.I. Yogyakarta High Karo Pematang Siantar North Sumatra High Cengkareng Timur Jakarta Barat Jakarta High Tambakrejo Surabaya East Java High Pancuran Gerobak Sibolga North Sumatra Moderate Astana Cirebon West Java Moderate Kauman Surakarta Central Java Moderate Lirboyo Kediri East Java Moderate Antang Ujung Pandang South Sulawesi Moderate Ngestiharjo Kulon Progo D.I. Yogyakarta Low Wiroborang Probolinggo East Java Low Rantepao Tana Toraja South Sulawesi Low Hulu Banteng Lor Cirebon West Java Low * There were 3 ND sites, all located on Java. To ensure confidentiality of the sites and the respondents in this small sample, the ND sites are not listed here. They are referred to in the study as Site 1, 2 and 3. 4 Pradhan, Rao, Rosemberg, 2010, The Impact of the Community level activities of the Second Urban Poverty Project Indonesia: Evaluation of the Urban Community Driven Development Program 5 The Rapid Appraisal was carried out as a follow up to look more closely at the PAPG and ND pilots. The ap- proach consisted of national level interviews, field visits to 11 kelurahan in 6 cities (February 2012) including key informant interviews, visits to investment sites, and review of project documentation. The sites, listed in Table 3, were selected from a master list of all ND and PAPG sites with an aim to provide some representation in geographic coverage, a bias towards more dense urban environments, and a variation in performance and local government engagement. Two sites in Solo and Surabaya were included as innovation sites that could offer particular learning. Interviews were carried out with community members, local government representatives, and facilitators. A set of semi-structured interview questions for each category of interview was developed including basic information on location description and key development/poverty issues, project description, perceived project successes/strengths, perceived project challenges/weaknesses, and solicitation of recommendations/other issues. As with the RAND study, the Rapid Appraisal was designed to offer a snapshot of locations and is interpreted as indicative of potential trends. BOX 1: IMPACT EVALUATION OF THE URBAN POVERTY PROJECT 2 (UPP2) An impact evaluation of the UPP2 project, a predecessor of the PNPM-Urban Program, was carried out from 2004-2007 by the Development Economics Group at the World Bank. The objective of the evaluation was to detect whether there are discernable differences in the welfare and participation of UPP2 recipients compared to similar groups of individuals who did not participate in the program using difference in differences estimation. The evaluation specifically looks at the Kelurahan Grants component of the UPP2 project, representing 46% of total project cost. The study focuses on ad- dressing questions on how the project affected beneficiaries living in communities which received a grant in terms of i) welfare; ii) access to credit; iii) access to infrastructure; and iv) participation in local institutions. Field surveys were carried out in January-February 2004 (baseline), August 2005-January 2006 (mid- term), and August-September 2007 (final). The sample included over 15,000 household members at the baseline and final rounds, and close to 10,000 at the mid-term surveys in 73 treatment and 38 con- trol kelurahans across Indonesia. It is important to note that the final survey was meant to be carried out at the end of the project (December, 2010), however, in 2007 the Government announced the intent to scale up UPP into a national program (PNPM), which would ‘contaminate’ the control group. As such, the ‘final’ survey was carried out substantially earlier than planned and thus captured impacts only at an interim, rather than final stage of the project. Key findings from the impact evaluation include: i) an overall improvement in welfare in urban areas but no statistically significant impact in treatment communities (relative to control communities); ii) an improvement in access to adequate sanitation in treatment communities; iii) a considerable increase in participation in community organizations in both treatment and control communities; iv) a shift of KSM economy members with previous access to non-UPP2 loans to UPP2 loans which carry lower interest rates (average drop from 4.3 to 2.2 percent/month); and v) evidence that those who are elected to the BKMs are more likely to be males, educated, affluent and have official connections, and KSM economy5 members are more likely to be women, less affluent, and less connected than BKM members but more so than the general population. These findings raise further questions about participation, the role of facilitators, BKMs, and the role of the Program vis-à-vis household welfare impact given the small size of the grants. Many of these issues are addressed under the RAND evaluation. Source: Pradhan, Rao, Rosemberg, 2010, The Impact of the Community Level Activities of the Second Urban Poverty Project. 5 Groups are set up under the BKM to manage funds for specific activities – infrastructure, social, and economic; these are known as KSMs. The study fo- cused only on KSM-economy – i.e. those that manage the revolving loan funds – and not at KSM-social or KSM-infrastructure. 6 Policy Note TABLE 3: STUDY SITES, PAPG AND ND RAPID APPRAISAL Kelurahan Kota Province Program Klumprit Sukoharjo Central Java PAPG Blimbing Sukoharjo Central Java ND Pasar Lama Banjarmasin S. Kalimantan PAPG Cakranegra Seltan Mataram Central Java PAPG Pagutan Mataram Central Java ND Karuwisi Makassar S. Sulawesi ND Sinrijala Makassar S. Sulawesi ND Watulondo Kendari S.E. Sulawesi ND Lapulu Kendari S.E. Sulawesi PAPG Podosugih Pekalongan Central Java ND Kraton Kidul Pekalongan Central Java PAPG Indonesia: Evaluation of the Urban Community Driven Development Program 7 8 Policy Note Section 2: Main Evaluation Findings The key findings of the two studies are organized by topic and presented below. Unless otherwise noted, findings refer to the RAND study covering the PNPM-Urban. The final section on assessing the size of block grants and the ND and PAPG pilots includes findings from both studies. An introductory discussion provides some discussion and context on each of the issues evaluated. 2 . 1 Assessing the CDD Approach Community Driven Development is a fundamental part of the PNPM program with a focus on empowering communities to make decisions about their investment needs and priorities. The benefits are intended to improve equity and inclusiveness, efficiency and improved local governance. Experience from other coun- tries has shown that CDD provides an approach to both alleviate the physical issues of urban poverty such as unsafe housing, poor infrastructure, and utilities, while also promoting the intangible elements of collective community empowerment, strengthening of social networks and promoting more open communication between communities and government institutions. CDD can also enhance the sustainability of projects by strengthening the responsiveness of communities, as well as the institutional relationships between com- munities, local and national government bodies, private developers and NGOs.6 That being said, CDD has limitations particularly when it comes to implementing large scale investments or in particularly complex urban environments which may for example, require resettlement. Improving entire road networks, drainage networks, or the provision of utilities across the country cannot be effectively planned and implemented at the community level. Areas that are at particularly high risk to climate and natural hazards may require com- plex infrastructure planning or resettlement decisions which are beyond the scope of only the community. In addition, fundamental to the CDD approach is the participation process which can be substantially more time consuming than Government led upgrading programs. The World Bank. Urban Upgrading and Community Driven Development. Webpage. . 6 Indonesia: Evaluation of the Urban Community Driven Development Program 9 BKM Meeting, Makassar Photo: World Bank The PNPM-Urban approach to participation relies on volunteer community based organizations (BKM) which have an elected leader. It is in a sense, a de facto system of delegated representation through the BKM. The BKM formulates a community development plan to guide activities, with a small team for project administra- tion and subproject implementation. BKM receives funding from PNPM-Urban for activities designed to meet the goals of community development plans. Community volunteers play an important role including initial planning, actual administration, implementa- tion, and maintenance of projects. A basic element linked to fostering participation in the program is ensur- ing that knowledge about the program and information about its activities is widely disseminated to the public, including how and when to participate in voting and planning, and the existence/status of projects undertaken. In the evaluation, the CDD approach was assessed by looking at four main issues: i) the success of overall so- cialization in building basic awareness and understanding of PNPM-Urban processes, funding and projects, ii) community participation in planning, decision making activities, and project implementation activities; iii) inclusiveness towards women; and iv) poverty targeting and the extent to which PNPM-Urban meets the needs of the poor. Findings. On the whole, most respondents in the study perceive that PNPM-Urban provides important and needed services. It was clear from the responses that PNPM-Urban infrastructure programs are well-received and beneficial to the community, and viewed as being well chosen and targeted. Community organizations are also perceived to be working relatively well, independent of Government programs and structures. BKM representatives, RT leaders (the smallest neighborhood unit) and RW leaders (one level up from RT) regularly attend KSM meetings in all sites. The issues and challenges that emerged from the interviews and focus groups were related to socialization of the program, the responsibilities of the BKM (community organiza- tion/Board of Trustees), KSM (community voluntary contribution group), relatively low participation rates by community members, and the extent to which the needs of the poor are met. 10 Policy Note Main Evaluation Findings Socialization of the program takes place mainly through facilitators and BKMs. Within the study communities, at the level of officials and community leaders there is a good understanding of PNPM-Urban’s goals and processes. At the community level, however, there is a relatively low understanding of the program and this was particularly acute for the poor. A common concern linked to the system of delegated representation through the BKMs is related to the avail- ability and political weight of the delegates. Limits on the number of members on the BKM may leave some RTs unrepresented in the larger kelurahans. In most of the BKMs, not every RT in the kelurahan had a representative on the BKM itself; and only half the BKMs reported having community leaders join in their meetings. Direct participation by community members in decision making about project selection and implementa- tion (primarily through attending meetings) is relatively low, estimated at approximately 20% for decision making. This is not, however, surprising particularly for urban areas where the opportunity costs of participa- tion are high. Participation in infrastructure project implementation was somewhat higher, though this varies by community. During implementation, participation is mostly in the form of providing labor, either directly or indirectly (e.g., help in clearing the sites, cleaning, and women preparing snacks for male workers). The most frequently cited barrier for participation, especially for PNPM-Urban volunteers in KSM and BKM, was the time-cost of the process, which can be substantial. An IEG evaluation survey of CDD projects (mainly rural) in Benin, Brazil, Madhya Pradesh, and Uttar Pradesh found the proportion of community members participating in meetings ranged from a range of 16 percent (Uttar Pradesh) – 72 percent (Benin). The report raises the argument that a low level of inclusion in community organizations responsible for subproject selection and management is not in itself problematic. “It is unrealistic and perhaps inefficient to expect communities to collectively undertake such activities, and a group of villagers could instead be chosen to do soon behalf of the community. Though valid, this argument raises concerns regarding the ways in which community representatives are selected and the inclusion of weaker social groups.” 7 Women’s participation in the PNPM-Urban is found to be especially low, attributed to social norms, family roles, and in some cases limited outreach to women. Some communities reported that they lost most of their female members who eventually stopped participating in PNPM-Urban community meetings as they felt they were sacrificing time with their families, especially children. In other cases, female respondents reported that only male heads of household were invited to PNPM-Urban meetings, or that when household invita- tions were received, these were assumed to be for the male heads of household; female-headed households interviewed did not report being invited. The low participation is attributed to many factors – limited or ineffective socialization efforts, challenges in sustaining participation over time due to fatigue, the opportunity costs of time away from work and family, and those who reported that they felt their voices weren’t heard and thus didn’t bother to participate. With regard to the extent to which the PNPM-Urban meets the needs of the poor, the overall responses were favorable though this is somewhat difficult to interpret. Responses regarding the value of PNPM-Urban infrastructure programs were also favorable with investments generally being perceived as beneficial to the community, well chosen and targeted. That being said, when community members and key informants were asked about the most important needs of the poor and about what elements of PNPM-Urban are most beneficial, many respondents emphasized the need for social and economic services. The majority indicated that loans, and training, health services and education (support with school expenses, and skills training) were the greatest needs of the poor. With regard to infrastructure, priorities included housing improvements, drainage, roads, solid waste collection, toilets and clean water. In interpreting these results it is important to consider there could be some bias in the responses given that infrastructure needs are already being ad- dressed through the PNPM program and that people tend to prefer programs that provide direct individual benefits to them rather than public goods such as infrastructure where benefits are more diffuse even if they are significant. 7 IEG, World Bank, 2005, The Effectiveness of World Bank Support for Community Based and Driven Development. Indonesia: Evaluation of the Urban Community Driven Development Program 11 Good practices in participation. There are a few good practices specific to the CDD approach that were identified in the communities studied. In communities where substantial efforts aimed at socialization and outreach to encourage participation within communities were made, for example by holding informational exhibitions and publications, these efforts reached a lot of people and attracted much interest. In many cases the BKM’s level of effort was cited as responsible for the success of socialization. In communities with high levels of participation, previous experience with community mobilization, local culture and education levels were noted to be important. In sites were female participation was said to be ‘good’, there was already some presence of active women leaders or groups which facilitated this. 2 . 2 Links with Local Government Indonesia’s institutions have evolved dramatically over the past decade with its subnational governments now playing a major role in service delivery. As local governments take on a greater role in service deliv- ery, it is increasingly important to ensure that the PNPM-Urban operates effectively in the context of this decentralization, particularly in ensuring good collaboration between the PNPM-Urban and local govern- ments. The study assessed how the linkages between the PNPM-Urban program and local government and sectoral programs to identify if there was much collaboration or duplication, and seek out factors that contributed to good collaboration. That being said, many respondents voiced concern that the involvement of local govern- ment could reduce community ownership, jeopardizing the participatory nature of the program. In principle, the PNPM-Urban promotes collaboration with local government through the Renta (Annual Plan), which is presented to the kelurahan musrenbang, annual local government development planning meeting process, at the start of each year to be integrated or adopted into the local government planning. Local lurahs (civil servant heads of kelurahan) also have some responsibility for reviewing and signing-off on community project proposals. At each administrative level the Ministry of Public Works acts in accordance with the local government to form a coordination working group to provide information, technical support and to solve administrative problems. In addition, PNPM-Urban facilitators work with the BKMs to form as- sociations, (BKM Forums at the kecamatan and kota level) to work collectively and influence government planning while also building local government capacity to work with BKMs and the BKM Forum. The Poverty Alleviation Partnership Grant (PAPG) was introduced as a pilot in 2009 to strengthen linkages between com- munities and local governments but is addressed separately in section 2.6 as a pilot. PJM bazar, Majene Sulawesi Selatan Photo: PNPM 12 Policy Note Main Evaluation Findings Findings. In practice, linkages between the PNPM-Urban Program and local government vary considerably across sites. Most government officials were aware of PNPM-Urban activities. In some communities linkages with local governments was quite strong, in others there was little coordination. In some communities, offi- cials reported that they were not invited to PNPM-Urban meetings or project sites. There appears to relatively little duplication in the scope of activities. In the more successful communities, active use of the musrenbang, was a crucial mechanism for coordination between PNPM-Urban and the local government at the kelurahan level. While the process is only carried out an- nually, it means that the PNPM-Urban is aware of what is planned, and can respond to emerging needs that may evolve between that time frame, given that the PNPM-Urban decision making process is faster. This helps to mini- mize duplication of efforts. A second area identified as contributing to successful links between communities and local governments was the existence of a strong commitment by the Department of Public Works and the head of village to effectively communicate government activities with BKM members. Coordination is thought to work best when the same individuals are present at both the PNPM-Urban and musrenbang meetings. In the communities where the relationships between PNPM-Urban and local government officials were less positive, the underlying factor most commonly reported was that the lurah was unwilling to participate in the PNPM-Urban. Some of this may be due to lingering effects from the early stages of the UPP which dis- couraged too much involvement by the Lurah in an attempt to foster community participation. 2 . 3 Capacity Building and Effectiveness of Facilitators Facilitators are an integral part of the PNPM-Urban Program and can play a key role in the success of the program at the community level. They are on the frontline, and therefore can have enormous influence and impact on the program’s effectiveness. Yet Mansuri and Rao (forthcoming) point to a paradox with regard to the role of facilitators, particularly with regard to participation -- they are paid to play the role that a social activist would in mobilizing communities, but their incentives do not necessarily align with those of a social activist. They are responsible for large numbers of communities, are relatively poorly paid, and are typically rewarded based on project targets rather than community-based goals. The study looked at the role of facilitators, how effective they are, if they adequately meet the needs of the communities, what the characteristics of strong versus weak facilitators are, and what role might facilitators play going forward. Findings. It is clear from the feedback that the importance of facilitators is significant as they play a key role in some of the most central elements of the program. They are responsible for i) assisting the KSM and BKM in respective responsibilities of proposal writing, budgeting and cost control, technical guidelines for projects, infrastructure maintenance, prioritization of projects, etc; ii) socialization and community outreach activities; iii) supporting community mapping; and iv) project implementation inspections through monitoring con- struction works, ensuring compliance with guidelines. In most of the communities in the study, the BKMs report that the facilitators meet with them about once a month with a majority of KSMs reporting every 2-4 weeks. The most frequent type of help was with financial reporting and management, and project monitoring and implementation. The attributes of highly effective facilitators were identified as institutionally savvy, good interpersonal skills, friendly, awareness of the needs of the community, understanding of local culture, good communication skills, and a proactive attitude. Feedback from communities who were satisfied with facilitators’ performance reported frequent visits to construction sites, provision of technical guidance to the KSM, and sharing of information. Identified challenges that may have some impact on how effective facilitators are, mainly related to i) the work load of facilitators, particularly on administrative tasks, which is heavy leaving only limited time for each Indonesia: Evaluation of the Urban Community Driven Development Program 13 kelurahan; and ii) frequent rotations which may disrupt facilitators’ ability to meet community needs. While this practice has many benefits, it does not always coincide with the project cycle. Some respondents also reported that facilitators work with a greater number of communities and groups than they can effectively handle, reducing their time available for each community, in particular for socialization activities. Facilitators also pointed to a lack of adequate training for their various roles and responsibilities which limited their ef- fectiveness. 2 . 4 Control mechanisms and Governance The challenge of good governance in service delivery has a long history in Indonesia, and a program such as the PNPM-Urban which operates at a national level delivering thousands of subprojects each year makes it particularly challenging. Over time the program has developed many features to enhance institutional ef- fectiveness through accountability measures including simple and practical procedures for financial control, information disclosure, feedback, complaints, follow-up and resolution. The entire program is designed to generate demand for good governance at the grassroots’ level. To enhance accountability, the GOI has also established two monitoring and evaluation working groups that cover the PNPM-Urban; one for Management Information Systems (MIS) and one for reviewing and finalizing performance indicators for the program. The National PNPM Steering Committee has adopted an integrated, shared MIS and website. The MIS has been developed as an advanced web-based system that is open to the public, providing updates and information on project progress against performance indicators for each cycle of the PNPM-Urban. Other mechanisms to ensure good governance are built into the PNPM-Urban as part of its Better Governance Action Plan which include: (i) randomized audits; (ii) severe penalties for misuse of funds; (iii) rewards for well performing BKMs and KSMs; (iv) an accountability matrix for all operations; and (v) a complaints handling service. The study assessed how effective the current governance structure and control mechanisms that are in place are through the interviews and focus groups carried out during the field work, with an aim to identify pos- sible areas for improvement. Findings. The program has multiple layers of controls, and on the whole, they appear to be working. In most sites, the multiple approaches to auditing are being completed on schedule and are key to limiting or preventing the misuse of funds. Few respondents reported complaints of misuse of funding within PNPM- Urban. While some of this may be attributed to reporting bias, in many cases respondents pointed to the many layers of control mechanisms and auditing under the Program that help avoid these problems. Unan- ticipated audits are particularly useful as the auditors show up unannounced. In addition, many respondents attributed a purportedly low prevalence of corruption to the participation and implicit monitoring of com- munity members. This is particularly the case in communities where residents were very involved in project planning and implementation. While the many layers of controls may be contributing to the administrative burden on facilitators, BKMs, and KSMs, they appear to be working relatively well. The open governance approach to the MIS provides an excellent platform for monitoring project performance and potentially a large role as a coordinating mecha- nism for a wide range of stakeholders. That being said, the scale and scope of the MIS presents challenges. It can be difficult to navigate, and the validation procedures are not always adhered to, which can result in inconsistencies in the data. There is also much scope for enhancing the system to become a more integral part of the control mechanisms for the program and a means for monitoring outcomes, possibly introducing efficiency gains in the way information is collected and reported, thereby reducing some of the current time consuming reporting activities. 14 Policy Note Main Evaluation Findings A bridge built by community in Gorontalo Photo: PNPM 2 . 5 Quality of Infrastructure Approximately 70 percent of the community grants are used for infrastructure investments which mainly in- clude roads, bridges, drainage, public toilets, infrastructure improvements (housing and public facilities), and clean water projects (drinking water, bathing, and water sources (wells and natural springs)). In most CDD programs around the world, it is thought that community engagement in the identification and implemen- tation of projects more closely addresses needs and priorities of beneficiaries, and results in higher quality projects that are more likely to be well maintained. The study aimed to assess the overall quality of infrastructure investments, identify what factors con- tribute to high and low quality works, whether current standards are adequate, and how cost effective infrastructure works are. To assess this, engineers carried out a formal evaluation based on standardized protocols for specific project types with the goal of determining whether PNPM-Urban projects met basic construction and materials standards, and practices, particularly on maintenance and performance. A total of 243 projects in PNPM-Urban communities and 69 projects in ND sites were assessed. The largest share of projects in the sample were roads (45 percent), followed by drainage (23%) and infrastructure improve- ments (23%). Findings. Overall, infrastructure quality was found to be quite high based on both engineering assessments and qualitative assessments. A high percentage of drainage, infrastructure improvements and public toilet projects received good scores, while for roads and water projects the scores were more evenly mixed be- tween good and sufficient. Almost all respondents, including some government officials, were favorable about infrastructure quality noting that PNPM-Urban projects were often of better quality than government constructed projects. This was attributed to the participatory approach which involves better oversight at the community level. In a few cases, assessed project quality was low due to substandard materials or poor workmanship. Critiques during the focus groups and beneficiaries were concentrated around issues of low quality materials, and on technical issues. With regard to maintenance, respondents perceive that the community is more likely to maintain PNPM- Urban projects than other projects, however, in most cases the projects reviewed have not been in use for a sufficiently long period of time to make such an assessment. For ditches, gutters, drains, and roads, it is typically the people living in the immediate vicinity of a project who are in charge of maintenance. For some types of infrastructure (e.g., trash tubs) maintenance is carried out by paid workers. In a small number of ke- lurahan, community members contribute money for infrastructure repairs. Indonesia: Evaluation of the Urban Community Driven Development Program 15 The study was not able to assess in detail the cost-effectiveness of projects because budget data is often insufficient or questionable and counterfactuals are not available. In many cases, the budget for a single project was used to fund multiple projects, especially where residents were perceived to have diverse needs. Without comparable data for non-PNPM-Urban projects, it is difficult to assess relative effectiveness for PNPM-Urban infrastructure projects. Interview respondents were asked to provide subjective assessments on whether PNPM urban projects were cost-effective. Some responded that they were often more efficient than government projects due to community contributions (lower total costs), the absence of complex ten- der procedures, and better oversight through community participation. Several other studies show CDD’s cost effectiveness as compared to equivalent works build through other government delivery mechanisms.8 In Indonesia (rural), Nepal and the Philippines, small infrastructure delivered through CDD was between 13- 16 percent lower depending on the investments. These cost savings came from eliminating the middle man or contractor overheads. 2 . 6 The Adequacy of Glock Grants and the PAPG/Neighborhood Development Scheme Approach Under the current program design, communities receive relatively small grants ranging from US$15,000 to US$30,000 in the larger kelurahans. A scheme aimed at more comprehensive neighborhood upgrading was introduced on a pilot basis under the Poverty Alleviation Partnership Grand (PAPG), and subsequent Neighborhood Development Scheme (ND). These programs provide a useful comparator for assessing the adequacy of the block grants given that the amounts are substantially larger (approximately US$108,000 per kelurahan for ND). The PAPG was designed to increase partnership between communities and local governments largely by requiring participating local governments to apply for the funds (thereby demonstrating interest) and to commit their financial and technical support to PNPM Urban projects. The ND was introduced as an effort to introduce more systematic urban upgrading by significantly increasing the size of the grant, and including spatial planning as a core activity which would be incorporated into official development planning. While the grant amount is much larger than the PNPM-Urban sites and is mainly for infrastructure that can contribute to economic or social development, a considerable fraction (30 percent) of the funding is allocated to plan- ning (including hiring outside experts) and “marketing activities.” The purpose of the planning and marketing support is to increase the community involvement in the project and to share the plan with potential funders (e.g. local government and the private sector) so as to “channel” additional resources to the communities. All ND activities and grant funds must be focused on one priority area, which is selected based on several parameters, including the poverty rates and other measures of need. Because the PAPG and ND are still pilots, they were treated separately in the RAND study. With the study design focusing on in-depth community analysis, it was only possible to include three ND sites making it difficult to draw conclusions on the ND program. That being said, key findings were illustrative and were fol- lowed up through the Rapid Appraisal which covered an additional 11 sites to better understand the issues raised. An impact evaluation of the ND is being planned, though findings would only be available in 2015. The activities in the PAPG/ND sites included infrastructure as well as economic and social. The three RAND sites, were categorized as “Site 1 - typical”, “Site 2 - unsuccessful” and “Site 3 - successful” based on site charac- teristics including community participation, local government involvement, and the quality of ND planning products. In site 1, the community decided to use the ND funds to build a garbage compost and recycling facility, two concrete roads near the facility, and purchased bins for recycling and composting which were distributed in different parts of the kelurahan. In site 2, the construction of a stone retaining wall for flood protection, and repairs to the road were financed. In site 3, the ND project financed paving of roads alongside a stream and construction of a footbridge over the stream. See Wong, discussion draft, “What have been the Impacts of World Bank CDD Programs, CDD Impact Evaluation Review and Operational and Research 8 Implications.” 16 Policy Note Main Evaluation Findings Findings. Relatively few of the traditional PNPM-Urban sites reported that the current block grant size was inadequate for project needs. This may reflect the fact that project requests were already scaled to expecta- tions about available resources. A second piece of suggestive evidence is that in practice, when infrastructure assessments were conducted, planned budgets for projects were frequently divided up into several smaller projects rather than spent on the larger projects that were initially proposed. Analysis of the MIS data sug- gests a similar finding -- when looking at a sample of Kelurahan enrolled in the PAPG program (where com- munities receive higher grants per capita), these communities were used primarily to fund more, rather than larger, projects. The size per project and type of project does not appear to change relative to the regular PNPM-Urban communities. While this data is subject to many caveats, it is suggestive. PAPG. In the communities studied, the PAPG has demonstrated the ability to forge constructive partner- ships between local government and communities, however, in many cases the result appears to emphasize community participation in local government activities rather than local government support of community driven development activities. In addition, the scale of PAPG projects appears to be limited and most often driven by the local government wishing to expand ongoing work using the PAPG approach as a way to leverage more funds. That being said, the approach has served to foster stronger relationships and in the cases where the program was followed on with the ND, the relationships provide an important platform for coordination on the spatial planning and larger investments. ND. On the whole, the field work from both studies indicated that there is some concern that the larger, more complex projects are harder to manage. The larger projects require strong financial management and skilled labor, thus making it more difficult to involve (untrained) community members in implementation. Allocat- ing the larger funding blocks equitably across different areas or groups of residents also can be complex. The structure of participation is similar to the PNPM-Urban sites, with delegated representation. Participa- tion, however, appears to be more difficult in the ND sites due to the complexity of projects which taxes non-trained volunteers, and the smaller number of residents who perceive direct impact by the projects, as the scheme is dedicated to the poorest neighborhood only. For example in the three RAND sites, the initial turnout to discuss the plan or location of a priority area drew many participants, but once the decisions were made, attendance at the meetings dropped off. In one of the sites, over 100 people came to the first meeting, but when residents found out that the grant only went to one area (poorest) with nothing more for the rest of the kelurahan, many never attended again. There were also reported problems with participation fatigue, and volunteer labor only being available on the weekends, driving up costs. It is also worth noting that the ND pilot communities were chosen because they were high performing under PNPM-Urban. Many kelura- han who are less well performing may lack the capacity to effectively plan and implement ND projects. These will also likely be poorer communities where the need for economic development is greatest. Training program as part of social activities, Kendari Photo: PNPM-Urban Indonesia: Evaluation of the Urban Community Driven Development Program 17 The community spatial planning (CSP) process in ND is extensive, accounting for 30 percent of the block grant. While there are detailed guidelines on the CSP, there appear to be real challenges in implementation in some communities. As a planning process, the CSP appears to be somewhere between a technical urban planning process (set to very specific and perhaps overly technical guidelines), a community mapping and prioritization exercise, and a project. The urban planner has a difficult task in both developing a technical urban planning product with very clear specifications in a relatively short period of time, and engaging in a participatory process and ensuring community priorities are incorporated. In several of the Rapid Appraisal communities, the final CSP often appear to be too complex to resonate with the community, and not suf- ficiently linked with local government’s plans to help influence infrastructure and maintenance decisions. A key element in the process of problem identification, planning, and activity decision making that appeared to be missing in the communities studied is the discussion of options for consideration. While the sample is relatively small, the limited evidence suggests that the process appears to often skip over the discussion of options for addressing a particular problem with the decisions pre-determined even before the planning process began. This may in part be due to time and facilitation constraints. The types of investments under the ND were broadly categorized as community upgrading or public goods (or “icons”). The community upgrading activities appear to be similar to PNPM-Urban investments, meaning that funds are often used to do more activities from the CDP that needed funding rather than undertaking new larger programs. The “icon” investments such as an outbound facility for tourism or community hall were more apparent in semi-urban areas where land constraints are not an issue. In assessing the selection of investments in relation to the greatest potential for impact on communities, several observations emerged. Decisions appear to be influenced by: i) visibility – some communities prioritize highly visible ‘icon’ invest- ments particularly in the initial pilots where communities were encouraged to “think big.” Current prioritiza- tion now emphasizes poverty reduction activities though many communities still prefer ‘icon’ investments; ii) complexity – some communities avoid investments that require addressing the highly complex issues such as land use; and iii) doing what is familiar – there may be a bias for community members and facilitators to do “more of the same” with a higher level of resources given their experience and comfort level. Facilitators were highly valued in the pilot sites as with the PNPM sites, with respondents emphasizing the need for more time for facilitators to spend on implementation and guiding construction. In the case of the ND, because the program is new, the facilitators had challenges in understanding and implementing the guidelines. With regard to links with local government, the level of cooperation and collaboration varied across sites. Two factors are noted that may be reducing the potential for cooperation between local government and ND. First are political issues – in one site there was considerable animosity towards BKM members, who were also members of the local council. Second is the fact that control and supervision is carried out by the Province rather than by the City level government. 18 Policy Note Section 3: Discussion and Recommendations The main findings of the evaluation indicate that overall the PNPM-Urban program is an effective approach for community participation, for addressing basic infrastructure, and to some extent other needs at the com- munity level. It is a core part of the GOI’s poverty reduction strategy as well as the national urban strategy cur- rently being prepared. Respondents from the focus groups report that the PNPM-Urban program provides important and needed services, and that the infrastructure programs are well-received and well-targeted. Community organizations are also perceived to be working relatively well, independent of Government pro- grams and structures. The findings of the evaluation also point to areas for improvement which could enhance the overall op- erational efficiency and impact of the program. These areas include efforts to increase project activities for social and economic activities, ensure more participation by women and the poorest in communities, further capacity building of facilitators, improvements to the MIS, and more systematic program alignment with the local government budgeting processes (Musrenbang). The assessment of the pilot PAPG and ND communi- ties identified some challenges in implementation with some implications for its future direction. Several recommendations for program design emerging from the study are outlined below. Many of these recommendations are being addressed under the new PNPM-IV program and some may be further main- streamed over time. Notably the feedback from communities prioritizing social and economic programs may require a more substantial design shift, possibly towards a “Generasi” style program as in the PNPM- rural. There are also broader questions related to the future of the PNPM. As a program which operates in all urban areas in Indonesia, there is scope for expanding its role to further address urban poverty which is projected to rise with rapid urbanization, as well as to achieve broader urban development goals. A discussion of options is also discussed below. Indonesia: Evaluation of the Urban Community Driven Development Program 19 Makassar Photo: World Bank i) RECOMMENDATIONS FOR PNPM-URBAN DESIGN CHANGES Several aspects of the program were identified in the evaluation which would benefit from some design changes. Introducing any changes, however, would be best done through an initial piloting and evaluation to minimize any unforeseen negative impacts, particularly given the large size and reach of the program. Fortunately, the program has a long history of gradual change based on experimentation and evaluation and thus the structure has much potential for absorbing innovation. Deepening efforts to promote community participation, particularly for women. The CDD approach is rooted in the participation of community members in the decision making and implementation process. The experience with participation has been mainly through representation by the BKM and KSM with much scope for more involvement by community members, particularly the poor and women. This could be achieved through further outreach specifically targeted at females and low-income residents to encourage participa- tion in community meetings, as well as through introducing new approaches for attracting these groups and eliciting proactive participation. Some ideas include holding female-only meetings, allocating special funds for proposals from women, or providing childcare arrangements for participants. Such approaches are being currently explored and would need to be piloted and carefully monitored to be able to identify any unanticipated outcomes. Allocating a larger share of funding for social and economic programs. A majority of spending currently goes to infrastructure, yet communities pointed to needs for social and economic programs for the poor during the field evaluation. This could be achieved by increasing the share of activities allocated to social and other economic activities. Currently, social activities (10 percent of the program) are aimed at the poorest of the poor, mainly intended to benefit individuals. Economic activities are mainly linked to the revolving loan fund which is being phased out. In order to redesign the program to better meet social and economic needs of the poor (which is currently planned for the PNPM-Urban IV) it would be important to better understand those needs and how the social and economic programs function. A model for exploration in the urban context is the existing Generasi program under the PNPM-Rural. The PNPM-Generasi program targets poverty, health and education goals through a program of community block grants with explicit performance bonuses for communities. Each village receives a block grant and 20 Policy Note Disscussion and Recommendations with the assistance of facilitators and service delivery workers, villagers undertake a social mapping and participatory planning exercise to decide how best to use the block grant funds to reach 12 education and health targets related to maternal and child health behavior and education behavior. A performance bonus is given to communities based on the villages’ performance on each of the targeted indicators. A recent impact evaluation has overall found improvement in the targets it was designed to address, particularly in poor com- munities. The community incentives approach had mixed results, in part because it was found to be some- what complex for communities to understand.9 An increased focus on social activities could be designed to help the urban poor in accessing existing social protection programs for which they may be eligible (Raskin, BSM), assist with school fees which were reported as particularly onerous for poor urban households, and could facilitate access to training and microcredit which have also been identified as priorities. Any introduc- tion of such a design shift would require additional facilitators and extensive training. Strengthening relations between local governments and PNPM-Urban. Over time, decentralized gover- nance will increasingly rely on local governments for the implementation of programs and policies at the local level. This provides enormous opportunities for communities and local governments to work together, though current approaches to this are often ad-hoc. The PAPG pilot and some good practice PNPM-Urban communities offer some important lessons that could be incorporated into the development of good practice guidelines on how to work more closely in planning. For example, the use of incentives to ensure local government participa- tion in important PNPM-Urban meetings and vice-versa, and steps on how to institutionalize the PNPM-Urban program into Musrenbang process. Such approaches provide opportunities for improving collaboration. Consolidating the role of facilitators. Facilitators are integral to the success of the program. Several areas have been identified where some design changes could potentially free up time for the facilitators to focus more on socialization activities to promote more participation in meetings, and better align facilitators with community needs. A major area for review and potential reform is the administrative procedures, with an aim to reduce those that are less necessary or could be handled in other ways, for example through ICT technol- ogy or changes in the MIS. A second area is a review of the cycle for facilitator rotations to coordinate this better with the project cycle to minimize disruptions. Finally, facilitators would benefit from further training in particular areas identified in the study including interpersonal skills, administrative skills, effective meth- ods for encouraging participation, and on identifying and formulating social and economic programs. The creation of an incentive based rewards system for improving facilitators performance has much potential for further motivating results on the ground and is recommended. Enhancing the MIS. The MIS is an impressive and important tool for capturing information on project imple- mentation. The system would benefit from further enhancements such as the inclusion of basic GIS data, built-in validation checks, capturing information when block grants are split among multiple projects, and possible streamlining of the data currently collected. The system could also be further designed to be used for auditing and evaluation purposes. Enhancements could also include further use of ICT to improve ef- ficiency and quality of monitoring. Any changes in the MIS will require extensive training on data collection, data entry and data analysis. Sustaining the quality of infrastructure and maintenance. To ensure continued quality of infrastructure and to promote sustained maintenance, a relatively easy starting point is to prepare guidance manuals and additional training to communities and facilitators on good practices, including how to improve communi- cations between PNPM-Urban implementers and the Department of Public Works (PU) in ensuring links of infrastructure with city level planning and systems. ii) OPTIONS FOR A LONGER TERM ROLE OF THE PNPM URBAN In urban areas, approximately 18 percent of the population are poor or near poor, representing some 20 million people. As the country urbanizes, this number is expected to increase, surpassing rural poverty by the year 2020. Analysis of the urban poor, as discussed in Indonesia: Urban Poverty and Program Review (2012) Indonesia’s PNPM Generasi Program, Final Imap 9 Indonesia: Evaluation of the Urban Community Driven Development Program 21 points to the key characteristics of the urban poor—they tend to have low education levels, work in the in- formal sector with low wages, live in low quality housing, lack tenure security, and have lower access to basic services than the non-poor. Subgroups among the urban poor that are particularly vulnerable include recent urban migrants, child laborers, street children and those living in informal settlements. By region, some two thirds of all urban poor live in Java, though the poverty rate in NT is substantially higher. Focus groups carried out in a sample of poor urban communities across Indonesia identified a number of key challenges for residents. Those most frequently mentioned were inadequate incomes, difficulty in find- ing jobs, and expenses related to schooling. Other themes that came up were poor infrastructure and lack of capital. The Government of Indonesia has a comprehensive poverty reduction strategy with a number of large social assistance programs as well as the PNPM-Urban program. The overall package of programs aimed at the ur- ban poor generally addresses key needs with the exception of microcredit, and a comprehensive approach to urban slums (including addressing land issues). Many of the social programs face challenges with target- ing, including both problems of undercoverage and leakages. As Indonesia urbanizes, the demand for land, basic infrastructure, housing, transport, and social services at the community level will grow enormously. It is likely that many new residents will live, at least temporarily, in informal settlements which currently are not serviced. In this context, a clear policy on informal settle- ments is needed, as well as an increasing role for local governments in delivering services at the local level. In complex urban areas where the need is particularly high, more traditional slum upgrading programs may be more appropriate. Slum upgrading programs incorporate spatial planning and community participation, but are typically focused on providing a set of pre-defined basic services, often linked to city-wide infrastructure planning, and are implemented through local governments. In some countries such programs also include social components that are designed to address problems of employment, crime and violence, youth, and health care. Examples of successful programs in countries such as Brazil, South Africa, and Thailand are useful models. The PNPM Urban has the potential to play an important and growing role in small scale infrastructure and possibly in service delivery over time given the well established structure and relatively well functioning program presence throughout Indonesia. It is well-suited to addressing multisectoral needs and increas- ingly working with local governments. Similar transitions have taken place with some of the social invest- ment programs originating in the late eighties and nineties in a number of countries. As an example, the Jamaica Social Investment Fund (JSIF) began as a traditional social fund in 1996, and has since evolved as an institution which mobilizes resources and channels these to community-based socio-economic infra- structure and social services projects. JSIF falls under the Office of the Prime Minister with a national part- nership between central and local government, and communities and private and public organizations. It has a well established management structure and much experience with safeguard policies, financial management and procurement procedures. It also has MoUs with key partner agencies such as the Na- tional Water Commission, National Solid Waste Managmeent Authority, Rural Electrification Program, local councils and others. Critics of such an approach point to the lack of building local institutional capacity for sustained service delivery. The continued introduction and evaluation of new programs and approaches in the PNPM will allow for the experimentation of changes to meet the evolving needs of the urban poor. Piloting new social and economic programs, spatial planning in slum areas, and strengthening relations with local governments are all important for addressing some of the existing shortcomings of the program. The PNPM-Urban program can serve as a base to mobilize communities, but new programs and policies will be needed to proactively address growing needs. Exploring other countries’ models of local service delivery through knowledge exchange will help to ensure that the country is well placed to respond to the changing needs in urban areas ahead. 22 Policy Note References Berger, N. Glick, P. Perez-Arce, F., Rabinovich, L. and Yoong, J. 2011, Process Evaluation of the Program Nasional Pemberdayaan Masyarakat (PNPM)- Urban, Rand Corporation. Beard, V.A. and A. Dasgupta. 2006. Collective Action and Community Driven Development in Rural and Urban Indonesia. Urban Studies, 43(9): 1451-1468 Dasgupta, A. and Beard, V. A. (2007), Community Driven Development, Collective Action and Elite Capture in Indonesia. Development and Change, 38: 229–249. doi: 10.1111/j.1467-7660.2007.00410.x Devas, Nick. 1981. “Indonesia’s Kampung Improvement Program: An Evaluative Case Study.” Ekistics 286: pp 19-36 Directorate General Cipta Karya Implementation Guide for Community Based Settlement Environment (PLPBK), Ministry of Public Works. Guggenheim, S., T. Wiranto, et al. (2004). Indonesia's Kecamatan Development Program: A Large-Scale Use of Community Development to Reduce Poverty. Scaling Up Poverty Reduction: A Global Learning Process and Conference, Shanghai, China Guggenheim, S. (n.d). Crises and Contradictions: Understanding the Origins of Community Development Project in Indonesia. Jakarta, World Bank. Joint Donor and Government Mission (2008), Gender in Community Driven Development Project: Implication for PNPM Strategy. Working paper on the findings of Joint Donor and Government Mission, (Jakarta: Government of Indonesia, The World Bank, AUSAID, ADB, DFID). Lincoln, Y.S., and E.G. Guba. "Processing Naturalistically Obtained Data." Naturalistic Inquiry (1985): 344-56. Mansuri, G. and V. Rao (2004) ‘Community-Based and -Driven Development: A Critical Review’, The World Bank Research Observer19(1): 1–39 McLauglin, Karrie, Adam Satu, Michael Hoppe (2007) " Indonesia: Kecamatan Development Program. Qualita- tive Impact Evaluation" Mulya, Shiddiq Sarana. (2010). PNPM-P2KP “Community-Driven Infrastructure Development Activities.” The- matic Studies on P2KP Implementation Performance: “Infrastructure.” Synopses Series (2). Schuler, Nina, and Dwiyani, Risye. Rapid Appraisal of the PNPM Neighborhood Development and Poverty Alleviation Partnership Grant Mechanism, 2012, processed. Ochoa, Catalina (2011). The Community Based Neighborhood Development Approach: Evolution, Definition, and Key Aspects Moving Forward. New York, NY: World Bank. Olken, Benjamin A. "Direct Democracy and Local Public Goods: Evidence from a Field Experiment in Indone- sia." American Political Science Review 104.2 (2010): 243-67 Indonesia: Evaluation of the Urban Community Driven Development Program 23 Pradhan, Menno, Vijayendra Rao and Cristina Rosenberg(2010)” The Impact of the Community level activi- ties of the Second Urban Poverty Project (UPP),” draft available at http://www.aiid.org/uploads/File/proj- ects/12_UPP_impact.pdf?PHPSESSID=843e721de6b3e0880444023a29242f1b World Bank Concept of the Poverty Alleviation Partnership Grant for Neighborhood Development – PAPG- ND, World Bank. World Bank (2007). "Gender in Community Driven Development Projects: Implications for PNPM Strategy." Working Paper on the Findings of Joint Donor and Government Mission. World Bank. (2010). Targeting effectiveness of current social assistance programs in Indonesia Poverty Group. World Bank Jakarta Office. World Bank (2009). PNPM Urban – Additional Financing Project Paper, World Bank World Bank (2010). PNPM III - Project Action Document, World Bank Voss, J. (2008). Impact Evaluation of the Second Phase of the Kecamatan Development Project in Indonesia, World Bank. 24 Policy Note THE WORLD BANK