45619 FINAL REPORT RAPID EVALUATION OF GOVERNMENT OF INDONESIA'S COMMUNITY-BASED POVERTY PROGRAMS: RURAL INFRASTRUCTURE October 2007 LP3ES Lembaga Penelitian, Pendidikan dan Penerangan Ekonomi dan Sosial Institute for Social and Economic Research, Education and Information CONTENTS Contents ............................................................................................................ i Tables ............................................................................................................ ii Annex ............................................................................................................iii Abreviations ............................................................................................................iv A. Data Sheet of PPIP ............................................................................................................ v B. Data Sheet of P2MPD vii C. Data Sheet of PMPD ix Executive Summary x Chapter I Introduction 1 Chapter II Evaluation Methodology ............................................................................................. 2 A. Scope of Evaluation ............................................................................................. 3 B. Evaluation Methodology ............................................................................................. 4 C. Location ............................................................................................. 5 Bab III Document Review Analysis ............................................................... 6 A. Program Disign 6 B. Program Management 10 Chapter IV Field Finding and Analysis of Program Evaluation 12 A. Achievement of Result 12 B. Program Evaluation and Quality Qontrol 13 C. Location Selection/Poverty Targetting 14 D. Cost Structure, Effectiveness and Fund Flow 17 E. Leakages and Fund Flow 21 F. Technical Review of Infrastructure 25 F. Community Organization 26 G. Capacity Enhancement 27 H. People Satisfaction of Program 28 I. Program Sustanibility 30 Chapter V Conclusion,Lesson Learnt and Recommendattion 33 A. Conclusion and Lesson Learnt 33 B. Recommendation 36 Appendix TABLES No. Tabel Description Hal 1 Village location and the project evaluated 5 2 Program Achievements 13 Concept/guidelines and facts of the selection of program village 16 3 location 4 Composition of the Allocation of Program Costs 18 5 Composition of Project Financing 18 6 Allocation of Funds for Community 19 7 Cost Structure and Cost Effectiveness 20 8 Financial Audit 22 9 Compatibility of fund flow to community 24 10 Technical criteria and facts of infrastructure constructed 25 11 Community organization 26 12 Program of the enhacement of the capacity of district government 28 and local community 13 People's satisfaction level 30 14 Condition of the continuity of infrastructure 31 ANNEXES Annex 1. List of Institutions/resources person Annex 2. List of project document reviewed Annex 3. Interview and focus group discussion guide Annex 4. Research questions of evaluation Annex 5. List of questions/government interview Annex 6. Interview guide with community groups ABREVIATIONS ADB Asian Development Bank BAWASDA Badan Pengawas Daerah BAPPENAS Badan Perencanaan Pembangunan Nasional MRKP Badan Pemeriksa Keuangan Pembangunan CPMU Central Project Management Unit DIPA Daftar Isian Project Anggaran DSF Decentralized Support Facilities FGD Focus Group Discussion LP3ES Lembaga Penelitian, Pendidikan dan Penerangan Ekonomi dan Sosial LSM Lembaga Swadaya Masyarakat LSPBM Lembaga Simpan Pinjam Berbasis Masyarakat O&P Operasi dan Pemeliharaan PIMPRO Pimpinan proyek PMD Pemberdayaan Masyarakat Desa P2MPD Proyek Pendukung Pemberdayaan Masyarakat dan Pemerintah Daerah PMPD Proyek Pemberdayaan Masyarakat untuk Pembangunan Desa PPIP Proyek Peningkatan Infrastruktur Desa PPMU Provincial Project Management Unit PPN Pajak Pertambahan Nilai POKMAS Kelompok Masyarakat SDM Sumber Daya Manusia TKK Tim Kerja Kabupaten A. PROJECT DATA SHEET PPIP SCOPE DESCRIPTION 1. Name of Program Rural Infrastructure Support Project (RISP) / 2. Department Directorate General of Cipta Karya, Departement of Public Works 3. Contact Person Agoes Widjanarko Director General mmudp@pu.go.id 4. Period June 20, 2006 up to March 31, 2009 5. Source of Fund US$ 50.000.000 (ADB) US$ 5.960.000 (APBN) 6. Objectives Increasing the village community's welfare through the improvement of the poor community for the basic infrastructure at the village level. Some indicators : 1. to provide the village infrastructure based on the community need, good quality and managed by community group it self 2. to increase the community capacity to implement and manage the rural infrastructure 3. to increase the jobs opportunity for community at the village level 4. to increase the local government capacity in the community facilitation for development activity in rural area; 5. To influence the transparent, accountability and sustainability in the rural infrastructure development process. 7. Program Component 1. Infrastructure project to support the community accessibility such as rural road and bridge 2. Infrastructure project to support the community food production such as rural irrigation. 3. Infrastructure project to support the community basic need such as rural water and sanitation 8. Location (province/District) Four Providences (East Java, East Nusa Tenggara, South Sulawesi and South East Sulawesi) at the 48 Districts are follows : 1. Kab. Pacitan 2. Kab. Trenggalek 3. Kab. Madiun 4. Kab. Bangkalan 5. Kab. Pamekasan 6. Kab. Sampang 7. Kab. Bondowoso 8. Kab. Situbondo 9. Kab. Sumba Barat 10. Kab. Sumba Timur 11. Kab. Kupang 12. Kab. Timor Tengah Selatan 13. Kab. Timor Tengah Utara 14. Kab. Belu 15. Kab. Alor 16. Kab. Lembata 17. Kab. Flores Timur 18. Kab. Sikka 19. Kab. Ende 20. Ngada 21. Manggarai 22. Rote Ndao 23. Manggarai Barat 24. Kab. Selayar 25. Kab. Bulukumba 26. Kab. Bantaeng 27. Kab. Jeneponto 28. Kab. Takalar 29. Kab. Sinjai 30. Kab. Pangkajene Kepulauan 31. Kab. Barru 32. Kab. Pinrang 33. Kab. Enrekang 34. Kab. Luwu 35. Kab. Tana Toraja 36. Kab. Luwu Timur 37. Kab. Luwu Utara 38. Kab. Buton 39. Kab. Muna 40. Kab. Konawe 41. Kab. Kolaka 42. Kab. Konawe Selatan 43. Bombana 44. Wakatobi 45. Kolaka Utara 9. Sub-District and Villages 571 Sub-District and covered at the 1.840 villages. B. PROJECT DATA SHEET P2MPD SCOPE DESCRIPTION 1. Name of Program Community and Local Government Support (CLGS) 2. Department Badan Perencanaan Pembangunan Nasional (BAPPENAS) 3. Contact Person Basah Hernowo Director of Human Settlements and Housing eddytk@bappenas.go.id 4. Period March 25,1999 up to March 31, 2005 5. Source of Fund US$ 120.000.000 (ADB) 6. Objectives Develop and strengthen the decentralization process and to support acceleration for recovery of economic impact crisis. The objectives of this programs are : 1. to support the decentralization process, 2. to empowering the urban/rural community to be more involve in the regional development 3. to increase the transparent, controlling and reporting procedure in the development implementation at the district/city level 4. to reduce the poverty situation through increasing the community access for the public services and basic infrastructure 5. To create the job opportunity and support the local economic activities. 6. To increase the basic infrastructure function. 7. Program Component 1. There are two types of infrastructure · Type A: development and improvement of the small scale public infrastructure which was selected, planned and implemented by community it self. The kind of infrastructure such as ; water and sanitation, small irrigation, rural road and bridge, drainage, school building, community market etc · Type B: development and improvement or expanded of the large scale public infrastructure and managed by District/City government. 2. Capacity building related to increase the local government and community in the development planning, implementing and evaluation and monitoring trough management consultant and facilitator. 8. Location Phase I ( Province East Java, West Java, and Central Java) and Phase II ( Province South (provience/District) Sumatera, Lampung and South Sulawesi) and covered at 48 District/City likes : : 1. Kab. Cianjur 2. Kab. Garut 3. Kab. Kuningan 4. Kab. Majalengka 5. Kab. Sukabumi 6. Kab.Tasikmalaya 7. Kota Sukabumi 8. Kab. Banjarnegara 9. Kab. Banyumas 10. Kab. Boyolali 11. Kab. Cilacap 12. Kab. Grobogan 13. Kab. Kebumen 14. Kab. Klaten 15. Kab. Magelang 16. Kab. Purbalingga 17. Kab. Sragen 18. Kab. Wonogiri 19. Kota Magelang 20. Kab. Blitar 21. Kab. Jember 22. Kab. Jombang 23. Kab. Kediri 24. Kab. Lumajang 25. Kab. Madiun 26. Kab. Magetan 27. Kab. Nganjuk 28. Kab. Ngawi 29. Kab. Pacitan 30. Kab. Pamekasan 31. Kab. Ponorogo 32. Kab. Sumenep 33. Kab. Trenggalek 34. Kab. Tulungagung 35. Kota Blitar 36. Kota Kediri 37. Kota Madiun 38. Kab. Musirawas 39. Kab. Musibanyuasin 40. Kab. Ogan Komering Ilir 41. Kab. Ogan Komering Ulu 42. Kota Palembang 43. Kab. Tulang Bawang 44. Kab. Lampung Tengah 45. Kota Bandar Lampung 46. Kab. Bone 47. Kab. Sinjai 48. Kota Makasar 9. Sub-District/Villages - C. PROJECT DATA SHEET PMPD SCOPE DESCRIPTION 1. Name of Program Community Empowerment for Rural Development Program (CERD) 2. Department Directorate General of Village Community Empowerment, Departement of Home Affairs 3. Contact Person Aris Mulyana Project Manager PMPD Jl. Raya Pasar Minggu, Km 19, Jakarta Selatan 12072 62-21-79198111, Fax: 79194240 alili@centrin.net.id, cerd@cerd.or.id 4. Period : 12 Mar 2001 : 30 Jun 2007 5. Source of Fund US$ 65.000.000 (ADB) US$ 50.000.000 (ADB) 6. Objectives To reduce the number of poor community in rural area through improvement the village economic condition and increasing the poor community income, especially for community which was living close with economic growth area. Several steps to achieve the objectives are · Increasing the community capacity in the planning and managing the development activity in their village and increasing the local government apparatus to facilitate the rural development. · Supporting the local investment activity and increasing the linkages among urban and rural area with rural infrastructure development that needed for small scale and micro enterprise development. 7. Program Component 1. Increasing the capacity for decentralization implementing such as institutional development, human resource and provide the village grant. 2. Development of financial institution and economic in rural area that covered community based saving and borrowing unit and small scale and micro enterprise development. 3. Increasing the rural infrastructure such as the connecting infrastructure to growth area, infrastructure to support the village economic and infrastructure for housing 4. Project management and monitoring. 8. Location South Kalimantan, Central Kalimantan East Kalimantan, South East Sulawesi , North Sulawesi (Provience/District), and Central Sulawesi and covered at 13 District/City as follows : 1. Kab. Tanah Laut 2. Kab. Banjar 3. Kab. Pasir 4. Kab. Kutai Kartanegara 5. Kab. Malinau 6. Kab. Barito Selatan 7. Kab. Kapuas 8. Kab. Kendari 9. Kab. Konawe Selatan 10. Kab. Minahasa 11. Kab. Boloon Mongondow 12. Kab. Donggala 13. Kab. Poso 9. Sub-District/Villages 568 villages distrMrsted in 77 kecamatans/subdistricts Executive Summary In 1996, 22.5 million Indonesians or 11.3 percent of the country's total population was poor. After the Asian Economic Crisis, a 1998 survey showed that the number had risen to 49.5 million (24.2 percent). As of 2003, there had been a decline in the total number of poor with 37.4 million Indonesians (17.4 percent of total population) being classified as poor. Despite this decline, poverty is still a critical issue for the Government of Indonesia. The Government has focused efforts on developing rural infrastructure to enhance people's access to markets and social services. There are at least three large-scale, community-based rural infrastructure programs that have been implemented by the Government: first, the Empowerment Program for Local Communities and Regional Governments (P2PMD), which is implemented by the National Development Planning Agency (Bappenas); second, the rural infrastructure improvement program (PPIP) that is the continuation of PKPS-BBM IP and implemented by the Ministry of Public Works (Dirjen Cipta Karya); third, the community empowerment program for rural development (PMPD) carried out by the Ministry of Home Affairs (Dirjen PMD). The Indonesian Government, through Bappenas and with technical assistance from the World Bank, has asked LP3ES to conduct an independent evaluation of these three programs. The evaluation aimed to determine whether the programs have been effective and have achieved the originally stated objectives. Evaluation results are expected to provide input to the government for further developing the National Community Empowerment Program or PNPM. The evaluation was conducted from July­August 2007, using a meta-evaluation approach supported with key informant interviews and field visits. The evaluation covered 10 aspects: achieving results; cost effectiveness and budget structures; fund leakages and fund flows; technical valuation of infrastructure projects; community organization; capacity development; people's satisfaction with the programs; and program continuity. Based on documents reviewed and field visits, including interviews and observations of the completed infrastructure projects, it can be said that all the three programs have achieved the objectives of providing basic infrastructure, particularly roads and bridges. When the programs began, they also helped communities through the provision of temporary jobs and additional income. Nevertheless, LP3ES concludes that the three programs have not significantly improved the condition of poor people. No impact evaluation has been done but interviews with some beneficiaries of the projects indicated that socio-economy profile of poor communities in the project locations has not changed significantly in terms of asset ownership and savings, physical condition of homes, diversification of their livelihood, and the quality of food. Although LP3ES acknowledges that changing the overall condition of impoverished citizens depends on many other variables (e.g. culture, land carrying capacity and policies), it is also important to consider focusing community-based infrastructure programs on developing productive infrastructure that can more effectively contribute to poverty alleviation. Regarding provision of infrastructure, the three programs have: improved people's economic access (in terms of availability of roads to link villages to markets); increased asset (land) values; and improved access to basic infrastructure that ultimately helped improve communication and interaction with other villages. The programs also built crucial infrastructure projects including clean water and sanitation projects. There are mis-targeting issues found in all three programs. For example, there are cases where participating villages were found to be better off than neighboring villages that were not included in the program. Also, the selection process of project beneficiaries provided room for local political elites to intervene and manipulate projects to suit their own interests. Furthermore, village heads tended to use the "even distribution" principle (bagi-rata). Even distribution of funds often meant that the needs of poorer groups were been sacrificed to maintain social cohesion. The optimum achievements of the three programs (PMPD, PPIP and P2MPD) thus are still limited to physical aspects and direct, short-term impacts. Other objectives such as poverty reduction, community and local government empowerment, as well as governance improvement are still far from satisfactory. Regarding community empowerment, for example, many village meetings were treated as formalities to fulfill program disbursement requirements. The genuine involvement of communities in all aspects of program implementation such as management, technical construction, and community operation and maintenance organizations was often still limited. Programs with the highest community participation levels were those where infrastructure projects were built through the community-based approach. When a community's need for a particular infrastructure project is high, community participation levels rise, as do levels of infrastructure usage and maintenance--and thus sustainability. Several important findings from this evaluation can be summarized as follows: a. The basic principles adopted by the programs such as democracy, good governance, transparency, and women's emancipation have not been implemented as planned. At the local government level, the implementation of good governance in the programs is still below expectations and the situation is understood as a learning phase. On the other hand, the process and practices of transparency and accountability have been functioning at the community group (Pokmas) level, albeit still in small numbers and only for certain implementation aspects (such as book-keeping). Conceptually, the three programs have tried to improve participation, transparency and accountability through complaint handling mechanisms for communities. However, the complaint mechanism does not work effectively nor is it managed seriously. There should be one unit specially assigned to handle complaints, including information dissemination, data processing, and follow-up. b. Project implementation duration is generally designed for one year. In fact, in many areas, the projects only have 3-4 months to finalize their implementation. This is caused by the lateness in the issuance of government's budget (DIPA) at the local level. As the result, project activities are mostly oriented to achieving physical targets while ignoring the aspect of strengthening community institutions (Pokmas). Understandably, community empowerment processes took place only to fulfill programs procedures stated in existing documents. Community post-project role such as community participation and responsibility in maintaining the facilities is virtually non-existent. c. Institutional capacity development carried out in the projects still tends to serve administrative purposes and the interests of project management only. As a result, almost all community groups were inactive when the project ended. However, in several cases, women's groups who manage savings and loan programs under PMPD and productive facilities built in PPIP programs are still exist. The project should provide a 3-month transitional period before it ends to give communities the opportunity to prepare and consolidate their resources in order to strengthen their organization and to formulate steps to operate and maintain the facilities built. d. In all programs, funds have been directly channeled to Pokmas. This has positive impact in building trust between the government and the Pokmas. However, there are some cases where the funds received by the Pokmas are lower than that promised. For example, in OKI and Sleman districts (P2MPD). There are also cases where the funds were still charged the value added tax (PPN) despite clear project guidelines stating that the funds are tax free. See, for example, the case of P2MPD project in Sleman district (Yogyakarta). e. The infrastructure projects developed by the three programs are relatively good, particularly the types of facilities constructed and their functions. Although these infrastructure projects were simple and low cost, the community felt strong positive impacts, both economically and socially. f. The technical quality of infrastructure projects is good and meets the technical specifications set by the government. The local people generally did not give an enthusiastic response to the joint cooperation contract systems (KSO) used for developing infrastructure in their areas. But, they respond well to projects managed by the local people themselves (swakelola rakyat). g. Communities are generally satisfied with the benefits and functions of the infrastructure built in their areas, but they are not satisfied with the transparency of budget, fund disbursement, procurement of materials and complaints handling mechanism. h. Procedures of promoting local community's participation were implemented based on existing guidelines, but the quality of the participation remains below expectations. Participation is measured largely by community willingness to participate in construction of infrastructure, while less emphasis is placed on participation in project management processes. Therefore, several decisions are still dominated and influenced by village elites. Meanwhile, women's participation is often limited to meetings. There are, however, some exceptions including the P2PMD program in Sleman where women's participation is relatively high in all aspects of program implementation. i. Provision of information regarding realization of activities is still limited, especially for marginal groups, such as the poor and illiterate. Thus, not all people in the project location understand important issues such as eligibility, beneficiary criteria, and how the project is decided and implemented. So far the programs only provide program board information on the project. j. Maintenance and operation of infrastructure projects is overseen by maintenance groups or KPP. However, most of them are not yet effectively maintaining the infrastructure. Sibatua Village in Pangkep (PPIP) is an exception. Its KPP is effectively maintains infrastructure in the area. The saving and loans groups (LSPBM) of PMPD also seem to be capable of managing their funds and show indications of continuity. k. All projects have monitoring and evaluation systems (M&E) at all levels from the local community up to the district and provincial governments. In some cases, M&E was conducted by independent agencies (the case for P2MPD project). The systems also include mechanisms to handle complaints from project executors. However, in all programs M&E and complaint handling have not been functioning well. Almost no evaluation has been conducted on the programs and very few monitoring reports are documented. Even when conducted, the results of the M&E activities (including supervision and people's complaints) have not been made a standard mechanism to improve overall project quality. In order to improve the implementation and effectiveness of the programs, some recommendations are as follows: a. The Government of Indonesia must solve issues related to getting the government's budget (DIPA) to the local level. The 1-year implementation period in the program's design is crucial to project success. The current reality of a 3­4 month implementation period is inadequate, particularly for programs requiring community participation in all project phases. The coordination between the central and local governments should be increased so that there are no delays in budget funds provided by local government. b. Central government should provide socialization of the project implementation to local government long before the project start date so that the local government will have enough time to integrate the required project costs into the local budget planning system. Socialization to community members at the village level should be carried out through various ways, such as leaflets and posters put at in strategic places (such as village or sub-village offices), so that the community members understand the design and the purpose of the project. c. To better target the poor, all projects should begin with social and economic assessments at the village level. Selection of community proposals should be done by an independent team/institution, not by a project implementer, to better assess their feasibility and potential social and environmental impacts. d. Create a project component specifically for women beyond savings and loan activities. e. Evaluation of the three projects showed that infrastructure projects built by the community are cheaper, functional, better in quality, as well as more sustainable compared than those built by contractors. The government should give wider opportunity to communities to implement simple infrastructure projects, such as irrigation system, jetties and clean water facilities, valued below Rp. 300 million. f. Place competent, high quality facilitators in the field with appropriate experience and skill sets. Quality facilitators help ameliorate elite capture of the decision- making process, ensure that participatory processes function effectively, organizational management runs democratically, and community institutions function in a sustainable manner. g. Capacity building for local government apparatus (i.e. the executing team), village apparatus, and community group (pokmas) is very important. Critical for these groups are good governance principles (transparency, accountability, participation) and rights based approach in development, as well as community-based, self- monitoring and evaluation. h. Poverty reduction programs focusing on village infrastructure should be managed under one system to standardize mechanisms of technical, financial, and community empowerment thus minimizing community confusion over different program requirements, approaches, and methods. In implementation, programs can be adjusted to meet specific needs in the field. CHAPTER I Introduction 1.1. Background The economic crisis that hit Indonesia in 1998 has caused a drastic increase in the number of poor people. This poor condition is marked by the skyrocketing prices of essential goods, high unemployment and a decline in productive undertakings. As of 1996, poor people reached 22.5 million (11.3%), and rose to 49.5 million (24.2%) in 1998. Although the government has developed various poverty reduction programs since the emergence of the crisis, the quantity and quality of poor people have not changed meaningfully. In 2003, poor people reached 37.4 million (17.4%). Efforts made to reduce poverty since Indonesia was ravaged by economic crisis in 1998 include labor intensive projects, food assistance, education, healthcare and productive activities like development of labor force and small enterprises with an aim to increase people's income. The projects include direct cash assistance that addressed fuel price hikes. One general characteristic of these poverty reduction programs is its approach that covers infrastructure and economic development (capital) and has the characteristic of community development. In addition to that, in relation to the issue of governance, the process of carrying out the projects followed the principles of openness, accountability, participation and gender equality. The Indonesian government, through National Development Planning Agency (Bappenas) and with technical support from the World Bank, has asked LP3ES to conduct an independent evaluation in order to determine the effectiveness of poverty reduction programs carried out by related ministries, notably in the infrastructure sector that are implemented with community-based approaches. There are at least 14 large scale programs identified, 3 of which are rural infrastructure programs as follows: Firstly, the community and district government empowerment program (P2PMD), which include investment loans for reducing the impacts of the economic crisis and the improvement of the capacity of village institutions and the district government through the development and reparation of infrastructure and facilities managed by National Development Planning Agency (Bappenas). Secondly, the program for increasing and improving village infrastructure (PPIP), which is the continuation of PKPS-BBM IP that aims to improve the welfare of the village people through the improvement of people's accesses to basic infrastructure service. This program is carried out by the Ministry of Public Works (Director General of Cipta Karya). Thirdly, the community empowerment for village development program (PMPD), which is directed to reduce the poverty among people who live around growth centers through the development of infrastructure and economic enterprises in the framework of the inter-linkages between villages and cities. The Ministry of Home Affairs (Director General of PMD) carried out this program. . 1.2. Purpose As stated in the term of reference (TOR), this evaluation aims to collect data and gather information about the process of project implementation, from planning to evaluation stages. Evaluation includes the benefits and impacts of projects. Evaluation results are expected to provide input for the government in carrying out national program of poverty reduction (National People Empowerment Program, PNPM). Meanwhile, the special objectives of the evaluation are: a. To study and determine which of the three infrastructure programs is effective in achieving their stated objectives. b. To recommend steps for program improvement based on best practices of community development and poverty reduction programs. CHAPTER II EVALUATION METHODOLOGY A. Evaluation Scope Referring to above purposes, the evaluation scopes are as follows: a. Achievements Compatibility with the objectives that were defined based on the results of the internal and external evaluations. The goals were well defined in the framework of achievement of results and program performance. Program goals reflect poverty reduction objectives. Problems faced during the course of program implementation. Reports on program results and output reflect the real situation in the community in terms of the development of accesses to infrastructure, time efficiency, labor absorption and others. b. Program evaluation and quality control Systems of monitoring and evaluation carried out in the program. The program carried out monitoring and evaluation on a regular basis as part of quality control. Types of the evaluation conducted (process, methodology and impact) and the results are reliable. c. Selection of location and poverty targets The selection process of program location followed required standards and gave priority to poor areas. The selection of program beneficiaries was based on the poorest groups of village people. Some considerations were taken into account in the selection of locations that benefited from the program. d. Cost effectiveness and budget structure A picture of the total cost of the whole program. The portions of funding (loan, grant and national level/regional budget. Break-down of the components of program cost (assistance for the community, capacity development, technical and administrative assistance). The budget is quite rational from the aspect of unit and total cost. Cost effectiveness at the field. e. Leakage and fund flow Independent financial audit (BPK or other parties). Audit findings. Funds were used according to stated purposes. Compatibility between funds received by the community and those stated in plan documents. f. Technical review of facilities Compatibility between the construction of facilities built and technical specifications. The functions of infrastructure and facilities built. The quality of facilities at the field. g. Community organization The formation and selection process of community organization. Facilitation activities carried out. Community organization members reflect poor people. Continuity of the roles and functions of community organization in the program. The type of capacity development program for strengthening the existence of community organization. h. Capacity development Types of community capacity development carried out at the local level (district/city government, facilitators and community organizations). The process of carrying out activities (training materials, methods, facilitators and time). The quality of capacity development activities that supports program implementation. i. Satisfaction of the program Transparency in program information. People's knowledge about the program. Availability of information and publication about the program. Meetings of the local people for discussing the program. The level of the local people's and stakeholders' satisfaction of program implementation and program results. j. Program continuity Components of program continuity. Activities are planned by the people and carried out by the people, and aim to encourage program beneficiaries to continue it. Activities of operating and maintaining facilities. Forms of the maintenance of facilities carried out by beneficiaries. B. Evaluation Methodology In order to collect data and information in accordance with above mentioned scope within the limited time-frame, this evaluation uses meta-evaluation approach. Evaluation is conducted by analyzing documents related to the program (implementation guidelines, reports on program progress, evaluation report) and others. In addition to data collection and information gathering, rapid appraisal is also made through focus group discussions that involve the local people who receive the program and direct observation of the infrastructure built and the organization officials and the book-keeping of the community organizations. The gathering of information, the conducting of in-depth interviews and the holding of focused group discussions were made by taking samples of those directly involved in the carrying out of the three projects, including project management, consultants and beneficiaries. The details of evaluation respondents are as follows: · National Level: Project Leaders and National Consultants of P2MPD Project (Bappenas), PPIP Project (Ministry of Public Works, Cipta Karya) and PMPD Project (Ministry of Home Affairs, PMD Direcorate General). · District Level: PMU at Bappeda, Dinas PMD, Dinas Cipta Karya, Local Consultants, TKK (Sekwilda) · Village Level: Kecamatan Government Officials, Village Head, Facilitators, Local Community, Management of Community Organization, The evaluation work was conducted from June 4-July 9, 2007, in 4 phases. Firstly, proposal preparation, the collecting and reviewing of project documents, preparation of questionnaires and evaluation guidelines, and the completion of contract administration and permit obtaining. Secondly, decision on rapid assessment location plans in consultation with the World Bank (DSF), finalization of questionnaires and evaluation guidelines, recruitment and the coaching of local research assistants, and preparation for field visits. Thirdly, data collection and information gathering in 9 villages in 6 districts. Fourthly, data processing and analysis, compilation of study findings, outlining of report drafts, presentation of study results at Bappenas and the World Bank, and the making of final reports. C. Location Due to the limited time and consideration of program locations, the selection of districts to be evaluated was done purposively based on two considerations: the geographical location should represent Java and other islands (eastern regions were not visited due to time constraints); and should represent grouping based on project performance categories of good, average and poor (based on performance on timely disbursement and implementation of construction, timely and quality of reports, and reported quality of infrastructure built). The categorization was done by LP3ES based on information from project executors at Bappenas, Ministry of Home Affairs and Ministry of Public Works. After the districts were selected, the villages were then randomly selected from village lists provided by the project management. Three villages from each project are selected as follows: Tablel 1 : Village location and the project evaluated dievaluasi Location RISP Project CLGS Project PMPD Project OKI District, South 1. Ulak Kemang - Sumatra Village, poor category 2. Desa Suak Batok Village, average catgegory Banjar District, South - Pejambuhan Village, Kalimantan good category Sleman District, - Sidomulyo Village, - Jogyakarta Special good category Territory Bangkalan District, East Mertayasah - - Java r Village, good category Keramat Village, poor category Pangkep District, South Sibatua Village, - - Sulawesi good category Minahasa District, North - - 1. Touliang Village, Sulawesi average category Raringis Village, poor category CHAPTER II PROGRAM CONCEPT AND MANAGEMENT This chapter describes program concepts and program implementation management. Program design and management plan describe the initial concepts of the program and related changes in line with the progress of program implementation. Information on program design and management came from reviewing a number of program documents supported with interview of program management at the national level, as well as field assessment. The documents reviewed include program general guidelines, monitoring guidelines, progress reports and annual reports. In some study locations, the study team also obtained and reviewed financial and audit reports. However, concerning evaluation reports, the study team only found one evaluation for P2MPD (year 2005). The secretariat of other programs tried to convince the study team that program evaluation have been conducted but unfortunately the evaluation reports were not well documented. The Program Design is a summary of the contents of the reviewed reports, which was later compared with facts on the ground. Complete list of reviewed documents can be found in Appendix 7. A. Program Design Purpose and Components The three programs evaluated were designed in response to the impacts of the economic crisis that started in 1997, notably the impacts on poor people. Aside from the emergency purpose, the program is also meant to improve the condition of the local people's economy by using people empowerment approaches. As for P2MPD Program, aside from above mentioned purposes, it also aims to enhance the capacity of regional government officials in the implementation of regional autonomy policies (See Box 2.1). The three programs evaluated were designed to address the impacts of the economic crisis that hit Indonesia in 1997, especially the impacts on poor people. Aside from the emergency purpose, the program is also meant to improve the condition of the people's economy using the approach of community empowerment. As for P2MPD Program, aside from above mentioned purposes, it also aims to enhance the capacity of district government officials in relation to the implementation of regional autonomy policies (See Box 1.1). The three programs have components of infrastructure development, which can be planned, carried out and maintained by the local people. For P2MPD, there is also component of infrastructure development done through cooperation with technical contractors. Aside from the component of infrastructure development, some programs also have component on human resources development for district government officials (PMPD and P2PMD Programs) and on development of financial and economic institutions of the village community (PMPD Program). (See Box 2.1). P2MPD Program was actually a merge of two proposals, namely a decentralization reformation program and a poverty eradication project. The initial design of P2MPD Program contained only two important elements namely reduction of the impacts of the economic/monetary crisis and support for decentralization. However, when Indonesia began to be able to control the impacts of the economic crisis, in the third year of the program, the element of sustainability was added into the P2MPD Program. Box 1.1. Purpose of Program Purpose of PMPD Eradication of poverty in rural areas by improving the condition of the village economy and increasing the income of poor people especially those living near economic growth centers. This can be reached by: · Empowering the village community by enhancing the local people's capacity in planning and managing their village development activities, and increasing the capacity of government officials in facilitating rural village development. · Supporting local investment activities and strengthening the relationships between rural and urban areas by building rural infrastructure and facilities that are needed to develop productivity of small and micro enterprises. Purpose of P2MPD Develop and strengthen the process of the implementation of regional autonomy and support the acceleration of the recovery of crisis impacts. More specifically, the program aims to: 1. Support efforts of administration decentralization 2. Empower village community so it can play an active role in regional development 3. improve transparency procedures, state governance, control, accounting and reporting at the district/city level 4. alleviate poverty by enhancing the access of poor people to basic public service 5. create job opportunities and encourage economic activities at the local level 6. improve the functions of basic infrastructure and facilities Purpose of PPIP Increase village community's welfare through improving poor people access to basic rural infrastructure. Box 1.2. Program Components PMPD Components 1. Capacity enhancement for the decentralization of development implementation, which covers institutional development, human resources development, and provision of grants for the village 2. Development of financial and economic institutions in rural areas, which covers the formation and development of community-based saving and loans institutions (LSPBM) and the development of micro and small enterprises in rural areas. 3. Improvement of rural infrastructure, which covers infrastructure that links villages to growth centers, infrastructure that supports rural economic activities, and residential infrastructure. 4. Management and monitoring P2PMD Components 1. Infrastructure and Facilities Work (2 types) · Type A: build, improve and/or repair of public and small scale infrastructure and facilities that are selected, planned and carried out by the local people, which include clean water facilities, irrigation, roads and local bridges, waste processing facilities, drainage, marketplaces, school buildings, sanitation facilities (like MCK/bathing-washing-toilet structures), water transportation facilities (jetty facilities). · Type B: build, repair and/or enlargement of public infrastructure and facilities whose types are similar to Type A, but of larger scale and are managed by District/City Government. 2. Capacity development in the form of the enhancement of the capacity of District Government and the local community in planning, implementing, monitoring and evaluation by providing management consultants and facilitators. PPIP Components 1. Infrastructure work that supports accessibilities namely rural roads and bridges . 2. Infrastructure work that supports food production namely rural irrigation. 3. Infrastructure work that supports the fulfillment of people's needs namely the provision of drinking water and rural sanitation. Matching Grant PMPD and P2MPD programs require matching grant from the district government with varying amounts and purposes. PMPD Program requires contribution from the district budget that increase annually from 20% for the first year up to 64% for the fourth year. Document review shows that many districts failed to follow this scheme so that the district's portion was reduced to only 44% in the fourth year. Meanwhile, P2MPD requires matching grant from the district government with breakdown as follows: 25% for physical development of Type A, 50% for physical development of Type B, and 10% for project administration. The requirement for fund contribution from the district government on one side can be seen as a method of giving authority to the district government in line with regional autonomy policies and getting the district government more seriously involved in program implementation. On the other hand, however, serious consideration needs to be given to the amounts of contribution from the district government so as to ensure that it will not create a burden on the district government's finances. Based on interviews with program executors, as far as PMPD Program was concerned, there is resistance on the part of DPRD (district parliament). Many districts object to the amounts of contribution they should give so that their contribution was reduced to 44% from 64%. Moreover, the matching grant requirement has reduced district governments' interest in taking part in the program. The initial plan was that PMPD Program will cover 750 villages in 20 provinces, but they were reduced to only 575 villages in 6 provinces. Document reviews showed that the issue of financial contribution from the district government was initially confusing to the district program implementers because the program was new to them. As a result, in the initial phase of program implementation, the district governments and DPRD were late in approving contribution for P2MPD Program. Interview with projects implementers also revealed the same issue. Sequence of Program Implementation As shown in available documents, implementation of the three programs started with activities of preparing the local community though socialization, consultations for the selection of participants, planning and the decision on proposals on the types of facilities to be built. Based on proposals from the local people, construction work was carried out. The sequence of activities was nearly similar between the three programs. They differed only in the intensity of the process. For example, the socialization of PMPD last longer, while for PPIP and P2MPD, the implementation including the socialization only took 3-5 months. For PMPD Program, the sequence of the implementation of program components has become an issue. The initial plan was that the infrastructure component designed to support the local village's access to marketplaces would be done after their economic activities were developed. Consequently, infrastructure would be built one year after the component of people empowerment was done. Document review found that in reality, infrastructure was constructed prior to the development of community empowerment activities. The late provision of consultants was the main reason behind the late implementation of community empowerment component. Field assessment also found that the component of physical development was more emphasized than the component of community development. This situation gave impression that the local people were mobilized when the development of infrastructure started. Program Principles and Approaches The three programs hold relatively similar principles and approaches. The principles are transparency, accountability, participatory, continuity and flexibility. P2MPD Program specifically mentioned the principles of democracy and the involvement of women. Meanwhile, the approaches adopted by the three programs generally cover empowerment, pro-poor, decentralization, partnership, capacity building and integrity. Those principles and approaches tend to be idealistic and still cannot be fully put into practiced. Transparency, for instance, is often understood as merely putting up notice boards about project implementation. Good governance practices actually have been started at the community level. Clear division of tasks and authority among program beneficiaries, good book-keeping that provides information about fund received and disbursement, and meetings of villagers to discuss the development of activities are some examples of the good administration practices at the community level. These have been practiced in a number of locations like in Penjambuan Village (Banjar District), Sidomulyo Village (Sleman District) and Touliang Village (Minahasa Disrict) although still in simple forms and not in a continuous way. This showed that the programs basically have provided a learning process for the local people to practice the principles of openness, accountability and democracy in receiving and managing activities. The principle of empowerment that serves as spirit of the three programs is very essential in the context of the shifting of development paradigm from state-based to community-based. This is not only aims to develop the community's self-reliance in solving problems but also to give the community equal political bargaining power with other parties. Unfortunately, in implementation, the three programs have not optimally pushed forward the process of community empowerment in a substantial way. For example, in the PPIP and P2MPD programs, the involvement of the local people in planning was still artificial because it only aimed to fulfill existing standard procedures. Only a few people took part in that process and in a very short time. B. Program Management The three programs evaluated have similar management patterns that involve government, community and consultants. The executing ministry has management structure in different levels from national level down to the provincial, district, sub-district, village and community levels. The steering committee or coordinating team was formed at the national, provincial and district levels, while at the community level, a group tasked to organize community members was formed to carry out program activities. Because program designs are also linked to the implementation of regional autonomy policies, the presence and functioning of the team at the provincial level tend to be formally administrative while at the district level, the team plays a more intensive role in all program phases. At the district level, District Coordination Team (TKK) chaired by the District Head, District Secretary, or District Department Head with team members from related government agencies is formed. Field findings showed that in general, the performance of TKK is good, especially in coordination and program planning, and in ensuring that the selection process of community proposals went smoothly. Meanwhile, the frequencies of field visits to monitor program progress were very limited. On consultants, there are consultants at the national and district levels, depending on the components of the program implemented. But normally, consultants comprise of community empowerment consultants, consultants for village economic development, and technical consultants. At the sub-district and village levels, facilitators carry out the function of consultant, and their number varies, depending on the program. Field findings show that at P2MPD Program in OKI, 1 facilitator handles up to 5 villages, while at PMPD Program in Banjar, 2 facilitators work hand in hand (technical facilitator and empowerment facilitator) for two villages. The ratio between the number of facilitators and the number of villages handled by them determine the quality of the preparation of the community and the quality of the facilities built. In fact, at the community level, facilitators often spearheads the program as they motivate the local community and hold consultations with them about projects to be carried out in their areas. CHAPTER IV MAIN FINDINGS The main findings of the three programs are presented simultaneously because many similarities were found among the programs. The summary of detailed similarities and differences among the three programs by research focus can be found in Appendix 1. Meanwhile, the summary of the strengths and weaknesses of the three programs can be found in Appendix 2. A. Achievement of Results The evaluation of the general guidelines on program implementation, report documents and evaluation shows that the main purpose of the three programs is to empower the community to develop better infrastructure in rural areas. Other purpose includes increasing the capacity of district government officials in the implementation of regional autonomy policies. These in turns are expected to improve the welfare of rural people and to reduce the number of poor people in rural areas by creating more job opportunities, increasing local investment activities and productivity of small and micro enterprises (especially PMPD). In addition, the programs also intend to build practices of transparency, accountability, and community participation at all levels as part of local good governance. From the guidelines, it can be said that the purposes of the programs are already well formulated in relation to government efforts in addressing economic and social crisis, including the crisis of confidence over government services that are not functional optimally. However, field findings show that there seems to be incompatibility between the purpose stated in the concepts of activities and program implementation on the ground. For example, PPIP Project aims to enhance the capacity of district government officials in facilitating the local community to carry out development in rural villages. However, limited activities for such capacity enhancement make it difficult to push forward for government officials as "development facilitators". Also, programs impacts on poverty reduction, changes of development processes to become more transparent and accountable, and strengthening the implementation of regional autonomy polices have not achieved the expected results. For transparency for example, in P2MPD Project construction activities with contractual models, the information on the amounts of contract value and the types of contract work were not available to the community so that most of them did not know about the activities carried out by contractors. The achievement of results on these aspects was seen more as a learning process by the program implementers. Nonetheless, the achievement on rural infrastructure development/construction is considered good judging from the physical aspects of the facilities/infrastructure built/repaired (number and quality) and from the aspect of creating temporary job opportunities for poor people in rural areas during the economic crisis. The programs are also quite good in getting community to participate in the construction of facilities and infrastructure ­ especially those using community-based approach. Field observation conducted in 9 villages of the programs shows that infrastructure had been built in the areas and had created job opportunities for the local poor people. For example, in PPIP Project in Bangkalan District, the jetty facility built in the areas has created jobs for the local people although it was limited only on the time of construction. Also, with the operation of the jetty, fishermen no longer found it difficult to park their boats and were no longer worried that they boats would be washed away. PMPD Project is slightly better in achievement compare to P2MPD and PPIP in the aspect of the involvement of women and poverty reduction (with the operation of saving and loans program). Table 2 showed several achievement variables in terms of discrepancies between the results of document review and facts on the ground. Table 2: Program Achievements No. Variable Document Review Field Fact 1 Objective Reduce the number of poor Achievements of programs people, and increase poor implementation are not specifically people's and nearly poor targeted for poor or nearly poor people's accesses to basic people, but for the general infrastructure in rural areas community in the villages. Objectives met are the availability of basic infrastructure in the village and the availability of job opportunities for the village people. 2 Project functions The project function is to - Most infrastructures built are and general support the acceleration of the functioning, although there problems development of villages by are cases where the quality is infrastructure provision. not so good - Usually, only the construction Problems generally faced by of infrastructure that can the programs are: provide job opportunities. -community empowerment - The building of infrastructure - community's technical cannot accommodate all poor capabilities in construction people as workers - Decision making is still dominated by village elites and the strengthening of the roles of community groups is still less due to limited time 3 Increasing of Program reports explain that - Access increased in locations access to basic the programs are quite where people chose to built infrastructures, successful in enhancing poor roads and bridges. But, for reduction of time people's accesses to all types other infrastructure such as needed to access of infrastructure. This has had clean water and irrigation, facilities, and impact on the improvement of only a few have been built increase in job the social and economic and as such, the increase in opportunities. condition of the local people. access is still limited. - Job opportunities increased only during construction times (1-2 months). Field assessment showed that achievements in the implementation of rural infrastructure program are also determined by the length of time for implementing activities. For example, PMPD Project was completed in 2 years, while PPIP and P2MPD projects were finished in one year, respectively. Obviously, one year is not enough to have significant impacts at the community level, especially on the capacity and dynamism of community organizations (kelompok masyarakat, pokmas). Empowerment efforts require a relatively long time before the results and benefits can be seen. This is even worse when the project was completed in only 3-4 months (for example, PPIP in Bangkalan and Pangkep, P2MPD in OKI). As the result, programs performance became un-satisfactory compared with their very high goals, which are often cannot be easily translated into clear agenda with specific activities. The same thing happened with the achievements of the component of enhancing the capacity of government officials and the strengthening of Civil Society Organization (CSO) at PMPD Project. Aside from the fact that in general the criteria of achievements are not that clear, differences in the time period for project completion clearly also determined the quality of project completion activities. Concerning the aspect of activities that aim to reduce poverty in the rural community, in general there is no report that explicitly and quantitatively points to a certain number of people who have been alleviated from poverty as the results of the completion of the three projects. Concerning the involvement of poor villagers in the projects, not all of them are ready to work following contractors' models that provide low wages while work load is big and working rules are strict; so many of them refused to work for the projects. This situation hampered the success of the projects especially in terms of the job provision for local community and the improvement of their economy. B. Program Evaluation and Quality Control The three programs evaluated were completed with monitoring and evaluation systems. The systems aim to provide knowledge about the progress of project activities on the ground and the problems encountered. Monitoring and evaluation results are expected to provide a feedback to improve the performance of the program. General guidelines of all programs state that monitoring and evaluation are conducted structurally, functionally and in participatory manner. Structural monitoring is conducted by government officials who are in charge of the program from the national, provincial and district levels. Functional monitoring is conducted by consultants from the national and regional levels. Meanwhile, participatory monitoring is expected to be carried out by stakeholders and the general public through groups formed by the community who benefits from the program. According to the guidelines, monitoring is regularly conducted, usually on a monthly basis at the district level, three-monthly at the provincial level, and every semester at the national level. Field findings show that functional monitoring by consultants has been carried out quite well. Structural monitoring by government officials was also carried out but not as regularly as stipulated in the guidelines. Field findings in the case of OKI District for example show that TKK has never conducted field monitoring while the project officer conducted field monitoring when the physical work was underway. The project officer's field visit was meant to check the latest progress of project execution before the next disbursement of project fund was made. The reason cited for why monitoring was not adequately conducted was that there is no adequate fund allocated for it. The project officer preferred to trust facilitators to carry out monitoring task. However, progress reports on the project were submitted periodically because such reporting determines the next phases of fund disbursement of the projects. In fact, progress reports on the project were prepared by consultants/facilitators. At the community level, participatory monitoring was not optimally conducted due to the low capacity of the local people to carry out monitoring work. Facilitators play leading roles in monitoring. Meanwhile, the program also makes evaluation to measure the effectiveness, efficiency and impacts of program implementation. Evaluation scheduled for mid-term and end of project will be based on the indicators stated in the logical framework matrixes. However, as stated in Chapter II, only one evaluation report was available, namely evaluation report on the final completion of P2MPD Program (2005). The independent evaluation of P2MPD was made in relation to the completion of the project. The evaluation was made to appraise the performance of the project, which was compared with the plan of CLGS/P2MPD Project Phase II. More specifically, this evaluation aims to identify the strengths and weaknesses of project design and implementation, propose some issues that requires attention for planning the next step of the project, appraise social and economic changes and existing regulations that can impact the design of the next step of the project. The evaluation was conducted by reviewing reports and by conducting interviews with some parties related to the project. Unfortunately, due to limited funding and time, the evaluation did not consist of field visits. With such a method of evaluation, the results mostly refer to project documents and the opinions of project executors. From the aspect of methodology, this is a weakness because, regardless of reasons, field confirmation is necessary to check the validity of the situation reported. Another weakness of the report is the fact that it does not include the lists of the documents reviewed and the lists of people interviewed. With such weaknesses, the credibility of the evaluation report is questionable. C. Location Selection and Poverty Targeting In accordance with the direction and purpose of the program in poverty alleviation, it was decided that the programs locations (districts and villages) should be under-developed, isolated or marginalized areas from both infrastructure availability and social and economic condition. Almost all projects set criteria for location selection as follows: · Areas most severely affected by the crisis · Areas with relatively high number of poor people (families) · Areas where no similar project (PPK, P2KP and others) has been carried out However, some projects set additional criteria. For example, PMPD Project gives priority to villages with economic potentials or located near growth centers or already have economic links with such centers. Meanwhile, PPIP Project focuses on villages in disadvantaged areas as decided by the Governor/District Head based on decree from the Minister for the Acceleration of the Development of Disadvantaged Areas (Pengembangan Daerah Tertinggal, PTD). The selection of locations was conducted in phases at all levels with each level having the authority to make decision. For almost all projects, the Central Government decided the districts project locations while taking into account input from ADB (in the case of PPIP and PMPD projects). Meanwhile, the District Head decided sub districts/villages project locations with consultation with related district agencies and sub district heads. Agreed villages were then stated in the decision letter (SK) of the District Head. For PPIP projects, villages are decided with SK from the Minister of Public Works No. 241/A/KPTS/M/2006 after being approved by DPR (House of Representatives). In its later development, the selection of location villages were discussed and agreed on with the State Minister of PTD. This selection process involving all levels is basically quite positive because it provides districts with authority to select location according to their capacity. This allows for greater compatibility between selected locations and the criteria set. However, the Central Government's authority to make final decision on project locations can cause villages proposed by districts to be rejected and replaced, although the number of such cases has not been significant. This happened for example in PPIP Project in Bangkalan District. The project officer said that three villages had been proposed by the District Government as project locations in 2006, but they were not accepted by the Central Government. This happened due to differences in the criteria of disadvantaged areas between the district government and the Central Government. All three projects have problems where the criteria of village/location selection have not been followed, notably the criteria of villages with big population or with big numbers of poor families and villages that have not received similar assistance programs. The majority of program villages only match one of the criteria; some even did not fulfill any criteria at all. (e.g. P2MPD Project in Ulak Kemang Village, OKI). Some detail findings from document reviews and field assessment on selection of location can be seen in Table 3. Table 3: Findings on the selection of program location Variable Document Review Field Fact 1. Location Criteria Areas most seriously affected Areas affected by the impacts by the crisis of economic crisis The number of poor villagers There are poorer areas/villages (families) is quite big that were not selected Does not have similar projects Location village already from other organizations (PPK, received other program of P2KP, and others) similar type Program location area is relatively developed 2. Program Not specifically mention Most program receivers are the beneficiaries criteria criteria of program general population. Some poor beneficiaries. Only PMPD people are involved as workers Project defines poor people in the programs and women as beneficiaries of Women play quite big roles in the program. the implementation of saving and loans program (PMPD) and in the building and management of infrastructure (P2MPD) The selection of program locations based on poverty criteria cannot be applied consistently in all programs. For instance, P2MPD Project in Sleman District. Sidumulyo Village with 18.6% poor households was selected to be program location for Type A and Type B facilities. Meanwhile, Sidoarum Village with 33.28% poor households was not selected. The selection of poor beneficiaries was often also not easy due to social (self-reliance potentials) and political considerations (pressures from elites and vested interests). For example, in OKI District - which has been split into two districts (OKI and Ogan Komering Ulu) - in South Sumatra, around 50% of program target villages actually were not from the poor category. In the context of the types of physical facilities built and their benefits for the poor, the majority of the local people preferred roads and bridges although there are also some social and economic facilities like clean water tanks, boat berthing facilities, irrigation systems and others. Community argued that roads did not cause many crucial technical problems. Also, they viewed that roads gave many more benefits and functions to all community groups compared to irrigation facilities that are more beneficial to land/capital owners (although irrigation also contributed to the rise in agricultural production and thus helping agricultural workers). Admittedly, roads/bridges built did not directly address the needs of the poorest villagers considering that they normally did not always use roads as means to increase their economic activities. Such a situation is found in nearly all locations of the three projects that were evaluated. Based on the results of interviews and field visits, several constraints and problems are encountered in the selection of locations are as follows: · Difference in the perception and identity of disadvantaged villages between the regional government and the central government caused the central government to turn down the regional proposals because they viewed that the villages selected were not from the poor category. · Collusion and conflicts of interest between elite groups from the district and the village remained strong so that the selection of program villages often did not follow existing guidelines. This issue included support from local political elites (DPRD members) in the selection of locations, mainly in their constituency areas. · Village elites remained dominant in village consultations that decided project implementation. Village elites often avoided conflicts among local people that might caused by the projects by driving the community to choose types of infrastructure that would ensure equal distribution of facilities so that the facilities would be utilized by general public and not by certain groups of people. D. Cost Structure, Effectiveness and Fund Flow The cost structure of the fund allocation is directed to finance the building of infrastructure and project management activities. For PMPD Project, it is also directed to finance the development of the community's saving and loans groups. Infrastructure building cost is the cost allocated for building physical facilities that is carried out by the local people (self- management) and by contractors. Meanwhile, management cost includes the cost for consultants, trainings, facilitators, and surveys that help to smooth the whole process of community development. Yet, funds were not allocated to cover the management cost after the project was built and the project management was to be transferred to the local people. In fact, the cost structure related to the components differs in terms of proportion. The biggest allocation goes to infrastructure development. Meanwhile, the allocation of management cost as compared with total costs of the program cannot be easily verified because the characteristics and configuration of management activities of each program differ significantly. However, it was found in program documents that the allocation of management costs varied from 2-12% for the three programs (Table 4). Table 4: Composition of the Allocation of Program Costs Proportion of the Project Costs of Infrastructure Proportion of Proportion of Building Management Costs others PMPD 44.8% 12 % 33.2% P2MPD 93 % 7 % - PPIP 98 % 2 % - The other proportion for PMPD Program is the allocation for financing development of saving and loans institutions (LSPBM) and for the empowerment of government officials through financial assistance for formal education (scholarships). The realization of the empowerment of government officials has been used to provide scholarships for District Government officials, from Diploma programs to undergraduate (S1 and S2) programs. In general, the reporting on the realization of cost allocation is not clearly made, notably in the reporting of the realization of government budget funds. This is because the some of the costs allocation items in the projects cannot be found in the usual district government budget allocation items. For example, in Minahasa District, most types of activities related to PMPD Program are stated in the account of "Other Spending". This could provide incorrect information for those who use such financial reports. Composition of Project Financing The loan schemes require that, for any allocation of funds for each program, the District Government must provide matching grant. The financing proportions plan that comprise of loan sources and APBD is directed to give participatory responsibility to the District Government to support project execution. The general proportions are: 60 - 80% from loan and 20 ­ 40% from APBD. The composition of the required matching grant as follow: Table 5: Composition of Project Financing Project Classification Proportion of Loan Proportion of Funds Matching Grant PMPD Year I - III 80 % 20 % Year IV ­ VI 56 % 44 % P2MPD Type A 69 % 31 % Type B 50 % 50 % Project 0 % 100 % Administration PPIP 100 % 0 % Meanwhile, since PPIP Program that does not require matching grant, the District Government did not provide such funds. Furthermore, the program provides operational costs for the District team that reached around 2% of the total allocated fund. In field visits and interviews, a number of problems arise concerning the mobilization of matching grants: · The time of drafting budgets in the regions was adjusted with the process of planning costs of each district. For Minahasa District, for example, accompaniment from 2001 could not be realized because the district budget had been ratified. The allocation of loans could be realized in the next budget year. · Several district governments have other needs that are more urgent, or they have limited financial capacity so that they cannot provide accompaniment funds or their amounts are lower than planned. Allocation to the Community Operational Guidelines of each project stress that assistance is channeled to the local community in the form of activities so that the local people can enjoy the process and outcome of project execution, including the building of infrastructure and the formation of financial institutions at the village level (PMPD). Fund allocation for infrastructure in the three programs for each village is as follows: Table 6: Allocation of Funds for Community Allocation of Funds to be Project managed by Community (per Note Village) * PPIP Rp 250 million P2MPD Rp 75 million Other than projects carried out by contractors PMPD Rp 50 million p Rp 150 million Other than assistance for LSPBM Although fund allocation for the community has been made, the process of considering allocation amounts is not clear. Even, reports from auditors say Sleman District Government, in carrying out P2PMD, did not make calculations about the allocation for each village and the district government only distrMrstes funds in equal amounts to a number of villages that had been decided. In addition, a 10%-value added tax was imposed on allocation funds received by the local people although existing operational guidelines stated that funds received by the local people were not subject to taxation. In connection with the development of LSPBM, the greater portion of funds is allocated for its formation and development. Meanwhile, the allocation of funds for capital development is less equally distributed among LSPBM organizations. In fact, the documents of PMPD Project state that each group is provided with Rp 30 ­ 50 million depending on the preparedness of community groups that cover the formation of the groups, their administration and the initial savings of their members. However, facts on the ground are different as evidenced in two location villages in Minahasa District. Although the villages can develop their assets to reach Rp 50 million each, they did not receive allocation funds for their capital development. Rationality of cost unit and activity In the three programs, clear guidelines about cost unit are not found, which are to be used to outline activity costs. However, the process of planning the infrastructure development project that involves facilitators, especially technical facilitators, has been helpful in the designing of the physical plan of the infrastructure and its cost plan. In addition, guidelines for applying cost units are clearly stated only in PPIP Program, and are not found in other programs. That is why, for the two programs, the setting of cost units uses prevailing district standards. Field findings show that the local people take an active part in the drafting of budget plan (RAB) together with facilitators. Facilitators also complete the physical planning of infrastructure. In general, local mass organizations know about cost standards based on local market prices. In OKI District (P2MPD Project), it was found that wage price units were purposely raised on fields that it aimed to increase the local people's buying power. The same thing was found in Minahasa District, on fields that the move was meant to keep up with the rising prices of raw materials. The rising costs made the local people understand problems around the unmatched budget, and then they become able to criticize cost units of APBD projects in their village as expressed in a discussion forum in OKI and Minahasa districts. It is said that cost standards are not available at each project. As a result, in a district, due to the absence of same cost reference, the outputs of activities differ although the input of the cost of activity implementation is the same. Guidelines for applying cost units are clearly stated in PPIP Program, but they are not found in other programs. A similar thing is revealed by MRKP findings on the absence of operational directives (juklak) and technical directives (juknis) in P2MPD Program. Table 7: Cost Structure and Cost Effectiveness Variable Concept Field Finding Rationality of Cost Units · Clear guidelines on cost · Cost standards are and Activity Costs units for drafting plans of decided using district activity costs regulations or market · Stipulation of cost units prices uses prevailing distgrict · Facilitators help in the regulations. planning of projects of · Planning of the project of infrastructure infrastructure development development and in the involves facilitators, drafting of project budget. notably technical facilitators. Cost Effectiveness · Cost effectiveness is · Development carried out by measured with the amounts the community is genraly spent and the utility and highly high cost-effective. , impacts of the project Output volumes are bigger developed. . than the original plan as compared wsith the types of project execution through contracts with third parties. Economic impacts on poor people as the party who carries out project development and receives it. Cost Effectiveness Cost effectiveness is the relationship between activity output and the input. This is closely related to the need for information about activity cost standards. Cost effectiveness is measured between the cost and the utilities and impacts of the project. The development of infrastructure, especially with resources from the local people, has high cost effectiveness because output volumes are much bigger than the initial plan as compared with the types of project implementation that are contractual and involve third parties, and also has economic impacts on poor people who carry out and benefit the project. This is visible in the case of PMPD in Banjar District. The LSP-BM office was initially planned to be made of wood and measure 4 x 4 meters, have iron sheet roof and cemented floor was changed into a permanent brick structure measuring 4 x 6 m with cemented roof, ceramic floor, and the project was estimated at Rp 31,000,000. The same thing happened with the plan of building of a rice barn measuring 6 x 6 m, valued at around Rp 46,000,000, which was changed into a structure of 8 x 6 m, valued at around Rp 55,000,000. Project effectiveness is not only seen from the aspect of costs being compared with output but also from the level of outcome as found in the PPIP case in Pangkep District. There, the building of a wooden bridge was planned to use supporting wooden pillars of 12 x 12 cm, but in realization they were 14 x 14 cm so that procurement of the items increased by 30% and the money used to buy them came from the local people. The project of improving roads leading to fish ponds has economic impacts on the local people because the frequencies of harvesting from the ponds have increased. Also, the area of ponds that can be cultivated has increased. E. Leakages and Fund Flow Mechanisms of Fund Channeling Formally, the scheme and mechanism of the three projects basically adopt the model of de- concentration funds, for which the management and documentation of budget planning, decision on fund allocation and final accountability lie in the hands of the national level project leader. In the meantime, in the provincial/district level, people in charge of the projects are appointed, who carry out administrative and financial work according to their job description. Guidelines, systems and procedures of the three projects were decided based on a Circular from the Director General of Treasury at the Ministry of Finance. Meanwhile, the financing systems and procedures for accompaniment funds from APBD follow the rules stated in Government Regulation No. 15 Year 2000 on the Management and Accountability of Regional Government Finances and the Decree of the Minister of Home Affairs No. 29 year 2002, whose implementation processes were improved this year with the issuing of the Decree of Minister of Home Affairs No. 13 year 2006. In the informal context and in the absence of government regulations, funds for realizing activities are disbursed to mass organizations through their bank accounts or the accounts of community organizations, with guidance from facilitators and with supervision that follows Program mechanisms. Document papers of transactions for budget realization are kept at mass organizations as documentation and as objects of the internal control of the Program. Aside from the development of LDPBM at PMPD projects, there is no technical guidelines for the financial management of the projects at community groups. But, in some districts, the local people conducted financial management and submitted reports periodically, which they did with high awareness. For the continuation of projects that support the principle of transparency, it is advisable that guidelines for financial management also need to be regulated down to the community level. In relation to the channeling of funds to the community, there is another finding namely the difference between documents on the guidelines of project implementation and their real implementation on the field, where the payment of foreign assistance loans are subject to value added tax (PPN). This was found in OKI and Sleman districts, which carry out P2PMD Project. As a matter of fact, the guidelines book states that payment through foreign assistance loans is not subject to PPN. In addition, field interviews also show that the disbursement of assistance loans to the community is not in accordance withy agreements, as found in OKI and Sleman districts. In OKI, the local people informed that funds worth 37 million that should have been channeled to the community turned out to be only Rp 25 million. As projects managed by the government, fund management systems for the three projects follow stipulated budget rules/policies. In implementation, almost no fund manipulation was found in the implementation of budget management procedures. It is only that, in fact, abiding by the procedures of government budget has different impacts in Bangkalan District and Pangkep District concerning PPIP Project. Because there was deadline for fund disbursement in the 2006 budget year, Bangkalan District Government accelerated the implementation of program activities. Limited time (from the time budget funds were disbursed) caused project implementation was hurried up so that the quality of project work was not good. Meanwhile, Pangkep District Government delayed the withdrawal of funds in the next fiscal year, which caused a delay in the completion of projects activities in the village. As for OKI Regency, assistance loans were already prepared in 2003 while accompaniment loans were ready only in 2004, which caused project work started only in that year. As a matter of fact, consultants started working in 2003. The project officer made an additional agreement (addendum) with consultants namely extending the working period of consultants but their fees were not raised. Funds were channeled to the community by transferring them from the state accounts or district government accounts to the accounts of community groups. But, in the process of MRKP audit, in Pengkep District, it was found that after the funds were transferred to the accounts of community groups, some people of the groups immediately withdrew the funds. Although their action did not breach existing rules, there are worries about the safety of placing funds at banks and irregular use of the funds at the community level. Financial Audit Internal control of the project is the control provided by the project management, at national and district levels. In general, internal control is the task of the working unit namely District Control Agency (Bawasda). Project officers said they had not found extreme irregularities like corruption and manipulation. Control is also conducted by community groups, generally through informal control, which means that community members can conduct control directly and raise questions to community groups that have been formed. Especially for P2MPD Program, control by community groups is conducted through a special team that is given the task to control. Documents of implementation guidelines of each program also state that external audit is conducted by MRKP. MRKP not only carries out financial audit, but also operational audit. Meanwhile, the procedures of managing district finances include external audit by MRK so that in a random way each project in some districts can be become the object of MRK audit. Table 8: Financial Audit Variable Concept Field finding Internal Audit and · Internal control of the project · Internal control of the project Control l is conducted by District is conducted by Bawasda. . Control Agfency (Bawasda) · Internal control by Control is also conducted by community components also community components has been carried out. Independent audit is · But, follow up measures for conducted by MRKP. control results are not clear. · Audist is also conducted by · Results of MRKP audit in MRK (random audit). Minahasa last year show that work volumes are lower than they should be. The finding of MRKP audit in Pangkep District shows that some groups have taken the greatst portion of money after funds were transferred to their village's bank accounts from state accounts or distgrict government accounts. Internal control have been carried out by each district, and the internal control has been regularly conducted by District Control Agency. Meanwhile, internal control by the community also has been practiced. However, follow up measures on the internal control remain unclear. This is why internal control appears to be merely an administrative formality. Several findings of the audit conducted by Bawasda (Distri9ct Audit Agency) are as follows: a. In Minahasa District (PMPD): The volume of the work carried out by contractors is less than the formally required, and accordingly contractors are obliged to return funds to the government b. In Bangkalan District (PPIP): Agreement with the local community does not contain description about the costs of each activity c. The district government does not prepare Operational Directives and Technical Directives to put into practice General Guidelines d. The district government does not conduct analysis of fund allocation in order to set the highest ceiling of the funds allocated to the villages, and this is because the district government only set average allocation amounts to all the villages. e. Administrative work and financial report are not properly done, which is evidenced by: · The working on the general book-keeping does follow existing rules · Financial administrative work of Type A Project in the program receiving village is not strictly carried out. · Of the 54 villages, only 33 produced accountability reports · Financial report (Annual Plan and Actual Project Expenditure and Financial and Cumulative Project Expenditure and Financing) was not produced by the project management Compatibility of the Using of Funds The aim of each program is empower poor people, through infrastructure development or other forms of projects. Each program cannels funds to target villages in the form of project financing, through community groups or contractors. The funds channeled were based on proposals from the local community that comprised the objective, the type of the facilities constructed, construction design and the breakdown of the funds required. It finds that funds had been distrMrsted and used according to plans originally agreed on. The disbursement of funds through community groups has given additional benefits in the form of extra income of poor villagers, especially those involved in the project development process and receiving wages as laborers. The increase in their income was also due to their undertakings following the completion of the infrastructure project. This situation is found in Pangkep District, where, after the building of a wooden bridge and the improvement of area roads were completed, the small undertakings of local villagers as shrimp/fish pond farmers further developed. A similar situation is also found in Bangkalan District (processing of marine products) and Sleman District (small trading). Meanwhile, the using of funds by the local community through the scheme of financial institution development went according to plan. All villages that received LSPBM development funds in PMPD Program have succeeded to form financial institutions, and even the project location village in Minahasa District saw its assets rising by Rp 50 million after two years. LSPBM members also can use loans to develop their businesses and fulfill the needs of other families. Compatibility between Funds Going to Local People and the Budget Each project has clear budget structure: the amounts channeling to the local villagers and those going to other components. The amounts to be channeled to the villages are stated in different documents, while those going to community groups are stated in the agreement between the project management and the community groups. The agreement states the amounts to be channeled, and the procedures and requirements of fund channeling. Table 9: Compatibility of fund flow to community Variable Concept Field finding Using of fund in · Project purpose is empower · Funds have been channeled relation to project poor people through and used according to agreed purpose infrastructure development and plan others · Fund disbursement through · Channel funds int he form of community groups can help project financing, through increase the income of poor community groups or through people. contractors · Another impact is felt through · With PPMD Project, the rise in the undertakings of channeling of funds is also poor people after the project is directed in the form of the completed development of saving- borrowing institutions (LSPBM) Compatibility · It is clarified: the amounts ·The amounts channeled are in between Channeling channeled to the community accordance with plan The of Funds to and the amounts going to other amounts received by Community and components community groups in OKI and Budget · The amounts channeled to the Sleman districts are smaller village are put into different than the agreement because documents transaction of the payment is · The amounts channeled to the made tax object (PPN) Aside community are put into from tax obligation, in OKI agreement between project there are also other cuts so that officer and community groups the community recieve smaller amounts It was found that the level of compatibility between the funds received by the local community and the budget decided was quite high at nearly all programs. The cutting of the amounts of funds received by the local community was mainly caused by the reduction for Value Added Tax (PPN). The local people questioned this matter, arguing that community groups receiving such funds are not business enterprises like contractors. The value of the reduction amounts could be small, but the cutting of funds at different rates as happened with PMPD Program in OKI District has caused the project fail in the sense that the facilities constructed did not follow technical designs that had been decided (output and quality). In other words, corruption and less transparency have given quite big contrMrstion to the failure in the infrastructure development. F. Technical Review of Infrastructure The development of basic infrastructure carried out through that three programs essentially aims to enhance local poor people's accesses to economic resources (roads, bridges, irrigation). Meanwhile, their basic needs for god health is tried to be fulfilled by the development of clean water facilities, toilets and bathrooms (MCK) and others. In addition, the infrastructure program also serves as a lesson for the local people to learn about development from, by and for the people, which supports self-reliance of the local community. In such a learning process, all necessary infrastructures have to be constructed according to required quality and technical standards. The development of the basic infrastructure requires simple technical capabilities but still it has to refer to quality standards and technical criteria that have been stipulated. Technical criteria for village roads, for instance, are that they go with capacity designs so that they can be passed by vehicles with carrying capacity of 5 tons, which transport local people's products, and the roads are 3 meters wide at the minimum. Technical criteria for connecting roads or roads for farm businesses are that they are meant for simple vehicles like two- wheeled buggies, mini trucks, motorcycles with carrying capacity of 3 tons maximum. Technical criteria for clean water supply are that the water tanks have capacity to take in water enough for 10 poor families per day at the minimum or 3000p5000 liters, and water quality must meet the standards of water quality stipulated by the Ministry of health or WHO (See Table 9). All the technical criteria must be included in the concepts and documents that take the form of guidelines of the implementation of construction techniques. The guidelines also state the columns of project work and the unit price as well as total prices of all technical work. In the process of carrying out the community-based construction work, quality orientation and the level of the functioning of the infrastructure constructed are one of the outcome criteria in the rural infrastructure projects. Table 11 shows the results of field findings about the technical quality of facilities constructed in the three projects. Table 10 : Technical criteria and facts of infrastructure constructed Variabel Concept (guidelines) Field Fact 1. Type, quality and volume - Road/bride: design of - In relation to the load of plans, of infrastructure maximum load 5 tons, not all projects give atention to minimu width 3 m. Materials: it, river rocks, cement, sand and - The local people less understand baako asphalt and others that standards of material quality meet SNI standards. Wood - But, the compatibility with the meets SNI standard. Volume: volumes of work of rroad/bridge min width 3 m and length bulding is very good. . follows design in the design - Project costs are relatively the document. same namely Rp 312 million for - Clean Water: Criteria design P2PMD Project in Sidomulyo and the quality of dringking Village, Sleman and Rp 250 water tanks 3000 liters, million for Kramat Village, diameter of pipes and their Bangkalan, but quality length follow "bestek" specifications for P2MPD are documents, teh quality is in stricter than PPIP. accordance with SNI standard. - The quality of materials used by Water quality must follow the local people is not strictly standards of the Ministry of examined, excluding those stated Health or WHO. in labels. - Irrigation Channel: Desaign - Concerning compatibility with of the quality of materials volume and time of work, follows SNI standard. Volume generally they go with provisions of Work: width of channels is in "bestek" documents. stated in "bestek: documents. - Even, the qulity of materials is - Boat berthing facility: examined visually only, because Quality design must follow tghere is no detailed report on SNI Standard, example: wood, that matter . cement, brick iron and others. - The absence of strict spefications Volume of work must be in and do not always follow accordance with "bestek: existing rules of specifications documents. Storage/Rice (District's Public Works Barn: design of building Offcie/Dinas PU). materials must follow the - The development of PPIP quality of SNI standard, example: bricks, cement, sand, projects like roads tends to be wood, roof-tile and others. flexible and is adapated to the Volume of work must go with condition and skills of the local the "bestek" document. people. The above table clearly shows that that infrastructure development carried out in the 3 programs basically goes with the volumes and timeframe for project implementation as stated in the guidelines. However, serious attention is not given to the requirements for quality materials used for infrastructure development projects, including projects carried out the local people themselves or those carried out by contractors. From the discussions with the local people and field observations, several technical problems are identified in the infrastructure development and they were caused by several factors including: · Field Facilitators/Consultants do not have enough sensitivity to the quality of infrastructure despite the fact that they have backfield of civil, technical education. · The local people do not have knowledge and skills that are adequate to read pictures/technical specifications and the need for improving the quality of infrastructure and materials used to build them. Regarding the volumes of project work and the time schedule for completing it, several villages are found to have build infrastructure with quality and quality exceeding stated targets, like the PPIP project in Sibatua Village in Pangkep District, where the local people succeeded to build a connecting bridge between village, which are two times longer than stated in the initial plan, with their own resources. However, some villages have not succeeded to build quality infrastructure, including Raringis Village in Minahasa District (PMPD). The village planned to build a relatively long bridge worth 40 million, but later they constructed a small bridge and roadside ditches. When the evaluation team visited the area, the projects were not yet completed, which meant it failed to meet the scheduled deadline. Similarly, in Ulak Kemang and Suak Batakok, OKI (P2MPD Project), the roads constructed were not up to required quality standards and were less strong so that the evaluation team found them in so poor condition. G. Community Organization Organization of the community is very important in the development of community-based infrastructure development. Community organizations are institutional instruments that make it possible to measure the performance of project work. The three programs gave different names to community organizations formed like CSOs, Community Groups (Pokmas), Task Force Team (Tim Inti), Tim Pokja Prasarana and LSPBM for the community-based working group of saving and borrowing. All these groups went through a strengthening phase with following sequence: preparation, formation, strengthening through facilitation education and training, drafting of organization program, carrying out of program activities and the monitoring and evaluation of the community organization and the guarantee for its sustainability. Summary of evaluation findings on community organization is shown by Table 11. Table 11 : Community organization Variable Concept (guidelines) Field fact 1. Preparation - Phases are carried out with - Usually, it is formally carried out socialization, PRA, by facilitators and project meetings with villagers, officials from the district through village consultations. . project socialization. . - There is no evidence that efforts were already made to make the local people aware of the importance of the projects and PRA. - Meetings of villagers/village consultation meetings are held as a formality only without involving many poor people and women (except P2MPD Project). . 2. Formation of - Organizations are formed - Often, the formation of organization democratically organization was dominatred by - Criteria of the election of village elites. organization's management - Election of the organization's personnel are based on moral management personnel is often standards (honesty, based on whether or not people responsibility and to be elected are influential accountability) figures. - Statutes/Bylaw, according to - Statutes and bylaw are based on existing rules, are outlined operational directives/technical based on the real needs of directives that have been the local people. . stipulated/need for projects 3. Training of Training is based on existing Training held is based on people's Management needs for training (needs need for project/program in the Personnel assessment) of the individuals, form of training on infrastructure organizations and programs. development training and technical training on LSPBM bookkeeping. 4. Facilitgation of Designing of Integrated Rural - Drafting of plans only focus on Program Development Program. rural infrastructure and LKSMB Designing development. - Outlining of the plan of doing construction work that comprises time, labor and materials used. . 5. Facilitation of - Facilitation of the - Saving-borrowing program is program implementation of rural carried out by the local people implementation infrastructure development; (skilled laborers and/or core from preparation to quality groups, working group for control. . construction). - Facilitation of the - Saving-borrowing programn is implementation of LSPBM implemented by the through: production of management of LSPBM. regulations on borrowing Construction management and saing, book-keeping, carried out has only focused on collection of capital from the the acheivement of volumes and local people, revolving of the cost of work is not funds and monitgoring and compatible with quality. evaluation. . - LSPBM management has - Facilitation places emphasis stressed accountability and on accountability and transparency. . transparency in financing. 6. Facilitation of Facilitation of monitoring and - Monitoring & evaluation are program evaluation comprises conducted using notice boards monitoring and monitoring of volumes, funds thas state project volume, cost, evaluation. and the quality of activities. without mentioning the qulaity. - Monitoring and evaluation are conducted through monitoring book and financial evaluation of LSPBM is conducted by facilitgators and project managment personnel. 7. Facilitation of the - Facilitation of the - There is no evidence that training preparation for preparation of program has been held by facilitators for program continuation by holding O&M about the infrastructure continuation. trainings and giving constructed. Infomrally carried explanations to the local out by successful LSPBM. . people about technical matters about the maintenance of projects and the replication of infrastructure programs after they are finished. Facilitgation of the preparation for the continuation of LSDPBM with continued trainings and also with the creation of cadres for project management so that the LSPBM system will continue to develop and make progress. The table above clearly shows that a gap remains between the ideal obligations stated in the criteria/guidelines and the reality on the field. However, in the case of PPIP Project in Pangkep, the formation of Pokas went as expected because the program socialization Village Consultation (mudes) forum proceeded with the assistance of district consultants and sub district facilitators. OMS/Pokmas were formed in democratic process including the election of their management personnel. The organizations as technical executors have carried out their tasks in accordance with existing technical guidelines. In addition, OMS/Pokmas also made technical plans and drafted budgets according to available funds. The forms of their activities are clear and can be easily carried out so that the local people can continue their activities well (guarantee of continuity/sustainability). H. Capacity Building Field findings in the three program locations show that efforts to capacity building for local community has not run well. The capacity building for community leaders and community members in the forms of trainings, workshops and others were basically faced with limitation in terms of type, frequency and the participant. Capacity building in the three programs mainly aimed to support the smoothness of the development of infrastructure like Operation & Maintenance training, fund management and others. Meanwhile, trainings for the development of the local community attitudes and skills for the improvement of their self- reliance, cooperation and the leadership of Pokmas were limited. In addition, there is limitation in the resoles and functions of facilitators in efforts of increasing organization capacity and community empowerment, in the provision of trainings and facilitation. This seems to be the result of the weakness of the process of facilitators selection and recruitment. Qualifications and the backfield (education) of facilitators were less relevant to their tasks of empowering the local people (Table 13). In addition, the process of holding trainings for facilitators and the substance of local consultant's preparation of vacillators as project executors were less deep, and there was impression that they were held merely as formality of meeting project execution rules. Table 12 : Program of the enhacement of the capacity of district government and local community Variable Concept (guidelines) Field fact Increasing of the capacity S1 and S2 formal education in S1 and S2 at several universities of district government planning and the management of technical construction of community-based rural infras- tructure. Increasing of the capacity S1 and S2 formal education in S1 and S2 in sociology at of community organizations community development different universities Actually, PMPD Project provided formal education programs for project officials from district government circles to obtain S1 and S2 degrees. However, the fields of education taken by the officials seemed not to go with needs on the field for community empowerment (Table 13). In addition, the number of the program beneficiary is very limited, and did not directly touch the needs of poor. In the future, it should be considered to provide programs Which could directly prepare and improve skilled human resources who have adequate knowledge and understanding on the problems of their community, particularly for the shake of community empowerment. I. People's Satisfaction of Program One yardstick to measure the success of a project is satisfaction level of the local people concerning the process of program implementation and the results turned out. In relation to the three projects, people's satisfaction levels are measured with two techniques namely structured interview and FGD technique. The aspects of measuring villagers' satisfaction are transparency of program information, people's knowledge about the program, availability of information and publication about the program including things related to the mechanisms of handling people's complaints about the program up to the satisfaction levels of the local people and stakeholders concerning program implementation and results. At the concept level, the three programs have pushed the local people (especially those in program locations) to have adequate knowledge about the program, which can later help encourage them to take part in program implementation up to program control. This is conducted through two forms of activities namely socialization of the program through meetings of local villagers and publications about the program. Moreover, the three programs also have designed schemes for handling people's complaints about irregularities and negative impacts of program implementation. All this shows that the program was designed merely to measure the level of people's satisfaction concerning the physical results of the infrastructure constructed, but also to measure the processes of project realization. In this evaluation, on the whole peoples satisfaction can be put into 4 variables namely satisfaction from the technical aspect of facilities including their function and quality, mechanism of project management, financial management and the existence and sustainability of community organization. In regard to technical performance, 60% of respondents are satisfied with the facilities constructed (Graph 1). They feel not only satisfied with the functions and quality of the constructed facilities, but also on the impacts of the construction. The presence of the facilities, the local community are stimulated to further increase the economic productivity of the community. In case of fish pond cultivation in PPIP program in Pangkep District, the community members might have asset and utilize the land for economic productivity that was formerly idle one. Graph 1 also shows that the level of people's satisfaction of the performance of infrastructure development is 40-60% (management performance is 50%, financial performance is 60%and the performance of community organization is 40%), which in all are between the average and good levels. This means that the projects are not fully successful as expected. The measurement of the performance was made through focused group discussion (FGD) that used scale rating matrix technique that is based on the local people's perception and valuation as a group. Level of Community Satisfaction on Village Infrastructure 80% 70% 60% 50% 40% 30% 20% 10% 0% Technical Management Financial Community Performance Performance Performance Performance Graph 1. People's satisfaction on the rural infrastructure project Table 13 shows the level of the local people's satisfaction concerning each performance variable that can visually be shown through the table below: Table 13 : People's satisfaction level Variable Concept (guidelines) Field fact 2. Technical All technical criteria, Most technical criteria on work Performance quantitatively and qualitatively, volumes have been met, and have been met and are correctly even volumes are increased applied albeit with regular budgert. Differences in stipulations on technical standards for village infrastructure (PPIP, PMPD & P2MPD) cause the adjustment of the process of infrastructure development with the condition of the local people's skills so that output turned out relatively differs from one another, although their quality and quality met technical requirements. . 3. Performance of Mechanisms of project Most have applied management project management that is transparent mechanisms according to time management and accountable. . schedules and budget absorption, but the level of transparency and accountability is still very limited; as seen through reports on work that are not detailed especially concerning costs and materials so that it becomes difficult to control planned costs and realized costs in the field. 4. Financial There is effectiveness and Several villages have aplied performance effiency in budget spending efficiency and effectiveness in budget spending in quite a good way like in Pangekp (PIPP). But, some others have shown similar poerformance (cases in OKI, P2MPD Project). . 5. Performance of Community participation is clear Iinformation (socialization) of community from planning, controlling of the the project, concerning its form organization development of infrastructure up and mechanisms, does not to O&M and the replication and reach marginal people continuation of the program. . (illiterate and poor people). ) Part of the village people have shown participation in the whole process of infrastructure development, but few villages have succeeded to increase participation of women in that process like in Sleman (P2MPD). . J. Program Sustainability One very important aspect of any program is project continuity (sustainability). Both book of project manual as well as reports of project activities outlines that found that all the projects of rural infrastructure development, at the concept level, provide phases of operational activities or the utilization and maintenance, from the physical aspect of the infrastructure and the activities of saving-borrowing institutions (notably PMPD). In addition, at any program a Team or Operational/Utilization and Maintenance Group was formed. It was formed at the community level as part of program activities. Basically, the main purpose of program sustainability is the growth of the capacity of community organizations of the beneficiaries in managing the facilities constructed and the capacity to expand the existing facilities with available resources, notably for facilities that are village assets (like the improvement and widening of roads, and boat berthing facilities). Field findings show that, of Table 12 above, the three projects evaluated seem to have community based-organization handling operational and maintenance at the community level. However, in terms of the function the constructed facilities, each program performs different conditions. The O&M organization in Bangkalan, for instances, is less active as compared to the one in Pangkep. In Bangkalan, almost all of the facilities constructed (95%) are roads and bridges. The management of the constructed are not or less productive in approaches, in a sense that the local people are required to pay all the facilities they utilize. The community do not have clear financial sources for maintaining the roads and bridges. As for boat berthing facilities, fees actually have been collected from those utilizing them. But, such fees are still collected by certain individuals from the local community and are not managed by Pokmas for O&M. Meanwhile, in Pangkep, the infrastructure constructed includes water terminals and water tanks. Water tanks are constructed on the ground where villagers/consumers might buy water. So that formed Water User Group (KPP) could take its role for O&M management by utilizing profits from water payment. From the two cases, it can be concluded that there are two factors determine project sustainability: applicability and utility of the facilities (whether the facilities are productive or not) and the capacity building of the formed community groups. Similar findings are found at the P2MPD and PMPD projects. Most groups are not yet prepared to carry out maintenance activities in a systematic and sustainable way. There are no prepared mechanism of organizational fund raising and in organizing the water users. What often happened was that facilities were maintained in an ad hoc way, like with kerja bakti (mutual help activities) in a regular way or when problems arose. This such a condition happened because ­in the villager's point of view-- the management of saving-borrowing activities have to be carried out on a daily basis and in a continuous way. Meanwhile, the management of O&M of infrastructure need to be organized only when problems arise or when the facilities constructed are damaged. Table 14 : Condition of the continuity of infrastructure No Variable Concept (guidelines) Field fact 1 Availability of the Sustainable funds are available, Component of sutainable funds component of in limited amounts. Only in is so limited, and the time pf sustainable fund PMPD Project there are special project execution is also very locations for sustainable limitged. As a result, the activities, especially in the preformance of O&P strengthening of LSPBM. orgnaizations in the community gnerally is not adequate for managing the infrastructure constructed. 2 Type of activities for - Sustainable activities Preparation of guidelines of strenthening the comprise: preparation of maintenance is made by Field continuity of projects guidelines on maintenance, Facilitator, but less in a for the community. training for Pokmas/KPP participatory way. Guidelines in maintenance, and of maintenance tend to focus preparation of cadres of on technical aspects, and give Community so little attention to the aspect Empowerment. . of the institutional development of Pokmas/KPP. Trainings conducted by Facilitator in fact mostly use one-way socialization method, and the time allocation is not adequate ­ 2-3 hours. The existence of community empowerment cadres is not yet functional in making Pokmas/KPP functioning in the maintainanc eof infrastructure. . 3 Type of O&P Types of O&P activities for In practice, the types of O&P activities infrastructure comprise: activities that are complete are - Infrastructrure transformed in a short time so management that the local people cannot - Service provision understand that knowledge. - Maintenance Moreover, in gneral, procedures Pokmas/KPP is not yet capable Financing to provide adequate finances. 4 Execution of O&P O&P activities are carried out Pokmas/KPP and the village activities by Pokmas/KPP together with government do not yet play village government. their roles as expected namely provide funds for the projects since the assets constructed belong to the village. Based on above findings, it can be concluded that local community groups not playing their roles optimally in the management of the infrastructure constructed was caused by several factors as follows: a. The existence of the local community and their village should be more used to support the smoothness and success of infrastructure development. Although several activities are carried out to prepare the local people to play their roles after the completion of the infrastructure projects, the process of carrying out the activities and its intensity are less than expected and tend to be artificial. b. The short time for carrying out the project for each location has led to the fact that all those involved in the execution of the project (government apparatuses and consultants) focus their activities on fulfilling their administrative tasks. On the other hand, facilitators in carrying out their base it more on contracts in which activities of facilitating the community are carried out only during the course of project implementation. c. Community groups and cadres who have proven their functions well during the course of project implementation are not optimally utilized as catalysts for continuing the management of infrastructure (notably roads) with new projects or are not given other incentives in the form of productive venture CHAPTER IV CONCLUSION AND RECOMMENDATION A. Conclusion and Lesson Learnt Based on the analysis on the documents, in-depth interviews and field observation to the infrastructure and facilities constructed in the study sites (9 villages in 6 districts, and in 6 provinces) it can be said that the purpose of the three programs have been achieved as of planned, particularly in the realm of basic rural infrastructure (road and bridge). On the other hand, it is admitted that the three programs have not succeeded in developing productive infrastructure and giving impacts to poverty eradication. But, the three programs, at the starting point of project implementation, specifically have helped the local people to face economic crisis by providing jobs and accordingly increase their income. Meanwhile, the program of infrastructure and facility development has increased the accessibility of the local people's economic ventures, has increased the value of their assists (land) and the accessibility of basic social facilities like accesses of communication and interaction between community members, clean water supply and others. The successes in the provision of the infrastructure were also indicated by the level of people's participation mainly in the building of infrastructure that used community-based approaches. This also shows that as long as certain types of infrastructure are needed by the local people, the level of their participation and the functions and the continuity of the infrastructure will increase. However, evaluation shows that the three programs have not immediately improved the condition of poor people. Interviews with poor people who are beneficiaries of the projects indicate that the social and economic profile of poor people in project locations have not significantly changed in terms of assets ownership, saving, physical condition of their house, diversification of the source of livelihood and the quality of their foods. Admittedly, any change with the poor condition of the local people is also determined by other variables (culture, land supporting forces and polices), and not merely due to the impacts of the existence of the three programs. However, it should be considered that, in the future, community-based infrastructure program should place more emphasis on the development of productive infrastructure so that the programs can give greater contrMrstion to poverty eradication efforts. Meanwhile, the optimal achievements of the three programs (PMPD, PPIP and P2MPD) are limited to the physical aspects of the infrastructure and their implications. Meanwhile, the processes and values behind the physical achievements are still not enough to turn out satisfactory results. Example, concerning the selection by the local villagers of the programs they are about to receive, not all villagers involved in the selection process were poor people, which was not in accordance with the initial plan. Moreover, the selection of target people who will receive projects allowed intervention by local political elites who have vested interests. Accordingly, although existing rules provided that the poorest people are given priority to receive projects, it cannot be easily materialized since village consultation meetings were held merely to fulfill procedural requirements for the projects and also because of the considerations of equal distrMrstion of development projects. This limited process were not only found in the selection of target villages but also in the involvement of the local people in the project management, technical construction and the strengthening of community organizations for the operation and maintenance of the infrastructure constructed. As such, the three programs basically have succeeded to provide infrastructure construction, but they were not followed by achievements in the processes including successes in the practicing of good governance at the local government level like transparency in project financial management, accountability and the practicing of control mechanism. Roughly, several important findings were also based on the results of evaluation of the implementation of the three programs. The findings are as follows: a. The building of village infrastructure was generally well carried out notably from the aspect of the selection of the types of facilities and their utilities. Accordingly, although they were constructed with simple methods at low costs, the infrastructure constructed are viewed by the local people as having given positive impacts them, economically and socially. b. Seemingly, the basic principles of the project that were used like democracy, good governance, transparency, the involvement of women and others so far have been forced into the projects just to address reformation demands that are still popular in Indonesia. It seems that the practicing of local good governance in the projects remains far from expectations and people are aware that what have been achieved so far are still at the learning phase. The process and practices of transparency and accountability are exactly found in the circles of community groups (Pokmas), and several pokmas have made financial recording and reporting that are quire transparent. c. Project documents indicate that each type of projects has monitoring and evaluation (M&E) systems at the community, district, provincial and government levels. Also, there is M&E conducted by independent institutions (the case of P2MPD Project). M&E includes the handling of complaints/objections about project implementation. However, monitoring is mostly conducted by facilitators rather than by the project leader. It is merely a formality in order to fulfill requirements of project fund disbursement. People's complaints are tackled, but it happened only in PMPD Project in South Kalimantan. Moreover, community groups (KSM) in a point of view that not all results of M&E, control and the handling of people's complaints have been basis for the of project quality improvement. It is also a result that there is no independent outside agency that was assigned to undertake evaluation to the projects. So far, it is only one report of project evaluation was by evaluation team. d. In-depth interviews with project executors in the districts and with officials of community groups show that the projects were normally designed to be carried out in 1 (one) year. In reality, however, only 3-4 months could be allocated for it. This was because the government was late to issue DIPA. As a result, activities were often oriented to meet physical achievement targets while ignoring the aspect of community empowerment and the strengthening of community groups. So it is understandable that the process of community empowerment as stated in the documents was followed merely to fulfill existing procedures when the projects were being carried out. The local people almost played no role after the projects were completed, for example their participation and responsibility in the maintenance of facilities. e. The institutional capacity building carried out in the projects still tend just to meet project administrative and management requirements. Even, FGD and observations in project locations highlight that the poor capacity of community organizations (Pokmas) have seriously caused that almost all community groups were not sustained by the end of project. Several women groups manage saving and borrowing funds from PMPD and PPIP projects whose facilities are productive, and accordingly the roles and interaction of the local community are still visible. f. Generally, funds have been directly channeled to community groups. This has positive impacts on the efforts to build Pokmas/OMS' trust. However, it further need an operational guidelines as well as supporting training on financial management at the community level. The aim of these activity is to increase transparency and accountability in the community fund expenses. In addition, there is still a tax cut payment practice on overseas project loans (PLN) such as a case of P2MPD Project in Sleman. In-depth interviews with the personnel of Pokmas in field and also data in the financial documents/reports concerning the projects also indicated problems of the channeling, management and the cutting of the amounts of project funds that went to the local people although the number of such cases are not significant. g. The technical quality of the infrastructure constructed is relatively good in general in terms of volume and technical specification, but not from the aspect of the materials used. There needs to be standards guidelines for costs and the minimum quality of materials used. This aims to improve the quality of infrastructure constructed. h. Focus group discussion showed that the level of people's satisfaction of the programs was 40-60%. Their dissatisfaction was mainly related to transparency in budgeting and in the mechanisms of fund disbursement and in the procurement of construction materials like cement, and in the handling of people's complaints. i. Although the local people's participation goes with existing guidelines as stated in annual progress project reports, the quality is far from satisfactory. Participation is still measured with the local people's willingness to give contrMrstion in the process of infrastructure construction. Meanwhile, attention is not given to people's participation in terms of decision making process and in sounding their views. It is not surprising, that village elites still dominate decision making processes. People's participation includes participation of women, which is still limited to attending meetings. An exception is the case of P2MPD Project in Sleman where women's participation is quite high. j. Concerning the aspect of project sustainability, in most cases it was generally formed "group of users and maintenance/KPP" in the community. But the groups did not function properly to maintain the existing infrastructure and facilities. As the evaluation team observation, that KPPs in the tree projects have mostly not functioned properly such cases in Sibatua Village in Pangkep District (PPIP) and in the LSPBM group in Minahasa (PMPD), B. Recommendations a. Learning from the documents and the execution of the three projects that were evaluated, the Indonesian government should maintain policies for not carrying out empowerment projects with funds from State Budget (APBN) and donor countries, and with implementation cycles of only 3-4 months. With such cycles, the processes and outcome of such projects will not be adequate, especially concerning the importance of involving the local people in planning, monitoring and evaluation that eventually could be conducted as formality only. Even worse, the district government is often late to give support (DIPA) for project implementation. In addition, facilitators in doing their work face limitations in terms of time and orientation, and their work is not really based on the need to empower the local people to settle their problems. b. Project implementation needs to be socialized by the central government to the district government far before the project is carried out so that the district government has enough time to integrate project financing into the system of district budget planning. Meanwhile, socialization to the village community should be conducted through different ways and means like placing leaflets at the village/hamlet office, and information that can reach illiterate villagers needs to take certain forms like the using of visual aids in socialization meetings or the utilizing of community radio, which aims to enable the local people to know about project design and purpose. c. The implementation of infrastructure projects has not optimally reached poor people, from the process of selecting villages that can receive projects to the selection of the types of the infrastructure to be constructed. Accordingly, in the future any project offered needs to be initiated with activities of mapping out rural social and economic activities, which will serve as a basis for making sure that the majority of villagers who will benefit the project are poor people, and for determining the type of the infrastructure that can be constructed. Also, the proposals and suggestions from the local people should not be selected by project executors. That task should be carried out by an independent institution/team in order to decide the viability of the project and possible social and environmental impacts caused by the project (social-environmental impact assessment). d. Many cases of project implementation, especially implementation by the government, show that women play so little roles and participation in decision- making concerning project management. On the other hand, many projects were successfully managed when women were optimally involved like the case of PMPD Project. That is why there needs to be special components (outside infrastructure) that are managed by women, and not merely saving-borrowing units. They could include micro ventures. e. Results of the evaluation of the three projects show that the types of infrastructure constructed by the local people are much more cheaper and functional and have much better quality, and guarantee project continuity as compared with the types of infrastructure constructed by contractors. Accordingly, the government needs to crate more opportunities for the local people to carry out rural infrastructure projects with value of below Rp 300 million, which use simple technologies like irrigation networks, boat berthing facilities and clean water facilities. f. However, carrying out infrastructure projects as pointed out by point c above requires intensive facilitation, notably by qualified facilitators with competence in community organization. This is meant to reduce and even eliminate the domination of village elites in decision making concerning the whole process of project planning, execution and evaluation, and, at the same time, guarantee the effective process of participatory planning, democratic management of the organization, and the self-supporting and the continuity of the functions of village institutions. g. Increasing the capacity of district government apparatuses (project execution management, village apparatuses and Pokmas) is so important notably in the fields of good governance (transparency, accountability, participation) and the theme of development with perspectives of rights. In addition, the increasing of the local people's capacity in monitoring and evolution by themselves (community-based self monitoring and evaluation). h. Programs of poverty eradication through village infrastructure development need to be carried out using one management system (one door policy) so that there will be standardization in the selection, planning, monitoring and evaluation of projects, from the technical and financial aspects and from the aspect the strengthening of community organizations. Surely, doing this work has to go with the needs on the field. As a result, the community that receive the program will not face complications in responding to poverty eradication programs of different types with different approaches and methods. Annex 1: List of evaluation respondents (Institutions and Individuals ) No. LOCATION INSTITUTION INFORMATION SOURCE 1. Oki District- S. 1. Bappeda OKI H. Sodikin, SP Sumatra Project Officer of P2MPD 2. Project Consultant Faisal SE CV Berta Alamba Engineering: Ulakang Village Head - Mr. Ujang Keti Selabato Village - 3. Common villagers - Mr. Fahrul Roji - Mrs Ruslah - Mr. Muhammad - Mr. Alimuddin - Mrs Eti 2. Sleman District 1. District Coordination Sekwilda of Sleman District Team for P2MPD Project 2. Bappeda Sleman Ir. Dwiyanto Sudibya Project Leader of P2MPD Project 3. Governance Affairs Mr Moko Section Head of Sub Division of Village Community Empowerment Section 4. Project Consultant Mr. Pujo and Mr. Sriyono 5. Head of Sidomulyo H. Mulyadi Village, Godean Subdistrict 6. Common Villagers - Mrs Wartiyem, Pokmas chairman - Mrs Sumiasih, villager - Bapak Kartijo, villager 3. Bangkalan District 1. Kimpraswil Mr. Ari, ST Saker Chairman/Project Officer 2. Project Consultant Mrk. Miski Asisten to Project Officer 3. Mertayasa Village Head Ir. Heru 4. Common Villagers - Rahmat, SE - Sumadi, representative of Mertayasa Village - Abdul Azis, Chairman of Pokmas Keramat - Rahmat, Keramat Villager 4. Banjar District 1. PMD Office and Linmas - Mr. Harun Office (Community PMD Section Head Protection) - 2. Project Consultant Moh. Heru 3. Penjambuan Village Head Ahmad Yuni 4. Common villagers - Sahrul Poklak Chairman 5. Pangkep District 1. Subdin Cipta Karya Ir. Dzulkifli, M Si Ka Subdin 2. Sibatua Village Head M. Roy Hartono 3. Common villagers - Mr. Ilyas OMS Chairman - M. Ansar Subdistrict Facilitator - M. Anas (Beneficiary) - Irwan (Beneficiary) - Hj. Marwiyah (Beneficiary) - H. Baharudin Community leader 6. Minahasa District 1. MRMPD Minahasa Welly Pantow MRMPD Project Officer - Mr. Youke LSPBM Chairman Villagers - Mrs Nornitje (Beneficiary) - Hans A. Maki Planning Forum Chairman 7. Jakarta 1. P2MPD Mr. Hari, P2MPD Project Officer 2. PMD - Mrk. Haryo, Technical Section - Mrk. Aris Mulyana, Msi, PMPD Project Officer Mr. Ir. Dani Junaedi, 3. Ministry of Public Works Kimpraswil Annex 2 LIST OF REVIEWED DOCUMENTS No. Project Title Document I P2MPD (Program ADB. October 2006. Completion Report, INO: Pendukung Pemberdayaan Community and Local government Support Sector Masyarakat dan Pemerintah Development Program. Jakarta. Daerah) ADB. March 1999. Report and Recommendations of The President to Board of directors on Proposed Loans and technical Assistance Grant to The Republic of Indonesia for CLGS Sector Development Program. Jakarta. Bappenas and ADB. March 2005. Final Report: Community and Local Government Support ­ Sector Development Program. Jakarta. Bappenas. 2005. Appendices Final Report. Community and Local Government Support ­ Sector Development Program. Jakarta. Bappenas. March 2005. Independent Evaluation of CLGS Project. Jakarta. Bappenas. Oktober 2001. Pedoman Umum Program Pendukung Pemberdayaan Masyarakat dan Pemerintah Daerah (P2MPD). Jakarta. Bappeda kabupaten Ogan Komering Ilir. Kontrak antara Pemerintah RI dan PT. Pola Cipta Alamba Konsultan dan YPPM untuk Pekerjaan P2MPD ADB Loan No. 1678-INO. Jambi. PT POLA ALAMBA KONSULTAN, Konsultan Manajemen daerah (KMD) Kabupaten Ogan Komering Ilir. 2004. Adendum Surat Perjanjian Pekerjaan, No. 02/258/KMD-P2MPD/2004. Kayu Agung. PT POLA ALAMBA KONSULTAN, Konsultan Manajemen daerah (KMD) Kabupaten Ogan Komering Ilir. Februari 2003. Laporan Bulan II, Pemeritah Kabupaten Ogan Komering Ilir, Program Pendukung Pemberdayaan Masyarakat dan Pemerintah Daerah (P2MPD) tahun Anggaran 2002. Kayu Agung. Palembang. PT. ASANA CITRA YASA berasosiasi dengan YAYASAN "Cipta Mandiri" YOGYAKARTA. Desember 2004. Laporan Akhir, Pekerjaan Konsultan Manajemen daerah dan Fasilitator Program Pendukung Pemberdayaan Masyarakat dan Pemerintah Daerah (P2MPD) Kabupaten Sleman, tahun anggaran 2004. Yogjakarta. PT POLA ALAMBA KONSULTAN, Konsultan Manajemen daerah (KMD) Kabupaten Ogan Komering Ilir. 2004. Adendum Surat Perjanjian Pekerjaan, No. 02/258/KMD-P2MPD/2004. Kayu Agung. PT POLA ALAMBA KONSULTAN, Konsultan Manajemen daerah (KMD) Kabupaten Ogan Komering Ilir. Februari 2003. Laporan Bulan II, Pemeritah Kabupaten Ogan Komering Ilir, Program Pendukung Pemberdayaan Masyarakat dan Pemerintah Daerah (P2MPD) tahun Anggaran 2002. Kayu Agung. Palembang. PT. ASANA CITRA YASA berasosiasi dengan YAYASAN "Cipta Mandiri" YOGYAKARTA. Desember 2004. Laporan Akhir, Pekerjaan Konsultan Manajemen daerah dan Fasilitator Program Pendukung Pemberdayaan Masyarakat dan Pemerintah Daerah (P2MPD) Kabupaten Sleman, tahun anggaran 2004. Yogjakarta. Secretariat of the CLGS-SDP. March 2005. Appendix Final report. Jakarta II PMPD (Pemberdayaan ADB. September 2000. Report and Masyarakat untuk Recommendations of The President to Board of Pembangunan Desa) directors on Proposed Loans and technical Assistance Grant to The Republic of Indonesia for CERD Project. Jakarta. Departemen Dalam Negeri. 2006. Consolidated Annual Report 2006: Community Empowerment for Rural Development Project (CERD). Direktorat Jenderal Pemberdayaan Masyarakat Desa. Jakarta. Departemen Dalam Negeri. April 2004. Panduan Program Monitoring dan Evaluasi Kinerja Pengelolaan Proyek PMPD. Jakarta. Departemen Dalam Negeri. Juni 2003. Pedoman Umum Proyek Pemberdayaan Masyarakat untuk Pembangunan Desa (PMPD). Jakarta Tim Proyek PMPD/CERD Kabupaten Banjar. Implementasi PMPD/CERD di Kabupaten Banjar 2001-2006. Banjar Departemen Dalam Negeri. 2006. Revisi Rencana Pembangunan Jangka Menengah Desa (RPJM) Kabupaen Banjar 2005-2006. Banjar. Departemen Keuangan Republik Indonesia, Direktorat Jenderal Pembendaharaan. Peraturan Direktur Jenderal Perbendaharaan Nomor per- 43/PB/2005 tentang Petunjuk Pelaksanaan Penyaluran dan Pencairan Dana Loan ADB III PPIP (Program Number 1765-INO (SF) dan 1766-INO Community Peningkatan Infrastruktur Empowerment and Rural Development (CERD). Perdesaan) Jakarta. ADB. November 2005. Report and Recommendation of the President to the Board of Directors, Proposed Loan Republic of Indonesia: Rural Infrastructure Support Project. Jakarta. ADB. November 2005. Initial Poverty and Social Assessment, INO: Rural Infrastructure Support Project. Jakarta. ADB. November 2005. Draft Design and monitoring framework, INO: RISP. Jakarta. Departemen Pekerjaan Umum. Juni 2006. Pedoman Pelaksanaan Program Peningkatan Infrastruktur Perdesaan (PPIP). Jakarta. Departemen Pekerjaan Umum, Direktorat Jenderal Ciptakarya. Tahun 2006. Keputusan Menteri Pekerjaan Umum Republik Indonesia, Nomor : 241.A/KPTS/M/2006 tentang Penetapan Desa Sasaran Program Rehabilitasi dan Peningkatan Prasarana Perdesaan, RISP-ADB. Jakarta. Lembaga Penelitian SMERU. Juni 2006. Kajian Cepat PKPS-BBM Bidang Pendidikan Bantuan Operasional Sekolah (BOS) 2005. Jakarta LP3ES. 2006. Studi Evaluasi Pelaksanaan Program Kompensasi Pengurangan Subsidi Bahan Bakar Minyak Bidang Infrastruktur Pedesaan (PKPS-BBM IP) Tahun 2005. Jakarta. PUSKA UI. 2006. Laporan Akhir hasil Asesmen Cepat Program PKPS-BBM 2005 Bidang Kesehatan. Jakarta Annex 3 Guidelines for Interview and FGD I. TRANSECT A. Aim 1. Complete information in mapping 2. Directly observe the condition of physical infrastructure and land utilization 3. Understand existing problems and the methods adopted by farmers to settle them and the potentials of resources in evaluation locations B. Participant Participants total around 15 people who represent different groups of village stakeholders. 1. Community leaders 2. Officials of engagement of Village Infrastructure Program 3. Village Apparatus (Village Head/Head of Village Economic And Development Affairs Section) 4. MRD/LKMD officials 5. RW/RT Head 6. Women Groups (PKK, Posyandu, Dasawisma) C. Stationeries & Others 1. A4 Paper 2. Pencils 3. Sketches of locations 4. Drawing of Project Design D. Method 1. Team explains the purpose and aim of the topics to be discussed 2. Team and location villagers agree on the sites where to start observation 3. Participants and the team observe and discuss important matters so as to complete information that has been obtained. 4. Make rough sketches about area borders, sorts of infrastructure, physical condition of existing infrastructure 5. Improve sketches E. Time 180 minutes II. RATING SCALE A. Function a. To have better knowledge about the links, existence, utility and roles of infrastructure from the viewpoints of the local people b. To have knowledge about activities of local villagers, their needs and the benefits of infrastructure c. To have knowledge about the benefits of any kind of infrastructure as felt by the local people. B. Participants Participants total around 15 people who represent different groups of village stakeholders. Group 1: 1. Community leaders 2. Officials of Village Infrastructure Program 3. Village Apparatus (Village Economic and Development Affairs Head) 4. MRD/LKMD officials 5. RW/RT heads Group 2: 1. Leaders/officials of women organizations 2. Village female officials 3. Female villagers (poor female villagers prioritized) C. Stationeries & Others 1. "Plano" papers 2. Ink markers 3. Pieces of carton; round with different sizes 4. Glue D. Method 1. Discussion 2. Dialog E. Steps of Activity Step 1: 1. Facilitator explains to the participants the purpose and aim of activity. 2. The participants are invited to get involve in discussion and the facilitator explains about the size of the carton and how to place them. The big and small sizes of carton and the distance between carton pieces placed on a board depict the link and benefits of infrastructure for local villagers. The bigger the carton pieces the bigger their benefits, and the closer the distance between the carton pieces the easier to access 3. Facilitator asks participants to analyze the benefits and access of infrastructure by sticking carton pieces to "Plano" papers. Together with participants, facilitator decide on the central point of links. 4. Participants choose the size of carton pieces according to benefits and the access as they feel it, and then they write down the sorts of infrastructure. 5. Participants discuss and stick carton pieces according to the access and benefits of each sort of infrastructure. Then they analyze and compare one sort of infrastructure with the other. Facilitator Team records and help smoothen discussions. 6. Participants can make/change the roundness of their carton and then stick it again to the board in accordance with their wishes and assessment of the infrastructure. 7. Facilitator discusses with participants about their problems and expectations concerning the infrastructure, and give suggestions for improvement. Bagian 2: 1. Facilitator asks participants to analyze the impacts of existing infrastructure on the social and economic life of the local people. 2. The study team stick carton pieces to the board and prepare a symbol for each sort of social and economic impacts of the infrastructure. The economic impact, for example, is the local people's income levels, reachable facilities for farming, low transportation costs, and others. Social impacts, example, access to school, access to healthcare services, easy traveling, easy socialization and social activities, and others. 3. Facilitator asks participants to identify benefits or impacts of the programs, and write it down on the left column of the matrix. 4. Facilitator asks participants to discuss their evaluation of the benefits and impacts of the programs by giving scores and formulating indicators, making comparison between the condition of their life before and after the programs were carried out in their areas. Example, farmers' income levels: Before the infrastructure was constructed unhusked rice sold for Rp 2000 perkilo while it was Rp3000 per kilo after the infrastructure was developed in their area. 5. Facilitator and participants discuss and give explanation about their assessment and indicators concerning each benefit and impact of the programs. F. Time 120 minutes IV. Focus Group Discussion (FGD) A. Function/Purpose 1. To have deeper information and understanding on the level of the local people's participation in the programs, in socialization/social marketing, program implementation and program monitoring. 2. To have deeper information and understanding on transparency and accountability of the programs 3. To have deeper information and understanding on gender perspectives of the programs in all areas 4. To have deeper information and understanding on institutions and mechanisms of submitting complaints concerning the programs B. Participants The number of participants total around 15 people who represent different groups of village stakeholders. Group 1 (men): 1. Community leaders 2. Officials of Village Infrastructure Program 3. Village Apparatus (Village Head/Head of Village Economic and Development Affairs) 4. MRD/LKMD officials 5. RW/RT chairmen Group 2 (Women): 1. Leaders/officials of women organizations 2. Village female officials 3. Female villagers (poor women prioritized) C. Materials 1. Plano papers 1. Pieces of papers 2. Ink markers D. Method 1. Brainstorming E. Steps of Activity 1. Facilitators explains about the purpose and aim of discussion 2. Facilitator asks participants to discuss each sub-topic above (directives of questions are in the guidelines of questions), by giving them questions and asking them to give responses on a piece of paper and stick it to the Plano paper. 3. Facilitator asks participants to discuss each response and analyze it together. 4. This prices is repeated so that each question is discussed and analyzed F. Time 120 minutes NEEDED INFORMATION (rating scale will also be identified) On Pokmas · How a community group (Pokmas) was formed? · How did the projects play their roles in the formation of pokmas? · How could pokmas be selected to receive a project? · How many poor villagers are members of pokmas? Capacity Building · What training or something of that kind has been received by pokmas? · Who gave that training? · What do the local people view of the benefits of training? Satisfaction of Project Beneficiaries · Do the local people know about the projects? · Can information about the program be known by the public? · Do the local people know about project activities in their villages? · Were meetings of local villagers held to discuss about projects to be developed in their areas? · Are there mechanisms of handling people's complaints? Do people know about the mechanisms? · On the whole, do the local people feel satisfied with the projects carried out in their areas? Continuity · What is the level of the continuity of the projects constructed? · What have been done by the local people to guarantee the continuity of the projects? · Are there O&M activities carried out by the local people? What are the mechanisms used? Does the local community's O&M operate according to plans? If not, what are the constraints? How does O&M settle their problems? Others · Mechanisms of submitting complaints · Check the questions submitted to the government Annex 4 RESEARCH QUESTIONS Achieving Results 1. According to evaluations, both internal and external, is the program meeting its stated objectives? Are the program's objectives well-formulated so that its achievement and performance can be assessed properly? Do these objectives reflect the overall goal of poverty reduction in Indonesia? 2. According to progress reports, how is the program functioning? What are the common problems encountered during implementation? 3. What are the outputs and reported results on the field in terms of increasing access to rural infrastructure; time savings; and employment generation? Programs Evaluation/Quality Control 4. Does the program have any monitoring and evaluation system? Is it being monitored and evaluated regularly as part of quality control of the program? 5. Are the evaluations credible? What type of evaluation is conducted (impact evaluation, process evaluation)? Is the evaluation design adequate to reach its intended purposes (selection of methodology, sampling, data collection mechanism, data analysis)? What kind of methodology is used (randomized experiment, matching, double difference, qualitative)? How is the quality control of the evaluation? Poverty Targeting/Site Selection 6. Is the program operating in poorest areas? 7. Are poorer segments of the community benefiting? Cost Effectiveness/Budget Structure 8. What is the overall budget? What portions are funded through loans, grants, and government internal resources (rupiah murni)? 9. What is the budget breakdown, broken down by assistance to communities (e.g. community grants and financial assistance, capacity building, TA, administration) 10. Do budgets appear to be reasonable in terms of unit costs and overall costs? 11. Are field activities cost effective (value for the money)? Funds Flow/Leakages 12. Are there any independent audits by MRKP and/or other agencies? If yes, what are the findings? 13. Were the funds used for their intended purposes? 14. Based upon field missions, are the amounts received by communities match with what was originally planned? Technical Review 15. Are there any technical reports assessing the quality of infrastructure constructed under these programs? 16. How is the quality of the infrastructure on the field? Community Organizations 17. How are community groups formed, and how well does facilitation and TA work? How were these groups chosen? Are the poor represented in these groups? Are the groups sustainable? What type of training or capacity building do these groups receive, if any? Capacity Building 18. What capacity building of local officials, service providers or communities has been undertaken by the program? 19. What has been the quality of the capacity building support? Beneficiary Satisfaction with Program 20. Information Transparency: Do people know about the program? Is program information available publicly? Do they know how activities were selected in their village? Were public meetings held? 21. Redress: Is there a handling complaints mechanism and do people know where to send complaints? 22. Satisfaction levels: Were stakeholders (local officials, village beneficiaries) satisfied with the implementation of the program? What are their opinions about the program? Sustainability 23. What are the sustainable components of the program? 24. What is the program doing to ensure sustainability of benefits? 25. Are operations and maintenance activities undertaken? If so, how and by whom? Annex 5 Question List Interviews with Government Respondent Note : C : National Pr : Province D : District 1. Aspect of Managerial Performance A. Implementing Organization a. (C+Pr) Are government institutions/offices or regional governments responsible for the implementation of the projects? b. (C How about the involvement of institutions/offices in the handling of the projects from planning, implementing, monitoring and evaluating project activities? c. (C+D) Is there division of responsibility between the central government institutions and district government institutions in the implementation of projects, and how the responsibility is divided? d. (C) Were staffs/Consultant Team (technical Assistance) recruited to assist government institutions? What are the names of the consultants? e. (C+D) How about the performance of consultants in giving technical assistance to institutions, at the Jakarta, provincial, district and field levels? f. (D) In carrying out projects, were community groups (Pokmas) formed? Who facilitated the formation of such groups? g. (D) What were the activities of Pokmas in the planning of activities on the field, the carrying out of activities, and in the phase of monitoring and evaluating field activities? h. (D) How about the performance of Pokmas in carrying out its activities in each phase? B. Governance of Activity a. [D] What was the form of transparency in budget drafting and transparency in the using of budget funds for project implementation? b. [D] Was the community given access to information about the whole program/activity plan and the amounts of budget allocated? What form and media used to give access to information about program activities and budget amounts? c. [D] Did people play a role in decision making related to project activities? In what forms do the local people take part in decision making? Using the model of representation by certain people or being open to the community at large? d. [D] When there were complaints from the people, how they were handled? And are there institutions responsible for handling such complaints? And how those airing complaints were involved in settling the problems they submitted? e. [D] In the process of carrying out project activities, from planning, executing, monitoring and evaluating, are there guidelines or standard rules used? Are monitoring and evaluation conducted according to plans? If not, why? 2. Aspek Financial Performance* a. [D] How were budget plans (RAB) drafted? Who were involved in budget drafting and what were the phases of doing it? b. [D] How financial structure and the types of spending were planned, and how they were put into detail? c. (C+D] How the budget plans were executed, and how was it matched with each activity plan? d. [C+D] What is the level of cost effectiveness, between the spending of budget funds and each activity plan? e. [D] Did the amounts spent go with the results achieved in the programs (value of money)? ­ Were big amounts were spent with small benefits or the other way around? f. [C+D) How about the procedures of channeling and managing available funds? Is it quite efficient and transparent in implementation? And, were the local people given access to know budget plans and the spending of project funds? g. [C+D] What are the standard operational procedures/SOP of standard financial management, from planning up to auditing? How are existing SOP implemented? h. [C+D] How is financial audit conducted? And who carries it out? What are audit findings? 3. Aspect of Technical/Physical Performance a. [D] What physical assistance is provided for the local people, and what kind of assistance is preferred by them? b. [D] How about the compatibility between the kinds and forms of physical assistance and the local people's needs? c. [K] How about the quality of existing physical facilities (comfortability, safety and compatibility)? d. [D] Does the condition of existing physical facilities go with the technical specifications set? And do the specifications go with people's expectations? e. [D] How about the level of the utilization by the local people of the physical facilities? f. [D] Does the existence of the physical condition of the facilities go with the local condition (materials, facility design, operational capacity and maintenance)? 4. Aspect of Communityh Performance a. [C+D] Was the participatory approach adopted as an approach methodology for the activities of the projects? And what are the guidelines for practicing the participatory approach? (Socialization, planning, implentation, monitoring, OMK) b. [D] How were the process, level and form of community participation put into activities? Do the local people feel the benefits of the form of community participation? c. [D] How about the level of the continuity of the process, level and form of community participation after the projects end? d. [D] What is the level of people's satisfaction of the results of project activities and the whole process of the activities? e. [D] Do the projects give attention to the aspect of gender equality, and are there guidelines for implementing it? f. [D] What are the form and process of the implementation of gender perspective on the field? g. (D) Has the project models been replicated in other areas or other programs, for government projects, projects of regional governments, or non-government projects? *Additional detailed questions about financial performance a. [C+D) Were there funds coming from the government (State Budget/APBN or Regional Government Budget/APBD)? What are their portions as compared with assistance funds or loans? b. [C+D] Where did government funds come from? Were they from APBN and APBD? Or a combination between the two? c. [C+D) Which working unit has authority to manage the using of such funds? d. [C+D] Are funds also channeled to the local people? e. [C+D] In what forms are they channeled? f. [C+D] How about the budget structure and composition? g. [C+D] What is the portion of funds going to facilitation experts and supporting staffs who are from outside the local community? h. [C+D) Have experts and facilitators worked according to people's expectations? i. [C+D] What is the portion of general administrative costs as compared with total costs of project activities? j. [C,D) Have the whole scope or work and its volumes been fully done? k. [D] Was there change in the volumes of project work from the original plan? Why was there change? l. [D] Compared with market prices, were they and the unit price of project work volumes was relatively similar? m. [D]Were all resources planned have been used for each working unit? n. [D] Is there remainder of resources not used? Why was there remainder? And what will it be used for? o. [D] What is the process of financial administration for a project work? How is it conducted at Project Executor at the Government? p. [D) Are there guidelines for conducting financial administration? From whom? q. (D) Did each party (Project Executor and the Government) make accountability reports? What were their forms? What evidences were made report supplements? r. (D) Who has authority to examine records of financial administration and financial reports, internal and external ones? s. (D) Did any external auditor audit the finances and financial reports? Mention it. t. (D) What problems were often faced in financial administration activities? u. (D) How about the breakdown of funds allocated for project work? What is the relationship between the breakdown and the scope of project work v. (D) Were the volume of work and the unit cost in the project budget were in accordance with the volume of each working scope? w. (D) What was the form of the utilization of funds managed by the local people? Did they go with plans? x. (D) Was the any difference between the budget and its realization? What caussed the difference? y. (D) Has the project work given economic benefits to the local people? In what forms? Annex 6 Guidelines for Interviews with local people TECHNICAL ASPECT 1. Were the majority of the local people involved to design plans of developing village infrastructure and facilities? 2. Did community members give contrMrstion in the form of labor and/or building materials or money? 3. Are there community groups responsible for the maintenance and management of the infrastructure? 4. Did the community groups get trainings about methods of operating and maintaining the rural infrastructure? 5. Do the infrastructure constructed function as planned and do they fulfill technical requirements? 6. Do the infrastructure constructed have met the local people's needs? 7. Do local community members have a sense of belonging to the infrastructure and use them according to plan? 8. Are there any plans of building new village infrastructure? 9. Do the infrastructure constructed have positive impacts on the environment, and social and economic welfare of local villagers? ASPEK KEUANGAN 1. Were there funds from the Government (APBN or APBD)? How much? 2. Were funds also channeled to the local people? 3. In what forms were they channeled? 4. Were there experts and facilitators? 5. Have the experts and facilitators performed their tasks as expected by the local people? 6. Have all the whole scope of project work and its volume been fully materialized? 7. Was there any change in project work volumes? Why did change happen? 8. Compared with market prices, was the unit price of project work volumes was relatively similar to the market prices? 9. Have all resources planned have been used for each work unit? 10. Is there remainder of resources that is not yet used? Whey was there remainder? What is the plan to use it? 11. Were there individuals (community groups) appointed to carry out financial administration? 12. Are there guidelines for carrying out financial administration? From where? 13. Is there any obligation to make accountability reports? What is the form of such reports? What evidences are made report supplements? 14. Who have authority to examine financial administrative records and reports? 15. Are there external parties who examine project finances and financial reports? Mention them! 16. What problems were often faced in the process of carrying out financial administration activities? 17. What is the form of the utilization of funds managed by the local people? Does such utilization go according to plan? Has project work given economic benefits to the local people? In what forms? ASPECT OF COMMUNITY INVOLVEMENT AND EMPOWERMENT 1. How was a community group (Pokmas) formed? 2. What was the role of the projects in the formation of Pokmas? 3. How was Pokmas selected to receive the projects? 4. How many poor villagers are members of Pokmas? 5. What trainings were held for Pokmas? 6. Do the local people know about the project? 7. Can information about the program be obtained by the public? 8. Do the local people know about project activities in there areas? 9. Were meetings of local villagers held to discuss projects to be carried out? 10. Are there mechanisms of handling people's complaints? Do the people know about the mechanisms? 11. Do the local people feel satisfied with the projects constructed? 12. How about the level of the continuity of the projects constructed? 13. What have been carried out by the local people to guarantee the continuity of the projects? 14. Did the local people carry out O&M activities? What are the mechanisms for it?