Document of The World Bank Report No: ICR2181 IMPLEMENTATION COMPLETION AND RESULTS REPORT IDA-H4180 and IDA-36750 ON A CREDIT IN THE AMOUNT OF SDR 15.3 MILLION (US$ 19.34 MILLION EQUIVALENT) AND A GRANT IN THE AMOUNT OF SDR 9.3 MILLION (US$ 15.00 MILLION EQUIVALENT) TO LAO PEOPLE’S DEMOCRATIC REPUBLIC FOR A POVERTY REDUCTION FUND PROJECT MARCH 22, 2012 Sustainable Development Department South East Asia Unit East Asia and Pacific Region CURRENCY EQUIVALENTS (Exchange Rate Effective March 22, 2012) Currency Unit = Lao KIP 7,990 = US$1 FISCAL YEAR October 1 – September 30 ABBREVIATIONS AND ACRONYMS AF Additional Financing APR Annual Performance Review AusAID Australian Agency for International Development CAS Country Assistance Strategy CDD Community-Driven Development EAP East Asia and Pacific Region EFA Education For All EGPF Ethnic Group Policy Framework FCRM Feedback and Conflict Resolution Mechanism FM Financial Management GAC Governance and Anti-Corruption GFDRR Global Facility for Disaster Reduction and Recovery GoL Government of Lao ICR Implementation Completion and Results Report IDA International Development Association IFRs Interim Unaudited Financial Reports IFSA Integrated Fiduciary Supervision Assessment ISR Implementation Status Report LUFSIP Lao Upland Food Security Improvement Project QAG Quality Assessment Group QALP - 2 Quality Assessment of Lending Portfolio QPR Quarterly Progress Report M&E Monitoring and Evaluation MIS Management Information System MTR Mid-Term Review NLCRDPE National Leading Committee for Rural Development and Poverty Eradication NT2 Nam Theun 2 O&M Operations and Maintenance PDO Project Development Objectives PDR Peoples Democratic Republic PG Poverty Grading PIU Project Implementation Unit POM Project Operational Manual PRF Poverty Reduction Fund PRFP Poverty Reduction Fund Project PSIA Poverty and Social Impact Assessment SDC Swiss Agency for Development and Cooperation SDR Special Drawing Right SIL Specific Investment Loan VNPA Village Needs and Priorities Assessment Vice President: Pamela Cox Country Director: Annette Dixon Acting Sustainable Development Leader: Julia Fraser Project Team Leader: Ingo Wiederhofer ICR Team Leader: Ingo Wiederhofer LAO PEOPLE’S DEMOCRATIC REPUBLIC POVERTY REDUCTION FUND PROJECT TABLE OF CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1.  Project Context, Development Objectives and Design ....................................................................... 1  2.  Key Factors Affecting Implementation and Outcomes ...................................................................... 6  3.  Assessment of Outcomes .................................................................................................................. 15  4.  Assessment of Risk to Development Outcome ................................................................................. 21  5.  Assessment of Bank and Borrower Performance ............................................................................. 21  6.  Lessons Learned................................................................................................................................ 25  7.  Comments on Issues Raised by Borrower/Implementing Agencies/Partners................................... 27  Annex 1.   Project Costs and Financing................................................................................................ 28  Annex 2.   Outputs by Component ....................................................................................................... 29  Annex 3.  Economic and Financial Analysis ....................................................................................... 40  Annex 4.   Bank Lending and Implementation Support/Supervision Processes .................................. 44  Annex 5.   Beneficiary Survey Results ................................................................................................. 47  Annex 6.   Stakeholder Workshop Report ............................................................................................ 48  Annex 7.   Summary of Borrower’s ICR and/or Comments on Draft ICR .......................................... 49  Annex 8.   Comments of Cofinanciers and Other Partners/Stakeholders............................................. 52  Annex 9.   List of Supporting Documents ............................................................................................ 53  Map (IBRD33431) .................................................................................................................................... 54  A. Basic Information Country: Lao PDR Project name: Poverty Reduction Fund Project (PRF) Project ID: P077326 L/C/TF Number(s): Credit #: 3675-LA Grant #: H4180-LA ICR Date: ICR Type: Core Lending Instrument: SIL, IDA(AF) Borrower: Lao PDR Original total commitment: Disbursed amount: SDR 15.3 million SDR 24.6 million SDR 9.3 million (AF) Environmental category: B Implementing Agencies: Lao Poverty Reduction Fund Cofinanciers /Other External Partner: Swiss Agency for Development and Cooperation B. Key Dates Process Date Process Original Date Revised/Actual Date(s) Concept review: January 24, 2002 Effectiveness: February 3, 2003 September 11, 2007 (AF) October 10, 2008 (AF) Appraisal: April 25, 2002 Restructuring(s): July 8, 2008 (AF) February 4, 2008 (AF) Approval: June 25, 2002 Mid-term Review: November 24, 2005 July 15, 2008 (AF) Closing: September 30, September 30, 2011 2008 C. Ratings Summary C.1 Performance Rating by ICR Outcome: S Risk to Development Outcome Moderate Bank Performance: MS Borrower Performance: MS C. 2 Detailed Ratings of Bank and Borrower Performance Bank Ratings Borrower Ratings Quality at Entry: MS Government: S Quality of Supervision: MS Implementation Agency MS Overall Bank Performance: MS Overall Borrower Performance: MS i C.3 Quality at Entry and Implementation Performance Indicators Implementation Performance Indicators QAG Assessments (if any) Rating Potential Prob. Project at any time(Yes/No): No Quality at Entry (QEA): 3 (MS) Problem Project at any time(Yes/No): No Quality of Supervision (QSA): 4 (MU) D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) 1. General Education 28 28 2.General Transportation 28 28 3.Other Social Services 25 25 4. Central Government Administration 11 5 5. Sub-national government Administration 8 14 Original Actual Priority Theme Code (as % of total Bank financing) 1. Rural Services and Infrastructure 23 45 2. Decentralization 22 11 3. Participation and Civic Engagement 22 22 4. Gender 22 11 5. Other Social Protection and Risk Management 11 11 E. Bank Staff Positions At ICR At Approval Vice President: Pamela Cox Jemal-ud-din Kassum Country Director: Annette Dixon Ian C. Porter Acting Sustainable Julia Fraser Emmanuel Y. Jimenez Development Leader: Project Team Leader: Ingo Wiederhofer / Helene Christopher Chamberlin Carlson Rex ICR Team Leader: Ingo Wiederhofer ICR Primary Author: Janelle Plummer ii F. Results Framework Analysis Project Development Objective (i) Assist villagers to develop community public infrastructure and gain improved access to services; (ii) Build capacity and empower villages in poor districts to manage their own investment planning and sub-project implementation in a decentralized and transparent manner; and (iii) Strengthen local institutions to support participatory decision-making and conflict resolution processes at the village, khet (sub-district) and district levels, involving a broad range of villagers, including women and the poor. Revised Project Development Objective The original PDO remained unchanged throughout the project. However, additional key performance indicators were added at Additional Financing. (a) PDO Indicator(s) Baseline Value Original Target Values Formally Revised Actual Values Achieved (from approval Target Values at Completion or Target documents) (from Additional Years Financing Project Paper) (i) Financing community infrastructure activities, and other community public capital investments endeavors and training activities at the village level. Number of PRF Number of infrastructure 1,873 - cycles I-V Cycles I-V training and facilities built, by (completed) 1,873 sub-projects investment projects district, compared to + [actual from AF] financed baseline data. 1,500 sub-projects - In 5 provinces, 21 cycles VI-VIII districts, 161 kum bans 2,340 sub-projects* 3,373 sub-projects 3,179 sub-projects In 5 provinces, 22 In 21 Districts (94% of revised target) Districts, 220 khets/kum at completion bans. In 7 provinces, 28 districts, 213 kum bans Date achieved 25-Jun-2002 25-Jun-2002 4-June 2008 16-Sep-2011 Comments Total number of districts reached. *Original targets above shown in Financing Agreement. The project (incl. % achievement) achieved 80% of the anticipated 2340 sub-projects in the original credit (cycles I-V); and 94% of the 3,373 in additional financing (cycles VI- VIII), including training sub-projects. Given the demand driven nature of the project, and the flexibility in the amounts allocated to individual sub-projects it was difficult to estimate the actual number and type of sub-projects to be delivered. The lower actual value achieved following Additional Financing would also be expected as the costs of individual sub-projects increased over the project to improve quality and the number of lower cost training and livelihoods activities declined as a % of the total sub-projects. (Note: The actual value of 3,373 reported in last ISR inadvertently included financing provided under component 3a of the Lao Upland Food Security Investment Project). iii Utilization rates of No baseline. No targets were set at Schools: public infrastructure appraisal  Number of students per On average 46 students and services provided teacher in PRFP schools per teacher (range 6-83) by the PRFP in sample villages. of sample.  >80% of sample villages with PRFP Enrolment rate of 90% of school sub-projects have schools visited.* net school enrolment rate >80% among children ages 6-10.3 No baseline. Health Dispensaries:  In >60% of sample 81% of sample had PRFP health centers, minimum package of minimum package of services. health services is provided.4 No baseline. Road/Bridges:  >80% of sample 50% of roads in sample villages with PRFP road of 12 had year round use sub-projects will have by small vehicle. access to district center by small vehicle all year round. No baseline Water and Sanitation:  >80% of sample 80% of villages opting villages with PRFP for water sub-projects had water/sanitation sub- access to clean water. projects have universal access to clean and safe water. Date achieved 25-Jun-2002 25-Jun-2002 4-June 2008 16-Sep-2011 Comments Data obtained from end of project Technical Utilization and Beneficiary Assessment. (incl. % achievement) *However the above assessment noted that the ratio of children enrolled, but not attending school was high, especially for girls. Overall quality of 0 No target at appraisal. >70% of inspected Overall: 71.2%* PRFP sub-projects PRFP sub-projects meet the PRFP technical Roads: 67%** guidelines. Irrigation: 50% Schools: 79% Water supply: 100% Health dispensary: 71% Date achieved 25-Jun-2002 25-Jun-2002 4-June 2008 16-Sep-2011 Comments Data obtained from end of project Technical Utilization and Beneficiary Assessment. (incl. % achievement) * Accounting for the proportions of each type of sub-project, and excluding capacity building projects over 71.2% of sub projects met technical specifications. **No standard designs were established for roads due to variation in topological conditions. Beneficiary N/A No target at appraisal.  > 70% of allocated 77% satisfaction with public sub-projects were goods and services identified as priorities at provided by the PRFP village level (during the VNPA meetings).5  > 80% satisfaction 81.1% level from the health clinic:70% beneficiaries regarding roads: 82.3% improved infrastructure deep wells: 76.7% and services. spring fed: 81.8% Date achieved 25-Jun-2002 25-Jun-2002 4-June 2008 16-Sep-2011 iv Comments Sub-projects coming from community lists. Data from PRF M&E unit. (incl. % achievement) Beneficiary data obtained from end of project Technical Utilization and Beneficiary Assessment. Pro-poor targeting 0 Number and % of poor khets / kum bans served.* khets / kum bans Cycle I: 60 Cycle I: 121 Cycle II:130 Cycle II:188 Cycle III: 150 Cycle III: 239 Cycle IV: 220 Cycle IV: 252 Cycle V: 220 Cycle V: 161 Target Number of khets / kum bans: 220 Villages** Villages: Target set at 50% of PG0-2 PG0-2 villages. Cycles I-V range 30-49% Cycles VI-VIII range 42- 84% 25-Jun-2002 25-Jun-2002 4-June 2008 16-Sep-2011 *Target figures for kum bans obtained from Financing Agreement. The project reached more than the target number of kum bans, then focused support on fewer poorer villages and increased unit costs to improve quality. **Monitoring shifted to the village level at AF to improve poverty targeting. - - % of infrastructure sub- % of infrastructure sub- projects in villages projects built in PG0 without road access villages was 12%; in PG1 No target was set at AF. villages was 40% (cycles VI-VIII). Date achieved 25-Jun-2002 25-Jun-2002 4-June 2008 16-Sep-2011 Comments Poverty targeting overall reached its targets. A target of 15% for villages without road access was set by the (incl. % achievement) task team after AF and entered in the ISR in 2010. Given these are the lowest levels of poverty grading, one can assume that all PG0 villages did not have road access, and that a significant number of PG1 villages also lacked road access. (ii) Building local capacity at the village level in poor districts to manage public investment planning and implementation Participation rate of 0 No. of proposals 31.6% attendance of women and small submitted by women and Women in PRF District ethnic groups in number funded. Decision Meetings planning and decision- making meetings. % of ethnic minority No target set at AF. 100% non-Lao speaking facilitators at khet and % non-Lao speaking facilitators at kum ban district level. facilitators at kum ban level. and district level % villagers participating See Intermediate % of women Outcome Indicators participating % of community contributions No. of community and district meetings. [no targets] Date achieved 25-Jun-2002 25-Jun-2002 4-June 2008 16-Sep-2011 Comments The team set a target of 15% in the ISR. Meetings consist of 33% PRF officials, 33% District officials and 33% kum ban facilitators, (i.e. 15% women represents just under 50% of the community participants). This target was reached. Attendance of non-Lao-Thai ethnic group in PRF District Decision Meetings-the team used as a proxy for the no. of non-Lao-Thai speaking kum ban facilitators. In practice, all kum ban facilitators for non-Lao speaking communities were from those ethnic groups and would attend the required planning and decision making meeting: half of those would be women (so the % attendance would be 100% and 50% respectively). The figure as a % of the total number of participants of the meetings is not illustrative as it is contingent on PRF and District officials. v Feedback and conflict 0 Not target set at appraisal No target set. resolution Number and distribution Number and types of No. of relevant of complaints and their cases received and complaints: 32 resolution resolved; number of days No data available on needed to resolve cases. complaint handling /redress. Date achieved 25-Jun-2002 25-Jun-2002 4-June 2008 16-Sep-2011 Comments In the latter stages of PRFI, the project redesigned the complaints handling system and being piloted a new approach in 2 districts in the final year of the Project. Community awareness 0 No target >70% of sub-projects Committees formed 100% and understanding of will have practiced benefits and maintenance plans after 63% maintenance funds responsibilities in the 3 years practicing after 3 years. PRFPs operation and process. Date achieved 25-Jun-2002 25-Jun-2002 4-June 2008 4-June-2011 Comments The number of maintenance plans was not quantified, however the number of O&M committees formed and number of funds established for a sample of projects were reported in the Technical Utilization and Beneficiary Satisfaction Assessment. Actual figures affected by inability of communities to pay for maintenance and rehabilitation of cyclone Ketsana damage over and above costs of routine maintenance. (i) Strengthening the capacity of local institutions to support participatory decision-making at the village, kum ban and district levels. Level of understanding 0 No target. Number of trained Data not available. of participatory officials who recognize approaches by the the significance of Output data shows that relevant local participatory approaches 181 training events on the government officials principles and approaches of PRF, implementation and O&M and participation reached 310,308 officials at provincial, district and kum ban level.* Proportion of relevant Data not available. line agencies officials holding regular dialogue with communities Date achieved 25-Jun-2002 25-Jun-2002 4-June 2008 4-June-2011 Comments This indicator was introduced to evaluate the understanding of bottom up participatory planning and top-down sector planning of public investment. The qualitative description in the Technical Utilization and Beneficiary Satisfaction Assessment showed that this experience was mixed, with some district officials still preferring the decisions to be made by technical officials and others recognizing the benefit of community decision-making. *Data from PRF M&E unit. (table 2.12) (b) Intermediate Outcome Indicator(s) 1 Baseline Value Original Target Values Formally Revised Target Actual Values (from approval Values Achieved documents) at Completion or Target Years Component 1: Sub project grants for infrastructure and training (PDOI) 1 Some figures previously included in the ISRs may seem higher than those reported in this results framework – due to either (i) the inclusion of LUFSIP funds or (ii) the ISR reporting Cycle VIII data. vi Sub projects and 2,340 sub-projects* 1,873 - cycles I-V 1,873 sub-projects training sessions In 5 provinces, 22 (completed) (Cycle I-V) approved, contracted, Districts, 220 khets / + In 5 provinces, 21 completed by sector, kum bans. 1,500 - cycles VI-VIII districts,252 kum province, district, kum 19 districts, bans, 1,880 villages ban and village. 1,640 villages 3,373 sub-projects (overall) 3,179 sub-projects* (overall) In 7 provinces, 21 districts. >80% of infrastructure 100% recorded.** recorded in official ministry books by Sept 11. 7.5% contribution >70% of villages, 21% roads/public community’s labor and in- works; kind contribution in sub- 16% health related project implementation is infrastructure; >10% 20% agriculture infrastructure.*** Date achieved 25-Jun-2002 25-Jun-2002 4-June 2008 16-Sep-2011 Comments *Because the types and size of sub-projects were demand-led and unknown, targets for the no. of sub-projects are difficult to define. The targets set at AF reflect the profile of choices that villagers were making. (incl. % achievement) ** All PRF projects were recorded by Line Ministries; however this did not prove to be a robust indicator of line ministry buy-in and sustainability. ***Community contributions to sub-projects exceeded targets but varied substantially across sector given the difficulties communities had in raising cash. Most contributions were in the form of labor. This is an outcome indicator of Component 2. Data is not available on the % of villages from which this contribution came. Component 2A: Community capacity building and empowerment % of Proposals 0.00 Number of proposals > 50% of village proposals 91% coming from submitted by women come from the women’s Women’s lists and number funded lists. Date achieved 25-Jun-2002 25-Jun-2002 4-June 2008 16-Sep-2011 Comments Data available for cycles VI to VIII. 88% reported in ISR (25-May-2011) refers to cycle VIII only. Average (incl. % achievement) across cycles VI to VIII is nearly double that anticipated. Component 2B: Capacity building of local institutions Participation of local 0.00 Types and number of 55 training events Government officials trainings provided to local held at district level, in District Decision government officials. with 19,293 Meetings No target. participants. List of training events provided in Annex 2. Participation of 622 officials were district/province officials at involved in district District Prioritization decision meetings in Meetings and District cycles VI to VIII Decision Meetings No. of village visits by No. supervision visits to District staff visited PRF staff and villages by the Provincial kum bans 4 times per facilitators; no. of and District staff. year as required by villages visited. the annual cycle. No. of provincial TA 681 community team members, district facilitators. facilitators, khet vii facilitators and PMT by month, quarter, year; district/province No specific targets were set for these indicators in the AF. Date achieved 25-Jun-2002 25-Jun-2002 4-June 2008 16-Sep-2011 Comments No target was set in the AF results framework; (incl. % achievement) The end evaluation noted very mixed results with regard to the engagement of district officials stating, on the one hand, that “there is a remarkable influence of district administrations in sub-project selection� and on the other, that officials complained they were not sufficiently involved. Overall data suggests that 13% of sub- projects were derived from district (not community/kum ban) priorities with very few in the latter years of the project. Table 2.9 Component 3: National project management Assessments and - No target No target. Regular routine reports submitted on reporting (QPR, APR, schedule IFRs) were submitted on time. Final evaluations and data for the ICR were significantly delayed. Date achieved 25-Jun-2002 25-Jun-2002 4-June 2008 16-Sep-2011 Comments (incl. % achievement) Ceiling on 0 <25% of costs spent on <25% 15% on average management costs national project management Date achieved 25-Jun-2002 25-Jun-2002 4-June 2008 16-Sep-2011 Comments See Table 3.1 (incl. % achievement) Audits contracted and Yes Yes Yes produced on schedule Date achieved 25-Jun-2002 25-Jun-2002 4-June 2008 16-Sep-2011 Comments Audits produced on time and unqualified throughout project. The audit report for the year ending September (incl. % achievement) 30, 2011 has been submitted and found to be satisfactory. Staff appointments - Staff appointments are Overall, 290 made competitively; staff consultants were hired are qualified and capable. on a competitive basis in the Finance, Community Development, Engineering and M&E units of the PRF office, including Directors and advisory positions. Date achieved 25-Jun-2002 25-Jun-2002 4-June 2008 16-Sep-2011 Comments The ICR mission noted the capability of staff on the whole, with some skill areas stronger than others. (incl. % achievement) Problems with turnover at the beginning of the project were resolved. Few positions remained vacant. viii G. Ratings of Project Performance in ISRs Actual Date ISR Implementation Disbursement No. Archived Development Objective Progress (US$ million) 1 8/1/2002 Satisfactory Satisfactory 0 2 3/5/2003 Satisfactory Satisfactory 0 3 9/30/2003 Satisfactory Satisfactory 1.06 4 3/8/2004 Satisfactory Satisfactory 1.88 5 7/7/2004 Satisfactory Satisfactory 2.14 6 12/28/2004 Satisfactory Satisfactory 3.17 7 4/6/2005 Satisfactory Satisfactory 4.65 8 1/20/2006 Satisfactory Satisfactory 8.08 9 12/12/2006 Satisfactory Satisfactory 13.34 10 9/24/2007 Satisfactory Satisfactory 18.19 11 6/25/2008 Satisfactory Satisfactory 22.68 12 6/25/2009 Satisfactory Satisfactory 26.34 13 5/1/2010 Satisfactory Moderately Satisfactory 31.27 14 6/26/2011 Satisfactory Moderately Satisfactory 36.51 15 10/11/2011 Moderately Satisfactory Moderately Satisfactory 37.27 H. Restructuring NONE I. Disbursement Profile ix 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal At appraisal, social indicators in Lao were some of the lowest in the East Asia and Pacific (EAP) region. Maternal and child mortality and illiteracy rates were extremely high. Most remote village and mountainous areas did not have schools, and almost 70 percent of the labor force was uneducated. To bring about a sharp reduction in poverty by 2005, the Government of Lao PDR (GoL) adopted a sustainable growth with equity strategy that was articulated through four pillars: agriculture/forestry, education, health and infrastructure, especially rural roads. The Government had also committed to a broad decentralization framework and to the creation of an environment conducive to poverty reduction at the local level. This included efforts to improve poverty reduction targeting in districts, villages and households. The PRF was designed to support these efforts by targeting poor districts with direct flows of investment and capacity -building to strengthen decentralized decision-making and implementation down to the village level. The primary challenges to the delivery of local services and infrastructure in rural areas were institutional, physical and sectoral: (i) the weak capacity of relevant government institutions to undertake a national scale project, (ii) the geographical remoteness and difficulty of physical access to a number of areas, and (iii) the highly diverse set of needs to be addressed involving many sectors. At the time of preparation, poor health and education indicators and low productivity were considered to be the result of low access to health and education services, weak agricultural extension, limited irrigation and a sparse road network. Reconciling the weak institutional capacity with this broad scope of operational activity was a critical dimension of the problem to be addressed. The rationale for Bank assistance aligned with three of the four CAS objectives. It intended to contribute to the efforts to reduce poverty through sustainable growth, to focus on social service delivery and infrastructure, and to consolidate donor and private sector support. The PRF would contribute to each objective by respectively: (i) targeting poor districts and allowing all villages within those districts to compete for resources, (ii) delivering small scale investments and services at the village level, and (iii) encouraging the participation and collaboration of the private sector in support of training, monitoring and evaluation. The intention to put in place a mechanism through which project resources could flow directly to villages was a fundamental decision of government supported by the Bank. The Bank engagement in the project was further justified by the complementarity of the PRF with other education, agriculture and infrastructure projects and, importantly, to establish a potential channel for the utilization of revenues from the Nam Theun Two (NT2) hydropower scheme for poverty reduction purposes. Government ownership and commitment was also high for the country context, with the strong commitment of government at central and provincial levels and across ministries, the commitment of leadership at political and operational levels. Given global and regional experience in community-driven development projects, the Bank was well placed to support the proposed project both in its design development and ongoing implementation. 1 1.2 Original Project Development Objectives (PDO) and Key Indicators The project specified three key development objectives: (i) Assist villagers to develop community public infrastructure and gain improved access to services:  Sub-projects approved, contracted, completed by sector, by district, by annual cycle  Quality of finished infrastructure  Increase in access to infrastructure/service  Demand of the poor of different sub-project types  Number and percentage of ethnic minority beneficiaries  Number and % of poor khets served.  Number and % of poor districts served.  % of beneficiaries satisfied with outputs.  Number of participating communities with maintenance plans.  Average unit costs of completed sub-projects by project type, by district, by province. (ii) Build capacity and empower villages in poor districts to manage their own investment planning and sub-project implementation in a decentralized and transparent manner:  Number of complaints filed with provincial PRFP office and percent resolved  Number of community procurements undertaken  Number of contractor procurement undertaken  Percent of community procurements being undertaken  Average amount of community procurement and contractor procurements  Number of male and female khet and district facilitators trained  Percent of male and female facilitators trained  Percent of ethnic minority facilitators at khet / district level  Number of village visits per khet per month by districts and khet facilitators. (iii) Strengthen local institutions to support participatory decision-making and conflict resolution processes at the village, khet and district levels, involving a broad range of villagers, including women and the poor:  Number of khet meetings and number of participants in each  Number of district forums and number of participants in each  percent of women participating  number of women attending special women’s meetings  percent of women participating in district forums  number of proposals submitted by women  number of proposals funded that are submitted by women  number of target districts participating in annual sub-project cycle  percent counterpart contribution by khet, by district, and by province annually. 2 1.3 Revised PDO and Key Indicators The overall PDO remained substantively unchanged at additional financing, although the wording was revised; indicators were reworded to improve measurability, new indicators were added and a number of new targets were established. PDO I: Financing community infrastructure activities, and other community public capital investments endeavors and training activities at the village level.  Utilization rates of public infrastructure and services delivered by the project, measured for each type of sub-project: - Schools: Number of students per teacher; >80% of sample villages with PRFP school sub-projects have net school enrolment rates >80% among children ages 6-10. - Health Dispensaries: in >60% of sample PRFP health centers, minimum package of health services is provided. - Road/Bridges: >80% of sample villages with PRFP road sub-projects will have access to district center by small vehicle all year round. - Water and Sanitation: >80% of sample villages with PRFP water/sanitation sub-projects have universal access to clean and safe water. - Overall quality of PRFP sub-projects: >70% of inspected PRFP sub-projects meet the PRFP technical guidelines.  Beneficiary satisfaction with the public goods and services provided by the PRFP: - >70% of allocated sub-projects were identified as priorities at village level (during the Village Needs and Priorities Assessment (VNPA) meetings). - >80% satisfaction level from the beneficiaries regarding improved infrastructure and services.  Pro-poor targeting: - % of infrastructure sub-projects in villages without road access. PDO II: Building local capacity at the village level in poor districts to manage public investment planning and implementation.  Participation rates of women and small ethnic groups in planning and decision making meetings.  % of non-Lao speaking facilitators at kum ban and district level.  Feedback and Conflict Resolution Mechanism: - Number and types of cases received and resolved; - Number of days needed to resolve cases.  Community awareness/understanding of benefits and responsibilities in the PRFP’s operation and process: - >70% of sub-projects will have operational maintenance plans after 3 years. PDO III: Strengthening the capacity of local institutions to support participatory decision- making at the village, kum ban, and district levels.  Level of understanding of participatory approaches by relevant local government officials: - Number of trained government officials who recognize the significance of participatory approaches. - Proportion of relevant line agencies officials holding regular dialogue with communities. 3 1.4 Main Beneficiaries The PDO captured the target of the PRF project as “communities in poor districts chosen on the basis of population-weighted poverty incidence�. All villages within a selected district were eligible to participate, with higher priority being given to villagers evaluated locally as poor. Beneficiaries included a broad range of poor villagers, including women and ethnic minorities�, and local institutions. 1.5 Original Components The project was structured in three components:  Component 1: Village Project Sub-project Grants. (US$16.10m total financing 74.2%, US$14.98m IDA). This component was designed to finance grants for eligible sub- projects identified by communities. Eligible local infrastructure projects included transport infrastructure, schools and nurseries, clinics, community halls and other public buildings, communal water supply systems, irrigation and drainage systems and local markets. Coverage of 21 districts in 5 provinces was agreed. The ceiling of any sub- project was set at US$25,000; works were expected to average US$6,400. It was anticipated that this menu of options would expand as capacity developed.2  Component 2: Local Capacity Building. (US$3.17m total financing 14.6%, US$2.17m IDA). The component was focused on the empowerment and capacity building at the community level. It was designed to provide support for Provincial technical support teams, as well as district and khet facilitators, including all operational, office and travel requirements to support the participatory processes at the sub-national level.  Component 3: National Project Management. (US$2.44m total financing 11.2%, US$2.19m IDA). This component supported central level staffing, management, monitoring, reporting an operating the Poverty Reduction Fund, including all operational costs and technical assistance. 1.6 Revised Components Additional Financing (US$15.0 million IDA) was approved on July 15, 2008. It was supplemented with US$4.8 million parallel financing from the Swiss Agency for Development and Cooperation (SDC). The additional financing was provided to continue supporting the development of local infrastructure and community empowerment in the poorest districts in another three cycles of implementation until a revised end date of September 30, 2011. The project was scaled up and restructured to enhance development impact by (i) refocusing activities on government’s “first priority� districts only; and (ii) enhancing training of government officials (not only local PRF staff) in participatory approaches.3 2 The FA showed 22 districts and also noted that all villages within a targeted district were eligible to participate in the Project. 3 The FA for AF notes the project components as Part A: Village Sub-project Grants - Carrying out of Sub-projects through the provision of Sub-grants made available by the Poverty Reduction Fund out of the proceeds of the Grant to a village or group of villages to finance the carrying out of the corresponding Sub-project in accordance with the provisions of the relevant Sub-grant Agreement. Part B: Capacity Building for Communities and Local Institutions (a) Strengthening of Beneficiary capacity to implement Sub-projects under Part A of the Project, including Sub-project identification, planning and implementation, through 4 Accordingly, revisions were made to the following components and funding allocations:  Component 1: Village Project Sub-project Grants. (US$8.37m additional financing IDA; US$4.8million SDC) Component 1 was revised to ensure the coverage of the project improved pro-poor targeting. The project removed seven districts that were not considered government “first priority�, added 9 others in the target provinces and included another province (Luang Namtha) with three districts, resulting in a new set of 19 first priority target districts. The change in coverage did not aim to reach more beneficiaries but rather focus assistance on those with the greatest need. Another change was the modification of the eligible list of sub-projects to exclude support of livelihood activities which, as private goods, broadened the scope of the project and required a different set of technical skills which the project was unable to deliver effectively across such a wide set of districts. After Additional Financing, the final allocation to Component 1 represented 71.48% overall.  Component 2: Local Capacity Building. ((US$2.93m additional financing IDA). In addition to the community capacity building elements of this component (US$2.44 million), a new sub-component, Capacity Building for Local Institutions (Component 2b), was added (US$0.49 million). The new component was introduced in 2008 to assist district officials in their dialogue with communities, increase district and province officials awareness of the benefits of participatory planning, and training them in participatory planning methods, and supporting capacity building manuals and more frequent visits to villages.  Component 3: National Project Management (US$3.69 additional financing IDA). Continued support for national level management through the established Project Implementing Unit. 1.7 Other significant changes The activities funded with the additional financing remained consistent with the original project objectives. Implementation arrangements through the PIU were retained. Other efforts to improve the progress toward the PDO were included and approved at additional financing:  Targeting of non-Lao speaking ethnic groups through an information, education and communication strategy and recruitment of skilled staff (communications officers and translators) able to better involve non-Lao speaking communities.  Increased collaboration with line agencies through national level dialogue with line ministries and through enhanced support at the district level included under the new sub- component 2(b).  Increased focus on O&M to be achieved through the development of maintenance plans with assigned responsibilities for communities and relevant district level line agencies for improved sustainability of sub-projects. the provision of technical assistance, training, and goods for provincial technical support teams, and district and Kum ban facilitators; and (b) strengthening the Recipient’s institutional capacity at the district and provincial levels to improve quality of service delivery, promote participatory planning and facilitate dialogue between districts and communities, though the provision of training and technical assistance. Part C: National Project Management Supporting the management and operation of the Poverty Reduction Fund through the provision of technical assistance, vehicles and other goods. 5  Changes in the technical design of the project. Reflecting lessons learned, a number of technical changes were included in the Project Operational Manual (POM) following additional financing. These included revisions to the technical guidelines to improve construction quality standards, improved communication efforts to promote understanding of the PRF principles of community engagement.  Changes to improve pro-poor targeting of infrastructure. Village infrastructure endowments were surveyed annually from 2008 to determine where infrastructure sub-projects should be focused. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design, and Quality at Entry Soundness of the background analysis. The background analysis for the project recognized the challenging context of Lao PDR at the time of preparation and appraisal. There was a strong rationale for the project within the CAS. The involvement of communities in making decisions over local development and investment was new to Lao, and the preparation adequately took account of the national context. The PRF was developed within the framework of the burgeoning community-driven development (CDD), social fund and local development fund operations, being adopted throughout the East Asia region at the time, and relevant lessons were thoughtfully incorporated. The counterpart team was given the opportunity to see and discuss the model through exchanges with Indonesian government counterparts. The project effectively supported the government’s ambitious strategy and targets for improving human development indicators and focused on achievable ends. Further analysis at preparation, however, particularly around local political economies, may have enabled more detailed preparation and subsequent support for the implementation processes envisaged. Project Design. The PDO focused on the development of relevant goals toward infrastructure services, capacity building and community participation. These were innovative and achievable within the scope of the operation and engendered a new approach in the context of a centralized state. The selection of the social fund model was consistent with the strong focus of the GoL on poverty reduction through enhanced basic services delivery and rural infrastructure. The model incorporated the potential benefits of bottom up planning and village management of funds to enhance development effectiveness, though it did not pursue the more rigorous empowerment ends of the CDD model. This was appropriate given the highly centralized system and the narrow space for voice and participation outside government and state affiliated structures prevailing in Lao PDR at the time, and also the low capacity of remote communities for grassroots financial management. A second key parameter of the project design was the selection of the institutional arrangements, in particular the establishment of an autonomous project implementing agency attached to the Office of the Prime Minister.4 The project team and government counterpart considered obvious 4 The PRF was legally established by decree as an autonomous entity attached to the Prime Minister’s office, reporting to an Administrative Board. The executive director of the PRF managed a pyramidal structure of national, provincial and district level consultants who carried out technical, administrative, fiduciary and community development roles. The organizational structure is decentralized, with most tasks being decided and carried out locally. Links are made to the sector ministries at each level and to 6 alternatives (e.g. implementation through line departments). The decision to ring fence the Fund and focus on delivering infrastructure services and piloting new methods was made with due note of ambitious service delivery targets and the limited capacity and understanding of local government structures. It also seems to have been prudent in hindsight, as the decentralization policy and structures under discussion at appraisal during the project implementation period. If there is a question about this institutional arrangement, it is not about its appropriateness at appraisal but whether this model should have evolved to support the development of evolving sub-national institutional structures. The project was complex by nature of its diversity and coverage of micro-projects at the grassroots level. It was also highly innovative for Lao PDR at the time. Community participation at this scale had not been attempted; a top-down decision-making model was the norm for the types of services and infrastructure envisaged. The PRF was established with one primary, well- positioned government counterpart and a set of three straightforward components: grants, capacity building and project management. The three components were rational and provided a sound structure for implementing, budgeting and reporting. Social and environmental impacts of these micro-projects were modest. Although the three parts to the PDO reflected the key ingredients of a social fund (service delivery, community participation and institutional strengthening), their structure and equal weighting were problematic. The PRF’s PDO results framework was also loosely defined, indicators were a mix of outcomes and low level outputs, many without targets, and they did not adequately capture the capacity and empowerment of village management and strengthened local institutions. The overall design of the PRF was relevant and workable, although details needed for effective implementation were lacking – especially given its innovative nature. The challenges of delivering and maintaining small scale infrastructure in such remote areas were not fully recognized, and the potential safeguards issues could have been given more attention. The participatory process, fundamental to the project approach, was not fully disaggregated across the stages of participation. The implications of cultural and linguistic diversity for effective participatory processes were not adequately considered. Given the challenges of working in a state with a high propensity for top-down processes, with marginalized ethnic minority and women beneficiaries and multi-income target districts, more detailed preparation may have led to more effective participation in the start up phase. Government commitment. The PRF has enjoyed high-level political commitment throughout. It was anchored by the Prime Minister’s Office, the Administrative Board of the PRF being chaired by a Deputy Prime Minister. The long-serving Executive Director of the PRF was well respected and an effective advocate for the project. This commitment has been consistent throughout from those engaged in the design, oversight and direct implementation of the PRF. The project has not however always enjoyed the full support of all line ministries, with some questioning the participatory approaches that underpin PRF delivery and others the institutional autonomy of the PRF. Nevertheless, the contention that greater commitment at entry from line agencies may have the governor’s offices at provincial and district levels. However, the PRF functions as a largely ring fenced service delivery agency. 7 resulted in more coordination remains unproven. The weak cooperation of some line ministries at the national level only emerged over time, and resolution of technical issues raised by ministries (e.g. technical standards, coordinated planning and operations and maintenance) is unlikely to have solved the inter-governmental issues encountered. Cooperation at the provincial and district levels has largely been satisfactory. Assessment of Risks. At appraisal the overall risk rating of the project meeting its PDO was considered Modest (M). Three main risks were identified; (i) political interference in PRF operations (M), (ii) community groups unable to formulate proposals (M), and (iii) not enough facilitators qualified to work with communities (M). The strong leadership of the PRF ensured it was able to establish its autonomy and was not politically captured at the national level. Community groups formulated proposals. Facilitators were identified and overall project management ensured the annual processes were implemented. However the overall assessment of risk at entry was implausibly low, given that the project was new in the Lao context, challenging in any context, and contained many unknowns that were impossible to anticipate. In hindsight, substantial risks in reaching the poor and local elite capture emerged, high risks with regard to local capacity and grass roots procurement and financial management capacity became apparent quickly, and risks regarding sustainability and technical quality emerged over time. While the poverty targeting risk was adequately addressed, the capacity risks were only partially mitigated through revisions at Additional Financing. The risks framework, rightly, introduced a risk concerning the misallocation of resources (missing at appraisal). Sustainability risks might have also been included. The Quality Assessment of Lending Portfolio – 2 (QALP-2) undertaken by the Quality Assurance Group (QAG) of the World Bank in 2010 rated the Quality of Design as Moderately Satisfactory (MS): Strategic Relevance and Approach were rated Satisfactory (S); Realism of Project Design and Risks was rated Moderately Satisfactory (MS); and Fiduciary, Safeguards and Governance and Anti-corruption aspects were rated Moderately Satisfactory (MS). 2.2 Implementation Major factors contributing to effective implementation. The implementation of the project was relatively smooth for a complex project with a pyramidal implementation structure reaching from the national level down to villages. Government commitment at the highest level enabled the machinery of government, including the Ministry of Finance, to support the PRF to work with autonomy and thereby focus on its poverty reduction ends (e.g. in the management of the special account). Management was effective; there was a strong champion of the project from design to completion. A stable cadre of mid-level managers emerged, and they have good knowledge of the delivery systems. Major factors affecting implementation. Following a delay in start up, disbursement throughout the project was consistent and on-track, exceeding the estimated disbursement profile during the Additional Financing phase. The key factors affecting implementation were:  Staffing and compensation. In the first years of implementation, the project suffered from a high turnover of staff (35% in 2005) particularly in key senior positions, due to inadequate compensation. A salary grid was put in place and phased increases in salaries were established in early 2007. 8  Weak capacity at the lower levels of implementation. Staffing district positions (and some provincial) proved difficult given the remoteness of many locations. Training was limited, and although this improved as the project became established (and was boosted at Additional Financing), overall, training of PIU staff was insufficient.  Working in decentralized environments. Other concerns that flow from a decentralized project implemented by communities included: the lack of banks in some provinces (e.g., Saravane), low capacity of kum ban facilitators and an inadequate market of private contractors willing to work in remote locations.  Inter-ministerial collaboration. Problems with line ministry collaboration at central level were raised at various times. While this did not affect progress in the funding and implementation of sub-projects, it did affect progress in relation to local capacity building. Practical coordination at the lower levels improved, with signs that some provinces have sound coordination and relationships at district level.  Managing Information. At the national level the delays in putting in place a robust and reliable MIS system affected early efforts to collect and manage data. The primary external factor to affect implementation was the 2007-8 price increases for construction materials, which caused some contractors to cancel contracts because they could not complete them for agreed prices. The mid-term review (MTR) was conducted in December 2005, by which time disbursement was largely on track. The government had established a functioning PIU and 1,347 sub-projects had been completed. The MTR itself did not employ a strategic evaluative approach that considered relevance, appropriateness, effectiveness and efficiency of the project, but it identified a few practical concerns (staff turnover training of PRF staff, sub-project size) and slowed project expansion. A more strategic and rigorous approach to the MTR might have enabled earlier identification of issues that were then only addressed at Additional Financing. A significant implementation quality concern emerged during cycle VI, when field visits and an internal assessment raised concerns regarding the accuracy of poverty targeting. Overall, the proportion of projects implemented in the three poorest grades of villages was in the region of 67%, with the poorest targeting occurring in the first cycle (53%) and the highest in cycle VII (75%). In cycle VI, sub-projects implemented in the group of villages (PG0-2) was as low as 22%, while the relatively better-off villages in the target districts received the lions share (78%). By 2009, following the introduction of a quota for targeting of poorer villages on the basis of annual assessments, the project staff became familiar with the issue and met targets of 66% of sub-projects and at least 50% of sub-project budgets. The preparation of the Additional Financing (AF) grant prompted deeper reflection and more robust analytical work on project strengths and weaknesses, which helped to inform the design of the AF phase. The AF attempted to address the weakness in the capacity building components as well as geographic coverage and targeting. While the latter improved, the sustained capacity- building investments necessary to systematically sensitize and engage local government officials, and to effectively strengthen the capacity of remote and poorly educated communities, remained insufficient. 9 The QALP-2 undertaken in 2010 noted that the PRF implementation progress was Moderately Satisfactory, recognizing that solutions were found to emerging problems but also finding weaknesses with respect to targeting the poorest, compliance with procurement procedures, supervision of social and environmental safeguards, and the weak buy-in and capacity of local government. These were all critical to the project implementation but the QALP-2 process was completed too late to have significant impact on project implementation. The project was not considered at risk at any time during implementation. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization The project design noted the importance of Monitoring and Evaluation, allocated sufficient budgets and established M&E as a primary task of the PIU but preparation proceeded without a reliable baseline. A monitoring framework was agreed at appraisal and included a range of instruments at the village, khet, provincial and national levels. A Management Information System (MIS) was envisaged along with the staffing necessary to manage the various monitoring and reporting processes. However, key indicators put in place at appraisal lacked clarity and specificity in places, and included output indicators (no. projects built, no. proposals submitted, no. visits) that were not adequate to assess progress toward the PDO. Targets were not set for most indicators. The indicators were revised at Additional Financing in 2008 with an effort to improve the relevance and measurability of indicators, but some targets were still not set. While the revision led to a more rigorous results framework which focused more on outcomes and clarified intermediate outcomes by component, the lack of targets for several indicators remained a significant shortcoming. Monitoring and evaluating a project with such broad coverage is a difficult task, and the project faced and resolved a number of challenges during implementation. The process of developing the M&E systems in PRF was a learning process, with gradual improvement over the duration of the project. In particular, the project initially struggled with a low-cost in-house system that compiled information of variable quality. The MIS was substantially upgraded in 2005 and became a robust and reliable management information system by 2007. Data quality gradually improved. Though a baseline was not produced and data was not analyzed in the early years, analysis did improve over time and ultimately provided information on whether PRF was/was not reaching the poorest communities. Qualitative monitoring was not undertaken or not utilized effectively until the latter years of the project (e.g. the sustainability report). Perhaps most importantly, the project adapted en route with the adoption of village poverty profiles for targeting purpose from 2008 onwards, when experience showed that the poverty data provided by the National Statistics Center did not accurately reflect realities on the ground. Although the Bank conducted a useful evaluation in early 2008 to inform preparation of the Additional Finance phase, evaluation is a particular weak spot and back loading of evaluations to the end of the project contributed to a general lack of familiarity with the content of the results framework and a lack of awareness of impacts as they emerged. This exposed the project to risks that might have been avoided if corrected earlier in implementation. The process of obtaining information for the ICR underlined the internal weakness in data utilization and analysis; the independent assessments which were to provide the necessary information were delayed and inadequately managed. The PIU recognizes the limitations of its M&E capabilities, and efforts are underway in PRF II to strengthen the capabilities of the unit. 10 2.4 Safeguard and Fiduciary Compliance 1. Safeguards The project was correctly categorized as category B – partial assessment. Although projects were small in size, two safeguards were triggered: environmental assessment (OP4.01) and indigenous peoples (OP4.10). Safeguard issues arose during project implementation but were minor and mitigated through standard procedures.  Environmental Assessment. Compliance with OP4.01 on Environment Assessment was rated satisfactory throughout the project. The project’s village grant component had modest environmental impacts which were due mainly to the construction or refurbishment of roads and issues of water quality. . The 2010 QALP-2 noted that there had been insufficient monitoring of project compliance with the environmental assessment safeguard. Subsequent to this, a mission was conducted and found that there were no significant negative impacts due to civil works, but noted a lack of safeguard reporting, and limited understanding of safeguard issues and procedures during sub-project selection. As a result, monitoring was stepped up, additional training was provided and World Bank staff in-country were enlisted to provide environmental safeguard support in the implementation support missions in the last eighteen months of implementation.  Indigenous Peoples. Compliance with OP4.01 on Indigenous Peoples was also satisfactory. The project data shows that ethnic minorities comprised on average 36% of the project’s beneficiaries. The participatory approach adopted for planning and implementation of the project ensured that the needs of ethnic minorities were addressed in a culturally sensitive manner. The results framework included a specific target that could be monitored (no. of non-Lao speaking facilitators) though it did not prove to be a robust indicator. As part of the project preparation, a social assessment described the particular problems associated with improving access for ethnic groups to services. At additional financing the focus on reaching ethnic minorities was improved through the pro-poor targeting. Other social safeguard issues which emerged were isolated, minor problems rather than systemic non-compliance. While these cases were managed transparently once the PIU became aware of them, they do raise some doubt about the appraisal of social safeguards. In particular:  Involuntary Resettlement. The Involuntary Resettlement safeguard (OP4.12) was not triggered by the project. With respect to the issue of resettlement, the PRF documents note “forced (involuntary) resettlement is not allowed under any conditions and voluntary resettlement requires an early special approval from Vientiane prior to any decision in the inter khet meeting (POM, 2003). In the main, this requirement was understood at national and provincial level and largely respected. The PRF’s approach effectively provided a local development model for remote communities utilizing participatory approaches. However, the GoL’s national village consolidation policy was not without effect on the project. The POM specified that PRF funds were not to be used in (i) villages that had been recently resettled, or (ii) villages where there were imminent plans for consolidation. All road sub-projects were for road rehabilitation, as a result of which right-of-way and land ownership issues applied only to marginal road widening. 11 A Poverty and Social Impact Assessment (PSIA) conducted by the World Bank in 2007, noted that cases of resettlement had been reported, mostly where Provinces had not followed project guidelines and procedures. In all, about 18 out of the 1,880 PRF beneficiary villages (1%) were flagged as having possible resettlement issues: three villages in Bachieng district and two villages in Soukouma district (Champasak province), one village in Samuai district (Saravan province), and 12 villages in Nalae district (Luang Nam Tha province)). Of the latter, 11 are potentially at risk of resettlement as a result of a planned hydropower project not associated with the World Bank. For these villages, the PIU has confirmed that it is following up on the resettlement plan to ensure that if relocated after the closing of the project, these villages will receive appropriate compensation and replacement for the infrastructure provided by the PRF project. This will be monitored carefully under PRF II, which has triggered the Involuntary Resettlement safeguard. 2. Fiduciary compliance a. Financial Management Financial Management (FM) arrangements in PRF met the minimum requirements of OP/BP 10.02 and operated fairly smoothly over the course of the project. PIU staff gained continuous training and experience until their performance surpassed that of most other projects; systems (accounts packages and an internal control unit) were gradually put in place. The project rightly identified risks regarding advances and these were tightly controlled; quarterly Interim Unaudited Financial Management Reports (IFRs) were submitted on time and were mostly accurate; annual audit reports were also timely and consistently unqualified. The Bank conducted annual transaction reviews and an integrated fiduciary review, which helped the project to develop capacity and focus on weaker areas. Those internal control issues that emerged did not raise significant FM flags. Nevertheless, a number of FM risks were identified at appraisal and persisted throughout the project: the risk of misallocation of resources at the local level in the 300-400 transfers made annually, the scale of the operation, and the weak capacity of the country. The development of a steady track record also meant that PRF became the vehicle of choice for disaster response activities after 2009, over-stretching PIU staff and resulting in minor problems (in reporting, records, trainings). Community-level procurement and financial management was, unsurprisingly, challenging in an environment marked by low levels of literacy, numeracy and understanding of basic book and record keeping. The fiduciary reviews nevertheless noted that some provinces/districts had stronger FM practices than others, indicating that concentrated capacity building, monitoring and compliance checks could result in improvements in FM performance. The QALP-2 noted weaknesses in FM and procurement (see below), suggesting a risk of a disconnect between fiduciary performance and supervision ratings. However, the Integrated Fiduciary and Safeguard Assessment (IFSA) undertaken by the Bank in 2011 rated PRF as Satisfactory. The report noted that (in the provinces visited) Provincial staff were capable, but raised questions about the appropriateness of procedures applied given the significant level of community payments. A checklist procedure for all community payments has since been put in place and is being implemented in PRF II. 12 b. Procurement Procurement in PRF can be considered in terms of (i) national level procurement, mainly the procurement of goods, services, consultants (conducted through a standard procurement plan at the national level) and (ii) the highly decentralized procurement of sub-projects at kum ban level.  With regard to national level procurement, subject to World Bank prior and post review, PRF had a moderately satisfactory track record with minimal deviation from procedures.  With regard to decentralized procurement, between 248 and 548 contracts are awarded annually to multiple beneficiaries. The project has also seen over 65% sub-projects procured by kum ban representatives for the construction of facilities (schools, health clinics) and infrastructure (roads, bridges, culverts). Procedures for these procurements were specified in the POM, and were subject to post review. Thirteen post reviews were carried out over the duration of the project, and an IFSA was carried out in early 2011. These post reviews documented minor procedural non-compliances in 28% of the contracts reviewed. This is not inconsistent with other CDD projects in EAP where experience suggests that minor procedural issues arise in an isolated, non-systemic way. Up to 2010, no technical deviations had been identified though the number of technical supervision visits was limited in coverage and scope. Over the 8 year duration of PRF I and thousands of contracts, only one instance of a possible indicator of collusion (2010) on a sub-project was reported. The QALP-2 review of 2010 raised a number of questions regarding the procurement performance of PRF and rated procurement as Moderately Unsatisfactory (MU). Two procedural non-compliances were noted in the review: (i) the practice of price negotiation with winning bidders resulting in a lower final price, and (ii) less than 3 contractors bidding and contracts being awarded. The PIU argued that, while on the whole they could and would meet these requirements in typical situations, on occasion, especially in remote mountains areas, these procedures became a constraint to the implementation of sub-projects in poor rural villages – the difficulty of finding contractors to take on construction contracts a long distance from district centers over difficult terrain was a significant constraint to implementation; and that neither practice resulted in inflated prices of contracts. Several issues raised by the QALP -2 review had been addressed during the Additional Financing design – improving procurement capacity and tightening procedural controls. Nevertheless, further efforts were made following the QALP -2 to enhance supervision and take additional corrective actions in light of the recommendations. The IFSA conducted in 2011, while limited in scope (and noting a possible non-compliance of the supply of materials on a community contract, rated procurement on 7 randomly selected sub- projects as Satisfactory (5) and Moderately Satisfactory (2). The procurement challenges of such a broadly decentralized project operating in remote areas seem to have been underestimated at design and under-supported during implementation. There are indications that the project was slow to act on recommendations and similar requests were repeated each year. Comparatively, the procurement capacity of the PRF PIU is still considered significantly stronger than that of most other projects in Lao PDR. PRF II has further tightened procurement controls, and enhanced procurement staffing. 13 2.5 Post-completion Operation/Next Phase The PRF II project became effective on October 12, 2011, with a gap of only 12 days following the closure of PRF additional Financing phase. The overall success of PRF I from the perspective of government and Development Partners was also reflected in the co-financing arrangements put in place for PRF II, with the GoL (US$10 million), the SDC (US$10 million) and the Australian Agency for International Development (AusAID, AUD20 million) more than matching the IDA Grant (US$25 million) for PRF II. Counterparts and the task team worked intensively to ensure that the transition to PRF II was smooth. PRF II, which will run until 2016, anticipates broadening the project footprint to reach approximately 275 kum bans. As it will retain about 50% of PRF I kum bans, it will continue to address the capacity of these communities. The budget of US$65 million is also more targeted to allow for more intensive support in fewer villages than was achievable in phase 1. In response to a GoL request to reinstitute support for livelihoods through the PRF, a livelihoods and nutrition pilot is also being implemented with parallel financing from a Japanese Social Development Fund grant (US$2.6 million). Two key issues affect the sustainability of completed sub-projects: buy-in from line ministries for the maintenance of infrastructure, and the continued commitment of communities. While this remains a critical concern and there are instances where the breakdown of these two requirements has affected the utilization and satisfaction of services, on the whole the project reached the targets set for operations and maintenance. Community contributions were 25%, well over the target of 15%; O&M committees were formed for 100% of the infrastructure and facilities constructed; and 63% of these committees collected maintenance funds (albeit minimal) and practiced maintenance three years after completion. All infrastructure and facilities were recorded in line ministry books and maintenance arrangements were routinely formalized. However, the commitment and capacity to maintain assets financed through PRF vary in practice. Limited sector budgets affect the levels of maintenance. This is especially acute for roads sub-projects, as the GoL has generally not allocated sufficient budgets for the maintenance of rural roads. The learning dimension of PRF improved over the duration of the project. PRF II has taken on board many of the lessons of the first phase of the operation. It is notable that these lessons have been taken up and applied at various levels – from the PDO down to village level procedures. Of particular note is that the PDO is more sharply focused on access to infrastructure and services, noting that capacity building and participation are a means to this end. Technical revisions included measures for improved poverty targeting. A revision of GoL policy shifts the focus of poverty targeting from the district to the narrower focus of the kum ban as the unit for poverty targeting. Multi-year investment plans will be introduced to improve the participatory process and strengthen linkages with local government and sector planning and budgeting. Four rounds of grants per kum ban will deepen investment to improve impact and enhance participation through repeated processes. The use of sector guidelines and personnel in project appraisal and supervision, as well as procedural changes will be established to improve technical quality. PRF II also foresees a greater focus on gender, and the feedback and complaints handling mechanisms. Fiduciary controls have been tightened through the establishment of an internal controls unit and the recruitment of additional procurement personnel. Finally, there is an increased focus on M&E, including plans for regular qualitative analysis and an impact 14 evaluation. While these revisions address a number of operational issues and are likely to improve effectiveness, the issue of sustainability is still a risk recognized by the project. Recommendations for the future. Future efforts might also focus on: (i) greater collaboration with other projects in Lao PDR which are largely concerned with the strengthening of local governance; (ii) strengthening of project governance features to further enhance project transparency and accountability; (iii) more community and local government capacity building; (iv) more rigorous qualitative analysis and experimental fieldwork, especially in relation to gender, voice and engagement; and (v) strengthening of the PIU M&E team. See Section 6. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Rating: Satisfactory The objectives and overall design of the PRF were relevant at appraisal, additional financing and at the time of closing in 2011. The PRF project strongly supported key priorities of the Lao PDR to achieve a sharp reduction in the incidence, depth and severity of poverty through four strategic pillars (agriculture/forestry, education, health and road infrastructure). By supporting infrastructure building and community empowerment, PRF is in line with the National Growth and Poverty Eradication Strategy and the National Socio-Economic Development Plan (2006- 2010). It also supported the complementary objectives of the CAS (2005), in particular the focus on social service delivery and investments in infrastructure. The operation was central to achieving country and Bank objectives. At Additional Financing, the project adapted to a critical lesson regarding local government capacity building and refocused efforts to ensure poor villages were benefiting first and foremost. As completion approached, in order to continue its contribution to poverty reduction, government requested a second phase of the project building upon the same principles, approaches and implementing arrangements. One of the overall project achievements, and an indicator of relevance, is that the GoL embraced the PRF approach as a key vehicle for rural poverty reduction. 3.2 Achievement of Project Development Objectives Rating: Satisfactory Project performance was satisfactory in achieving the primary part of the PDO – improved access to public infrastructure and services at the village level. Linkages to the poverty reduction overarching goal were direct and outcomes visible. The project achieved a very broad coverage in poverty-related investments. Largely consistent with targets set at Additional Financing, the project implemented 3,179 sub-projects in 28 poor districts in 7 provinces: built 616 schools, 86 bridges, 65 health dispensaries, 669 clean water systems and 156 irrigation schemes; over 3,000 km of rural roads were upgraded. The focus on education (34% of total budget), public works and transport (32%), health/water and sanitation (22%), agriculture infrastructure (4%) provides a strong indication of its focus on rural basic services. With regard to sector indicators, the end of project assessment of a sample of built infrastructure found that services were being provided. Schools had on average 46 students per teacher (range 6-83) with an enrolment rate of 90% of the schools visited. 81% of health clinics in the sample offered a minimum package of health services to local communities. While 80% of villages opting for water sub-projects had access to 15 clean water, only 50% of roads in the sample had year round use by small vehicle. With the exception of the all-weather requirement for roads, these values suggest that most sector targets, while modest, were met. Technical quality targets (>70%) were reached overall, but varied substantially, from the highly successful water supply systems (100%), to schools (79%) and the few but problematic irrigation sub-projects (50%). Over the eight cycles of implementation, 90- 95% of the sub-projects provided remote rural villages with infrastructure and substantially improved their access to services. The average number of beneficiaries per year (cycle) was over 400,000. Beneficiary satisfaction was on average 81.1%. The sector-wise breakdown was as follows: health clinic (70%), roads (82.3%), deep wells (76.7%), spring fed water supply (81.8%), electricity (100%). Community halls (60%) and market halls (65%) received much lower satisfaction ratings and were added to the negative list at Additional Financing. After some initial challenges, poverty targeting improved over the life of the project. In cycles I- V, the percentage of sub-projects and/or budgets spent in the poorest villages (categories PG0-2) only ranged from 30-49%. This was due, inter alia, to inadequate poverty data, a lack of defined poverty targets, and the weak capacity of PRF staff at start-up. By cycle VII and VIII however, this expenditure had reached 78% with project team efforts to carry out poverty profiles, clarity on the PRF targeting and more skilled facilitators, reaching out from district centers. Overall, this accounts for about 58% of the budget being spent in the poorest villages (PG0-2). The proportion of ethnic minority beneficiaries (about 36% overall) also suggests that the project was largely successful in targeting poor villages and meeting priority needs in difficult, remote areas of the country. Parallel interventions, notably the GOL’s intensive efforts at rural electrification, converged to bring about visible increases in access to public infrastructure. These sub-projects were delivered efficiently overall – notwithstanding the normal caveats of CDD projects such as thinly spread budgets and the challenges of maintaining technical quality. Unit costs were low when compared with other government built infrastructure (35-42% of the cost of roads and 82-88% of the cost of schools). Moreover the project delivered these results utilizing a very different approach than had been seen hitherto in Lao, establishing the seeds of CDD and introducing a new needs-based approach to local development. Critically it did so in the most remote and some of the poorest parts of the country. Given the overarching goal of the project, this objective – access to village infrastructure and services – has a significantly higher weighting than the other capacity building objectives. Performance against the second objective (Building Local Capacity at the village level) was moderately satisfactory. A total of 1,113 training events were conducted in kum bans, for approximately 68,000 community members. Capacity improved and communities were empowered to play a central role in project processes, although the fieldwork conducted for the ICR highlighted the lack of women’s engagement in public meetings. 852 community members (kum ban representatives), including women, were mobilized to participate in district level decision-making processes, manage project planning, implementation and finances. The accountability meeting, a social audit process, was introduced by the project. This project was largely alone among other top-down development interventions in its efforts to build capacity for participation. These achievements, while limited, can be directly attributed to PRF interventions. 16 The primary indicators for this objective did not set many numerical targets, but did include indicators that would start to measure community participation and capacity. The attendance of women at district decision meetings was sound (50% of community representatives) (although the indicator itself is unclear); all facilitators at kum ban level were non-Lao speaking, community contributions reached 25%, well over the 15% target, and the indicator which measured responsiveness to women’s priorities suggests that 91% of the sub-projects were on women’s lists, well over the 50% target. Communities were mobilized in all cases to establish O&M committees and 63% of these had mobilized funds. However, an effective conflict resolution processes did not emerge until the final year of the project. Performance against the third objective (Strengthening the capacity of local institutions to support participatory decision-making) was moderately satisfactory. Enhanced funds for capacity building of local institutions were provided at Additional Financing. 181 training events were provided for over 310,000 provincial, district and kum ban facilitators. Progress was sufficient for the project to deliver the sub-projects intended to reduce poverty, but increased capacity was largely limited to the PRF project units. Targets were not set for this objective. District officials participated in key meetings as expected. 55 training events were conducted on the principles and approaches of PRF implementation (including community participation) for about 19,293 district and 1,607 provincial officials. The moderate shortcomings in achieving this objective may, however, affect the sustainability and replicability of the participatory approach. The technical support of local officials, not just PRF staff at the district level, is vital for post-project sustainability of the sub-projects built under the PRF. While capacity building is hard to measure, further consideration could have been given to the indicators of this part of the PDO. The QALP-2 Assessment of the likelihood of achieving the PDO was Satisfactory/Likely. 3.3 Efficiency An economic return was not estimated at appraisal. Analysis conducted for this completion report to estimate the benefits of infrastructure investments by drawing on similar operations, suggests average returns around 21%. A 2011 comparative assessment of the unit costs of PRF and Government financed infrastructure illustrates that, with some variation between the north and south of Lao PDR, costs were between 35% and 88% lower in PRF financed projects due to in-kind community contributions, community-based procurement, knowledge of local costs and a unit-costs database. Administrative costs were initially high (60% and 35% in the first and second year respectively) but dropped to 8-10% in subsequent years. Overall this resulted in a 15% average operational cost ratio, lower than the 25% ceiling established at appraisal. The rolling nature of the project saw administrative costs increase each time new provinces were added to expand coverage. If training and facilitation costs from Component 2 are included as overheads for capital investments (i.e. empowerment/participation is not considered an outcome of PRF but a cost of delivering infrastructure), then the administrative cost overall is estimated at 27%. Notwithstanding the benefits of community participation in the construction phase of village development a technical quality trade-off was noticeable, as semi-skilled labor and lesser quality materials were used, at times reducing quality, beneficiary satisfaction and utilization rates. This 17 risk was not identified at appraisal. Over the range of infrastructure types, an acceptable level of quality was achieved in 100% of spring fed gravity water supply systems, but only 50% of weirs. Utilization ranged from 99% roads to only 78% for weirs constructed with project funds. See Table 3.3 Annex 3. The non-cash benefits accruing to PRF sub-projects include the time saved in collecting water; the time and cost savings of more immediate access to transport produce and/or non-forest timber products to markets. The combinations of new village infrastructure and facilities such as roads that linked with schools and markets, meant that beneficiaries increased productivity and prioritized education for children. Other savings of better infrastructure included the normal annual damage due to flooding. In sub-projects carried out under a community force account, community members received income for work carried out, and contractors also employed local labor, thereby generating employment for short periods in remote villages. Beneficiary satisfaction surveys indicated a village satisfaction rate of 81% (for the sample of 82 sub-projects assessed in eight districts in four provinces). The average satisfaction rate varies marginally from 79% in Xieng Khouang province to 81% in Savannakhet province but with marked differences across the most popular sectors: electrification (95%), schools (83.5%-85%), road access (82.3%) and bridges (81%). The benefits of the project also derive from the local capacity building activities which aimed to support improved community engagement and empowerment but are impossible to measure. Beneficiary feedback on the operational cycle and participation opportunities, indicate very high levels of satisfaction with the approach and the sense of ownership of assets (96%). Beneficiaries identified that the knowledge acquired through the capacity building and implementation processes came at individual, household and community levels, in particular an improved social capital in villages was reported in the end of project beneficiary study. This study also indicated high levels of O&M participation for schools, health dispensaries, water supply systems, community / market halls, weirs; 100% of communities established committees and 63% collected maintenance funds and practiced maintenance. 3.4 Justification of Overall Outcome Rating Rating: Satisfactory The overall outcome rating of satisfactory reflects the strong alignment of the project with government and Bank strategies and its continued relevance. PRF achieved very sound results in delivering local services and infrastructure to poor and very poor villages in remote mountainous regions of the Lao PDR, meeting the targets for improving services in priority poor areas. The project had some success in enhancing community capacity and empowerment, though less success in strengthening local government institutions. In delivering these results, the project was relatively efficient – unit costs were low and economies of scale and duration kicked in after the first few years, though the lack of accurate data makes comment on efficiencies quite general. 18 3.5 Overarching Themes, Other Outcomes and Impacts A. Poverty Impacts, Gender Aspects and Social Development Baseline data was not available at the beginning of the project, which has limited the ability of the project to gauge the impacts of its interventions. Nevertheless, studies undertaken at milestones throughout the project have highlighted the achievements and limitations of PRF in respect to poverty impacts, gender and social development. Poverty Impacts. While it is possible to document the physical accomplishments of the PRF, and to expand on some of these to known outcomes, a number of parameters of the project were critical: grants were provided for sub-projects in priority poor districts of the country; the focus of grants met community needs (91%); and the eligible list of sub-projects and the sub-projects selected focused on basic services: education, health, water supply and roads. The project did not intend to cover all aspects of poverty reduction. By design, it predominately focused on improving access to infrastructure services and facilities for poor rural communities. In terms of process, the project initially struggled with measurement and definitions of poverty. Data compiled by the National Statistics Center was found to be out-of-date and unreliable impeding PRF poverty targeting up to cycle V. Qualitative assessments suggest positive impacts as projects were selected to meet priority needs. A qualitative assessment carried out in 2008 by a Bank team noted that stakeholders reported positive impacts of improved infrastructure and facilities on communities in target villages – due to improved road access, water supply, schools and health facilities though some regional variation was noted. The final Technical, Utilization and Beneficiary Satisfaction Assessment (2011) noted that water supply was considered the most relevant sub-project followed by primary school, health center construction and weir construction. Gender Aspects. Women’s voice and participation in Lao PDR continues to be limited by traditional customs, particularly in minority ethnic groups in remote rural areas. The project addressed gender concerns directly and indirectly. Given the increasing focus on non-Lao speaking districts, and the poverty and gender marginalization in these areas, the poverty targeting implicitly brought with it gender targeting. The project also aimed to make women full participants in the planning, decision and implementation process of the PRF project; including gender targets for PRF staff as district and khet facilitators, and khet representatives, gender- aware processes to enable effective participation of women and the representation of their needs. While the project made some progress in this area, the disempowerment of rural women in Lao PDR, particularly those in remote areas, remains a constraint. Nevertheless, on those indicators measured (i) attendance of women in PRF District Decision meetings – the target of 50% of the community representatives was reached (31.6% of the participants in the meeting); and (ii) the proposals coming from women’s lists�, the target of 50% was well surpassed at 91%. Gender disaggregated data on district and kum ban facilitators and representatives is encouraging. Counting attendance however often belies the quality of the participation taking place. Qualitative assessments suggest that while women participated in meetings, and the needs/views of women are reflected in the sub-project investments, their role in decision-making may be 19 minimal in practice and empowerment of women does not appear to have been extended beyond the boundaries of the project. Social Development. Poverty, gender and ethnic minority groups are closely linked in Lao PDR. The concerns of ethnic minorities were met by the project design through the demand-driven and participatory processes that promoted inclusive local level decision-making. Direct measures to involve ethnic monitories were difficult in practice, especially in the startup phase: the staff hiring policy was thwarted by a lack of skilled kum ban facilitators, training of staff on ethnic minority issues was not really launched and ethnic minorities were not always on an equal playing field in the first years of the project. The M&E system was weak, including M&E of ethnic minority data. However, over 33% of the sub projects reached ethnic minority beneficiaries, often in extremely remote, previously inaccessible, villages. As noted elsewhere, community participation was central to the project approach, reflected in both the objective and the mechanism for planning and implementation. This approach exposed beneficiaries and local government officials to a bottom up approach – in some cases for the first time in Lao PDR. Procedures were developed and supported through district level facilitators and the engagement of community representatives (at kum ban level) who took primary responsibility for the management of sub-projects and “grass roots� financial management. While capacity limitations were apparent and lessons were learned, the process was successfully rolled out in an increasing number of provinces, with some significant regional adaptation. PRF established, across seven provinces, a community-based model which showed all stakeholders (beneficiaries, PRF staff and district line ministries officials) the benefits of community engagement in local development decision-making. The PSIA survey (Xiengkhor district, 2007) highlighted the importance of the PRF project as a first attempt to work nationwide on a truly participatory basis, focus on the country’s priority districts for poverty reduction, mobilize a high level of interest, awareness, commitment and ownership among villagers, and demonstrate transparency and accountability. The assessment raised questions as to whether the PRF was reaching the poorest areas within the poorer districts at the time. These findings contributed to the refined poverty targeting adopted in 2009. B. Institutional Change/Strengthening While the decision to establish a separate, multi-sector PIU clearly facilitated effective delivery of grants, the question of whether institutions have been strengthened as a result remains open to debate. Three key institutional outcomes are apparent at the various levels over which PRF has had influence: (i) the PRF has resulted in a change in government’s policy and platform for tackling poverty; (ii) the project has reached 21 local level institutions, bringing about a change in the options available to them for implementing government projects and establishing a model for multi-sector, needs-based approaches for replication; and (iii) state and non-state stakeholders have learned the importance of communities as agents of development change. These three areas of institutional change are related to the poverty reduction objectives, not capacity building objectives. In order to broaden the institutional change process, the second phase of PRF II must now ensure that these project achievements are internalized in country systems: broadening line ministry experience and knowledge so as to better understand and contribute their offices to reaching poor communities; achieving greater integration of PRF at the district level; and 20 reaching out to new districts to link into initiatives that have focused efforts on the development of local governance. It is perhaps in this last area of local governance where institutional change will be most needed if decentralization policies are brought into force. C. Other Unintended Outcomes and Impacts (positive and negative): Perhaps the most significant positive unintended outcome was the use of the PRF as a disaster response instrument. Following cyclone Ketsana in September 2009, the PRF was called upon to support the rehabilitation of damaged infrastructure in a number of affected villages, significantly expanding project activities through the Lao Upland Food Security Investment Project (LUFSIP) and a grant from the Global Facility for Disaster Risk Reduction supported by AusAID. In addition, the World Bank, the SDC and the European Union (also through LUFSIP) have increasingly drawn upon the PRF as a vehicle to support rural service delivery efforts. 4. Assessment of Risk to Development Outcome Rating: Moderate At the time of evaluation, the risk that development outcomes will not be maintained is moderate. The decision to continue the PRF model under a PRF II project that engages both development partners and government in co-financing, establishes a robust platform for ownership, commitment and eventual sustainability. Financial flows are predictable and financial management is sound, though there is a need to address procurement challenges. The project will need to intensify its efforts towards handover and cross fertilization at local levels. Linked to this, and a continuing technical risk, is the quality of some of the infrastructure constructed under PRF – with both sustainability and ownership implications. While targeted communities have engaged in participatory processes and some local capacity has been built, community empowerment, which is essential for lasting engagement, and overall governance, particularly accountability and transparency, require further strengthening. The political situation is unlikely to change, and political capture of investments is likely to decrease given the revised targeting and nature of PRF investments. In addition, the project has integrated several measures to enable PRF II to mitigate against disaster risks. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance A. Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory Overall, the Bank adequately identified, facilitated preparation of, and appraised PRF in order to achieve the PDO. Technical, financial and economic issues were duly considered. The overall design was strategically relevant and remained so until completion; it drew on Bank experience elsewhere but was nevertheless sensitive to the operating context in Lao PDR. In 2003, the project was innovative, seeking to inculcate bottom up planning in a top-down centralized environment. Key aspects of the project design included the flexibility in the types of sub- projects eligible for finance (allowing the PRF to work across a large number of sectors) as well as the pyramidal extension structures of a semi-autonomous agency put in place to reach the 21 poorest villages and minority ethnic groups in remote areas. The achievements in these areas should be seen in the context of low capacity local staff located in remote districts, with difficult access from provincial centers. The PDO, however, was not well articulated (in three unequal objectives), and the design was weak in key details. In particular, there was insufficient technical definition in participatory planning, implementation and M&E. Similarly, the strategy for engaging and building capacity of local level and line ministry stakeholders was inadequately defined and budgeted (although an effort was made to address this at additional financing). Risks in both these areas were underestimated. The results framework did not provide an effective basis for measurement and monitoring of some outcomes; targets were not set for many indicators, and the reliance on government data for initial poverty assessments affected poverty targeting. More thorough identification of risks at appraisal would have resulted in better identification of problems in implementation and more prompt responses, i.e., a practical and operational risk framework could have been used as a management tool. The QALP-2 assessment completed in 2010 concluded that the quality of design was moderately satisfactory (3), but that the strategic relevance and approach was satisfactory (2), while the realism of project design and risks was rated as moderately satisfactory (3). The moderate shortcomings/weaknesses identified included the arrangements for M&E, institutional analysis, the quality of the risk assessment, the ownership of line ministries as well as the effectiveness of Bank management of preparation. B. Quality of Supervision Rating: Moderately Satisfactory Task team commitment to the project has been high throughout and has been commended by all Government and Bank stakeholders. Task team leadership changed four times, with differing styles and priorities but without significant problem. From the documentation, supervision in the early years appears, however, to have been light. The frequency, intensity and technical implementation support of missions -- less than twice a year from a headquarters-based team -- was insufficient for a project adopting a new untested approach and implementing hundreds of sub-projects every year. Supervision missions did not consistently include the technical (engineering) expertise needed for a project spending over 90% on construction of infrastructure and facilities, and environmental safeguards expertise was not deployed consistently. The light approach to reporting earlier in the project is also illustrated by the MTR (December 2005) mission documents. The project experienced significant challenges in the early years, and would have benefited from more candid and rigorous feedback and follow-up in the early years. Given that sub-projects were completed on an annual cycle it is not clear why the project’s greatest challenge in these first years (poverty targeting) was not addressed earlier. The initial light approach to supervision changed with an evaluation at the end of cycle V (2007) and a Bank-led qualitative assessment, as well as efforts to improve monitoring. At Additional Financing (2008), the team introduced a number of modifications to address weaknesses identified, in particular improving pro-poor project targeting, focusing the project on delivery of public goods, and enhancing capacity building in government (although the latter was not implemented effectively). In late 2009, task management of the PRF was transferred from the 22 EAP Human Development Department to the Social Development Unit of the EAP Sustainable Development Department. Following this change, country office engagement improved, more operational and technical expertise was brought to bear. Fiduciary supervision, both procurement and financial management, seems to have received less attention than would be expected in a project with so many annual transactions. Post reviews were conducted regularly, but there is scant evidence of follow up. The Implementation Status and Results Report (ISR) ratings suggested that implementation progress was satisfactory for the period 2002-2008, and that it declined in the latter stage (2009- 2011). The satisfactory ratings for implementation in the first stages of the projected are more likely to have reflected an observation that the project was managing to implement a complex project utilizing a new approach; and the later moderately satisfactory rating is indicative of the fact that some basic problems with the quality of implementation and M&E had been identified. In hindsight, with the exception of M&E, these ratings do not reflect the trajectory of implementation performance, which seemed to improve over the eight years of the project as the project staff became more familiar with the project objectives principles and fiduciary/safeguard systems, and the pro-poor targets were met. A rating of moderately unsatisfactory for quality of Bank supervision in the QALP-2 triggered a further intensification of project supervision efforts. The new task task team refocused on fiduciary and environmental safeguards, and further improved technical supervision. The Bank’s team performed satisfactorily in terms of ensuring adequate transitional arrangements to PRF II. A follow-on operation was established seamlessly, reflecting the lessons of PRF I, and with only a minimal gap between closure the PRF project and effectiveness of the PRF II project. C. Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory Overall, Bank performance was moderately satisfactory: the project design was relevant to the GoL development strategy, appropriate to the institutional context and complementary to the Bank’s country portfolio, and remains so as PRF II proceeds. Some design details missing at the outset were developed in the course of project implementation, Commitment to supporting the implementation of an innovative project in a difficult context was high. Earlier periods of weaker supervision and fairly low management attention were eventually addressed. In the later stages, supervision was tighter, with improved country-based support. The transition to a new operation was seamless. Overall, strong task team commitment was noted throughout and supervision quality improved over the life of the project, though weaknesses in fiduciary and safeguards supervision were noted. 5.2 Borrower Performance A. Government Performance Rating: Satisfactory The GoL located the PRF in the Office of the Prime Minister under the political leadership of the Deputy Prime Minister, securing the political support necessary to achieve its objectives. An intergovernmental committee – the National Leading Committee for Rural Development and Poverty Eradication (NLCRDPE) – was established to provide strategic guidance, high-level 23 coordination and oversight. This executive arrangement has protected the project, to the degree possible, from the inevitable pressures exerted on a project cutting new ground and operating across multiple sectors at the grass roots. The Ministry of Finance was generally supportive and played an enabling role by providing the Fund with the space to operate and, to some extent, agreeing to more flexible procedures to reduce administrative procedures. Replenishments were timely. Various concerns arose over the eight years of implementation when the Leading Committee did not seize its potential role in fostering inter-ministerial cooperation, but the project nevertheless functioned effectively in delivering infrastructure services to poor communities. Securing the cooperation of line ministries is difficult in such contexts – efforts to draw in line agencies improved over the duration of the project, with increasing cooperation from the Ministry of Education for instance, but it continues to be an issue to be addressed in PRF II. The Governor’s offices of the provinces where PRF operated, and crucially the District offices, were also collaborative for the most part. Overall, government commitment is illustrated not only by the length of the project but by Government’s decision to commit US$10 million to PRF II from the national budget, and its efforts to ensure a smooth transition to and continuity of the successor project. A demonstrated track record encouraged the SDC to contribute co-financing during the Additional Financing phase of the project. B. Implementing Agency Performance Rating: Moderately Satisfactory The PRF implementing agency, operating under the leadership and management of an influential executive director, was steadfastly committed to, and largely successful in meeting the service delivery development objective of the project. The pyramidal structure of national, provincial and district level units was varied in capacity but these institutional arrangements delivered results on the ground, ensured high level commitment and attracted co-financing. The unit was less effective in ensuring capacity-building at district and community levels.  National Management Unit. After problems with turnover in the early stages of the project, the national level PIU reached a steady status. It was well staffed with committed, skilled professionals leading most sub-units, supporting the sub-national level as necessary to implement the sub-projects across the annual cycle and disbursing funds effectively. The technical quality of sub-projects has been adequate, with incremental improvements over the course of project implementation. FM has proceeded without cause for concern; record keeping is satisfactory. Procurement issues have arisen but have not adversely affected project outcomes or caused any concern for overall project governance. An effective complaints handling and feedback system was established only in the final year of the project. Monitoring was initially weak but improved over time. Capacity for evaluation remained weak throughout, which hampered the ability of the PIU to adapt to lessons learned in the course of project implementation. The PIU facilitated the smooth transition of the project to PRF II, and effectively assumed significant additional disaster response activities from 2009 onwards.  Provincial and district capacity. Provincial level units suffered from capacity constraints particularly in their understanding of the project objectives (with variation between 24 provinces). The capacity of District level teams seems to have limited the development of the participatory approaches, although staff are generally committed, procedures are followed, and structures are robust. Sustained training of provincial and district-level staff was not accorded sufficient resources and attention. Some of the challenges of the project seem to have occurred at this level, especially in relation to project targeting, with some gaps in knowledge as to the pro-poor objectives of the project or otherwise some problems with local incentives and pressures. Linkages with line ministries appear to function adequately at lower levels. C. Justification of Rating for Overall Borrower Performance: Rating: Moderately Satisfactory The borrower’s performance has been sound overall, though a number of weaknesses have affected the quality of project implementation. These include the variable understanding and absorption of the principles of participation, limited coordination with some sector ministries, inadequate investment in staff capacity-building, insufficient attention to complaints handling and the moderately unsatisfactory M&E. 6. Lessons Learned Poverty targeting requires clearly defined frameworks and monitoring. The initial problems and the time taken to identify and resolve the poverty targeting approach suggest that future operations would benefit from clearer design targets, stronger indicators, improved assessment and closer monitoring. In particular, the original design would have benefited from a framework of parameters, accurate poverty assessment, clear and measurable indicators, and a process for intensive monitoring and supervision to verify results in the first years. These process-related actions should be linked to efforts to fully familiarize project staff with the poverty dimensions of the project objective. Aligning objectives with institutional arrangements. The decision to work in a parallel structure which controlled the quality of staffing and ensured that systems and structures were developed to deliver results on the ground possibly did so at the expense of institutionalization of the processes and capacities in sub-national and line ministry structures. The lesson from the Lao experience is that this trade off can be justified given the specific poverty reduction objectives, the intention to reach villages in remote mountainous regions and the need to “protect� the participatory and multi-sector demand-driven approach – all of which would likely have been compromised if the project had been delivered through centralized, top-down line departments without the experience, capacity for outreach or commitment to innovate. Aligning ambitious poverty reduction objectives with reliable implementation arrangements was justified and remains valid while similar conditions prevail. The vital institutional element was the positioning of the Fund in the Prime Minister’s Office where it benefited from high-level championship and received a level of leadership that would not have been possible if dispersed across line ministries. Coordination with projects supporting system development. As the project became established and began to prove its effectiveness, there could have been more efforts to develop synergies and cross-fertilization with those projects that were promoting organizational change 25 and that focused on strengthening local governance systems. As PRF II moves forward, it would be of mutual benefit for the PRF to forge closer links with projects that focus on the development of local governance. PRF continues to deliver results at the grass roots in a participatory manner and to build the capacity and ownership of district level officials in village level investments. The efforts of projects implementing community and local development activities should be learning platforms for the development of relevant capacity and governance in local level structures, albeit in the context of only partial efforts toward decentralization. Line ministry coordination and collaboration. Multi-sectoral needs-based approaches and decentralized service delivery, present dual challenges to line ministries worldwide. The lesson of Lao PDR is that, if coordination and collaboration are fostered with line ministry officials at the local level, some of these barriers can be overcome. While this lesson may be most relevant to remote local government agencies, developing “collaborative space� in local government is achievable. Efforts can and should take place at different levels. Community participation. While the project made every effort to implement a demand-driven approach in a top-down centralized planning context by ensuring local community engagement in the planning and decision-making process, the processes established did not always ensure that the voice of the poorest in villages was carried through the kum ban to the final decision making arena (the district) and did not always result in a more empowered community, more able to voice needs. Improvements in the knowledge and facilitation skills of the PRF district staff and community facilitators, and the awareness of the capacity of the communities themselves, are vital. A number of design attributes affected qualitative engagement of communities. These include: (i) measures of participation should be qualitative as well as quantitative and should be assessed regularly; (ii) longer planning cycles ensure commitment and durability of the process, (iii) the importance of the project to community interface, and the need for sustained investment in the development of capacity at the community and local government interfaces. Project governance. The project design and implementation preceded the recent review of CDD projects in the Bank which raised a number of issues concerning the governance of these projects. Notwithstanding the implicit focus on community participation in planning, implementation and financial management, as well as the Bank’s fiduciary controls, other governance-related activities in PRF I were limited. As the project continues to expand, it will be vital that efforts are made to coordinate a broader set of activities that focus on strengthening project governance, including: (i) a focus on project transparency and access to information at both the national and community levels; and (ii) attention to conflict of interest issues. The “accountability meeting� which promotes reporting by the kum ban representatives and is a forum for transparency and social audit, could form a key element of an expanded local transparency and accountability framework. Enhanced evaluation. More rigorous qualitative analysis and experimental fieldwork, especially in relation to voice and engagement, and capacity building should be conducted as the project proceeds, with far increased budgets and efforts to appoint skilled consultants that have a track record elsewhere. As the project moves forward, the evaluation capacity in the PIU should be strengthened. Bank execution of some of key evaluations should be considered. Funds should be allocated for more comprehensive and reliable reporting, and international experience engaged to 26 improve quality, build capacity, and enhance utilization of data to inform timely learning and adaptive management. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies: None. (b) Cofinanciers: None. (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society): Not applicable. 27 Annex 1. Project Costs and Financing (a) Project Cost by Component (in US$ million equivalent) Components Appraisal AF Cofinancing Total Actual Percentage Estimate Estimate SDC Cost 1 2 3 4 5 6 (1+2+3) (US$ mill) (US$ mill) (US$ mill) (US$ mill) (US$ mill) 1. Sub-project Grants 14.98 8.37 7.27 30.62 30.96 101% 2. Local Capacity Building 2.17 2.93 - 5.10 108% 5.50 3. National Project Management 2.19 3.69 - 5.88 7.05 120% Total Baseline Cost 19.34 15.00 7.27 41.60 43.51 105% Physical Contingencies 0.00 Price Contingencies 0.00 Total Project Costs 19.34 Project Preparation Facility 0.00 - - - Total Financing 19.34 15.00 7.27 41.60 (b) Financing Appraisal Percentage Actual Source of Funds Type of Financing Estimate of (US$ million) (US$ million) Appraisal Borrower Counterpart Fund 2.37 2.37 100% IDA Loan 19.34 19.34 100% Total PRF I 21.71 21.71 100% IDA (AF) Grant 15.00 15.00 100% SDC (AF) Grant 4.80 7.27 151% Total PRF AF 19.80 22.27 112% Overall total 41.51 43.98 106% Explanation: The credit (IDA-36750) and the AF Grant (IDA-H4180) were approved in SDR. Figures at appraisal (above) are based on the exchange rate at approval: SDR =US$1.37 (2003) for the PRFI; and SDR=US$1.59 (2008) for PRF-AF. Actual disbursements applied the rate of SDR=US$1.59. 28 Annex 2. Outputs by Component Component 1: Village Sub project Grants (US$30.62 total; US$23.35 million IDA; US$7.27 SDC) a. Overview of Sub-projects Implemented Sector focus. Component 1 financed 3,179 sub-projects totaling $30.96 million in 28 poor districts in 7 provinces and included 616 schools, 86 bridges, 65 health dispensaries, 669 clean water systems and 156 irrigation schemes; over 3,000 km of rural roads were upgraded. By the closing date, 56% of the budgets supported social services (34% education, 22% health while 33% was spent on roads, bridges, and 4% on irrigation). The focus was almost entirely (93%) on construction with training accounting for 7% and costs for non-construction, non-training expenditure representing only 2.9% of the budget. The variation over the life of the project is shown in Table 2.2. Table 2.1: Sub-projects and expenditure by sector No. sub-projects* % by sector % by budget Education 842 26 34 Public Works and Transport 594 19 33 Health (including water systems) 818 26 22 Agricultural infrastructure 162 5 4 Capacity building /ITE 763 24 7 Total 3,179 Source: M&E Unit PRF* *A total of 3,373 sub-projects were completed including LUFSIP Table 2.2: Proportion of project investment by type (%) Cycle Cycle Cycle Cycle Cycle Cycle Cycle Cycle Total I II III IV V VI VII VIII % Construction 95.9 90.1 91.4 89.5 92 91 89 90.7 91 Training 8 4.6 5.4 9.1 5.4 6.1 6 4.2 5.8 Other 3 4.3 3.1 1.2 2.5 2.1 4.3 3.2 3.2 Source: M&E Unit PRF Provincial coverage. Sub-projects were implemented in 28 districts located in 7 provinces. The phasing over the life of the project (see Table 2.3) has resulted in some variation in the levels of support. The number of sub-projects implemented by province varied from 1,225 in Huaphanh (cycle I with five first priority districts) to 83 in Sekong (cycle VIII). Table 2.3: Provincial coverage Province Population Investment Investment Project Project Kip (mill) per capita Beneficiary Investment Kip (mill) % % Huaphanh (cycle I) 179,327 100,530 560,601 26.8 34 Savannakhet 159,587 66,345 415,732 23.8 23 Champasack 100,894 32,695 324,057 15.1 11 Xiengkhuang (cycle III) 82,614 40,419 489,259 12.3 14 Saravanh 39,131 28,241 721,726 5.8 9 Luangnamtha (cycle IV) 73,384 17,477 238,160 11.0 6 Sekong (cycle VIII) 34,623 6,097 176,107 5.2 3 Total 669,560 291,808 29 Source: M&E Unit PRF Beneficiaries. The 3,179 projects reached about 1,500 villages. Although it is difficult to estimate the population that benefited from these projects directly and indirectly, the total population of villages influenced by the project reached 650,000 in cycle VI. Direct beneficiaries (defined as villagers where the sub-project was implemented/built) started with 150,280 in cycle I, and peaked in cycle IV with 413,905. Of these, the average percentage of ethnic minority beneficiaries was 35.6% and direct ethnic minority beneficiaries constituted 15.9%. Table 2.4: Geographical Coverage Cycle Cycle Cycle Cycle Cycle Cycle Cycle Cycle I II III IV V VI VII VIII Provinces 3 3 5 5 5 6 7 7 District 10 14 20 21 21 19 21 21 Khet /Kum ban 121 188 239 252 161 195 214 213 Source: M&E Unit PRF Table 2.5: Project Beneficiaries Cycle Cycle Cycle Cycle Cycle Cycle I II III Cycle IV V Cycle VI VII VIII All beneficiaries (population) 213,998 428,504 479,670 652,098 530,340 316,227 315,996 360,895 Direct beneficiaries population 150,280 271,046 249,198 413,905 351,605 175,010 183,996 205,549 % 70.2 63.3 52.0 63.5 66.3 55.3 58.2 57.0 Small ethnic groups population 41,705 121,008 158,588 301,154 174,423 136,501 133,965 151,164 % 19.5 28.2 33.1 46.2 32.9 43.2 42.4 41.9 Direct beneficiaries population 21,326 59,806 70,229 141,940 72,951 48,420 61,038 809,64 % 10.0 14.0 14.7 21.8 13.8 15.3 19.3 22.4 Source: M&E Unit PRF Poverty targeting. With regards to the poverty focus of PRF activity, the poverty targeting of the sub- projects increased over the duration of the project – from 32% in the poorest categories (PG0-2) of villages in the first 5 years to 70% in the final year (see table 2.4c). In cycle I-V, the % of sub-projects and/or budgets spent in category PG0-2 villages was 53%, by cycle VIII this had increased to 70%. Table 2.6: Poverty Targeting of Sub Projects (PG0: poorest villages-PG5: Better off villages) Poverty Cycle I – V Cycle VI Cycle VII Cycle VIII Grading No. No. No. No. % % % % villages villages villages villages PG0 394 32 251 17 259 18 159 11 PG1 90 7 348 24 407 29 403 29 PG2 161 13 404 28 406 28 418 30 PG 0-2 645 53 1,003 69 1,072 75 980 70 PG3 259 21 314 22 283 20 342 24 PG4 319 26 140 10 70 5 87 6 PG 3-4 578 47 454 31 353 25 429 30 Total 1,223 1,457 1,425 1,409 Source: M&E Unit PRF 30 Table 2.7: Infrastructure Sub-projects Implemented in Villages Based on Poverty Grading Percentages – PG0 (poorest village) - PG5 (Better off village) PG0 PG1 PG2 PG0-2 PG3 PG4 PG5 PG 3-5 Cycle I 2 7 21 30% 20 21 29 70% Cycle II 4 12 23 38% 21 21 20 62% Cycle III 5 19 22 46% 19 19 17 54% Cycle IV 5 19 19 44% 18 21 16 56% Cycle V 5 25 18 49% 22 20 10 51% Cycle VI 5 17 20 42% 41 17 0 58% Cycle VII 22 34 28 84% 14 2 0 16% Cycle VIII 10 33 35 78% 18 4 0 22% Source: M&E Unit PRF b. Quality of infrastructure Quality of constructed infrastructure has been the subject of some concern especially in the latter years of the project when monitoring improved and sustainability was questioned more. Utilization data collected in 2010 shows average utilization of 96% overall. The lowest utilization occurs in drilled and dug wells (78%), irrigation (78%) and bridge construction (85%), with the highest rates for schools (99.7%). Roads are used irrespective of their condition however (and beneficiary satisfaction is a better indicator of this). See table 2.8. The project has had a significant number of problems with wells (nearly one quarter of the sample are not used), a number being damaged by flooding during the Ketsana cyclone in September 2009 (i.e. Savannakhet, Attapeu, Saravanh, and Sekong provinces). The lowest levels of utilization occurred in Saravanh province where 18% of the sub-projects built were not utilized, compared with only 1-2% in Xiengkhouang and Huaphanh, largely due to the effects of cyclone Ketsana. Table 2.8: Sub-project Utilization by type of sub-project (of sample studied) Project type No. projects Used % Unused % School renovation/construction; Community halls 306 305 99.7 1 0.3 Rural road upgrade 269 267 99 2 1 Bridge construction and renovation 34 29 85 5 15 Electricity system 9 9 100 0 0 Dispensary construction 26 24 92 2 8 Spring fed gravity and water supply 297 289 97 8 3 Drilled well, drug well 50 39 78 11 22 Irrigation, dam, weir 51 40 78 11 22 Community markets 9 9 100 0 0% Total 1,051 1,011 96% 40 4% Source: PRF Technical Utilization and Beneficiary satisfaction Assessment 2011 31 Table 2.9: Sub-project Utilization by province (of sample studied) No. Province Used % Unused % projects Huaphanh 479 470 98 9 2 Savannakhet 271 257 95 14 5 Champasack 79 76 96 3 4 Xiengkhouang 125 124 99 1 1 Saravanh 61 50 82 11 18 Luang Namtha 36 34 94 2 6 Total 1,051 1,011 96% 40 4% Source: PRF Technical Utilization and Beneficiary satisfaction Assessment 2011 Technical assessment. The final PRF technical assessment undertaken in 2010-11, indicated that, of a sample of 80 projects of various types, 50% of irrigation systems and wells were of unacceptable quality, and that 33% of roads were of poor quality. (of 12 rural roads assessed, 5 are not passable during the raining season). Quality for health, schools and water supply was found to be better, but around 20% of these sub-projects were considered to be of poor quality. Electricity systems, community halls and bridges were all good quality. While later projects are built to a higher standard, technical assessment of PRF roads noted a number of causes of poor quality construction:  a lack of standard designs for rural road construction  different sizes for culverts and bridges  incomplete surfacing and compaction  thickness of earth cover  the earth near road site. Improvements in technical quality have been made since 2007. In cycles I and II, district and provincial PRF offices were responsible for designing sub-projects. In cycle III, the PRF developed standard designs. In cycle IV, communities recruited supervisors, selected by PRF, to follow-up on technical aspects of infrastructure with background in civil engineering and provided training. Each community supervisor followed in average three sub-projects that he/she regularly inspected, to advise the kum ban team on implementation process and issues to address. PRF technical advisors certified the progress or the completion of the sub-projects into the field inspection forms and at the kum ban accountability meetings. From cycle VI to cycle VIII, the survey design process was conducted with the systematic participation of the local authorities and concerned line ministry officials at district level. Inspection is now staged and linked to payments. Nevertheless, PRF faced constraints in terms of quality control that are inherent in the CDD approach. A large proportion of the infrastructure is built by communities (on a community force account) resulting in occasional weaknesses in respect to PRF technical guidelines and standard designs. Moreover the PRF menu of options allows a large scope of projects and requires a high level of expertise in different fields. The decision to empower communities through the PRF process, the coverage and the remoteness all make it difficult to ensure a proper quality control. This difficulty has been addressed to some extent in PRF II design by adding a step in the PRF cycle to clarify and formalize responsibilities and assign 32 specific tasks to key stakeholders from District Government, private sector (contractors) and kum ban Authorities involved in the sub-project implementation and supervision. The PRF second phase will also mainstream disaster risk management practices and encourage spot improvement approaches for roads sub-projects. Operations and Maintenance. The project recognized the importance of line ministry ownership of PRF financed infrastructure and set a target of 80% of PRF village infrastructure being recognized by line ministries. By the end of the project 100% of PRF village infrastructure had been recorded in the official ministry books. The project also intended to promote and monitor the number of target beneficiaries with maintenance plans and prioritized the establishment of community operations and maintenance committees in each province. The utilization study noted that of the 1,051 sample, 95% had established committees, 3.4% were in progress and only 1.8% had no O&M plan. c. Community Contributions Community contributions to PRF sub-projects have been monitored throughout the project. At appraisal it was agreed that communities should contribute more than 10% of the cost of the sub-projects, in more than 70% of the villages. This target was consistently surpassed (annual average 17.5%). However a significant variation was seen between the original phase of the program (Cycle I-V) and the additional financing phase (Cycle VI-VIII) as the PIU sought to reduce community contributions that came in the form of wood (in order to reduce the potential impact of the sub-projects on deforestation). By sector, by far the greatest contributions were made for roads and irrigation. Contributions were provided by communities in kind, especially in the form of labor, or by providing transport for construction materials. Many communities also voluntarily contributed small amounts of land to widen roads, construct irrigation canals etc. Table 2.10: Overall Community Contributions (% of PRF budget) Cycle Cycle Cycle Cycle Cycle Cycle Cycle Cycle I II III IV V VI VII VIII % PRF budget 83 82 83 85 80 84 89 89 84 % community 17 18 17 15 20 16 11 11 15.5 contribution Source: M&E Unit PRF Table 2.11: Community Contributions by sector (% of PRF budget) Cycle Cycle Cycle Cycle Cycle Cycle Cycle Cycle Overall I II III IV V VI VII VIII Education 9.7 10.5 10.8 11.6 11.4 13.1 6.4 9 10.3 Public Works 31.0 23.9 21.5 16.1 27.2 21.4 15.3 13.8 21.2 Health 9.3 17.1 15.9 20.6 19.6 16.3 15.1 14.8 16 Agricultural 27.0 19.2 29.5 30.4 16.9 13.7 13.5 12.4 20.3 Infrastructure ITE 9.3 7.7 2.8 4.1 3.1 0.0 0.0 0.0 3.3 Source: M&E Unit PRF 33 d. Employment generation The project provided significant support to communities that were mobilized to carry out the works themselves. Of the overall expenditures on sub-projects from cycle I to Cycle VIII, 68% (2,168 projects) were carried out by communities under a community force account arrangement (valued at US$15,720,338), and 32% (1,011 were carried out by a contractor). The project does not have data on the number of labor days this amounts to or the number of days / estimated wages paid to villagers to provide labor to contractors. Component 2: Local Capacity Building (US$5.50 million IDA) Component 2 supported local capacity building in three primary areas: capacity building of the provincial technical support team, capacity building of district facilitators and capacity building of community level (khet / kum ban) facilitators. At additional financing, it was recognized that capacity development was lagging and additional funds of US$0.49 million were allocated to local capacity building (under a new component 2b). Indicators were clarified and component 2 was aligned with PDO parts (b) and (c). To empower villages in poor districts to manage their own public investment, planning and sub-project implementation in a decentralized and transparent manner, capacity building activities were undertaken in three primary areas:  manage public investment planning and implementation.  provide technical consulting advice, and  training to strengthen local capacities, and form a Provincial Technical Support Team, who will in turn provide technical engineering, and financial management assistance at the Poverty Reduction Fund ' s provincial level. A total of 681 community facilitators (227 female) were recruited as PRF staff to mobilize commune members, and to work with communities to ensure and enhance participation. This was conducted through 3 phases:  preparation (socialization) including village meetings; village needs and priority assessments, kum ban socialization meetings, district socialization meetings and provincial workshops.  planning including village meetings, kum ban decision meeting, district decision meetings for the prioritization of sub-projects, preparation of design and budgets, procurement methods, kum ban sub-project design and budget meeting, and district decision meeting to finalize funding commitments.  implementation including information meetings, mobilization of labor, training on construction, release funds, construction, certification of progress, cross kum ban assessment and monitoring, kum ban handover meeting, accountability meeting, operation and maintenance. a. Community capacity building and empowerment Capacity building. The purpose of the community capacity building was to ensure that the participatory processes envisaged for planning and implementation of sub-projects were put in place. Capacity building was achieved through training of kum ban facilitators, implementing team, procurement teams and kum ban representatives. 34 Table 2.12: Details of Community Training No. Budget allocated No. % budget trainings (US$) beneficiaries Animal Raising 203 336,762 18% 5,328 Crop Raising 175 191,591 10% 5,458 Natural Resources Management 236 490,614 26% 31,964 Scholarships 88 200,763 11% 317 Training of Village Volunteer 9 24,443 1% 88 Livelihood training 148 222,081 12% 4,500 Gender Awareness training 80 44,482 2% 16,747 Village Saving Group 101 250,623 13% 2,875 Birth Attendance 17 34,874 2% 309 Village Veterinary Training 54 80,793 4% 722 Literacy Training 1 3,506 0.2% 25 Other 1 4,918 0.3% 73 Total 1,113 1,885,450 100% 68,406 Source: M&E Unit PRF Outcomes. Outcomes were measured by the number of sub-project proposals submitted by communities and subsequently approved, reaching a high of 93% by the end of the project, well over the target of 50% set down at additional financing. Table 2.13: Sub-projects submitted and approved by communities (from community lists) Cycle Cycle Cycle Cycle Cycle Cycle Cycle Cycle Average I II III IV V VI VII VIII % % % % % % % % Community 85 71 67 62 76 87 86 93 77 Kum ban 9 20 9 5 3 12 12 6 10 District 9 9 24 32 21 2 1 1 13 Source: M&E Unit PRF Women’s’ participation. A target of 50% was also set for the number of sub-projects that met women’s needs. Sub-projects in the last three cycles, the period during which this data was measured, reached about 90% as indicated by the number of sub-projects selected from as priorities on women’s lists. The building block approach which allowed women to participate in separate groups than men at the village level was seen as being particularly effective, enabling more articulation of their household needs (education, water supply) and less for large infrastructure projects. However while the PRF mobilized women for project ends, very few have opted for a more public role in kum bans or districts after the cycle was completed. A field visit conducted for the ICR noted that women’s participation in joint village meetings was weak. 35 Table 2.14: Sub-projects reflecting women’s priorities (no. sub-projects selected from women’s and men’s lists) Target Cycle VI Cycle VII Cycle VIII % % % Women priorities 13 11 8 Women and men priorities 79 82 80 Sub-projects reflecting women’s priorities 50 92 93 88 Source: M&E Unit PRF Ethnic Minority participation. The PRF also intended to ensure its activities effectively reached ethnic minorities. Through its Ethnic Group Policy Framework (EGPF) PRF aimed to provide guidance for addressing ethnic group needs and aspirations in line with the World Bank’s safeguard policies. Survey (CBA) results suggest that close to 90 percent of ethnic groups saw that their needs were taken into account. This is supported by the efforts made by the project to ensure language was not a barrier to ethnic minority participation and benefit. Most facilitators could speak Non-Lao Tai languages. b. Capacity Building for Local Institutions An evaluation conducted in 2008 noted that not all officials recognize the benefits of the participatory approach, and that longer term engagement would be vital to build that commitment over time. The same study noted that the need to better coordinate CDD mechanisms in Lao PDR, overlapping and leaving gaps in coverage. At additional financing, PRF designed a new capacity building component aimed at contributing to the government’s objective to better harmonize and coordinate ministries and other GoL organizations involved in poverty reduction and to ensure the long term sustainability of PRF funded infrastructure. This sub-component intended to advocate the use of participatory planning to strengthen the capacity of line ministries to engage communities in the planning process. The objective was also to exchange with local institutions and to receive their feedback on the PRF work and approach in order to improve PRF processes, procedures and manuals. It also aims at enhancing the coordination between local institutions to contribute to efficient and sustainable management of local development. The capacity building study conducted in 2011 assessed the efficacy of this capacity building, the awareness and understanding of PRF approaches, and the levels of communication and coordination. Training provided. The training courses within the PRF Local Institution Capacity Building component generally included trainings to improve understanding of the principles and approaches of PRF, implementation, O&M and participation. Details of the trainings provided are set out in Table 2.12. Table 2.16: Details of Provincial, District and Kum ban Training Events (focused on facilitators understanding of participatory objectives) Budget No. held % budget No. beneficiaries Local Capacity Building (US$) at provincial level 23 38,254 9% 1,607 at district level 55 54,905 13% 19,293 at kum ban and village level 103 340,624 79% 289,408 Total 181 433,782 310,308 Source: M&E Unit PRF 36 Assessment of capacity building. The capacity building assessment generally found the training activities to be considered by participants as good in quality, with the training events on participation and on the principles of PRF particularly appreciated. Key findings of the study were:  Kum ban representatives and district officials show a better (‘good’) understanding of the PRF approach having acquired knowledge on (i) the benefits of participatory approaches in responding to the needs of villagers, and (ii) the need to create capacity of communities to enable local people to get actively involved in their own development.  In districts where PRF is operating a high number of district officials have been targeted for virtually all of the capacity building efforts. Many of the local government staff has participated in training on PRF principles, while others were recruited as trainers or resource persons for technical training on the implementation, operation and maintenance.  A high percentage of kum ban staff members have participated in some kind training event, with up to 100 percent participation in specific technical training for specified target groups.  Both groups indicated a high level of contentment (more than 90 percent) with training events in terms of their timely announcement and the preparation of trainers, though some trainees would have preferred to see more time for discussions and a better documentation of training materials.  Similar to the results from an assessment of training, interviewees, through a ranking exercise, indicated that other capacity building measures, such as workshops, meetings, field visits, written guidelines measures and IEC tools – with the notable exception of study tours - achieved their goals to a high degree. Feedback and conflict resolution: PRF put in place a Feedback and Conflict Resolution Mechanism (FCRM) to receive feedback and to address emerging issues. FCRM Committees were composed of PRF and government staff members operating on village, kum ban, district and provincial levels. In general, the FCRM is perceived by committee members and communities in positive terms as a mechanism that allows villagers to express their opinions confidentially, to ask for help to solve problems or conflicts. Nevertheless, the numbers of complaints were low – over 8 years, only 32 relevant complaints were received concerning in 32 sub-projects / 29 villages. An enhanced FCRM was piloted in two districts in 2011 in order to inform PRF II, and had made encouraging progress by the end of the project. Outcome of capacity building. An outcome indicator for capacity building identified at additional financing was the participation of local government officials in PRF District Decision Meetings. Table 2.14 illustrates the very solid attendance of Government officials (non PRF project staff) at the key meetings where sub-projects are selected. No per diems were paid for attendance in these meetings. Table 2.17: Attendance at PRF District Decision Meetings PRF % GoL Staff Communities Total Female Staff female Huaphan 36 125 116 277 79 28.5% Savannakhet 58 133 204 395 136 34.9% Champasack 28 32 44 104 32 31.9% Xiengkhuang 33 78 96 207 66 34.4% Saravanh 23 38 36 97 23 30.8% Luang Namtha 129 388 517 75 14 23.7% Sekong 28 69 76 173 45 26.0% Grand Total 258 622 852 1,732 548 31.6% Target 15% Source: M&E Unit PRF 37 Component 3: National Project Management (US$5.88 million IDA; US$6.67 million cost) The funds allocated to component 3 for national level project management were utilized for administrative, monitoring, reporting and operational costs of the PRF at the central level, including necessary technical assistance and training in specialized areas of expertise. An initial allocation of about US$2.19 million was increased to US$7.05 million at Additional Financing. US$178,500 operating costs were covered by GoL from the period 2003-2010. Overall, 290 consultants were hired on a competitive basis in the Finance, Community Development, Engineering and M&E units of the PRF office, including Directors and advisory positions. Table 2.18: Component 3 Expenditure* 2003 2004 2005 2006 2007 2008 2009 2010 2011 Total % Consultants 180,584 307,711 217,076 86,744 334,008 318,934 641,965 802,575 801,332 3,690,928 54% Salary Office equip. 25,201 3,321 4,812 9,135 395 14,680 190,182 53,825 5,890 307,441 4% Vehicles 714 - - - - 19,000 8,556 213,496 - 241,765 4% OC ** 45,889 61,196 72,290 65,699 77,807 180,308 345,822 207,633 623,713 1,680,357 24% Civil work 40,154 4,821 132 - - - 22,117 6,962 77,092 151,277 2% Training - 17,926 106,569 53,013 41,986 23,894 14,723 33,528 53,111 344,751 5% Audit fee 61,608 37,582 99,190 1% TA 69,669 130,900 200,569 3% M&E 71,788 72,500 144,288 2% Total 292,542 394,975 400,879 417,656 454,196 556,816 1,223,364 1,318,018 1,802,120 6,860,566 *IDA financing for PRF subsidized consultant and operating costs for the additional LUFSIP activities and for SDC resources, which were utilized to finance sub-projects only. **Not covered by Government Source: M&E Unit PRF Table 2.19: Component 3 Outputs Technical Assistance, M&E Unit Outputs planned Year Number Financial Annual Financial Audit 8 2003-2011 M&E Technical & Beneficiary assessment 1 2005-2006 Technical & Beneficiary assessment 1 2006-2007 Technical, Utilization & Beneficiary assessment 1 2011 Participatory assessment 1 2011 Capacity building assessment 1 2011 Internal outcomes assessment 1 2009 Internal sustainability assessment 1 2010 Baseline survey 1 2003 Final survey 1 2005 Annual progress report 9 2003-2011 Quarterly report 19 2003-2011 38 Semi-annual report 3 2003-2005 Communication PRF magazine 19 2003-2009 Other communication materials N/A 2003-2010 Bulletins 25 2003-2011 Website 1 2003-2010 TV & radio spots 9 2003; 2007-2010 Manuals 10 2003; 2008; 2010 Booklets 5 2003 Articles 500 2003-2011 Videos 6 2003; 2007-2010 Workshop and PRF Board meetings 17 2004-2010 meetings PRF annual workshops 8 2004-2010 Source: M&E Unit PRF 39 Annex 3. Economic and Financial Analysis Due to the needs based approach to sub-project selection, project appraisal documents did not provide an estimated EIRR. There is also no accurate information on poverty reduction in the districts where PRF is operating and where there are data, issues regarding the attribution of the cost-benefits make a calculation of an EIRR challenging. In other countries that have measured the benefits of CDD projects and social funds, favorable economic rates of return for individual sub-projects have been illustrated.5 PRF shares the features of these projects, namely (i) support from local community facilitators to mobilize communities and ensure technical quality; (ii) decentralized decision-making to more accurately match resources to needs/preferences; (iii) utilizing local contractors and labor. Utilizing the sector internal rate of return figures developed for the Vietnam Community-Based Rural Investment Project, but taking into account the various proportions of education, roads/bridges, agriculture infrastructure and health/ water supply that were constructed in PRF (see table 2.1) results in an EIRR calculation of 21%. The following analysis better illustrates the cost effectiveness, efficiency and the general cost benefits of the project through value for money investment in improving infrastructure and facilities, adopting needs- based participatory approaches and building capacity. Administrative costs. At appraisal, the GoL established a requirement that the ratio of administrative6 costs to total costs would not exceed a ceiling of 25%. While this was not regularly monitored, the overall ratio of operational costs appears to be in the region of 15% (based on the assumption that national project management costs are all administrative and sub-national costs all focused on capacity building). Unsurprisingly for a CDD project launched in 3 provinces, some in remote rural areas, the up-front costs were high, reaching 60% in the first year, and 35% in the second year as the project was established. The ratio declined to around 10% was sub-project implementation began in earnest and reached a steady ratio at about 8-10% until more provinces were introduced (adding establishment costs) and categorization changed (when local facilitation was included in the national operating costs). If training and facilitation costs from Component 2 are included as overheads for capital investments (i.e. empowerment/participation is not considered an outcome of PRF but a cost of delivering infrastructure), then the administrative cost overall is estimated at 27%. Table 3.1: Operating Costs (as a percentage of total costs) 2003 2004 2005 2006 2007 2008 2009 2010 2011 Total actual costs 0.49 1.68 4.04 5.45 5.35 5.37 6.36 7.63 7.10 Operating costs 0.29 0.58 0.40 0.42 0.45 0.55 1.22 1.31 1.8 Operating costs 60% 35% 10% 8% 8% 10% 19%* 17% 25%** (%) * Establishment costs in Sekong Province; Kum ban facilitators allowance re-allocated from component 1 to Operating costs. ** Sub grant budget declined in final year as project closed Source: M&E Unit PRF 5 EIRR in similar projects in region: Community Based Rural Investment Project, Vietnam 17.4-23.0%; Rural Investment and Local Governance Project, Cambodia: 18-25%; Note: no EIRRs were calculated able for the similar Cambodia and Thailand Social Funds. 6 Administrative Costs as per PAD / Disbursement Letter included the categories consultant service, goods, works, incremental operating costs and training in component 3 (see table 2.25). The GoL contribution to the operating costs covers by percentages in each category and the total amount of the GoL contribution USD 178,500 from the period 2003-2010. 40 Unit costs. Cost effectiveness indicators for the project were checked a number of times throughout the project. In order to contain the very high unit costs of construction that had been found in Lao PDR before appraisal, efforts were made to ensure cost effectiveness and acceptable/satisfactory economic returns by checking against average unit cost rates across various categories of infrastructure. Table 3.2 provides a comparative assessment of the unit costs of PRF and Government/other roads, irrigation systems, schools, water systems and dispensary/health clinics and illustrates that, across that range, and despite the differentiated cost structures in the north and south of Lao, costs were between 35% and 88% lower in PRF financed projects. Unit rates were kept down because of the in-kind support of communities and the oversight of rates by the national project management team. Table 3.2: Average Unit Cost for Selected Infrastructure 2009-2011 (Kip thousands) Unit PRF GOL/Other PRF GOL/Other NORTH SOUTH Public works Rural roads# km 30,000 70,000 42% 35,000 100,000 35% Weir* meters 10 15 66% 12 25 48% Irrigation Weir** meters 20 30 66% 23 35 66% Earth Canal meters 200 600 66% 35 130 27% Education School+ sq m 1,518 1,725 88% 1,607 1,964 82% Spring Fed Gravity meters 36 50 72% 36 50 72% Public health system Dispensary ++ one 240 350 68% 320 500 64% # significant variation due to topographic and hydrological conditions and availability of earth different conditions * concrete + Riprap) height < 2m high: ** concrete + Riprap) height > 2m high + 8x7m size including sanitation; ++ ADB design including medical equipment Source: M&E Unit PRF Notwithstanding the benefits of community participation in the construction phase of village development and the successes of PRF in this regard there is – typically in CDD projects – a technical quality trade-off as semi-skilled labor and lesser quality materials are used. PRF is no exception. Table 3.3 provides a snapshot of this trade off. It should be noted that technical quality however is not only due to the community role in the construction process; a commonly occurring problem is the non-availability of acceptable quality gravel and sand. 41 Table 3.3: Savings, quality, beneficiary satisfaction and utilization of infrastructure Quality Beneficiary Utilization rating satisfaction Savings* rates** [Acceptable] Public works Rural roads 58-65% 67% 82.3% 99% Irrigation Weirs 34-52% 50% 73.0% 78% Education Schools 12-18% 79% 83.5% 99% Spring Fed Gravity system 72% 100% 81.8% 97% Public health Health Dispensary 64-68% 71% 70.0% 92% Source: Technical Utilization and Beneficiary Satisfaction Assessment, 2011 * From table 2.2; ** PIU data Beneficiary Satisfaction. The Beneficiary Assessment (2011), carried out in conjunction with technical utilization, indicated a village satisfaction rate of 81% for the 82 sub-projects assessed in 8 districts in 4 provinces. The average satisfaction rate varies marginally from 79% in Xieng Khouang Province to 81% in Savannakhet Province but with marked differences across sectors. The highest satisfaction rates are in electrification (95%), schools (83.5%-85%) and in road access (82.3%) and bridges (81%). The lowest satisfaction rates are in community hall (60%) and market halls (65%). The latter were built early in the project and removed from the eligible list when efforts were made to improve poverty targeting and reflected a degree of local capture at district level that reduced cost benefit in some specific provinces in their first cycles. The breakdown of the satisfaction rates by provinces and type of sub-projects is shown in table 3.4. Table 3.4: Beneficiary satisfaction rates of Sub-projects Average Satisfaction Rate (%) Savannakhet Saravan Luangnamtha Xieng Total Khouang Primary School 82.5 82.5 85.0 84.0 83.5 Secondary School 0.0 0.0 0.0 85.0 85.0 Access Road 82.5 81.3 0.0 83.3 82.3 Bridge 90.0 90.0 75.0 75.0 81.0 Culvert 0.0 70.0 0.0 80.6 79.4 Dispensary 70.0 67.5 85.0 60.0 70.0 Deep Well 76.7 0.0 0.0 0.0 76.7 Spring Water System 77.5 81.3 84.0 0.0 81.8 Weir 71.7 0.0 75.0 0.0 73.0 Irrigation System 0.0 70.0 0.0 72.5 71.7 Electricity 95.0 95.0 0.0 0.0 95.0 Community Hall 0.0 0.0 60.0 0.0 60.0 Market Hall 0.0 0.0 0.0 65.0 65.0 Total 81.0 79.7 80.7 79.0 81.1 Source: PRF Technical Utilization and Beneficiary Satisfaction Assessment 2011 The non-cash benefits accruing to PRF sub-projects are similar to those in poverty reduction projects elsewhere. For improved water supply, beneficiaries noted the time savings in collecting water; for roads – the time and cost savings due to quicker, more immediate access including being able to transport produce/non-forest timber products to markets, and getting children to schools located in district centers; for irrigation – increases in incomes due to increased productivity and savings from damage due to 42 flooding. PRF is notable in its wide spread of sub-project types (e.g. not all the funds was spent on rural roads) and the level of achievement in remote areas. In cycle VII, 22%, and cycle VIII, 16%, of sub- projects were built in locations where there was no road access prior to the project. In those cases where communities elected to carry out the works themselves, PRF provided economic benefit not just in terms of the resulting infrastructure but because the income received by community members for work carried out, did not leave the village but could be utilized by village households. In those projects carried out by contractors, this benefit was less marked, but in many cases contractors still employed local community members to provide labor, thereby generating employment for short periods in sometimes remote villages. Empowerment benefits. The benefits of the project also derive from the local capacity building activities which aimed to support improved community engagement and empowerment. Beneficiary feedback on the operational cycle and participation opportunities, indicate very high levels of satisfaction with the approach and the sense of ownership of assets (96%). Beneficiaries identified that the knowledge acquired through the capacity building and implementation processes came at individual, household and community levels. Benefits in relation to the community, noted the impact on social capital in the communities – the sense of community, the shared understanding of needs and the new ability of communities to work together and use representatives on their behalf. At the household level, the beneficiaries highlighted their increased ability to act as a member of development activities, their increased trust and confidence and at the individual level they noted their increased knowledge in identifying and prioritizing needs. Feedback, however, also highlighted their continued lack of knowledge in negotiation. Measures of ownership and sustainability. The project identified at an early stage the need to put in place vehicles to ensure ownership for post-project sustainability to maintain benefits and the record of maintenance committees and funds are notably high. The technical utilization beneficiary survey indicated high levels of O&M participation for schools, health dispensaries, water supply systems, community / market halls, weirs (100% committees established) and 63% of sub-projects will have collected maintenance funds and practiced maintenance 3 years after completion. Other studies have noted the drop off in operation and maintenance due to damage past repair (post-Ketsana), the low standards of construction, the burden of maintenance on communities, particularly roads, and the severe lack of resources for maintenance. Nevertheless, the project displays high levels of ownership of assets by communities – inevitably linked to the high levels of community contribution, and the potential for regular O&M regimes when institutionalized mechanisms of support can be established under PRF II. 43 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Senior Social Development Operational Cycle/Training John Victor Bottini EASSO Specialist Strategy Operational Cycle/MIS/Units Steve Charles Burgess Consultant EASES Costs and Project Costing Soniya Carvalho Lead Evaluation Officer IEGPS Social Funds/Microenterprise Christopher Chamberlin Lead Operations Officer EASHD Task Team Leader Nitaya Chatnantawej Program Assistant EACTF Project Costing Tasanee Chokwatana Program Assistant EACTF Project Support Nina Masako Eejima Legal Counsel LEGES Legal Counsel Lead Social Development Scott Guggenheim EASSO Peer Reviewer Specialist Enurlaela Hasanah Research Analyst EASSD Implementation Plan Svend E. Jensby Consultant EASSO Safeguards/Social Assessment Lao Social Organizations / Keiko Miwa Education Specialist EASHD Gender Issues / Social Assessment Procurement Assessment and Othip Mongkolsawat Procurement Specialist EAPPR Procurement Plan Disbursements and Financial Rosa Muleta Disbursement Management Senior Financial Sector Olivier Lambert EASFS Micro finance development Specialist Senior Financial Management Behdad Nowroozi EAPFM Financial Management Specialist Peer Reviewer/Poverty Kaspar Richter Lead Econonist ECSP3 Targeting Implementation / Funds Flow Maryam Salim Senior Operations Officer EASHD / Operations Peer Reviewer / Monitoring Lynne Sherburne-Benz Country Program Coordinator EAC-EAP and Evaluation & Targeting Financial Management Financial Management Nipa Siribuddhamas EAPFM Specialist Assessment Nitipat Taemphairojana Team Assistant EACTF Project Support Supervision / ICR Chinnakorn Chantra Procurement Specialist EAPPR Procurement Sladjana Cosic Social Development Specialist EASER Social Accountability Minna Hahn Tong Consultant EASHD Project Support Sidet Kim Program Assistant EACTF Procurement Florian Kitt Knowledge Management EASER Communications 44 Officer Social Development Stephanie Anne Kuttner EASHD Gender, Project Support Consultant Jennica Larrison Consultant EASHD Project Support Ngozi Blessing Obi Malife Program Assistant EASER Project Support Oithip Mongkolsawat Senior Procurement Specialist EAPPR Procurement Senior Financial Management Donald Herrings Mphande AFTFM Financial Management Specialist Emiko Naka Consultant EASHD Implementation Support Natsuko Kiso Nozaki Consultant ECSHD Project Support Sybounheung Phandanouvong Social Development Specialist EASTS Social Safeguards Viengkeo Phetnavongxay Environment Specialist EASTS Environmental Safeguards Piriya Pholphirul Economist EASHD Monitoring and Evaluation Senior Social Development Janelle Plummer EASTS ICR Primary Author Specialist Helene Monika Carlsson Rex Senior Gender Specialist EASER Co-Task Team Leader Iamele P. Rigolini Senior Economist EASHD Task Team Leader Senior Human Development Maryam Salim EASHD Task Team Leader Specialist Financial Management Nipa Siribuddhamas EAPFM Financial Management Specialist Souphanthachak Sisaleumsak Procurement Specialist EAPPR Procurement Souksavanh Sisoulidavanh Program Assistant EACLF Project Support Senior Infrastructure Sombath Southivong EASTS Technical Project Support Specialist Kwanpadh Suddhi-Dhamakit E T Consultant EASHD Project Support Manida Unkulvasapaul Environment Consultant EASTS Environmental Safeguards Philaiphone Vongpraseuth E T Consultant EASTS ICR Support Ingo Wiederhofer Senior Operations Officer EASER Task Team Leader (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) USD Thousands (including Stage of Project Cycle No. of staff weeks travel and consultant costs) Lending FY02 27 240.42 FY03 2 27.11 FY04 0 0.00 FY05 0 0.00 FY06 0 0.00 FY07 0 0.00 FY08 0 0.00 Total: 29 267.53 Supervision/ICR 45 FY02 0 0.00 FY03 12 105.61 FY04 13 99.84 FY05 18 85.53 FY06 11 68.70 FY07 16 97.22 FY08 8 49.12 FY09 12 57.46 FY10 30 129.91 FY11 24 98.76 FY12 13 75.86 Total: 145 868.01 46 Annex 5. Beneficiary Survey Results NOT APPLICABLE 47 Annex 6. Stakeholder Workshop Report NOT APPLICABLE 48 Annex 7. Summary of Borrower’s ICR and/or Comments on Draft ICR Comments Received from the Project Implementation Unit of the Poverty Reduction Fund In 1996, the sixth Congress of the Party set the objective to free the Lao PDR from the status of least- developed country by 2020, especially in eradicating mass poverty by 2010. As rural poverty in the Lao PDR is directly linked to access to resources, social services and livelihood opportunities, the Government’s rural development strategy has thus two major components: improving access to essential factors of development, and a comprehensive, poverty-focused planning process at the district level to ensure that all initiatives are mutually self-supporting and complementary. In 2002, the Government of Lao PDR invited the World Bank to support its efforts to deliver development resources at the village level, targeted to the poorest districts in the country and mediated through strong participatory processes and decentralized decision making. To this end, the Poverty Reduction Fund was legally established by a Decree of the Prime Minister of Lao PDR (No. 073 / PM) on 31 May 2002 as an autonomous entity, overseen by a Board of Directors, and attached to and chaired by the Committee for Planning and Investment (it moved in 2006 to the Prime Minister’s Office). The PRF was given the authority to set its own personnel recruitment, procurement, and remuneration standards to help it attain its objectives. The PRF operational scope and design drew from best practices from a variety of World Bank funded social fund and community driven development projects under implementation at the time in East Asia, and adapted to fit with the distinct conditions of Lao PDR (based on the experience of the Committee for Planning and Cooperation). Therefore, the PRF’s mission is to support and establish local capacity, procedures and systems in line with the Government of Lao PDR’s decentralization policy, to efficiently and effectively channel and utilize various funding resources including loans, government and other donor’s funds, to reduce the poverty of rural communities. This was done through a program of highly participative, multi-sectoral, accountable grant-based, village-managed activities such as small-scale infrastructure projects, life skills development and income generation that enable rural poor and ethnic minorities to benefit from the country’s move towards improving socio-economic conditions for all citizens. After 8 years of implementation, the PRF program has actively participated in the implementation of the National Growth and Poverty Eradication Strategy (NGPES) and the National Socio-Economic Development Plans (NSEDP). The PRF program has proven its capacity to effectively and efficiently deliver resources to poor villages in remote areas by using the bottom up approach for the identification and planning of priorities by the communities, the management of funds at the village level, as well as the implementation of the sub-projects by allowing beneficiaries to choose whether they will implement the project themselves or contract the implementation to a contractor. On average, around 400 sub-projects were delivered on a yearly basis, with a strong focus towards women’s priorities (80% of the sub-project selected), and benefiting the poorest and small ethnic groups (for more than 1/3 of the total number of beneficiaries), with a level of community satisfaction reaching 80% in average. The bottom-up approach leads to a great level of ownership and sustainability. More than 2/3 of the sub-projects were implemented by using the community force account method, 96% of the sub- projects are still utilized at the end of the PRF additional phase, and 2/3 of the sub-projects have a maintenance fund in place and an Operation and Maintenance team operational. The high level of commitment and ownership of the communities are also expressed through the level of community contribution (16%), and contribute to the efficiency of sub-projects costs (between 35% and 88% in 49 comparison with other projects with similar focus in Lao PDR), and counterbalance the program’s relatively high operating costs. These excellent results were achieved thanks to the continuous support and advice of the Government of Lao PDR through the PRF Board as well as the donors, who worked in a cooperative manner to ensure that the PRF met its intended objectives and to improve aspects of the program’s overall performance as necessary in the course of implementation. For instance, during the additional financing period (2008- 2011), the Program reinforced its strong focus towards delivering support to the poorest; improving coordination with line ministries and understanding on the participatory planning approach, as well as reinforcing the quality and sustainability of the sub-projects, based on the lessons learned from 2003 – 2007. To ensure that disbursement did not lead PRF to wastage, leakage or fraud, intensive mechanisms for social and financial control were instituted in accordance with World Bank guidelines. This was reinforced by PRF local staff clearance, with villagers as signatories, of all sub-project disbursements on the basis of confirmed physical progress. The participatory process decision making was therefore subject to a strong set of controls over disbursements to minimize risks. In other words, the transparency and accountability of the PRF were subject to strict procedures, leading to no significant fiduciary issues. Nevertheless, in terms of procurement, minor procedural non-compliance were found repeatedly in a number of the contracts reviewed by the World Bank. These issues are mainly the consequences of the low level of literacy in some of the areas covered by the PRF, the difficulties of supervising a large number of sub-projects implemented on yearly basis (and nationwide), the remoteness of some sites and the difficulty of finding contractors to take on construction contracts in some remote areas. Therefore, there is a need to further simplify financial management and procurement procedures to improve community and contractors understanding and compliance, and also to strengthen the capacity at the community level through proper training and use of IEC tools. Despite the PRF impact on reducing Poverty is broadly recognized, the MIS was originally designed in- house and was found inefficient to accurately report on the PRF outcomes, despite the improvements made during the course of the project to mitigate this issue. The indicators were also not always reflecting PRF targeted achievements and therefore were revised and more “outcome-focused� during the PRF additional phase. Weaknesses were also found in the capacity of the PRF staff to properly analyze the data collected, thus limiting its value as a management tool. Therefore specific attention will be paid during the second phase of the PRF to ensure proper monitoring and evaluation and reporting towards the PDO and agreed key performance indicators. The program aimed to create stronger links between the local government and the aspirations of villagers with project staff at district, province and national levels coordinating and building linkages. Nevertheless, the PRF legal autonomy, the improved capacity of the PRF staff to deliver without need for outsource, the limited budget available to support coordination and capacity building of the local institutions and the communities may have been factors that have limited the implementation of this aspect of the PRF. Nevertheless, the data collected show that the benefits of the participatory planning process was step by step understood and integrated by the local authorities as the percentage of priorities selected by the communities and changed at upper level (kum ban and district level) was greatly reduced over time (30% in the original phase against 12% in the additional financing phase). This important aspect of the program will continue to be a strengthened during the second phase of the PRF. The other constraints that have affected the overall results of the program were associated with the high PRF staff turnover at the beginning of the program implementation, difficult access to bank facilities for the remotest communities, low capacity of some communities to manage their own sub-projects especially in term of procurement and ensuring quality of infrastructures quality, remoteness that leads to 50 difficulties in accessing sub-projects sites and that contributed to some sub-projects exceeding budget ceilings, lack of skilled local contractors, complex processes and procedures (i.e. procurement and safeguards), limited capacities of some PRF consultants and concerned sector technical staff, lack of coordination and collaboration with line ministries, lack of relevant sector infrastructure standards in key sectors (e.g. roads), and weaknesses in the Feedback and Resolution mechanism. Many of these issues have been solved during the first phase of the Project, and the remaining issues have been adequately taken into account in the design of the second phase. Due to the intended strategy of the PRF to give priority to the diffusion of the community based model on the wide scale versus concentration of its efforts in a limited number of provinces and districts, only a 7% of the total number of priorities identified by the communities in target districts were addressed per year. This situation may have led to frustration and planning fatigue at the village, kum ban and district level. This issue has also been well addressed in the design of the second phase of the PRF. Finally, there is a need for the PRF to further improve its coordination and collaboration with other donors and agencies in order to complement or supplement PRF support and vice versa. The PRF has made a promising start in this regard during the second phase of the PRF with the diversification of its activities (including disaster recovery), the formalization of Memoranda of Understanding with key sector partners, and the further diversification of financing. Based on its impressive record of rapid disbursement, large coverage, rapid delivery of services across a range of sectors and focus on small rural infrastructures, the project has itself become a channel for other funding sources focusing on improved access to water sources (SDC additional fund for water related infrastructures), food security in rural areas (LUFSIP sub-component 3a) and post-disaster response activities (LUFSIP sub-component 3b and a grant from the Global Facility for Disaster Risk Reduction). On this last point, the PRF has experienced the effects of natural disasters caused by cyclone Ketsana, with some of its infrastructures being impacted. Therefore, as part of the second phase, the PRF has strengthened its overall Disaster Risk Management and Response capabilities to ensure disaster-resistant technology is integrated into standard infrastructure design and delivery. Sub-projects Budget Source of funds implemented US$ PRF (cycle I-VIII) 3,179 36,476,032 GFDRR (cycle VIII) 14 313,582 LUFSIP sub-component 3A (cycle VIII) 233 2,332,962 LUFSIP sub-component 3B (cycle VIII) 92 3,342,129 Total 3,518 42,464,705 The access to more funding sources from different donors during the Cycle VIII gave the opportunity for the PRF to experience both scaling up of its coverage, diversification of its activities and the need for a stronger management of its funding sources and monitoring of its activities in a short period of time. The difficulties encountered in managing a budget that double its financial resources allocated to sub-grants in comparison with the other cycles highlighted some of the challenges meet by the PRF and lessons learned also feed the design of the PRF II, in its way to become a Government National program. The performance rating by the implementation, completion and results report related to the assessment of outcomes, assessment of Risk to Development Outcome, and the assessment of Bank and Borrower Performance are properly justified and therefore gives a realistic and fair reflection of the strengths and weaknesses faced by the PRF in the light of the overall socio-economic and institutional context the PRF had started and evolved all along these 8 years of implementation. 51 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Comments Received from the Swiss Agency for Development and Cooperation As a co-financing partner since 2008, SDC appreciates the open-minded analysis of the project made in the ICR. The report indeed captures well the significant results achieved by the Project, but is also critical on its shortcomings. SDC agrees with the overall conclusion that PRF has succeeded in achieving objective 1 (poverty reduction), but less successful regarding objective 2 “local capacity building�; and objective 3 “local institutions strengthening�. SDC therefore shares the assessment that the project performance has been only “Moderately Satisfactory�, even if on the ground, the Project has significantly contributed to the improvement of the living conditions of a very high number of poor villagers. The conclusions on institutional change/strengthening, as well as the lessons learnt (esp. re coordination with projects supporting system development) are very pertinent. SDC shares the analysis made in the report that as the project moves forward, the issue of institutional sustainability will gain importance and should be addressed with due attention. In this regard, SDC agrees with the analysis that the i) integration into local (sub-national) set-up; ii) linkage with other local government capacity development initiatives; and iii) cross-sectoral coordination between line agencies; are key elements that will need to be considered with particular attention in the implementation of PRF II to ensure that the project is well anchored into government system. SDC is aware of the complexity of monitoring and evaluation of project such as the PRF. SDC recognizes the weaknesses of the M&E system described in the report, but is confident that these have been addressed in the PRF II design. SDC considers that the rating of the Government performance is quite high, given the somewhat limited operational support received by the project during its implementation, and the difficulties faced by the NLCRDPE in its coordinating role. SDC however fully acknowledges and values the strong support that PRF has received at the highest policy level. Finally, while the issue of resettlement is addressed in the report, there is no conclusion on the role of the PRF in this regards. An assessment of how PRF is indeed an instrument contributing to avoid resettlement by providing services to remotest areas would have added to the analysis. In conclusion the report confirms SDC’s own assessment that PRF is an instrument well-suited to the Lao context, which provides new avenues to improve local service delivery and more inclusive citizens’ participation in public decisions. The report will constitute an important element for improving the future performance of PRF. 52 Annex 9. List of Supporting Documents World Bank Project Appraisal Document June 2002 World Bank Project Agreement August 2002 Report on Social and Economic Indicators Survey November 2004 World Bank Mid-Term Review December 2005 MEK / ACCMIN Final Report for Technical Assessment December 2006 Consultants Mixai Techno Beneficiary Assessment of PRF Project-Final Report January 2007 Engineering World Bank Poverty and Social Impact Assessment March 2007 World Bank, IMF, Lao PDR Public Expenditure Review Integrated Fiduciary May 2007 ADB & EU Assessment Tom Schacher Technical Consultancy Mission Report (22 November 2007-15 January 2008 January 2008 World Bank Social and Environment Safeguards February 2008 World Bank Project Information Document April 2008 World Bank Project Appraisal Document-Additional Financing June 2008 World Bank Financing Agreement July 2008 World Bank Community-Driven Approaches in Lao PDR Assessment of October 2008 PRF World Bank Second Quality Assessment of Lending Portfolio-QALP2 and April 2010 Task Team Laos: Review of Transparency, Accountability, and Corruption October 2010 in Public Sector World Bank Request for Clarification-Procurement Post Review for November 2010 Contracts in 2009-2010 World Bank Lao PDR Integrated Fiduciary Supervision Assessment-IFSA February 2011 Country Report World Bank Project Appraisal Document - PRF II April 2011 Synesis Consultancy Assessment of Poverty Reduction Fund Participatory Approach June 2011 World Bank Financial Monitoring Report June 2011 Mixai Techno PRF Technical, Utilization, and Beneficiary Satisfaction June 2011 Engineering Assessment 2011-Final Report Icon�Institute GmbH Assessment on Capacity Building of Local Institutions in the September 2011 und Co. KG Poverty Reduction Fund World Bank Measuring Impact of Com unity-Driven evelopment October 2011 Projects on Gender GoL PRF I Completion Report December 2011 World Bank Procurement Post Review Supervision Mission Reports 2003-2011 World Bank Aide Memoires of the Project Supervision Missions 2003-2011 World Bank Implementation Status and Results Reports 2003-2011 53 IBRD 33431 100°E 102°E 104°E 106°E 108°E This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and any other information shown on this map do not imply, on the part of The World Bank Group, any judgment on the legal status of any territory, or any endorsement or acceptance of such boundaries. CHINA LAO P.D.R. To Lincang To Gejiu To Daluo Gnot-Ou Ou 22°N 22°N Phongsaly Boun-Nua PHONGSALY PHONGSALY MYANMAR VIET NA M LUANG Muang NAMTHA Khoa Luang To Namtha Hanoi g kon Meung Muang Xiangkho BOKEO Me Xai Nambak u ng Sam-Neva O Ban Ta Be LUANG Viangxai Huaisai OUDOMSAI PHRABANG HOUAPHAN To Chiang Rai Xiangkhoang Plateau 20°N Pakbeng Luang Phrabang SAYABOURY SAYABOURY SAYABOURY Kham Nan Plain of Jars Phokhoun Xiang Khoang Sayaboury Kasi XIANG KHOANG Phou Bia Gulf of Vangviang (2,817 m) Ban Nalé Nalé Xaisomboun Nam Ngum Tonkin VIENTIANE Reservoir BOLIKHAMSAI Pone g Hong Paksane n o ek Khamkeut Paklay M Kadi VIENTIANE ng Me 18°N Xanakham 18°N ko ng PREFECTURE OF KHAMMOUANE VIENTIANE MUN. Cammon Plateau Ban Na Phan To Xe Thakhek e e Vinh Ba ng a a To fai Khon Kaen N oy To Khon Kaen Xebangfai 0 50 100 Kilometers SAVANNAKHETXepon SAV SAVANNAKHET Xéno Xéno 0 25 50 75 100 Miles T HA ILA ND Savannakhet Phin To Qui Nhon g S e B a n g hio n Samouay LAO PEOPLE'S 16°N 16°N SARAV Saravane SARAVANE SARAVANE DEMOCRATIC on D REPUBLIC Khongxedon SEKONG Sekong SELECTED CITIES AND TOWNS Pakxong To Ubon PROVINCE CAPITALS Ratchathani CHAMPASSAK Bolovens CHAMPASSAK Champassak Plateau NATIONAL CAPITAL Attapeu Sanamxai RIVERS TTAPEU ATTAPEU ATTAPEU MAIN ROADS RAILROADS Khong 14°N 14°N PROVINCE BOUNDARIES CA MBO DIA INTERNATIONAL BOUNDARIES 104°E 106°E NOVEMBER 2004