Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report December, 2016 ACRONYMS AND ABBREVIATIONS ANCOVA Analysis of covariance FGD Focus Group Discussion GFWS Gravity Flow Water System GoL Government of Lao PDR HH Household ICC Intracluster correlation coefficient IDI In-Depth Interview LECS Lao Expenditure and Consumption Survey MDG Millennium Development Goals MIS Management Information System NGES National Growth and Poverty Reduction Strategy NPRDPE National Program for Rural Development and Poverty Eradication NSEDP National Socio-Economic Development Plan OLS Ordinary Least Squares PAP Pre-analysis Plan PDO Project Development Objective PMU Project Management Unit PRF Poverty Reduction Fund PRSP Poverty Reduction Strategy Paper ACKNOWLEDGEMENTS This report has been prepared by the World Bank Lao Poverty Reduction Fund II Impact Evaluation Team. John Voss (World Bank, Consultant) led the Impact Evaluation Team and prepared the Final Report. Susan Wong (World Bank) provided guidance as Task Team Leader for the Impact Evaluation. Satoshi Ishihara (World Bank Laos, Task Team Leader for Lao Poverty Reduction Fund II) and Ingo Wiederhofer (World Bank) provided critical sup- port. Andrew Beath (World Bank), Emmanuel Skoufias (World Bank), and Louise Scura (World Bank) were peer reviewers. Indochina Research Ltd. implemented the field survey. The team would like to express its appreciation to General Manager Chansada Souvalansy; Project Leader, Khamsook Phom- mavongsa; and Qualitative Component Project Leader Bouasavanh Khanthaphat as well as the large team of enumerators, data entry operators, and management staff for their excellent work. Financial support for the impact evaluation and the baseline survey comes from the De- partment of Foreign Affairs and Trade of the Government of Australia, the Swiss Develop- ment Corporation, and the World Bank. The Government of Lao PDR has provided tremendous support to the impact evaluation over the past five years. Special thanks to Dr. Bounkouang Souvannaphanh (Executive Di- rector, PRF), Julien Rossard, and Hatthachan Phimphanthavong, for their generous support of the Lao PRF II Impact Evaluation and more specifically, the endline survey work. CONTENTS Executive Summary ......................................................................................................................... 7 I. Background .............................................................................................................................. 15 a. Introduction ............................................................................................................................. 15 b. Purpose of the PRF II impact evaluation ........................................................................... 16 II. PRF II Implementation ...................................................................................................... 17 a. Selection of project locations .............................................................................................. 17 b. Project Development Objective ........................................................................................... 18 c. Description of project activities .......................................................................................... 18 d. What has PRF II done? .......................................................................................................... 19 III. Research Design ................................................................................................................... 21 a. Introduction to the research design: randomized controlled experiment approach 21 b. Sampling design ..................................................................................................................... 21 c. PRF implementation in treatment Kumban ..................................................................... 23 d. Hypotheses .............................................................................................................................. 24 e. Data collection activities and survey instruments ......................................................... 27 f. Qualitative component design ............................................................................................. 29 g. Estimation methodology ...................................................................................................... 30 h. Accuracy and credibility of estimates ............................................................................... 31 i. Idiosyncratic imbalance (balance tests from baseline) .............................................. 31 ii. Attrition ................................................................................................................................ 31 iii. Contamination ................................................................................................................... 32 iv. Spillovers ............................................................................................................................. 32 v. Enumerator error ................................................................................................................ 32 vi. Publication bias ................................................................................................................. 33 IV. Results for Access to and Utilization of Basic Infrastructure and Services ..................................................................................................................................... 35 a. Introduction ............................................................................................................................. 35 b. Access to roads ....................................................................................................................... 36 c. Access to markets .................................................................................................................. 38 d. Access to protected water sources .................................................................................... 39 e. School building quality .......................................................................................................... 41 f. Access to health care .............................................................................................................. 42 g. Impact on poor households .................................................................................................. 43 V. Engagement with Local Government and Inclusion in Development Decision-making for Improved Service Delivery .................................................. 45 a. Introduction ............................................................................................................................ 45 b. Community engagement with local government ........................................................... 46 c. Participation in Village Decision-making .......................................................................... 49 VI. Discussion and Recommendations ........................................................................... 52 Annex A: Detailed Results .............................................................................................................. 55 Annex B: Power Calculations and Balancing Tests .................................................................. 60 Annex C: Detailed Methods Description ..................................................................................... 64 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 7 EXECUTIVE SUMMARY 1. The Lao People’s Democratic Republic has made significant progress in reducing poverty and increasing access to services over the past 20 years. Poverty has declined steadily: from 46 percent in 1993 to 23.2 percent in 2012/2013. However, Lao PDR remains one of the poorest countries in the region with an estimated per capita income of US$1,660 in 2015 and is classified by the United Nations as a lower-middle-income country. Consid- erable differences in poverty rates persist among different geographic areas and ethnic groups with all three major non-Lao-Tai groups, who constitute about 65 percent of the population, still recording poverty rates of 40 percent, compared to 15 percent among Lao- Tai (considered the majority group). Non-income poverty also remains a serious issue as the country faces multiple challenges with respect to nutrition, measles immunization, skilled birth attendance, and some dimensions of gender equality. 2. The Government of Lao PDR (GoL) prioritized and articulated its poverty reduc- tion strategy in the 2004 National Growth and Poverty Reduction Strategy (NGPES - the PRSP), which identified 47 districts as priority areas for poverty reduction interventions. The second phase of the Poverty Reduction Fund Project (PRF II) contributed to the Gov- ernment’s poverty reduction agenda by focusing on reducing poverty in relatively remote and inaccessible areas through financing investments in small infrastructure that facili- tate poor communities’ access to basic services and markets as well as contributing to strengthening citizens’ engagement and voice in local development. PRF II expanded the first phase of the program to four new provinces (Phongsaly, Oudomxay, Luang Prabang, and Attapeu), completing an additional 1,426 subprojects, including 479 in water and sani- tation (34 percent); 451 in the education sector (33 percent); 284 in roads and bridges (20 percent); 134 in agriculture, forestry, and energy (9 percent); and 78 in the health sector (5 percent). 3. The PRF II Project Development Objective (PDO) is to improve the access to and the utilization of basic infrastructure and services for the project’s targeted poor com- munities. Based on this, the project has identified a set of key outcome indicators included in its Results Framework: • Improved access to and utilization of basic economic and social services in subdis- tricts (kumbans) supported by the PRF: o % increase in access and utilization of health services o % increase in access to and utilization of protected water sources o % increase in access to and utilization of roads • Lowest two quintiles benefit from the above services. • Greater than 75 percent satisfaction levels reported by beneficiaries in targeted vil- lages regarding improved services and local development planning. • Decision-making on allocation of PRF resources involve at least 40 percent women and 60 percent poorest community members 8 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 4. PRF II uses kumban-based block grants supplemented by facilitators to conduct community development planning and fund the construction of infrastructure sub- projects. Villagers prepare village development plans that are integrated at the kumban level through an inclusive process led by elected village representatives. Kumban plans are then revalidated on an annual basis through a participatory process at the village, kum- ban, and district levels. The planning process includes a detailed assessment of communi- ties’ needs using social mapping and other relevant tools to identify priorities and ensure that the voices of vulnerable groups are heard. Kumban facilitators assist communities to develop plans and also monitor progress. Subprojects are implemented at the village level. Each targeted kumban receives up front a four-year budget to inform its planning and prioritization. An average annual budget allocation of US$42,000 is provided to PRF II target kumbans, for a total average investment amount per kumban of US$168,000 over four years, with subprojects financed and implemented on an annual basis. Subprojects are selected for financing at the kumban level by the PRF kumban committee (consisting of elected villagers including women and ethnic groups). PRF district staff, district local gov- ernment, and sector officials provide technical validation of proposals. The final decision for subproject financing is made at the kumban level by the PRF kumban committee based on transparent criteria and process as specified in the Project Operations Manual. 5. The PRF II Impact Evaluation uses a randomized treatment assignment designed to provide accurate and unbiased estimates of program impact. A set of hypotheses con- cerning the potential impact of the program based on the PDO and accompanying Results Framework and in consideration of PRF implementation during the period of evaluation were developed and accompanying indicators selected to evaluate specific outcomes. 6. Participation in the PRF II impact evaluation was assigned randomly by kumban, the unit of project implementation. Forty-four kumbans were selected to participate in the evaluation, across 11 districts in 4 provinces. In each of the 11 districts, 2 treatment and 2 control kumbans were selected randomly for a total of 22 treatment and 22 control kum- bans. Within each kumban, 100 households were surveyed randomly, stratified by village to ensure complete geographic coverage, for a total target sample of 4,400 households.1 Data collection occurred in two stages, consisting of two quantitative survey rounds and two qualitative studies: a Baseline Survey in September–October 2012 and Baseline Study in August–October 2012 (before project implementation began)2 and an Endline Survey in September–October 2015 and Endline Study in August–October 2015. The qualitative study was designed to complement the quantitative research design to determine the key factors influencing outcomes between PRF and control locations and the ways in which PRF II is a driver of impacts. The study used a combination of 64 key informant interviews 1 Aside from poverty rates, no other data were available at the kumban level that enable a matched pair or other matching approach to supplement the randomized treatment assignment. 2 Some limited project organization and planning activities occurred in Attapeu Province before data collection began. However, it is not expected that this implementation will impact results as it did not extend beyond initial Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 9 and 111 focus group discussions with 677 respondents to interact with local government of- ficials, PRF local staff, and community members in 16 villages (8 treatment and 8 control) selected from quantitative survey locations. The randomized treatment assignment allows for a straightforward approach to estimating project impacts for indicators across the seven hypotheses discussed below in Section III: the results at endline in PRF locations are compared with those in control locations. Baseline data is used as a control for any random pre-project differences when available. The results of the qualitative study are incorporated with the results from the quantitative survey into this report. 7. This report presents findings from the impact evaluation of PRF II, conducted be- tween September 2012 and October 2015 after 36 months of implementation. The main findings are as follows and summarized in Table 1: Access to and Utilization of Basic Infrastructure and Services • PRF II generates impacts where the benefits of the infrastructure provided can be realized in or near the village. PRF II generated significant positive impacts for sub- project types where benefits could be obtained in the village: water subprojects in- creased access to protected water sources, school building quality improved from the perception of community members, and time to travel to the nearest village in both the rainy and dry seasons was decreased. • Poor households share in the benefits from village infrastructure but constraints to benefits outside the village are magnified. Given the basic infrastructure needs of villages in the targeted kumbans, poor households are able to share in the benefits of village-located infrastructure, including water systems, school rehabilitation, and road access to nearby villages. In the case of water systems, they see an added ben- efit of access to protected water in the wet season, where although water may be plentiful, it is not taken from protected sources. • PRF II does not generate impacts when households need to travel far away from the village to receive benefits. Where households were required to travel far from the village, PRF II did not generate significant impacts: travel time to district cent- ers, access to roads in the dry and wet seasons, access to health care when sick, and measures of access to and utilization of markets outside the village. Given an average travel time of three hours to the district center and the remoteness of many PRF II villages, PRF II road projects are unlikely to be able to reduce the travel time to alter decision-making around seeking services or markets beyond nearby villages. These constraints are exacerbated for poor households with limited resources. • The key constraints to impacts located far from the village are the PRF II budget envelope, household resources, and uncertainty of service quality and outcomes: o Project budget envelope: Budget allocations per kumban do not allow for the repair or improvement of the larger kumban road network that would be neces- sary to reduce travel times to kumban and district centers. Road projects are only able to address one link to another village in the road network or within villages themselves. 10 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report o Household budget and resources: Even though health centers or road projects do provide greater access via some reduction in travel times, significant time and resources still must be devoted to reach health care services or outside markets to the extent that households prefer to sell goods to traders in the vil- lage and consult with traditional medicine practitioners in the village in the first instance. o Risk factors related to uncertainty: Exacerbating households’ budget and re- source constraints are uncertainties around potential benefits once the point of service is reached. Health centers are often poorly staffed, lacking resources and potentially discriminatory or providing poor quality of care; market prices in distant locations are uncertain in comparison with a competitive market from multiple traders visiting villages. These risks further reduce incentives to seek out benefits from PRF II constructed health and road subprojects. • PRF road improvements have increased the number of traders accessing the village. While road improvements over a limited set of segments between villages and district or kumban center markets was not sufficient to increase village access to markets, the number of traders accessing the village increased in the PRF locations due to cost reductions for travel as a result of road improvements. The larger number of traders created additional competition and increased the quantity of goods sold by villagers. Engagement with Local Government and Inclusion in Development Decision-making • PRF II creates greater voice for communities in decision-making. PRF II has in- creased respondents’ perceptions that their input in village affairs and decision-mak- ing is sought to a greater extent and has significant influence, including women and the poor, in addition to the full sample. Although many programs operating in PRF II locations involve communities in decision-making, PRF II has a stronger focus on community participation in every stage of subproject implementation and takes a more inclusive approach to ensure that all community members contribute. • PRF II increased attendance at general village meetings for communities as a whole and for poor women. Both poor women and the full sample of community members saw their attendance at village meetings held for any purpose (not limited to PRF II) increase as a result of PRF II. However, the perception of a stronger voice noted above and higher rates of attendance has not resulted in increased active participation, including speaking and joining a planning activity, for the full sample, women or the poor. • Perception of service outcomes in terms of village development are not impacted. Levels of satisfaction with specific service outcomes including education quality, and water were not impacted by PRF II for the full sample (inclusive of villages that did not receive PRF II subprojects). The findings showed decreased satisfaction with ac- cess to health care. As the findings were not limited to locations where infrastructure subprojects were constructed, given the relatively small number of subprojects and small budget envelope, PRF II is likely too small to overcome the large set of factors which determine outcomes, including other public infrastructure programs, and ser- vice provider resources and quality. Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 11 Table 1: Summary of Key Findings Indicator Finding Roads Access to roads (% of villages) No significant impact Time to nearest village (% of households) Reductions of 25 minutes in the wet season and 16 minutes in the dry season for PRF II villages compared with control villages Markets Selling goods outside the village No significant impact (% of households) Quantity of goods sold outside PRF II households sold fewer goods outside the village than house- the village (% of households) holds in control villages. Increased access for traders due to road improvement is a likely driver of the result. Water Access to protected water in No significant impact for the full sample. Poor households in PRF the wet season (% of households) II villages were 67 percentage points more likely to have access to protected water sources in the wet season compared with house- holds in control villages. Access to protected water in PRF II villages were 58 percentage points more likely to have the dry season (% of households) access to protected water sources in the dry season compared with households in control villages (61 percentage points for poor households). Schools School building quality PRF II households perceived school building quality as higher than (% of respondents) households in control villages. Health Care Seeking care when sick No significant impacts (% of respondents) Time to health facility No significant impacts (% of respondents) Community Engagement Communities with significant influence PRF II households were 10.3 percentages points more likely to on decision-making (% of respondents) state that their community had significant influence on decision- making in the village compared to households in control villages. Local government sought community PRF II households were 9.3 percentages points more likely to state input (% of respondents) that local government sought community input in the village compared to households in control villages. Satisfaction with local government ca- PRF II households were 3.7 percentages points more likely to be pacity to meet needs (% of respondents) satisfied with the capacity of local government to meet needs compared to households in control villages. Satisfaction with local capacity of de- PRF II households were 3.7 percentages points more likely to be velopment projects to meet needs (% of satisfied with the capacity of local development projects to meet respondents) needs compared to households in control villages. Participation in Village Meetings Attended a meeting PRF II households were 3.7 percentages points more likely to at- (% of respondents’ households) tend a village meeting in the last six months compared to control households. Attended the last village meeting Poor women were 3.3 percentages points more likely to attend a (% of respondents) village meeting in the last six months compared to control house- holds. Satisfaction with Services and Planning Satisfaction with village development No significant impacts plan (% of respondents) Satisfaction with the quality of No significant impacts education (% of respondents) 12 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report Indicator Finding Satisfaction with access to health care PRF II individuals were 6.6 percentages points less likely to be sat- (% of respondents) isfied with their access to health care compared to individuals in control villages. Satisfaction with access to water No significant impacts (% of respondents) Recommendations 8. As PRF I and II have addressed infrastructure deficiencies with primary schools in the village, PRF III should look at ways to facilitate increased access to junior second- ary schools and early childhood education. Over 33 percent of all subprojects in PRF II are primary school rehabilitation or expansion and in a small number of cases, new schools. As access to primary school is not a major constraint and infrastructure deficiencies in exist- ing schools are corrected, the largest obstacle to education access is at the junior second- ary level. The qualitative study determined that the critical constraints to junior secondary enrollment are lack of schools at the village level and lack of funds for transportation and boarding in kumban or district centers where the majority of secondary schools are located. Facilitating access could be achieved via grants for transportation, building dormitories, or subsidizing stays in existing dormitories. Improving access to secondary education could also improve completion rates at the primary level as many families pull children out of school before grade 5 as they do not see the purpose in children completing primary educa- tion when there is no opportunity to continue their education. This perspective was strong- est among poor and ethnic minority communities. PRF should work together with the Min- istry of Education on these initiatives. 9. PRF III should expand focus on protected water sources given the existing ex- tremely low rates of access. The average rate of access to protected water sources in the dry season throughout the entire sample is at 7.8 percent. As shown below in Section IV, water system subprojects can have a transformative effect on communities with respect to accessing clean water. Currently, communities across the four provinces in the sample use less safe collected rainwater during the rainy season and unsafe water from rivers and streams when rainwater is scarce during the dry season. The improvement in health, edu- cation, and economic outcomes that results from safe water has the potential to create larger impacts relative to other infrastructure types currently supported by the PRF. 10. PRF III should consider funding non-infrastructure needs: transport costs for ser- vice providers or service users (e.g secondary schools, nurses, midwives). Following on the first recommendation above, PRF III should raise awareness in communities as to the potential benefits of funding non-infrastructure activities as a means to improve service outcomes, particularly when infrastructure subprojects will not create large benefits due to travel costs. Road improvement can reduce travel times but in many cases not enough to encourage households to invest in the still long travel times required to reach service cent- ers. Instead, service providers (teachers, nurses, doctors) could be brought to the village on a periodic basis, in the same way that households prefer to sell goods to outside traders in the village rather than travel to farther away markets. Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 13 11. Improve linkages to larger road networks which allow easier access to kumban and district centers where markets and service points are available. Road subprojects should be focused on instances where roads significantly reduce the travel time and cost needed to reach service points and markets. If budget envelopes are not large enough to address the problem, consider bundling proposals for projects which both serve multiple villages and also provide improved linkages to larger road networks. Implementable main- tenance plans for cases where PRF funds are bundled together will be necessary to ensure benefits are sustained. 12. Increase integrated planning with district government and stakeholders to solve problems that are beyond the capacity of current kumban block grants. Further to the fourth recommendation, when PRF kumban block grants are not large enough to address road access problems to an extent which can benefit communities, integrate planning with district government and other stakeholders working in the kumban to ensure that gaps or deficits in funding or implementation can be resolved via integrated planning and pooling of resources to address specific needs. 13. PRF III should be more active as a broker bringing services to villages in a specific coordinator role at the district level. The PRF should play an active role in coordinating planning to address specific needs beyond the reach of individual stakeholders and pro- vide information to service providers at the district level for better resource allocation. PRF should encourage and facilitate line departments to use the PRF-built community platform to deliver services to remote villages cost effectively. 14 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 15 I. Background a. Introduction 1. The Lao People’s Democratic Republic achieved significant progress in poverty re- duction and access to services since the initiation of market-oriented economic reforms in the mid-1980s as the incidence of poverty has declined steadily over the last 15 years, from 46 percent in 1993 to 23.2 percent in 2012/2013. Over the same period about one- third of the population gained access to improved health, education, electricity, water, and sanitation services. However, Lao PDR remains one of the poorest countries in the region with an estimated per capita income of US$1,660 in 2015. 2. Considerable differences in poverty rates persist among different geographic ar- eas and ethnic groups with all three major non-Lao-Tai groups, who constitute about 65 percent of the population, still recording poverty rates of 40 percent, compared to 15 percent among Lao-Tai (considered the majority group). Non-income poverty also remains a serious issue as the country faces multiple challenges with respect to nutrition, measles immunization, skilled birth attendance, and some dimensions of gender equality. Stunting among children under five years of age is above 60 percent among ethnic groups, compared to the national average of about 44 percent. The gender gap remains high with less than 60 percent of women in poor households able to read and write, compared to over 80 percent of men who can. As with poverty, social indicators are worse in remote areas and among the non-Lao-Tai ethnic groups. 3. The Government of Lao PDR (GoL) prioritized and articulated its poverty reduc- tion strategy in the 2004 National Growth and Poverty Reduction Strategy (NGPES - the PRSP), which identified 47 districts as priority areas for poverty reduction interventions. The 7th five year National Socio-Economic Development Plan (NSEDP), which covered the period 2011–2015, continued the emphasis on achieving the Millennium Development Goals (MDGs) by 2015 and transitioning from Least Developed Country status by 2020. The Na- tional Program for Rural Development and Poverty Eradication (NPRDPE), which is a key input to the NSEDP, identified the following priority goals for increased Government atten- tion in rural areas: • Small-scale infrastructure and service delivery and livelihood development • Decreasing the service and income gap between rural and urban areas • Ensuring more integrated economic and social development, taking into account the importance of natural resource management and environmental conservation • Encouraging the participation and initiative of local communities based on the par- ticipatory development approach • Improved international and regional cooperation 16 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 4. The second phase of the Poverty Reduction Fund (PRF) Project contributed to the Government’s poverty reduction agenda by focusing on reducing poverty in relatively re- mote and inaccessible areas through financing investments in small infrastructure that facilitate poor communities’ access to basic services and markets as well as contributing to strengthening citizens’ engagement and voice in local development. PRF I, implemented from 2002 to 2011, had a successful track record in delivering services in remote areas quickly and at scale. PRF I provided support to approximately 2,185 communities in 8 out of 17 provinces and 30 out of 144 districts, including 23 priority poverty districts (out of 45). Since its establishment in 2002, 3,396 subprojects have been completed in around 2,000 villages. PRF II expanded the program to four new provinces, completing an additional 1,426 subprojects, including 479 in water and sanitation (34 percent), 451 in the education sector (33 percent), 284 in roads and bridges (20 percent), 134 in agriculture, forestry, and energy (9 percent), and 78 in the health sector (5 percent). b. Purpose of the PRF II impact evaluation 5. PRF I developed a monitoring and field reporting system to track progress in pro- ject implementation. However, evaluation efforts suffered from a flawed baseline and poor quality of data and lacked the ability to demonstrate project impacts for key areas of in- terest to the Government, including utilization of and access to services and community capacity. As a component to the second phase, the Government recognized the need to implement a rigorous evaluation approach to assess the effectiveness of PRF II and inform the design of PRF III or other future interventions. This report presents the results of a randomized impact evaluation, comprising two survey rounds with accompanying qualita- tive studies over a three-year period of evaluation (2012–2015). The impact evaluation was conducted on behalf of the Government with the assistance of the donor community. It compares changes in outcomes throughout four cycles (IX–XII) of implementation between PRF II villages and a set of control villages that have yet to begin participation in the pro- gram. The randomized treatment assignment approach enables the evaluation to attribute impacts on key outcome indicators to the project. Estimates of project impact are devel- oped via a series of hypotheses previously determined and disseminated in a pre-analysis plan (PAP) that was completed before examination of the endline survey data. 6. This report is organized as follows: • Section II presents the background for the Lao Poverty Reduction Fund project. • Section III describes the research design, analytical methods, and data collection. • Section IV presents the results of the impact estimation and qualitative study for ac- cess to and utilization of services. • Section V presents the results of the impact estimation and qualitative study for so- cial dynamics and engagement with local government. • Section VI provides conclusions and considerations for the future implementation of the PRF. Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 17 II. PRF II Implementation a. Selection of project locations 7. PRF II provides support to villages in 10 provinces: Savannakhet, Saravanh, Houa- phanh, Luang Namtha, Sekong, Xiengkhouang, Phongsaly, Oudomxay, Luang Prabang, and Attapeu. The first six provinces participated in PRF I and the final four began partici- pation with PRF II in 2012. The kumban (subdistrict) is the basic unit for poverty targeting. The project identified kumbans for PRF II on the basis of the following criteria: • Kumban poverty criteria specified in Government Decree #285/PM, specifically those related to poverty incidence, access to a road, access to water, and access to educa- tion and health services • Geographic location in terms of operational access and administrative cost-effec- tiveness • The presence of other similar programs in these kumbans 18 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 8. Based on these criteria, PRF II began implementation in 2012 in 38 districts across the 10 provinces, comprising a total of 274 rural kumbans. b. Project Development Objective 9. The PRF II Project Development Objective (PDO) is to improve the access to and the utilization of basic infrastructure and services for the project’s targeted poor com- munities. The project has identified a set of key outcome indicators included in its Results Framework that are used to evaluate performance against the PDO: • Improved access to and utilization of basic economic and social services in kumbans supported by the PRF: o % increase in access and utilization of health services o % increase in access to and utilization of protected water sources o % increase in access to and utilization of roads • Lowest two quintiles benefit from the above services. • Greater than 75 percent satisfaction levels reported by beneficiaries in targeted vil- lages regarding improved services and local development planning. • Decision-making on allocation of PRF resources involve at least 40 percent women and 60 percent poorest community members 10. While the impact evaluation will address all but the final two indicators (which are captured in the project’s management information system [MIS]), it is only one compo- nent of the portfolio of monitoring and evaluation activities that will be used to evaluate the program as a whole. These activities include data from the MIS, a Beneficiary Assess- ment Study, Technical Quality Audit, and thematic studies on gender, capacity building, and planning. c. Description of project activities 11. Community Development Grants. The primary intervention component is the use of kumban-based block grants for community development planning and subsequent con- struction of infrastructure subprojects. The kumban planning process is undertaken on a three-year rolling basis. Villagers prepare development plans at the village level that are integrated at the kumban level through an inclusive process led by elected village repre- sentatives. Kumban plans are then revalidated on an annual basis through a participatory process at the village, kumban, and district levels. The planning process includes a detailed assessment of communities’ needs using social mapping and other relevant tools to iden- tify priorities and ensure the voices of vulnerable groups are heard and included in the se- lection of the priorities. Kumban facilitators assist communities to develop plans and they also monitor progress. Subprojects are implemented at the village level. 12. Each targeted kumban receives up front a four-year budget to inform its plan- ning and prioritization. An average annual budget allocation of US$42,000 is provided to PRF II kumbans, for a total average investment amount per kumban of US$168,000 over four years with subprojects financed and implemented on an annual basis. Subprojects are selected for financing at the kumban level by the PRF kumban committee (consisting of elected villagers, including women and ethnic minority groups). PRF district staff and dis- trict local government and sector officials provide technical validation of proposals. The Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 19 final decision for subproject financing is made at the kumban level by the PRF kumban com- mittee based on transparent criteria and processes as specified in the Project Operations Manual, including the following: • 75 percent of subprojects must directly benefit the poorest communities within the kumban. • Subprojects must meet appropriate technical standards for infrastructure agreed upon with relevant sector ministries. 13. Local and Community Development Capacity Building and Learning. Communi- ties receive training to better assess their own needs; discuss identified needs with local authorities; implement and supervise the construction of small public infrastructure in- vestments, procurement, financial management, operations, and maintenance; and lastly monitor outputs and outcomes at the community and kumban levels. Village training ac- tivities are directly related to subprojects financed under Community Development Grants (such as establishment of Parent Teacher Associations for schools and water user groups). d. What has PRF II done? 14. PRF II has supported six categories of infrastructure. In the four provinces par- ticipating in the evaluation, a total of 439 subprojects were constructed over the period 2012–2015. The largest categories were Education, with 143 subprojects comprising 33 percent of total subprojects built, Water and Sanitation with 155 subprojects comprising 35 percent of total subprojects built, and Roads and Transport with 109 subprojects com- prising 25 percent of total subprojects built. Table 2 summarizes the distribution of sub- project types across the four provinces. Table 2: Distribution of Subproject Types by Province (2012–2015) Attapeu Luang Prabang Oudomxay Phongsaly Total % of total projects Agriculture/Forestry 4 2 3 3 12 3 Education 17 49 57 20 143 33 Energy 3 — — — 3 1 Health 4 — 7 6 17 4 Roads and Transport 13 40 28 28 109 25 Water and Sanitation 32 44 63 16 155 35 20 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 21 III. Research Design a. Introduction to the research design: randomized controlled experiment approach 15. The PRF II Impact Evaluation uses a randomized treatment assignment designed to provide accurate and unbiased estimates of program impact. A set of hypotheses con- cerning the potential impact of the program based on the PDO and accompanying Results Framework, and in consideration of PRF implementation during the period of evaluation were developed and accompanying indicators selected to evaluate specific outcomes. This section reviews the research design in detail: Section III.b describes the sampling design for both the quantitative and qualitative components. Section III.c reports on PRF II activities during the period of implementation. Section III.d discusses the set of hypotheses concern- ing PRF II potential impacts and accompanying indicators. Section III.e outlines the data collection process, including instruments and fieldwork. Section III.f reviews the data col- lection and methods for the qualitative study. Section III.g describes the estimation meth- ods and specifications used to develop impact estimates. Section III.g discusses procedures to ensure the accuracy and credibility of the results. b. Sampling design 16. Participation in the Lao PRF II impact evaluation was assigned randomly by kum- ban, the unit of project implementation. Forty-four kumbans were selected to participate in the evaluation, across 11 districts in 4 provinces. In each of the 11 districts, 2 treatment and 2 control kumbans were selected randomly for a total of 22 treatment and 22 control kumbans. The sampling frame comprised all kumbans which met the Government’s criteria for selection into PRF II, including a minimum estimated poverty rate greater than 40 per- cent. Within each kumban, 100 households were surveyed randomly, stratified by village to ensure complete geographic coverage, for a total target sample of 4,400 households.3 17. The kumban-level sampling frame was limited by the number of districts and kumbans planned for the four new provinces joining PRF II. The plan for implementation was for 14 districts and 114 kumbans in Phongsaly, Oudomxay, Luang Prabang, and At- tapeu Provinces. The breakdown of the sampling frame by district is shown in Table 3. Se- lection of the initial 114 kumbans were made by considering all kumbans in the 14 districts with poverty rates greater than 40 percent.4 All districts were new participants in Lao PRF II and were not a part of the original PRF I Project.5 Power calculations were conducted on a set of key indicators with resulting sample size requirements indicating that approximately 100 households in each of 40 kumbans consisting of 20 treatment (PRF II) and 20 control locations were needed to identify impacts at expected effect sizes for each indicator, a to- tal of 4,000 households.6 3 Aside from poverty rates, no other data were available at the kumban level that enable a matched pair or other matching approach to supplement the randomized treatment assignment. 4 Poverty rates were provided by the PRF II Project Management Unit (PMU) and were based on national house- holds surveys conducted in 2009 (National Household Survey [LECS] III). 5 The reason for the focus on new locations is to be able to establish a control group which has not previously received PRF assistance. Because under PRF I all kumban within a district received the project, a comparison of PRF II locations which previously received PRF I assistance would necessitate the use of comparison of kumban from different districts. This would not be ideal given the variation in governance environment, eco- nomic conditions, topography, and other factors across districts. 6 See Annex B for a detailed discussion of the power calculations conducted to determine required sample size. 22 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report Table 3: Province and District Kumban Poverty Distribution Province District Number of Kumbans with poverty rate > 40% Phongsaly Sam Phan 8 Mai 7 Oudom Xai Na Mo 5 Nga 9 Beng 7 Houn 13 Pak Beng 11 Luang Prabang Nam Bak 7 Phone Xay 10 Viengkham 10 Phoukune 7 Pak Zaeng 8 Attapeu Samakkyxay 8 Sanarmxay 6 Total 14 114 18. Sample selection was then conducted using the following steps: • District selection. A total of 11 out of the 14 districts were selected for the sample, apportioned to each province based on population. Phongsaly and Attapeu contained only two PRF II districts each, each of which was assigned automatically to the sam- ple; for Luang Prabang and Oudomxay, four and three districts, respectively, were selected randomly from the planned five in each district. • Kumban selection. Two treatment and two control kumbans were selected by simple random selection from within each district for a total of 44 kumbans.7 • Household selection. Within each kumban, 100 households were selected random- ly from for a total of 4,400 households. The 100 households were stratified across all villages in the kumban by dividing the 100 households by the number of villages. Households were then selected randomly using lists constructed in each village with the assistance of the village government.8 • Sampling weights were constructed to ensure that results are representative across the 11 districts, reflecting each sample household’s equal probability of being selected across all households in the sample.9 7 Oversampling of 4 kumbans above the 40 required by the result of the power calculations was conducted in case of loss of control. No loss of control did in fact occur. 8 Households were selected by compiling a list from the village head. A random starting number and interval were chosen by lot. Households were selected by taking the household of the randomly selected starting num- ber as the first household and then assigning subsequent households using the interval number going down the list. Households that refused to be interviewed or could not be contacted were replaced by additional households further down the list, again using the randomly selected interval number. 9 For a detailed discussion of sample weight generation, see Annex C. Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 23 Figure 1: Randomized Sampling Process in Phongsaly Province Phon s l Provinc Provinc District District 1: District 2: M i S mph nh Kumb n Tr tm nt Control * Tr tm nt Control* Control Tr tm nt * Control * Tr tm nt Kumb n 1 Kumb n 2 Kumb n 3 Kumb n 4 Kumb n 1 Kumb n 2 Kumb n 3 Kumb n 4 Tr tm nt Control Tr tm nt Control Control Tr tm nt Control Tr tm nt Hous holds 100 HHS 100 HHS 100 HHS 100 HHS 100 HHS 100 HHS 100 HHS 100 HHS 19. Table 4 provides a summary of the breakdown of treatment assignment by ad- ministrative units. Table 4: Treatment Assignment by Administrative Level Provinces 4 — District 11 — Treatment Control Kumban 22 22 Villages 146 128 c. PRF implementation in treatment Kumban 20. The primary intervention component to be evaluated by the impact evaluation is the use of kumban-based block grants for community development planning and subse- quent construction of infrastructure subprojects. As noted earlier, each targeted kumban received up front a four-year budget to inform its planning and prioritization. An average annual budget allocation of US$42,000 was provided to PRF II target kumban, for a total average investment amount per kumban of US$168,000 over four years with subprojects financed and implemented on an annual basis. PRF II was implemented during Cycles IX–XII of the overall PRF program and corresponded to the period November 2012 to August 2015, for the purposes of the evaluation. Table 5 details the extent of subproject construction during this period: Table 5: Number of Subprojects Constructed by Type in Treatment Kumban Type Subprojects Total Kumban Roads and Bridges 17 12 Schools 14 12 Health 4 12 Water 12 2 24 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 21. It should be noted that not all villages in a given kumban necessarily receive sub- projects, given the available budget envelope. Out of the 146 villages in the sample, only 45 villages (31 percent) implemented subprojects over the period of evaluation. Second, these subprojects vary across villages so that for any given infrastructure type, coverage is lim- ited across the entire sample.10 This variation presents challenges to impact measurement, as discussed in Section III.g. There are a number of subprojects for Cycle XII which were still being implemented at the time of the endline survey. The criterion for inclusion with respect to classifying a village as having implemented a particular infrastructure type is comple- tion by July 2015. 22. As noted in Table 6, the kumban planning process is undertaken on a four year rolling basis. Village meetings with facilitators were held during the first cycle of imple- mentation, in cycles when subprojects were implemented (generally only 1 per village over the period of evaluation) and occasionally in non-subproject implementation cycles for the purpose of validating the initial development plan. Table 6: PRF Facilitated Meetings Held by Cycle Planning Stage Subproject Number of Meetings Held in (Cycle IX Attapeu, Cycle X others) Construction Cycles Where No Subproject Was Constructed Attapeu Yes Yes 1 out of 3 cycles Luang Prabang Yes Yes 1 out of 2 cycles Oudomxay Yes Yes 1 out of 2 cycles Phongsaly Yes Yes 1 out of 2 cycles d. Hypotheses 23. The PDO for PRF II is to improve access to and the utilization of basic infrastruc- ture and services for the project’s targeted poor communities. The impact evaluation is designed to address the following set of PDO-level indicators from the Results Framework11: (a) Improved access to and utilization of basic economic and social services in kumbans supported by PRF • % increase in access to and utilization of health services • % households (HHs) with improved access to and utilization of safe water re- sources • % increase of HHs with access to all weather roads (b) Lowest two quintiles benefit from above services 10 Additional villages beyond the 45 implemented subprojects in Cycle XII did not complete construction by July 2015 and were thus excluded from the total. 11 For the PDO relating to satisfaction levels reported by beneficiaries regarding improved services and local development planning, given the small number of subprojects constructed in the impact evaluation sample, this indicator is more accurately measured through the project MIS. Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 25 24. Based on the Results Framework, a set of eight hypotheses were developed and 21 accompanying indicators, listed in Table 7, were selected based on the potential feasibil- ity of data collection and accuracy of responses. Hypotheses related to the market access dimensions of road subprojects, school infrastructure quality, and perception of service delivery were also included despite not being explicitly listed above after consultation with the PRF PMU staff and stakeholders. Table 7: Hypotheses and Indicators Hypothesis Indicators dir so secq lvl for I. PRF II increases access to i. Access to protected water source in protected water sources the dry season + HH WS-8 HH B ii. Access to protected water source in the rainy season + HH WS-10 HH B iii. Time to reach water source (rainy and dry seasons) − HH WS-N HH R II. PRF II increases access to i. Months access to road in rainy and roads dry seasons by car/truck + V IV-3/4 V R ii. Time to district center and nearest village by car/truck − HH MA-3, N V R iii. Cost to district center − HH MA-2, N HH R III. PRF increases access to i. Times per months goods are sold markets outside the village + HH MA-18 HH R ii. Quantity of goods sold outside the vil- lage + HH MA-11 HH R iii. Price differential for goods sold in the MA-15 village and market outside the village − HH MA-16 HH R IV. PRF II increases access i. Seeking care at medical facility if sick + I H-9 I B to health care services ii. Time to nearest point of health care service − I H-13 I R iii. Cost of transport to nearest point of health care service − I H-14 I R V. PRF II increases percep- i. Perception of quality of school building tion of quality of school in- and facilities + HH E-N HH C frastructure and facilities VI. PRF II increases satis- i. Satisfaction with planning process to faction with local develop- select village project + SC SC-N HH C ment planning processes ii. Satisfaction with capacity of sub- projects constructed to meet village needs + SC SC-23 HH C iii. Perception of influence of community over village affairs + SC SC-25 HH C iv. Perception of community participa- tion in decision-making + SC SC-24 HH C VII. PRF II increases satis- i. Satisfaction with education services + SC SC-N HH C faction with and perception ii. Satisfaction with access to health of services services + SC SC-N HH C iii. Satisfaction with access to safe water + SC SC-N HH C VIII. Households in the low- Consumption quintiles created via con- HH CF, CNF HH R est two quintiles of per sumption aggregate from HH module and capita consumption in the Market Price survey. sample benefit from access to roads, health care and safe water dir - Direction of Hypothesized Effect: +: positive, − : negative 26 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report so - Source of Indicator: HH: Household module; I: Individual module; SC: Social dynamics and governance mod- ule; V: Village module. lvl - Level of Indicator: HH: household; I: individual (sub-household); V: village. for - Format of Indicator: B: binary; R: real number; C: categorical. secq - Section and Question Number: WS: Sanitation-Water; IV: Part IV of village questionnaire; MA: Roads, Transport, and Market Access; H: Health; E: HH Roster and Education; SC: Social Capital and Government; CF: Consumption-Food; CNF: Consumption Non-Food; N: new question. 25. With respect to Hypotheses I, II and IV, the process of indicator selection was straightforward, utilizing indicators directly related to PDO-level indicators from the Re- sults Framework. For Hypothesis II, the decision was made to change the indicator for ac- cess to roads from a binary yes/no variable to months of access, based on the variability in access and duration of the rainy and dry seasons: in many cases roads were accessible during part of each season. Additional questions were asked on access, mode of transport, and time to the nearest village to capture the fact that many projects are only able to im- prove certain sections of roads rather than the entire route to the district center. 26. With respect to Hypothesis III, access to markets, the concept of the most im- mediate impact of new roads aside from accessing services was developed for the baseline survey to look at the extent which communities were able to sell goods in markets and the price they receive external to the village. While not specified in the Results Framework, im- pacts are potentially detectable through the impact evaluation at the household level. 27. For Hypothesis V, education, although originally not explicitly stated in the Results Framework, since a third of investments throughout the program were in schools and kin- dergartens, an approach to measure their impact was required. It is important to note that at baseline 95 percent of villages had a primary school and enrollment rates across grades was at 90 percent or above. The majority of school projects involved expansion of an ex- isting building or replacing an existing building with a new building. Thus, school projects were not providing a new school where none had existed before for the majority of cases. A number of indicators were then considered: • Enrollment. Rejected due to already high enrollment rates and the perception that remaining lack of enrollment is not due to infrastructure conditions. • Completion rate and attendance. While a new building might provide a greater per- ception of school quality and subsequent benefit to attendance from parents’ per- spectives, it is not clear that this effect could be strong enough to be detected. For attendance, there are significant measurement difficulties for households to recall absences and administrative data are difficult to access and unreliable. For comple- tion rate, given the longer-term nature of the indicator, there are additional factors outside the control of the project, such as teaching quality and curriculum which are likely to be more important than building quality. • Perception of school building quality. Ultimately, perception of school building qual- ity was chosen due to its feasibility in capturing impacts as the most direct result of school rehabilitation and expansion projects. Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 27 28. For Hypotheses VI and VII, local development planning and services, new questions were added (see Section V below) to capture satisfaction with service access and quality and the planning process in a new Service Satisfaction module. Indicators derived from ex- isting questions from the baseline survey around the community’s role in and influence over planning were also evaluated. 29. For Hypothesis VIII, a per capita monthly consumption aggregate was developed using an adapted version of the LECS IV national household survey. Considering that the threshold for inclusion in the sample was a poverty rate of 40 percent, the bottom two quintiles are a good proxy for poor households in the sample. 30. In addition to the indicators listed under these hypotheses based on the objectives outlined in the Results Framework, the evaluation also considers the impact of PRF II on participation and quality of participation in village meetings and decision-making.12 e. Data collection activities and survey instruments 31. Data collection occurred in two stages, consisting of two quantitative survey rounds and two qualitative studies: a Baseline Survey in September–October 2012 and Baseline Study in August–October 2012 (before project implementation began)13 and an Endline Survey in September–October 2015 and Endline Study in August–October 2015. Given the potential impact of seasonality on survey and study responses, the timing of both the quantitative surveys and qualitative studies for the endline survey was designed to match the dates for which the survey and study were conducted at baseline. Survey teams and qualitative study teams during endline data collection initiated fieldwork on the same days and sought to visit locations in the same order as in 2012. While delays or changes due to weather or access adjusted schedules slightly, all locations in 2015 were visited within one week of their date in 2012. Figure 2 summarizes the timing of the quantitative survey in the context of project implementation. Figure 2: Survey Timing and PRF Implementation Proj ct Impl m nt tion Proj ct Impl m nt tion Continu s B ins: Nov mb r 2012 with Y rl Block Gr nts 2012 2013 2014 2015 2016 2017 B s lin D t Endlin D t Fin l Imp ct Coll ct d: Coll ct d: Ev lu tion S pt mb r- S pt mb r- R sults: Octob r 2012 Octob r 2015 F bru r -M rch 2016 12 These additional topic areas were selected based on interest from key stakeholders. They were not included in the PAP but are based on a consultative process which occurred before data analysis began. 13 Some limited project organization and planning activities occurred in Attapeu Province before data collection began. However, it is not expected that this implementation will affect results as it did not extend beyond initial organization meetings at the village level. 28 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 32. The baseline survey met its target of 100 households for each of the 44 kumbans in the sample, except for a single village in Phongsaly Province, which had only 10 house- holds, leaving a total sample of 4,393 households at baseline. For the endline survey, households were tracked and re-interviewed at endline when they were still located within the kumban. In cases where households had moved outside the kumban, they were replaced with the household directly to the left of the original baseline household.14 In addition, the 10 households from the village in Phongsaly referenced above had all left the kumban with no opportunity for replacement.15 This reduced the total sample at endline to 4,383. Table 8 shows the breakdown of households by endline status. Table 8: Status of Baseline Sample Households at Endline Number of Households Located in the original baseline village 4,117 Located in another village in the kumban 23 Moved outside the kumban and replaced 243 Entire village moved outside the kumban 9 33. Both survey rounds used three survey instrument modules: a household survey, a village survey, and a market price survey. The instruments utilized questions from the LECS national household survey, conducted by the Department of Statistics, to the great- est extent possible to allow for future comparison. Before the baseline survey round, the instruments were field-tested three times in village locations which have similar poverty rates and significant minority populations, to ensure that interviews where translation of the instrument by enumerators was required produced similar comprehension from non- Lao speakers as with interviews conducted in Lao. A small set of questions were added to the household and village endline survey instrument to account for the PRF II implementa- tion experience and were field-tested twice. The survey instrument included the sections and corresponding respondents as listed in Table 9. Table 9: Survey Instrument Modules and Corresponding Respondents Instrument Section Respondent Household Roster, Housing Conditions, Head of household Access to Markets Access to Education, Health and Employment Individuals - Health: All individuals - Education: All individuals 6 years of age and older - Employment: All individuals 10 years of age and older Social Dynamics and Governance and One respondent per household: 50% male, 50% female Service Satisfaction assigned randomly (head of household or spouse). Village Survey Village head Market Price Survey At district market, conducted by field teams 14 Attrition bias is discussed below in part h of this section. 15 The entire village had moved outside the province and so the 10 households were dropped from the sample. Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 29 34. The profile of the sample by gender, ethnic group, age, and disability is presented in Table 10. Table 10: Sample Breakdown by Ethnic Group, Gender, Age, and Disability Ethnic Group % of Households Age % of Individuals Khmu 55.5 <6 12.8 Other ethnic groups 25.2 6–11 15.9 Hmong 11.5 12–17 12.9 Lao 7.8 18–30 17.0 30–50 17.7 Gender % of Individuals 50+ 23.9 Male 49.8 Female 50.2 % of individuals 8.8% with a disability16 f. Qualitative component design17 35. The qualitative study was designed to complement the quantitative research de- sign to determine the key factors influencing outcomes between the PRF and control lo- cations and the ways in which PRF II is a driver of impacts. The results of the study are incorporated into the findings below in Sections IV and V. The endline qualitative study was conducted simultaneously with the quantitative survey, in the same villages that were vis- ited during the baseline study: 16 villages were chosen from the locations surveyed under the quantitative component. Within each province, two districts were selected with one treatment and one control village in each district. Districts and villages were chosen purpo- sively to reflect geographical, ethnic, and socioeconomic variation.18 36. The study used a combination of 64 key informant interviews and 111 focus group discussions with 677 respondents to interact with local government officials, PRF local staff, and community members. The composition of community member focus groups considered gender, poverty, and ethnic minority status. 16 The sample of individuals with disability was too small to generate robust findings. 17 A more in-depth discussion of the qualitative component methods and design can be found in the full qualita- tive report that is submitted separately. 18 Discussions were held between the research team and PRF II district teams given the limited information on village characteristics at the national level. Villages which demonstrated higher poverty rates, remoteness from district and village centers, and ethnic group variation were selected by the field teams in consultation with the World Bank evaluation team. Every effort was made to ensure that the treatment and control vil- lages in each district had similar profiles with respect to these characteristics. 30 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report Respondents Main ethnic groups Type/Level of Respondent Year No. Male Female Total Lao-Tai Mon- Hmong Chine- FGD/IDI Khmer Iu-Mien Tibet Deputy District Governor 2012 8 IDIs 8 0 8 4 4 0 0 2015 8 IDIs 8 0 8 6 2 0 0 Rural Development Official 2012 3 IDIs 3 0 3 3 0 0 0 2015 8 IDIs 7 1 8 8 0 0 0 PRF District Coordinator/ 2012 8 IDIs 8 0 8 8 0 0 0 Team Leader (PRF staff) 2015 8 IDIs 7 1 8 8 0 0 0 Kumban staff (GoL staff) 2012 6 IDIs 6 0 6 0 6 0 0 2015 7 IDIs 7 0 7 1 4 2 0 PRF Kumban Coordinator 2012 6 IDIs 5 1 6 4 2 0 0 and Village Implementa- 2015 10 IDIs 5 5 10 2 7 1 0 tionTeam Village authority mem- 2012 16 IDIs 16 16 2 12 2 ber and Production Group 2015 21 IDIs 19 2 21 2 19 0 0 Leader Villagers - Luang Prabang 2012 29 FGDs 81 70 151 0 138 13 0 2015 28 FGDs 102 63 165 0 158 7 0 Villagers - Phongsaly 2012 22 FGDs 50 60 110 9 83 0 18 2015 26 FGDs 69 66 135 21 96 10 8 Villagers - Oudomxay 2012 24 FGDs 67 70 137 13 107 17 0 2015 25 FGDs 69 68 137 14 104 19 0 Villagers - Attapeu 2012 27 FGDs 70 73 143 8 135 0 0 2015 32 FGDs 78 100 178 16 162 0 0 Totals 2012 102 FGDs 314 274 588 51 487 32 18 41 IDIs 2015 111 FGDs 371 306 677 78 552 39 8 64 IDIs Note: FGD = Focus group discussion; IDI = .In-depth interviews g. Estimation methodology 37. The randomized treatment assignment allows for a straightforward approach to estimate project impacts for indicators across the seven hypotheses discussed above: the results at endline in PRF locations are compared with those in control locations. Two different specifications will be used to test the hypotheses: • Impact of treatment effects across the entire sample. For Hypotheses VI and VII, the impact evaluation will test the overall impact on indicators using a simple Ordinary Least Squares (OLS) approach. Baseline data are used when available to control for any random pre-project differences despite the randomized treatment assignment as a robustness check. • Impact of specific subproject types for Hypotheses I–V. Due to the smaller sample size associated with a given infrastructure type, an Instrumental Variables (IV) ap- proach is used to identify the impact of a particular type for the relevant treated lo- cations as construction was not widespread enough to generate detectable impacts across the entire sample. Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 31 38. For a detailed description of the specifications used, see Annex C. h. Accuracy and credibility of estimates 39. Sample kumbans were assigned treatment using a randomized assignment. This ensures that the methods outlined provide unbiased estimates of PRF II impacts. However, practical aspects concerning the project implementation and data collection may never- theless introduce bias or imprecision, thereby affecting estimates. The following sections note potential sources of such errors and provide the information necessary to evaluate their potential effects on the results, given actions taken to mitigate these effects. i. Idiosyncratic imbalance (balance tests from baseline) 40. The primary objective of the baseline survey was to determine that the randomized assignment of Lao PRF II kumbans and control kumbans was successful in ensuring that pre-project conditions across the two groups for factors which affect key outcome indi- cators are identical and that there is no idiosyncratic imbalance. To test this, comparison of means and comparison of distributions tests were conducted on 49 outcome indica- tors and village/household characteristics.19 In general, the balancing tests confirm that the treatment and control areas are statistically similar for most outcome indicators of interest. A detailed presentation of the results are found in Annex B. ii. Attrition 41. The randomized treatment assignment ensures that at baseline there is no selec- tion bias with respect to factors impacting outcome indicators. However, selection bias may be introduced if attrition is a direct result of treatment assignment. Table 12 shows the number of households lost between baseline and endline. Comparison of means tests demonstrates that there is no significant difference in attrition rates between treatment and control groups. Table 12: Attrition Between Baseline and Endline Re-interviewed Replaced due to Attrition Treatment 2,090 110 Control 2,050 133 19 Comparison of Distribution Tests were also conducted. The results did not differ from standard Comparison of Means tests. 32 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report iii. Contamination 42. Impact estimates can be influenced by the presence of non-PRF II infrastructure in both treatment and control locations. The PRF PMU conducted a thorough review of all infrastructure types constructed by entities at the village level for all kumbans in the sam- ple, consulting with PRF and district and village government staff. The results are shown in Table 13. While there are some differences, particularly with respect to education, a com- parison of the number of non-PRF II subprojects built in PRF II locations with the number of subprojects built in control locations shows that a systematic attempt to divert resources into control locations as a result of PRF II treatment assignment is unlikely. Table 13: Infrastructure Subprojects by Kumban PRF II-built Non-PRF II Built Control (in PRF II Kumban) Roads (new, upgrades, repairs) 17 8 6 Schools/preschools 14 37 53 Health clinics 4 1 4 Water systems 12 11 7 Total 47 57 70 iv. Spillovers 43. When project activities in treatment locations induce a behavioral response in control locations, estimates can be biased due to responses correlated with outcomes of interest. In this case, spillover is unlikely given the isolated nature of kumbans, the ten- dency of road networks to connect to district centers rather than other kumbans in the case of road projects, and the village-specific nature of most other infrastructure types. Based on a review of road network mappings completed by the survey firm in each kumban, infrastructure subprojects were not located in such a way as to benefit control kumbans in terms of access to or utilization of services. v. Enumerator error 44. Enumerators can face significant pressure in the field to collect data quickly as field teams fall behind schedule, seeking to decrease the time needed to administer sur- veys. This can result in nonsystematic errors related to question flow, delivery, recording accuracy, and potentially, fabrication of data. While this does not generally introduce bias, it can reduce the precision of estimates. To mitigate the potential effect of enumerator er- ror, the following steps were taken: • A week-long intensive training and selection program was designed for enumerators for the endline survey for a pool of enumerator candidates drawn from individuals who had participated in baseline surveys or similar quantitative surveys, frequent testing of enumerator candidates, and selection of enumerators based on perfor- mance in training, • Data was entered into tablets to reduce errors in skip codes and transition between modules. Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 33 • Data was uploaded daily or every few days and reviewed by a qualitative control team managed by the survey firm and reviewed by World Bank staff, allowing for correc- tion if errors were found. • Field monitoring of endline survey enumerators was conducted by the World Bank and PRF PMU staff. vi. Publication bias 45. To preclude the post-data collection selection and manipulation of indicators to produce significant results, often seen as a requirement for publication, a PAP was completed, peer reviewed, and posted before the data were reviewed and analysis was conducted. A small set of changes were made to the PAP based on changes in the survey instrument and data collection process but methods which were not planned for the origi- nal PAP were not introduced. An additional set of indicators related to participation and decision-making were not included in the PAP but were inserted after a consultative pro- cess with the PRF PMU staff and stakeholders before the endline data were received. These changes are discussed in detail in Annex C. 34 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 35 IV. Results for Access to and Utilization of Basic Infrastructure and Services a. Introduction 46. PRF II seeks to improve access to and the utilization of basic infrastructure and services for the project’s targeted poor communities as part of its PDO through disburse- ment of block grants to fund public infrastructure subprojects at the village level. This sec- tion presents the impacts of PRF II on outcomes related to specific infrastructure subproject types: road construction and repair, health clinics/dispensaries, school construction and rehabilitation, and protected source clean water systems. Section IV.b addresses village access to roads in the wet and dry seasons by cars/trucks. Section IV.c considers the im- pact of PRF II on access to markets. Section IV.d presents evidence on household access to protected water sources. Section IV.e reviews individual access to and utilization of health care services. Section IV.f assesses the perception of school infrastructure quality. Sec- tion IV.g looks at the same indicators for Sections IV.b–IV.f for poor households, defined as households in the bottom two quintiles of the per capita monthly consumption distribution. Results are presented in the following paragraphs. Coefficients demonstrating significant impacts at the 5 percent level are denoted in bold and with a double asterisk (**). Coef- ficients demonstrating significant impacts at the 10 percent level are denoted in bold and with a single asterisk (*). Detailed results are found in Annex A. The main findings from the quantitative study for each subsection are highlighted in bold in the text. Percentages are expressed in decimal form in the tables but converted to percentage points in the text. As noted in Section III, the estimation of impacts uses a comparison against control groups in all cases and controls for baseline data when feasible.20 47. Due to the budget envelope available at the kumban level, the coverage of PRF II villages with respect to any specific infrastructure type is too limited to detect signifi- cant impacts across the entire sample.21 Therefore, as noted in Section III.g, the analysis uses an approach to identify impacts only in the subsample of locations where relevant infrastructure has been constructed or where a village is in a position to benefit from in- frastructure in another village. In some cases where significant impacts are found, the full sample is also considered. The specific subsample used is identified in the tables. The cri- teria for a village to be included in the subsample based on infrastructure type are listed in Table 14. Table 14: Criteria for Inclusion in Infrastructure-specific Subsamples Type Criteria Water subsample PRF II water system in the village School subsample PRF II school subproject in the village Health care subsample PRF II dispensary/clinic in the village or road section constructed on the route from village to clinic Roads subsample PRF II road is connected to the village 20 In general, estimates are presented without inclusion of baseline data to ensure consistency across indicators for which baseline data do and do not exist. If the addition of baseline data reduces the significance level from 5 percent to 10 percent, it is noted with a footnote. If the addition of baseline data renders a finding insignifi- cant, it is discussed in the text. 21 Estimates of PRF II impact were estimated for all key indicators in Section IV under Hypotheses I–V. None were found to be significant at the 10 percent level. 36 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report b. Access to roads 48. Road subprojects account for 25 percent of all subprojects in PRF II locations and 36 percent of subprojects in sample kumbans. Of the 17 road subprojects in the sample kumbans, 14 connected villages to the larger village road network within the kumban22 and three improve intra-village roads designed to connect hamlets and agricultural field loca- tions to the village center. Table 15: Effects of PRF II on Access to Roads Hypothesis II: PRF II increases access to roads Coefficient P-value Obs Months of Access Dry Season, roads subsample –0.689 0.786 274 Months of Access Wet Season, roads subsample –0.879 0.731 274 Time to District Center Dry Season (minutes), roads subsample –80.300 0.474 274 Time to District Center Wet Season (minutes), roads subsample –119.300 0.428 274 Time to Nearest Village Dry Season (minutes), roads subsample –114.300** 0.047 274 Time to Nearest Village Wet Season (minutes), roads subsample –73.100** 0.036 274 49. PRF II reduced travel times to the nearest village, as shown in Table 15. When con- sidering only the subsample of villages with road subprojects or connected to another vil- lage with a road subproject, the travel time from PRF II villages to the nearest village is reduced by 114 minutes in the dry season and 73.1 minutes in the wet season. PRF II had no impact on the time to the district center or months of access to roads in either the rainy or dry seasons. 50. Villagers across socioeconomic, gender, and ethnic groups as well as local offi- cials viewed road access as the largest factor in escaping poverty, with incentives to produce agricultural goods for sale in outside markets or to traders as the means. Vang- bong, Viengkham District, Luang Prabang: A middle-income male said, “Our village is poor because there’s no road access. If there’s road access, trading will be more convenient: vil- lagers will have ways of trading. And villagers’ livelihood will get better.” An example of PRF addressing road access and the resulting impacts is in Nambak District, Luang Prabang. Village Officials in Yalo village said, “PRF constructed the road to Kiewchali village in Nam- bak. After having road access, villagers could access the Kumban and district easily; they grow more crops; they could make more money and be able to buy motorbikes and trac- tors. People have improved their houses. The road construction benefits villages in Kumban Songcha including Songcha village. They could access their production land more easily; traders from outside the village could come and buy in the village; and they could sell their produce in the market.” 22 This includes one bridge which served the same function of connecting the village to the larger village road network in the kumban. Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 37 51. Figure 3 and Figure 4 show the two types of road projects representative of the majority of road projects in the sample kumban: a road connecting a village to fields and to the larger road network, with targeted villages and road link in red.23 In both cases, it is important to note that the road segments which are constructed and/or rehabilitated are small segments of the larger road network and do not address problems with road access between other locations. Villagers do see value in roads which allow greater access to fields. The qualitative study indicated that after a PRF-widened road of 3,200 m to the villagers’ production fields, in Ban Tangdu, Beng District, Oudomxay, was completed in August 2015, villagers started to grow more maize and other crops to sell. Figure 3: Village Connected to Fields ແ�ວງ: ອ�ດຕະປຶ ຊກຸມບານ: ບານປາກບ ຫ�ວໜາທິມຕອງແ�ມແ�ນຜ�ງຂອງເ�ນທາງສ�ລ�ບແ�ລະກຸມບານ ນ�ເ�ຄ�ພໍ ບານດອນ ແ�ນທເ�ນທາງຂອງກຸມບານ ແ�ະ ກ�ນ�ດເ�ນທາງທສາງໂ�ຍ ທລຍ 18A ຫອງການກຸມ ບານປາກບ ບານປາກບ ຈ�ນວນບານທ�ງໝ�ດໃ�ກຸມບານມີ ເ�ືອງ: ສະໜາມໄ� ບານໜອງແ� ບານໂ�ນສະອາດ ນ�ເ�ປຽນ 4 Source: hand drawn by PRF II Impact Evaluation quantitative survey teams. 52. However, road projects can struggle to provide access to district centers and larg- er road networks as the feasible length of construction given PRF II kumban budgets is limited. The average time needed to access the district center is 139 minutes in the dry season and 192 minutes in the wet season. As PRF II funds are typically only able to address a few kilometers of one road segment within the larger road network, a reduction time of 24 minutes in the wet season will not significantly affect travel times to district centers or in some cases kumban centers. This can lead to situations where an identified road/ bridge priority at the village level cannot be funded and villages are forced to select options which are not as useful given existing needs. A poor Talieng female villager, Tadseng Village, Sanxay District, Attapeu, said, “I wanted PRF to build a bridge for crossing the Namtae, the bridge was my first priority when PRF asked us in 2012. In the rainy season we cannot go to market, cannot go to hospital, we have to wait 1–2 days, if the rains continuing for 3–4 days, we cannot go out of village for whole weeks up to 10 days, last year, my household was short of rice, then the rain was heavy, the river overflowed and even though we had money to buy rice we could not go to the market. Last year, I asked PRF again and they said forget about the bridge, it’s too expensive; your village will get a kindergarten school. Yes, but it’s not necessary.” 23 These maps were hand-drawn by the survey teams. 38 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report Figure 4: Village Connected to Larger Road Network Source: hand drawn by PRF II Impact Evaluation quantitative survey teams. c. Access to markets 53. Road access has the potential to increase access to markets, incentivizing crop production and the types of crops produced, the frequency of sale of products, and the price obtained. A poor woman discussing access to markets, Samphan District, Phongsaly, where a PRF road was constructed, said, “We usually carried bamboo shoots and some forest products and walked up and down the hill to Samphan district market. It takes us 1 day to go and come back just to earn 20,000–50,000 kip. We did not want to do it, but we needed cash for our children’s education. Nowadays, we do not need to go to the markets as there are 3–4 buyers at our village, and the prices are almost the same and we still have time to do work.” When villagers cannot access markets, there is less incentive to grow ad- ditional crops beyond subsistence level, even if they are not poor. Middle-income men, Van- gbong, Viengkham District, “We are facing difficulty because we don’t have road access. It’s so difficult to sell produce in the market especially in the dry season. If there’s road access… villagers will produce more. We want to grow maize and cardamom. We saw other villages grow and get out of poverty, we want to do this too.” 54. The impact evaluation considered the impact of PRF II roads on access to nearest markets, rate and quantity of selling goods outside the village, and the price difference be- tween selling in the village compared with the nearest market. As shown in Table 16, PRF II had no significant impact on the rate of sale of goods outside the village or the price dif- ferential for villages in the roads subsample in comparison with control households and Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 39 found that villagers sold goods fewer times per month (−3.81) and in less quantity (−663 kg) outside the village. The finding on times per month selling goods outside the village is not significant with the addition of baseline controls. 55. Although the sale of goods outside the village was reduced, the qualitative find- ings indicated that increased road access did lead to additional traders accessing the village. While villagers were not able to utilize roads to reach markets farther away at the kumban or district centers due to the lack of road improvements across all segments be- tween villages and markets, road improvement even on limited sections brings traders and increased competition for village-produced products. Traders can more easily and cheaply access villages as a result of better roads rather than households travelling to markets themselves. Although roads can reduce local travel times, the time and cost of reaching districts is still too high to warrant making a trip. Villages instead seek road improvements to increase the number of traders and decrease the cost of trader access to push up prices closer to those in district or kumban center markets. Table 16: Effect of PRF II on Access to Markets Hypothesis III: PRF increases access to markets Coefficient P-value Obs Selling goods outside the village (% of households), roads subsample 0.096 0.176 2,672 Times per months goods are sold outside the village, roads subsample –3.81* 0.085 268 Quantity of Goods Sold Outside the Village (kg), roads subsample –663** 0.027 4,383 Price differential for goods sold in the village and nearest market outside the –20,722 0.611 1,437 village (kip), roads subsample 56. This finding is supported by the reasons given by households who did not sell goods outside the village: 48 percent of households indicated that the primary reason for selling inside the village was due to the lack of a vehicle or a lack of traders from outside the village rather than road access/travel time. The qualitative study indicated that villagers owning large transport such as a car or truck would prefer to bring goods to the market themselves, but in the absence of vehicle ownership (the large majority of households), it is more cost-effective to sell to traders, given existing prices and transport costs. Villagers in Tad Saeng and Dakseang, Sanxay District, Attapeu noted the effect of road improve- ments on price differences between village and outside markets: “Now we have more trad- ers coming to our village, more Vietnamese traders than local. Prices are not much different between selling in the village and the market, for example, 1 kg live chicken is 25,000 kip in Daksaeng village and about 30,000 kip in the district market, not enough to balance the cost of transport and lost time.” d. Access to protected water sources 57. Water subprojects comprise 26 percent of all subprojects in PRF II locations and 35 percent of subprojects in sample kumbans. In sample kumbans, 12 water-related sub- projects were funded: 3 drilled wells, 2 water system renovations, and 7 new spring-fed gravity systems, all constructed approximately two years before the endline survey. At baseline in 2012, only 5 percent of households had access to protected water sources, with the vast majority using collected rainwater in the rainy season and unprotected water sys- tems, springs, or rivers/streams in the dry season. The average distance to fetch water in the dry season is only 100 meters, but these sources may be unsafe or not accessible, given broken water systems, polluted rivers, and water fees, particularly for poorer households. 40 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report In Namloy, Sampan District, Phongsaly, villagers have access to water, but in the dry sea- son—March to July—supply falls. Middle-income and poor women’s focus groups discussed the difficulties in the dry season: “In the dry season there is not enough water because we cut the forest upstream to plant cardamom. Apart from these problems, our water is dirty because the road is passing through the river. Every day trucks, hand tractors and bikes, animals cross the river. We really hope PRF will help us to improve or build a new water sys- tem as we do not know how to fix it.” A poor Khmu woman from Ban Sibounheuang, Houn District, Oudomxay, said, “My family is poor. We don’t have money to connect water to our house. We get water from the Beng river which is 2 km away to wash our dishes but the water is not clean like before because people from a banana plantation dump their garbage in the river.” Table 17: Effects of PRF II on Access to Protected Water Sources Hypothesis I: PRF II increases access to protected water sources Coefficient P-value Obs Access to protected water source - dry season (% of households), water subsample 0.582** 0.048 4,383 Access to protected water source - wet season (% of households), water subsample 0.522 0.198 4,383 Time to reach water source in dry season (minutes), water subsample –9.350 0.321 4,383 Time to reach water source in rainy season (minutes), water subsample –5.320 0.542 4,383 58. PRF II had a significant impact on access to protected water sources in the dry season, as seen in Table 17. For the water subsample where PRF II has constructed wa- ter subprojects, villagers saw an increase in access to protected water of 58.2 percentage points, in comparison to villagers in control kumbans . There is no significant impact in the wet season, likely reflecting the practice of rainwater collection as the primary source. 59. Although there was no significant impact found within the water subproject sub- sample on time to fetch water, evidence from the qualitative study shows that access to protected water sources can have a significant effect when distances are greater. A middle-income women’s group member from Vangbong Village, Vangbong District, Luang Prabang: “Women can spend more time in the field because we don’t have to worry about carrying water from the river. We can take a bath at night, and we can use toilets. It’s also convenient when there’s a wedding, village meeting, and village festivals.” 60. The qualitative study also noted that the budget envelope can be a problem for in- stalling gravity-fed water systems which are viewed as superior to more affordable bore- holes from a water quality standpoint but which in many cases cannot be constructed due to higher cost. In Tad Saeng, Sanxay District, Attapeu, villagers were unsatisfied with PRF built boreholes, questioning the quality of the water. In addition, there were cases of PRF boreholes being built on private land (Saxamxay District, Attapeu) instead of a large system built for the entire community. Finally, some hamlets of villages located outside the village center must travel several kilometers to fetch water even when protected systems are constructed. These hamlets are frequently populated by ethnic minority groups. An Akha ethnic group member, Tintok Village, Mai District, Phongsaly: “Sometimes my family has no water. I come back from the farm late so I do not take a bath. Normally, women carry the water for the family. Some days my wife works hard on the farm or garden, so she has no time to get the water.” Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 41 61. The qualitative study noted the importance of implementable maintenance plans to sustaining the benefits of new water systems. Tangdu Village, in Beng District, Ou- domxay, has a gravity flow water system (GFWS) with five standpipes. There is a water user group to which each HH contributes 12,000 kip per year. In late 2015, villagers lacked sufficient water, especially during the daytime due to a technical problem with the GFWS, requiring them to resort to older wells. The water management group and the village gov- ernment were unable to fix the problem. e. School building quality 62. Education-related subprojects account for 33 percent of PRF II-funded activities and 30 percent of activities in the sample. PRF II funded 14 education-related subprojects in sample kumbans, all encompassing the rehabilitation (7 subprojects) or expansion (7 sub- projects) of primary schools. Two of the expansion subprojects involved adding a classroom to accommodate early childhood education (kindergarten), which previously had been us- ing primary school classroom space. In sample areas, PRF II did not build a new school in a village which previously had none. As a result, PRF II impacts are focused less on creating new points of access and instead on improvements to the school building infrastructure, including providing a more stable, reliable and larger space to conduct classes, and toilets. The high incidence of primary schools in villages (95 percent) and existing high rate of en- rollment at baseline in 2012 (91 percent) indicate that school rehabilitation or expansion is unlikely to increase the volume of students who enroll and their frequency of school attend- ance. The qualitative study further identified attendance fees, uniform costs, gender bias, and ethnic minority status as key drivers of enrollment and attendance rather than school building quality or availability. In addition, while improvements in infrastructure are likely to contribute to learning outcomes (improved health due to toilets, teacher attendance, learning atmosphere, hours in classroom), the feasibility of the sample size to detect such an impact given the large number of other factors (teaching quality, learning resources, parental support, and so on) is limited. 63. Instead, perception indicators for school building quality from the perspective of household respondents are used as a proxy to assess whether factors related to school in- frastructure quality which impact higher-level education outcomes are being significantly affected by PRF II. Respondents were shown a card with six steps on a staircase from ‘1’ at the bottom to ‘6’ on top. They were asked to identify a poor quality school with ‘1’ and the best quality school with ‘6’. They were then asked on which step they would place the school in their village. As shown in Table 18, PRF II had a significant impact on the perception of school building quality: households in the school subsample of PRF II villages placed their schools 1.66 steps higher, on average, in comparison to control villages. Table 18: Effect of PRF II on Perception of School Quality and School Building Quality Hypothesis V: School building quality Coefficient P-value Obs Perception of quality of school building and facilities 1.66** 0.003 4383 (steps on card from 1 to 6) 64. The qualitative study captured only a single PRF constructed school—a kindergar- ten in Tad Saeng, Sanxay District, Attapeu, designed to move preprimary children from the primary school classroom to their own space. Villagers indicated that they thought it 42 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report would encourage children to follow on to stronger attendance and performance in primary school among minority groups. The women’s minority group respondents said, “We got the kindergarten from PRF this year and we are very happy. The building looks very nice and clean. This building might encourage our children to go to school instead of just playing or going to the field with their parents.” 65. PRF II also does not address secondary school access as there were no funded activities related to junior secondary school construction or funding of transportation or boarding costs for secondary students in kumban centers or district centers where existing junior secondary schools are located. The qualitative study noted that although enrollment rates in junior secondary school are increasing, road access and cost of attend- ance and boarding remains a major problem. Even when roads are good enough for travel by motorbike or car, only less-poor families have the resources to pay for transportation costs or dormitory rooms. In Hadsaykham, Sanamxay District, Attapeu, a Lavae male from the focus group wanted to send their children for post-secondary education but lacked the means: “We raise the issue every time there is a village meeting and we proposed to the village committee to find some scholarships for the students that finished school to study in secondary school. Every year the number of students increases but only those with a special relationship with government officials can continue studying while we don’t have enough money and don’t know people who work in the government. Our children could not continue studying in the school, that’s why teenagers who graduated feel lost when they want to study and there is no support from the village committee.” f. Access to health care 66. PRF II-funded health-related activities constituted 4 percent of all subprojects and 9 percent of subprojects in the sample. Four health subprojects were implemented: 2 new dispensaries and 2 dispensary rehabilitations. Given the range of factors influencing health outcomes (resourcing of clinics and quality and availability of care), the potential impacts for PRF II are through increased access. For this reason, road projects can also contribute to health access via reduced travel times and cost and villages affected by road subprojects are included in the health subsample. However, PRF II had no significant impact on the rate of seeking care when sick, time to reach the health care facility, or cost of transport to the health facility, as seen in Table 19. Table 19: Effect of PRF II on Access to Health Care Hypothesis IV: PRF II increases access to health care services Coefficient P-value Obs Seeking care when sick (% of individuals) 0.036 0.861 8,434 Time to health facility (minutes) –20.2 0.763 3,152 Cost of transport to health facility (kip) –33,834 0.374 3,152 67. The qualitative study noted that poor roads, particularly in the wet season and long travel times are a primary obstacle to access. The cost of travel to a health center or district hospital and the cost of treatment are often prohibitively expensive for poor villag- ers. If they are in dire need, they frequently have to go into debt or sell a major asset such as a buffalo. As a villager in Yalo Village, Nambak District, Luang Prabang noted, “If the road could be used all year round, it would be much more convenient and faster to take sick people to the health centre.” Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 43 68. The time and cost burdens are magnified by the potential for poor service quality at the kumban center. Staff absence, lack of resources and high prices lead many villagers to either consider traditional medicine options locally or travel farther to district centers. Mokkha, Viengkham District, Luang Prabang, Village Development Fund group: “The ser- vice of nurses is not good. They didn’t talk nicely to us and they sell medicine at a higher price than in the market.” A wealthy women’s group member, Ban Phonxay, Houn District, Oudomxay, said, “If any members of my family fall ill, we go to the district hospital because there are more doctors and medicines there. Even if we go to the health center here, we have to go and buy medicines in the district anyway.” 69. Poor and minority ethnic group households face additional barriers due to dis- crimination, further increasing the risk of spending resources on accessing care. A poor Hmong villager in Houn District, Oudomxay, said, “Many Hmong families don’t go to the health center because the medicine is expensive. Mostly, we practice traditional treatment. We feed our ghost by sacrificing cows and pigs. If we don’t get better, we go to the health center.” A poor male from the villagers group in Yalo said, “When we go to the hospital, the doctor would first of all ask us if we had already seen the shaman. If we hadn’t seen him, the doctor would not provide any treatment. Instead he would tell us to do the traditional ceremony first. He recommended this because he is a member of our ethnic group.” 70. The qualitative study also noted that staffing of new PRF-constructed facilities can be a problem with long waits until staff are allocated from the district despite an ex- isting agreement with the district that new facilities should be resourced as soon as they are constructed. The new dispensary in Laleo Kumban, Sampan District, Phongsaly, has been waiting since the end of Cycle XI in 2014. At the time of the survey in October 2015, staff had still not arrived. g. Impact on poor households24 71. The impact of PRF II on poor households across the indicators considered in Sections IV.a–IV.e follows the same pattern with similar magnitudes of impact, as shown in Table 20, including access to protected water sources in the dry season, quantity of goods sold out- side the village, and perception of school building quality. A key difference is a significant impact on access to protected water sources in the wet season. This is potentially due to the lack of rainwater collection systems among poor households. It is not surprising that no impacts were found for health care and access to markets, as the constraints on potential impacts (cost and time of travel and risk of service quality) are greater for poor households. Table 20: Significant Impacts on Poor Households Hypothesis VIII: Households in the lowest two quintiles of per capita Coefficient P-value Obs consumption in the sample benefit from access to roads, health care and safe water Access to protected water source - dry season, water subsample 0.611** 0.030 1,753 Access to protected water source - wet season, water subsample 0.671** 0.090 1,753 Quantity of Goods Sold Outside the Village (kg) –766** 0.030 1,753 Perception of quality of school building and facilities (steps on card from 1 to 6) 1.51** 0.007 1,753 24 Only impacts significant at the 10 percent level or less are presented. 44 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 45 V. Engagement with Local Government and Inclusion in Development Decision-making for Improved Service Delivery a. Introduction 72. PRF II seeks to achieve access and utilization objectives through inclusive commu- nity and local development processes with an emphasis on ensuring sustainability. This section considers the impact of PRF II on the capacity of communities and marginalized subgroups within those communities (the poor and women) to affect development planning and decision-making to hold local governments accountable and improve service delivery quality for overall village decision-making.25 Respondents in this section of the Household Instrument were randomly assigned to male and female household members.26 73. The primary potential driver of impacts are the facilitated village meetings and subsequent subproject implementation activities. PRF II held meetings with communities in the initial year the project began (2012) and then generally only when subprojects were implemented, as noted in Table 5. As noted in Table 11, a number of other programs were supporting infrastructure subprojects in villages during the evaluation period, both in PRF II and control kumbans. The qualitative study looked at differences between PRF II imple- mentation processes with respect to community involvement. Many of them used similar facilitated processes where communities identify needs and support subproject implemen- tation. The qualitative study noted three key differences: • PRF II has a stronger institutional footprint including more representatives at the vil- lage and kumban levels with better lines of communication. • PRF II communities are more active throughout the entire implementation process rather than only focusing on needs identification and subproject proposals. • PRF II places a stronger and more regulated emphasis on ensuring participation from women, the poor, and minority groups in contrast to other programs which merely encourage such participation. 25 Section V does not apply specifically to PRF processes and context. It considers the full set of village affairs and accompanying decision-making beyond PRF meetings. 26 Results for ethnic groups were not estimated due to the difficulty in determining which ethnic groups were disadvantaged. As noted in the sample profile above (Table 10), 92.2 percent households were non-Lao and thus non-Lao ethnic groups were the majority in most villages. As noted in the qualitative study, households in the bottom 40 percent of the consumption distribution are a good proxy for disadvantaged ethnic groups within a given village as these households tend to be farther away from the village center and have more dif- ficulty communicating in the Lao language. 46 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 74. This section will look at the extent to which these differences are able to drive im- pacts on engagement, participation, and ultimately satisfaction with local government policy and service delivery performance, for the entire sample and for women and the poor. The sample included all PRF II Impact Evaluation villages regardless of whether they received infrastructure subprojects. The indicators are not designed to address satisfac- tion with particular PRF II processes or outcomes, but rather the overall dynamics within the village, to be able to compare with villages in control kumbans.27 Section V.b looks at community engagement with local government, accountability with respect to local gov- ernment addressing needs, and resulting satisfaction with service delivery. Section V.c considers participation by the community in decision-making as a whole.28 b. Community engagement with local government 75. This section considers five indicators that are reflective of deeper community in- volvement and engagement with local government, addressing community influence and participation in development planning and decision-making, and satisfaction with the out- comes of development activities. With respect to community influence and participation, PRF II demonstrated significant positive impacts. As noted in Table 21, respondents in PRF II villages were 9.3 percentage points more likely to state that local government sought input from communities on issues of concern in the village and development planning in contrast to control village respondents. PRF II village respondents were also 11 percentage points more likely to state that communities had significant influence on decisions related to development planning and village affairs than respondents in control villages. 76. Findings from the qualitative study support these results. PRF II processes allow for deeper and sustained involvement at every stage of the process in contrast to other implementation actors. A poor male respondent in Namloy, Sampan District, Phongsaly, said: “I remember I went to the meeting with both PRF project and CARE project in 2012. The CARE project came to make village development plan with us before PRF, when we were in the meeting with PRF we told them we already did this with CARE project and we told them the same things we listed with CARE. PRF had more detailed discussion about where to select the subproject site. For example, if we said we need tracks they will ask which tracks are most used and benefits the majority of people. I think this method is very useful because we all agreed which one is number 1, 2, & 3 and all participants have the opportu- nity to be involved in decision-making.” 77. Local government actors were supportive of the project’s capacity to more easily coordinate activities with the district government through use of PRF II-established vil- lage actors, nominated from the community: Rural Development Office, Viengkham Dis- trict, Luang Prabang: “They established Kumban facilitators, village coordinators/actors. It is more convenient for communication and cooperation. The village actors could assist the project to drive the work smoothly and effectively. When district could not reach the village authorities, they could contact the PRF village actors to coordinate for them.” 27 Other data collection tools, including the MIS address issues of satisfaction and participation for specific PRF II activities. 28 As in Section IV, coefficients demonstrating significant impacts at the 5 percent level are denoted in bold and with a double asterisk (**). Coefficients demonstrating significant impacts at the 10 percent level are denoted in bold and with a single asterisk (*). Detailed results are found in Annex A. Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 47 Table 21: Community Role in Decision-making and Satisfaction with Development Out- comes Hypotheses VI and VII: PRF II increases satisfaction with local Coefficient P-value Obs development planning processes and satisfaction with and perception of services Communities have significant influence on decision-making (% of respondents) 0.110** 0.010 4,383 Local government sought community input (% of respondents) 0.093** 0.001 4,383 Satisfaction with local government capacity to meet needs (% of respondents) 0.037** 0.052 3,603 Satisfaction with the capacity of local development projects to meet needs (% of respondents) 0.037* 0.077 4,383 Satisfaction with village development plan (% of respondents) 0.450 0.806 4,383 Satisfaction with the quality of education (% of respondents) 0.182 0.409 4,383 Satisfaction with access to health care (% of respondents) –0.069** 0.003 4,383 Satisfaction with access to water (% of respondents) –0.041 0.240 4,383 78. With respect to satisfaction with the capacity of local government and develop- ment projects to meet needs and satisfaction with the village development plan, PRF II showed weaker impacts.29 As noted in Table 21, respondents in PRF II villages were 3.7 percentage points more likely to be satisfied for both the capacity of local government and local development projects to address identified needs. However, the finding with respect to local development projects is not significant with the introduction of baseline controls. While other actors have an impact on outcomes related to village government, develop- ment projects and planning (including local government and other donor-supported pro- grams), some PRF II implementation problems highlighted by the qualitative study provide some additional insight into potential project-related factors: • Budget envelope. As noted earlier, in some cases PRF II budgets were not large enough to implement projects addressing the most critical needs. Male ethnic minority group, Houn District, Oudomxay, “PRF informed us that they would build a health center for us because they had a limited budget, so villagers agreed. Actually, we really want to have a bridge to cross the river to our production field.” • Quality. In some cases, poor quality of construction had reduced the value of the infrastructure. Rural development officer in Beng District, Oudomxay: “PRF’s work is low quality compared to the money they spent; for example, the track widening to the villagers’ fields in Ban Tangdu in 2015. The distance was 3,200 m and the value was 270,000,000 Kip. However, this track has already partially collapsed. Compared to other projects, PRF’s work is low quality. PRF chose only the company they knew.” Quality issues seem to be related to procurement processes which select firms that do not ensure sufficient quality materials and technical expertise. • Transparency. Despite strong participation in planning, the result of implementation process, in particular why certain decisions were made, was not always clear. Village implementation team, Hadsaykham, Sanamxay District, Attapeu: “The information that most of villagers need to know from PRF is why we got the subproject and why we didn’t get the other subproject we wanted. PRF staff or headman didn’t give us the reason why we didn’t get the subproject that we proposed. That makes people confused and ask questions about the PRF budget. For example, we heard that the 29 It is important to note with respect to these findings that the full sample is being considered and not only communities where relevant water, health, and school subprojects were constructed. 48 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report value of the water wells is 3 million kip for 5–7 meters depth, but in some wells the water is rusty and cannot be used but it still costs the same amount, in some places they could not reach the water as they didn’t dig deep enough and the company just left it and still cost the same but PRF didn’t care about the quality of work; villagers have to collect money to hire the company to dig deeper to reach water. That why the expenditure and cost of subproject is unclear and the PRF coordinator and account- ing also didn’t know about the size of the budget, they just only did the transfers and paid per diems to the monitoring team.” 79. Finally, with respect to service delivery quality, there were no significant impacts on the quality of education children are receiving, and access to water. In addition, PRF II household respondents were 6.9 percentage points less likely to be satisfied with ac- cess to health care, in comparison with control respondents. This is potentially the re- sult of communities identifying health care access as a critical need but unable to secure funds due to limited budget envelope or limited capacity to improve sufficient roads needed to more quickly access services, as noted above in several examples from the qualitative study. 80. These patterns generally hold when looking specifically at women and poor house- holds, as seen in Tables 22 and 23: significant impacts are found for community influence in decision-making and local government seeking community input at similar levels of magnitude to the full sample; women and the poor demonstrate slightly higher levels of satisfaction with the capacity of local government to address needs. However, with re- spect to capacity of local development projects to address needs, no impact was found for the poor and women. No significant impacts are found with respect to satisfaction with village development planning, quality of education, and access to health and water. This is potentially due to the fact that the poor and women have less influence in decision- making despite attendance at meetings. Although PRF II targets the poor and women as active participants in the process, this does not ensure active participation. Viengkham District PRF team, Luang Prabang: “If the poor have fewer voices than other groups, their priority won’t be selected. Also if the priority of the poor is opposed by other interest in the village then they might not be brave enough to push their own interests in a meeting.” Table 22: Women’s Perception of Community Roles in Decision-making and Satisfaction with Development Outcomes Women Coefficient P-value Obs Communities have significant influence on decision-making (% of respondents) 0.104** 0.001 2,033 Local government sought community input (% of respondents) 0.102** 0.001 2,033 Satisfaction with local government capacity to meet needs (% of respondents) 0.045* 0.068 2,033 Satisfaction with local capacity of development projects to meet needs (% of respondents) 0.038 0.136 2,033 Satisfaction with village development plan (% of respondents) 0.150 0.504 2,033 Satisfaction with the quality of education (% of respondents) 0.040 0.216 2,033 Satisfaction with access to health care (% of respondents) –0.047 0.132 2,033 Satisfaction with access to water (% of respondents) –0.044 0.280 2,033 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 49 Table 23: Poor Households’ Perception of Community Roles in Decision-making and Sat- isfaction with Development Outcomes Poor households Coefficient P-value Obs Communities have significant influence on decision-making (% of respondents) 0.103** 0.001 1,753 Local government sought community input (% of respondents) 0.104** 0.000 1,753 Satisfaction with local government capacity to meet needs (% of respondents) 0.060** 0.038 1,753 Satisfaction with local capacity of development projects to meet needs (% of respondents) 0.042 0.163 1,753 Satisfaction with village development plan (% of respondents) 0.018 0.451 1,753 Satisfaction with the quality of education (% of respondents) 0.004 0.859 1,753 Satisfaction with access to health care (% of respondents) –0.055 0.105 1,753 Satisfaction with access to water (% of respondents) –0.011 0.801 1,753 c. Participation in Village Decision-making 81. PRF II seeks to maximize participation in the decision-making process across all subgroups with respect to its internal processes, including women and the poor. This sec- tion looks at the extent to which participation in PRF activities has influenced overall par- ticipation in village affairs. The evaluation considers four indicators associated with public participation in meetings held in the last six months: attendance, speaking in meetings, joining in planning development activities, and filing an official complaint with village of- ficials. Meetings included all those in the village and are not limited to PRF meetings. While these indicators looked at participation from households in general, the evaluation also looked at participation by women and poor households in the most recent village meeting. As shown in Table 24, the results demonstrated that PRF II had a significant impact on meeting attendance in the last six months: PRF II households were 3.3 percentage points more likely to send a member to a meeting in comparison to control respondents. In ad- dition, women in poor households were 3.4 percentage points more likely to attend the most recent meeting in comparison with poor women in control households. PRF II ex- hibited no impacts on more active participation in village affairs and meetings, including speaking at meetings, activity planning, and filing complaints. 50 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report Table 24: Participation in Village Decision-making in the Last Six Months and Last Meeting Full Sample - Last Six Months Coefficient P-value Obs Attended a meeting (% of respondents’ households) 0.033** 0.037 4,383 Spoke at a meeting (% of respondents’ households) 0.017 0.426 4,383 Joined development activity planning (% of respondents’ households) –0.016 0.520 4,383 Filed an official complaint (% of respondents’ households) –0.017 0.123 4,383 Bottom 40% - Last Six Months Attended a meeting (% of respondents - bottom 40% of consumption) 0.021 0.351 1,753 Spoke at a meeting (% of respondents - bottom 40% of consumption) 0.011 0.74 1,753 Joined development activity planning (% of respondents - bottom 40% of consumption) –0.019 0.598 1,753 Filed an official complaint (% of respondents - bottom 40% of consumption) 0.011 0.474 1,753 Most Recent Village Meeting Attended most recent meeting (% of respondents - bottom 40% of consumption) 0.010 0.616 1,571 Women attending last meeting (% of respondents’ households) 0.016 0.265 3,954 Poor women attending a Last Meeting (% of women - bottom 40% of consumption) 0.034* 0.091 1,571 82. Findings from the qualitative study broadly support these results. Villagers are more active in seeking out information by attending meetings in PRF II locations but are reluctant to participate actively, including speaking or joining a planning activity, particu- larly women and ethnic minority groups. A poor woman from Yalo, Nambak District, Luang Prabang, said, “Sitting and listening to the meeting is important for me. It helps me know about village development such as if there will be a project coming in to the village or not; and know what is going on in the village.” This perception extends to village officials. In Tin- tok, Mai District, Phongsaly, a village authority member indicated, “I could say that women actively attend the meeting, since PRF supported women and men to participate the meet- ing, they could select their own priority needs between men and women. In the past, not many women join the meeting, even they joined but they did not ask any questions.” 83. Key constraints on attendance and active participation include language, when meetings are held in Lao or Khmu, and lack of time, leading many poor and ethnic mi- nority members to focus on work in the fields. From Ban Sibounheuang, Houn District, Oudomxay, a Hmong women group member said, “We don’t want to come to the meeting because we have young children. We have many children and no one helps us. Some women have to stay in the field. Some of us want to join the meeting but our husbands didn’t let us.” Another woman in the group said, “When there’s a meeting, my husband attended. After the meeting, he never told me anything. So I don’t know what happened in the village.” Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 51 52 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report VI. Discussion and Recommendations 84. This report presents findings from the impact evaluation of PRF II, conducted be- tween 2012 and 2015 after 36 months of implementation. The main findings are as follows: Access to and Utilization of Basic Infrastructure and Services • PRF II generates impacts where the benefits of the infrastructure provided can be realized in or near the village. PRF II generated significant positive impacts for sub- project types where benefits could be obtained in the village: water subprojects in- creased access to protected water sources, school building quality improved from the perception of community members, and time to travel to the nearest village in both the rainy and dry seasons was decreased. • Poor households share in the benefits from village infrastructure but constraints to benefits outside the village are magnified. Given the basic infrastructure needs of villages in the targeted kumbans, poor households are able to share in the benefits of village-located infrastructure, including water systems, school rehabilitation, and road access to nearby villages. In the case of water systems, they see an added ben- efit of access to protected water in the wet season, where although water may be plentiful, it is not taken from protected sources. • PRF II does not generate impacts when households need to travel far away from the village to receive benefits. Where households were required to travel far from the village, PRF II did not generate significant impacts: travel time to district cent- ers, access to roads in the dry and wet seasons, access to health care when sick, and measures of access to and utilization of markets outside the village. Given an average travel time of three hours to the district center and the remoteness of many PRF II villages, PRF II road projects are unlikely to be able to reduce the travel time to alter decision-making around seeking services or markets beyond nearby villages. These constraints are exacerbated for poor households with limited resources. • The key constraints to impacts located far from the village are the PRF II budget envelope, household resources, and uncertainty of service quality and outcomes: o Project budget envelope: Budget allocations per kumban do not allow for the repair or improvement of the larger kumban road network that would be neces- sary to reduce travel times to kumban and district centers. Road projects are only able to address one link to another village in the road network or within villages themselves. o Household budget and resources: Even though health centers or road projects do provide greater access via some reduction in travel times, significant time and resources still must be devoted to reach health care services or outside markets to the extent that households prefer to sell goods to traders in the vil- lage and consult with traditional medicine practitioners in the village in the first instance. o Risk factors related to uncertainty: Exacerbating households’ budget and re- source constraints are uncertainties around potential benefits once the point of service is reached. Health centers are often poorly staffed, lacking resources and potentially discriminatory or providing poor quality of care; market prices in distant locations are uncertain in comparison with a competitive market from multiple traders visiting villages. These risks further reduce incentives to seek out benefits from PRF II constructed health and road subprojects. Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 53 • PRF road improvements have increased the number of traders accessing the village. While road improvements over a limited set of segments between villages and district or kumban center markets was not sufficient to increase village access to markets, the number of traders accessing the village increased in the PRF locations due to cost reductions for travel as a result of road improvements. The larger number of traders created additional competition and increased the quantity of goods sold by villagers. Engagement with Local Government and Inclusion in Development De- cision-making • PRF II creates greater voice for communities in decision-making. PRF II has in- creased respondents’ perceptions that their input in village affairs and decision-mak- ing is sought to a greater extent and has significant influence, including women and the poor, in addition to the full sample. Although many programs operating in PRF II locations involve communities in decision-making, PRF II has a stronger focus on community participation in every stage of subproject implementation and takes a more inclusive approach to ensure that all community members contribute. • PRF II increased attendance at general village meetings for communities as a whole and for poor women. Both poor women and the full sample of community members saw their attendance at village meetings held for any purpose (not limited to PRF II) increase as a result of PRF II. However, the perception of a stronger voice noted above and higher rates of attendance has not resulted in increased active participation, including speaking and joining a planning activity, for the full sample, women or the poor. • Perception of service outcomes in terms of village development are not impacted. Levels of satisfaction with specific service outcomes including education quality, and water were not impacted by PRF II for the full sample (inclusive of villages that did not receive PRF II subprojects). The findings showed decreased satisfaction with ac- cess to health care. As the findings were not limited to locations where infrastructure subprojects were constructed, given the relatively small number of subprojects and small budget envelope, PRF II is likely too small to overcome the large set of factors which determine outcomes, including other public infrastructure programs, and ser- vice provider resources and quality. Recommendations 85. As PRF I and II have addressed infrastructure deficiencies with primary schools in the village, PRF III should look at ways to facilitate increased access to junior second- ary schools and early childhood education. Over 33 percent of all subprojects in PRF II are primary school rehabilitation, expansion, and in a small number of cases, new schools. As access to primary school is not a major constraint and infrastructure deficiencies in exist- ing schools are corrected, the largest obstacle to education access is at the junior second- ary level. The qualitative study determined that the critical constraints to junior second- ary enrollment are lack of schools at the village level and lack of funds for transportation and boarding in kumban or district centers where the majority of secondary schools are located. Facilitating access could be achieved by providing grants for transportation, build- ing dormitories, or subsidizing stays in existing dormitories. Improving access to secondary education could also improve completion rates at the primary level as many families pull children out of school before grade 5 as they do not see the purpose in children completing primary education when there is no opportunity to continue their education. This perspec- tive was strongest among the poor and ethnic minority communities. PRF should work together with the Ministry of Education on these initiatives. 54 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 86. PRF III should expand focus on protected water sources given the existing ex- tremely low rates of access. The average rate of access to protected water sources in the dry season throughout the entire sample is at 7.8 percent. As shown earlier, water system subprojects can have a transformative effect on communities with respect to accessing clean water. Currently, communities across the four provinces in the sample use less safe collected rainwater during the rainy season and unsafe water from rivers and streams when the water level runs low during the dry season. The improvement in health, education, and economic outcomes that results from safe water has the potential to create larger impacts relative to other infrastructure types currently supported by the PRF. 87. PRF III should consider funding non-infrastructure needs: transport costs for ser- vice providers or service uses (secondary schools, nurses, midwives, and so on). Follow- ing on the first recommendation, PRF III should raise awareness in communities as to the potential benefits of funding non-infrastructure activities as a means to improve service outcomes, particularly when infrastructure subprojects will not create large benefits due to travel costs. Road improvement can reduce travel times but in many cases not enough to encourage households to invest in the still long travel times to reach service centers. Instead, service providers (teachers, nurses, doctors) could be brought to the village on a periodic basis, in the same way households prefer to sell goods to outside traders in the vil- lage rather than travel to farther away markets. 88. Improve linkages to larger road networks which allow easier access to kumban and district centers where markets and service points are available. Road subprojects should be focused on instances where roads significantly reduce the travel time and cost needed to reach service points and markets. If budget envelopes are not large enough to address the problem, consider bundling proposals for projects which both serve multiple villages and also provide improved linkages to larger road networks. Implementable main- tenance plans for cases where PRF funds are bundled together will be necessary to ensure benefits are sustained. 89. Increase integrated planning with district government and stakeholders to solve problems that are beyond the capacity of current kumban block grants. Further to the fourth recommendation, when PRF kumban block grants are not large enough to address road access problems to an extent which can benefit communities, integrate planning with district government and other stakeholders working in the kumban to ensure that gaps or deficits in funding or implementation can be resolved via integrated planning and pooling of resources to address specific needs. 90. PRF III should be more active as a broker bringing services to villages in a specific coordinator role at the district level. The PRF should play an active role in coordinating planning to address specific needs beyond the reach of individual stakeholders and provide information to service providers at the district level for better resource allocation. PRF should encourage and facilitate line departments to use the PRF-built community platform to deliver services to remote villages cost effectively. Annex A: Detailed Results Table A 1: Hypothesis I: PRF II increases access to protected water sources (water subsample) IV estimation method Baseline controls included Coefficient P-value Obs Control mean Coefficient P-value Access to protected water source - dry season (% of households) 0.582** 0.048 4,383 0.038 0.547** 0.039 Access to protected water source - wet season (% of households) 0.522 0.198 4,383 0.067 0.496 0.164 Time to reach water source in dry season (minutes) –9.350 0.321 4,383 2.64 Time to reach water source in rainy season (minutes) –5.320 0.542 4,383 2.66 Note: Indicators in bold are significant at the 10 percent (one asterisk *) or 5 percent (two asterisks **) level. Table A 2: Hypothesis II: PRF II increases access to roads IV estimation method Baseline controls included Coefficient P-value Obs Control mean Coefficient P-value Months of Access Dry Season, roads subsample –0.689 0.786 274 6.1 0.113a 0.918 Months of Access Wet Season, roads subsample –0.879 0.731 274 3.4 0.828 0.749 Time to District Center Dry Season (minutes), roads subsample –80.3 0.474 274 150 –90 0.307 Time to District Center Wet Season (minutes), roads subsample –119.3 0.428 274 206 –104.7 0.319 Time to Nearest Village Dry Season (minutes), full sample –24.8** 0.031 274 54.5 Time to Nearest Village Wet Season (minutes), full sample –15.8** 0.021 274 60.1 Time to Nearest Village Dry Season (minutes), roads subsample –114.3** 0.047 274 54.5 Time to Nearest Village Wet Season (minutes), roads subsample –73.1** 0.036 274 60.1 Note: Indicators in bold are significant at the 10 percent (one asterisk *) or 5 percent (two asterisks **) level. a. Baseline controls for Months of Access use Access to a Road in Wet/Dry Seasons as a proxy as Months of Access was not available at baseline. Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 55 56 Table A 3: Hypothesis III: PRF increases access to markets (roads subsample) IV estimation method Baseline controls included Coefficient P-value Obs Control mean Coefficient P-value Selling goods outside the village (% of households) 0.096 0.176 2,672 0.124 0.135 0.108 Times per months goods are sold outside the village –3.81* 0.085 268 2.4 –1.18 0.268 Quantity of Goods Sold Outside the Village (kg) –663** 0.027 4,383 3.034 –660** 0.027 Price differential for goods sold in the village and nearest market outside the village (kip) –20,722 0.611 1,437 18,257 –41,341 0.596 Note: Indicators in bold are significant at the 10 percent (one asterisk *) or 5 percent (two asterisks **) level. Table A 4: Hypothesis IV: PRF II increases access to health care services IV estimation method Baseline controls included Coefficient P-value Obs Control mean Coefficient P-value Seeking care when sick (% of individuals) 0.036 0.861 8,434 0.343 0.116 0.511 Time to health facility (minutes) –20.2 0.763 3,152 56.5 –36.2 0.587 Cost of transport to health facility (kip) –33,834 0.374 3,152 25,010 –9,188 0.798 Note: Indicators in bold are significant at the 10 percent (one asterisk *) or 5 percent (two asterisks **) level. Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report Table A 5: Hypothesis V: PRF II increases perception of quality of school infrastructure and facilities IV estimation method Baseline controls included Coefficient P-value Obs Control mean Coefficient P-value Perception of quality of school building and facilities (steps on card from 1 to 6) 1.66** 0.003 — — — — Note: Indicators in bold are significant at the 10 percent (one asterisk *) or 5 percent (two asterisks **) level. Table A 6: Hypotheses VI and VII: PRF II increases satisfaction with local development planning processes and satisfaction with and perception of services IV estimation method Baseline controls included FULL SAMPLE Coefficient P-value Obs Control mean Coefficient P-value Communities have significant influence on decision-making (% of respondents) 0.110** 0.010 4,383 0.492 0.110** 0.001 Local government sought community input (% of respondents) 0.093** 0.001 4,383 0.823 0.093** 0.001 Satisfaction with local government capacity to meet needs (% of respondents) 0.037** 0.052 3,603 0.715 0.041** 0.035 Satisfaction with the capacity of local development projects to meet needs (% of respondents) 0.037* 0.077 4,383 0.790 0.035 0.104 Satisfaction with village development plan (% of respondents) 0.450 0.806 4,383 0.840 — — Satisfaction with the quality of education (% of respondents) 0.182 0.409 4,383 0.816 — — Satisfaction with access to health care (% of respondents) –0.069** 0.003 4,383 0.710 — — Satisfaction with access to water (% of respondents) –0.041 0.240 4,383 0.715 — — FEMALE RESPONDENTS Communities have significant influence on decision-making (% of respondents ) 0.104** 0.001 2,033 0.509 0.103** 0.001 Local government sought community input (% of respondents) 0.102** 0.001 2,033 0.824 0.102** 0.001 Satisfaction with local government capacity to meet needs (% of respondents) 0.045* 0.068 2,033 0.721 0.048* 0.062 Satisfaction with local capacity of development projects to meet needs (% of respondents) 0.038 0.136 2,033 0.783 0.031 0.224 Satisfaction with village development plan (% of respondents) 0.150 0.504 2,033 0.855 — — Satisfaction with the quality of education (% of respondents) 0.040 0.216 2,033 0.838 — — Satisfaction with access to health care (% of respondents) –0.047 0.132 2,033 0.732 — — Satisfaction with access to water (% of respondents) –0.044 0.280 2,033 0.719 — — POOR HOUSEHOLDS (Bottom 40% per capita consumption) Communities have significant influence on decision-making (% of respondents) 0.103** 0.001 1,753 0.418 0.102** 0.001 Local government sought community input (% of respondents) 0.104** 0.000 1,753 0.792 0.102** 0.001 Satisfaction with local government capacity to meet needs 0.060** 0.038 1,753 0.674 0.064** 0.040 (% of respondents) Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 57 58 IV estimation method Baseline controls included Coefficient P-value Obs Control mean Coefficient P-value Satisfaction with local capacity of development projects to meet needs (% of respondents) 0.042 0.163 1,753 0.773 0.034 0.275 Satisfaction with village development plan (% of respondents) 0.018 0.451 1,753 0.817 — — Satisfaction with the quality of education (% of respondents) 0.004 0.859 1,753 0.799 — — Satisfaction with access to health care (% of respondents) –0.055 0.105 1,753 0.674 — — Satisfaction with access to water (% of respondents) –0.011 0.801 1,753 0.677 — — Note: Indicators in bold are significant at the 10 percent (one asterisk *) or 5 percent (two asterisks **) level. Table A 7: Hypothesis VIII: Households in the lowest two quintiles of per capita consumption in the sample benefit from access to roads, health care and safe water IV estimation method Baseline controls included Coefficient P-value Obs Control mean Coefficient P-value Access to protected water source - dry season, water subsample 0.611** 0.030 1,753 0.015 0.608** 0.023 Access to protected water source - wet season, water subsample 0.671** 0.090 1,753 0.034 0.674* 0.060 Quantity of Goods Sold Outside the Village (kg) –766** 0.030 1,753 253 –452** 0.1060 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report Perception of quality of school building and facilities (steps on card from 1 to 6) 1.510** 0.007 1,753 3.100 Note: Indicators in bold are significant at the 10 percent (one asterisk *) or 5 percent (two asterisks **) level. Table A 8: Additional indicators: participation in village affairs IV estimation method Baseline controls included FULL SAMPLE Coefficient P-value Obs Control mean Coefficient P-value Attended a Meeting (% of respondents’ households) 0.033** 0.037 4,383 0.846 0.031** 0.040 Spoke at a Meeting (% of respondents’ households) 0.017 0.426 4,383 0.422 0.018 0.363 Joined Development Activity Planning (% of respondents’ households) –0.016 0.520 4,383 0.585 –0.030 0.230 Filed an Official Complaint (% of respondents’ households) –0.017 0.123 4,383 0.743 –0.016 0.120 POOR HOUSEHOLDS (Bottom 40% per capita consumption) Attended a Meeting (% of respondents - bottom 40% of consumption) 0.021 0.351 1,753 0.807 0.019 0.398 Spoke at a Meeting (% of respondents - bottom 40% of consumption) 0.011 0.740 1,753 0.336 0.011 0.727 Joined Development Activity Planning (% of respondents - bottom 40% of consumption) –0.019 0.598 1,753 0.504 –0.033 0.351 Filed an Official Complaint (% of respondents - bottom 40% of consumption) 0.011 0.474 1,753 0.053 0.117 0.464 ATTENDANCE AT MOST RECENT VILLAGE MEETING - POOR AND WOMEN Attended most recent meeting (% of respondents - bottom 40% of consumption) 0.010 0.616 1,571 0.974 Women attending last meeting (% of respondents’ households) 0.016 0.265 3,954 0.138 Poor women attending a Last Meeting (% of women - bottom 40% of consumption) 0.034* 0.091 1,571 0.111 Note: Indicators in bold are significant at the 10 percent (one asterisk *) or 5 percent (two asterisks **) level. Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 59 60 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report Annex B: Power Calculations and Balancing Tests Power calculations 1. Table B 1 lists the total number of kumbans required to meet the sample size re- quirements, assuming 100 households are surveyed per kumban. A sample size of 44 kum- bans and 4,400 households will be enough to satisfy requirements for all key indicators listed below. Roads are calculated based on number of kumbans and villages rather than households. Table B 1: Number of Kumbans to Meet Sample Size Requirements All Rural Rural - No Rural - Four Target Road With Road Provinces -Rural Poverty 34 38 36 36 Enrollment Secondary (12–15) 22 30 22 20 Enrollment Secondary (15–17) 42 40 42 42 Percent Seeking Health Care When Sick 30 30 36 30 Percent Seeking Health Care When Sick and Perceived as Serious 34 36 34 42 Sanitation 34 38 34 36 Drinking Water Source Dry Season 42 38 38 40 Access to Road 44 Road Passable in Rainy Season 42 Note: Water Source: Acceptable is piped in water or protected well. Sanitation: Acceptable is modern or normal toilet. Poverty: Poverty Rate calculated using National Poverty Lines for 2007/2008 LECS. Health: Seeking Care When Sick and Serious Drops Response of ‘Not Serious Enough’. Education: Primary Enrollment Rates already above 96 percent for all categories. All power calculations completed using Optimal Design for Multilevel Longitudinal Research Software and con- firmed using standard power calculation modules in STATA. Sample sizes reflect a power of 80 percent at a 5 percent significance level, with an assumed range for an effect size of +/−5 percent for all indicators. Additional Note on Kumban versus Village Level Variation and Power Calculations 2. Although there are a small number of kumban and randomization is conducted at that level, the kumban is a recent phenomenon with little actual governmental and ad- ministrative impact on the region. Thus, variation is based on geographic/physical proxim- ity rather than membership of an administrative unit. Within kumban, there is significant variation from village to village with respect to service delivery, road access, and economic status. Therefore, the variance due to being located within a kumban is very low relative to the village level and is reflected in the intracluster correlation coefficient (ICC) calculations. When we look at the number of villages in the sample (320), similar power calculations us- ing village level parameters indicate that this will be sufficient to examine the utilization Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 61 rate and other binary variable questions, including subsamples. Thus, while there will be some loss of power due to concentration of the village locations in kumban, this will be far less in the Lao context in comparison than with other countries. Balancing Tests 3. The primary objective of the baseline survey is to determine that randomized as- signment of Lao PRF II subdistricts and control subdistricts was successful in ensuring that pre-project conditions across the two groups for factors which affect key outcome indicators are identical. To test this, comparison of means were conducted on 49 outcome indicators and village/household characteristics.30 The results are found in Table B 1. 4. In general, the balancing tests confirm that the treatment and control areas are statistically similar for most outcome indicators of interest. Here we highlight a few indi- cators where differences are statistically significant and potentially represent pre-project differences between the two groups. The cost of transport both to access health facilities and district markets demonstrates some evidence for significant differences. In the case of access to markets, households in the treatment group are spending on average approxi- mately 14,000 kip per trip more than households in the control group. For access to out- patient health care, treatment households are spending approximately 11,000 kip less to reach a health facility.31 On access to education, there is also some evidence for pre-project difference at the secondary school level. For children ages 13–15, treatment groups are 8 percentage points more likely to be enrolled than similarly aged children in control house- holds. Finally, for social dynamics and governance indicators, treatment households are significantly more likely to contribute materials or funds to community development pro- jects (such as repairing roads or drainage ditches) with a 17 percentage point difference in comparison to control households. 5. It is worth noting that for a large group of indicators, it is likely from a statistical standpoint that a small group of indicators will demonstrate differences purely by chance. However, the post-project implementation analysis to determine impact will attempt to correct for the identified pre-project differences discussed. All remaining indicators do not demonstrate significant differences between treatment and control households, including key household welfare and access indicators such as per capita consumption, food con- sumption percentage of total consumption and access to outpatient health care and pri- mary education. 30 Comparison of Distribution Tests were also conducted. The results did not differ from standard Comparison of Means tests. Results are available on request. 31 The significant level of the difference in the mean rate of contribution to of materials or funds to community development projects between treatment and control households is at the 5 percent level. The significance level for cost of transport to district markets approaches the 5 percent level. This is in contrast to cost of ac- cessing health care, rate of secondary enrollment for children ages 13–15 and perception of access to health care and education which are only significant at the 10 percent level. For balancing tests, the 10 percent level threshold is used as the standard to identify significant differences in pre-project conditions. 62 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report Table B 2: Balancing Tests Indicator Mean Mean T-statistic P-value Observations (treatment) (control) Sought Care when Sick 36.9% 36.8% 0.01 0.990 9,345 Cost to Health Care Facility (kip) 23,873 44,333 1.82 0.075 1,409 Secondary Enrollment 83.2% 75.5% 1.83 0.074 1,657 Primary Enrollment 88.4% 91.6% 1.19 0.241 4,620 Time to Secondary School (minutes) 25.3 24.7 0.15 0.881 1,156 Cost of Transport to Secondary School (kip) 619 473 0.58 0.566 1,156 Access to Safe Water Source (No rain collection) 4.2% 5.1% 0.25 0.807 4,393 Access to Safe Water Source (Including Rainwater) 73.8% 78.1% 0.72 0.475 4,393 Time to District Center (minutes) 256 252 0.04 0.971 4,365 Cost to District Center (kip) 22,031 37,345 1.84 0.072 3,424 Access to Sanitation 31.9% 33.9% 0.31 0.755 4,393 Price in District Centers (kip) 245,547 334,978 0.58 0.562 349 Price in Village (kip) 245,547 188,818 0.93 0.356 981 Per Capita Consumption (kip) 375,418 350,216 0.46 0.646 4,393 Food Consumption Percentage of Total 61.4% 61.8% 0.15 0.879 4,393 Access to Information on Use of Village Funds 23.3% 16.4% 0.98 0.332 4,393 Access to Information on Use of Community Funds 10.2% 10.2% 0.00 0.999 4,393 Access to Information on Use of Project Funds 26.3% 17.7% 1.21 0.234 4,393 Access on Information on Project Planning 29.3% 22.6% 1.36 0.180 4,393 Attended Most Recent Village Meeting 95.5% 95.5% 0.01 0.992 4,393 Spoke at a Meeting in the Last Six Months 36.8% 30.1% 1.50 0.141 4,393 Detailed Knowledge of the Village Development Plan 32.5% 33.6% 0.23 0.820 4,393 Petitioned the Government in the Last Twelve Months 6.9% 10.3% 0.78 0.441 4,393 Government Seeking Input from the Community Regularly 83.4% 84.6% 0.23 0.822 4,393 Community Can Reverse Decisions Taken by the Village Head 43.6% 45.9% 0.35 0.730 4,393 Satisfaction with Village Government Capacity to Handle Problems 36.2% 29.7% 1.10 0.277 4,393 Community Has Significant Influence in Village Affairs 42.4% 50.2% 1.54 0.131 4,393 Willingness to Contribute Time to Community Development Projects 94.9% 95.4% 0.29 0.771 4,393 Willingness to Contribute Resources to Community Development Projects 69.0% 51.7% 2.44 0.019 4,393 Village and Household Characteristics Access to Electricity 32.8% 29.1% 0.33 0.742 4,393 Farming Primary Occupation 92.1% 92.9% –0.49 0.625 4,440 Poor 37.4% 33.8% 0.51 0.611 4,440 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 63 Indicator Mean Mean T-statistic P-value Observations (treatment) (control) Number of Household Members 5.8 5.7 0.70 0.485 4,440 Gender (female) 50.6% 50.1% –0.83 0.409 25,125 Age 23.2 22.6 1.09 0.281 25,125 Years of schooling 1.7 1.6 0.49 0.624 25,125 Primary School in Village 97.9% 97.7% 0.13 0.898 274 Secondary School in Village 7.7% 11.2% –1.46 0.152 268 Health Post in Village 9.6% 15.6% –1.48 0.147 274 Time to Nearest Health Facility 106 93 0.65 0.522 240 Rice Shortage Last 12 Months 3.4 3.2 0.47 0.637 268 Price of Rice 3,900 4,140 –0.74 0.465 240 Daily Wage for Laborer 35,862 39,630 –0.86 0.399 56 Truck Access in Dry Season 92.6% 94.5% –0.43 0.67 231 Truck Access in Rainy Season 32.2% 39.1% –0.56 0.576 231 Access to a Rod 82.9% 85.9% –0.33 0.741 274 Number of Households 79.8 72.6 0.73 0.472 274 Number of Minority Group Households 63.9 62.8 0.15 0.883 274 64 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report Annex C: Detailed Methods Description Estimation of Treatment Effects Estimation of Treatment Effects for the Entire Sample (Hypotheses VI–VII) 1. All hypotheses with indicators for which baseline data is available will be tested us- ing an analysis of covariance approach of the following form: (1) Y1vi = a + tr1Tv +-Y0vi + gv + uvi, where Ytvi is the outcome of interest at time t (0 = baseline, 1 = endline), in village v and household i, Tv is a dummy representing PRF treatment, g are village/district fixed effects, and u is the error term. The coefficient of interest representing the impact of PRF is repre- sented by tr1. All standard errors are clustered at village level. The analysis of covariance approach provides a more flexible treatment of random variance between treatment and control groups in comparison with the standard difference-in-difference estimator. 2. In cases where baseline data is unavailable, the following specification will be used, where tr1 represents the impact of PRF on the indicator of interest: (2) Y1vi = a + tr1Tv + gv + uvi. 3. The standard difference-in-differences method was chosen given the relatively im- mediate and consistent impact that new infrastructure is expected to have on the indica- tors of interest. In particular the assumption of parallel time trends between treatment and control groups in stronger for impacts of this type. Estimation of Effect of Specific Types of Projects on Individual Indicators32 (Hypotheses I-–V) 4. As noted in Section II, Lao PRF II funds the implementation of subprojects selected by the village community. Given that for all but two villages in the sample, only one type of subproject was built over the period of evaluation, detection of overall treatment effects across the entire sample which are directly related to specific infrastructure subproject types is less feasible. Thus, for hypotheses pertaining to specific infrastructure types (I- –V), estimation of the overall treatment effects of the program is supplemented by esti- mating the treatment-on—the-treated effect of particular infrastructure types in villages where each type has been located or villages which stand to benefit from such infrastruc- ture constructed in other villages in the kumban. 32 This section closely follows the methods used in the NSP Impact Evaluation and can be found at the following web address: http://www.nsp-ie.org/paps.html. Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report 65 5. To undertake this approach, the choice of project type needs to be accounted for as it is endogenous to the treatment assignment: the choice is only revealed for villages which receive the project. Once treatment status is determined, the choice of project type is driven by a set of unobserved variables. These unobservables may not be distributed equally between the entire control sample and the subset of villages choosing a particular infrastructure type, potentially biasing results in a comparison of a subset of households in treatment villages with the entire control sample. 6. To control for such unobservables, we compare the villages from the treatment group that have chosen a project of a particular type to villages in the control group that would have chosen the project of the same type if they had received support from PRF II. Formally, denote by Cvj a dummy that indicates that a village v will choose the project of type j, if given a chance, and the corresponding outcome of interest by Yvj. Denote by Y0vj and Y1vj potential outcome Y in village v without treatment and with treatment, respectively. Cvj is only observed in treatment villages in the case where Tv = 1; the realized value of Yvj is thus (3) Y1vj if Tv = 1 and Cvj = 1, Y0vj otherwise. 7. The estimate of interest is the treatment-on-the-treated effect: (4) ttot = E [Y1vj - Y0vj | Cvj = 1]. 8. It can be demonstrated that the treatment-on-the-treated effect can be estimated using an instrumental variable regression with Cvj as the endogenous variable and Tv as the instrument.33 The treatment assignment is uncorrelated with unobserved variables in the error term driving project choice because treatment was randomized and not conditioned on such variables. However, it is correlated with Cvj as project selection only occurs in vil- lages where treatment is received. 9. To then estimate the effect of particular infrastructure subproject type, the follow- ing instrumental variable regression is used: (5) Y1vi = a + tivj Cvj + gp + uvi, where Cvj is instrumented by Tv. Standard errors are clustered at the village level. 33 See http://www.nsp-ie.org/paps.html Appendix A for formal proof. 66 Lao Poverty Reduction Fund Phase II Impact Evaluation: Final Report Part 2: Sampling Weights 10. Sampling weights were constructed to control for differential probability of selec- tion to populations within administrative regions to ensure that the results reflect a repre- sentative depicture of the reference population, in this case the 11 districts which comprise the sample. The weights are calculated as the inverse of the sampling probability: P(household) = P(kumban) * P(village/kumban) * P(household/village), P(kumban): kumban selected/total kumban, P(village): villages selected/total village (in this case always 1), P(household): households selected/total households. Part 3: Changes from the PAP • Change from village fixed effects to district fixed effects. In cases where village or village group fixed effects perfectly predict outcomes in any one location, district ef- fects are used. • Dropping of distance to water source and road distance indicators. The survey firm noted that villagers demonstrated lack of knowledge of distances to water sources, other villages and district centers. As a result these variables have been dropped. Vil- lage head responses are used for distance and time estimates to nearest village and district center. • Road access in rainy or dry season changed to months of access to more accurately reflect road usage capability more accurately as, particularly in the rainy season, roads are passable for a portion of the season. • The original difference-in-differences estimator was replaced by an ANCOVA ap- proach described above. This change was made before the endline data were received. However, all regressions were also run using the difference-in-differences estimator when baseline data were available and did not demonstrate changes in level of signifi- cance, direction of impact, or large changes in magnitude of impact. World Bank Vientiane Lao PDR Xieng Ngeun Village, Chao Fa Ngum Road, Chanthabouly District P.O Box: 345, Vientiane, Lao PDR Tel: (856-21) 266 200 Fax: (856-21) 266 299 Website: www.worldbank.org/lao The World Bank 1818 H Street, NW Washington, D.C. 20433, USA Tel: (202) 4731000 Fax: (202) 4776391 Website: www.worldbank.org