Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00005049 IMPLEMENTATION COMPLETION AND RESULTS REPORT Credit No. IDA 53300 ON A CREDIT FROM THE INTERNATIONAL DEVELOPMENT ASSOCIATION IN THE AMOUNT OF SDR 97.6 MILLION (US$ 150.00 MILLION EQUIVALENT) TO THE Socialist Republic of Vietnam FOR THE VN - Central Highlands Poverty Reduction Project (CHPov) September 28, 2020 Social Sustainabilty & Inclusion Global Practice East Asia And Pacific Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization CURRENCY EQUIVALENTS (Exchange Rate Effective December 31, 2019) Currency Unit = Vietnamese Dong (VND) VND 23,172 = US$ 1 US$ 1.38 = SDR 1 FISCAL YEAR January 1 – December 31 Regional Vice President : Victoria Kwakwa Country Director : Carolyn Turk Regional Director : Benoit Bosquet Practice Manager : Susan S. Shen Task Team Leader(s) : Sean Bradley, Nghi Quy Nguyen ICR Main Contributor : Anton Glaeser (FAO), Sean Bradley ABBREVIATIONS AND ACRONYMS CDB Commune Development Board CDD Community Driven Development CHPov Central Highlands Poverty Reduction Project CPF Country Partnership Framework CPO Central Project Coordination Office DDI Dietary Diversity Index DPMU District Project Management Unit EMs Ethnic Minorities ESMF Environmental and Social Management Framework GoV Government of Vietnam EIRR Economic Internal Rate of Return ICR Implementation Completion and Results (Report) IDA International Development Association EFA Economic and Financial Analysis IE Impact Evaluation IEM Indigenous Ethnic Minorities ISM Implementation Support Mission ISR Implementation Status & Results (Report) LEG Livelihood Enhancement Group M&E Monitoring and Evaluation MDRI Mekong Development Research Institute MEM Migrant Ethnic Minorities MIS Management Information System MPI Ministry of Planning and Investment MTIP Medium Term Investment Plan MTR Mid-Term Review NPV Net Present Value NTP National Targeted Program O&M Operation and Maintenance PAD Project Appraisal Document PDO Project Development Objective PPMU Provincial Project Management Unit RF Results Framework SDR Special Drawing Rights SEDP Socio-Economic Development Plan SRI System of Rice Intensification TTL Task Team Leader RF Results Framework RFI Results Framework Indicator ToC Theory of Change WB World Bank i TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 5 A. CONTEXT AT APPRAISAL .........................................................................................................5 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION ................................................................7 II. OUTCOME ...................................................................................................................... 8 A. RELEVANCE OF PDOs ..............................................................................................................8 B. ACHIEVEMENT OF PDOs (EFFICACY) ........................................................................................9 C. EFFICIENCY ........................................................................................................................... 13 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 15 E. OTHER OUTCOMES AND IMPACTS ......................................................................................... 15 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 17 A. KEY FACTORS DURING PREPARATION ................................................................................... 17 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 18 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 19 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 19 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 21 C. BANK PERFORMANCE ........................................................................................................... 24 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 25 V. LESSONS AND RECOMMENDATIONS ............................................................................. 26 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 29 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 49 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 52 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 53 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 66 ANNEX 6. CHPOV IMPACT EVALUATION SUMMARY ............................................................. 68 ANNEX 7. CHPOV END-LINE SURVEY; KEY DATA TABLES ....................................................... 73 ANNEX 8. KEY PROJECT DOCUMENTS ................................................................................... 88 ANNEX 9. LIVELIHOOD ENHANCEMENT GROUP SUCCESS STORIES ........................................ 90 ANNEX 10. CHPOV TIMELINE AND KEY MILESTONES ............................................................ 96 ii The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P128072 VN - Central Highlands Poverty Reduction Project (CHPov) Country Financing Instrument Vietnam Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Socialist Republic of Vietnam Ministry of Planning and Investment Project Development Objective (PDO) Original PDO The proposed project development objective of the Central Highlands Poverty Reduction project is to: enhance living standards by improving livelihood opportunities in Project Communes of upland Districts of the central highlands of Vietnam. Page | 1 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 150,000,000 134,220,275 116,730,890 IDA-53300 Total 150,000,000 134,220,275 116,730,890 Non-World Bank Financing 0 0 0 Borrower/Recipient 9,400,000 7,505,000 7,436,400 Total 9,400,000 7,505,000 7,436,400 Total Project Cost 159,400,000 141,725,275 124,167,290 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 27-Dec-2013 22-Jul-2014 16-Jun-2017 31-Dec-2019 31-Dec-2019 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 24-Oct-2019 108.62 Change in Components and Cost Cancellation of Financing Reallocation between Disbursement Categories KEY RATINGS Outcome Bank Performance M&E Quality Moderately Satisfactory Moderately Satisfactory Modest RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 14-Apr-2014 Satisfactory Satisfactory 0 Page | 2 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) 02 01-Dec-2014 Moderately Satisfactory Moderately Unsatisfactory 0 03 16-Mar-2015 Moderately Satisfactory Moderately Unsatisfactory 1.25 04 29-Sep-2015 Moderately Satisfactory Moderately Unsatisfactory 6.03 05 25-Apr-2016 Moderately Satisfactory Moderately Satisfactory 13.25 06 23-Dec-2016 Moderately Satisfactory Moderately Satisfactory 27.13 07 27-Jun-2017 Moderately Satisfactory Moderately Satisfactory 51.33 08 24-Jan-2018 Moderately Satisfactory Moderately Satisfactory 65.86 09 27-Jun-2018 Moderately Satisfactory Moderately Satisfactory 78.37 10 14-Dec-2018 Moderately Satisfactory Moderately Satisfactory 92.45 11 24-Jun-2019 Moderately Satisfactory Moderately Satisfactory 103.48 12 03-Jan-2020 Moderately Satisfactory Moderately Satisfactory 117.45 SECTORS AND THEMES Sectors Major Sector/Sector (%) Agriculture, Fishing and Forestry 24 Other Agriculture, Fishing and Forestry 24 Public Administration 30 Central Government (Central Agencies) 3 Sub-National Government 27 Social Protection 23 Social Protection 23 Transportation 23 Other Transportation 23 Page | 3 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Private Sector Development 100 Jobs 100 Social Development and Protection 25 Social Inclusion 25 Participation and Civic Engagement 25 Human Development and Gender 46 Health Systems and Policies 20 Health Service Delivery 10 Adolescent Health 5 Child Health 5 Nutrition and Food Security 26 Nutrition 13 Food Security 13 Urban and Rural Development 30 Rural Development 30 Rural Infrastructure and service delivery 30 ADM STAFF Role At Approval At ICR Regional Vice President: Axel van Trotsenburg Victoria Kwakwa Country Director: Victoria Kwakwa Carolyn Turk Director: John A. Roome Benoit Bosquet Practice Manager: Jennifer J. Sara Susan S. Shen Task Team Leader(s): Sean Bradley, Son Thanh Vo Sean Bradley, Nghi Quy Nguyen ICR Contributing Author: Sean Bradley Page | 4 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. In 2012, Vietnam had already achieved remarkable poverty reduction results over the previous two and a half decades transforming into a lower middle-income country. However, overall progress in further reducing poverty had slowed, large gaps between the poorest and better off households were growing, and the gaps between different regions of the country, and urban and rural areas, were persisting. The gap between the rate of urban poverty (at 6%) and rural poverty (at 27%) was particularly striking. In addition, there was a stark difference in the well-being between the Kinh/Hoa majority and the country’s ethnic minorities (EMs). In 2010, EMs accounted for 70% of individuals in the lowest income decile, up from 53% in 2006, while making up less than 15% of the total population. The poverty gap in the two poorest and predominantly EM regions in Vietnam stood at 15.6% for Northern Mountains and 11.5% for Central Highlands, compared with a national average of 5.9%.1 EMs also lagged in terms of assets and access to essential public goods. 2. At the time of project preparation, an estimated 74% of the EMs in the Central Highlands region lived below the Government Statistics Office-World Bank poverty line. The region ranked at the top in terms of income inequality,2 had the worst rates of stunting and wasting among children in the country,3 the lowest rate of primary school enrolment and less than half of EM children were enrolled in lower secondary school. 3. The World Bank has been a long-term partner of the Government of Vietnam in its successful poverty reduction efforts. This included support beginning in 2001 with the first Northern Mountains Poverty Reduction Project (P059936) and continued with the World Bank-financed development policy lending support for the Program to Support the Mountainous Areas and Ethnic Minorities Communes (P107062), commonly referred to as “Program - 135-2”, in 2007. The Government therefore requested World Bank support for a project targeting the poorest districts in the six Central Highlands provinces of Dak Lak, Dak Nong, Gia Lai, Kon Tum, Quang Nam and Quang Ngai. 4. The project was strategically relevant to and fully aligned with the Government of Vietnam’s (GoV) development priorities and strategies as reflected in the Socio-Economic Development Plan (SEDP), 2011-2015, to reduce poverty and to ensure inclusion and equal opportunity for all communes in the country. The project was also well aligned with the World Bank Group’s Vietnam Country Partnership Strategy, FY12-FY164, specifically regarding Outcomes 3.2 (Improved basic infrastructure and public service delivery and access), and 3.1 (Increased Opportunities for the Poor and Household Resilience to Shocks). Theory of Change (Results Chain) 5. The project’s Theory of Change (ToC), as depicted in Figure 1 below, was retrospectively constructed by the Implementation Completion and Results Report (ICR) team based on the project description in the Project Appraisal Document (PAD). 1 Vietnam Household Living Standards Survey 2010 data and calculation, Government Statistics Office. 2 An absolute inequality measure which is the ratio of income level at the 95th percentile over the income level at the 10th percentile reveals that the dispersion of income between the top “rich” and the “poorest” is the highest in the Central Highlands. 3 According to the Vietnam Household Living Standards Survey and Nutrition Surveillance data. 4 Report no. 65200-VN. Page | 5 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Figure 1: CHPov Theory of Change Structure Page | 6 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Project Development Objectives (PDOs) 6. The PDO as stated in the Financing Agreement is “to enhance living standards by improving livelihood opportunities in Project Communes of upland districts of the Central Highlands of Vietnam”. Components 7. The project consisted of four components, as summarized below. 8. Component 1: Village and Commune Infrastructure Development (total estimated cost of US$54.7 million, of which IDA US$52.4 million; actual US$42.38 million, of which IDA US$42.22 million) provided grants for the design, construction, operation and maintenance of small-scale village and commune level infrastructure. 9. Component 2: Sustainable Livelihoods Development (total estimated cost of US$35.2 million, entirely IDA financed; actual US$25.77 million) provided grants to strengthen and diversify income sources, and improve nutrition and food security by enhancing the productive capabilities and dietary intakes of project beneficiaries, and grants for market linkages support for commodities having commercial market potential. 10. Component 3: Connective Infrastructure Development, Capacity Building and Communications (total estimated cost of US$53.0 million, of which IDA US$51.4 million; actual US$40.65 million, of which IDA US$40.13 million) supported the development of selected intra-communal and inter-communal level infrastructure and other investments supporting socio-economic connectivity, capacity building and technical support for project implementation and management, and communication and outreach activities. 11. Component 4: Project Management (total estimated cost of US$16.5 million, of which IDA US$11.0 million; actual US$16.99 million, of which IDA 10.23 million) provided support for project management, monitoring and evaluation. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION Revised PDOs and Outcome Targets 12. The PDO was not revised nor were there any changes to the outcome indicators and targets. Revised PDO Indicators n/a Revised Components 13. There were no changes to the description of the project components. Other Changes 14. The project was restructured in October 2019 to cancel SDR 11.5 million (estimated at US$15.8 million), which caused a reduction in the estimated level of financing for all four project components (see details in Annex 3). This cancellation of credit funds led to a reduction in the value of the eligible expenditure categories by: (a) SDR 4.9 million for Category 1 (Goods (including vehicles), Works, Non-Consulting Services, Consultants’ Services, Training and Workshops, and Incremental Operating Costs under Part 3 and 4 of the Project), and (b) SDR 6.6 million for Category 2 (Grants under Parts 1 and 2 of the Project). The restructuring also led to a revision of disbursements estimates. Page | 7 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Rationale for Changes and their Implication on the original Theory of Change 15. The above cancelation freed up IDA resources to be reallocated to another pending Bank-financed activity5 and facilitated the closure of the project in accordance with the original project design. These changes did not affect the ToC of the project. However, the reduction in financing (combined with a 10% reduction in US dollars available under the credit due to changes in the SDR-US Dollar exchange rate) may have somewhat affected the extent to which the project was able to fully achieve its intended outcomes and objective. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 16. The PDOs remain directly relevant to the World Bank Group’s Vietnam Country Partnership Framework (CPF) for FY 18-20.6 The project contributes directly to CPF Focus Area 1 (Enable inclusive growth and private sector participation) under Objective 2 (Promote private sector and agri-business development) by strengthening market linkages and private sector partnerships through the establishment of the Livelihood Enhancement Groups (LEGs) and Productive Partnerships (see below). The project’s targeting of geographic areas with high ethnic minority populations, and emphasis on the participation of the poor and women are also well aligned with CPF Objective 5 (Broaden economic participation of ethnic minorities, women, and vulnerable groups). 17. The PDO also remains strategically relevant to and aligned with the GoV’s current development priorities and strategies as outlined in its Socio-Economic Development Strategy, 2016-2020, which emphasizes rural development, poverty reduction, and area-based support to ethnic minority regions of the country. These priorities are reflected in several current national programs including the New Rural Areas Program, the National Targeted Program (NTP) for Sustainable Poverty Reduction, and the Master Plan on Socio-economic Development in Ethnic Minority and Mountainous Areas (2021-2030). 18. Based on the extent to which the PDO remains directly relevant to the Bank CPF, as well as GoV’s development priorities and plans, the relevance of the PDO is assessed as HIGH. 5 The cancelled IDA credit from CHPov was used for the Climate Change and Green Growth DPO (P172006). 6 Report no: 111771-VN. Page | 8 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome 19. The PDO: “to enhance living standards by improving livelihood opportunities in Project Communes of upland districts of the Central Highlands of Vietnam”, includes both the higher level objective of enhancing living standards and the means adopted by the project to achieve this, i.e., improving livelihood opportunities. This was to be achieved among an estimated 540,472 beneficiaries (RF#1) in the project’s target district and communes. Ultimately, the project reached an estimated total of 638,536 beneficiaries, mainly as a result of an increase in the population of the project’s target communes. 20. The project was designed to achieve three key intermediate outcomes: (a) improved access to infrastructure and services (including those that would enhance economic activities), (b) improved nutrition and livelihood diversification,7 and (c) enhanced community engagement in public investment activities. These outcomes are directly linked to project activities and outputs (as shown in the ToC figure above), and to several results measures in the project’s results framework (RF). 21. Therefore, the ICR assesses efficacy of the project based on the achievement of or contribution to the following outcomes, as measured by the indicated RF indicators (RFI): • Outcome 1: enhanced living standards, as measured by change in food and non-food consumption of poor households (RFI #3); • Outcome 2: improved access and connectivity, as measured by improvement of poor households in access to services, infrastructure and utilities (RFI#4), and changes in connectivity (RF#11); • Outcome 3: improved nutrition and livelihood diversification, as measured by change in productive and durable assets (RF#8), and dietary diversity (RF#10); and • Outcome 4: enhanced community engagement, as measured by increase in participation of women in decision making on public investments (RFI#7), and percentage of villagers whose identified development priorities are financed (RFI#2). 22. Assessment of the project’s outcomes is based on data and analysis from the baseline and end-line surveys, and the project’s impact evaluation (IE).8 The IE was designed around a robust difference-in-differences evaluation method.9 However, both the end-line survey and the impact evaluation found that the counterfactual “control” communes received relatively equal levels of financing for similar types of investments. This funding was provided as part of the on-going NTPs that used similar criteria as the project to channel poverty-reduction funding to the same geographic areas as covered under the CHPov. The IE concluded that these non-CHPov communes are therefore not a pure control group. As such, and unless otherwise indicated, the percentage changes in the results measures discussed in the remainder of this report are taken from the project’s end-line survey report.10 23. The IE did note, however, that those sub-outcome indicators where a positive impact of the CHPov is detected 7 These two distinct outcomes are combined because they are linked in the project’s implementation strategy where support for n utrition included diversification of household livelihoods activities toward more nutritional foods through gardens, small livestock raising, etc. 8 The CHPov end-line survey was carried out by Mekong Development and Research Institute, a consultant firm hired by the Government, between September and December 2019. The CHPov impact evaluation was prepared by a World Bank team led by Obert Pimhidzai and finalized in May 2020. The impact evaluation used data from the end-line survey and applied accepted difference-in-differences methodology in reaching its conclusions. 9 A difference-in-differences evaluation model is designed to identify causal linkages between project efforts and specified development outcomes by comparing the status of key indicators before and after project implementation in areas that receive project support (“treatment”) and areas that do not receive support (“control”). 10 The Central Highlands Poverty Reduction Project Endline Report; Mekong Development Research Institute, Hanoi, 2020. Page | 9 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) are related to investments the project devoted more resources to as compared with the NTPs, such as commune roads and drinking water systems. A summary of the project’s IE design, results and comparison of project and non- project investments from other sources is presented in Annex 6. 24. Several outcome-level RF indicators (for access, connectivity, dietary diversity and asset accumulation) are composite index measures that aggregate a number of sub-indicators to arrive at a single measure or score. As such, these measures average out change across multiple sub-indicators that make up the index, which can mask important improvements in specific areas of interest. For this reason, the discussion below presents both index averages and individual sub-indicator measures for key areas of change that relate to specific CHPov investments. Annex 7 presents detailed tables of before and after measurements from the end-line survey for all index indicators. Outcome 1: Enhanced living standards 25. The PDO level indicator: “change in food and non-food consumption of poor households”, was meant as the main measure to assess the impact of CHPov on “enhanced living standards” and, in effect, reducing poverty. Overall, the end-line survey found that consumption of poor beneficiaries in the project communes increased by 48.7% between 2014 and 2019,11 which significantly exceeded the target of 10%. Non-poor households saw only an 8.3% increase, suggesting that the poverty targeting efforts of the project also had strong positive results. Food expenditures of poor households increased by 32% and non-food expenditures by 78%. This positive trend is also reflected by the reduction of food-expenditures as share of total household expenditures from around 57% to 50%, which also suggests improving living standards. This change in consumption is linked to an overall increase in household expenditures by 18.3% over the same period. These improvements were seen across all groups of project beneficiaries (including the majority Kinh/Hoa, EMs indigenous to the Central Highlands [IEMs], and ethnic minorities who had migrated to the Central Highlands [MEMs]). As such, the first outcome of the project has been exceeded. Outcome 2: Improved access to public services and infrastructure, and connectivity 26. This outcome was expected to be achieved through project investments under Component 1 (block grants for construction, repair, operation and maintenance of village and intra-commune level infrastructure) and Component 3 (block grants for inter-commune level infrastructure to enhance physical connectivity). This financing led to the construction and/or repair of 1,173 rural roads (666 kms), 122 schools/classrooms, 164 irrigation systems (serving approximately 1,733 ha.), 150 water systems (serving approximately 11,400 households), 30 bridges, and other public buildings and utilities (such as communal houses or in-village electrification). Operation and maintenance (O&M) financing supported critical maintenance needs of an additional 209 rural roads, 24 bridges, culverts or drainage systems, and 108 commune-level schools. 27. The results of these sub-project investments in the construction, repair or maintenance of public infrastructures were measured through: RFI#4—“percent change in access of poor households to services, infrastructure and utilities”, and RFI#11—“percent change of transport connectivity index”. The index measure12 for RFI#4 showed a modest 2.3% overall improvement in access between the start and finish of the project. However, at the sub-indicator level much more significant positive changes in access were noted for several areas relevant to CHPov investments, and more so for the poor and indigenous EMs. For instance, there was an overall reduction of 30% in travel time to schools (with larger savings for IEM groups and poor); an overall decrease of 26.3% in travel time to access commune health centers (and more than 30% decrease for IEM and poor households); a nearly 27% reduction in travel time to the commune people’s committee office; an overall 7.3 percentage point increase in 11 In real terms adjusted to 2019 prices, overall annual per capita consumption of the poor in project areas increased from VND 8.166 million to VND 12.142 million. 12 RFI#4 was constructed based on 12 sub-indicators of household access to four main service domains-- education, health, water supply and agricultural services. Page | 10 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) access to drinking water (and 9.5 percentage points for the poor), and a 10.2 percentage point increase in access to irrigation systems (though lower for poor households). 28. Component 3 financed priority medium-scale (inter-commune) infrastructure that would facilitate “connectivity” between communes, which would in turn support economic integration of EMs and other poor households. Connectivity was measured by RFI#11 (percent change of transport connectivity index), which was a composite measure of household travel times13 to key infrastructure and destinations such as schools, health station, market, and different administrative centers.14 Overall, the index showed a modest 2.8% improvement in connectivity from the beginning to the end of the project, and between project and control communes.15 However, at sub-indicator levels, statistically significant improvements were seen in connectivity to primary schools (by 9.7%), to commune health facilities (by 9.4%) and to local markets (by 6.8%). A separate end-line analysis also showed similar improvements with a reduction of 3 minutes in travel time to local schools (almost 30%) and of 5.4 minutes to local health facilities of 5.4 minutes (26%). 29. The project also identified “social” connectivity as an important pathway for the predominantly EM poor of CHPov to be able to enhance their living standards,16and promoted this element of connectivity through a robust training program that included almost 20,000 beneficiaries,17 local government officials, and other project staff (see detailed discussion of RFI#12 in Annex 1), and wide-spread communications that specifically targeted EM populations to facilitate interactions with local government authorities and markets. While social connectivity was not explicitly analyzed in the end-line survey, two specific data points suggest that the project had important impacts in this area. First, the increase of 16.9 percentage points in community members’ participation in local development affairs, and 31.4 percentage point increase in supervision of local construction activities (with the greatest change being for the poor (31.8 and 36.2 percentage points respectively) and indigenous EMs (19.4 and 39.0 percentage points). Second, the impact evaluation found that households in project communes strengthened their non-farm participation and livelihood diversification strategies, which suggests that project inputs to support external linkages were working. 30. Overall, this outcome is assessed to have been substantially achieved. Outcome 3: Improved nutrition and livelihood diversification 31. The achievement of this outcome was supported by investments under Component 2: Sustainable Livelihood Development. This component’s investments included block grants and training support to LEGs for food security and nutrition, income diversification, and market linkages. The project supported a total of 4,383 LEGs benefitting 63,531 households (approximately 45% of the total target area). The intended outcome of this support was principally measured through two RF indicators: RFI#10 “percent change in dietary diversity of poor households”, and RFI#9 “percent change in productive and durable assets”. Increased dietary diversity was meant as a proxy for improved nutrition, while increases in productive and durable assets were meant as a proxy measure for increased income from diversified livelihood activities and market linkages. 32. Dietary diversity was calculated based on the Dietary Diversity Index (DDI) developed by USAID, which includes consumption information on 12 main food groups.18 Comparing baseline and end-line survey data shows 13 These times (both for motorized and non-motorized connectivity) where then translated into a 5-point scale to compare changes over time. 14 The index included one destination that was unlikely to be affected by project investments—connectivity to provincial centers—which showed a significantly negative result of -9.6%, which in turn negatively affects the overall index score. 15 This overall connectivity measure was better for poor and indigenous EM households, at 4.1% and 3.7% improvement respectively. 16 See: Pro-poor Rural Development Initiatives in Vietnam. April 2013. World Bank Report No. ACS3453. 17 Of whom 54% were women and 69% ethnic minority beneficiaries. 18 The DDI calculated for the project was based on recollection of what was consumed over the previous 12 months, which the end-line report notes can be prone to overestimation of the actual DDI of households. Page | 11 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) that the consumption of all food groups increased, with the exception of roots and tubers. Overall, the project contributed to a 21% increase in the DDI.19 In particular, significant increases were observed for the proportion of households consuming fruits (23%), meat (26%), eggs (28%), and for sugar and honey (27%). These results indicate a shift toward a healthier diet among beneficiaries (with an increase in fruits and higher protein foods, and reduction in carbohydrates). As such, the project was successful in improving nutrition in the project areas. 33. The project’s results framework does not include a specific indicator to measure the diversification of livelihoods opportunities, however several different sources of information reflect on this outcome. First, of the total number of LEGs supported by the project, an estimated 68% were classified as “income diversification” groups. While this group of 2,935 LEGs were not necessarily different livelihood types, the method or scale of their activities supported under the project were oriented toward ensuring more sustainable income from their livelihood activities (often through diversification). In addition, while not strictly a measure of diversification, it is also important to note that 59% of LEG members were women. Second, as evidence of this diversification, the end-line survey shows that the level of poor households raising cattle and poultry increased by 13 and 11 percentage points respectively (an improvement of 24.5% and 14.9% respectively), which was significantly higher than the increase among non-poor households.20 The survey also revealed important shifts in crop cultivation practices (moving from annual to perennial crops), which should support more sustainable income generation.21 The proportion of households producing annual crops dropped by 8 percentage points, while the proportion producing perennial crops jumped by 11 percentage points. The impact evaluation also showed that households’ average incomes from primary wage employment increased by 18.2 percent and total household wage incomes by 7.7 percent. This was a result of workers – mostly men - switching to wage jobs as a primary rather than a secondary occupation. It is possible that the changes in agriculture related work noted above, and in establishing linkages outside of the communes, contributed to this change. Taken together, these findings imply that the project helped to increase household livelihood diversification.22 34. This diversification of livelihoods is also reflected in an increase in household assets, as measured by RFI#9 “percent change in productive and durable assets of households participating in LEGs”. The end-line survey found an overall improvement in asset ownership of 9.3% for the composite measure of nine productive assets, with a higher increase for poor households. At the sub-indicator level, the study showed significant increases of households owning cattle and poultry (as noted above), water pumps (by 15.3%) and motorbikes (by 8.9%). 35. Based on these findings this outcome is assessed to have been substantially achieved. Outcome 4: Enhanced community engagement in public investment decisions 36. The project is built on a Community Driven Development (CDD) platform that promotes the participation of beneficiaries in decisions and actions around the use of development resources. In addition to improving efficiency in the use of such resources,23 this increased participation in development activities will tend to result in investments being more responsive to the development priorities of the community and would increase satisfaction with the support being received. This outcome was supported by investments under Component 3 relating to capacity 19 The dietary diversity score changed from 0.56 at baseline to 0.68 at end-line in project communes, a 21% improvement. In addition, when comparing project and non-project communes, the impact evaluation found that the overall household dietary diversity score improved by 3.8% overall, by 6.5% among the poor, and by 8.3% for female headed households. 20 Pig farming, however, did not show similar positive results at end-line, thought to be due to the African swine fever outbreak. 21 The Central Highlands Poverty Reduction Project Endline Report; Mekong Development Research Institute, Hanoi, 2020. 22 Impact Evaluation of the Central Highlands Poverty Reduction Project, World Bank, May 2020. 23 For a discussion of CDD outcomes see: Radical Decentralization: Does Community-Driven Development Work? Katherine Casey, Annual Review of Economics. 2018, Vol. 10.; or Community Driven Development: Myths and Realities. World Bank Policy Research Working Paper No. 8435. Page | 12 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) building and communications, and the technical assistance support provided to communes by the cadre of commune facilitators financed under Component 4. The achievement of this project outcome was measured through three interrelated RF indicators: RFI#7 “increased rates of participation of women”, (and of community members in general); RFI#2 “rate of responsiveness of development financing to expressed community priorities”; and RFI#6 “general satisfaction with support being provided by local government”. 37. Participation. Participation in local development activities in the project communes increased across all groups. The end-line survey found a 16 percentage point increase in participation in public meetings (representing an almost 30% improvement) and a more than 30 percentage point increase in supervision of local works (albeit from a very low base of 4.5%). In addition, the end-line survey found that the proportion of women taking part in commune/village meetings and expressing their opinions (a more meaningful measure of “participation”) on project selection increased by 5 and 10 percentage points, respectively (RFI#7). Notably, women from female-headed households seemed to be more active in joining local activities than other women. The survey also found significant increases in participation levels for IEM women, improving by more than 12 percentage points from a low base of 23.3% participation (or more than a 50% improvement). 38. Responsiveness. The extent to which the project responded to the expressed priorities of the community was measured through RFI#2 “proportion of poor villagers whose identified development priorities are satisfied”. The underlying logic of this measure was to test how well the participatory and transparent process of identifying investments was working and be a measure of improved living standards for individual households in that their priority public investment concerns have been responded to. Comparing preferences from baseline data to implementation of sub-projects in each commune from the project’s administrative data shows a high proportion (87%) of households that had at least one development priority addressed, while 25% of households had all of their top development priorities financed by the project.24 39. Satisfaction (RFI#6). The end-line survey reports that overall household satisfaction with Government support in the project area in general (i.e., including support other than under the project) increased by 13.8 percentage points (an improvement of almost 32%). More specifically, satisfaction increased across five domains: the selection of infrastructure projects (10.7 percentage point increase), the quality of those projects (6 percentage point increase), the quality of local roads (4.4 percentage points), the quality of local irrigation systems (32.4 percentage points), and with agricultural extension services (15.6 percentage points). 40. Based on these findings the achievement of this outcome is substantially achieved. 41. Efficacy Rating. Based on the assessed achievement of the four outcomes above, efficacy of the project is rated SUBSTANTIAL. Justification of Overall Efficacy Rating 42. Given that the four outcomes linked to the achievement of the PDO are rated as substantially achieved or exceeded, and that the end-line survey data used to substantiate these conclusions are robust, a substantial efficacy rating is justified. C. EFFICIENCY Assessment of Efficiency and Rating 43. Project efficiency was assessed through an analysis of: (i) the project’s economic rate of return computed through an economic and financial analysis (EFA), as compared to the appraisal estimates; and (ii) administrative and 24The stated priorities that were compared against actual project investments were those indicated by baseline survey participants, and not those identified during annual planning cycles. As such, the measure suggests minimal elite capture of the project resources. Page | 13 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) operational efficiency. Additional reviews of per beneficiary and per unit of output costs were also carried out. The detailed analysis of efficiency is contained in Annex 4. 44. The project’s economic benefits are derived from various elements of its investments. Most important to this analysis are the results relating to the project’s livelihoods activities (Component 2) that led to greater production and household income, as suggested by the increased levels of consumption in the project areas. Second are the effects of public infrastructure that directly contribute to production (irrigation) or access to markets for the sale of agriculture products (roads). Lastly, are the project’s investments that indirectly contribute to beneficiary welfare through improved health (water, nutrition education, and roads that improve access to health clinics), increased participation of women, and capacity to engage with local authorities and private sector markets. However, some benefits cannot be quantified and only the economic benefits derived from the project’s livelihood activities and the road and irrigation investments were quantified in the ex-post EFA for CHPov (see annex 4, para 6). As such, the EFA estimates for the project should be considered conservative. 45. The project’s EFA at completion finds an economic Net Present Value of VND 1,485 billion, equivalent to US$ 63.9 million, and an Economic Internal Rate of Return (EIRR) of 29%, showing positive and substantial returns to investment (see Figure 2 below). The EFA analysis found that most project benefits are derived from the community infrastructure investments, which accounted for a larger share of the budget (approximately 60%). The project EIRR at completion was higher than the ex-ante project design estimate of 15%. 46. A sensitivity analysis to assess the effects of lower rates of production or sales of goods, and for higher prices of goods sold at market, found that in all scenarios the EIRR was above the discount rate of 5%.25 The potential impacts of COVID-19 were not factored into the analysis. However, COVID-19 has been considered in the discussion of risk to outcomes. 47. Operational and Administrative Efficiency. The project was completed by the original closing date of December 31, 2019. Overall, project management costs were approximately 13.5% of total project costs at project close, which, while above the original projections of approximately 10%, are considered reasonable for this type of project.26 These higher costs reflect the challenges of working through a fully decentralized structure (with implementing units in all 26 districts) in one of the most isolated regions of the country. This percentage is approximately at the mid-point of a range of project management costs for a group of large-scale national CDD projects (ranging from 5% to 21%) worldwide, suggesting reasonable administrative efficiency. Moreover, CHPov is above average in allocating more than 75% of total project costs for direct sub-project investments, as compared with this same group of CDD programs.27 48. Procurement performance improved significantly over time with delays falling from 84% in the first year of implementation to 1.3% by the close of the project.28 The project encouraged wider use of competitive bidding for infrastructure contracts valued below the Government’s allowed threshold for single-source selection, resulting in cost savings. Procurement practices of Government provincial and district staff were also enhanced, which should help further public financial management efficiencies. 25 The economic discount rate of 5 percent was used given the very low 10-year bond rates at the time of the ICR (around 4 percent). While lower than the 6 percent recommended in the Bank guidelines on estimating discount rates (Discounting Costs and Benefits in Economic Analysis of World Bank Projects, OPSPQ, 2016), given an estimated EIRR of 29% the project’s NPV remains positive even at 6 percent. 26 The increase in percentage management costs is directly linked to the reduction in value of the overall project investments. While other project component costs were reduced, project management costs that tend to be front-loaded and contain a high degree of fixed costs were not proportionally reduced. 27 Based on analyses done for World Bank Report No: ACS13685 Islamic Republic of Afghanistan; Strategic Directions for the National Solidarity Program, that looked at six large-scale and long-running national CDD programs. 28 Procurement delay is measured as the ratio of total contracts signed by year end to total contracts planned for the year. Page | 14 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) 49. A project restructuring in the final year of implementation led to the cancellation of SDR11.5 million, reducing overall IDA financing to SDR86.1 million (refer to Section I.B. “Significant Changes During Implementation” for further discussion of this cancellation). By the project’s end-disbursement date, SDR83.8 million of expenses had been documented, or approximately 97.34% of the revised credit resources (see Annex 3). 50. The operational efficiency of the project was also supported by a reasonably stable task management with the original lead Task Team Leader (TTL) remaining through to the end of the project, and co-TTL changing only once after the project Mid-Term Review (MTR). Fiduciary and safeguard staff were also very stable. However, technical support for the livelihoods component and Monitoring and Evaluation (M&E) was less consistent, resulting in certain shortfalls in guidance and oversight see M&E section). Figure 2: Summary CHPov EFA at Completion 51. Assessment of Efficiency and Rating. Based on the project’s EFA at completion and the operational and administrative aspects of project implementation, project efficiency is rated as SUBSTANTIAL. D. JUSTIFICATION OF OVERALL OUTCOME RATING 52. Given the assessments of “relevance of PDO” rated as high, “efficacy” as substantial (with moderate shortcomings), and “efficiency” as substantial, overall project is rated MODERATELY SATISFACTORY. A rating of “MS” (rather than “Satisfactory”) is warranted due to: (a) the lack of strong counter-factual evidence from the project’s impact evaluation due to the control communes having received similar levels of investments from other National Targeted Programs, and (b) more modest composite results for two key outcome measures for connectivity and productive assets. E. OTHER OUTCOMES AND IMPACTS Gender 53. The project demonstrated a commitment to increased participation and inclusion of women and required at least 30% female participants in the participatory village planning meetings where priorities were established. The Page | 15 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) rate of participation of women in LEGs and in training activities was notably higher, at 59% and 54% respectively,29 which shows the project’s successful commitment to focus on women. The end-line survey noted a significant increase in women’s participation in commune/village meetings and in expressing their opinions about project selection. There was an overall increase of 10 percentage points above baseline for all women in expressing their opinions, and 12.5 percentage points for indigenous EM women (more than 50% improvement over the base-line value). Ethnic Minorities 54. The project explicitly applied a targeting formula that took into consideration both poverty and the concentration of ethnic minority populations. The project was also built on a platform of community driven development that incorporates the principles of free, prior and informed consultations and consent for all related project investments to ensure that development activities meet the priorities of local communities. The project also applied quotas, especially for LEG participation, to ensure on average at least 50% EM household participation. The project exceeded this commitment and achieved a participation rate of EMs in LEGs of around 85%. The participation of EMs in project related training was also higher than originally targeted, with 77% EM participation in agricultural techniques training, 67% participation in nutrition training, and 52% participation in LEG management training. Limited knowledge of the Vietnamese language by EMs posed a challenge to full participation in planning meetings as well as in training sessions, but the project compensated by hiring community facilitators from the area or using interpreters. The mid-term beneficiary assessment found no violations regarding the application of OP 4.12 on Indigenous Peoples. Institutional Strengthening 55. The project contributed to institutional strengthening and organizational development of various levels of local government (provinces, districts and communes). At the commune level, capacity building measures enabled the Commune Development Boards (CDB) to fulfil their role as investment owners, and the Commune Supervisory Board to oversee the implementation of the selected infrastructure activities. Targeted skills training and exposure visits, especially to village leaders and board members, enabled communes to take ownership of the development activities and to actively participate in decision making, planning, and implementation. Individual members will, for the most part, continue as members of the Commune People’s Committee and the capacities transferred through the project would be retained in these Government administrative structures. The capacity of the Provincial and District Departments of Planning and Investment was enhanced through extensive training on fiduciary, environmental and social safeguards procedures. These staff were also exposed to elements of project management, communications, and community outreach for EM communities. A cadre of 130 community facilitators were exposed to the development challenges of the poorer and culturally diverse areas of the country and were trained in basic project management and participatory skills; many will take these skills with them to subsequent employment. 56. Approximately 63,500 farmers (1 per household) were organized into more than 4,383 LEGs to facilitate training in production techniques relevant to their selected agricultural activities and collaboration between the members for knowledge exchange, joint input supply and/or marketing. While these structures were effective for the delivery of project support, given strongly held negative views of cooperatives in rural Vietnam, their sustainability remains a challenge. While an estimated 50% of LEGs has ceased to function as groups at project close, the individual group members remained active in areas where the project had assisted them. Where LEGs have continued to function, clear benefits from collective action (in sourcing and transporting inputs or marketing production) have tended to be very strong. 29 Based on data collected during the ICR mission. Page | 16 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Mobilizing Private Sector Financing 57. Overall, the limited success with establishing market-linkage LEGs under the project (with only 17 market- linkage LEGs operating at the end of the project) also limited the amount of private sector financing mobilized under the CHPov. The limited numbers of market-linkage groups formed reflects the lack of beneficiary familiarity with, access to and linkages with predominantly Kinh-dominated, market opportunities; challenging agro-climatic environments; and difficult logistical connections with suppliers and markets. The low levels of private sector engagement also suggest an unfamiliarity and therefore aversion to implied risks associated with engaging in these communities and with clients less well known. Poverty Reduction and Shared Prosperity 58. The CHPov was by design as a poverty reduction project targeting a geographic area and population groups that were specifically identified based on poverty rates. The evidence of substantial increase in the annual per capita rates of consumption (18.3% overall, and 48.7% for the poor) as measured by the end-line survey is strong support for the project’s poverty reduction effects. The end-line survey also found that the poor enjoyed above average improvements in: access to safe drinking water, travel time to schools and commune health facilities, ownership of important productive assets such as cattle, poultry and motorcycles, and general satisfaction with the project. Other Unintended Outcomes and Impacts 59. No other outcomes / impacts have been observed. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 60. Project objectives and design were guided by the Government’s poverty reduction efforts, which at the time were pursued through several targeted area programs. Lessons from previous and ongoing World Bank supported poverty reduction projects, specifically the Northern Mountain Poverty Reduction Project Phases I and II, were also considered. The project was designed along the lines of a classic demand-driven, participatory CDD project focusing on village, communal and district infrastructure, as well as livelihood development. Given the poverty context in the country at the time, the targeting of the communes in Central Highlands with the highest percentage of ethnic minorities was especially relevant. 61. The project’s results framework included a mix of output, process and outcome indicators for the three areas of project support (infrastructure, livelihoods, and participation). The PAD did not clearly enunciate the linkages between components, intermediate outcomes, and the PDO. In addition, the term “enhance living standards” was left open to interpretation and the suggested means of “through increased livelihood opportunities” was overly narrow in focus given other relevant project investments and activities. However, considerable resources were dedicated to ensuring that a robust evaluation and monitoring system was in place (see section on M&E). 62. The project drew heavily from the Northern Mountains project in terms of operational procedures, Government staff exchanges, and overlapping Bank teams. It also took advantage of the existing participatory district and commune-level planning under the New Rural Areas strategy to identify the first 18-months of project infrastructure investments, thus enhancing project implementation readiness. Planning and implementation in the following years followed a more typical CDD approach. 63. Critical risks were adequately assessed during project preparation and most risk mitigation measures that Page | 17 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) could be addressed by the project were efficiently implemented, with the exception of a strong M&E/ management information system (MIS) and a more explicit and, perhaps, a more strategic collaboration with the Department of Agriculture and Rural Development as explained under the “Quality at Entry” section. B. KEY FACTORS DURING IMPLEMENTATION 64. Regular, at least biannual, implementation support missions (including a mid-term review) were conducted and detailed Aide Memoires containing useful and relevant recommendations were prepared. Overall, more than 20 supervision and specific technical support missions were undertaken. The project benefitted from low turnover of Task Team Leaders (only three co-TTLs over the project period), as well as fiduciary staff, providing continuity in guidance and supervision. The overall implementation progress of the project was initially rated as Moderately Unsatisfactory for the first 1.5 years of implementation due to start-up delays, and Moderately Satisfactory thereafter through to the end of the project. The DO rating was rated Moderately Satisfactory for the duration of the project as the key outcome measures for consumption, access, dietary diversity, assets and connectivity were not available until the end-line survey was completed and the DO could be more accurately estimated. 65. Project implementation performance was affected by a slow start in terms of setting up the needed project infrastructure and recruitment of necessary staff, especially at provincial and district level. Delays in recruiting provincial and district-level livelihoods staff as well as the Technical Assistance Team initially resulted in inadequate technical guidance for already started livelihood activities. These issues were however successfully resolved during the second year of implementation. 66. Several issues regarding budgeting, finance and disbursements also affected implementation. First, the onset of the Medium-Term Investment Plan (MTIP) of 2016-2020 created discrepancies between project budgets (based on the overall IDA and Government financing envelope for the project) and annual allocations (the funds that central and provincial implementing units were allowed based on MTIP parameters). Second, these discrepancies slowed annual approval of implementation plans and budgets in turn slowing down annual implementation. They also generally led to a reduction in the value of approved annual budgets for the project’s implementing units.30 Third, the processing of withdrawal applications by the Ministry of Finance was at times significantly delayed (in some cases by more than one month) leading to cash-flow constraints of implementation units. 67. The project was restructured in 2019 to cancel SDR11.5 million (also see discussion of cancellation in Section I.B. “Significant Changes During Implementation” and Section II.C: “Efficiency”), which led to a relatively proportional reduction in the value of all three technical components of the project (see Annex 3). The task team advised Government against the cancellation and recommended a six-month project closing date extension to allow the full and efficient utilization of remaining project funds. The cancellation, coupled with the loss of the US Dollar value of the original credit due to SDR-USD exchange rate changes, is likely to have reduced somewhat the levels of potential project achievement. 68. Roads constructed under Component 1 were designed per the master plan for road networks in the communes/villages, generally with a carrying capacity of around 3 tons for light trucks. However, much heavier trucks are using these inter-village roads and causing damage. Local government authorities have had mixed success in addressing this, and this could reduce the productive life of these roads. Factors outside the control of government and/or implementing entities 69. In 2019 the project area experienced two significant natural disaster events that likely had negative effects 30The MTIP limits on ODA financing also contributed to the request by Government to cancel SDR 11.5 million in the final year of project implementation as provinces were unable to utilize their full allocation of project funding. Page | 18 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) on some measures of project-related change during the end-line survey and may have had negative impacts on project results. First, a severe outbreak of African swine fever affected the project areas and the LEGs who were involved in pig rearing. The project responded quickly to the outbreak to agree on revised LEG proposals with the affected groups, which nonetheless postponed support during the period when end-line data was being gathered. Second, in August 2019, toward the very end of the project, several provinces of Vietnam were affected by floods, and four project provinces (Dak Lak, Dac Nong, Gia Lai, and Kon Tum) were among the nine most affected provinces in the country. Losses -- of homes, livelihoods, and lives -- were widespread.31 These floods affected the project’s ability to fully implement planned infrastructure and slowed down LEG operations in the final year. They also potentially affected the measures of outcome. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 70. The project’s M&E system was guided by the project’s RF, which consisted of 12 indicators.32 The RF had a reasonable balance of output, process and outcome measures for the three project components and PDO. The PAD outlined how and at what frequency data on each results indicator would be collected, either through annual progress reports, at mid-term review, or through the final impact evaluation, and definitions or measurements of indicators were partly detailed. The key performance indicators under the PDO were: (i) total number of beneficiaries; (ii) proportion of poor villagers whose identified development priorities are satisfied; and (iii) percentage change in food and non-food consumption of poor households. The first two PDO indicators were at the time standard measures used for CDD programs globally. The third indicator was considered critical in measuring the achievement of “enhanced living standards” that were identified in the PDO. 71. The project’s evaluation strategy was built around a robust difference in differences impact evaluation design, which was developed by a Bank team drawn from the Poverty Global Practice. The CHPov impact evaluation was part of the regional commitment made (under IDA-16) to apply rigorous evaluation methods that include both baseline measures and plausible counterfactuals to determine the ultimate results of the project. This team prepared the detailed methodology for the project’s evaluation and worked with Government on the design of the project’s baseline survey. Seven of the project’s results indicators were drawn from the IE, with the remaining output indicators to be captured through the project’s MIS. 72. The overall results framework would have benefitted from additional analysis to clarify the relationship between components, intended outcomes, and the ultimate objective of the project. The indicators could also have benefitted from more clarity in formulation and estimated levels of achievement. Specifically, the PDO could have been more precise, or at least the term “enhanced living standards” defined for the purposes of measuring project results. Component outcomes were not explicitly expressed in the PAD nor their linkage to the PDO, both of which had to be derived from the description of the project components as part of the ICR assessment. In addition, it can be argued that too great an emphasis was placed on complex (composite index) measures of outcome (e.g., for assets, access and connectivity) and a long-term impact evaluation to determine the project’s results. The RF could 31International Federation of Red Cross and Red Crescent Societies, 2019. Emergency Plan of Action Operation Update Vietnam: Floods. 32An additional operational indicator (RF#13: percentage delays in procurement activities as per approved annual procurement plan) was added early on during implementation without a formal restructuring of the project. Page | 19 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) have given more consideration to additional intermediate outcome measures relating to livelihoods and priority infrastructure investments either through the project MIS or periodic specialized studies.33 73. The project relied on a web-based MIS platform that was designed to use mobile-data connectivity and smart- phone/tablet capabilities to facilitate real-time data collection led by the commune facilitators. This system was designed to simplify and accelerate basic project progress data collection and reporting (including for the remaining six RF output indicators) on a regular basis. The overall system, called “community collect”, was based on the prototype system developed by the World Bank Social Development Global Practice, which also committed to provide technical support in system installation. General aspects of the M&E system and the MIS were outlined in the PAD, while the procedural details and reporting formats were in the Project Implementation Manual. The project’s MIS was designed to work from the commune level up to the Central Project Coordination Office (CPO) in Hanoi, and specific staff at all levels had designated responsibilities for this purpose. M&E Implementation 74. The design and execution of the project’s impact evaluation was out-sourced and the baseline survey was successfully carried out in early 2015 prior to significant field-level implementation. Baseline values were established for six of the seven outcome measures in the RF by 2016.34 However, some confusion persisted on the definition and means of measuring some of the RF indicators until the time of the end-line survey, when the assessment methodology and interpretation of some indicators were sharpened to produce more meaningful results. 75. Because the IE was outsourced, the task team and government placed more attention on the design and operationalization of the MIS to monitor physical progress, which nonetheless proved challenging. After some start- up delays and glitches, there were initial signs that the system was functioning reasonably well with web-based sub- project data and photos available, and regular standardized progress reports prepared. However, continued technical problems with data-collection forms and a failed launch of the program’s version 2.0 platform, led the Government to suspend further efforts to fully operationalize the system in 2018. The required key output data were made available through parallel excel-based systems at provincial and central levels. 76. Optimal functioning of the project’s M&E system was also impeded by a lack of qualified staff. Several project staff and consultants at different implementation levels also had received limited M&E training.35 The MTR reviewed the status of M&E and recommended several additional intermediate outcome measures. However, Ministry of Planning and Investment (MPI) project management resisted the recommendation for further data collection as it felt that this would further overburden the community facilitators. MPI was also reluctant to add additional studies under the project given that the State Auditor had questioned the conduct of special studies (as part of the MTR) during the project’s annual audit. 77. In addition to the baseline survey, the project conducted the end-line survey cum impact evaluation in 2019. The project also undertook mid-term review studies, including a beneficiary assessment and technical and economic assessment of infrastructure investments (albeit of uneven quality) in 2017. Timely quarterly and annual progress reports were produced, as well as a Rapid Assessment of Livelihood Enhancement Group Outcomes (2019), and several technical reports on infrastructure and livelihood activities, including some success stories. Existing data allowed MPI to finalize its own Project Completion Report (PCR) in May 2020. 78. The results of the impact evaluation were confounded by the fact that the non-project supported communes received roughly equal levels of financial support for similar types of investments that the CHPov was also providing 33 WB, June 2017. Aide Memoire Mid Term review, p.34. 34 With the exception of RFI#2 on development priorities satisfied. 35 WB, June 2017. Aide Memoire Mid Term review, p.34. Page | 20 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) to project communes, thus rendering the counter-factual, “without project”, role of these communes moot. This possibility could have been checked in advance of the end-line survey and possible adjustments made to the approach used for the ultimate evaluation of the project. This was not done and may have been a missed opportunity to improve understanding of the direct impacts of the project. M&E Utilization 79. Initial data from the baseline that highlighted drinking water and nutrition issues in project communes led to enhanced efforts to support these types of investments under the project. The project’s progress reports and monitoring data were periodically confirmed by the Bank through field assessments. The exact extent to which the monitoring systems were used to guide management decision by District Project Management Units (DPMU), Provincial Project Management Units (PPMU) and CPO is however not known. The end-line survey and related impact evaluation report are currently being used to better understand the outcomes of the project and should readily help MPI, the Committee for Ethnic Minorities Affairs and other key stakeholders guide further NTPs that focus on EM poverty. Justification of Overall Rating of Quality of M&E 80. The M&E system suffered from design and implementation challenges that had a negative effect on its utility. Balancing these with the robust baseline and end-line studies carried out, the M&E system is rated as MODEST. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Environmental Safeguards Compliance 81. The project was classified as an Environmental Category “B – Partial Assessment” and three environmental safeguard policies, namely Environmental Assessment (OP 4.01), Forests (OP 4.36) and Pest Management (OP 4.09), were triggered. MPI and the project provinces prepared and disclosed (on September 16, 2013) an Environmental and Social Management Framework (ESMF)36 as part of the project’s feasibility study. In accordance with the country’s environmental regulations and the World Bank OP/BP 4.01, the ESMF served to guide the project in screening, assessing and mitigating project environmental and social impacts. The ESMF also contained a “Negative List” for identifying ineligible subproject proposals. Overall, the risk of potential negative impacts was considered to be minor, localized, and temporary in nature and reasonably easy to mitigate in the context of the project. 82. The government recruited or appointed adequate staff at provincial and district levels to oversee environmental (and social) safeguard matters and provided sufficient training for commune representatives and beneficiaries. Central office technical staff, in coordination with district/commune agriculture officers, provided guidance on Good Agriculture Practices and waste management to LEGs involved in livelihood sub-projects. Guidelines on construction of animal pens, techniques of crop cultivation and livestock raising, proper use of fertilizer and pesticides and hands-on training on treatment of animal waste were provided under Component 2. Local authorities also made efforts to provide storage tanks for empty pesticide containers. Farmers were made aware of proper handling and disposal of agro-chemicals and their containers to avoid health and environmental hazards. The central project office in coordination with provincial management units (and local communities) regularly carried out monitoring and supervision of construction contractors’ compliance with agreed mitigation measures and periodically reported to the Bank throughout implementation. No major incidents and accidents happened during implementation and there were no complaints from beneficiaries about environmental impacts associated with construction activities. 36WB, 2013. Central Highlands Poverty Reduction Project (P128072), Environmental and Social Management Framework, (Draft for disclosure). Page | 21 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) 83. Bank missions found the project to be generally in compliance with triggered policies. The overall safeguard rating remained at moderately satisfactory for the project’s duration, but implementation under individual policies gradually improved to satisfactory as the knowledge of project teams improved. Field studies by the Bank team’s engineer reported that a number of infrastructure projects had incomplete documentation files, including environmental reports.37 Other implementation support missions found that all subprojects included the required Environmental Codes of Conduct (ECOPs) and approved Environmental Protection Plan, which were incorporated in bidding documents. However, it was also noted that works contracts did not reference these documents, therefore making it difficult to hold contractors accountable for implementing the intended impact mitigation measures. This was noted by the Bank and rectified later on by the respective project implementing units.38 Environmental, social health and safety requirements were included in project documents, but they were not regularly monitored and reported on as part of construction supervision.39 The project encountered initial challenges adhering to the agreed Environmental Codes of Conduct and Good Agriculture Practices for rice and maize subprojects, that led to an initial MS ratings for OP 4.09, Pest Management.40 Social Safeguards Compliance 84. Bank supervision missions found the project to generally be in compliance with relevant social safeguard policies. The project triggered two social safeguard policies: OP 4.10 Indigenous Peoples and OP 4.12 Involuntary Resettlement. Social safeguard instruments, including a Resettlement Policy Framework and the Social Assessment were prepared, reviewed, approved and disclosed (on August 12, 2013) per the policy requirements. The project was considered responsive to the requirements of OP 4.10 and the PAD itself was considered an Indigenous Peoples Plan or Framework. Project implementation was responsive to the requirements of the operational policy, as demonstrated by the high percentage of ethnic minorities participating in the overall project and in livelihood groups. The project’s overall estimation of EM participation is 85% of LEG members and 69% of trainees. Key ethnic minority languages were used in both village meetings and for various audio-visual communication materials 85. Minor land acquisition impacts were generally addressed through voluntary donation criteria and mechanisms set forth in the project’s resettlement policy framework and the ESMF.41 Early project missions noted shortfalls in the proper documentation of land acquisition, which was the cause of the only consistent MS rating under safeguards.42 Towards the second half of project implementation units at district level gradually improved their efforts in order to comply with the mentioned provisions. Communities were better informed on compensation policies, and a proper documentation system for land donation practices was put in place. As of December 2019, the project reported that more than 1,200 households in the project areas had donated approximately 106,700 square meters of affected lands (equal to 2.16% of the households’ total land holdings). No physical relocation was recorded and only one complaint relating to land donation was received by the project (in Kon Tum province), which was resolved to the satisfaction of the affected person. 86. The establishment of a functioning grievance redress system through various channels (mobile text messages, voice messages, project website, etc.) faced difficulties throughout project implementation related to problems with the MIS. Fewer than 100 comments or complaints were filed through this system and only one was addressed in this 37 BKG Group, 2017. Quality inspection and evaluation of investment extent and efficiency for completed infrastructure works of Central Highlands Poverty Reduction Project. 38 WB, June 2017 and March 2018. Aide Memoires. 39 WB, August 2019. Aide Memoire. 40 WB, January 2015, Aide Memoire, p.24. 41 Compensation in different forms was generally arranged by local authorities, contractors or community members themselves. 42 WB, November 2016 and March 2018. Aide Memoires. Page | 22 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) manner. All other grievance cases were reportedly settled either through project staff, e.g., Community Facilitators, or through personal contacts with commune and district authorities. Despite having the official project hotline widely disseminated on most project communications materials at commune levels, target beneficiaries and community members were reported to doubt the efficacy of this official mechanism or fear that by complaining in this manner they would risk the continued support under the project. Financial Management 87. The project complied with all financial covenants regarding the submission of annual audited financial management reports and quarterly Interim Financial Reports, which were submitted on time and were of acceptable quality. Qualified project consultants were in place at central, provincial and district levels to manage project finances, which were overseen by relevant Government staff. The World Bank and central government provided training in project financial management to project staff to enable them to manage project finances. The project’s accounting and reporting functions, internal controls and the disbursement were maintained adequately. 88. As previously noted, the onset of the MTIP of 2016-2020 created discrepancies between the project operating budget, which was based on the overall IDA and Government financing envelope for the project, and the authorized allocations, which were the funds that central and provincial implementing units were allowed based on MTIP parameters. In general, the authorized allocations were less than the operating budget, which limited the extent to which the different management units could implement project activities.43 The processing of withdrawal applications by the Ministry of Finance took on average three weeks, but was at times much more significantly delayed (by 1-2 months) leading to cash-flow constraints for implementation units. 89. Financial management under the project was generally satisfactory but was rated moderately satisfactory during the course of the project implementation due to MTIP and withdrawal application processing issues. There were no significant audit findings. By relying heavily on project consultants for the day-to-day management of the project’s finances, the Government, however, missed an opportunity to build financial management capacity within MPI, provincial and district governments. Procurement 90. In accordance with the requirements of the financing agreement, the government recruited and installed relevant staff at central, provincial and district levels for overseeing procurement under the project. These staff were guided by a detailed Project Implementation Manual and received relevant training from central consultants, with support from Bank staff, to help ensure compliance with Bank procurement guidelines and GoV regulations. 91. The Project Implementing Agencies, including the CPO, PPMUs, DPMUs and CDBs, made considerable efforts to improve their performance to address several issues that contributed to procurement and project implementation delays. These delays related to: (a) slow annual budget allocation and preparation of the procurement plan; (b) slow recruitment of design consultants for works; (c) lengthy internal processes for review and approval of designs; (d) delays at the stage of solicitating quotations and bid evaluations; and (e) delays in execution of works. 92. Contract management was initially sub-optimal, especially for the construction of infrastructure at the community level. Bank mission aide memoires highlighted several relevant issues, including: (a) contractors not respecting warranty periods; (b) lack of minutes of checking and approval of field laboratories of the contractors; (c) lack of acceptance minutes for input materials; (d) lack of material transportation diagrams; and (e) lack of minutes of handing over of the finished construction items.44 Given these issues, the overall rating of procurement remained 43This fact contributed to the cancellation of project financing. 44BKG Group, 2017. Quality inspection and evaluation of investment extent and efficiency for completed infrastructure works of Central Highlands Poverty Reduction Project. Page | 23 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) moderately satisfactory for the duration of the project. Implementation support missions noted gradual improvements in many areas, including: (a) more effective use of procurement consultants to assist in procurement activities; (b) more timely recruitment of design consultants for works; (c) streamlining of internal approval processes; and (d) improved procurement and contract management. As noted under the discussion of RF#13, annual procurement delays45 fell from 84% in 2014 to 1.3% in 2019. C. BANK PERFORMANCE Quality at Entry 93. The project preparation team was adequately staffed and conducted four full preparation missions up to appraisal. The team drew relevant lessons from several global and in-country operations. The team carried out a quality enhancement review that drew on experiences of lead rural development and livelihoods support specialists from within the Bank. 94. The team coordinated closely with and engaged staff from the successful Northern Mountains Poverty Reduction Projects, as well as the Agriculture Competitiveness Project, in designing the project. Key lessons drawn from these experiences included involving Provincial and District Peoples Committees directly to monitor implementation and ensure participation of other departments; structuring and support of LEGs; and parameters and procedures for sub-project selection and implementation. 95. The design of Components 1 and 2 was based on well-established CDD/livelihood models and in-country experiences that had proven effective. Component 3, for connective infrastructure, capacity building and communication, was added in response to preparation analysis that showed EM connectivity to markets spanned both physical and social/cultural issues. 96. The project took advantage of the participatory district and commune-level planning that had taken place under the New Rural Areas strategy to identify the first 18-months of project infrastructure investments. In this way, the project was thus able to quickly start implementation as first year investment plans needed only to be validated and detailed, rather than undergoing a lengthier process of social analysis and prioritization. Planning and implementation in the following years followed a more typical CDD approach. 97. The project was closely aligned with the World Bank Group’s Vietnam Country Partnership Strategy, FY12- 1646. It was also consistent with the Government’s Socio-Economic Development Strategy 2011-2015, which emphasised addressing EM poverty, as well as the recent launch of the New Rural Areas strategy. 98. Fiduciary and safeguard considerations and requirements were given appropriate attention, as were key risks. The hybrid fiduciary arrangements that mesh Bank and GoV procedures required additional technical support and capacity building for provincial and district units, which were built into the project. Specialized social and environmental safeguard arrangements were outlined in the project’s ESMF. Risk assessment, as detailed in the PAD, anticipated the most relevant risks. However, the customized MIS to help mitigate risks to delivery and monitoring did not fully materialize. 99. As noted, (see Section VI.A. “Quality of Monitoring and Evaluation (M&E)”) the project’s M&E system encountered some challenges, which task team efforts were not able to fully resolve. Importantly, the impact evaluation design was overly complex for Government to adequately manage and there was an overreliance on a prototype MIS that ultimately proved overly optimistic. The task team could have considered more modest 45 Annual procurement delay is measured as the percentage of works contracted by year’s end versus the number of works activities planned for that year. 46 World Bank Report no. 65200-VN. Page | 24 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) approaches to the M&E functions and targets given the project’s challenging operational context and the range of support provided under the project. Quality of Supervision 100. The Bank provided a range of appropriate technical and operational skills through its regular implementation support missions over the course of the project. More than 20 supervision and specific technical support missions were carried out during project implementation, including more frequent missions in the early and late years of the project. The Bank team carried out the planned mid-term review in June 2017 and undertook several discrete technical support missions on infrastructure quality, nutrition support, review of safeguards, and the MIS in response to specific issues or concerns identified during regular supervision missions. The task team also effectively engaged with the Poverty Global Practice on the impact evaluation design and implementation, with the Health team in support of nutrition activities, and with the Food and Agriculture Organization (FAO) to secure specialized support for the livelihood component. Aide memoires, management letters and ISRs were appropriately candid and highlighted issues requiring Government and Bank management attention. Ratings appear to have been realistic based on progress and actions taken by Government. The task team was especially stable with the original co-TTL and all core Bank safeguard and fiduciary staff remaining with the project throughout. 101. Supervision support did, however, suffer from some key shortcomings. There was turnover in the support provided to the livelihood component from preparation through implementation that adversely affected guidance provided to Government. The market linkages sub-component presented challenges that the task team was also not able to fully resolve (as the difficulties generally related to commercial interests outside of control of both Bank and project personnel). Key M&E risks and recommended actions identified in early Aide Memoires and at the MTR (see M&E section) were ultimately not acted upon by Government. Risks to the sustainability of road investments as a result of overloaded trucks (again, beyond the control of task team and project staff) and insufficiently operationalized O&M committees were also flagged regularly but were not fully resolved. Justification of Overall Rating of Bank Performance 102. Based on the above review and the limitations outlined in overall project implementation, and consistent with the rating of project outcome, Bank performance is rated MODERATELY SATISFACTORY. D. RISK TO DEVELOPMENT OUTCOME 103. The main risks to the project’s development outcomes are: (i) the continuation of the O&M needed for the infrastructure built by the project;47 (ii) the expansion and implementation of the participatory socio-economic planning approach beyond the project districts; (iii) the continued provision of appropriate agricultural advisory services to the farmers to maintain or improve their production and marketing efforts once the project has closed; and (iv) the COVID-19 outbreak. 104. As discussed earlier, community’s capacity to control heavy truck traffic on project roads appears to be limited. It also remains to be seen if the O&M groups supported by the project will be in a position to continue their maintenance work (for all commune infrastructure) without a dedicated budget allocation. This is a particular challenge in Vietnam (and was flagged in the evaluation report for the Northern Mountains Poverty Reduction Project Phase 1) for all CDD projects. 105. The project’s success of engaging ethnic minorities and women more actively through participatory approaches and decision making at the village level through the CDBs is not guaranteed, as these are not permanent 47BKG. Joint Stock Company, 2017. Quality inspection and evaluation of investment extent and efficiency for completed infrastructure works of Central Highlands Poverty Reduction Project. Page | 25 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) structures. Nonetheless, CDB members generally remain with the Commune People’s Committee and can therefore continue to facilitate this type of participatory planning. The evidence from the impact evaluation also showed general increases in EM and female EM participation and voice in commune development issues more generally. 106. The agricultural extension services provided by private sector companies and the Department of Agriculture and Rural Development may not be sustained because private providers were paid a percentage of the sub-project grant, which will no longer be available to farmers. The project design did not envisage putting in place formal institutional linkages with the Department of Agriculture and Rural Development nor to build the capacity of the department’s extension services, something that could be pursued in future operations. However, where farmers are engaged in traditional crops and livestock that the Department of Agriculture and Rural Development normally supports, appropriate advisory services should be accessible. Where private sector linkages are required for non- traditional agriculture production, farmers will have to weigh the potential costs with the benefits of securing outside technical support. Private providers who also supplied inputs will have an incentive to continue supporting those farmers with whom they have established a productive or commercial linkage. 107. The spread of the COVID-19 virus in Vietnam has so far been controlled well and the negative health impacts have been contained, however, the global pandemic will have ramifications for Vietnam’s economy. Preliminary national data collected since the COVID-19 outbreak suggest substantial reductions in income for vulnerable (and especially EM) households; data from Central Highlands show an increase of approximately 30% in the number of people reporting to be unemployed (between December 2019 and June 2020), and 53% of farming households indicating income losses estimated at more than one-third.48 V. LESSONS AND RECOMMENDATIONS 108. Investment Project Financing-type financing can effectively complement programmatic support for enhanced outcomes. The project financed similar types of infrastructure and certain livelihood investments as some of the rural and poverty focused NTPs that Government currently supports. However, such operations, including CHPov (and the predecessor Northern Mountains projects) placed greater emphasis on: (a) soft facilitation (and communication) skills and inputs to enhance participation; (b) more customized livelihood support; and (c) social and environmental safeguards. As a result, improvements with participation (including of women and of indigenous EMs), satisfaction with agriculture support, and dietary diversity were significantly greater in CHPov communes versus the NTPs. In addition, provincial authorities noted leveraging the support under the project to facilitate work in NTPs. As such, the utility of Investment Project Financing to pilot concepts and strategies, reach or address particularly marginal groups or areas, and to augment parallel NTP implementation would argue for their continued support, especially where Government is still experimenting with approaches or is faced with specialized areas of need or demand where a one-size-fits-all approach would not be appropriate. 109. Inter departmental coordination across different ministries is challenging without formal arrangements (and financing) in place. Using the private sector to fill the gap can help as long as financing is available to sustain that support (either through the project or by users). While the project was able to more than double the rate of satisfaction with agriculture extension services in project communes, the overall rate remained fairly low at 28%. This points to a general dissatisfaction with both public and private extension services in these predominantly EM 48Data from: (i) UNWOMEN, UNDP (2020), COVID-19 Impact on Vulnerable Households and Enterprises in Viet Nam: A Gender-sensitive Assessment” (RIM-2020). June 2020; (ii) GSO (2020), Labor and Employment Report – Quarter II 2020, Hanoi; and (iii) IFAD, IPSARD, ADB (2020), Impact assessment of COVID-19 on livelihood of farmers in Vietnam, Hanoi. Page | 26 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) areas of the country, which likely reflects a disconnect between what extension services are supporting (and how) and what is important to EM farmers. The project was only partially able to bridge this gap. Given the importance of agriculture development under the project, at design the project should have included a stronger institutional link to appropriate technical services that could be sustained. 110. Community supervision of infrastructure must be adequately paired with appropriate technical oversight of contractors. The project applied standard community-based supervision procedures through the project- facilitated Commune Supervisory Boards that drew from local civil society organizations (women’s union, farmers association, etc.). Evidence from the end-line survey suggests that these structures were quite successful in increasing the level of community supervision of local works. However, despite basic training there are limitations to the Commune Supervisory Boards’ technical skills. Therefore, the project engaged a technical supervision firm at provincial level to conduct technical reviews and spot-checks of all civil works implemented in the province, and especially those larger works contracted by districts. In addition, a supervising engineer was required for all works activities that were contracted out (versus direct community implementation). Despite this 3-tiered oversight structure, CHPov experienced some problems with execution, quality of works, and sub-project handover procedures.49 In future projects of a similar nature, the linkages between such community supervision efforts and the formal and independent supervisory consultants should be strengthened. The independent consultants also need to be more closely monitored by the project (which was done in the latter half of CHPov’s lifetime to correct the identified issues). 111. Specialized community engagement and facilitation support, which requires relevant language skills, is essential to reach the most isolated EM communities. In the absence of continued support from community facilitators, the local government authorities at the district level are constrained in their ability to reach and assist the most isolated communities. In addition, ensuring the substantive participation of EMs in particular requires specific outreach and encouragement that is generally not available. The increased levels of participation of women and indigenous EMs under CHPov warrant the investment in this specialized facilitation. This type of support is different and separate from other technical outreach and services from relevant line agencies and the local government. 112. Carefully consider the type, ambition and methods of measuring results for multi-sectoral/CDD operations, particularly in complex environments. CHPov was an ambitious project that looked to improve a wide cross-section of outcomes in a very complex and challenging environment. In retrospect, the project team likely selected overly ambitious PDO outcomes (enhanced living standards, changes in consumption, etc.) and some overly optimistic targets. This is especially true under CDD operations that will tend to spread investments across multiple sectors, and therefore reducing the extent of impact in any one area of measure (e.g., access to clean water). Given this likely dampening of results across sectors, it is important to include some key intermediate indicators of results that fall between simple output measures and higher-level outcomes. 113. There are limitations and risks in using complex, and costly, difference in differences impact evaluations for CDD operations. The relatively small and multi-sector nature of CDD investments (even when done over multiple years) increases the need for larger survey samples to improve the power of the analysis, which increases the cost of the exercise. In addition, the normal program approach of purposefully targeting project areas (usually based on poverty measures) will be in tension with a desire to randomly assigning project support to facilitate identification of those areas that do not receive project support (the project’s “counter-factual”). Lastly, even when project support can be assigned randomly, or other non-project areas can be identified through techniques such as propensity score matching, the risk of other funds flowing into non-project areas can be high. Indeed, this risk may be increased when 49 BKG, 2017. Quality inspection and evaluation of investments Central Highlands Poverty Reduction Project. Page | 27 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) local authorities are given discretion over local development funds and chose to “compensate” non -project areas with other funding sources, as appears to have happened under CHPov, rendering the “counter-factual” communes moot. In light of these challenges and risks, task teams should think very carefully about the type of evaluation approach to be used that will best help the project, and the client, learn from the project effort. In the case of CHPov, it might have been wiser to rely less extensively on a robust IE and more on process and outcome studies. . Page | 28 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS B. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Enhance living standards by improving livelihood opportunities in Project Communes Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Direct project beneficiaries Number 0.00 540472.00 638536.00 30-Sep-2013 31-Dec-2019 31-Dec-2019 Female beneficiaries Percentage 0.00 50.00 50.14 31-Mar-2017 No. Beneficiary Households Number 0.00 120000.00 141897.00 30-Sep-2013 31-Dec-2019 31-Dec-2019 Comments (achievements against targets): Measure changed to individuals versus households to capture individual LEG member beneficiaries. Page | 29 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percent Change in Proportion Percentage 0.00 20.00 32.10 of Villagers Satisfied with Support Received for 30-Sep-2013 31-Dec-2019 31-Dec-2019 Development Project Comments (achievements against targets): The average improvement in the satisfaction level of villagers is 32.1%, which consists of the following changes in satisfaction: (i) the selection of infrastructure projects (increased from 73% to 84%), (ii) the quality of those projects (from 59% to 65%), (iii) the quality of local roads (from 45% to 49%), (iv) local irrigation system (from 29% to 62%), and (v) with agricultural extension activities (from 12% to 28%). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Change in food and non-food Amount(USD) 8238500.00 9062350.00 12191700.00 consumption of poor households 30-Sep-2013 31-Dec-2019 31-Dec-2019 Comments (achievements against targets): Data were collected in VND. Converted to USD using end-of-project rate of exchange is: baseline: $355.54, endline: $526.14 Page | 30 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) A.2 Intermediate Results Indicators Component: Village and Commune Infrastructure Development Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Annual planned versus Percentage 0.00 88.00 76.20 completed infrastructure investments 30-Sep-2013 31-Dec-2019 31-Dec-2019 irrigation canals Meter(m) 0.00 0.00 42450.00 constructed or repaired bridges constructed or Number 0.00 0.00 30.00 repaired water systems constructed Number 0.00 0.00 150.00 or repaired classrooms constructed or Number 0.00 0.00 122.00 repaired roads constructed or Kilometers 0.00 0.00 666.00 repaired Page | 31 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Comments (achievements against targets): Overall for the 5-year project implementation period a total of 2,226 sub-projects were planned under components 1.1 and 3.1, and 1,697 were completed (76.2%). However, an original target of the percentage of planned vs. completed sub-projects was not established. Nonetheless, given that approximately 12% of the credit was cancelled the task team proposes that a target of 88% would be a reasonable expectation given this reduction in available financing. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Change in access of poor Text 38.6% 20.0% 39.5% households to services, infrastructure and utilities 30-Sep-2013 31-Dec-2019 31-Dec-2019 Comments (achievements against targets): The overall index measure only found a 2.3% improvement (between project and non-project communes) across all indicators for the poor. However, there was an overall reduction in time to travel to schools of more than 30% (with larger savings for indigenous EM (IEM) groups and poor); an overall decrease of 26.3% of time in accessing commune health centers (and more than 30% decrease for IEM and poor households); a nearly 27% reduction in travel time to commune people’s committee; a 70% improvement in access to drinking water for the poor, and a 17.6% increase in access to irrigation systems for poor households. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Proportion of poor villagers Percentage 0.00 20.00 25.00 whose identified development priorities are 31-Dec-2014 31-Dec-2019 31-Dec-2019 satisfied Page | 32 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Comments (achievements against targets): Measure considers all households and not just poor as baseline survey found general agreement between poor and non-poor and inclusion of all households improved statistical significance. Results: 25% of households had all three priorities addressed in their communes; 81% of households had at least 1 or 3 priorities addressed. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Increase in participation of Percentage 0.00 25.00 44.60 women in decision-making on local public investments 30-Sep-2013 31-Dec-2019 31-Dec-2019 Comments (achievements against targets): The endline survey found that the proportion of women taking part in commune/village meetings and expressing their opinions (a more meaningful measure of “participation”) about project selection increased by 4.8 and 10 percentage points (a 15.5% and 44.6% improvement), respectively. In addition, the study found an overall 10 percentage point increase-- from 22.4% to 32.4%-- in women expressing an opinion at commune/village meetings, represents an almost 50% improvement over baseline. Component: Sustainable Livelihoods Development Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Livelihoods enhancement Number 0.00 3687.00 4383.00 groups (LEGs) and market link partnerships (MLPs) 30-Sep-2013 31-Dec-2019 31-Dec-2019 Page | 33 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) established female LEG members Percentage 0.00 0.00 59.00 Comments (achievements against targets): Original target established by Government as indicated in their Project Completion Report (May 2020) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Change in productive and Percentage 51.40 20.00 56.20 durable assets of households participating in LEGs 30-Sep-2013 31-Dec-2019 31-Dec-2019 Comments (achievements against targets): On average, asset ownership improved by 7% between baseline and endline, while the increase for poor households was 12%. A revised index that focused more on productive agriculture-related assets shows an overall 9.3% improvement at the endline. In addition, at sub-indicator level, the study reveals significant increases in households owning cattle (24.5%) and poultry (14.9%)-- both of which were important for livelihood diversification and household nutrition—water pumps (15.3%) and motorbikes (8.9%). However, other productive assets such as gardening and irrigated land, mobile phones and pigs fell short of reaching the proposed target increase of 20%. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Change dietary diversity of Text 0.573 on constructed 20% 0.68 Page | 34 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) poor households dietary diversity index 30-Sep-2013 31-Dec-2019 31-Dec-2019 Comments (achievements against targets): Overall, with the exception of migrant ethnic households, the project contributed to a 21% increase in the DDI. Sharp increases were observed for the proportion of households consuming fruits (23%), meat (26%), eggs (28%), and for sugar and honey (27%). Component: Connective Infrastructure Development, Capacity Building and Communications Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Change of transport Text 68.2 20% 70.1 connectivity index 30-Sep-2013 31-Dec-2019 31-Dec-2019 Comments (achievements against targets): Overall, the index showed a 2.8% improvement in connectivity from the beginning to end of the project, and between project and control comments. This overall connectivity measure was better for poor and indigenous EM households, at 4.1% and 3.7% improvement respectively. At sub-indicator levels, statistically significant improvements were seen in connectivity to primary schools (by 9.7%), to commune health facilities (by 9.4%) and to local markets (by 6.8%). A separate endline analysis also showed similar improvements with a reduction in travel time to local schools of 3 minutes (or almost 30%) and to local health facilities of 5.4 minutes (or 26%). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page | 35 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Client days of training Number 0.00 35594.00 56961.00 provided (number) 30-Sep-2013 31-Mar-2019 31-Dec-2019 Client days of training Number 0.00 12394.00 16758.00 provided - Female (number) 30-Sep-2013 30-Sep-2018 31-Dec-2019 Staff and beneficiaries Number 0.00 0.00 19451.00 trained Comments (achievements against targets): Original targets drawn from Government Project Completion Report (May 2020) Component: Project Management Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage delays in Percentage 0.00 5.00 3.10 1.30 procurement activities as per approved annual 30-Sep-2013 31-Dec-2015 30-Jun-2017 31-Dec-2019 procurement plan Comments (achievements against targets): Measures ratio of contracts signed by Dec. 31 to contracts planned for given year. Page | 36 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Page | 37 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) C. KEY OUTCOMES/OUTPUTS BY COMPONENT50 Objective/Outcome 1: Improved access to basic infrastructure and public services at village and commune level 1. RF#4: Percent Change in access of Poor Households to Services, Infrastructure, and Utilities—2.3% average change. Sub-set of index measure sub-indicator results for aspects most closely linked to CHPov investments: a. Average travel time to kindergarten: 31.8% improvement, 3.5 minute reduction b. Average travel time to primary school: 33.8% improvement, 5.7 minute reduction c. Average travel time to commune health facility: 31.9% improvement, 7.9 minute reduction d. Poor households with access to improved water sources: 18.9% increase e. Increase in households with irrigation systems in commune: 12.1% increase 2. RF#6: Percent Change in Proportion of Villagers Satisfied with Support Received from the Project— 32.1% average change. Sub-set of index measure sub-indicator results for aspects most closely linked to CHPov investments: Component 1 Outcomes a. Increase in proportion satisfied with Selected Investment Project List: 14.8% b. Increase in proportion satisfied with Project Quality: 10.2% c. Increase in proportion satisfied with Road Quality: 9.9% d. Increase in proportion satisfied with Irrigation System: 110.2% 3. RF#2: Percentage of households whose development priorities were satisfied (i.e., financed)—25% of HHs with all three top priorities financed under project, 81% of HHs with one of top three priorities financed. 4. RF#7: Increase in participation of women in decision-making on local public investments—15.5% improvement in women’s participation, and a 44.6% improvement in women expressing opinions in community meetings. There was also an overall improvement in participation of 29.2% for all community members. Sub-component 1.1: Design, construction or repair of small scale village and commune level infrastructure. Key Outputs by 1. Number of infrastructure sub-projects (SP): Total SPs 1,525; 1,053 SPs roads; 78 SPs classrooms; 90 SPs Subcomponents irrigation systems; 146 SPs water supply systems; 79 SPs new community houses; 79 SPs other infrastructure. 50 In this section of the ICR all percentage changes indicated are based on the percentage change between base-line and end-line values for the relevant indicators/measures. Page | 38 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) 2. Type of Infrastructure built: 445km roads; 98 classrooms; 35,6 km canals; 1,351 ha improved irrigation; 146 domestic water supply systems for 10,927 households, 79 community houses, 25 bridges, 10 underground spillways, 6 low voltage power lines; plus some miscellaneous such as village sport stadiums, etc. 3. Training: Total number of people trained in construction supervision: 685 (19% female), 572 trainees from village and commune level, 113 trainees from central, provincial and district level. Sub-component 1.2: Repair, operation and maintenance of communal infrastructure. 1. Number of infrastructure sub-projects (SP): Total SPs 466; 2. Type of Infrastructure built: 209 SPs for road maintenance; 24 SPs for repair of bridges, culverts, drainage ditches; 33 SPs for irrigation related repair; 78 SPs for domestic water system related repairs; 11 SPs for repair of community houses; and 108 SPs for repair of primary schools and kindergartens, and some other small repairs. 3. Training: Total number of people trained in O&M: 1,402 (16% female), 1065 trainees from village and commune level, 337trainees from central, provincial and district level. Objective/Outcome 2: Enhanced livelihood options and improved food and nutrition security 1. RF#10: Percent Change Dietary Diversity of Poor Households—overall 21.4% improvement. 2. RF#9: Percent Change in Productive and Durable Assets of Households Participating in LEGs—overall 9.3% improvement. Sub-set of index measure sub-indicator results for aspects most closely linked to CHPov Component 2 Outcomes investments: a. % increase in LEG households owning pigs 22.9%, and poultry 25.2% b. % increase in LEG households owning a motorcycle: 10.9% c. % increase in LEG households owning a water pump: 38.1% d. % increase in LEG households owning a mobile phone: 5.5% Sub-component 2.1 Self-reliance and Income Generation Key Outputs by 1. Number of LEGs established: 4,337 LEGs established; 1,402 LEGs (19,031 HHs) under Food Security & Subcomponents Nutrition (out of these 550 LEGs received two support cycles); 2,935 LEGs (43,715 HHs) under Livelihood Diversification.; overall around 50% of LEGs discontinued to work as groups at project end 2. Total number of LEG member households: 62,746 (60% females); Page | 39 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) 3. Training: Total number of members trained (i) in agricultural techniques, 42,416 (60% female; 77% ethnic minorities); in (ii) LEG management, 5,709 (51% female; 52% ethnic minorities), (iii) in nutrition 1,647 (66% females; 67% ethnic minorities) Sub-component 2.2: Market Linkages Initiative 1. Number of LEGs established: 46 LEGs established under market linkage sub-component; percentage discontinued unknown, but LEGs producing gac, bananas, maize, sweet potatoes, pigs, goats are reported to be not effective.51 2. Total number of LEG member households: 785 (60% female); 3. Training: number of members trained in agricultural techniques unknown; other trainings included under SC2.1 above. Objective/Outcome 3: Improved inter-commune connectivity, capacity building & outreach 1. RF#11: Change in Transport Connectivity Index—overall improvement of 9.4%. Sub-set of index measure sub-indicator results for aspects most closely linked to CHPov investments: a. 7.7% improvement in connectivity to primary schools Component 3 Outcomes b. 8.0% improvement in connectivity to commune health facilities c. 9.1% improvement in connectivity to local markets d. 3.3% improvement in connectivity to commune peoples center e. 7.1% improvement to district center Sub-component 3.1 Development of selected intra-communal and inter-communal level infrastructure and other investments supporting socio-economic connectivity. 1. Number of infrastructure sub-projects (SP): Total SPs 172; 144 SPs roads; 5 SPs classrooms; 15 SPs irrigation Key Outputs by systems; 4 SPs water supply systems; 4 SPs other infrastructure. Subcomponents 2. Type of Infrastructure built: 221km roads; 2 suspension bridges; 3 underground overflows; 6.7 km canals; 382 ha improved irrigation; 6 dams; 4 domestic water supply systems for 445 households; 1 irrigation pumping system; 14 advisory contracts. Sub-component 3.2 Training and Capacity Building for all aspects of project management 51 GoV Project Completion Report. Page | 40 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) 1. Training: Organized 542 training and capacity building events for project staff and governmental officers. 2. Number of trainees: 19,451 (30% female; 43% ethnic) Note: training outputs pertaining to other sub-components are listed under the respective sub-components above Sub-component 3.3 Communications and outreach activities. 1. Information, communication and education material: around 450,000 items of information material including: 270 DVDs, 100 CDs, 8 audiobooks, 400,000 posters and leaflets, 1,000 picture books, around 6,300 commune and community meetings, 33,750 farm calendars, 115 television reportages, around 450 articles and newsletters, etc. Page | 41 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) D. DETAILED DISCUSSION OF ACHIEVEMENTS OF INDIVIDUAL RESULTS FRAMEWORK INDICATORS 1. The following discussion draws heavily on the findings from the project’s end-line survey report that was prepared by Mekong Development Research Institute. Data referenced below can also be found in the selected tables in Annex 7, which are taken from the end-line report. PDO-level Indicators 2. RF#1: Number of direct beneficiaries (Core Indicator). The indicator measured the outreach of the Project and the percentage of women reached as a sub-indicator. The number of direct beneficiaries has been defined as “entire population of villages having at least one infrastructure investment (regardless of type) throughout the project cycles, plus the number of LEG beneficiaries in villages having no infrastructure investment.”52 Accordingly, the project reached 638,536 beneficiaries, or almost 120% of the original target of 540,472, with 50.14% female (against a target of 50%). The percentage of women participating in the livelihoods groups was even higher at 59%. Using the total population of a community as the basis for estimating project beneficiaries is standard practice for CDD projects given the overwhelmingly public nature of project investments. Based on this assessment the indicator target is considered to have been exceeded. 3. RF#2: Proportion of poor villagers whose identified development priorities are satisfied. This indicator was meant to measure if the development priorities identified by poor villagers at the beginning of the project have actually been selected for financing and have been implemented by the project. This meaning was also reflected in the Vietnamese translation of the indicator, although limited to infrastructure (“percentage of infrastructure proposed by poor villagers implemented”). This was how the indicator was defined in the original RF of the project that is found in the PAD.53 The underlying logic of this measure is to test how well the participatory and transparent process of identifying investments was working. If the priorities of the poorest households were being responded to then we would conclude that the process is effective in facilitating the demand of the community (and avoiding elite capture). However, the baseline and end-line surveys interpreted this indicator differently and sought to compare changes in development priorities as an indication that priorities have been met. This overlooks the reality that priorities that may persist in each commune could reflect an over-demand for a given type of investment and not a lack of supply. In other words, while households may have selected village roads as their priority, and village roads may have been invested in, there may remain additional roads in need of repair or improvement. 4. Therefore, this indicator was measured by the ICR team in a different manner than outlined in the end-line and IE reports. In addition, in reviewing baseline preferences of poor and non-poor households it was found that in general the preferences for development were aligned in both groups. Therefore, the IE team recommended using the priorities identified by all households as this improved the statistical relevance of the “priorities”. As can be seen in the table below, comparing preferences from baseline data to actual implementation of sub-projects in each commune from project administrative data reveals a high proportion of households (87%) having at least 1 development priority satisfied. In addition, 25% of households had all development priorities “satisfied”, or financed, by the project. The analysis does also note, however, that 13% of households had none of their top three priorities satisfied under the project. This is possibly the result of households having identified priorities that significantly overlap with the types 52 WB, 2019. Aide Memoire – Implementation Support Mission, Annex 2. 53 PAD, page 21, footnote 19. Page | 42 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) of infrastructure widely financed under the National Target Programs (i.e., markets, office of commune people’s committee, health clinics, and electricity). 5. The top four priorities identified by households at baseline were inter-village roads (28.3%), clean water (16.4%), in-field roads (8.7%), and irrigation (8.3%). The composition of infrastructure types financed by CHPov followed these same priorities with component 1.1, financing 1,525 infrastructure subprojects, of which 69.1% were roads, 9.8% for clean water, and 5.9% for irrigation systems. Given these findings, the indicator target is rated as having been exceeded. Table 1-1: Analysis of Satisfaction of Development Priorities by Province Quang Quang Kon Dak Dak Overal Nam Ngai Tum Gia Lai Lak Nong l HHs having at least 1 priority was satisfied 96.6% 94.7% 84.6% 81.4% 84.6% 84.6% 87.0% HHs having 100% priorities were satisfied 57.0% 46.4% 15.9% 11.9% 12.4% 22.8% 25.1% HHs having none of priorities were satisfied 3.4% 5.3% 15.3% 18.5% 15.5% 15.5% 13.0% Total 100% 100% 100% 100% 100% 100% 100% * This condition measures those households where one of their top three priorities were ineligible for financing under CHPov. 6. RF#3: Percent change in food and non-food consumption of poor households. This indicator measured the changes in expenditures for food and non-food items. It was the main tool used to evaluate the impact of the CHPov towards enhanced living standards and, in effect, reducing poverty. Overall, the rate of per capita consumption of poor beneficiaries in the project communes increased by 48% between 2014 and 2019, 54 which significantly exceeds the proposed target of 10%. In addition, non-poor households saw only an 8.3% per capita increase, suggesting that the poverty targeting efforts of the project also had strongly positive results. Food expenditures of poor households increased by 32% and non-food expenditures by 78% against the baseline. This positive trend is also reflected in a reduction of food-expenditures as share of total household expenditures from around 57% to 50%, which also suggests improving living standards. 7. This change in consumption is linked to an overall increase in household expenditures by 18.3% over the same period (or 3.5% per annum). These improvements were seen across all groups of project beneficiaries that includes the Kinh/Hoa, EMs indigenous to the Central Highlands (IEMs), and indigenous minorities who have migrated to the Central Highlands (migrant ethnic minorities, MEM). The expenditure per capita in all 6 project provinces in 2019 is higher than the expenditure poverty line, reflecting in part an improvement in household living standards.55 Based on the above the above assessment the indicator target was exceeded. Component-level Indicators 54 In real terms adjusted to 2019 prices, overall annual per capita consumption of the poor in project areas increased from VND 8.166 million to VND 12.142 million. 55 Although not statistically significant, the impact evaluation found that household expenditures within project communes increased by 2% more than in the non-project communes. Page | 43 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Component 1: Village and Commune Infrastructure Development 8. RF#4: Percent change in access of poor households to services, infrastructure and utilities. This indicator was constructed based on 12 sub-indicators gauging household access to four main service domains concerning education, health, water supply and agricultural services. The end-line survey of project communes shows significant positive change in access to several services relevant to CHPov areas of infrastructure investments (partly as a result of investments in local roads). For instance, there was an overall reduction in time to travel to schools of more than 30% (with larger savings for indigenous EM (IEM) groups and poor); an overall decrease of 26.3% of time in accessing commune health centers (and more than 30% decrease for IEM and poor households); a nearly 27% reduction in travel time to commune people’s committee; a 70% improvement in access to drinking water for the poor, and a 17.6% increase in access to irrigation systems for poor households—the latter two being areas of priority project investment. However, the overall index measure only found a 2.3% improvement (between project and non-project communes) across all indicators for the poor. Given an overall project target of 20% increase, the target for this indicator is considered partially achieved. 9. RFI#5: Annual planned versus completed infrastructure investments. This performance measure was not applied as originally envisioned due to peculiarities of the Government’s planning and implementation cycle. Most importantly, the Government allows for adjustments in plans during the course of the year to reflect changes in approved versus allocated budget, cost variations in individual subprojects, changing community priorities, and as a result of unforeseen circumstances. Plans are adjusted during the course of the year to align them with actual implementation. In addition, Government allows implementation from one calendar year to carry over into the next year (up until December 31 of the following year). However, over the course of the 5-year implementation phase, the overall average of planned versus completed infrastructure projects was 76.2%, as shown in the table below. This level of output is roughly consistent with the overall value of project financing as compared with the original budget, and therefore this target for indicator is assessed as substantially achieved. 10. Table 1-3 below indicates the allocation of completed subproject types by provinces. The main subprojects as a percentage of total were: village and inter-commune roads (70.5%), drinking water systems (8.8%), irrigation construction and repair (8.3%), the construction of community centers (4.7%) and construction or repair or schools (4.9%). Quality of infrastructure, as measured by satisfaction levels of beneficiaries, improved particularly for water supply and irrigation systems. However, field surveys by Bank implementation support teams found some quality issues especially with roads, likely leading to the lower rates of satisfaction for this area of investment. Table 1-2: Total number of infrastructure sub-projects planned and implemented (by Province) Number of Number of Total Number of Number of Total % Sub- Provinces Approved Approved Approved completed completed Completed projects Subprojects 1.1 Subprojects 3.1 Subprojects subprojects 1.1 subprojects 3.1 Subprojects completed Dak Lak 363 33 396 307 21 328 82.8% Dak Nong 321 59 380 245 30 275 72.4% Gia Lai 343 43 386 314 35 349 90.4% Kon Tum 380 59 439 321 40 361 82.2% Quang Nam 129 29 158 119 28 147 93.0% Quang Ngai 423 44 467 219 18 237 50.7% Total 1,959 267 2,226 1,525 172 1,697 76.2% Page | 44 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Table 1-3: Total number of infrastructure sub-projects implemented (by subproject type and Province) Total Sub-project Type Quang Ngai Dak Lak Quang Nam Gia Lai Kon Tum Dak Nong Completed Sub-projects 1.1 3.1 1.1 3.1 1.1 3.1 1.1 3.1 1.1 3.1 1.1 3.1 Road 125 18 281 21 59 23 261 35 157 28 170 19 1,197 School 9 6 12 5 26 2 20 3 83 Irrigation 14 7 10 1 16 29 6 14 8 105 Water Supply 50 25 2 2 61 2 8 150 Community House 14 4 5 12 18 26 79 Other 7 4 3 18 13 2 0 47 Culverts and sluice 5 5 2 17 7 36 gates Total 219 18 307 21 119 28 314 35 321 40 245 30 1,697 11. RF#6: Percent change in proportion of villagers satisfied with support received from the project. The end-line survey reports that overall household satisfaction with support in the project area56 increased by 13.8 percentage points (an improvement of almost 32%). Specifically, satisfaction was shown to have increased across five domains: (i) the selection of infrastructure projects (increased from 73% to 84%, or a 15% improvement), (ii) the quality of those projects (from 59% to 65%, 10% improvement), (iii) the quality of local roads (from 45% to 49%, 9% improvement), (iv) local irrigation system (from 29% to 62%, 138% improvement), and (v) with agricultural extension activities (from 12% to 28%, 117% improvement). 12. Although the project succeeded in more than doubling the satisfaction rates for agricultural advisory services, at 28% the actual satisfaction rate in 2019 was still low. While this suggests some shortcomings in either coverage, quality, or frequency of trainings under the project, given the separate role that the Department of Agriculture and Rural Development plays in providing similar support, this may not be a clear assessment of CHPov. The second highest improvement in satisfaction, by 138%, was observed for irrigation infrastructure (from 29 to 62%), another important area of project investment. Lower rates of change in satisfaction were recorded for local road quality, which may reflect either continued unmet demand for roads, or problems encountered by misuse of and damage to project roads by over-weight vehicles. However, given these overall positive results, and the fact that both irrigation and agricultural extension were important areas of investment by the project, this indicator target is assessed as fully achieved. 13. RF#7: Percent increase in participation of women in decision-making on local public investments. The end-line survey found that the proportion of women taking part in commune/village meetings and expressing their opinions (a more meaningful measure of “participation”) about project selection increased by 4.8 and 10 percentage points (a 15.5% and 44.6% improvement), respectively. Participation rates also differed amongst women in different groups. Women from female-headed households seemed to be more active in joining local activities than other women. The end-line survey also found larger improvements in participation for indigenous Ethnic Minority (IEM) women, as compared with Kinh/Hoa women. In 2014, female IEM participation was 23.3% while 27.3% of Kinh/Hoa women participated. However, by 2019, the corresponding figures of these two groups were 35.8% and 33.4%, implying a 54% 56 Information on satisfaction was collected for all support provided to the communes, and not limited to CHPov efforts. Page | 45 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) improvement in IEM women’s participation. In addition, the study found an overall 10 percentage point increase-- from 22.4% to 32.4%-- in women expressing an opinion at commune/village meetings, represents an almost 50% improvement over baseline. The improvement for IEM women was even greater. Based on this evidence the indicator target is fully achieved. Component 2: Sustainable livelihoods development 14. RF#8: Number of livelihoods enhancement groups (LEGs) and market link partnerships (MLPs) established. No specific target for this measure was established in the PAD or later on given the demand- driven nature of the project. Under Sub-Component 2.1 (self-reliance and income generation), a total of 4,337 LEGs supporting 63,561 households were established. Two types of sub-project activities were supported: (i) food security and nutrition groups (which were 30% of total), and (ii) income diversification groups (70% of total). Out of these about 13% of the LEGs received a second round of support. Field crops, such as rice and maize, constituted 21% of subprojects and dominated the food security groups, while 14% of LEGs were involved in other field crops and vegetables. The most popular livestock activities (as measured by the number of LEGs), after the project excluded the choice of cattle due to high costs, were raising pigs (23%), goats (21%), and poultry (16%). 64 LEGs engaged in more specialized commodities such as sugarcane, green bean, ginger, turmeric, ginseng, avocado, passion fruit, black pepper, bamboo rats, wild ducks, pigeons, rabbits, etc. The total percentage of female LEG members was 59% and the average number of members per LEG was 15 farmers. 15. Under Sub-Component 2.2 (market linkage initiatives), only 46 LEGs comprising 785 households, were established. These included groups dealing with the cultivation of pineapple (15), gac (Momordica cochinchinensis) (5), bananas (8) and hybrid maize (4). The rest of the market linkage LEGs were engaged in areas such as goat breeding, mulberry cultivation for silkworm raising, and acacia seedlings, among others. The MTR estimated that LEGs operating under a farmer-agribusinesses partnership experienced an increase of their farm-gate price by some 15-20% and their production had become more efficient due to good practices and innovations brought by the project. At the same time, key challenges identified were: (i) lack of a more standardized contract farming arrangement, (ii) insufficient understanding of the value chain concept by the Provincial Project Management Unit staff, (iii) insufficient quantities produced by smallholder farmers, and (iv) lack of interested and capable agribusiness partners. The last issue was likely key in the relatively low numbers of market linkage groups overall. 16. Despite challenges faced by Component 2 that include the short life-span of LEGs (only slightly more than 50% of the LEGs were still active at the end of the project) and the low number of market linkage groups, the project managed to establish a large number of LEGs with high rates of participation of women (59%). Based on the review of results under this output measure, the ICR assesses the indicator target as being achieved. 17. RF#9: Percent change in productive and durable assets of households participating in LEGs. This composite index indicator was conceived to measure the results of improved livelihoods, which were thought would lead to the purchase of household assets (19 in total were measured57). On average, asset ownership improved by 7% between baseline and end-line, while the increase for poor households was 12%. A revised index that focused more on productive agriculture-related assets shows an overall 9.3% improvement at the end-line. In addition, at sub-indicator level, the study reveals significant increases in households owning cattle (24.5%) and poultry (14.9%)-- both of which were important for livelihood 57 Some assets in the original index would be considered luxury rather than “productive” goods. Page | 46 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) diversification and household nutrition—water pumps (15.3%) and motorbikes (8.9%). However, other productive assets such as gardening and irrigated land, mobile phones and pigs fell short of reaching the proposed target increase of 20%. Based on these findings the indicator target is assessed as being partially achieved. 18. RF#10: Percent change in dietary diversity of poor households. This indicator was measured by calculating a Dietary Diversity Index (DDI) based on the Household Dietary Diversity Score index developed by USAID. It was used to understand the change in dietary diversification of poor households as a result of several aspects of project implementation (food security LEGs, access to markets, etc.). The DDI is based on 12 sub-indicators, representing consumption information on one of the 12 main food groups, consumed over the last 12 months.58 19. The comparison of data from baseline and end-line survey shows that, with the exception of roots and tubers, the consumption of all food groups increased. In particular, sharp increases were observed for the proportion of households consuming fruits (23%), meat (26%), eggs (28%), and for sugar and honey (27%). Overall, with the exception of migrant ethnic households, the project contributed to a 21% increase in the DDI. Therefore, this indicator target is assessed as fully achieved. 20. In addition, the World Bank’s impact evaluation analysis found that the household dietary diversity score increased by 3.8 percent between treatment and control communes, which was even more positive among IEM and female headed households. The change in the household dietary diversity score increased by 6.5% among households belonging to EMs as compared to those of the Kinh. Similarly, households headed by women are likely to have a score of about 8.3% higher than those headed by men. Component 3: Connective infrastructure development, capacity building and communications 21. RF#11: Percent change of transport connectivity index. This indicator measured household travel time to key social infrastructure and destinations such as kindergartens, primary schools, commune health station, market, commune people’s committee, district and provincial centers, as a composite index measure of physical connectivity (Transport Connectivity Index (TCI)). These times (both for motorized and non-motorized connectivity) where then translated into a 5-point scale to compare changes over time. Overall, the index showed a 2.8% improvement in connectivity from the beginning to end of the project, and between project and control comments.59 However, at sub-indicator levels, statistically significant improvements were seen in connectivity to primary schools (by 9.7%), to commune health facilities (by 9.4%) and to local markets (by 6.8%). A separate endl-ine analysis also showed similar improvements with a reduction in travel time to local schools of 3 minutes (or almost 30%) and to local health facilities of 5.4 minutes (or 26%). In comparing these achievements with the RF target of 20% improvement, the review assesses the indicator target as partly achieved. 22. RF#12: Number of people trained (and person-days of training provided) by the project. The project invested a great deal of effort in training and capacity building. CPO and PPMUs organized 542 capacity building training events for project staff and governmental officers that involved 19,451 participants for a total of 56,961 person-days of training (of which 30% were women). In addition, almost 50,000 members of LEGs were provided training in general farming techniques, animal husbandry, disease prevention and 58 The end-line report notes that the use of a 12-month recollection period instead of a 24-hour period is prone to overestimation of the actual DDI of households. See MDRI, 2020. Impact Evaluation: CHPov; Executive Summary, p.15. 59 This overall connectivity measure was better for poor and indigenous EM households, at 4.1% and 3.7% improvement respectively. Page | 47 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) nutrition. This training was strongly targeted to women and EM participants with women in excess of 59% of trainees, and 73.8% EM trainees.60 It is also noteworthy that the nutrition trainings included 34% men. With the exception of training related to agriculture services, it is not possible to assess if the provided trainings were considered useful or of good quality by the beneficiaries as post-training evaluations were not carried out. However, for agriculture extension services, which were measured as part of the impact evaluation, the proportion of satisfied project beneficiaries increased from 12% to 28%. Based on this range of data relating to training outputs and outcomes, the review concludes that this performance indicator was fully achieved. Table 1-4: Inclusion of women and of ethnic minorities in training activities Total number of % of female % of ethnic % of participants from Training Topics trainees* participants participants commune and village levels 1. Agriculture techniques 42,416 60% 77% 98% 2. Managing LEGs 5,709 51% 52% 84% 3. Community development 9,036 31% 43% 48% 4. Nutrition 1,647 66% 67% 89% 5. Other topics 1,576 39% 63% 14% Total 60,384 54% 69% 86% * Counts duplicate participants (i.e., single person participating in more than one training). Component 4: Project management 23. RF#13 Percentage delays in procurement activities as per approved annual procurement plan. This indicator measures the relative effectiveness of CPO and provincial PMUs in both planning and procurement. The measure is calculated based on the total number of contracts signed in a given year, divided by the total number of procurement packages planned for in that same year. Given the newness of the project and the additional requirements for World Bank implementation, it would be expected that the rate of delay would be high to begin with but would improve over time. An end-target of no more than 5 percent “delay” was proposed in the results framework. A review of the data from the project shows a steady improvement in procurement performance across all Project Implementation Agencies from the first year of project implementation to the last, as shown in Table 1-5 below. Given these results the indicator target is fully achieved. Table 1-5: Analysis of Procurement Delays by Project year (all PIUs combined) 2014 2015 2016 2017 2018 2019 Percentage procurement delay, % (all PIAs) 84.5 22.2 18.7 3.1 n/a* 1.3 Source: World Bank (WB) Task Team calculations based on annual project administrative data. * The workplan and budget for 2018 was a combined 2018-19 plan and therefore the procurement delay was not calculated for the 2018 project year. 60As compared with the estimated 60% overall EM population in the target districts at the time of project start (per the project’s Social Assessment Report). Page | 48 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Sean Bradley Task Team Leader Son Thanh Vo Task Team Leader Steve Jaffee ARD Coordinator Markus Kostner Social Development Coordinator Hoai Van Nguyen Procurement Specialist Cung Van Pham Financial Management Specialist Nghi Quy Nguyen Social Specialist Thuy Cam Duong Social Specialist Khang Van Pham Environment Specialist Lan Thi Thu Nguyen Environment and Forestry Specialist Andrew Beath Evaluation Specialist Ida Christenssen Nutrition and Food Security Specialist (FAO) Takayuki Hagiwara Livelihoods Specialist (FAO) Aidan Gulliver Agriculture Economist (FAO) Ngozi Blessing Obi Malife Program Assistant Tam Thi Do Program Assistant Supervision/ICR Sean Bradley Task Team Leader Nghi Que Nguyen Co-Task Leader Page | 49 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Son Thanh Vo Co-Task Leader Chris Jackson ARD Coordinator Hoai Van Nguyen Procurement Specialist Cung Van Pham Financial Management Specialist Lan Thi Thu Nguyen Environmental Specialist Andrew Beath Evaluation Specialist Kyung In MIS Specialist Martin Henry Lenihan Social Specialist Nga Thuy Thi Nguyen Procurement Team Huong Lan Dao Health and Nutrition Specialist Hoang Tung Engineer Dao Ngoc Nga M&E Specialist Vu Thi Bich Ngoc Statistician/IE Specialist Nguyen Thi Hien Minh Communications Specialist Inna Punda Livelihoods Specialist (FAO) Anton Glaeser ICR contributor (FAO) Anne-Christelle Ott ICR contributor (FAO) Dung Thuy Vu Team Member Tung Hoang Team Member B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY12 22.900 135,928.46 FY13 35.407 257,450.13 FY14 29.227 140,234.45 Total 87.53 533,613.04 Page | 50 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Supervision/ICR FY14 8.838 43,889.03 FY15 18.225 99,256.13 FY16 33.095 157,881.17 FY17 28.800 140,485.33 FY18 20.525 118,356.12 FY19 12.742 98,865.85 FY20 18.000 137,875.66 Total 140.23 796,609.29 Page | 51 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) ANNEX 3. PROJECT COST BY COMPONENT Table 1: Total Estimated Project Costs (including counterpart funding) at Approval, upon Restructuring, and at Close (US$ million) (see Annex 4, Table 1 for detailed breakdown) Of which Of which Amount at Actual at Actual as Actual as Amount at IDA GoV Project Project Percentage Percentage of Components Approval Financed Financed Restructuring Closing of Approval Restructuring (US$M) (US$M) (US$M) (US$M) (US$M) (%) (%) 1. Village and Commune Infrastructure 54.70 52.40 2.30 47.06 40.94 74.8 87.0 Development 2. Sustainable Livelihoods 35.20 35.20 0.00 31.50 27.39 77.8 87.0 Development 3. Connective Infrastructure Development, Capacity 53.00 51.40 1.60 45.99 40.52 76.4 87.1 Building and Communications 4. Project Management 16.50 11.00 5.50 16.64 15.32 92.8 92.1 Total 159.40 150.00 9.40 141.73 124.17 77.9 87.6 Note: Amounts shown for “Project Restructure” and “Actual at Project Closing” based on Loan data for Credit 53300. Table 2: IDA Commitments, Cancellation, and final Disbursements (SDR million) IDA Credit 53300 SDR M 1. Signed Amount 97.60 2. Cancelled 11.50 3. Disbursed 83.81 4. Un-disbursed 2.29 5. Percent Disbursed* 97.34% * Difference between % disbursed and % actual costs relates to SDR-USD exchange rate changes over life of project. Page | 52 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) ANNEX 4. EFFICIENCY ANALYSIS General Efficiency Considerations 1. This annex analyzes the efficiency of the Central Highlands Poverty Reduction Project in Vietnam (P128072), financed by the World Bank as an input to the project’s Implementation Completion and Results Report. Project efficiency was assessed by i) the actual project costs and duration for realizing project objectives versus the plan; ii) the actual costs per beneficiary and per unit of output iii) the project economic rate of return as computed through an economic and financial analysis, and how these results compare to design estimates. 2. To assess the project efficiency, it is important to consider the discussion above regarding the lack of statistical significance between treatment and control communes for several of the key outcome measures (consumption, access, assets, and connectivity) at the aggregate level. However, also as noted, there is evidence that the control areas were “contaminated” through the allocation of NTP funds for similar types of investments. Therefore, measures of key project outcomes are drawn from the before and after results found in project communes as documented in the end-line report.61 3. The efficiency of project management is assessed based on the original projections compared to actual costs, as a percentage of total project costs. This assessment of project management costs is also compared to global averages for managing CDD projects. A comparison of these management costs to those of NTPs was not done, nor were the additional benefits of the environmental, social and fiduciary arrangements installed under the project calculated. 4. Overall, the project succeeded in meeting all of its output targets in the given timeframe and with a lower budget than initially envisaged. The initial IDA credit amounted to US$150 million. Following SDR-US$ exchange rate losses and a cancellation of SDR 11.5 million in the final year of implementation, the final value of the World Bank lending to the project was approximately US$118.4 million, or just over 20% less than the original projected budget. 5. The project Economic and Financial Analysis (EFA) at completion finds an economic Net Present Value of VND 1,489 billion, equivalent to US$64.1 million, and an Economic Internal Rate of Return (EIRR) of 29%, showing positive and substantial returns to investment. While both infrastructure and LEG investments were modelled, the analysis finds that most project benefits are derived from the community infrastructure investments, consistent with the fact that infrastructure investments accounted for a larger share of the budget. The project EIRR at completion was higher than the design estimate of 15%. 6. Moreover, it is important to note that the project had several benefits that could not be quantified. These included the increased participation of women in decision making processes, nutritional benefits for the project communities (the one key aggregate outcome measure that did show a statistically significant net positive increase between treatment and control communes) and improvements in the quality of life of the community inhabitants as well as increased capacity and the skills gained from the various trainings provided by the project. In addition, only two types of infrastructure investments were modelled, so the estimate of benefits are considered conservative. 7. However, the EFA at completion relies to a large extent on assumptions or evidence from a relatively small number of cases and data rather than systematic evidence from the project M&E and Management Information System. Specific data on production or outcomes as a result of infrastructure investments is limited. For instance, 61 See: The Central Highlands Poverty Reduction Project Endline Report; Mekong Development Research Institute, Hanoi, 2020. Page | 53 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) while all the LEG investments and part of the infrastructure investments aimed to increase agricultural productivity, the project did not seek to systematically measure these improvements in agricultural productivity. In addition, even though roads were the main investment of the project, accounting for 43% of total IDA project expenditures, the Results Framework did not collect evidence on the returns from road investments, and the road models are very sensitive to some of these assumptions. While the demand-driven nature of the project made anticipating which specific areas of production or measures of infrastructure improvements should have been monitored from the project’s beginning, after one or two years adjustments could have been made to the M&E system to start collecting information on the most common types of investments. 62 While some efforts were made in this direction, including the study carried out on the livelihoods activities, this assessment was insufficient to provide adequate evidence for a robust efficiency analysis. 8. In summary, the returns from the project as measured through the EFA are positive, but the evidence on which the EFA is based relies on limited data and assumptions of the benefits of project investments at the outcome level. The project efficiency is rated as SUBSTANTIAL given that: a) the results are robust enough to more conservative assumptions and b) not all project benefits could be quantified and as such the project benefits are likely to be higher than the more conservative estimates of the sensitivity analysis in practice. 9. Project costs and duration. The project lasted 65 months, as foreseen in the design. Project costs were lower than foreseen in the design, as the Government requested the cancelation of SDR 11.5 million in 2019. By project close, and as a result of exchange rate losses and the noted cancelation, total World Bank financing of the project was worth approximately US$118.4 million, or 80% of the original estimated financing. 10. Project management costs, amounting to 13.5% of total costs, were above original estimates due primarily to the reduction in overall project investments (in part given that management costs tend to have a substantial fixed element such as technical assistance or systems development, and tend to be front-loaded). However, the project’s administrative efficiency is still considered acceptable based on global CDD average project management costs.63 Indeed, higher management costs would have been expected for CHPov given its complex (multi-sectoral) nature and the challenging context, including geographically isolated areas and marginalized beneficiary groups, in which it operated. 11. Costs per beneficiary. The total number of households (HH) amounted to 141,897 (638,538 beneficiaries) compared to an initial target of 120,000 households (540,472 beneficiaries). Because project expenditures were also lower than planned, the cost per household was quite low, at US$880 per household compared to US$1,328 per household at design—which was closer to actual per beneficiary household costs of the Northern Mountain Poverty Reduction Project, P113493, (at US$1,294 per HH). While CHPov has significantly lower unit costs for LEGs, it is difficult to conclude if the support was more efficient in the Central Highlands region. Given the longer time period of implementation under the Northern Mountains project it is possible that the higher unit costs reflect an increased level of follow-up assistance provided to existing groups, thus increasing the per unit costs of support. It is also possible that with the greater overall coverage of households under Northern Mountains (82% of beneficiary households versus 45% under CHPov) the Northern Mountains project was reaching a greater percentage of the most isolated households that would have cost more to serve. However, the EFA was not able to further test these hypotheses. 62 Indeed, such was recommended at the MTR but was rejected by Government for reasons of audit challenges and workload of community facilitators. 63 Based on analyses done for World Bank Report No: ACS13685 Islamic Republic of Afghanistan; Strategic Directions for the National Solidarity Program, that looked at six large-scale and long-running national CDD programs. Page | 54 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Table 1. Costs by Component and Financier Table 2. Cost per Household and Beneficiary Unit Planned Actual Beneficiary districts Unit 26 26 Beneficiary communes Unit 130 130 Beneficiary households (HH) Unit 120,000 141,897 Beneficiaries Unit 540,472 638,538 Total cost per HH US$ 1,328 892 Total cost per beneficiary US$ 295 198 IDA cost per HH US$ 1,250 846 IDA cost per beneficiary US$ 278 188 12. Costs per beneficiary are higher for Component 2 than for Component 1, mostly because Component 1 beneficiaries are considered to include the entire population of villages having at least one infrastructure investment (regardless of type) throughout the project cycles. Table 3. Beneficiaries by Component Page | 55 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) 13. Error! Reference source not found. and 14. 5 show the scale and costs for community infrastructure projects and district infrastructure projects. They show that district level infrastructure projects were generally larger but also that they had much higher unit costs compared to community infrastructure. This is because the types of investments were different. For instance, roads at district levels tend to be roads designed to support larger/heavier vehicles thus requiring additional inputs. Table 4. Scale and Costs of Community Infrastructure Projects Cost per sub- Number of sub- Average scale Cost per unit, Cost per unit, Cost per sub- Community Infrastructure Beneficiary HH Unit project, million projects per sub-project million VND USD project, USD VND Roads 1,032 155,255 km 0.4 1,506 649 64,803 27,941 Schools 78 13,269 classrooms 1.3 381 479 16,404 20,610 Irrigation 90 10,883 ha 15.0 38 573 1,643 24,653 Water supply system 146 10,927 households 74.8 6 442 254 19,027 Other 16 2,555 1.0 927 927 39,905 39,905 Table 5. Scale and Costs of District Infrastructure Projects Cost per sub- Number of sub- Average scale Cost per unit, Cost per unit, Cost per sub- District Infrastructure Beneficiary HH Unit project, million projects per sub-project million VND USD project, USD VND Roads 141 94,893 km 1.6 2,268 3,557 97,607 153,113 School 5 647 classrooms 5 288 1,383 12,399 59,517 Irrigation 18 2,427 ha 21 126 2,673 5,421 115,043 Water supply 4 445 households 111 22 2,418 935 104,071 15. Costs per output appear to be high in comparison to the Northern Mountains Poverty Reduction Project, but this might reflect higher infrastructure costs in the Central Highlands or differing technical specifications, rather than an efficiency issue. . Discussions with the project engineer suggest that unit costs in Central Highlands were consistent with national standards and the lower costs in Northern Mountains were the result of lower technical standards. In addition, a comparison of road costs in Lao Cai (in the north) shows comparable unit costs. It is also noted that the data used to compare road costs are per linear kilometer, when a comparison of per square meter might be more appropriate to better see differences in technical specification. However, such data was not available at the time of the EFA. Economic and Financial Analysis Overview of the Methodology and Activities Modelled 16. This section presents the hypothesis and methodology used to construct the EFA of the CHPov in the context of the ICR. The objective of the analysis is to estimate the costs and benefits of the project from the perspective of project beneficiaries (financial analysis) and from the perspective of society and the economy (economic analysis), to assess the viability of the project. 17. This section will first introduce the methodology of the analysis, including the activity models used to represent project investments and the main assumptions behind these models. The section will then present the specific parameters and results of the financial analysis, followed by the specific parameters and results of the economic analysis. 18. The methodology is a cost-benefit analysis that estimates the costs and benefits of the project and the activities supported by the project. The data used for the analysis primarily come from the following sources: i) data provided by the Project Management Unit on outputs and costs per output; ii) data collected by an M&E consultant on the outcomes from infrastructure projects, mostly on irrigation; iii) business plans prepared as part of the project; iv) the Rapid Assessment of Livelihood Enhancement Group Outcomes; and v) discussions with Page | 56 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) various people involved in the project. 19. The benefits of the project primarily come from: • Component 1: Village and Commune Infrastructure Development; • Component 2: Sustainable Livelihoods Development, with the food security and nutrition and livelihood diversifications Livelihood LEGs sub-projects and the market linkage LEGs sub-projects; • The connective infrastructure in Component 3: Connective Infrastructure Development, Capacity Building and Communications. 20. To quantify some of the benefits of the project, activity models were prepared to represent the main investments enabled by the project and their associated benefits. For each model, a without project (WOP) situation is compared a with project (WP) situation. In the WOP situation, we consider the situation of beneficiaries prior to the project intervention while in the WP situation we consider the beneficiaries’ investments and activities enabled by project support. A model is considered profitable if the beneficiaries can derive more income following the project investments. Because all project activities were for groups, LEGs and communities, each model captures the number of households, as described in Error! Reference source not found., either based on the average number of households per type of LEG, or of the community. Table 6. Summary of Models MODELS WOP WP HH per model Livelihood Enhancement Groups Paddy rice Lower yield, lower inputs Training, higher yield and more inputs 15.0 Maize Lower yield, lower inputs Training, higher yield and more inputs 15.0 Poultry Seasonal labour Poultry 15.0 Pig breeding Seasonal labour Pig breeding 15.7 Goats Seasonal labour Goat rearing 15.1 Market Linkage Groups Sub-Projects Pineapple: market linkages Lower yield, lower inputs Training, higher yield and more inputs 46 Community infrastructure Irrigation non-irrigated land Higher yield and more double cropping 121 Roads No road Additional crop land, higher prices 150 District infrastructure District irrigation non-irrigated land Irrigation in place, higher yield and more double cropping 135 District road No road Additional crop land, higher yields and prices 673 21. The EFA is based on the assumption that the control group of the end-line survey benefitted from additional government investments and cannot be used as a counterfactual for the EFA. It is also based on the fact that the end-line data does not adequately measure improvements at the activity level, because it does not distinguish beneficiaries for each type of support received. In other words, while there is end-line data on rice yields, this data includes all the households that received project support, but we do not have the yields for the households that received rice LEGs sub-projects more specifically. Hence, while the baseline data is used, the end-line data is not used for the purpose of the EFA. Therefore, other sources of data were used to estimate the improvements due to the project investments. LEGs and Market Linkage Models 22. In the project, LEGs were categorized as Food Security and Nutrition LEGs or Livelihood Diversification LEGs for Sub-Component 2.1 and as Market Linkages LEGs for Sub-Component 2.2. Food Security and Nutrition LEGs were more likely to focus on paddy, maize and poultry while Livelihood Diversification LEGs were more likely to invest in pigs, goats and other crops. Hence, in the models, it is considered that the paddy and maize LEGs are Page | 57 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) undertaken by producers who already grow these crops. The livestock activities are considered as new activities, and the without project situation is simply seasonal labor, to reflect the additional labor time required for livestock activities. 23. LEGs refer to groups of activities while sub-projects refer to project-financed activities for each LEG. There were a total of 4,902 sub-projects – 4,885 under Sub-Component 2.1 and seventeen under Sub-Component 2.2 – and 4,383 LEGs – 4,337 under Sub-Component 2.1 and 46 under Sub-Component 2.2 (see Error! Reference source not found.). Of this total of sub-projects, some LEGs received two rounds of support-- 542 Food Security and Nutrition LEGs and 6 Income Diversification LEGs. Table 7. LEGs and Sub-Projects 24. Representative models were built for the most common LEG investments. These models aim to capture a typical sub-project, which would in practice vary based on the local context, number of beneficiaries and specificities of the project. The size of the LEG investments, described in units of production as shown in Error! Reference source not found., are based on project data compiled for the ICR, with some discrepancies, as shown in Error! Reference source not found.. The full costs of investments64 in the representative models are slightly higher than the project data investment costs, 17% higher on average, but this is considered acceptable given that business plans are likely to underestimate beneficiary contributions. It was not always possible to change the number of units of production in the models, because of lack of information on how costs might vary if units of production increase or decrease.65 Table 8. Production Units and Scale of the Models MODELS Unit Size Livelihood Enhancement Groups Paddy rice Hectares 6.4 Maize Hectares 10.6 Poultry Chickens 500 Pig breeding Pigs 43 Goats Goats 26 Market Linkage Groups Sub-Projects Pineapple: market linkages Hectares 12 64 Includes all the costs of the sub-projects for the first year, including training and excluding labor costs. 65 For instance, how the costs of a pig pen or a training would vary if the number of pigs or beneficiaries changes. Page | 58 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Table 9. Average Cost per LEG, VND Average cost per LEG sub-project, VND Cost in the model Project cost Rice 145,612,278 159,292,825 -9% Maize 179,725,696 159,292,825 13% Pig 264,318,311 139,396,454 90% Goat 120,082,515 125,855,685 -5% Poultry 134,212,658 136,057,717 -1% Average, modelled activities 168,790,291 143,979,101 17% 25. The without-project situation for the rice and maize models are based on the data on annual crops collected in the baseline survey. The yield reflects the yields observed in the baseline survey, an average of the poor household sample and the full sample. The budgets for fertilizer and seeds also reflect budgets of annual crops in the baseline survey. 26. The with-project situation reflects the business plans, with some adjustments. In particular, the yields for the rice and maize LEGs are more conservative than the target yields in some of the business plans collected, which included yields of 6 tons/ha for rice and 7 tons/ha for maize. On average in project provinces, the rice yields reach 5.4 tons/ha and the maize yields reach 5.3 tons/ha.66 Here, the analysis assumes that project beneficiaries go from yields below these averages (as confirms the baseline), showing that the project successfully target poorer households, to yields above these averages, as shows Error! Reference source not found.. Table 10. Yield Parameters for LEGs Yield parametres for LEGs, Kg/ha WOP, 1 WOP, 2-5 WOP, 6-10 WOP, 1 WP, 2-5 WP, 6-10 Rice 3150 3150 3150 4400 5500 5500 Maize 3200 3200 3200 4800 6000 6000 % increase Rice 40% 75% 75% Maize 50% 88% 88% 27. For the goat model, the Livestock Sector Investment Policy Toolkit67 was used to model a herd that would keep a more or less constant size, starting with 24 female goats and 2 male goats. For the pig and poultry models, all animals are assumed to be sold after a few months and no herds are modelled. The pig model considers that pigs are bought for breeding and that the parents are also eventually sold. The poultry model considers that chicks are purchased and sold for meat after three months. 28. Mortality rates were included in all livestock models. For pigs and poultry, no data was available and a standard mortality rate of 5% was assumed. For goats, the mortality rate was considered to be between 6% and 7%, depending on the age of the goat. The additional impact of the swine fever on pig mortality was modelled in the economic analysis, as will be explained in that section. Infrastructure Models 29. Investments for infrastructure are based on data provided on outputs provided by the project and costs per output. Error! Reference source not found. summarizes the production units and the scale of the models, based on these data. Annual maintenance costs are assumed to be 5% of the investment cost and the lifespans are 66 Simple average for all provinces. 67 See http://www.fao.org/3/ca7635en/CA7635EN.pdf. Page | 59 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) considered to be ten years. The benefits are assumed to start the year following the investment, but there are 10 years of benefits after the investment, so the models cover eleven years. Table 11. Production Units, Investment Costs and Scale of the Models Community infrastructure Unit Size Cost per Unit, VND Community infrastructure Irrigation Hectares 15.0 38,164,203 Roads Km 0.4 1,505,541,146 District infrastructure District irrigation Hectares 21.2 125,941,487 District road Km 1.6 2,267,677,413 30. For the irrigation models, the models are similar for the community irrigation scheme and the district irrigation, and only the size and cost of the investment scheme differ, based on project data. This does not reflect the fact that the benefits for district irrigation would be different in practice, because the underlying investment is more structural. 31. The benefits of the investment are the increase in yields due to better water management and the increase in the surfaces on which producers can grow two crops per season. The improvements in yields and increased crop surfaces are based on evidence from 13 different irrigation schemes in three provinces where paddy rice was primarily grown. The data revealed an increase in the planted area, from 91% of the area to 100% for the winter- spring crop and from 27% of the area to 75% for the summer-autumn crop. For the community irrigation, the model is based on paddy production. For district irrigation, it is based on both paddy and some pepper production in the summer-autumn crop. Yields also increased by 25% during the winter-spring crop and 12% for the summer- autumn crop for paddy, and by 40% for pepper.68 In addition, the irrigation schemes would have increased water conservation and reduced soil erosion, but these benefits were not quantified. 32. For the road models, the models are also similar for the community roads and the district roads, and only the size and cost of the investment differ, based on project data. As for irrigation, the investments would have been more varied in practice, but there was insufficient information to accurately capture the wide range of investments. As noted above, the evidence on the benefits was only anecdotal. As a result, the models are based on limited data and assumptions on the benefits that the roads enabled including i) an increase in yields of 10%; ii) an increase in products being sold of 50%, because of the increase in yields and the assumption that less frequent harvests prior to the roads led to below-potential harvested quantities; iii) a decrease in post-harvest losses for transported goods, from 10% to 5% and iv) savings from reduced transport time. The kilos of product are assumed to be valued at VND 7,000 per kilo, which is higher than the price of rice and maize per kilo, but lower than the price of cash crops such as coffee and cashew nuts. Financial Analysis Methodology and Assumptions 33. For the financial analysis, each model compares the costs and benefits of the beneficiaries’ activity with project and without project. This analysis aims to ensure that project beneficiaries were able to derive benefits from the proposed investments and to sustain the activities. In this project, the financial analysis was only done 68The evidence from the sites showed that paddy was mostly grown, with alternative crops including pepper and coffee. Because no data was available on production costs per hectare for pepper, the model assumes a constant production area with and without project. The production costs are therefore constant and the value of the additional yield of pepper is considered as additional revenue. The additional production area is therefore considered to be paddy rice. Page | 60 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) for the LEGs and the market linkages groups, because infrastructure investments are considered to be public investments. 34. In the financial analysis, each model was prepared with all the costs, including the training costs for the LEGs and the labor requirements and costs. Most costs are valued at market price. The production consumed by households is also valued at market price, and family labor is valued at the opportunity cost of labor, estimated at VND 80,000 per day. To compute the indicators of profitability for each model, the training costs and labor opportunity costs were excluded. 35. A discount rate of 10% was used for the analysis and all the benefits are assumed to start in year 1, together with the investments, because most of the project support for LEGs, training and inputs, would have to take place in the context of the activity. Results 36. The WP situations all have positive margins in the second year (after the initial investments), that range from US$63 per household for poultry to US$635 per household for goat rearing. Returns to labor are also higher than a day of work’s wages, which are about VND 120,000 per day. 37. When considering the incremental margins, which compare the WP situation to the WOP situation, all models are profitable with the exception of rice. According to the baseline survey, most households seemed to already use significant quantities of fertilizer on their rice. The marginal returns to additional inputs are likely to be low at that point. Hence, while yields increase as a result of the training and additional costs, the costs increase more, resulting in a lower margin. For the other models, the Net Present Value (NPV) of the additional margin ranges from US$7,478 per group in the maize model to US$ 31,442 for goat LEGs and US$16,768 for the pineapple market linkage group. The goats rearing model is particularly profitable because no feeding costs are included, as the households would typically cut and carry the food. The labor costs of this cut-and carry process are not included in the financial results, which is why the model seems more profitable. The pineapple market linkage group represents a higher-investment higher-return activity. The Internal Rates of Return are not particularly meaningful for the LEGs, as the benefits from the model are assumed to start in year 1 together with the investments69, so the initial cash flow is positive or only mildly negative in most models. Table 12. Results of the Financial Analysis Margin per Results of the Financial Margin per Return to Return to NPV, '000 HH (yr 2), NPV, USD IRR Analysis HH, USD labour, VND labour, USD VND VND Livelihood Enhancement Groups Paddy rice 2,659,950 114 152,871 7 (275,472) (11,857) NA Maize 5,070,000 218 281,667 12 173,731 7,478 181% Poultry 1,475,109 63 130,040 6 146,616 6,311 NA Pig breeding 3,483,571 150 248,246 11 167,820 7,223 87% Goats 14,764,180 635 275,522 12 730,473 31,442 NA Market Linkage Groups Sub- Projects Pineapple: market linkages 13,727,771 591 440,208 19 389,575 16,768 28% Economic Analysis 69 The investments correspond to inputs and a training, so they would be concomitant with the activity. Page | 61 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Methodology and Assumptions 38. The economic analysis compares the situation with-project to the situation without-project, similarly to the financial analysis. The additional benefits are the benefits attributed to the project. The analysis is done over a 10- year period with a discount rate of 5%, reflecting the interest rate on medium term Vietnamese bonds. 39. In addition to the models of the financial analysis, the infrastructure models were added to the economic analysis. 40. Economic prices were computed by removing taxes, subsidies and other transfers. A shadow exchange rate was computed using the formula below. All the economic models are based on economic prices. All the costs are included in the economic models, including training costs and the opportunity cost of labour. Figure 3 - Computation of the Shadow Exchange Rate Table 13. Conversion Factors Financial Economic Conversion Conversion Factors price/index price/index factor Import substitute or import 1.0 1.0 1.0 Import substitute or import 1.0 1.0 1.0 Export good 1.0 1.0 1.0 Non-tradable 1.0 0.9 0.9 Labour 120,000 80,000 1 Labour opportunity cost 80,000 80,000 1 FX 23,233 23,603 1 41. Following the conversion of financial prices into economic prices, the additional benefits of the modelled activities were aggregated in line with the activities of the project. In the case the Sub-Component 2.1, the modelled activities only represented about 55% of all the LEGs supported. Non-modelled activities included cattle and fish. In the aggregation, the non-modelled LEGs were included and distributed among the modelled LEGs, for a total of 4,337 LEGs, in line with the project output. The LEGs rather than the Sub-Projects were aggregated, to avoid double counting LEGs that received two rounds of sub-project support. 42. For market linkages, the phasing only accounts for twelve models/LEGs, even if the project output is 17. This was a necessary adjustment to avoid inflating the total investment in Component 2.2, because the investment in the modelled activity was higher than the average investment for the 17 sub-projects. 43. The phasing of activities represents project phasing where data were available. Where data on the years during which each investment was made was not available, in the case of the market linkage groups and the district infrastructure, the phasing assumes that activities were phased linearly from 2015 to 2019. Page | 62 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Table 14. Phasing of Project Activities PHASING 2014 2015 2016 2017 2018 2019 Total Livelihood Enhancement Groups - 728 1,040 1,072 521 976 4,337 Paddy rice - 53 76 78 38 71 316 Maize - 106 152 156 76 142 633 Poultry - 84 120 124 60 113 502 Pig breeding - 246 351 362 176 330 1,464 Goats - 239 341 351 171 320 1,421 Market Linkage Groups Sub- Projects - 239 341 351 171 320 1,421 Pineapple: market linkages - 3.0 3.0 3.0 3.0 - 12 Community infrastructure - - - - - - - Irrigation - 87 90 291 408 5 881 Roads - 87 122 466 271 86 1,032 District infrastructure District irrigation - 4.5 4.5 4.5 4.5 - 18 District road - 35.3 35.3 35.3 35.3 - 141 44. To take into consideration the fact that not all LEGs would succeed or be sustainable, a success rate was added. The success rates are 70% for the LEGs, with the exception of goat rearing, where 50% was used. This reflects the fact that goat rearing was both very popular and profitable among LEGs that successfully implemented the activity, but also that many households did not manage to sustain the activity and many goats died prematurely. The success rates for infrastructure are higher, 90%, as they require fewer commitments from beneficiaries. In the case of rice, it is considered that beneficiaries give up the activity70 after one year because it is not profitable. Hence, the benefits for that activity are overall negative. 45. In addition, for pig breeding, the aggregation shows a decrease of 19% of benefits from 2019 onwards. This captures the impact of the swine fever, which resulted in a decrease in the pig population of 19% compared to 2018. 46. Most project costs were already included as costs in the models (i.e.- training costs for LEGs, infrastructure costs, etc.). However, some costs, mostly training and capacity building in Component 3 and project management in Component 4, were not included. While no data was available on disbursement by activity, these additional costs are estimated to represent at least Sub-components 3.2, Sub-Components 3.3 and Component 4, so about 25% of total disbursements. These costs were added as cost to the final economic analysis. An additional VND 10 billion, approximately US$430,000, were considered as costs from 2021 to 2023, to account for potential future costs. 70 This is does not mean that they stop growing rice, but rather that they stop adopting the improvements suggested by the project. Page | 63 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Table 15. Adoption/Success Rates Adoption/success rates 2014 2015 2016 2017 2018 2019 Total LEG Paddy rice 70% 37 53 55 27 50 222 Maize 70% 74 106 109 53 100 444 Poultry 70% 59 84 87 42 79 352 Pig breeding 70% 172 246 253 123 231 1026 Goats 50% 119 170 176 85 160 711 Market Linkage Groups Sub-Projects Pineapple: market linkages 80% 2 2 2 2 0 10 Community infrastructure Irrigation 90% 78 81 262 367 5 794 Roads 90% 78 110 420 244 77 930 District infrastructure District irrigation 90% 4 4 4 4 0 17 Community road 90% 32 32 32 32 0 128 Results 47. The roads and irrigation models have positive returns, and community roads in fact account for 53% of total project benefits. Unfortunately, the model is not based on systematic data, and the model is very sensitive to assumptions, and in particular the 50% increase in products harvested and sold at the market. This is discussed in more detail in the sensitivity analysis. On the other hand, the models do not capture some benefits of the investments, including increased resilience from floods, water conservation and soil protection for the irrigation infrastructure. 48. Most of the benefits, 81%, are derived from the infrastructure projects. The LEGs and market linkage groups only account for 19% of benefits. This can be explained by a number of factors that include: i) infrastructures were the largest cost component of the project by far and ii) the lower success rates and sustainability of LEG groups (see for instance the rice models and the swine flu). 49. Using these assumptions, the project is found to have an economic NPV of VND 1,489 billion, equivalent to US$64.1 million. The EIRR is 29%. Error! Reference source not found. summarizes the economic benefits. 50. Because the EFA is so sensitive to assumptions in the road models, a sensitivity analysis was conducted to assess the impact of i) a higher price assumptions for goods sold at the market, of 10,000 VND per kilo and ii) a more limited increase in sales. The results of the analysis are summarized in Error! Reference source not found.7. In all scenarios, the EIRR is above the discount rate of 5%. In scenario 3, the district roads have negative returns, which impacts the overall project returns. Table 16. Sensitivity Analysis 51. One major assumption in the analysis is that all the roads are considered to be field to market roads, or at the very least to lead to increased yields and sales, while in practice the roads would be more varied. On the other hand, the community-driven approach implies that the roads would have been assessed as necessary for the Page | 64 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) community, with some kind of benefits. 52. The economic and financial analysis at design estimated that the project EIRR were 15%. The final EIRR is higher than this initial assessment. Table 17. Economic Analysis Results Page | 65 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Central Project Coordination Office comments on World Bank ICR Report, (draft version of 2020-08-14) General comments: 1. This is a good version of the report. GNTN is rated as Moderately Satisfactory (compared to the Moderately Unsatisfactory in May version). 2. We note the following ratings, which we agree with: a. Relevance of PDO: Substantial b. Achievements of PDO: Substantial (outcome 1: Substantial; outcome 2: Substantial; outcome 3: Modest) c. Efficiency: Substantial 3. Very good lesson-learned and recommendations (which we endorse): a. At design: clarification of indicators and sub-indicators in the RF b. At implementation: MIS, agriculture advisory services c. At post: sustainability of infrastructure works after the project completion d. Policy level: programmatic approach/sectoral support vs a single project Detailed comments: • Page 7, Table 3, No. 5: Results not considered. Comment: It should be treated similarly as No. 8 (no. LEGs formed) • Page 7, Table 3, No. 5: Partially achieved. Comment: Definition of “Partially achieved” and “Not achieved” should be added and consistently used. • Page 8, para #27: Over the course of the 5-year implementation phase, the overall average of planned versus completed infrastructure projects stood at …%. Comment: 76.24% (per CPO calculation) • Page 28, Annex 1, RF #5: Not measured. Recommendation: WB & CPO can provide number of planned sub-projects and number of completed sub-projects as an alternative way of measurement • Page 28, Annex 1, RF #6: YES (overall avg. 32.1%). Comment: Again, definition for “Yes” and “Partial” should be added and consistently used. • Page 29, Annex 1, RF #7: 25%. Comment: Increase percentage Page | 66 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) • Page 29, Annex 1, RF #7: YES. Comment: Cannot be “Yes” • Page 29, Annex 1, RF #8: YES. Comment: As a comment above, this is also “no target”, but measured • Page 29, Annex 1, RF #12: 60,384. Comment: Pls refer to PCR: client days: 56,961 (person days); women = 29.42%. • Page 29, Annex 1, RF #12: 2,902. Comment: Pls refer to PCR: total number of classes organized = 542; Only take into account activities funded by SC 3.2 • Page 29, Annex 1, RF #13: 0%. Comment: Pls refer to PCR: delayed/cancelled procurement = 1.32% • Page 31, para #2: However, the baseline and end-line surveys interpreted this indicator differently and sought to compare changes in development priorities as an indication that priorities have been met. Comment: It is not clear to the CPO what this statement means. • Page 33, para 8: the above indicator is not considered to have been achieved. Comment: This conclusion is different from the No.4 in RF table. • Page 33, Table 3: Number of infrastructure sub-projects implemented. Comment: WB and CPO now can provide this data. Therefore, the roughly calculated figure is 76.24% completed vs 100% planned. Page | 67 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) ANNEX 6. CHPOV IMPACT EVALUATION SUMMARY71 1. Per the requirements of the IDA-16 impact evaluation initiative, a rigorous impact evaluation (IE) was developed for CHPov. The evaluation of the project’s impact uses a difference in difference with matching methodology. Propensity score matching is used first to select comparison communes within the project provinces and second to generate household weights in the difference in difference regressions to estimate project impacts on outcomes. The analysis is performed using a panel data set from a baseline survey conducted in 2014/15 and an end line survey conducted at the end of 2019. Evaluation design 2. With project intervention areas purposefully selected, the impact evaluation follows a non-randomized clustered evaluation design. The commune is the treatment unit given that resources went to communes or village level subprojects which benefited all households in the area. The evaluation treatment group includes all 130 project communes to maximize the statistical power and representativeness of estimates. We apply statistical matching procedures to select control communes given that a regression discontinuity design is not practical. 3. The process of selecting project communes presents district-level and commune-level discontinuities in principle, but a regression discontinuity design is not an appealing evaluation strategy because of a large bandwidth problem. Applying the district-level discontinuity selects 26 districts that have a higher poverty rate than project communes but have the lowest rate of poverty among the 54 non-CHPov project districts in the six provinces. Applying the commune-level discontinuity selects five communes that have the highest aggregate selection score among the seven non-project communes in each of the 26 districts. Identification in regression discontinuity designs requires communities on either side of the threshold to be comparable in all dimensions, except the treatment. This is unlikely to be satisfied in either case given the proportionately large bandwidths. Poverty rates in the 26 poorest districts ranged from 43 percent to 86 percent, and those in the 26 poorest in non- project districts ranged from 27 to 43 percent for example. Narrowing the bandwidth to solve this problem significantly reduces the sample size, external validity, and statistical power. This makes a regression discontinuity design less attractive compared to statistical matching techniques. 4. Instead, the evaluation selects the comparison communes using propensity score matching (PSM). This approach approximates randomization through statistical modelling of the probability of a commune being assigned to a program, i.e. the propensity score, to identify non-project communes similar to project communes in observable characteristics. Pairs of treatment and non-treatment communes are matched based on closeness of their propensity score. A critique of PSM is that its accuracy is limited by the existence of non-program units fundamentally comparable to program units and the ability of the propensity score model to identify them. Slight misspecifications of the statistical model can result in substantial bias of estimated treatment effects for example (Kang and Schafer, 2007 and Smith and Todd, 2005). Checking for balance between treatment and comparison groups in relevant baseline indicators, then adjusting the model to improve balance, and re-match, can guard against such misspecifications but it also can aggravate the reduction of bias in estimated treatment effects if improving balance on some covariates increases bias in others (Hainmueller, 2011 and Imai and Ratkovic, 2013). 5. The evaluation applied the nearest neighbor propensity score matching to select 130 comparison communes. We estimate the propensity score by running a logit regression model on project and non-project communes in the six CHPov provinces on data from the Rural Agriculture and Fishery Census (RAFC) conducted in 71 This annex is drawn from the World Bank report “Impact Evaluation of the Central Highlands Poverty Reduction Project in Vietnam”. World Bank. 2020. Page | 68 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) 2011. This data clearly shows that treatment communes have a significantly higher share of ethnic minority population, higher incidence of poverty and more remote but less populated than unmatched non-project communes, hence the need to carefully select a comparable set of communes for the evaluation. In addition to variables factored in project commune selection criteria (i.e. poverty rate, ethnic minority population share and no plan for hydropower development in the commune), outcome variables and characteristics expected to affect project outcomes are included along with other geographic data.72 Primary outcomes 6. The evaluation focuses on those project results indicators measured based on changes compared to non- project areas as outlined in the project document. These indicators are: • Proportion of poor villagers whose identified development priorities are satisfied • Percent change in food and non-food consumption of poor households • Percent change in access of poor households to services, infrastructure, and utilities • Percent change in productive and durable assets of households participating in LEGs • Percent change in dietary diversity of poor households. • Percent change of transport connectivity index. Data and methods 7. The evaluation uses panel data from the baseline and end line surveys funded by the project to measure the project results indicators, including those derived from this evaluation. The baseline survey was conducted between December 2014 and January 2015 before the full roll-out of project activities. The end line survey was conducted between October and November 2019, just before the project completion date of December 2019, which was also the deadline for completion of all project funded activities, including this survey. 8. Data collection consisted of two sets of instruments. One set is the Household Questionnaires with variants for male and female respondents. This collected information on household characteristics, education, health, income sources, agricultural production, employment, consumption and households’ assets and production tools. It also included questions on access to basic public services, satisfaction with local infrastructure, and participation in the decision-making process for selecting infrastructure works. The other set of instruments was the Individual Local Leader Questionnaires administered to a Commune People’s Committee representative and a village-level representative. Both gathered the respondents’ general information (such as age, gender, ethnicity, education and position details), access to basic services and amenities (like education, health care, markets, administrative center) in the commune/village and support programs to the commune/village two years prior to the baseline (for the baseline survey) and during 2014-19 (for the end line survey). Both the baseline and end line surveys applied the same questionnaires except for some sections where questions not relevant for measuring the outcome indicators were trimmed. The end line survey also included new questions to capture participation in LEGs. Sampling strategy 9. The surveys were implemented in 130 project and 130 matched non-project communes with the sample equally split between them. A two-stage sampling process was followed to select households. At the first stage, 780 villages were selected by systematic, equal probability sampling by ordering villages in each commune by their 72 Including population, land area, percent of land allocated to annual and perennial crops and forestry; major and minor road density, major and minor river density, and average distance to commune, district and province center and mean and variances of: elevation; slope; rainfall; temperature; and sunshine hours and the existence of development projects and programs by the government and other donors. From a long list of potential covariates, the final model was estimated using backward selection logit regressions to keep variables that are statistically significant at the 10% level (e.g. Caliendo and Kopeinig, 2008). Page | 69 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) village code and systematically selecting three villages in each commune. Three instead of just one village were selected per commune to reduce the intra-cluster correlation. A new listing of households in the 780 villages was conducted, gathering information on names of the spouse and head of household, ethnicity of the household head and whether members of the household spoke the Kinh (majority) language. The listing covered a total of 87,878 households. Listed households from each commune were first ordered by ethnic grouping codes (for indigenous ethnic minorities, migrant ethnic minorities and Kinh), second by village code and then by household code. Then a systematic sample of 14 households was selected from the ordered list of households. To achieve an equal gender split, households were randomly assigned a gender of the respondent to have 7 males and 7 females’ respondents per village, with respondents being either a head-of-household, the spouse of head-of-household or a responsible adult. 10. The baseline sample consists of 3,640 households but the end line survey sample only included 2,600 households due to budget limitations. We randomly selected 10 out of the 14 households per village in the baseline survey for interviewing in the end line survey. This is better than selecting paired household using propensity score matching for three reasons. First, 72 percent of the control households would be selected regardless, meaning less gains from matching. Second, some of the remaining 28 percent control households would serve as replacements due to panel attrition, ruining the pairing and thus reducing the gain from PSM. Lastly, PSM could have resulted in uneven sample sizes across communes with some of the 130 control communes ending up with too few or no households at all in the sample. Results 11. The impact evaluation, based on a difference in difference approach combined with propensity score matching methodology, portrays mixed results on the impact of the project (Table 1). Estimates suggest the project improved household dietary diversity by 6.3 percent relative to the non-project communes among poor households and by 3.5 percent for all households. But no impact is detected on the other aggregate primary outcomes of the project namely, productive assets owned, transport connectivity, access to services, infrastructure, and utilities or share of villagers’ investment priorities that were fulfilled. These outcomes significantly improved in both project and non-project areas, but the gains were not statistically different between the CHPov project and non-project communes. Table 1: Summary of project impacts on primary outcome indicators Baseline End line Change Project Development Outcomes Value Value relative control group Poor villagers’ development priorities satisfieda - - 1.1 % Food consumption of poor households (VND ‘000) 3769 6083 4.1 % Non-food consumption of poor households (VND ‘000) 2831 5415 2.9% Access of poor households to services, infrastructure and 0.03 0.02 0.03 utilities (index: 0 – 1) Productive and durable assets of households participating in 42.9 49.9 -5.3% Livelihood Enhancement Groups (index: 0 – 100) Dietary diversity of poor households (index: 0 – 1) 6.7 8.1 6.5%** 2.1 percentage Transport connectivity index (index: 0 – 100) 55.2 60.5 points Notes: The satisfaction indicator is defined as the share of villages who expressed preference for a difference development priority between the start and end of the project. Page | 70 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) 12. Some positive impacts on sub-indicators on access to services and infrastructure are found, as well as reranking of some priorities and improvements in key livelihood outcomes. Positive impacts of the project are detected on increased accessibility to markets for poor households and accessibility to the commune center and district center for all households in project communes relative to comparison non-project communes. A re-ranking of some of villagers’ top priorities relative to the control groups is observed, suggesting that some of the villagers’ preferences were at least fulfilled. The share of villagers citing inter-commune roads as a top priority investment declined by more than 6 percent relative to the control group, as investment in health centers and electricity moved up the priority list. There was increased community participation in project selection by 4 percent (or a third of the control baseline outcome standard deviation), which could have positively influenced the allocation of investment to different sub-projects. The difference in difference estimates show that households’ average incomes from primary wage employment increased by 18.2 percent and total household wage incomes by 7.7 percent. This was a result of workers – mostly men - switching to wage jobs as a primary rather than a secondary occupation. These findings imply that the project strengthened household non-farm participation and livelihood diversification. 13. The absence of impact on aggregated primary outcomes is partly attributed to significant investments made under the National Targeted Programs (NTPs) that also benefited control communes. The amount of infrastructure investments in the CHPov project communes was only 10 percent higher than investments in similar infrastructure in non-project communes (see table 2 below). However, the spending patterns of CHPov project and non-project communes were different, with the former investing more in clean water systems and commune roads, which translated into positive impacts on clean water and connectivity to markets and the commune center. Nutrition interventions, where the dietary diversity indicator shows an impact, is another area the CHPov project devoted more resources through livelihood enhancement groups, than the NTP program in a typical commune. 14. While the results on the impact of the project are mixed, there are some notable lessons this evaluation provides. First, the evaluation demonstrates that the CHPov implementation model significantly increased wage earnings overall, especially from primary wage jobs. Thus, it could serve as an example for enhancing non-farm wage incomes for households in remote areas. Second, its model for community participation, while not tremendously successful, was an improvement compared to non-project communes. This both offers lessons for improvements, while revealing the big challenge that remains regarding improving community participation in community development programs in Vietnam. Lastly, differences in impacts between project and non-project communes mirror differences in spending on different types of investments implying that positive outcomes can be achieved with more targeted investments. Thus area-based poverty reduction policies still have a role to play in the development process in Vietnam. 15. The end-line survey73 demonstrates that those communes that received project support experienced substantial improvements across the range of key outcome measures as compared to the baseline. This includes a 48% change in overall consumption of poor households (and 30% increase for food); a 2.3% change in overall access to services by the poor (and larger specific improvements in access to community health facilities, improved water sources, and agriculture support services); an increase of 2.8% for the general beneficiary population in the project’s connectivity index (with larger improvements for access to primary schools, commune health facilities, the nearest market and the district center, as well as in general for poor households); a 13.8% increase in satisfaction levels with project support; a 9.3% increase in household durable assets; and a 21.4% increase in dietary diversity. 73 The Central Highlands Poverty Reduction Project Endline Report; Mekong Development Research Institute, Hanoi, 2020. Page | 71 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Table 2. Comparison of treatment and control communes’ infrastructure projects and total investments received since 2015 The percentage of communes having The total investment that communes Type of project the following project since 2015 received since 2015 (million VND) Treatment Control Difference Treatment Control Difference Inter-village road 86.8 88.5 -1.7 5295.5 4649.6 646.0 (3.0) (2.8) (4.1) (605.2) (636.9) (878.6) Inter-commune road 55.0 49.6 5.4 1946.2 2154.3 -208.0 (4.4) (4.4) (6.2) (497.3) (713.6) (869.8) Inter-farm road 67.4 55.0 12.5** 2409.0 1440.0 969.0** (4.1) (4.4) (6.0) (360.1) (285.8) (459.7) Irrigration works 72.1 70.2 1.9 1183.3 1960.9 -777.6 (4.0) (4.0) (5.6) (195.8) (449.3) (490.2) Market 7.0 12.2 -5.2 125.6 207.5 -81.9 (2.3) (2.9) (3.6) (86.5) (82.8) (119.7) Kindergarten 79.8 78.6 1.2 1207.2 1009.7 197.5 (3.5) (3.6) (5.0) (184.1) (141.8) (232.4) Primary/secondary school 82.2 71.0 11.2** 1980.0 1705.0 275.0 (3.4) (4.0) (5.2) (336.6) (319.6) (464.2) Village cultural house 71.3 80.9 -9.6* 1040.2 908.6 131.6 (4.0) (3.4) (5.3) (125.1) (107.2) (164.8) Commune health's center 51.2 58.0 -6.8 509.2 854.2 -345.0* (4.4) (4.3) (6.2) (93.2) (155.1) (180.9) Clean water system 62.0 56.5 5.5 1370.6 966.6 404.0 (4.3) (4.3) (6.1) (269.6) (203.0) (337.5) Electricity grid 57.4 56.5 0.9 592.0 839.3 -247.2 (4.4) (4.3) (6.2) (184.8) (303.1) (355.0) Commune people's committee 51.9 50.4 1.6 1271.1 1455.0 -183.9 (4.4) (4.4) (6.2) (194.3) (259.6) (324.2) Bridge/Drain 68.2 61.9 6.4 1053.0 808.0 245.0 (4.1) (4.3) (5.9) (211.0) (173.5) (273.2) All projects 19982.9 18958.6 1,024.4 (1618.0) (1634.8) (2,300.1) Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 Source: Baseline and Endline data analysis, CHPov Project, MDRI, 2019 Page | 72 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) ANNEX 7. CHPOV END-LINE SURVEY; KEY DATA TABLES74 Table 1: Project’s Results Framework Summary Table 2014 2019 Change INDICATORS Indicator 2: Proportion of poor villagers whose identified development priorities are satisfied % of poor villagers that identified corresponding infrastructure as one of the top three development priorities Inter-village road (%) 60.8 44.4 -16.4 Clean water works (%) 44.6 39.2 -5.4 In-field road (%) 32.6 41.7 9.1 Irrigation works (%) 29.0 33.1 4.0 Kindergarten (%) 22.9 15.2 -7.7 Village cultural house (%) 20.7 8.3 -12.4 Market (%) 18.4 9.6 -8.8 Inter-commune road (%) 16.8 20.2 3.4 Electricity (%) 16.2 0.6 -15.6 Bridge/Drain (%) 14.2 10.5 -3.7 Commune health center (%) 12.2 7.4 -4.8 Primary/Secondary school (%) 11.6 11.2 -0.4 Indicator 3: Percent change in food and non-food consumption of poor households Total annual expenditure per capita (thousand VND) 8,238.5 12,191.7 48.0% Food consumption (thousand VND/person/year) Holiday food and drink 313.4 414.6 32.3% Daily food and drink 4,118.9 5,358.2 30.1% Non-food consumption (thousand VND/person/year) Daily non-food expenditure 1,396.6 1,910.7 36.8% Annual consumption of other non-food 769.4 1045 35.8% Other cost as expenditure 567.4 1,363.1 140.2% Education expenditure 288.2 692.2 140.2% Health expenditure 784.5 1,407.9 79.5% Indicator 4: Percent change in access of poor households to services, infrastructure, and utilities Revised access index 38.6 39.5 2.3% Education (%) Kindergarten within 15-minute walk 80.6 80.8 0.2 Primary school within 30-minute walk 85.1 82.1 -3.0 Secondary school within 30-minute walk 63.9 65.0 1.1 Healthcare (%) Commune health facility within 20-minute drive 66.7 70.2 3.5 Immunizations availed at commune/village health 14.5 19.9 5.4 center in past year Pregnancy check-ups availed at commune/village 4.9 4.0 -0.9 health center in past year Health check-ups availed at commune/village health 45.7 27.4 -18.3 74 The Central Highlands Poverty Reduction Project; Endline Report. Mekong Development Research Institute. Hanoi 2020. Page | 73 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) 2014 2019 Change INDICATORS center in past year Household visited commune/village health facilities in 22.4 33.2 10.8 past year for other purposes Water (%) Access to improved water source 49.9 58.4 8.5 Water supply is sufficient (given improved source) 36.4 40.4 4.0 Agriculture (%) % of land irrigated by State-funded irrigation system 18.7 22.0 3.3 % of household received agricultural support within 13.7 25.5 11.8 past year Indicator 6: Percent change in proportion of villagers satisfied with support received from Project Total satisfaction level (%) % of households satisfied with selection of 72.9 83.7 10.7 infrastructure projects % of households satisfied with quality of those 58.9 64.9 6.0 infrastructure projects % of households satisfied with quality of local roads 44.6 49.0 4.4 % of households satisfied with irrigation system in the 29.4 61.8 32.4 village/commune Indicator 9: Percent change in productive and durable assets of households participating in LEGs Revised asset index 51.4 56.2 9.3% Composition % of ohouseholds having motorbike 81.6 90.5 8.9 % of households having water pump 40.2 55.5 15.3 % of households having mobile phone 84.8 89.5 4.7 Land for annual crops (log m2) 6.7 6.4 -4.4% Garden land (log m2) 7.8 7.8 0% % of households accessing to irrigated system in 66.9 70.4 3.5 commune % of households raising pig 30.1 37.0 6.9 % of households raising poultry 59.2 74.1 14.9 % of households raising cattle 25.7 50.2 24.5 Indicator 10: Percent change in dietary diversity of poor households Dietary diversity index 0.6 0.7 21.4% Composition (%) Cereals 100.0 99.9 -0.1 Tubers and roots 31.7 34.3 2.6 Vegetable 90.6 99.1 8.5 Fruits 29.2 55.3 26.1 Meat 43.4 67.0 23.6 Fish and seafood 71.1 77.7 6.6 Eggs 39.1 63.6 24.5 Bean and nuts 9.1 15.7 6.6 Milk 47.6 58.7 11.1 Sugar and molasses 38.9 64.8 25.9 Page | 74 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) 2014 2019 Change INDICATORS Oils and fat 89.7 96.5 6.8 Other foodstuffs 79.4 88.1 8.7 Indicator 11: Percent change in transport connectivity index Revised transport connectivity index 68.2 70.1 2.8% Accessibility to social infrastructures Kindergarten 3.1 3.1 1.0% Primary school 2.6 2.8 9.7% Commune health facility 2.5 2.7 9.4% Accessibility to key destinations Nearby market 2.2 2.4 6.8% Commune people’s committee 3.0 3.1 4.7% District center 1.4 1.5 7.4% Provincial center 2.2 2.0 -9.6% Page | 75 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Table 2. Household participation in local activities (%) Participate in local meeting Express opinion in the meeting Supervise construction work 2014 2019 Change 2014 2019 Change 2014 2019 Change Total 55.8 72.1 16.3 42.7 46.7 4.0 4.5 35.9 31.4 Household head's gender Male 57.4 73.7 16.4 43.4 48.2 4.9 4.7 37.1 32.4 Female 46.4 63.3 16.9 38.3 37.2 -1.1 3.9 29.8 25.9 Household head's ethnic Kinh/Hoa 60.1 72.9 12.8 46.8 58.9 12.1 5.2 26.4 21.2 IEM 55.7 75.1 19.4 36.6 39.5 2.9 4.3 43.3 39.0 MEM 49.7 61.6 12.0 56.3 54.4 -1.9 4.4 26.2 21.8 Poverty status Poor 49.9 70.5 20.6 31.8 38.6 6.8 3.4 39.6 36.2 Non-poor 61.0 73.6 12.6 50.8 54.1 3.3 5.6 32.5 26.9 Table 3. Women participation in local activities (%) Participate in local meeting Express opinion in the meeting 2014 2019 Change 2014 2019 Change Total 30.9 35.7 4.8 22.4 32.4 10.0 Household head's gender Male 25.5 30.1 4.6 16.8 26.6 9.8 Female 70.2 71.6 1.4 67.7 80.4 12.65 Household head's ethnic Kinh/Hoa 33.4 36.3 3.0 27.3 33.4 6.1 IEM 33.8 37.4 3.6 23.3 35.8 12.45 MEM 17.2 28.3 11.1 13.4 21.7 8.32 Poverty status Poor 35.1 38.9 3.8 21.7 37.7 16.04 Non-poor 27.8 32.8 5.0 22.7 29.0 6.31 Page | 76 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Table 4. Satisfaction of households in the project areas (%) Selection of infrastructure projects Quality of those infrastructure projects 2014 2019 Change 2014 2019 Change Total 76.5 94.2 17.7 68.8 81.9 13.1 Household head's gender Male 76.8 94.8 18.0 69.6 82.4 12.8 Female 74.7 90.9 16.2 64.0 79.1 15.1 Household head's ethnic Kinh/Hoa 82.4 97.5 15.1 72.7 79.8 7.1 IEM 73.7 93.2 19.5 67.6 84.5 16.9 MEM 76.5 92.9 16.4 66.9 76.8 9.9 Poverty status Poor 72.0 92.5 20.5 65.9 82.6 16.7 Non-poor 80.6 95.9 15.3 71.4 81.3 9.9 Quality of local roads Irrigation system in the village/commune Agricultural extension activities 2014 2019 Change 2014 2019 Change 2014 2019 Change Total 56.2 71.1 14.9 29.4 44.2 14.8 13.9 21.8 8.0 Household head's gender Male 54.9 69.5 14.7 28.2 44.7 16.5 14.7 22.8 8.1 Female 64.2 79.9 15.7 36.7 41.3 4.6 9.1 16.1 7.1 Household head's ethnic Kinh/Hoa 51.7 62.0 10.3 22.7 38.4 15.7 16.6 20.6 4.0 IEM 65.2 79.2 14.0 36.3 49.2 12.9 14.7 23.3 8.6 MEM 36.0 58.7 22.6 18.5 36.7 18.2 7.4 18.8 11.4 Poverty status Poor 58.5 75.8 17.3 33.5 48.1 14.6 11.8 22.5 10.7 Non-poor 54.2 66.7 12.5 25.8 40.5 14.7 15.7 21.1 5.4 Page | 77 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Table 5. Travel time from home to school (minute) Primary Lower secondary Upper secondary 2014 2019 Change 2014 2019 Change 2014 2019 Change Total 10.5 7.5 -28.7% 14.2 10.3 -27.5% 21.6 14.8 -31.8% Household head's gender Male 10.6 7.4 -30.5% 14.0 10.1 -28.2% 21.5 14.5 -32.5% Female 10.2 8.5 -17.2% 14.9 11.4 -23.5% 22.2 16.0 -27.8% Household head's ethnic Kinh/Hoa 9.3 7.7 -17.4% 10.8 9.1 -15.7% 15.6 12.6 -19.4% IEM 10.6 6.9 -34.7% 15.7 10.2 -34.9% 24.4 14.8 -39.3% MEM 12.2 9.2 -24.2% 14.2 11.9 -15.9% 21.8 17.5 -19.5% Poverty status Poor 11.0 7.5 -31.8% 17.0 11.3 -33.8% 26.0 16.0 -38.7% Non-poor 10.1 7.5 -25.5% 11.6 9.3 -19.6% 17.7 13.7 -23.0% Table 6. Travel time from household to commune health center (minute) 2014 2019 Change Total 20.6 15.2 -26.3% Household head's gender Male 20.6 15.0 -27.2% Female 20.6 16.2 -21.2% Household head's ethnic Kinh/Hoa 16.3 13.8 -15.3% IEM 21.6 14.7 -31.9% MEM 23.7 18.5 -22.0% Poverty status Poor 24.8 16.9 -31.9% Non-poor 16.9 13.7 -19.0% Table 7. Household perception about quality of in-field roads (%) Page | 78 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Positive responses Very good Good Neutral 2014 2019 Change 2014 2019 Change 2014 2019 Change Total 0.6 4.6 4.0 9.2 21.8 12.6 17.7 27.3 9.6 Household head's gender Male 0.6 4.5 3.8 8.6 21.8 13.2 17.5 26.1 8.6 Female 5.3 5.3 12.5 21.8 9.3 18.9 33.7 14.8 Household head's ethnic Kinh/Hoa 0.8 3.5 2.7 14.4 21.6 7.1 17.4 26.1 8.7 IEM 0.5 6.3 5.8 8.2 22.9 14.8 18.6 25.8 7.2 MEM 0.2 0.7 0.5 4.7 18.6 13.9 15.5 33.7 18.2 Poverty status Poor 0.4 5.4 5.1 7.7 23.2 15.5 19.4 25.4 6.1 Non-poor 0.7 3.8 3.1 10.5 20.5 10.0 16.2 29.1 12.9 Negative responses Bad Very bad 2014 2019 Change 2014 2019 Change Total 33.3 24.4 -8.9 39.3 21.9 -17.4 Household head's gender Male 33.6 24.7 -8.9 39.7 23.0 -16.7 Female 31.5 22.9 -8.6 37.2 16.4 -20.7 Household head's ethnic Kinh/Hoa 28.4 27.2 -1.2 39.0 21.6 -17.4 IEM 34.8 23.2 -11.6 37.9 21.7 -16.2 MEM 35.6 24.1 -11.5 44.0 22.9 -21.0 Poverty status Poor 32.9 24.1 -8.8 39.7 21.8 -17.9 Non-poor 33.7 24.7 -9.0 38.9 22.1 -16.9 Table 8. Household perception about quality of inter-village roads (%) Page | 79 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Positive responses Very good Good Neutral 2014 2019 Change 2014 2019 Change 2014 2019 Change Total 1.1 7.4 6.3 20.2 37.3 17.1 29.5 32.7 3.3 Household head's gender Male 1.2 7.5 6.3 19.9 37.4 17.5 28.9 32.6 3.6 Female 0.6 7.0 6.4 21.8 36.7 14.9 32.5 33.6 1.1 Household head's ethnic Kinh/Hoa 0.5 3.6 3.1 19.7 33.9 14.3 28.6 32.3 3.7 IEM 1.7 10.0 8.4 24.8 42.3 17.5 28.9 30.8 1.9 MEM 0.4 4.4 4.0 7.0 26.2 19.2 32.3 39.1 6.8 Poverty status Poor 0.9 10.1 9.2 22.2 38.2 16.0 29.9 32.1 2.2 Non-poor 1.3 4.8 3.5 18.3 36.3 18.1 29.1 33.4 4.3 Negative responses Bad Very bad 2014 2019 Change 2014 2019 Change Total 30.4 14.9 -15.5 18.9 7.8 -11.1 Household head's gender Male 31.0 15.0 -16.0 19.0 7.6 -11.3 Female 26.6 14.2 -12.4 18.5 8.5 -10.0 Household head's ethnic Kinh/Hoa 29.5 18.5 -11.0 21.7 11.7 -10.1 IEM 28.9 12.0 -17.0 15.7 4.8 -10.9 MEM 36.0 18.8 -17.2 24.4 11.5 -12.8 Poverty status Poor 28.9 13.7 -15.2 18.1 5.9 -12.2 Non-poor 31.8 16.0 -15.8 19.6 9.5 -10.1 Table 9. Household perception about quality of inter-commune roads (%) Page | 80 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Positive responses Very good Good Neutral 2014 2019 Change 2014 2019 Change 2014 2019 Change Total 2.6 8.4 5.9 37.2 39.6 2.5 30.1 30.3 0.2 Household head's gender Male 2.7 8.1 5.4 36.0 39.9 3.8 30.1 29.2 -1.0 Female 2.0 10.4 8.4 43.8 38.4 -5.3 29.9 36.6 6.8 Household head's ethnic Kinh/Hoa 1.6 5.5 3.9 32.3 34.0 1.7 28.8 29.0 0.2 IEM 3.7 11.8 8.0 44.8 46.5 1.6 28.7 29.7 1.0 MEM 0.4 2.3 1.9 21.2 26.4 5.2 36.1 33.9 -2.1 Poverty status Poor 3.2 11.2 8.1 39.8 43.1 3.3 31.4 29.9 -1.5 Non-poor 2.0 5.8 3.8 34.8 36.4 1.6 29.0 30.8 1.8 Negative responses Bad Very bad 2014 2019 Change 2014 2019 Change Total 21.3 13.7 -7.6 8.9 7.9 -1.0 Household head's gender Male 22.0 14.6 -7.3 9.2 8.3 -0.9 Female 17.5 8.9 -8.6 6.9 5.7 -1.2 Household head's ethnic Kinh/Hoa 26.0 15.3 -10.8 11.3 16.3 5.0 IEM 17.8 9.4 -8.4 5.0 2.7 -2.3 MEM 25.1 25.0 -0.1 17.2 12.3 -4.9 Poverty status Poor 18.5 10.3 -8.3 7.2 5.6 -1.6 Non-poor 23.8 17.0 -6.8 10.4 10.1 -0.4 Page | 81 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Table 10. Household perception about quality of inter-district roads (%) Positive responses Very good Good Neutral 2014 2019 Change 2014 2019 Change 2014 2019 Change Total 3.4 10.0 6.6 43.4 45.5 2.1 29.0 23.8 -5.1 Household head's gender Male 3.4 10.2 6.9 42.3 45.6 3.3 29.0 23.3 -5.7 Female 3.7 8.8 5.1 49.7 44.7 -5.1 28.8 26.8 -2.0 Household head's ethnic Kinh/Hoa 2.5 4.9 2.4 36.5 36.2 -0.3 27.3 24.9 -2.4 IEM 4.4 14.4 10.1 51.8 52.4 0.6 26.8 22.2 -4.7 MEM 1.8 3.4 1.6 28.2 36.7 8.5 37.7 27.5 -10.2 Poverty status Poor 3.8 13.6 9.7 45.5 48.0 2.5 31.7 24.4 -7.3 Non-poor 3.0 6.7 3.7 41.5 43.0 1.6 26.6 23.3 -3.2 Negative responses Bad Very bad 2014 2019 Change 2014 2019 Change Total 18.7 12.1 -6.6 5.5 8.6 3.0 Household head's gender Male 19.5 12.1 -7.4 5.8 8.8 2.9 Female 14.1 12.4 -1.8 3.6 7.3 3.7 Household head's ethnic Kinh/Hoa 23.5 13.7 -9.8 10.3 20.3 10.0 IEM 14.5 8.8 -5.7 2.6 2.2 -0.3 MEM 24.7 20.4 -4.4 7.6 12.1 4.5 Poverty status Poor 14.5 9.0 -5.4 4.5 5.0 0.5 Non-poor 22.5 15.1 -7.5 6.4 11.9 5.5 Page | 82 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Table 11. Travel time from household to public administrative centers (minute) Commune People's Committee District Center Provincial center 2014 2019 Change 2014 2019 Change 2014 2019 Change Total 20.4 14.9 -26.9% 56.6 43.8 -22.6% 143.4 100.7 -29.8% Household head's gender Male 20.3 14.9 -26.5% 56.5 44.2 -21.7% 143.1 101.3 -29.2% Female 21.1 15.0 -28.9% 57.5 41.8 -27.4% 144.8 97.1 -32.9% Household head's ethnic Kinh/Hoa 15.6 13.9 -11.2% 43.6 39.4 -9.7% 115.1 100.6 -12.6% IEM 21.2 14.0 -33.9% 60.2 42.2 -30.0% 156.4 94.7 -39.5% MEM 24.9 19.2 -22.9% 64.2 54.8 -14.7% 144.4 119.2 -17.5% Poverty status Poor 25.1 16.2 -35.5% 63.7 43.6 -31.6% 156.8 94.9 -39.5% Non-poor 16.2 13.8 -15.2% 50.3 44.0 -12.4% 131.3 106.1 -19.2% Table 12. Proportion of households having irrigation system in their commune/village (%) 2014 2019 Change Total 63.1 73.3 10.2 Household head's gender Male 62.4 73.1 10.7 Female 67.2 74.3 7.0 Household head's ethnic Kinh/Hoa 64.5 78.4 13.8 IEM 66.6 76.1 9.5 MEM 50.5 58.1 7.5 Poverty status Poor 63.6 71.3 7.7 Non-poor 62.7 75.2 12.6 Page | 83 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Table 13. Satisfaction of households on irrigation system (%) Dissatisfied Normal Satisfied 2014 2019 Change 2014 2019 Change 2014 2019 Change Total 40.3 22.3 -18.0 13.1 16.7 3.7 46.6 61.0 14.4 Household head's gender Male 41.3 23.1 -18.2 13.6 15.9 2.3 45.1 61.1 15.9 Female 35.4 18.0 -17.4 10.1 21.3 11.2 54.6 60.7 6.2 Household head's ethnic Kinh/Hoa 42.6 29.6 -13.0 22.3 21.3 -1.0 35.1 49.1 13.9 IEM 35.0 19.8 -15.2 10.6 14.6 4.1 54.4 65.6 11.1 MEM 57.3 19.2 -38.1 6.2 16.7 10.5 36.5 64.2 27.7 Poverty status Poor 37.9 16.0 -21.9 9.5 16.2 6.7 52.7 67.8 15.2 Non-poor 42.6 28.1 -14.5 16.3 17.1 0.9 41.1 54.8 13.7 Table 14. Average levels of annual per capita consumption by poor households, by gender of household head (thousand VND/person/year) Female-headed households Male-headed households Priorities 2014 2019 Change 2014 2019 Change Holiday Food and Drink 263.3 332.3 26.2% 321.8 428.2 33.1% Daily Food and Drink 4,098.1 5,155.5 25.8% 4,122.4 5,391.8 30.8% Daily Non-Food Expenditure 1,200.4 1,880.0 56.6% 1,429.3 1,915.8 34.0% Annual Consumption of Other Non-Food 753.2 1,031.2 36.9% 772.1 1,047.3 35.6% Other Cost as Expenditure 616.5 1,458.5 136.6% 559.3 1,347.4 140.9% Education Expenditure 302.2 1,358.3 349.5% 285.9 581.9 103.5% Health Expenditure 839.8 1,698.0 102.2% 775.3 1,359.9 75.4% Total Annual Expenditure Per Capita 8,073.5 12,913.9 60.0% 8,266.0 12,072.3 46.0% Observations 348 339 370 379 Note: The expenditure of both baseline and end-line are measured in the 2019 price. Table 15. Access to safe drinking water (%) Page | 84 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Safe drinking water Unsafe drinking water 2014 2019 Change 2014 2019 Change Total 65.4 72.6 7.3 34.7 27.4 -7.3 Household head's gender Male 65.2 71.5 6.3 34.8 28.5 -6.3 Female 66.1 78.4 12.3 33.9 21.6 -12.3 Household head's ethnic Kinh/Hoa 88.6 95.5 6.9 11.4 4.5 -6.9 IEM 53.5 61.5 8.0 46.5 38.5 -8.0 MEM 67.9 75.8 7.9 32.1 24.2 -7.9 Poverty status Poor 50.3 59.7 9.5 49.7 40.3 -9.5 Non-poor 78.8 84.8 5.9 21.2 15.2 -5.9 Note: Safe drinking water includes tap water, buying water, hand dug/constructed/drilled with pump wells, and rainwater. Table 16. Ownership of domestic assets (%) Motorbike Fridge Washing machine Hot water tank Gas stove 2014 2019 Change 2014 2019 Change 2014 2019 Change 2014 2019 Change 2014 2019 Change Total 85.7 90.9 5.2 22.7 37.3 14.6 6.6 13.7 7.1 4.5 14.0 9.5 49.7 67.4 17.8 Household head's gender Male 87.0 93.2 6.2 22.6 38.2 15.6 6.8 14.3 7.5 4.7 14.8 10.1 48.8 67.2 18.4 Female 78.4 78.9 0.6 23.6 32.8 9.2 5.5 10.4 4.9 3.3 9.6 6.3 54.5 68.5 14.0 Household head's ethnic Kinh/Hoa 97.2 97.4 0.3 58.9 84.9 26.0 21.4 42.8 21.4 13.7 37.3 23.7 93.5 96.7 3.2 IEM 77.5 86.9 9.5 6.9 14.3 7.4 1.0 2.0 1.1 0.7 3.6 3.0 30.1 50.2 20.0 MEM 94.0 94.4 0.3 18.8 43.5 24.7 2.6 9.9 7.3 3.0 14.2 11.2 46.0 80.7 34.7 Poverty status Poor 73.9 85.9 12.0 3.9 11.5 7.6 0.0 1.2 1.2 0.1 2.2 2.1 23.0 48.2 25.2 Non-poor 96.3 95.7 -0.6 39.6 61.7 22.2 12.5 25.4 12.9 8.4 25.1 16.6 73.5 85.6 12.1 Page | 85 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Table 17. Percentage of labor force working last 12 months (%) Self-employed in farm sector (including Waged and salaried work Self-employed in non-farm sector agriculture, forestry and fishery) 2014 2019 Change 2014 2019 Change 2014 2019 Change Total 40.9 43.9 3.0 80.6 82.4 1.7 5.6 7.0 1.5 Household head's gender Male 49.2 50.0 0.8 83.2 84.2 1.0 5.3 6.5 1.2 Female 31.9 37.4 5.5 77.9 80.4 2.5 5.8 7.6 1.8 Household head's ethnic Kinh/Hoa 40.4 38.9 -1.5 67.2 68.6 1.4 14.7 21.9 7.2 IEM 40.8 46.7 5.9 84.5 87.4 2.9 2.0 2.6 0.6 MEM 42.0 40.6 -1.4 86.0 82.1 -3.9 4.8 3.8 -1.0 Poverty status Poor 41.4 45.2 3.7 85.1 87.8 2.7 1.6 2.1 0.5 Non-poor 40.5 42.6 2.2 76.4 76.7 0.3 9.3 12.1 2.8 Table 18. Proportion of households participating in agricultural activities (%) Self-employed in agriculture Crop cultivation Annual crop 2014 2019 Change 2014 2019 Change 2014 2019 Change Total 97.2 97.6 0.4 93.2 92.7 -0.5 83.5 75.7 -7.8 Household head's gender Male 97.4 97.7 0.2 93.6 93.2 -0.4 84.2 75.7 -8.5 Female 95.5 96.8 1.4 90.6 89.6 -1.0 79.5 76.1 -3.4 Household head's ethnic Kinh/Hoa 93.7 96.4 2.7 82.2 85.0 2.8 56.2 49.7 -6.5 IEM 98.2 98.0 -0.3 96.8 96.3 -0.6 93.5 87.8 -5.6 MEM 98.9 97.9 -1.0 97.8 92.3 -5.5 92.5 74.1 -18.3 Poverty status Poor 98.1 98.5 0.4 96.6 96.2 -0.4 95.0 86.6 -8.4 Non-poor 96.3 96.7 0.3 90.2 89.4 -0.8 73.3 65.5 -7.8 Page | 86 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Perennial crop Fruit crop Livestock farming 2014 2019 Change 2014 2019 Change 2014 2019 Change Total 40.0 50.5 10.5 26.8 34.7 7.9 72.3 82.8 10.6 Household head's gender Male 41.4 52.0 10.6 26.8 34.4 7.5 73.4 82.6 9.2 Female 32.2 41.8 9.6 26.3 36.4 10.2 65.8 84.2 18.4 Household head's ethnic Kinh/Hoa 52.6 59.6 7.1 28.5 38.2 9.7 72.2 84.7 12.6 IEM 30.9 42.9 12.0 21.5 33.1 11.6 69.5 80.4 10.9 MEM 49.5 61.3 11.8 40.0 34.8 -5.3 80.5 87.7 7.3 Poverty status Poor 28.3 41.2 12.9 24.0 32.2 8.2 69.2 80.3 11.0 Non-poor 50.5 59.2 8.7 29.3 37.1 7.8 75.0 85.3 10.3 Table 19. Proportion of households raising livestock (%) Porker Cattle Poultry 2014 2019 Change 2014 2019 Change 2014 2019 Change Total 40.1 35.5 -4.6 37.9 47.3 9.4 76.7 83.8 7.1 Household head's gender Male 40.0 36.3 -3.7 37.5 47.8 10.3 77.1 84.2 7.1 Female 40.9 30.7 -10.2 40.7 44.5 3.8 74.0 81.4 7.3 Household head's ethnic Kinh/Hoa 25.4 26.1 0.7 24.2 25.4 1.2 89.1 90.5 1.3 IEM 50.4 44.9 -5.5 50.0 63.1 13.1 65.3 77.8 12.5 MEM 32.3 21.3 -11.0 24.1 31.6 7.5 90.1 91.7 1.6 Poverty status Poor 47.8 40.5 -7.3 43.5 56.2 12.7 67.8 78.9 11.0 Non-poor 33.7 31.0 -2.8 33.2 39.3 6.1 84.0 88.1 4.1 Page | 87 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) ANNEX 8. KEY PROJECT DOCUMENTS Central Highlands Poverty Reduction Project (CHPov “GNTN”) Key Project Documents, Technical Reports, Monitoring & Evaluation Reports Document Title Year Central Highlands of Viet Nam: Ethnic minority livelihoods, local governance context, April 2012 1 and lesson-learning study. E Shanks et al. Vietnam's Central Highlands: Analysis of issues related to ethnic minority labor market 2012 2 participation, WB Poverty reduction in the Central Highlands - Analysis of Ethnic Minority Labor-related June 2012 3 issues, WB Central Highlands Poverty Reduction - Operational Implications of Recent Analytical 2013 4 Work, WB Vietnam's Central Highlands: Ethnic Minority Livelihood Contexts, Strategies, and Past 2013 5 Interventions. E. Shanks et al.2013 6 The Rural Connectivity Index - Final Report. CAP & WB 2013 2013 7 Consolidated Project Feasibility Study Report July 2013 8 Integrated safeguards data sheet - appraisal stage Sept 2013 9 Environmental & Social Management Framework July 2013 10 Environmental Code of Practice (Source: Google translation from project website) Not known 11 Environmental Guidelines for remaining contracts (AM August 2019) Aug 2019 12 Resettlement Policy Framework (RP1467 v3) Aug 2013 Findings and Arrangements to Enhance Ethnic Minority Participation Indigenous People Sept 2013 13 Plan Vol3 Social Assessment Report (IPP649 v2) - The Central Highlands Poverty Reduction Aug 2013 14 Project 15 Project Appraisal Document (PAD) Nov 2013 16 Financing Agreement April 2014 17 Project Implementation Manual (PIM) (Vietnamese) 2014 18 Baseline Survey Report (Mekong Development Research Institute, MDRI) Sept 2015 19 Project Implementation Manual (PIM) Rev 2015 (Vietnamese) Google Translation 2015 Implementation Support Mission Report Component 2: Sustainable Livelihood. Ines Nov 2016 20 Punda Page | 88 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) 21 Infrastructure field visit report Gia Lai & Kontum Province (Hoang Tung) 2016 22 GNTN Results Framework – Indicator Formulation and Baseline Values 2016 23 Project Implementation Manual (PIM) 2014 and Review 2017 (Vietnamese) 2017 Quality inspection and evaluation of investment extent and efficiency for completed June 2017 24 infrastructure works of Central Highlands Poverty Reduction Project. BKG Group, 2017 25 Annual Project Progress Reports (in Vietnamese, a few in English) 2015-2019 2015-2019 26 Beneficiary Assessment Report CHPov (MDRI) April 2017 27 Project Mid-Term Review Report 2014-2017 (MTR) June 2017 28 Infrastructure field visit report Gia Lai Province (Hoang Tung) 2018 29 Infrastructure field visit report Kontum & Quang Ngai Province (Hoang Tung) 2019 Rapid Assessment of Livelihood Enhancement Group Outcomes, 2019. Dao Ngoc Nga, July 2019 30 Hoang Thu Hang Summary of discussion on lessons learnt from the Project implementation (CF Forum – Aug 2019 31 Hoi An) 32 Restructuring-Paper-VN-Central-Highlands-Poverty-Reduction-Project Oct 2019 33 Endline Survey-Impact Evaluation CHPov (MDRI) Feb 2020 34 Impact Evaluation of the Central Highlands Poverty Reduction Program (WB) May 2020 35 Government Project Completion Report (PCR) (First Draft in Vietnamese) April 2020 World Bank Implementation Status & Results Reports (12 reports) 2014-2019 36 (Source: WB website) 37 Project Aide Memoire (13) 2014-2019 38 Project Mid-term Review Mission Report Aug 2016 39 Selected LEG sub-project proposals 2014-2019 Page | 89 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) ANNEX 9. LIVELIHOOD ENHANCEMENT GROUP SUCCESS STORIES75 Cow rearing LEGs: (Buon Don and Krong Bong Districts, Dak Lak Province) Each cow-raising LEG had 11 members and each member received 1 cow, materials for the construction of a shed, and relevant technical training. Despite a small number of cows in these communes, and some difficulties with reproduction, group performance was quite good with a total of 41 calves born in two groups. At least one LEG member in Cu Pui commune was able to lend out her cow to another household, which later gave birth to two more calves. Many households have earned between 7 and 12 million Vietnamese Dung (VND) per calf (depending on selling time). LEG members indicated that this money was then used to build or improve their homes, or to purchase other farm inputs such as fertilizer or invest in further productive activities (such as coffee, pepper, etc.). The group in Tan Hoa commune (Buon Don district) used cow dung to also generated additional income for households “Some households are using dung to fertilize their own coffee or pepper plants, or else we sell directly as there are buyers coming directly to our shed to buy. With 2 cows, the household will have about 4-5 cubic meters of dung, which will sell at VND 3.5 – 4 million.” Group discussion with cow raising LEGs in Buon Don and Krong Bong Districts, Dak Lak Province 75Source: Rapid Assessment of Livelihood Enhancement Group Outcomes, 2019. Dao Ngoc Nga, Hoang Thu Hang. Photos taken from: GNTN Central Project Coordination Office, “5 Years and Miracle Changes, Project Photobook”. Hanoi. December 2019. Page | 90 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Goat Raising LEGs (Kon Thup Commune, Mang Yang District, Gia Lai Province) The goat raising LEG was established in 2017 with 15 members, mainly Nung and Tay migrant ethnic minorities. The group began with 30 female goats (2 goats per household) and 3 male goats of Bach Thao breed. By June 2019, the group had around 170 goats—some of which had been sold—and each household had kept 6 to 7 goats each. When the group started selling goats, the project also introduced them to buyers who now come to the households to purchase the goats. Currently demand is high and on average each goat sells for between VND 1 and 2 million in off-peak times, and for about VND 3 million at peak time. Households report all having earned money from the LEG activities. “The important thing is about caring and observation over our herd, whether we have interest in it. In the family, the husband often understands more about goats but both wife and husband have to grasp raising techniques. Our group has seen 3 households to escape from poverty and we cannot be happier about that.” – shared Mr. Thai, the group leader (Group discussion with goat raising LEG in Kon Thup Commune) Pig raising (Ba Trang Commune, Ba To District, Quang Ngai Province) Mr. Pham Van Men, a member of a pig farming LEG is from a poor household. Before joining the group in 2015, the income of his 3-person households was derived from two “sao” (~1000 square meters) of wet Page | 91 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) rice (cropped once/year), 6 sao of acacia (harvested after 4-5 years) and some chickens and ducks. The project supported the LEG to purchase the animals, provide training and some additional inputs. After Mr. Pham’s pig gave birth to 11 piglets, he applied for a loan from Vietnam Bank for Social Policies to buy another mother pig, thereby scaling up the number of his pig herd. He quickly became an example for other people in the hamlet to learn from. With the farming techniques he learned from participating in the LEG, his perception and understanding of farming practices changed, directly resulting in increased income for his household. His household went from being classified as poor in 2016, with income of around VND 800,000 per person per month, to almost doubling to VND 1,500, 000 per person per month. He believes that the project has encouraged a new way of farming for people from ethnic minority groups. Many people have become aware of the benefits of raising pigs in captivity instead of letting them forage freely. He also believes that the new way of organizing production, rearing techniques, and careful selection of animals are also important factors. However, he feels that the most valuable thing is how people from ethnic minority groups have started changing their mind-set towards agriculture and are now willing to invest in farming activities, thereby gradually changing their life. Pineapple farming LEGs (Cu Pui Commune, Krong Bong, Dak Lak Province) The group was established in 2016 with 20 members who are H’Mong people. The members started growing pineapple since June 2016 with 2 or 3 sao per household and 1,300 plants per sao. In 2018, the group harvested their first crop with 500 fruits per sao on average, which sold for VND 6,000 – 7,000 per fruit. Overall, these activities added between VND 6 and 10 million per year to the income of each household. However, in the second year the LEG members encountered challenges, with lower yields of approximately 70% of the first year, and sour fruit. Fortunately, the price of pineapples was higher that year-- reaching VND 10,000 per fruit—and therefore income remained steady for the group. In addition, about half of the group members conducted plant multiplication by themselves and were able to expand their individual growing areas by ½ to 1 sao. “We are still concerned about growing techniques, how to handle the falling of young fruits, poor fruiting or too sour fruit. We do not know if it is because of soil quality and we would feel more secure with technical assistance from the Project. There was Project staff assigned to assist us in the first year only and we don’t know who to ask now. If we cannot figure out how to handle issues with the crop, we will probably revert to growing cassava, which is what we have done here on this land for years.” Group discussion with LEG growing pineapple, Cu Te village, Cu Pui Commune, Krong Bong, Dak Lak Province Page | 92 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Gourd farming LEGs (Chư Krey Commune, Kông Chro District; Gia Lai Province) The gourd farming LEG with 18 member households, all from ethnic minority groups, started operating in June 2018. Being used to crops such as corn and various beans which are relatively easy to grow, LEG members were afraid at first of raising this unfamiliar crop. CHPov provided training courses on farming techniques, input materials (seeds, fertilizers, etc.), while LEG members contributed labor, watering systems, and pump. The Commune Development Board also assisted the group to access online videos of gourd farming and organized field trips to other gourd farms in the vicinity. LEG members succeeded in shifting from pure subsistence to market-oriented farming, and increasing crop productivity over time, with up to three crops per year, by applying the right techniques and fertilizers. Group members established a group savings fund to purchase inputs for the next crop, limiting their dependency on traders who came to the village and tended to charge higher prices. “Thanks for the support from the project for farming techniques and input materials. After 3 months, we are now able to sell our product and earn a lot of money. I’m very happy. Now with all the knowledge of team work, contributing and using team fund, we will continue to plant gourd for the next crops” reported Mr. Drop (Team leader) when being asked about his thoughts on the achievement of the subproject. Page | 93 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Mulberry production and silkworm raising (Dak Son Commune, Dak Glong District, Dak Nong Province) Mulberry farmers and silkworm producers in Dak Glong District depended entirely on traders passing through the commune since there was no organized production and marketing in place. In order to help the project area communes to establish a more market-oriented approach, the Provincial Project Management Unit of Dak Nong promoted the planting of mulberry trees for raising silkworm through a market-linkage approach. The LEG of the Dak Som Commune was the first to be connected with the Dak Nong Mulberry Company. The group comprised 15 households with an initial area of 4.5 hectares. The LEG included three well-off households who were able catalyze the group and to help the other member households with technical advice and methods of doing business. Mr. Nguyen Hieu Thanh, leader of the mulberry planting and raising silkworms group says: “Previously, on this hilly land, people only planted cassava and maize, and the income from these crops was not enough to escape poverty, so when the project supported this promising activity, all households agreed to start working”. Before project support the average income per household was around VND 5.4 million per household per month. After the establishment of the mulberry farm and expansion of production, income increased to an estimated VND 7.1 million per household per month. “Currently, silkworm raw material in Dak Nong only meets one-third of the factory's capacity. Therefore, our company has signed a commitment to buy all the cocoons of the LEG. At the same time, the company provides tools to ensure quality and quantity of silkworms, offers technical assistance, and provides assistance in mulberry and silkworm pest management." (Mr. Do Van Loc, Director of Dak Nong Mulberry Company) Page | 94 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Rice farming LEGs (Ba To village, Ba To Commune, Quang Ngai Province) “People have seen changes in paddy cultivation. They now master techniques of sowing. In the past, they used to soak the seeds and then sow directly but now they will soak, monitor until the sprout cleaves before sowing. Fertilizer is also being used according to project guidance and farmers are also sharing among themselves when to fertilize for best output. The paddy fields now look totally different. In the past, each field plot had its own growing progress and yields, but now all yields are much better. It is all thanks to CHPov and the comprehensive training provided to the group, by stages of paddy growth and directly in the field so that farmers could quickly understand and remember the techniques.” Interview with village leader in Mang Lung Village, Ba To Commune, Ba To village, Quang Ngai Province Page | 95 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) ANNEX 10. CHPOV TIMELINE AND KEY MILESTONES Project Year 2013 2014 2015 2016 2017 2018 2019 2020 Nov Project Appraisal Document 2013 Approval Date Dec 2013 Original Closing Date Dec 2019 April Finance Agreement 5330-VN 2014 Effectiveness Date July 2014 Sept Baseline Report 2015 Bank IE Team defines formulation of indicators and Feb 2016 baseline values Infrastructure Assessment Report (by BKG) Apr 2017 LEGs and Beneficiary Assessment Report (by MDRI) Jun 2017 Mid Term Review Report Jun 2017 Rapid Assessment of Livelihood Enhancement Group Jul 2019 Outcomes Restructuring Paper (Cancellation of SDR 11.5 million) Oct 2019 Mar End-line survey – impact evaluation (MDRI) 2020 May ISM 1 2014 Page | 96 The World Bank VN - Central Highlands Poverty Reduction Project (CHPov) (P128072) Project Year 2013 2014 2015 2016 2017 2018 2019 2020 ISM 2 Oct 2014 ISM 3 Jan 2015 ISM4 Jul 2015 Mar ISM5 2016 Interim Implementation Support Mission Aug 2016 Nov ISM6 2016 ISM7 Jun 2017 Mar ISM8 2018 ISM9 Oct 2018 ISM10 Feb 2019 ISM11 Aug 2019 Nov ISM12 2019 Nov ICR field mission (3 weeks) Dec 2019 2019 Page | 97