BANGLADESH Social Investment Program Project Report No. 106442 JUNE 16, 2016 © 2016 International Bank for Reconstruction This work is a product of the staff of The World RIGHTS AND PERMISSIONS and Development / The World Bank Bank with external contributions. The findings, The material in this work is subject to copyright. 1818 H Street NW interpretations, and conclusions expressed in Because The World Bank encourages Washington DC 20433 this work do not necessarily reflect the views of dissemination of its knowledge, this work may be Telephone: 202-473-1000 The World Bank, its Board of Executive reproduced, in whole or in part, for Internet: www.worldbank.org Directors, or the governments they represent. noncommercial purposes as long as full attribution to this work is given. The World Bank does not guarantee the accuracy of the data included in this work. The Any queries on rights and licenses, including boundaries, colors, denominations, and other subsidiary rights, should be addressed to information shown on any map in this work do World Bank Publications, The World Bank not imply any judgment on the part of The Group, 1818 H Street NW, Washington, DC World Bank concerning the legal status of any 20433, USA; fax: 202-522-2625; e-mail: territory or the endorsement or acceptance of pubrights@worldbank.org. such boundaries. Report No.: 106442 PROJECT PERFORMANCE ASSESSMENT REPORT Bangladesh SOCIAL INVESTMENT PROGRAM PROJECT (CREDIT IDA-3740) June 16, 2016 IEG Sustainable Development Independent Evaluation Group ii Currency Equivalents (annual averages) Currency Unit = Bangladeshi Taka 2003 US$1.00 BDT58.20 2004 US$1.00 BDT58.65 2005 US$1.00 BDT60.98 2006 US$1.00 BDT66.58 2007 US$1.00 BDT69.10 2008 US$1.00 BDT68.55 2009 US$1.00 BDT68.90 2010 US$1.00 BDT68.97 2011 US$1.00 BDT70.82 Abbreviations and Acronyms BDPC Bangladesh Disaster Preparedness Committee CAS Country Assistance Strategy CDD community-driven development CNRS Centre for Natural Resource Studies FGD focus group discussions GP Gram Parishads GS Gram Samitis HCP hard core poor ICR Implementation Completion Report IEG Independent Evaluation Group IEGSD IEG Sustainable Development Unit JG Jibikayan Groups KII key informant interviews M&E monitoring and evaluation MIS management information system MTR mid-term review O&M operation and maintenance PDO project development objective PKSF Palli Karma Sahayak Foundation PMC project management committee PPAR Project Performance Assessment Report PRSP I Poverty Reduction and Strategy Paper PIP participatory identification of poor SIPP I Social Investment Program Project I SIPP II Social Investment Program Project II SIPP III Social Investment Program Project III TLLs task team leaders VCO Village Credit Organization VDCs village development committees iii Fiscal Year Government: July 1 – June 30 Director-General, Independent Evaluation : Ms. Caroline Heider Director, IEG Financial, Private Sector & Sustainable Development : Mr. Marvin Taylor- Dormond Manager, IEG, Sustainable Development : Ms. Midori Makino Task Manager : Ms. Lauren Kelly v Contents Principal Ratings ............................................................................................................... vii Key Staff Responsible....................................................................................................... vii Preface................................................................................................................................ ix Summary ............................................................................................................................ xi 1. Background and Context................................................................................................. 1 2. Objectives, Design, and their Relevance ........................................................................ 2 3. Implementation ............................................................................................................... 6 4. Achievement of the Objectives ....................................................................................... 7 5. Efficiency ...................................................................................................................... 13 6. Outcome ........................................................................................................................ 14 7. Risk to Development Outcome ..................................................................................... 14 8. Bank Performance ......................................................................................................... 15 9. Borrower Performance .................................................................................................. 17 10. Monitoring and Evaluation ......................................................................................... 18 11. Lessons........................................................................................................................ 20 References ......................................................................................................................... 21 Appendix A. Basic Data Sheet.......................................................................................... 22 Appendix B. Methodology................................................................................................ 26 Appendix C. Gender Assessment ..................................................................................... 28 Appendix D. Village Grading Criteria ............................................................................. 32 Appendix E. List of persons Consulted. ........................................................................... 38 Appendix F. Jamalpur Map (IEG Site Visits)................................................................... 39 Appendix G. Borrower Comments ................................................................................... 40 This report was prepared by Abhinav Kumar Gupta, consultant, under the guidance of Lauren Kelly, Senior Evaluation Officer in the Independent Evaluation Group of the World Bank Group. Additional research and field support was provided by Kathryn Steingraber, Sonia Sarder and Sama Khan. The PPAR mission was conducted in March 2016. The report was peer reviewed by Elena Bardasi and panel reviewed by Judy Twigg. Vibhuti Khanna provided administrative support. vi Boxes Box 2.1. Natural Disasters in Bangladesh during the Project Period. ............................................. 5 Box 4.1. Water Contamination is a Critical Issue in Bangladesh .................................................. 11 Tables Table 4.1.Table Village Institution Grades in Jamalpur District at the end of SIP I. ...................... 8 vii Principal Ratings ICR* ICR Review* PPAR Outcome Moderately Satisfactory Moderately Moderately Unsatisfactory Unsatisfactory Risk to Moderate Significant Significant Development Outcome Bank Performance Satisfactory Moderately Moderately Unsatisfactory Unsatisfactory Borrower Satisfactory Moderately Satisfactory Moderately Satisfactory Performance * The Implementation Completion Report (ICR) is a self-evaluation by the responsible World Bank department. The ICR Review is an intermediate Independent Evaluation Group (IEG) product that seeks to independently verify the findings of the ICR. Key Staff Responsible Division Chief/ Project Task Manager/Leader Sector Director Country Director Appraisal Wahida Haque Constance A. Bernard Frederick T. Temple Completion Ousmane Seck Simeon Ehui Ellen Goldstein viii Independent Evaluation Group Mission: Improving World Bank Group development results through excellence in evaluation. About this Report The Independent Evaluation Group (IEG) assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the Bank’s self-evaluation process and to verify that the Bank’s work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, IEG annually assesses 20-25 percent of the Bank’s lending operations through field work. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which Executive Directors or Bank management have requested assessments; and those that are likely to generate important lessons. To prepare a Project Performance Assessment Report (PPAR), IEG staff examine project files and other documents, visit the borrowing country to discuss the operation with the government, and other in-country stakeholders, and interview Bank staff and other donor agency staff both at headquarters and in local offices as appropriate. Each PPAR is subject to internal IEG peer review, panel review, and management approval. Once cleared internally, the PPAR is commented on by the responsible Bank department. The PPAR is also sent to the borrower for review. IEG incorporates both Bank and borrower comments as appropriate, and the borrowers' comments are attached to the document that is sent to the Bank's Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public. About the IEG Rating System for Public Sector Evaluations IEG’s use of multiple evaluation methods offers both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (additional information is available on the IEG website: http://worldbank.org/ieg). Outcome: The extent to which the operation’s major relevant objectives were achieved, or are expected to be achieved, efficiently. The rating has three dimensions: relevance, efficacy, and efficiency. Relevance includes relevance of objectives and relevance of design. Relevance of objectives is the extent to which the project’s objectives are consistent with the country’s current development priorities and with current Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, Country Assistance Strategies, Sector Strategy Papers, and Operational Policies). Relevance of design is the extent to which the project’s design is consistent with the stated objectives. Efficacy is the extent to which the project’s objectives were achieved, or are expected to be achieved, taking into account their relative importance. Efficiency is the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared to alternatives. The efficiency dimension generally is not applied to adjustment operations. Possible ratings for Outcome: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Risk to Development Outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized). Possible ratings for Risk to Development Outcome: High, Significant, Moderate, Negligible to Low, Not Evaluable. Bank Performance: The extent to which services provided by the Bank ensured quality at entry of the operation and supported effective implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of supported activities after loan/credit closing, toward the achievement of development outcomes. The rating has two dimensions: quality at entry and quality of supervision. Possible ratings for Bank Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Borrower Performance: The extent to which the borrower (including the government and implementing agency or agencies) ensured quality of preparation and implementation, and complied with covenants and agreements, toward the achievement of development outcomes. The rating has two dimensions: government performance and implementing agency (ies) performance. Possible ratings for Borrower Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory ix Preface This is a project performance assessment of the Bangladesh Social Investment Program Project that became effective on March 18, 2003, and that ended on the extended closing date of December 15, 2012. Total estimated project costs, including three additional financings (AFs), were US$101 million. The original credit of US$18.24 million was signed on April 7, 2003, while the AF1 of US$8 million was signed on June 21, 2007; the AF2 of US$25 million was signed on February 7, 2008; and AF3 of US$50 million was signed on July 31, 2008. Actual project costs were US$ 60.3 million because of the cancellation of US$40 million in the AF3. A second phase of the project was approved in 2010 and closed in December 2015; however, the Implementation Completion Report (ICR) was not completed by the time of the Independent Evaluation Group (IEG) mission. A third phase of the project, the Nuton Jibon Livelihood Improvement Project, was approved with financing of US$200 million in August, 2015. This assessment learns from the cumulative implementation experience across all three phases, but only assesses and rates the first phase. This report was prepared by Abhinav Kumar Gupta, consultant, under the guidance of Lauren Kelly, senior evaluation officer in IEG, in collaboration with IEG consultants Kathryn Steingraber, Sonia Sarder, and Sama Khan. The team would like to recognize the strong support provided by the World Bank country office in Dhaka, Bangladesh, and the excellent assistance from the Social Development Foundation (SDF), both from the project management unit and from regional staff and community facilitators in the state of Jamalpur. This assessment was selected to provide inputs to the Bangladesh country study commissioned as part of IEG’s forthcoming Rural Non-Farm Macro Evaluation. It also includes an extended gender assessment, an appendix in this report and commissioned to provide input into IEG’s learning product “The Impacts of Community-Driven Development Interventions on Women’s Empowerment” (FY16, forthcoming). Methodology. A desk review of documentation was conducted that included a review of the project appraisal document (PAD), Implementation Completion and Results Report (ICR), legal and project files, the mid-term review (2006), and the end-line evaluation. A stakeholder analysis was conducted to identify key informants, and a semi-structured interview template was designed for individual interviews. A group interview module was designed and used to collect and synthesize inputs from members of community associations assisted by the project, including the Gram Samitis, Village Credit Organizations, and the Jibikayan Groups and Youth Groups in the district of Jamalpur. To conduct the gender analysis, IEG assessed the gender elements of the country assistance strategies, poverty reduction strategies, and project documents, and within the project documents, the level of gender integration, including the results framework and indicators. The desk-level assessment was supplemented by including gender-related questions in the individual and group interviews (see Appendixes B and C). x Following standard IEG procedures, copies of the draft Project Performance Assessment Report (PPAR) will be shared with relevant government officials and agencies for their review and comment. Comments received will be included in Appendix G of the report. xi Summary This is a project performance assessment of the Bangladesh Social Investment Program Project that became effective on March 18, 2003, and that ended on the extended closing date of December 15, 2012. Total estimated project costs, including three AFs made available because of the occurrence of two natural disasters, were US$101 million. The original credit of US$18.24 million was signed on April 7, 2003, while the AF1 of US$8 million was signed on June 21, 2007; the AF2 of US$25 million was signed on February 7, 2008; and AF3 of US$50 million was signed on July 31, 2008. Actual project costs were US$ 60.3 million due to the cancellation of US$ 40 million of the third AF. The stated project objective was to develop effective and efficient financing and institutional arrangements to improve access to local infrastructure and basic services through the implementation of community-driven, small-scale infrastructure and social assistance. For the purpose of this assessment, the objective is considered to be improved access to local infrastructure and basic services, which were intended to be delivered through effective and efficient financing mechanisms and by using a community-driven development approach. The relevance of the project development objective (PDO) is rated modest. Although the objective of improving access to local infrastructure and basic services is relevant, the objective statement lacked specificity: it does not include a reference to the category of beneficiary or to whom the services would be provided. This was corrected in the project’s second phase, whose objective was to “improve the livelihoods of extreme poor communities.” The multifaceted objective statement was also circuitously written. The objective places too much prominence on the project means, which confuses the theory of change. The objective overall is aligned with Bangladesh’s 2001-2005 and 2006-2009 country assistance strategies and Bangladesh’s 6th Five-Year Plan. The relevance of the project’s design is rated modest. The project was situated in two of the country’s poorest districts with limited access to infrastructure and basic services. While the objectives lacked a specific reference to its intended beneficiaries, the district-level target did allow the project to reach poor areas. This would have been a partially adequate targeting mechanism had the community-driven development (CDD) model been designed and implemented in an inclusive and accountable way. However, the relevance of project design was undermined a lack of poverty focus and of adoption and adherence to clear rules of engagement, including the use of social accountability tools. The relevance of design was enhanced at mid-term (2006) when measures were taken to increase community participation—including that of women and youth—and to direct services to the poor. The original results framework was unaligned with the project objective. The PDO indicators measured the project reach and the project means (a financing model), but not the project objective. Like other CDD programs that have been up and running when natural disasters have hit, the Social Investment Program was flexible enough to be used as a tool for disaster crisis response. With its extensive outreach, the program directed two additional rounds of financing to remote areas that had been affected by flooding and a cyclone. However, the model was too rigid. Although some activities financed under the crisis response were xii directly related to recovery, many of the additional activities retained pre-disaster planning features that lacked relevance from a crisis-response perspective. The high level of additional finance made available for the response outstripped capacity and required coverage in areas where the program, prior to the crises, did not have a presence. The objective of improving access to local infrastructure and basic services was substantially met. The project financed the construction of 1,734 community infrastructure works, including 2,490 km of earthen roads, 3,160 culverts, and 2,940 hand tube wells. Eighty-five 85 schools were repaired under the project. An impact assessment conducted at end-line reported that the culverts and earthen roads financed by the project significantly improved the mobility of persons living in the project area. The roads and culverts improved connectivity to educational institutes, health service centers, and markets. The repair of schools reportedly led to increases in the enrollment rate of students and increased the activity of school governing bodies. Tube wells enhanced access to safe drinking water, with reported savings accruing from a decrease in water-borne illness during the project period. However, the impact assessment did not report on the quality of the services delivered, including, importantly, the quality of the water in areas where arsenic is known to be a contaminant, or the quality of the tube wells, with regard to their flood resilience (since at least one of the original project districts was severely impacted by a flood after the wells’ construction). Sustainability of the works in general was found to be an issue, mainly for large repairs, and women were found to be the main contributors (time and labor) towards making minor repairs, based on IEG’s village-level interviews. The efficiency of achieving the PDO is rated modest. Financial and economic rates of return were estimated by both the PAD and ICR, but the types of activities covered by the two estimates differed considerably. The PAD found an economic rate of return of 30.6 percent for the estimated value of time that would be saved in accessing water, a benefit that would have been enjoyed mostly by women. The ICR estimated an economic return of 24.7 percent for income generating activities supported by access to finance. Efforts to mobilize group savings resulted in groups, on average, saving US$5.00 over the project period. While much more use was made of the seed capital provided by the revolving fund, the costs associated with mobilizing group savings far outweighed the actual capital accumulation achieved. The ICR also does not indicate whether the infrastructure built was flood resilient (unlikely in a CDD project). Since some of the project areas were affected by floods—and hence received AF—the ICR’s efficiency analysis should have discounted the loss (non-working tube wells, flooded culverts, and inaccessible schools) in its calculation of the returns on the productive activities. On the other hand, the ICR could have calculated the returns on health savings, owing to the increased access to potable water, during the project period. Development outcome is rated moderately unsatisfactory, on account of the modest relevance of the objective and of the design, substantial efficacy, and modest efficiency. Overall risk to development outcome is rated significant. According to the end-line assessment, only 20 percent of all community institutions supported by the project would be sustainable without continuing implementing support. A second phase of the project may have reinforced capacity, but this information was not able to be validated since the project xiii had not been self-evaluated by the time of the IEG mission. Operations and maintenance risks are also high: the PPAR mission found inadequate attention to and savings for the maintenance of the works financed under the first phase. Concerns are mainly related to major repairs that would require support from the Union Parishads. Parts of the project areas are also prone to natural disaster risks. More information is needed on the quality of the water delivered under the CDD model, especially in relation to the depths of wells in disaster-prone areas and associated risks of contamination. Overall, there is a need to situate the works provided through CDD projects in Bangladesh within a basin-wide water and disaster management planning approach. Lessons In rural areas, where citizens lie outside the reach of the state, facilitating the formation of strong institutions is fundamental to enabling the poor to access different sources of social and economic prosperity. The Social Investment Program transformed its approach to institution building at mid-term by ensuring that institutions were inclusive and responsive to the felt needs of the poor. This approach was reflective of the more effective design of rural livelihood programs under implementation elsewhere in the region at the time. Gender inclusion, including female leadership, in village development programs is a positive but insufficient tool to ensure high quality, sustained and effective leadership over a multi-year program period. Evaluations of CDD programs have shown that for leadership to be effective, it should be (1) rotated amongst members; and that (2) leaders should have the requisite time needed and demonstrate a high level of commitment towards nurturing the less well-off group members (with regard to savings, lending, and investment behavior). Elite female members of society have the ability to extend access and information to group members, but they also may lack the time and dedication needed to ensure effective implementation of the project and group’s goals. CDD projects offer a ready platform for outreach to disaster-affected populations, but additional disaster-related activities require thorough assessment of an implementing agency’s capacity to adapt, to manage quick influxes of AF, and to reach disaster- affected populations outside of a project area. The Social Investment Program provided a relevant and ready platform to respond to two successive natural disasters, but the SDF required rapid retooling to be able to respond to the crises in a timely way. The needs of crisis victims also differed significantly from those of other project beneficiaries. This complexity, related to the need to manage these dual streams, requires strategic management and a reconfiguration of service delivery mechanisms, project metrics, and monitoring and reporting means. Marvin Taylor-Dormond Director, Financial, Private Sector, and Sustainable Development Department 1 1. Background and Context 1.1 Bangladesh, with a population of about 150 million and a land area of 147,570 square kilometers, is among the most densely populated countries in the world. The economy of Bangladesh has grown steadily during the last decade due to macroeconomic stability, trade liberalization, an improved private sector investment climate, and financial sector reform. At the time of project design, however, about 35 percent of the population remained poor. Nationwide, headcount poverty fell from 48.9 percent in 2000 to 31.5 percent in 2010 and, given slowing population growth, led to there being 17 million fewer poor people in 2010 compared to 2000. Furthermore, growth during 2000to 2010 was pro-poor over the decade as a whole, that is, the increase in consumption per capita was proportionately higher for the poor compared to the non-poor. Although rural poverty has fallen significantly from 52.3 percent in 2000 to 35.2 percent in 2010, it lags urban poverty reduction; the figure for rural headcount poverty in 2010, which was 35.2 percent, was identical to what urban headcount poverty had been in 2000. 1.2 In 2000, the Human Development Index for Bangladesh was 0.468 and rose to 0.547 in 2010. Similarly, between 2000 and 2010, life expectancy at birth increased by 4.8 years, expected years of schooling by two years, and mean years of schooling by 0.8 years. Furthermore, in 2011, the Multidimensional Poverty Index was at 0.237, with a headcount ratio of 49.5 percent and intensity of deprivations, that is, education, health, and living standards at 47.8 percent. It was also found that 18.8 percent of the poor were near poverty, 21.0 percent in severe poverty, and 43.3 percent still below the income poverty line of $1.25 per day. Living standards contributed 44.9 percent to overall poverty in deprivations, followed by health and education with28.4 percent and 26.6 percent, respectively, in 2011. 1.3 In addition, participatory poverty assessments and Bangladesh’s Poverty Reduction Strategy found that correlates of poverty are linked to: (i) land deprivation: higher poverty rates associated with smaller landholdings and vice versa; (ii) a lagging demographic transition: poorer households larger in size and with more children and fewer adults; (iii) limited education: the poverty rate was higher for households headed by individuals with limited education; (iv) nutritional inadequacy; (v) gender inequality; and (vi) a lack of participation by the poor in the development process. 1.4 The World Bank-financed Social Investment Program (SIP) was designed to address these inequities. It targeted two of the country’s poorest districts (Jamalpur and Gaibandha) that were characterized by low levels of income and human development scores. The project was anchored in the then recently-created SDF, established by the Government of Bangladesh as a not-for-profit company in 2001, to promote community-driven initiatives through decentralized planning and management of development programs. Recognizing the challenges presented by weak local governance and their lack of reach, the project, especially after mid-term, focused on the development of capable and accountable local rural institutions as a platform for development planning and service delivery. It also used a CDD approach that included social mobilization and participatory identification of the poor, taking into account gender equity and the needs of youth. 2 2. Objectives, Design, and their Relevance 2.1 Project Development Objective The PAD and the project credit agreement stated the development objective of the project was “to develop effective and efficient financing and institutional arrangements for improving access to local infrastructure and basic services through the implementation of community-driven small scale infrastructure works and social assistance programs.” 2.2 Project Components: There were four components throughout the project from design to closure. The component costs at appraisal and actual costs are shown adjacent to each component below. A fifth component was added to the second additional financing credit, as explained in (e). Notwithstanding these additions and some substantive changes, the project’s objective remained the same. (a) Strengthening the SDF (appraisal: US$1.91 million; actual: US$7.63 million) The SDF, which was established as an autonomous not-for-profit company under the Ministry of Finance, was chosen to be the implementation agency for the project. This component was comprised of four sub-components: (i) information and communication; (ii) capacity building; (iii) monitoring, learning, and evaluation; and (iv) project management support (including environmental assessment and tribal development). Technical assistance, monitoring and evaluation studies, establishment and incremental staff expenses, and operating costs were financed under this component. (b) Institutional Development at the Community Level (appraisal: US$0.38 million; actual: US$5.00 million) This component supported communities in raising awareness, motivating and engaging the rural poor to participate in community-driven initiatives, changing attitudes and behaviors among local stakeholders, development of institutions at the village level, preparation of community action plans, and provision of an institutional development fund that included revolving funds. The component had two sub-components: (i) information and communication campaigns and (ii) formation and strengthening of community institutions and financing of their action plans. The component also financed local information dissemination and environmental assessment and screening. (c) Implementation of Community Action Plans (appraisal: US$19.35 million; actual: US$32.26 million) These plans were intended to prioritize community needs for small-scale infrastructure, livelihood, disaster recovery and preparedness (in cyclone and flood-affected areas financed by AF2 and 3), and social assistance based on informed choice and eligibility criteria. The project was to finance: (i) up to 85 percent of expenditures of community infrastructure sub-projects; (ii) services provided by nongovernmental organizations (NGOs) to implement social assistance programs, including one-time seed capital and legal assistance for the very poorest and vulnerable groups, and implementation of a tribal development plan; and (iii) costs of services for appraisal and supervision of sub-projects, including community financing sub-projects. (d) Pilot Private Financing in Community Utilities (appraisal: US$0.90 million; actual: US$0.40 million) The project financed technical assistance and a maximum of 70 percent of approved costs of water and electricity sub-projects. These resources were channeled to 3 leverage private financing of piped water supply to communities through matching grants. The technical assistance helped the SDF to identify, develop, appraise, and supervise pilot sub-projects in piped water supply and off-grid electricity. (e) Livelihood Restoration through Micro-Credit to Flood-Affected Communities (original: US$0.90 million; actual: US$15 million) This assistance was implemented from resources provided by AF credit II in December 2007 through the Palli Karma Sahayak Foundation (PKSF), an independent NGO, to provide micro-credit to flood-affected communities. It is treated as a stand-alone component by the ICR, separate from the four original components (p. 27). Since this component was entirely financed by an AF credit, there are no costs to show at appraisal. 2.3 The Social Investment Program Project (SIP) aimed at delivering critical social assistance and local infrastructure services through process oriented community-driven approach. It worked through empowering local poor communities by improving community infrastructures and access to income generating activities. The self-governed effective community institutions established to develop community-owned institutions as well as promoting participatory governance. 2.4 Community Group Formation: The project organized village-level institutions by mobilizing women and youth from vulnerable, marginalized, and poor households. At least 15 to 20 members were mobilized in groups called Jibikayan Groups (JGs). The JGs are considered the foundation of the project. Within these groups, members would implement their own savings and lending schemes, and through a project-financed revolving fund, they would decide upon, finance, and implement income generating activities. Members would meet in a general assembly, called as the Gram Parishads (GPs), which were supported by representative Gram Samitis (GSs), or executive bodies. 2.5 Village credit organizations were formed to receive and manage a village-level revolving fund—or the Shabolmi Fund. The fund made finance available for income generating activities. The main income generating activities undertaken by the Jibikayan Groups were poultry, goat rearing, cow rearing, cow fattening, petty trade, dairy, basket making, and driving rickshaws and vans. In a few cases, the beneficiaries were also undertaking value-added activities, such as puffed rice processing, chili primary processing, and salt processing. The proceeds from the loan are expected to increasingly make finance available for these types of activities for more group members. 4 Relevance of Objective 2.6 The relevance of the project development Objective is rated as Modest. While the objective of improving access to local infrastructure and basic services is relevant, the objective statement lacks specificity: it does not include a reference to the category of beneficiary (e.g. the poor). This was corrected in the second phase of the project, whose objective is to “improve the livelihoods of extreme poor communities and to strengthen the community institutions in selected districts.” The multifaceted objective statement was also circuitously written. The PDO places too much prominence on the project’s means, which confuses the theory of change. 2.7 The objective was and continues to be consistent with Bangladesh’s country assistance strategies (CASs). Two of the strategic objectives of the 2001 CAS were helping to build stronger institutions and governance across development programs and implementing an integrated approach to rural development, including supporting growth in agriculture and non-farm activities, making opportunities and assets available to the poor, and improving rural infrastructure (p. 18). The 2006-2009 CAS notes that further gains in human development will be difficult to achieve and sustain without service delivery systems that are more accountable, more responsive, and able to reach the very poor. “With government highly centralized, the poor find it difficult to hold public service providers accountable for delivering services and responding to their needs. Recent efforts to enhance local democracy and make more resources available to local governments provide an opportunity to strengthen local governance and increase the voice of the poor in identifying, designing, and implementing programs for rural infrastructure and social services” (p. 15). The objective is also in line with the aim of the government’s 6th Five-Year Plan to bring Bangladesh to middle-income-country status through a massive poverty reduction effort. Relevance of Design 2.8 The relevance of the project’s design is rated as Modest. The project was situated in Jamalpur and Gaibandha, two of the poorest districts in Bangladesh with limited access to infrastructure and basic services. While the project did not have a specific reference to the intended beneficiary group, geographic targeting enabled the project to reach poor areas. This would have been a partially adequate targeting mechanism had the CDD model been designed and implemented in an inclusive and accountable way. However, the relevance of design was undermined by a lack of poverty focus and of adoption and adherence to clear rules of engagement, including the use of social accountability tools. The relevance of design was enhanced at mid-term when measures were taken to increase community participation— including that of women and youth—and to direct services to more marginal parts of the population. 2.9 The original results framework was unaligned with the project objective. It sought to measure the core goal—the delivery of infrastructure and basic services—but not the means. The results frame and measurement systems did not provide a coherent way of understanding how the project added up to the sum of its parts. Were efficient and effective local financing institutions (in the form of group savings and lending) needed to support the project’s service delivery goal? How did the CDD model contribute to the efficiency and effectiveness of the 5 service delivery model? Terms, like community, participation, and beneficiary satisfaction were loosely applied. 2.10 Like other CDD programs that have been up and running when natural disasters have hit, the SIP was flexible enough to be used as a tool for disaster crises response. With its extensive outreach, the program was able to absorb and direct two additional rounds of financing to remote poor areas that had been hit by flooding and subsequently a cyclone (see Box 2.1). However, the model was too rigid. Although some of the activities financed under the crisis response period were directly related to recovery, many of the additional activities retained pre-disaster planning features that lacked relevance from a crisis response perspective. Even though the aim of using the project vehicle to reach disaster-prone areas was relevant, the rules and procedures associated with the CDD model, including hiring and staffing, were not amended to allow greater flexibility to achieve a sufficiently timely response. Rigidity around hiring, project staff placement, and the retention of CDD features (social mobilization and training) resulted in a two-year delayed response between the disaster and implementation of the first field activities in the disaster-hit areas. On the other hand, the AF period was too short to ensure effective implementation of the longer CDD goals sought in the expanded project areas. Box 2.1. Natural Disasters in Bangladesh during the Project Period. Overview. Several major natural disasters occurred during the project period including in 2004, 2007, and 2009. The 2007 flood-affected areas include SIP project areas, including the districts of Jamalpur and Gaibandha. Since 2007, natural disasters are estimated to have caused damages in the amount of Tk189.4 billion, or 4.7 percent of Bangladesh’s gross domestic product. In 2004, the country’s northeast suffered from flash floods that destroyed a substantial portion of the boro rice crop. The main wave of monsoon flooding started in early July, eventually affecting 36 million people (almost a quarter of the total population) living in the northwestern, northeastern, and central districts. The inundation caused nearly 800 deaths, affected 2 million acres of agricultural land, and destroyed infrastructure and public and private assets. While several areas were still experiencing an emergency situation in September, a localized monsoon depression swept over Bangladesh, bringing three times the normal rainfall, and resulting in flooding in Dhaka and the southwest and central areas of the country. Preliminary estimates show that total assets and output losses due to the floods are approximately US$2.3 billion, including both the public and private sectors. The 2007 floods directly affected over 13 million people in 46 districts, caused over 1,000 deaths, affected over 2 million acres of agricultural land, and damaged infrastructure, social and educational facilities, and private assets, including housing, crops, livestock, and fisheries. (World Bank 2008; World Bank 2005). Bangladesh was also hit by two cyclones between 2007 and 2009. Cyclone Sidr hit Bangladesh in November 2007, prior to the harvest season. It caused thousands of deaths and extensive economic losses, particularly in the southwestern districts (World Bank 2011). As a consequence of the 2007 cyclone, Sidr, SIP was expanded to the southwestern districts of Bagerhat, Barguna, Barisal, Patuakhali, and Pirojpur. Source: Shamsuddoha, Md. et al.; “Insights from Cyclone-affected Communities in Coastal Bangladesh”, Center for Participatory Research and Development (CRPD), 2013; United Nations, “United Nations Rapid Initial Assessment Report: Cyclone Sidr,” United Nations, 2007; World Bank, “Project Paper On A Proposed Additional Financing Credit in the Amount of SDR 30.7 Million (US$50.0 Million Equivalent) to the People’s Republic of Bangladesh for the Social Investment Program, Project Report No.: 43295-Bd”, World Bank, 2008; World Bank, “International Development Association Project Paper for an Emergency 2007 Flood Restoration and Recovery Assistance Program, Report No. 41772-Bd”, World Bank, 2007. 6 3. Implementation Project Costs 3.1 Total estimated project costs, including three AFs, were US$101 million. The original credit of US$18.24 million was signed on April 7, 2003, while AF1 of US$8 million was signed on June 21, 2007; AF2 of US$25 million was signed on February 7, 2008; and AF3 of US$50 million was signed on July 31, 2008. Actual project costs were US$ 60.3 million because of the cancellation of US$40 million in AF3. 3.2 The project was designed to be implemented in approximately 1,400 villages, representing 54 percent of all villages in the two target districts of Jamalpur and Gaibandha. The villages were selected on their relative levels of access to basic infrastructure and social assistance services, as well as the relative populations of persons living below the poverty line, and the presence of disadvantaged groups. Within these villages, 250,000 households were to benefit from infrastructure sub-projects, and an additional 300,000 poor and socially excluded were to benefit from the social assistance program. Beginning in 2007, the project undertook three successive rounds of AF with the aim of consolidating project activities. 3.3 The first AF was implemented after the mid-term review, which was conducted in 2006. The review identified some mistargeting, including exclusion of the poorest of the poor. It also identified a lack of ownership of some of the sub-projects, signalling a concern that the CDD process may not have been well implemented, and that as a result, both efficacy and sustainability were at risk. The project performed a mid-course correction by adopting a new system of beneficiary identification, namely through a participatory identification of poor (PIP) approach, that had been used elsewhere in the region. The course correction also remodelled the development of the village institutions. The new approach placed a greater emphasis on inclusion and accountability. The GP, which is the general assembly of the project, serves as an inclusive and accountable village institution and is organized into various entities such as the GS, JGs, and savings groups (SSCs), such as the village credit organizations (VCOs). 3.4 While the first AF was intended to support a consolidation of the project activities in the two target districts, the second and third financing rounds were made available in response to an unfortunate series of natural disasters. (See Box 2.1). The existence of the program, its implementing arrangements and outreach provided for a ready platform for disaster assistance, particularly to poor remote areas. The second AF was used to respond to a flood; AF was made available to assist some 500,000 flood victims, in 200 villages, mainly in the project areas, as well as in the neighbouring district of Sirajganj. The third AF was approved to respond to a cyclone whereby project activities were expanded to four affected districts in the south: Bagerhat, Barguna, Pirojpur and Patukhali The expansion provided livelihood assistance to about 120,000 families in 500 villages in 14 of the most-affected areas (upazillas). Just under one million people in these disaster-struck areas benefitted from infrastructure rehabilitation. 7 3.5 The original project closing date was changed from June 2007 to 30 June 2011, thus doubling its implementation period. This slower than expected implementation has three main causes: (i) the nature of project activities (confidence building, institution development and long term capacities building) which required more time than anticipated at appraisal. Also, since communities drove their own development, some delays were attributable to a slower pace set by the communities themselves; (ii) the disruptions caused by the flood and cyclones; (iii) the positions of SDF management lying vacant during some key periods for example, no managing director for 5.5 months, no general manager for programs for 1.5 years, and so on. 4. Achievement of the Objectives 4.1 The objective of the project, was to develop effective and efficient financing and institutional arrangements for improving access to local infrastructure and basic services through the implementation of community-driven, small-scale infrastructure works and social institutional development. Its achievement was rated Substantial overall. This PPAR is focused on the evaluation of the main aim of this objective, to improve access to local infrastructure and basic services. It considers the establishment of financing and institutional arrangements to be the means by which this aim was meant to be achieved, and the CDD model as the chosen approach. 4.2 The Results Framework. There were two PDO indicators used by the project. Neither were sufficient to measure the objective of improving access to infrastructure and services, with regard to accessibility, quality, and use of the services delivered. The first PDO indicator was “the number of hard- core poor and poor benefitting from the project.” With a baseline of zero, the project sought a target of 3 million and achieved the participation of 2.5 million by project close. This section begins by discussing issues associated with targeting (see para 4.3). The second PDO indicator was that “a direct financing model for hard-core poor and the poor is tested.” As stated previously, the development of a financing model was a means to achieve the project’s service delivery objective, not the development objective. Other intermediate indicators measure the project’s outputs, such as the number of small- scale infrastructure built, the number of youth that were employed, and the number of employment days generated for the flood-affected communities. A third type of indicator measured participation in the program, including female participation. Project targeting and results are discussed, in turn, below: 8 Targeting: Evidence on distributional impacts 4.3 The objective statement lacked a specific reference to a target group. The project sought to improve access to infrastructure and services, but it did not stipulate for whom the services would be made available. Partly as a result, the first half of the project period was challenged by a significant level of capture at the village level. While village development committees and project management committees were established by integrating a mix of different income categories (rich, middle class, poor, and hard-core poor), decision making and resource allocation were dominated by the first two categories of group participants. The original project design and early field implementation also did not focus on gender-specific needs. Early on, for example, while women were engaged as paid labor in road construction and maintenance, they did not participate in sub-project placement or design. These design flaws were corrected at the mid-term review, when the rules of the game for the establishment and governance of the local rural institutions were changed, so that these institutions were comprised of only the poor and very poor, and of women and youth. Results 4.4 Overall, data obtained on the performance of the participating villages in one of the two districts visited by IEG provides an overview of the relative results that were obtained by the villages in that district, which IEG understands can be extrapolated for the project as a whole. As shown in Table 4.1, roughly half of the village-level institutions in the Jamalpur District received A and B grades, or were well functioning at the end of the first phase. Table 4.1.Table Village Institution Grades in Jamalpur District at the end of SIP I. Grade No. of villages % of villages A 75 17 B 143 32 C 79 18 D 144 33 Source: SDF 4.5 In the Jamalpur District, one of two districts targeted by the project and visited by IEG, poor performance was associated with several factors that resonated also with poor- performing villages in the other district. The rural institutions were originally not constituted and “owned” by the very beneficiaries they intended to serve. This changed at mid-term, in line with best practice projects in the region that have established and strengthened institutions of the rural poor by including the poor both as members and as leaders. By building the social capital of the poor, these institutions better enable the poor to acquire the voice and knowledge necessary to connect to economic and social opportunities. At design, however, the groups were constituted in a way that sought formation for short-term gain around access to finance for needed infrastructure. Once the money dried up, there was less interest in using the rural institutions for anything more, especially for servicing the poor. 4.6 In addition, both financial and organizational training was initially provided to a small set of individuals who assumed leadership functions. IEG interviews conducted at the 9 village level corroborate the reported project-level findings that this practice, coupled with a lack of leadership rotation (including training), created group-wide dependency on a single individual, which put both the rural institution and the project at risk. Another reason for poor performance is associated with the incidence of natural disasters. Higher-performing villages sometimes dropped to a lower grade because of the capacity constraints imposed by floods or other exogenous shocks. Rural Savings and Lending Schemes 4.7 The project undertook an intensive-capacity building exercise to raise awareness about savings and lending. The project provided village-level institutions with facilitation support to mobilize savings and to teach group lending skills. The initial support stage lasted between three months to a year, after which time the project provided seed capital to finance a larger Revolving Fund. The revolving fund was intended to make available credit to finance income-generating and livelihood activities. The delay between the initial savings schemes and the introduction of the revolving funds was intended to support the development of financial management skills, prior to providing larger amounts of financing through the revolving fund. 4.8 The project helped 258,113 persons involved in the Jibikayan Groups mobilize US$1.23 million in savings, equivalent to an average of US$5.00 per member. The lower than expected mobilization of savings was due mainly to behaviour. Many groups emptied their savings funds upon receiving access to the larger Revolving Fund for infrastructure development and service provision. Delays in implementation also undermined the expected savings base that had been achieved during the life of the project. 4.9 Of the 258,113 persons participating in the Jibikayan Groups, roughly half accessed loans from either their village Savings Funds or the Revolving Funds. The loans were used for income-generating activities, both farm and non-farm. The main income-generating activities accessed by the project beneficiaries were poultry, goat rearing, cow rearing, cow fattening, petty trade, dairy, basket making, and rickshaws and vans. In a few cases, project beneficiaries undertook value-added activities such as puffed rice processing, chili primary processing, and salt processing. 4.10 While repayment rates were good—93 percent and 92 percent for loans taken from the savings fund and revolving funds respectively (the internationally accepted standard is 95 percent) - and while most loan takers were women, a lesson learned over the multiple phases of the project is that the model discouraged participation of the very poor. As observed in group interviews conducted by IEG, the very poor are reluctant to take loans because of a repayment schedule referred to as “on-time.” Loan takers are required to start repaying the loan, in installments, as early as the second week after the loan is taken. Interviews with members of rural savings groups pointed to the time that it takes to raise poultry, rear cattle, or fatten cows. Poor members felt that a three- to six-month grace period should be allowed for members engaged in productive activities, to allow for a maturation of those activities and for the generation of some early benefits. 10 4.11 Nevertheless, the project end-line assessment found that the availability of loan finance has not kept up with demand, with 88 percent of groups surveyed reporting that the members had to wait to receive a loan or that they were not able to get a higher loan amount to finance a livelihood activity, since the revolving fund growth was not able to match the inflation rate. Growth was also stymied by the use of a high institution maintenance fee: half of the 10 percent lending rate was used to finance the activities of village institutions. As part of the project’s subsequent phases, the Social Development Fund has recommended raising the lending rate, from 10 to 12 percent, to help continue to grow the funds to better meet beneficiary demands. 4.12 The external assessment found, and IEG focus groups helped to verify, that proximity to urban market centers was a determining factor in the choice of income-generating activity. Beneficiaries residing near urban centers tended to opt for non–traditional or more risky/high-reward livelihood opportunities, including vegetable production, tailoring, handicrafts, and vehicle driving. Rural remote beneficiaries, on the other hand, tended to invest in livestock rearing and poultry and cow fattening, and these activities constituted the majority of activities pursued by project beneficiaries. 4.13 An employment scheme targeting 7,893 unemployed youth was piloted towards the end of the project period. As part of this scheme, skill development loans were provided to youth to pay for skill-based trainings. Most of these trainings were designed to support employment in the non-farm sector and to obtain employment opportunities in urban areas. Trainings included computer operation, tailoring, mechanic skills, sewing, stitching, driving, and welding. In a few cases, youth participants accessed credit through the groups’ savings or revolving funds to fund training directly utilized to establish a rural enterprise. Infrastructure and Services 4.14 Against a target of 1800 small works, the project financed the construction of 1734 community infrastructure works, including 1376 works under the original project and 358 under the flood and cyclone AFs. Four major categories of infrastructure works were financed: 2,490 km of earthen roads, 3160 culverts, and 2,940 hand tube wells were built, and 85 schools were repaired. All villages, by project close, were reported to have established an operation and maintenance (O&M) plan and 77 percent of them (against the target of 80 percent) had mobilized O&M funds. 4.15 An impact assessment conducted at end-line reported that the culverts and earthen roads financed by the project significantly improved the mobility of persons living in the project area. The roads and culverts improved connectivity to educational institutions, health service centres, and markets. The repair of schools led to increases in the enrolment rate of students and increased the activity of school governing bodies. 4.16 Tube wells that were constructed were reported to enhanced access to safe drinking water, with reported savings realized from a decrease in water-borne illness. While this would have been the case for most of the project area, tube wells built in areas prone to and eventually affected by cyclones and flooding would have been susceptible to contamination. However, the project did not provide data on the depth of the wells that were installed, so it is 11 unclear whether the wells affected by flooding would have been able to provide access to safe drinking water after the occurrence of two natural disasters during the project period (See Box 4.1). Box 4.1. Water Contamination is a Critical Issue in Bangladesh In Bangladesh, the majority of irrigation and drinking water is supplied from underground sources due to the limitation of surface water. At the time of project appraisal, the population exposed to arsenic poisoning through drinking water was more than 25 million. Out of 64 districts of the country, 47 to 61 districts had arsenic content above the WHO-recommended maximum permissible limit of 0.05 mg/L. Thousands of people continue to suffer from arsenic-related diseases. By 2006, new research suggested that some 85 million people were at risk from arsenic in drinking water and in food crops. A national drinking water quality survey conducted in 2009 furnished data that were used to make an updated estimate of chronic arsenic exposure in Bangladesh. About 20 million and 45 million people were found to be exposed to concentrations above the national standard of 50 µg/L and the World Health Organization’s guideline value of 10 µg/L, respectively. Exposure varies widely in the 64 districts; among adults, arsenic-related deaths account for 0 percent to 15 percent of all deaths. Exposure to arsenic through drinking water sourced from groundwater is a global public health problem that is particularly devastating in Bangladesh. In rural areas, 97 percent of the population relies on tube wells—installed since the 1970s—to reduce disease from ingestion of pathogen-laden surface waters. Tube wells, affordably priced at about US$ 100, draw arsenic-containing groundwater from a shallow depth of 10 to 70 meters. Groundwater from depths greater than 150 m usually contains less arsenic and can be a sustainable drinking water source. However, the project data obtained by the IEG assessment do not allow for a validation of this appropriate depth. Source: Sources: International Water Resources Association Water International, Volume 26, Number 3, Pages 370–379, September 2001; Agriculture Ecosystems & Environment · April 2006; M.F. Hossain; American International University-Bangladesh. Arsenic in tube well water in Bangladesh: health and economic impacts and implications for arsenic mitigation. WHO; Sara V Flanagan et al. 2012. Gender-Related Findings 4.17 Gender was not a consideration in project design. Prior to mid-term, the reference to women’s participation is included in the reference to vulnerable groups (for example, poor and socially excluded female-headed household, disabled, tribal people, street children, and so on). These groups were provided assistance through a Social Assistance Program (SAP) aimed at increasing the capacity of excluded groups to participate in mainstream development activities through: (a) advocacy programs; (b) occupation skills training; (c) support for pregnant women and risk-pooling pilot initiatives; (d) legal aid support; and (e) grants for the graduated trainees and worst affected natural hazard victims. The approach, prior to mid-term was found by the ICR and the PPAR to be insufficient. SAP beneficiaries received a small, one time grant and these grants were too small to have made a meaningful impact on livelihoods or welfare. Neither women nor the vulnerable especially benefited from the SAP activities. 4.18 Prior to the mid-term, there was a heavy focus on infrastructure due to the nature of the group composition. VDCs and project management committees had been captured by 12 non-poor members, and were mainly male. During this period, the voices of the poor were suppressed. Criteria set at design, including targeted assistance to female headed poor households had not been respected during the early part of the project period in despite of the fact that there were specific targets set (30 percent of the VDCs; 30 percent of the CG members; and at least 25 percent of the project management committees were to have been women, the poor and vulnerable.) 4.19 The project only superficially included women in the infrastructure and works portion of the project. According to the PAD, “the targeting for community infrastructure works would be broad-based but include specific conditions for inclusion of the poor and women in the eligibility criteria for financing of sub-projects.” There was a particular focus on the benefits that would accrue to women from the increase in quantity and quality of water: ‘about 250,000 people (of whom at least 30 percent will be poor and women) will benefit through access to safe water, sanitation facilities, and other essential social, transport and trade infrastructure services through the implementation of about 1,800 community infrastructure sub-projects’. The PAD theorizes that ‘investment in safe drinking water and sanitation, by improving the health of rural populations as well as reducing time spent, especially by women, in transporting water, will contribute to poverty alleviation and increase the productive capacity of rural people, particularly women. Benefits of improved water supply both in terms of quantity and quality are expected to have a positive effect on health outcomes, and that these health benefits which in turn would lead to labor productivity and income gains via a variety of direct and indirect pathways, which would also benefit women. Since none of the results data is disaggregated, it is impossible to report whether these results were achieved, including for women. 4.20 The original community groups were also not provided with basic gender awareness training, knowledge and understanding on prevention of violence against women. The staff of SDF and other executing agents are also lacking clear understanding of gender awareness building and women empowerment focused elements. Original support through which these serves would have been accessed (e.g. legal aid support) did not materialize as envisioned. Based on the original project components, prior to the additional financings, the project focused more on the hardware. This was a missed opportunity to raise awareness around such issues as the dowry, early marriage, women’s health, pregnancy, and hygiene, and as stated above, violence against women. 4.21 Gender integration was fully achieved after the mid-term review and the restructuring of the project. This is due mainly to the support of a key member of the Bank’s Rural Livelihoods teams, which put poverty and gender squarely at the center of its rural development programs. The mid-term review recommended and the project implemented a full restructuring of the project’s village organizational structures. As such, after the mid- term review, women and girls constituted 95 percent of the membership of the 1407 village level institutions, i.e. Gram Samiti, Gram Parishad and the 20,414 village level Jibikayan Groups. Seventy-seven percent of the total 1224 no. of Gram Samitis are active and led by women. Interestingly, women appear to be the de facto participants of this project since many of the men residing in these areas were too busy or too uninterested in joining once the project was geared to being truly pro-poor. Many of the men in the project areas are day laborers and farmers. 13 4.22 All the groups received financial skill trainings, including saving and credit management training. The trainings were mostly limited to the leader and cashier of each institutions (Gram Samiti, Jibikayan Groups, VCOs, Social Audit Committees). While many of the recipients of the trainings were women, one of the downfalls of this design feature was that the women did not rotate. So while some women were trained, these skills were not adequately imparted to most female members of the group (non-leaders). 4.23 This finding is salient: it raises a more complex questions as to which women are empowered, whether empowerment is limited to a few of the many women who behave as general participants in CDD project, versus their leadership. It begs for more research across CDD programs on how women join, their need to be otherwise socially connected or financially endowed to assume leadership roles, and ultimately the distribution of benefits cross these different tiers of women groups. Quality, Operations and Maintenance 4.24 As part of the group interviews led by IEG in the District of Jamalpur, IEG sought to understand more about the sustainability of the project-financed infrastructure. Four years after project close, IEG asked members of active village institutions to speak to the current state of the infrastructure financed by the project. In the villages visited, a common theme that emerged was that villagers were able to conduct minor repairs regularly, especially on roads and culverts, but they were not able to finance major repairs. Minor repairs continue to be overseen by a maintenance committee organized by the project and supported by members of project-financed village institutions, mainly women. IEG also found that none of the village institutions visited maintained a formal Operation and Management Plan, and no O&M funds had been mobilized or set aside for future repairs. This raises questions as to the veracity of the reporting on Operations and Maintenance in the project close-out reports. 4.25 Meanwhile, IEG found that the local institutions created by the project were unable to leverage their needs (for example, for major infrastructure repairs) vis a vis the Union Councils (or Union Parishads), the smallest rural administrative and local government units in Bangladesh, formed under the Local Government (Union Parishads) Act of 2009. 5. Efficiency 5.1 The efficiency of achieving the PDO is rated as Modest. Financial and economic rates of return were estimated by both the PAD and ICR, but the types of activities covered by the two estimates differed considerably. The PAD estimated an economic rate of return of 30.6 percent for the estimated value of time that would be saved in accessing water, a benefit that would have been enjoyed mostly by women. The ICR estimated an economic return of 24.7 percent for income-generating activities supported by access to finance. Efforts to mobilize group savings resulted in groups, on average, saving US$5.00 over the project period. While more use was made of the seed capital provided by the Revolving Fund than the savings, the costs associated with mobilizing group savings far outweighed the actual capital accumulation achieved. The ICR also does not indicate whether the infrastructure built was flood resilient (unlikely in a CDD project). Since some of the project areas were affected by natural disasters, and hence received AF, the ICR’s efficiency analysis should 14 have discounted these loss (non-working tube wells, flooded culverts, inaccessible schools) from the calculation of the returns on the productive activities. The ICR could have also calculated the returns on health savings, owing to the increased access to potable water, during the project period. 6. Outcome 6.1 The rating of Development Outcome is Moderately Unsatisfactory. The relevance of the objective is rated Substantial, warranted by the relevance of the overall aim, but undermined by its lack of specificity with regard to the ultimate beneficiary. The relevance of design is rated Modest since the CDD approach was not grounded in best practice. While this was corrected at mid-term, just over half of the funds amounting to 56 percent of the original credit (excluding AFs) had been disbursed by that time. 6.2 Women were not initially integrated in project design or decision-making about the placement and use of infrastructure, but were the main beneficiaries of the remodeled design. While women received benefits through the group savings and lending schemes, and while these benefits are likely to accrue over time, they also bear the brunt of the operations and maintenance of the works conceived and built under the project. Men were less interested in attending group meetings after the project was reformed, since most of the seed money to fund contracts for works had been used up. Efficacy is rated Substantial, since the project was effective in improving access to infrastructure and basic services. However, the project should have measured and reported on the quality of the works, including whether the tube wells constructed were flood resilient and whether the water that was provided was uncontaminated. Efficiency is rated Modest since the project utilized two different methods to estimate returns, the methods were not comprehensive of the program as a whole, and several inefficiencies were unearthed during the assessment, including the very low rate of capital accumulation compared to facilitation costs. 7. Risk to Development Outcome 7.1 Overall risk to development outcome is rated as Significant. 7.2 Sustainability of Village Institutions. According to the end-line assessment, only 20 percent of all community institutions supported by the project are sustainable without continuing Foundation support. The second Social Investment Project Program was intended, among other things, to help these institutions become self-sustainable. Since project closed, steps have been taken to set up second tier level institutions and two district societies have been established and registered in Gaibandha and Jamalpur districts. SDF is currently working with the district societies to strengthen their institutional functioning including monitoring and evaluation and financial management, so that they can better provide support to the village organization beneath them. 7.3 Sustainability of Infrastructure. There is a risk that infrastructure built under the project will not be adequately operated and maintained. Contrary to the ICR’s statement that 83% of the villages under the consolidation phase (AF Credit I) developed operations and 15 maintenance plans for community infrastructure works and 79% implemented these plans as scheduled, the PPAR mission was unable to find any groups either designing or implementing these plans. Although the groups - mainly women - were regularly implementing minor repairs, they were facing challenges to finance major repairs that were dependent on the Union Parishad, which had limited resources to offer. (See paragraph 4.21 for more information.) 7.4 Exogenous Shocks. The project areas are also prone to risks due to natural disasters. While the subsequent phases of the Social Investment Program have increased resilience, especially through savings and attention to infrastructure repair, there is a need for more basin- and region-wide planning and stronger linkages to national disaster management initiatives. Data obtained by IEG from the Department of Public Health Engineering (DPHE) reveals that, as of 2015, 7,500 water points (tube wells) had been damaged in the North-West districts due to flooding. 7.5 The International Non-Governmental Organization Environmental and Sanitation Conditions Database1 shows that sanitation conditions have deteriorated in all flood-affected areas in the North-West of Bangladesh. In absolute numbers, 4,500 water points were non- functional after the recent flood. While the Government has assisted affected communities by providing water purification tablets, bleaching powder and fitkari to protect them from water- borne diseases, surface water still needs to be treated in the affected areas and the initiative needs to be taken to raise tube well platforms as a risk reduction initiative for the upcoming years. 8. Bank Performance Quality at Entry. 8.1 The quality at entry is rated as Moderately Unsatisfactory. The project is recognized for its contribution to testing new approaches to building rural service delivery platforms that address systemic institutional weakness at the time of project design. However, there were significant shortcomings in the design of the original project, and subsequent AF phases underestimated the capacity of the implementing agency to manage the consolidation and expansion of the CDD program while diverting a portion of its staff and resources to crisis response. In the course of six months, owing to the pressures put on the project by the successive natural disasters and the decision to respond, the Bank raised the level of project financing from US$26 million to US$101million. Disbursement averaged US$ 3.5 million annually between project effectiveness in 2003 and the end of 2007. While the additional finance came at the behest of Government, the size of the AF packages did not adequately involve consultations with the SIP management and governance teams. 1 Source: secondary data analysis, monsoon flooding 2015, Inter-cluster coordination and INGO-ESC, November 2015. 16 8.2 Quality at entry during the AF phases also demonstrated shortcomings. Both the Palli Karma Sahayak Foundation and the Bangladesh Disaster Preparedness Centre (BDPC) selected to implement the disaster response phase lacked the requisite experience. Weaknesses in quality at entry during the AF phases are associated with the cancellation of US$40 million during the third AF phase. Supervision. 8.3 The quality of supervision is rated as Satisfactory. Two of the four Task Team Leaders were located in Dhaka, which was deemed an advantage in addressing project implementation issues. According to the Project Team, supervision missions were undertaken every six months. These missions addressed substantive issues and provided technical assistance and guidance to the project implementation team. Task Team Leaders (TTLs) and the SDF were responsive to issues raised by missions. 8.4 Based on the findings of the mid-term review, Bank supervision developed several innovative, community-friendly models and tools that helped the project to reach the poor, women, and youth. These innovations, supported by the Bank after mid-term, included: (i) A participatory targeting methodology for the village community to identify the poor and hard-core poor. After the participatory exercise had been completed, the list of selected beneficiaries was publicly displayed for three months so that beneficiaries that were missed could add themselves to the list. (ii) User-friendly community operational manuals, developed with participation of the community after mid-term to help internalize the rules and guidelines of the CDD approach, and develop a shared understanding among community leaders, members of community institutions, and facilitating staff. (iii) A social auditing mechanism created through the Social Audit Committee, designed to promote transparency and accountability of village institutions (Gram Samitis, Committees and JGs). (iv) Community Professionals drawn from experienced community leaders who voluntarily developed their own villages before offering their skills and knowledge to new communities for a fee. This system of “community-to-community” capacity-building technical assistance was institutionalized through the establishment of community professional centres. Two such centres have already been developed in the original project districts and are offering services to villages in the AF areas. This promotion of a community-to-community learning approach through community professionals contributed immensely to improving implementation effectiveness and project sustainability. 8.5 Technical assistance and South-to-South exchanges were organized, drawn from similar projects in the region. These exchanges contributed to the Foundation’s capacity and its ability to contribute to meeting project performance targets. For the second and third AF credits, the supervision team provided continuous technical and capacity-building assistance to the Foundation. This included guidance for a community to community approach for scaling up the quality of village level processes (information drawn from Country Team). 17 The team also supported the Foundation in creating a Disaster Risk Management Unit and assisting expansion of its activities in disaster-affected areas. There were relatively minor shortcomings in the quality of supervision. 8.6 Overall Bank performance is rated Moderately Unsatisfactory in accordance with the IEG methodology (whereby, in case of a split rating, the overall rating is below the line when outcome is rated below the line). 9. Borrower Performance Government Performance 9.1 Government performance is rated as Moderately Satisfactory. The Economic Relations Division of the Ministry of Finance supervised all externally financed projects and was extensively briefed by all Bank supervision missions. The Division subsequently ensured that the implementing agency, the SDF, followed Bank procedures. The Government provided good support to CDD activities implemented by the Foundation and was responsive to the need for changes during implementation, including as a result of the mid-term review. 9.2 Government performance, however, was undermined by a six-month delay in appointing members of the Foundation’s Government body. The Government and Governing Body also met too infrequently with the Foundation, preventing timely decision making. The delay of the appointment of a Managing Director (for about five months in 2010) also caused delays in implementation. There were also reportedly inaccurate or inappropriate decisions regarding project scope and costs. Implementing Agency Performance 9.3 Implementing agency performance is rated as Moderately Satisfactory. The main implementing agency was the SDF. As a relatively new institution (created in 2001), the Foundation encountered several obstacles, including the lack of a human resource policy throughout the project, the absence of a Managing Director for five months in 2010 and of a General Manager for Programs for 1.5 years in 2008 and 2009, and an ineffective M&E system. However, the Foundation adjusted to changing circumstances over the course of implementation. During the last project year, the Foundation also made progress in several areas, including: (i) completion or rollover of unfinished tasks of the flood and cyclone AF Credits to the Social Investment Program Project II; (ii) successful preparation for and negotiation of the latter project; (iii) adoption and implementation of a new human resource policy; and (iv) a significant increase in delivery capacity. The Palli Karma Sahayak Foundation was a secondary implementing agency that implemented the livelihoods activity under the AF Credit II and, together with NGOs, implemented the disaster relief activities under AF Credit III. However, there were shortcomings associated with these agencies in terms of the experience needed to respond effectively to a disaster situation. 9.4 Overall Borrower Performance is rated Moderately Satisfactory. 18 10. Monitoring and Evaluation M&E Design, Implementation and Utilization 10.1 Monitoring and Evaluation Design, Implementation, and Utilization is overall rated as Modest owing to several shortcomings in design and implementation as explained below. 10.2 M&E Design: The design of the M&E system suffered from a number of weaknesses. The definitions of performance and results indicators in project documents were unclear. The indicators were not consistent among: (i) the PAD core text (the key performance indicators section); (ii) the PAD project design summary in the log frame in Appendix 1; (iii) the indicators developed for the three AF credits; (iv) the indicators reported by the SDF in its end-of-project Results Framework; and (v) the indicators measured by independent impact assessment surveys. 10.3 M&E Implementation: The project’s design shortcomings led to difficulties in implementation of M&E activities. Insufficient attention was paid to systematic reporting on concrete, practical indicators of project achievements, such as the use of infrastructure works, development of livelihood activities, and the contribution of these activities to overall welfare. M&E implementation was also made difficult by the post-midterm review switch to a more complex project. While the original M&E system was based on the PAD indicators, the new system, based on the Project Implementation Plan, was radically different. A results framework was created to reflect the new project design, but with inconsistencies among the indicators in the AF credits and the PAD.2 10.4 M&E Utilization: Several innovations developed at mid-term helped to strengthen the monitoring and reporting processes of the project. However, there is a lack of information on how these processes were used to provide targeted assistance to under-performing areas in a manner that utilized the project learning to course-correct and improve performance in real time. Notable innovations included the introduction of a village matrix that aggregated information on community activities and achievements and that was updated monthly in a participatory manner and posted on a board outside the Gram Samiti office. This matrix proved to be a tool for transparency and accountability. A third-party process monitoring system was implemented by an independent agency, the Centre for Natural Resource Studies. This system helped to restructure the project’s approach at the mid-term review, including providing monthly recommended action tables and reporting on irregularities and corruption cases. A web-based Management Information System was developed to track project 2 For example, the ICR notes that: “The results framework established in the original PAD was not strictly followed throughout the project, nor did the AF project papers provide a solid basis upon which to track progress” (p.28). As a result, Appendix 2, “PDO Achievement Table,” in a number of instances could not use the PAD results framework indicators because they were not reported but instead used proxy indicators from the SDF results framework (see ICR, Appendix 2 tables). 19 implementation, but its effectiveness was limited due to continuous changes in project design and the fact that the SDF was not equipped to handle management of the software. 10.5 Monthly monitoring review meetings took place at district and cluster levels to track progress and resolve implementation issues. Quarterly monitoring & learning forums including all field staff were held with community involvement that led to action plans to address emerging issues. Compliance with World Bank’s Policies Safeguards 10.6 The project was designated Environmental Category B. Two safeguards policies were triggered: (1) Environmental Assessment (OP/BP/GP 4.01) and (2) Indigenous Peoples (OP 4.20). The project put in place a screening process for the sub-projects. It also developed a Tribal Development Plan, in consultation with tribal populations. Three environmental specialists participated in supervision missions. The ICR, however, contains no details of the implementation of either operational policy or compliance. During the PPAR mission’s interviews, the project team accepted that the safeguards were not handled in a structured manner in SIP I. Financial Management and Procurement 10.7 Financial management conducted by the implementing agency (SDF) was consistently reported as satisfactory. The Foundation developed an internal audit arrangement to reduce fiduciary delinquency. It also appointed an External Audit Agency to investigate compliance issues. A governance and accountability framework was introduced and a community-driven financial management approach was followed. In an environment where misappropriation of funds was a major issue for development activities, the project established mechanisms to minimize leakages and maximize the share of project funding directly used by communities for development activities. Project funds were transferred from the Foundation directly to the accounts of communities, which were given responsibility for their management. Disbursements were made in installments based on Foundation appraisals of village proposals and completion of physical milestones. 10.8 A Community Operational Manual developed with and adopted by communities included a procurement manual to serve as a guide to procurement committees for the purchase of goods and contracting of works. At completion, an estimated 2,000 community members had been trained in procurement. However, the limited staffing of the Foundation procurement team meant that the pace of delivery of these trainings was slower than optimal. Community-level social audit committees, independent from other community and project institutions, had the mandate of reviewing procurement, funds management, and decision making processes. Bank procurement rules were adhered to by the Foundation, although in some cases it found them cumbersome and not sufficiently flexible, leading to delays caused by the need for Bank approval on a number of items. 10.9 The process monitoring system mentioned in the previous section was employed for ensuring financial accountability of interventions. In an estimated 90% of cases, the 20 independent Centre for Natural Resource Studies (CNRS) addressed problem cases directly with those concerned, without needing to involve Foundation management. At project completion, corruption cases occurred in about 20 to 25 villages (2% of villages) and concerned 40 to 50 people (0.02% of people involved). In rare cases, members of Foundation staff were involved and were consequently fired. 11. Lessons  In rural areas, where citizens lie outside the reach of the state, facilitating the formation of strong institutions is fundamental to enabling the poor to access different sources of social and economic prosperity. The Social Investment Program transformed its approach to institution building at mid-term by ensuring that institutions were inclusive and responsive to the felt needs of the poor. This approach was reflective of the more effective design of rural livelihood programs under implementation elsewhere in the region at the time.  Gender inclusion, including female leadership, in village development programs is a positive but insufficient tool to ensure high quality, sustained and effective leadership over a multi-year program period. Evaluations of CDD programs have shown that for leadership to be effective, it should be (1) rotated amongst members; and that (2) leaders should have the requisite time needed and demonstrate a high level of commitment towards nurturing the less well-off group members (with regard to savings, lending, and investment behavior). Elite female members of society have the ability to extend access and information to group members, but they also may lack the time and dedication needed to ensure effective implementation of the group goals.  CDD projects offer a ready platform for outreach to disaster-affected populations, but additional disaster-related activities require thorough assessment of an implementing agency’s capacity to adapt, to manage quick influxes of AF, and to reach disaster-affected populations outside of a project area. The Social Investment Program provided a relevant and ready platform to respond to two successive natural disasters, but the SDF required rapid retooling to be able to respond to the crises in a timely way. The needs of crisis victims also differed significantly from those of other project beneficiaries. This complexity, related to the need to manage these dual streams, requires strategic management and a reconfiguration of service delivery mechanisms, project metrics, and monitoring and reporting means. 21 References Asian Development Bank. 2010. “Country Gender Assessment Bangladesh.” Manila, the Philippines.Bangladesh Planning Commission. 2015. “Millennium Development Goals: Bangladesh Progress Report 2015,” General Economics Division World Bank. 2003a. “Social Investment Program Project. Project Appraisal Document” Washington, DC: World Bank. World Bank. 2003b. “Social Investment Program Project. Project Information Document.” Washington, DC: World Bank. World Bank. 2003c. “Social Investment Program Project. Project Agreement.” Washington, DC; World Bank. World Bank. 2006. “Social Investment Program Project. Mid-term Review Special Study of the Social Investment Program Project.” World Bank. 2007a. “Social Investment Program Project. Additional Financing I (Consolidation) Project Paper.” Washington, DC: World Bank World Bank. 2007b. “Social Investment Program Project. Additional Financing I (Consolidation) Credit Agreement.” Washington, DC: World Bank. World Bank. 2007c. “Social Investment Program Project. Additional Financing II (Flood) Project Paper.” Washington, DC: World Bank. World Bank. 2008a. “Social Investment Program Project. Additional Financing II (Flood) Project Agreement .” Washington, DC: World Bank. World Bank. 2008b. “Social Investment Program Project. Additional Financing III (Cyclone) Project Paper .” Washington, DC: World Bank. World Bank. 2008c. “Social Investment Program Project. Additional Financing III (Cyclone) Project Agreement.” Washington, DC: World Bank. World Bank. 2010a. Understanding the dynamics of gender and nutrition in Bangladesh: implications for policy and programming. Washington, DC: World Bank. http://documents.worldbank.org/curated/en/2010/07/16270714/understanding-dynamics-gender-nutrition- bangladesh-implications-policy-programming. World Bank. 2010b. “Social Investment Program Project. Project Appraisal Document for SIPP II.” Washington, DC: World Bank. World Bank. 2011. “Social Investment Program Project. Implementation Completion Report.” Washington, DC: World Bank. World Bank. 2016. “Social Investment Program Project Implementation Completion Review Report.” Washington, DC: World Bank. 22 Appendix A. Basic Data Sheet Social Investment Program Project, Credit IDA-3740 Key Project Data (amounts in US$ million)- Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Total project costs 22.54 60.03 266 Loan amount 18.24 - Co-financing - - Cancellation - - Appraisal Actual or estimate current estimate Approval Date Original Credit 18.24 17.69 (97%) April 7, 2003 Additional Financing 1 8 7.76 (97%) June 21, 2007 Additional Financing 2 25 22.94 (91%) February 7, 2008 Additional Financing 3 50 9.38(19%) July 31, 2008 Cumulative Estimated and Actual Disbursements FY03 FY04 FY05 FY06 FY07 Appraisal estimate (US$M) 1.12 1.83 5.80 12.8 15.2 Actual (US$M) 1.68 4.05 8.44 14.5 18.2 Actual as % of appraisal 150 221.3 145.5 113.2 119.7 Date of final disbursement January, 2012 Project Dates Original Actual Initiating memorandum 06/24/1998 06/24/1998 Negotiations 10/25/1999 01/15/2003 Board approval 03/14/2000 03/18/2003 Signing 04/07/2003 Effectiveness 04/20/2003 Closing date 06/30/2011 23 APPENDIX A Staff Time and Cost Staff Time And Cost (Bank Budget Only) Us Dollars (Including Travel And Consultant Costs) Stage of Project Cycle No. of Staff Weeks Lending FY03 37.01 62,625.01 FY04 FY05 FY06 FY07 Total: 37.01 62,625.01 Supervision/ICR FY03 9.22 33,320.36 FY04 39.29 90,054.35 FY05 50.76 120,035.11 FY06 47.8 154,548.75 FY07 39.87 92,904.54 FY08 32.46 108,699.37 FY09 33.53 111,076.18 FY10 24.97 88,071.21 FY11 32.77 104,331.33 FY12 7.52 16,050.99 Total: 318.19 919,092.19 APPENDIX A 24 Task Team Members Names Title Unit Lending Wahida Huq Task Team Leader SASRD Jeeva Perumalpillai-Essex Lead Operations Officer SASRD Gajanand Pathmanathan Sector Manager SASRD Said Al Habsy Chief Counsel LEGMS Kishor Uprety Sr. Counsel LEGMS L. Panneer Selvan Regional Safeguard Coord P.K. Subramanian Sr. Financial Mgmt Spec. SARFM Chingboon Lee Country Coordinator SACBD Enrique Pantoja Country Officer SACBD Clive Harris Sr. Private Sector Spec. SASEI Mohi Uz Zaman Quazi Sr. Transport Engineer SASEI Khawaja Minnatullah Sr. RWSS Specialist EWDSA G.M. Khurshid Alam Sr. Private Sector Spec. SASFP Nilufar Ahmad Sr. Social Scientist SASES Jeffrey Racki Sector Manager SASES Ivonna Teresa Kratynski Sr. Finance Officer LOAG2 Zafrul Islam Sr. Procurement Spec. SARPS Agustin Litvak Procurement Advisor SARPS Suraiya Zannath Sr. Fin. Mgmt Spec. SARFM Paul J. Martin Sr. Environmental Spec. SASES Mohammad Sayeed Disbursement Officer SARFM Marilou Uy Sector Director SASFP Shakila Khan Program Assistant SASRD Deborah Ricks Program Assistant SASRD Supervision Ousmane Seck Task Team Leader SASDA Meena Munshi Former Task Team Leader SASDA Parmesh Shah Lead Rural Dev. Spec. SASDA Pushina Kunda Ng‘andwe Rural Development Spec. SASDA Sugata Talukder Operations Analyst SASDA Suraiya Zannath Sr. Financial Mgmt. Spec. SARFM Ghazali Raheem Consultant (FAO) FAO Marghoob Hussein Sr. Procurement Spec. SARPS 25 APPENDIX A Toufiq Ahmed Procurement Specialist SARPS Nadia Sharmin Environmental Spec. SASDI Sr. Natural Resources Management Yuka Makino SASDI Spec. Moyeen Sabah Social Development Analyst SASDS Brenda L. Scott Information Assistant SASDO Amani Haque Program Assistant SASDO Henry Bagazonzya Sr. Financial Sector Spec. SASFP Winston Dawes Rural Financial Spec. SASDA C.S. Renjit Community Development Specialist Consultant Shakil Ahmed Ferdausi Sr. Environmental Specialist SASDI Mohi Uz Zaman Quazi Sr. Transport Engineer SASEI Sr. Rural Development Specialist Natasha Hayward SASDA Farzana Morshed Operations Analyst SASRD Mohammad A. Sadeque Procurement Specialist Melissa Williams Operations Officer SASDA Sr. Rural Development Specialist Samik Sundar Das SASDA Amin Khandaker Consultant Mona Sur Sr. Economist SASDA Sr. RWSS Specialist. Khawaja Minnatullah SASDN (Consultant) Anne Ritchie SASDA Renate Kloeppinger-Todd Community Financing Shakila Khan Tandra Program Assistant SASDO Erwin De Nys Sr. Water Resource Spec. LCSEN Vinayak Narayan Ghatate Consultant SASDA Luis Coirolo Consultant SASDA Mohinder S. Mudahar Consultant SASAR C.S. Reddy Consultant SASAR Harbans Lal Aneja Consultant SARPS Imtiaz Uddin Ahmad Consultant Md. Abul Kalam Khalifa Consultant Subrata S. Dhar Sr. Commication Officer SAREX Sadia Afroze Chowdhury Sr. Public Health Spec. SASHD Prasad Chandra Mohan Sr. Communication Spec. Maitreya B. Das Social Assess. Program & Baseline Survey 26 Appendix B. Methodology This project performance assessment seeks to evaluate the relevance, efficiency and effectiveness of the Social Investment Program Project. Both the ICR and ICRR report on the challenges of measuring the overarching objective of the project, namely, to “develop effective and efficient financing and institutional arrangements for improving access to local infrastructure and basic services.” Challenges to assessing effectiveness were associated with the inappropriateness of the M&E system and associated indicators and the lack of available data to measure this. The main methods used to conduct this PPAR were a desk review of project documentation, interviews with Government officials and implementation agency staff, and Village level group interviews. Village Level Group Interviews. The assessment conducted fieldwork in Jamaplur, one of the two project districts. The Village selection criteria included the relative performance based on a grading system and their proximity to town centers. The assessment conducted fourteen group interviews of different stakeholder groups in four villages, choosing two high performing villages (Grade A) and two low performing villages (Grade C). D villages were not included in the sample as the village institutions and Jibikayan groups in D villages were inactive). The selection of the villages was also chosen based on their proximity to the main town – or Jamaplur town – with a high and low performing village visited near the town and a high and low performing village visited approximately 40 km away. In each village, group interviews were organized with the project identified “hard core poor, poor and youth Groups.” These groups were randomly chosen in the villages. In addition, group interviews were also organized with the Gram Samitis and Village Credit Organization members. Since only four villages were visited, the names of the villages are not published here. 27 APPENDIX B Key Village Level Evaluative Questions and Data Sources Key Evaluation Key Questions Data Source Themes Creating Strong What institutions did the project create? Have these institutions been Project Documents and Accountable sustained? How was the institution constituted? How were/are Interviews Rural decisions made? Who participates and why? How is it financially Institutions maintained? How much time? Probe role of women, youth and vulnerable. Access to How was INF allocated? Was the infrastructure distributed in an Project Documents Infrastructure equitable fashion? Has the infrastructure been maintained – who Interviews finances and maintains infrastructure? Focus Group Discussions Site Visits Rural Finance How were rural savings and lending groups formed? Who participated? SIP Phase II and III data Who trained? How were messages communicated? How would financial literacy been measured? Interview with the Social Development Fund What is the current status of the Savings Base? What is the current status of Group Lending? FGDs with the Rural What do we know about who saves, borrows, and repays? Finance Committees What was the criteria for taking a loan? The second loans, or larger loans? What is the rate/service fee? What has been the repayment rate, and has this changed over time – if so, then why? Impact on How has the INF provided by the project enabled access to livelihood Project Documents Poverty, Welfare, activities or welfare enhancements? Interviews Livelihoods Focus Group Discussions How has the mobilization of savings and access to a source of lending (Youth, helped to provide income opportunities, or enhanced welfare? How were loans used, for what kind of activities. (Probe RNFE). How did the Revolving Grant mechanism – the one time grant – stimulate access to livelihood opportunities or have an effect on welfare? What are the major constraints you face in taking up new income generating activities? Youth What special opportunities were offered to Youth? How were the Project Documents targeted? How did these activities behave differently than other Interviews activities in the project? Anecdotally, how do rural youth perceive the Focus Group Discussions project as compared to other beneficiaries’? (Youth as compared to others). Gender See Gender Assessment (AnnexAppendix C). 28 Appendix C. Gender Assessment This assessment was designed to understand the gender dimensions of the Social Investment Program, Phase I. This study was commissioned as an input into the IEG learning product “The Impacts of Community-Driven Development Interventions on Women’s Empowerment” and doubles as a project performance assessment tool to collect and consider gender relevant project information. To conduct the gender analysis, the project performance team assessed the gender elements of the CAS, PRS, project documents, and within the project documents, the M&E framework and the results indicators. It supplemented this work by integrating gender questions into fifteen Focus Groups that were convened as part of the PPAR field mission. Country Context. Bangladesh’s significant growth over the past two decades has been accompanied by notable advances in poverty alleviation, primary school enrolment, gender parity in education, a lowering of the infant and under-five mortality rates and maternal mortality ratio, an improvement in immunization coverage and a reduction in the incidence of communicable diseases.3 As part of this process, women’s opportunities and public participation have significantly changed. Major progress has been made in closing the gender gap in school enrollment at both primary and secondary levels: girls’ enrollment currently outnumbers that of boys. Growth has been spurred by the rise of opportunities in the garment industry, which has provided a large number of formal sector jobs for women and that comprises more than 80 percent of its labor force.4 Bangladesh has achieved the eighth lowest political gender gap in the world. The proportion of seats held by women in the national parliament doubled from 10 per cent in 1990 to 20 per cent in 2011.5 In addition, the country has had a female head of state for longer than any other country, which has contributed to female presence in the political sphere, as well as a changing of traditional views towards women overall. Gender in the Country Assistance Strategies (2003-2016). Gender emerged as a crosscutting issue in the 2001 CAS: “gender equity and empowerment will be treated as integral elements” in the strategic objectives. It also focused on identification of performance indicators for gender outcomes and building the capacity of the project staff to undertake gender analysis with assistance from the IDA in partnership with other donors. Overall, institutional capacity building was given prime importance, whereas gender or women’s empowerment were not explicitly considered in the design of SIPP. The 2006 CAS continued to recognize gender as a crosscutting issue for Bangladesh country programs. In preparation of the CAS, country-wide consultations were held and reiterated the role of the Bank in facilitating the development process across different development issues including “gender and social protection issues”. The 2011-2014 CAS re-emphasized gender as a cross cutting theme across all pillars of development and reaffirms its commitment to gender mainstreaming with a special focus on women’s economic empowerment. 3 “Millennium Development Goals: Bangladesh Progress Report 2015”, General Economics Division (GED), Bangladesh Planning commission, 2015. 4 Country Gender Assessment Bangladesh, Asian Development Bank, 2010. 5 http://www.ilo.org/global/about-the-ilo/newsroom/news/WCMS_234670/lang--en/index.htm 29 APPENDIX C It speaks to the importance of “gender disaggregated analysis, gender-sensitive design and monitoring” for inclusive development. Gender in the PRSP. The government strategy for reducing poverty, as reflected in the I- PRSP, focuses on four areas: (a) expanding the scope for pro-poor economic growth through expanding income and employment opportunities for the poor; (b) fostering human development for the poor; (c) providing social safety nets for the poor against various anticipated or unanticipated income (consumption) shocks; and (d) favorably influencing participatory governance by enhancing the voice of the poor and strengthening women's empowerment. The Bangladesh Poverty Reduction Strategy Paper (PRSP) and the World Bank’s CAS 2001 acknowledge that inequality and social exclusions (particularly for women) reinforces income poverty. Gender Analysis of the Project Gender was not a consideration in project design. Prior to mid-term, the reference to women’s participation is included in the reference to vulnerable groups (e.g. poor and socially excluded female-headed household, disabled, tribal people, street children, etc.). These groups were provided assistance through a Social Assistance Program (SAP) aimed at increasing the capacity of excluded groups to participate in mainstream development activities through: (a) advocacy programs; (b) occupation skills training; (c) support for pregnant women and risk-pooling pilot initiatives; (d) legal aid support; and (e) grants for the graduated trainees and worst affected natural hazard victims. The approach, prior to mid- term was found by the ICR and the PPAR to be insufficient. SAP beneficiaries received a small, one time grant and these grants were too small to have made a meaningful impact on livelihoods or welfare. Neither women nor the vulnerable especially benefited from the SAP activities. Prior to the Mid-Term, there was a heavy focus on infrastructure due to the nature of the group composition. Village Development Committees and Project Management Committees had been captured by non-poor members, and were mainly male. During this period, the voices of the poor were suppressed. Criteria set at design, including targeted assistance to female headed poor households had not been respected during the early part of the project period in despite of the fact that there were specific targets set (30 percent of the Village Development Committees; 30 percent of the CG members; and at least 25 percent of the Project Management Committees were to have been women, the poor and vulnerable.) The project only superficially included women in the infrastructure and works portion of the project. According to the PAD, ‘the targeting for community infrastructure works would be broad-based but include specific conditions for inclusion of the poor and women in the eligibility criteria for financing of sub-projects.’ There was a particular focus on the beenfits that would accrue to women from the increase in quantity and quality of water: ‘about 250,000 people (of whom at least 30 percent will be poor and women) will benefit through access to safe water, sanitation facilities, and other essential social, transport and trade infrastructure services through the implementation of about 1,800 community infrastructure sub-projects’. The PAD theorizes that ‘investment in safe drinking water and APPENDIX C 30 sanitation, by improving the health of rural populations as well as reducing time spent, especially by women, in transporting water, will contribute to poverty alleviation and increase the productive capacity of rural people, particularly women’.6 Benefits of improved water supply both in terms of quantity and quality are expected to have a positive effect on health outcomes, and that these health benefits in turn would lead to labor productivity and income gains via a variety of direct and indirect pathways, which would also benefit women. Since none of the results data is disaggregated, it is impossible to report whether these results were achieved, including for women. The original community groups were also not provided with basic gender awareness training, knowledge and understanding on prevention of Violence against Women. The staff of SDF, CSO, and PO are also lacking clear understanding of gender awareness building and women empowerment focused elements. This was a missed opportunity to raise awareness around such issues as the dowry, early marriage, women’s health, pregnancy, and hygiene, and as stated above, violence against women. Rather, the first few years of the project were characterized by a focus on “the hardware” (INF) rather than the software necessary to ensure equitable and sustainable rural development Gender integration was fully achieved after the Mid-term and the Restructuring of the Project. This is due mainly to the support of a key member of the Bank’s Rural Livelihoods teams, which put poverty and gender squarely at the center of its rural development programs. The MTR recommended and the project implemented a full restructuring of the project’s village organizational structures. As such,after the MTR, women and girls constituted 95 percent of the membership of the 1407 village level institutions, i.e. Gram Samiti, Gram Parishad and the 20,414 village level Jibikayan Groups. Seventy-seven percent of the total 1224 no. of Gram Samitis are active and led by women. Interestingly, women appear to be the de facto participants of this project since many of the men residing in these areas were too busy or too uninterested in joining once the project was gearered to being truly pro-poor. Many of the men in the project areas are day laborers and farmers. All the groups received financial skill trainings, including saving and credit management training. The trainings were mostly limited to the leader and cashier of each institutions (Gram Samiti, Jibikayan Groups, VCOs, Social Audit Committees). While many of the recipients of the trainings were women, one of the downfalls of this design feature was that the women did not rotate. So while some women were trained, these skills were not adequately imparted to most female members of the group (non-leaders). This finding is salient: it raises a more complex questions as to which women are empowered, whether empowerment is limited to a few of the many women who behave as general participants in CDD project, versus their leadership. It begs for more research across CDD programs on how women join, their need to be otherwise socially connected or financially endowed to assume leadership roles, and ultimately the distribution of benefits cross these different tiers of women groups. 6 SIPP, PAD 31 APPENDIX C Gender Related Results: ICR and PPAR Findings The original project design and field implementation for the first 3-4 years of the project did not focus on gender specific demands or pay any special attention to address women’s needs in the community. As mentioned in the ICR, the community infrastructure subproject has suffered due to elite capture, where women were only engaged as paid labor in road construction and maintenance, but did not participate in planning and management. Although the project did not explicitly target women, active participation after mid-term in accessing savings and credit activities by women has certain positive impacts in the community as well as the households. The findings from the field research suggests that, the formation of women-only community institutions, their access to credit, and livelihood interventions helped build women’s capacity for collective decision-making and taking actions. It also helped strengthening social capital as measured by community cohesion and inclusion in the community institutions which categorically enhanced women’s empowerment -in the form of participation in community decisions and social and legal awareness. 
 Evidence of women’s participation in their own development process can be found in sources outside of the project documentation. For example, the Asian Development Bank’s Country Gender Assessment Report (2010) found that a large number of women have been integrated into community institutions and have taken responsibility for rural and urban development, which in turn has paved the way for women’s political empowerment (many of whom are now elected as members of the local government councils. 32 Appendix D. Village Grading Criteria Allocated Indicators Marks awarded to indicators marks TARGETING, UPDATING LIST & INCLUSIVENESS 5 Updated Wealth Ranking/ PIP list is  Reports available and updated within 6-months. - (2) available.  Reports available and updated within 6 – 12 months. - (1) 2  Reports available and not updated for last 12-months. - (0) Unresolved Complaints about the list  No conflict arisen about the checklist and do the process follow (1) 1  Otherwise (0) Inclusive Poor and Hard Core Poor  At least 80% poor and hard core poor included in JG (2)  About 70 - 79% poor and hard core poor included in JG (1) 2  <70% poor and hard core poor included in JG (0) Functioning of GP& GS 9 Regular meeting of GP held in time as per  Participation with quorum for last 3-meetings.-(3) COM  Last 3GP meetings held in proper time but 1 meeting held without quorum - (GP meeting - quarterly).- (2) 3  Otherwise. – (0) Regular meeting of GS held in time as per  Participation with quorum for last 3-meetings.-(3) COM  Last 3GS meetings held in proper time but 1 meeting held without quorum 3 (GS meeting- monthly).- (2)  Otherwise. – (0) 33 APPENDIX D Contents of meeting are properly  All meeting minutes of Gram Parishad, Gram Samiti and SAC decisions are discussed and decisions recorded. properly documented by the community and disseminated in time (at least last 3-meeting).- (3)  All meeting minutes of Gram Parishad, Gram Samiti and SAC decisions are not 3 properly documented by the community.- (1)  Meeting discussions are not rightly addressed and decisions not properly recorded.- (0) Knowledge on Project principles and role  Adequate. – (3) & responsibilities of Office Bearers  Moderately.- (2) 3  Un-satisfactorily. – (0) Eligibility Access of Village Fund  3rd installments of VDF - (5) (Applicable for batch-1 villages)  At least 2nd installment of VDF - (2)  At least 1st installment of VDF – (1)  Other wise – (0) Eligibility Access of Village Development  At least 2nd installments of IDF & CISF and 1st installment of SF (5) 5 and Risk Reduction Fund (VDRRF)  1st installment of IDF, CISF & SF (3) (Applicable for batch-3-9 villages)  Otherwise – (0) Construction of GS Office ( deed,  GS Office has been constructed and utilized by the community as per COM. – location, quality) and it’s use (3).  GS office has been constructing as per COM but not yet handed over to GS – 3 (2)  GS office has been constructed as per COM but not followed the rules (deed, location, & quality). – (0) Number of One-time grant recipients who  More than 20% of the assisted vulnerable are eligible to get SF loan– (2) are eligible to avail Shabolombi loan  Other wise –(0) 2 YOUTH SKILL TRAINING AND EMPLOYMENT 9 APPENDIX D 34 Use of youth skill development fund  At least 70% youth received skill development fund. - (2)  60 - 69% youth received skill development fund. - (1) 2  < 60% youth received skill development fund. - (0) Employment of youth (self & wage  At least 70 % youth employed (3) employment)  60- 69% youth employed. - (2) 3  50-59% youth employed. - (1)  < 50% youth employed. - (0) CRR of youth loan  At least 90% CRR (4)  80-89% CRR (3)  70-79%CRR (2) 4  60-69%% CRR (1)  <60% CRR (0) UTILIZATION OF SHABOLOMBI FUND 20 Transfer of SF from GS to VCO Accounts  GS transferred SF to VCO’s Bank Accounts within 15-days. – (1) 1  More than 15-days. – (0) Distribution of SF loan  More than 80% JG members received SF - (5)  At least 70-79% JG members received SF – (4) 5  60- 69% JG members received SF – (3)  At least 50- 59% JG members received SF – (2)  < 50% JG members received SF - (0) Idle fund in VCO’s Bank Account  Idle money for more than 15-days in the Bank (after receipt of installments) or 2 20% of revolving fund lying idle in the bank, If yes (0) and if no (2) OTR of SF  At least 95% OTR for last 3-months. – (4)  90-94% OTR for last 3-month – (3)  85-89% OTR for last 3-month – (2) 4  80-- 84 % OTR for last 3-months. – (1 ) < 80% OTR for last 3-months.- (0) CRR of SF  At least 90% CRR – (5)  80- 89% CRR – (4) 5  70-79% CRR – (3) 35 APPENDIX D  60- 69% CRR – (2 )  < 60% CRR.- (0) Recycling Ratio  The village receiving score above 1.5 - (3)  The village receiving score 1.3 - 1.4 - (2) 3  The village receiving score 1 and less than 1- (0) BOOKS OF ACCOUNTS OF GS & VCO 6 Satisfactorily keeping and maintaining  Books of Accounts maintained and updated as per COM. – (3) Books of Accounts of GS  Books of Accounts maintained but not updated in last one month. – (2) 3  Books of Accounts maintained un-satisfactorily and not updated. – (0) Satisfactorily keeping and maintaining  Books of Accounts maintained and updated as per COM. – (3) Books of Accounts of VCO.  Books of Accounts maintained and not updated for last one month. – (2) 3  Books of Accounts maintained un-satisfactorily and not updated. – (0) Procurement is done as per COM.  At least 80% of Procurement has been done as per COM guideline.- (2) 2  Less than 80% of Procurement has been done as per COM guideline.- (0) Functioning of SAC 6 Regularly meeting minutes recorded and  SAC meeting is held at least three times in a quarter during each of last four findings presented at GP meetings consecutive quarters with recording of minutes and its presentation in Gram Parishad – (3)  At least two meetings are held in a quarter during the last four consecutive 3 quarters with recording of minutes and its presentation in Gram Parishad – (2)  At least one meeting is held in a quarter during the last four consecutive quarters with recording of minutes and its presentation in Gram Parishad (1)  Otherwise – (0) Actions taken by GS to implement GP  SAC recommendations approved by GP have fully implemented for last 3- decision on SAC recommendations meetings. – (3)  SAC recommendations approved by GP have partly implemented for last 3- 3 meetings. –(2)  SAC recommendations approved by GP not implemented for last 2 meetings .- (0) APPENDIX D 36 Updated Village Information displayed  Update project information are displayed and accessible.- (3) and accessible.  Information are partially updated, displayed and accessible.- (2) 3  Information are not regularly updated and displayed. – (0) Action taken on misappropriation of fund.  No misappropriation occurred.- (3)  All money has been recovered -(2) 3  80% money have been recovered -(1)  No action taken. –(0) AMT recommendation resolved/  AMT recommendations have fully acted up on and corrected – (4) implemented  AMT recommendations have partially resolved. – (2) 4  No actions have yet been taken against AMT’s recommendation. – (0) Preparing & maintaining correct figures in  Village information correctly reported and available in the village level for last Village Matrix two months. – (4)  Village information partially correct and available in village level for last two 4 months. – (2)  Village information inconsistent and incorrect for last two months. – (0) Required numbers of CP produced  5 CPs produced and providing services. –(3)  At least 3 CPs produced and providing services. – (2) 3  < 3-CPs produced and providing services. – (0) Regular savings  At least 80% of the NJG members are depositing Savings regularly for last 3 months - (7)  About 70- 79% of the NJG members are depositing Savings regularly for last 3 months. – (4) 7  About 50- 69% of the NJG members are depositing Savings regularly for last 3 months. – (2)  Less than 50%. – (0) Operation & Maintenance of CIWs.  CIWs are maintained as per plan.- (2 )  There is a plan but not implemented – (1) 2  Other wise – (0) IMPLEMENTATION OF GAAP & CAP 4 Implementation of CAP  At least one CAP exercise has been conducted within the last 1 year and at least 2 70% of the recommendations in the action plan have been implemented (2) 37 APPENDIX D  At least one CAP exercise has been conducted within the last 1 year and at least 50% of the recommendations in the action plan have been implemented (1)  Other wise – (0) Implementation of GAAP  Risk has been identified and mitigation measures have been taken according to plan (2)  Updating of GAAP is being done in every six months and presented in the GS 2 (1)  Other wise – (0) Total Marks 100 Grade Score From 80 to above At least 95% OTR and 90% CRR of SF loan will be ensured for A grade villages. Otherwise, if any village get 80 score will not be considered as A grade. Within 70-79 B Within 60-69 C Below 60 D 38 Appendix E. List of persons Consulted. Name Designation Mazibur Rahman Assistant Executive Director, Bangladesh Executive Education Services Arun Kumar Mondal Regional Manager (Monitoring, Evaluation and Learning) Shahid Rahmot Kadir Specialist, Youth & Employment, Social Development Foundation Saiful Islam Specialists, Livelihoods, Social Development Foundation Lutfor Rahman Management, ME&L, Social Development Foundation Fazul Kader Deputy Managing Director, Palli Karma- Sahayak Foundation (PKSF) A.Z.M. Sakhawat Hossain Managing Director, Social Development Fund Sugata Talukdar Operations Analyst, The World Bank Mizanur Rahaman Specialist, Community Finance Mrittunjoy Roy Assistant Manager, MIS Sri Krishna Bhowmik Appraiser, Appraisal Monitoring Team Shamsul Alam District Community Finance Ashraful Alam District Officer, Institute Capacity Building Badsa Mia Community Facilitator, Sonakata Monirul Hasan Community Facilitator, Kastosinga Alal Mia Community Facilitator, Balujhuri Nurum Nabi Community Facilitator, Ujanpara 39 Appendix F. Jamalpur Map (IEG Site Visits) 40 Appendix G. Borrower Comments There are no comments received from the borrower.