Document of The World Bank Report No: ICR00002900 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-H1020, IDA-47190 & TF-53677) ON A GRANT IN THE AMOUNT OF SDR 35.2 MILLION (US$ 51 MILLION EQUIVALENT) FOR A REACHING OUT-OF-SCHOOL CHILDREN PROJECT AND ADDITIONAL FINANCING CREDIT IN THE AMOUNT OF SDR 23.1 MILLION (US$ 35 MILLION EQUIVALENT) TO THE PEOPLE’S REPUBLIC OF BANGLADESH December 23, 2013 Human Development Sector South Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective November 30, 2013) Currency Unit = Taka Tk. 1.00 = US$ 0.0129 US$ 1.00 = Tk. 77.68 FISCAL YEAR July 1 – June 30 ABBREVIATIONS AND ACRONYMS ACF Award Confirmation Form AF Additional Financing AOP Annual Operation Plan AY Academic Year BANBEIS Bangladesh Bureau of Educational Information and Statistics BCCP Bangladesh Center for Communication Programs CAMPE Campaign for Popular Education in Bangladesh CAS Country Assistance Strategy CBO Community-based Organization CHF Swiss franc CHT Chittagong Hill Tracts CMC Center Management Committee CPIA Country Policy and Institution Assessment DGA Development Grant Agreement DP Development Partner DPE Directorate of Primary Education DPP Development Project Proforma EA Economic Analysis ECNEC Executive Committee for National Economic Council EFA Education for All EMIS Education Management Information System ERP Education Resource Provider ESP Education Service Provider FA Financing Agreement FMR Financial Management Report GER Gross enrollment rate GOB Government of Bangladesh GPS Government Primary Schools ICCR Implementation Completion and Results Report (also referred to as “ICR”) IDA International Development Association IP Indigenous Peoples I-PRSP Interim Poverty Reduction Strategy Paper ISDS Integrated Safeguards Data Sheet KI Key Indicator LC Learning Center LGED Local Government Engineering Department M&E Monitoring and Evaluation MIS Monitoring Information System MoPME Ministry of Primary and Mass Education MoU Memorandum of Understanding MTR Mid-Term Review NER Net enrollment rate NFE Non-formal education NGO Non-Government Organization PAD Project Appraisal Document PD Project Director PEDPII Primary Education Development Project II PEDP3 Primary Education Development Program 3 PD Project Director PDO Project Development Objective RNGPS Registered Non-government Primary Schools ROSC Reaching Out-of-School Children project (also referred to as “ROSC-1”) ROSC-II Reaching Out-of-School Children Project II ROSCU Reaching Out-of-School Children Unit SDC Swiss Agency for Development Cooperation SKT Shishu Kallyan Trust SWAp Sector-wide Approach Tk Bangladesh Taka UEO Upazila Education Officer Vice President: Philippe Le Houerou Country Director: Johannes C.M. Zutt Sector Manager: Amit Dar Project Team Leaders: Dilip Parajuli/Subrata S. Dhar ICR Team Leaders: Susan Opper/Syed Rashed Al-Zayed Josh BANGLADESH REACHING OUT-OF-SCHOOL CHILDREN PROJECT CONTENTS DATASHEET ................................................................................................................. v 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 5 3. Assessment of Outcomes .......................................................................................... 15 4. Assessment of Risk to Development Outcome......................................................... 24 5. Assessment of Bank and Borrower Performance ..................................................... 24 6. Lessons Learned ....................................................................................................... 27 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 28 Annex 1. Project Costs and Financing .......................................................................... 30 Annex 2. Outputs by Component ................................................................................. 31 Annex 3. Economic and Financial Analysis ................................................................. 37 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 39 Annex 5. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 41 Annex 6. List of Supporting Documents ...................................................................... 48 MAP DATASHEET A. Basic Information Bangladesh Reaching Country: Bangladesh Project Name: Out-of-School Children Project IDA-H1020-BD, IDA- Project ID: P086791 L/C/TF Number(s): 47190,TF-53677 ICR Date: 12/20/2013 ICR Type: Core ICR GOVERNMENT OF Lending Instrument: SIL Borrower: BANGLADESH Original Total XDR 35.20M Disbursed Amount: XDR 51.13M Commitment: Revised Amount: XDR 58.30M Environmental Category: B Implementing Agencies: Directorate of Primary Education (DPE), Ministry of Primary and Mass Education (MoPME) Cofinanciers and Other External Partners: Government of the Swiss Confederation B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Effectiveness: Concept Review: 11/03/2003 Original Grant 09/22/2004 09/22/2004 Additional Fin. 06/07/2010 02/22/2011 Appraisal: 03/25/2004 Restructuring(s): 04/04/2013 Approval: 06/17/2004 Mid-term Review: 07/31/2007 02/10/2008 Closing: 06/30/2010 06/30/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Negligible to Low Bank Performance: Satisfactory Borrower Performance: Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Satisfactory Implementing Quality of Supervision: Satisfactory Satisfactory Agency/Agencies: Overall Bank Overall Borrower Satisfactory Satisfactory Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): 03/11/2004 Problem Project at any Quality of No None time (Yes/No): Supervision (QSA): DO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Other social services 55 55 Primary education 45 45 Theme Code (as % of total Bank financing) Education for all 50 50 Gender 25 25 Participation and civic engagement 25 25 E. Bank Staff Positions At ICR At Approval Vice President: Philippe H. Le Houerou Praful C. Patel Country Director: Johannes C.M. Zutt Christine I. Wallich Sector Manager: Amit Dar Michelle Riboud Project Team Leaders: Dilip Parajuli/Subrata S. Dhar Amit Dar/Hena G. Mukherjee Susan Opper/ ICR Team Leaders: Syed Rashed Al-Zayed Josh Susan Opper/ ICR Primary Authors: Syed Rashed Al-Zayed Josh F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The key objective of the Project is to reduce the number of out-of-school children through improved access, quality and efficiency in primary education, especially for the disadvantaged children, in support of GOB's national Education for All (EFA) goal. Revised Project Development Objectives (as approved by original approving authority) The PDO was unchanged during the project. (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Indicator 1 : The number of out-of-school children reduced by about 0.75 million. Value quantitative or 2.5 million 2 million 1.75 million 1.66 million Qualitative) Date achieved 06/30/2004 09/16/2004 02/22/2011 12/31/2012 Comments Target surpassed by more than 10%: ROSC reached total of 0.84 million (incl. % children. At Additional Financing, the target for reducing "Number of out-of- achievement) school children reduced” was revised to 0.75m from the original 0.50m. Percentage of students reaching expected competency level based on teacher's Indicator 2 : assessment in Bangla and Mathematics not less than 65%. Value Not Applicable (schools did quantitative or not exist prior to the project) 65% 80% Qualitative) Date achieved 06/30/2004 06/30/2004 12/31/2012 Comments (incl. % Target surpassed by nearly 25%. achievement) Indicator 3 : Student grade completion rate not less than 85%. Value Not Applicable (schools did quantitative or not exist prior to the project) 80% 85% 90% Qualitative) Date achieved 06/30/2004 06/30/2004 02/22/2011 12/31/2012 Comments (incl. % Target surpassed. At Additional Financing, the target was revised to 85%. achievement) Student transition rate to public and NGO schools to Grade 4 (from Grade 3) not Indicator 4 : less than 80%, or to Grade 6 (from Grade 5) not less than 50%. Not less than Not less than 80% 80% from Value from Grade 3 to 4 Grade 3 to 4 Not Applicable (schools did quantitative or not exist prior to the project) Qualitative) Not less than 70% Not less than 70% from Grade 5 from Grade 5 to 6 50% from to 6 Grade 5 to 6 Date achieved 06/30/2004 06/30/2004 02/22/2011 12/31/2012 Target surpassed by 40%. At Additional Financing, based on implementation Comments experience, the transition rate to Grade 6 was reduced to "not less than 50%" (incl. % from the original "not less than 70%." Transition rate from Grade 3 to 4 achievement) consistently remained over 80% but this indicator became less important over time as ROSC learning centers (LCs) provided the full five years of education. At preparation it was felt that some LCs might only be able to provide 3 years of education so that students would need to transition to other schools after Grade 3. Indicator 5 : About 2.9 million student years supported for beneficiary students. Value quantitative or 0 1.4 million 2.9 million 2.94 million Qualitative) Date achieved 06/30/2004 06/30/2004 02/22/2011 12/31/2012 Comments Target met. At Additional Financing, this was revised to 2.9m from the original (incl. % 1.4m. achievement) Indicator 6 : Average student attendance rate not less than 75%. Value Not Applicable (schools did quantitative or not exist prior to the project) Not less than 75% 90% Qualitative) Date achieved 06/30/2004 06/30/2004 12/31/2012 Comments (incl. % Target surpassed by nearly 20%. achievement) Indicator 7 : Average teacher attendance rate not less than 90%. Value Not Applicable (schools did Note less than quantitative or not exist prior to the project) 90% 90% Qualitative) Date achieved 06/30/2004 06/30/2004 12/31/2012 Comments (incl. % Target met. achievement) Students in each grade to have on average, the same number of textbooks as Indicator 8 : those in corresponding grades in the formal primary education system. Value Not Applicable (schools did quantitative or not exist prior to the project) 100% 99% Qualitative) Date achieved 06/30/2004 06/30/2004 12/31/2012 Comments Target met. The project ensured textbook provision in ROSC LCs was same as (incl. % (the National Curriculum and Textbook Board) textbooks provided to formal achievement) primary schools. (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Indicator 1 : Number of LCs established. Value (quantitative 0 15,000 22,500 23,695 or Qualitative) Date achieved 06/30/2004 06/30/2004 02/22/2011 12/31/2012 Comments Target surpassed. At Additional Financing, this was revised to 22,500 (including (incl. % an additional 7,500 in new Upazilas). Adding the Shishu Kallyan Trust schools achievement) covered by the project brings the overall total schools to 23,768. Indicator 2 : Number of students initially enrolled in LCs Value quantitative or 0 500,000 750,000 778,147 Qualitative) Date achieved 06/30/2004 06/30/2004 02/22/2011 12/31/2012 Comments (incl. % Target surpassed. At additional financing, target was revised to 750,000. achievement) Indicator 3 : Bi-annual disbursement of allowances and grants to be maintained regularly Value Not Applicable (schools did quantitative or not exist prior to the project) Yes Yes Qualitative) Date achieved 06/30/2004 06/30/2004 12/31/2012 Comments This was one of the basic disbursement procedures that was monitored (incl. % rigorously and it proceeded as planned. achievement) Community Outreach efforts as measured by number of participants in CMC Indicator 4: basic training sessions Value quantitative or 0 75,000 112,500 122,667 Qualitative) Date achieved 06/30/2004 06/30/2004 02/22/2011 12/31/2012 Comments Target surpassed by nearly 10%. The original target had estimated 5 participants (incl. % per LC/CMC. achievement) ROSC Committee is functional and provides policy and implementation Indicator 5 : guidance to ROSCU for project operation. Value Not Applicable (schools did quantitative or not exist prior to the project) Yes Yes Qualitative) Date achieved 06/30/2004 06/30/2004 12/31/2012 Comments ROSC Committee meetings were held with regularity and provided necessary (incl. % support to ROSCU on project implementation, and leadership in garnering achievement) broad-based support and ownership at the national level. Indicator 6 : M&E activities carried out as planned Value Not Applicable (schools did quantitative or not exist prior to the project) Yes Yes Qualitative) Date achieved 06/30/2004 06/30/2004 12/31/2012 All M&E activities in the PAD were carried out. These included: (i) collection and processing of self-reported data from LCs every six months to process Comments disbursement of grants and allowances; (ii) verification through 20% sample (incl. % field monitoring every year; (iii) impact evaluation baseline and follow-up achievement) surveys; (iv) quantitative (learning levels, tracer study) and qualitative assessments (service providers); (v) NFE study; and (vi) performance audits of grants and allowances. (ii)-(vi) were commissioned to third parties. G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 09/16/2004 Satisfactory Satisfactory 0.00 2 03/08/2005 Moderately Satisfactory Moderately Satisfactory 2.38 3 08/29/2005 Moderately Satisfactory Moderately Satisfactory 2.39 4 02/27/2006 Satisfactory Moderately Satisfactory 2.90 5 08/24/2006 Satisfactory Satisfactory 5.68 6 02/06/2007 Satisfactory Satisfactory 8.79 7 08/01/2007 Satisfactory Satisfactory 14.97 8 01/24/2008 Satisfactory Satisfactory 17.44 9 07/29/2008 Satisfactory Satisfactory 23.23 10 01/28/2009 Satisfactory Moderately Satisfactory 26.59 11 07/20/2009 Satisfactory Moderately Satisfactory 31.62 12 01/29/2010 Satisfactory Satisfactory 41.91 13 09/05/2010 Satisfactory Satisfactory 49.00 14 03/21/2011 Satisfactory Satisfactory 59.67 15 09/26/2011 Satisfactory Satisfactory 77.41 16 04/11/2012 Satisfactory Satisfactory 80.22 17 10/25/2012 Satisfactory Satisfactory 85.13 18 06/05/2013 Satisfactory Satisfactory 86.48 H. Restructuring (if any) ISR Ratings at Amount Restructuring Board Restructuring Disbursed at Reason for Restructuring Date(s) Approved DO IP Restructuring & Key Changes Made PDO Change in USD millions 02/22/2011 na S S 56.67 (i) Scale up operations to 30 additional Upazilas (Additional Financing); (ii) extend closing date to align with AF; (iii) adjust targets to align with AF and make more realistic; iv) increase IDA financing percentages in selected categories to meet changed disbursement needs as result of AF scale up, extension of closing date, and closing of Swiss co-financing TF-53677. 04/04/2013 S S 86.48 Re-allocate amounts of the na original Grant H01020-BD to each category. I. Disbursement Profile 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. The project was prepared at a time of general recognition that Bangladesh had made impressive gains in access to primary education, but significant challenges remained for reaching remaining millions of out-of-school children and delivering quality education to all. The out-of- school population was particularly disadvantaged: living in extreme poverty, in remote rural areas (including hard to reach regions in the Chittagong Hill Tracts and other tribal regions), on the streets, and with displaced families who had migrated from rural to urban areas. 2. Enrollment trends. By 2002, Bangladesh had a primary gross enrollment rate of 92 percent (up from 76 percent in 1991) with gender parity. Nearly 17.7 million students were enrolled in 78,000 primary schools across the 484 Upazilas.1 Yet, dropout rates averaged nearly 50 percent, and together with the net enrollment rate (NER) of 80 percent at the primary level, suggested that well over three million children of primary school age (6-10 years) were out of school either because they had never enrolled or had dropped out. Close to 8 million children aged 6-14 (the target age for the ROSC project) were also out of school in 2001. 3. Government response. The Government issued an Interim-Poverty Reduction Support Program (March 2003) that sought to narrow an ever-widening “quality divide” in education between the rich and the poor, and to increase access to education for the poor to ensure an efficient and sustainable approach to equitable, quality education. The related agenda in the education sector, the National Plan of Action for Education for all (EFA) 2003-2015, embraced all the international EFA goals for making education compulsory, accessible and all-inclusive. The Government enhanced its response by: (a) implementing (from July 2003) a second Primary Education Development Program (PEDPII) aimed at boosting access to schooling, improving education quality, and ameliorating overall management and oversight of the state’s provision of formal primary education; and (b) instituting (from fiscal year 2002/2003) a stipend program targeted at the poorest 40 percent of enrolled primary school students, estimated at the time to number over four million annually. Non-formal schools (“Learning Centers,” LCs) which provided primary education lay outside the ambit of PEDPII. 4. Rationale for Bank involvement. IDA supported GOB’s aspiration to reduce poverty substantially within a generation. The Country Assistance Strategy (CAS, 21326-DB dated February 8, 2001) took a three-pronged approach to: (i) consolidate gains in human development and support initiatives to address the next generation of development challenges in education, health and nutrition; (ii) implement an integrated approach to rural development; and (iii) accelerate and broaden private sector-led growth. Improving the quality of primary education and expanding access for the disadvantaged were key areas of IDA’s support. 5. While IDA and eight other Development Partners (DPs) were supporting primary education through the PEDPII (effective in May 2004), there was general recognition that, even if PEDPII achieved its targets, this would still leave as many as 1.8 million children aged 6-10 out of school by 2009. To complement PEDPII, which covered four categories of primary schools in the formal sector, the GOB decided that it needed an alternative arrangement to address the challenge of out- 1 Upazilas are sub-districts, and are the second lowest tier of regional administration in Bangladesh. 1 of-school children more fully. In 2004, Bangladesh became eligible for an IDA Grant which presented an opportunity to utilize grant funds for marginalized children whom PEDPII could not reach. ROSC was envisaged and designed as a pilot project to complement PEDPII to test alternative mechanisms for quality education service delivery. 1.2 Original Project Development Objectives (PDO) and Key Indicators 6. The PDO stated in the Project Appraisal Document (PAD) was: to reduce the number of out- of-school children through improved access, quality and efficiency in primary education, especially for the disadvantaged children, in support of GOB’S national EFA goals. The PDO statement in the Development Grant Agreement (DGA) was consistent with that in the PAD. 7. The PDO was to be measured by eight outcome indicators, also consistently stated in the PAD and DGA as follows: (i) the number of out-of-school children reduced by about 0.5 million; (ii) percentage of students reaching expected competency level based on teacher’s assessment in Bangla and Mathematics not less than 65 percent; (iii) student grade completion rate not less than 80 percent; (iv) student transition rate to public and NGO schools to Grade 4 (from Grade 3) not less than 80 percent, or to Grade 6 (from Grade 5) not less than 70 percent; (v) about 1.4 million student years supported for beneficiary students; (vi) average annual student attendance rate not less than 75 percent; (vii) average teacher attendance rate not less than 90 percent; and (viii) students in each grade to have, on average, the same number of textbooks as those in corresponding grades in the formal primary education system. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 8. The PDO was not revised. However, four of the PDO outcome indicators were adjusted in 2010 to reflect the broadened project scope under Additional Financing (AF). Targets for two outcome indicators were increased: the number of out-of-school children to be reduced was changed to 0.75 million (up from the original 0.5 million), and the number of student-years was changed to 2.9 million (up from the original 1.4 million). Also, adjustments were made in two additional outcome indicators to set more realistic targets on the basis of actual implementation experience. The target for student grade completion rate was raised to not less than 85 percent (originally 80 percent), and the target for student transition rate to public and NGO school to Grade 6 (from Grade 5) was reduced to not less than 50 percent (originally 70 percent). (See the ICR Datasheet). 1.4 Main Beneficiaries 9. Direct beneficiaries were an originally estimated 500,000 children (increased to 750,000 with the AF) to benefit from the expansion of LCs. The project prioritized access for the “disadvantaged” and activated this by targeting “needy” Upazilas according to four indicators: (i) NER lower than 85 percent, (ii) gender gap in enrollment greater than two percent, (iii) primary cycle completion rate not more than 50 percent, and (iv) poverty rate above 30 percent.2 The project would also benefit Shishu Kallyan Trust (SKT) schools under a program that MoPME was already operating to provide education to disadvantaged working children in urban areas. 2 The poverty headcount ratio at national poverty line was 40 percent in 2005. 2 10. Indirect beneficiaries included: parents/guardians/ward members of over 20,000 newly created Center Management Committees (CMCs)3 who would be supported by training to build up administrative and technical skills; and 24,000 teachers for ROSC-supported schools (labeled “Learning Centers” (LCs)), 80 percent of whom female and newly selected from among local communities, would receive training and salaries. The Ministry of Primary and Mass Education (MoPME) and Directorate of Primary Education (DPE) would benefit from consolidating their capacity to support the education system and from establishment of the project’s Management Information System (MIS) for data collection and analysis to inform program and policy decisions. 1.5 Original Components (as approved) 11. The main thrust of the project, representing over ninety percent of the IDA financing, was to increase access to quality primary education through direct support to communities to establish LCs. The communities would be assisted by local level institutions – mostly non-governmental organizations but also local education administrators. Originally, 60 Upazilas were selected, with an understanding that the number of Upazilas could be increased over the project’s lifetime. Project operations were organized into four main components and 12 sub-components, summarized as follows: Component 1: Improving Access to Quality Education for Out-of-School Children (Total estimated US$53.4 million, of which IDA = $US47.0 million). The objective was to increase access to, and facilitate completion of, primary schooling of disadvantaged children aged 6-14 by testing two approaches: (i) in 60 percent of the [n.b. originally] selected Upazilas, both education allowances for eligible students and grants to LCs would be provided; (ii) in the remaining Upazilas, only grants would be provided to LCs. One sub-component “Education Allowances to Out-of School Children” would disburse allowances twice a year to qualifying children through bank accounts managed by their mothers or guardians (total annual allowances per student ranged from BDT800 for class 1-3 students, to BDT970 for class 4-5 students). A second sub-component “Grants to Learning Centers/Schools” would disburse twice a year through bank accounts managed by the CMCs (grants per annum ranged from BDT25,000 for class 1-3 to about BDT31,000 for class 4-5). Financing was linked to the number of children enrolled. A portion of the grants was to be earmarked for quality enhancements and improving school management; the remainder used at the discretion of the CMC for teachers’ salaries (within defined parameters). A third sub-component “Shishu Kallyan Trust (SKT) Schools” financed grants to cover a part of their program management expenditures and education allowances for disadvantaged working children in urban areas who enrolled in SKT schools (SKT school grants ranged between BDT25,000-30,000 per annum and educational allowances were BDT1400 per student per year; the allowances aimed to meet direct and indirect costs of schooling). Component 2: Communications and Social Awareness (Total estimated US$2.7 million, of which IDA = $US2.0 million). The objective was to increase awareness about primary education in general, and about the project in particular. A “Social Awareness” sub-component focused on development and delivery of communications to stakeholders at family, community, Upazila, and national levels; development and distribution of informational materials to potential Education Service Providers (ESPs) and Education Resource Providers (ERPs); and logistical support for communications between the LCs and Upazila Education Officers (UEOs) in the government 3 The CMC would be composed of five parents/guardians, one female ward member, one Upazila Education Office representative, and one Education Service Provider, with the teacher as the member secretary to the CMC. 3 administration. A “Community Mobilization” sub-component covered development of training modules and related materials, activities at community level and preparation of Upazila community mobilization reports. Component 3: Project Management and Institutional Strengthening (Total estimated US$2.4 million, of which IDA = US$1.0 million). This component supported necessary activities for sound project management, as well as logistical and technical support. A “Project Management” sub-component provided for the operation of: (i) the ROSC Committee through which MoPME was to carry out its oversight role, and (ii) the ROSC Unit (ROSCU), headed by a Project Director, for day-to-day implementation. The “Institutional Capacity Building” sub-component covered training in management, educational development and financial management to strengthen central and Upazila administrations and local communities. The “Networking to Improve Quality” sub-component supported the ROSCU to create a network of ERPs and other experts for quality assurance and technical advice on education service provision. Component 4: Monitoring, Evaluation and Research (Total estimated US$4.3 million, of which IDA = US$1.0 million). The objective was to set up and maintain an effective system to: (i) monitor the use of grants and education allowances, and (ii) evaluate effects of these types of support through action research, thematic studies, and impact assessments. The majority of evaluations were to be carried out by contracted, independent agencies. One of the four sub- components, “Analytical Studies of Non-formal Education,” would support the GOB in institutionalizing a national, non-formal education framework. 1.6 Revised Components 12. The original components were not revised. However, under the Additional Financing approved in 2010, the scope of the project, particularly affecting Components 1 and 2, was expanded (see below). 1.7 Other significant changes 13. The project was implemented largely as planned. The processing of the Additional Financing (AF) was appropriate to help scale up a well performing project and enhance its impact. Significant changes during the AF include: (i) introduction of performance incentives for students, teachers, ESPs, ERPs, Upazila Education Officers (UEOs) and Assistant UEOs related to ROSC students’ completion of Grade 5; (ii) placing a ceiling of 75 LCs per ESP and a two-year maximum on the (previously unlimited) contract period between an ESP and a CMC; and (iii) starting a practice for the best performing teachers (per students’ Grade 5 results) to open new LCs for 2011 cohorts, obviating the need for ESPs in these cases. A single funding approach (grant-plus-allowance) was adopted for the 30 Upazilas added under the AF. These changes were made in efforts to refine implementation mechanisms and increase development impact. In terms of project restructuring, the following four Level-2 restructurings were approved in 2010, 2011, and 2013:  Additional Financing (Financing Agreement IDA-47190, dated June 7, 2010). This did not add new policy initiatives, but it extended support to: (i) enable students in the original 60 Upazilas to complete the primary cycle as there had been a delay in establishing LCs during early stages of the project, and (ii) scale up operations to an additional 30 Upazilas.  Amendment of the DGA of the original IDA Grant H-1020-BD (Letter dated June 16, 2010) was approved to: (i) extend the Closing Date to June 30, 2013 to align with that of the AF, and (ii) include “costs of transportation and distribution of textbooks” under the disbursement 4 category “Operating Costs” to cover additional genuine, eligible expenditures supporting the Project Development Outcome and build flexibility into the definition of “Operating Costs.”  Re-allocation of the proceeds, and increases in IDA financing percentages in selected categories of the original IDA Grant H-1020-BD (Restructuring Paper dated February 22, 2011) were approved to: (i) meet disbursement needs which changed as a result of the AF scale-up and extension of the original Grant Closing Date; and (ii) cover the gap left by the closure of the co-financing Trust Fund (TF053577) on June 30, 2010 that was financed by the Swiss Agency for Development Cooperation (SDC). This restructuring also covered retroactive application of an increase in IDA financing percentages in disbursement categories (Grants, Education Allowances and Consulting Services) resulting from the decision against extending the SDC TF due to the change in SDC’s overall country strategy for Bangladesh, to focus on income generation.  Amendment of proceeds of IDA Grant H-1020-BD (April 4, 2013) to reflect re-allocation of the amounts of the Grant to each category. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 14. The project was prepared in approximately six months (November 2003 to May 2004) which was rapid in view of the novelty of its concept, diverse geographical scope, and large number of potential stakeholders dispersed at local levels. The Bank team had a good skills mix, with expertise in education, economics, and operations (project management, monitoring and evaluation, communications and social awareness, financial management, procurement and costing). The two co-task team leaders combined good understanding of the formal education sector of Bangladesh. Both were involved in the PEDPII, which was an asset in analyzing institutional strengths and weaknesses in the education sector relevant for ROSC, and evaluating the costs-benefits of its approach. 15. The following factors in the preparation process affected the implementation and outcomes of the project: Responsive PDO. This reflects an effective response to Bangladesh’s circumstances and priorities for EFA. Consultations with major NGOs 4 actively involved in non-formal education and with key bilateral and multi-lateral development partners (DPs)5 are evident in the Aides-memoire of the preparation missions and contributed to refinements of the project concept and operational design. However, evidence points to concerns stakeholders shared over ROSC being designed and beginning implementation at a time when PEDPII was just getting underway – and thus straining MoPME/DPE’s capacity to manage both programs. It would have been preferable for the Bank team to reach consensus among all development partners on the plan to launch ROSC as a parallel project to PEDPII that was just starting as the first Sector-wide Approach (SWAp) in primary education in Bangladesh. 4 Bangladesh Rural Advancement Committee (BRAC), Bangladesh Association for Community Education (BACE), Campaign for Popular Education (CAMPE), Center for Rehabilitation Education Earning Development (CREED), Dhaka Ahsania Mission (DAM), Friends in Village Development Program (FIVDP), Heed Bangladesh, Proshika, Save the Children (SCF/USA), Kastbir Youth Action Group (KYAG), and Dirha Suchana Project (FIVDB and SCF project funded by USAID). 5 Education Local Consultative Group (ELCG), Asian Development Bank (ADB), Swiss Development Cooperation (SDC), United Nations Children’s Fund (UNICEF), United States Aid Agency (USAID), 5 Straightforward and coherent project design. The clear rationale for the project to test the effectiveness of alternative approaches for expanding access to quality education for disadvantaged, out-of-school children is well reflected in the project design. It introduces an innovative, blended approach of demand and supply, 6 clearly stating that a number of initiatives were subject to testing and revision over the course of implementation. Benefits of key lessons incorporated in project design. ROSC could not build on specific strengths of other projects since the innovation to deploy grants through a direct partnership between GOB and local communities who would be responsible for contracting NGOs was the first of its kind in Bangladesh. A number of lessons from the NGO sector were nonetheless reflected in the project design, namely: (i) mechanisms for quality assurance (e.g. teacher training) in alternative approaches to primary education, and (ii) community involvement as critical to improve coverage and quality of services in remote areas. The project design was also guided by the effects of the GOB stipend program that was showing how monetary incentives were encouraging the poor, and especially girls, to enroll in school; and by research showing that when educational institutions were granted decision making autonomy, there were improvements in education quality. These lessons deepened understanding of operational technicalities, but in retrospect did not adequately prepare the project to meet substantial political economy challenges it encountered at community level. Generally appropriate risk assessment. The Project Appraisal considered whether risks had been appropriately assessed and addressed. The overall risk rating was judged as “moderate;” only one risk was rated “substantial,” i.e. that “there could be leakages in the disbursements of educational allowances and grants.” The ensuing implementation experience suggests that most risks were appropriately assessed, except for two: (i) the ability to “empower communities” within the political economy context, and (ii) the domineering behavior of local level NGOs vis-à-vis vulnerable populations. These two risks were rated “modest” during preparation but found to be substantial in the first few years of project implementation. Corrective measures to address these issues were taken at Mid-Term and at the AF implementation. 16. Quality at Entry. A total budget of US$295,000 was spent on project preparation. A Quality Enhancement Review (QER) was held in March 2004, shortly before Appraisal. The main recommendation was that, given the innovative nature of the project, its design should be flexible, and a robust monitoring system needed to be in place. These recommendations were incorporated into the project design. Likewise, recommendations from two workshops the Bank team organized with national and international NGOs in non-formal education fed into the final definition of criteria for selecting ROSC project areas, procedures to reach “untapped” communities, safeguards for funds transfers to families and LCs, the outline for teacher training and LC quality control, and tools for project monitoring. 17. The Government was proactive in obtaining all necessary approvals and developing a clear Annual Operation Plan (AoP) and Operations Manual by project approval. Most of the readiness steps were already completed by project negotiations and in addition to the AOP included: (i) Project Concept Paper approved by the GOB Executive Committee for National Economic 6 While demand was augmented through no-cost facilities and stipends for beneficiaries, supply was generated through establishing LCs within easy reach of the families, and capacity building for CMCs, teachers, local level education officials and NGOs. 6 Council (ECNEC); (ii) procurement plan for the first 18 months of project implementation and a sample contract to be signed by CMCs and ESPs; (iii) terms of reference for a community organization and management study to serve as the basis for preparing a standardized training and dissemination package for building local level capacity; (iv) terms of reference for internal audit and monitoring of the project; (v) draft of the participation agreement with Commercial Banks; (vi) draft of the cooperation agreement between DPE and CMCs; (vii) draft of the Memorandum of Understanding (MoU) between MoPME and the Local Government Engineering Department (LGED) for its M&E services; (viii) list of qualifying Upazilas to be covered by the Project; and (ix) job descriptions for key project positions. 2.2 Implementation 18. The disbursement rate is the broadest parameter characterizing implementation progress. For the first three years, disbursement was low and did not surpass 20 percent (see DataSheet). Disbursement increased significantly after the Mid-Term Review and from 2009 continued with strong momentum – at approximately twice the rate of the first three years – until project closing. Of the original Grant, nearly 90 percent was disbursed by October 2010. The project was never rated at risk status. 19. Factors which favorably contributed to the project implementation: The Upazilas were selected by Appraisal and listed in the PAD and Development Project Proforma (DPP). As mentioned earlier, these were selected based on transparent and objective criteria. This helped offset risk of political disturbances to the continuity in (geographical) coverage needed to test effectiveness of the two approaches in Component 1. Dedicated institutional arrangements for ROSC were instrumental for steady attention to implementing the project’s new concept for GOB (MoPME/DPE) partnership with local communities delivering primary education in the non-formal sector. By project close, the ROSCU was successfully coordinating the set of systems and procedures which were tested and refined all during implementation. Decentralization mechanisms, including the use of commercial banks, were clearly articulated in partnership agreements and systematically monitored. The ROSC Committee was crucial for the high level political support that backed the Project Director (PD) in making the extraordinary progress after 2009. Robust, ground-breaking M&E. Out-sourcing M&E to agencies other than DPE was a first step in GOB’s openness to external third-party assessments and enhanced the project’s professional credibility in making claims about its results and development impact. The M&E reached the remotest LCs and delivered clear returns in signaling aspects which needed adjustment over the course of implementation. Flexibility within defined operational modalities and proactive use of action plans were among the major strengths of implementation and the ability to resolve issues as they arose. These features were particularly instrumental in amplifying attention to education quality which was considerably reinforced, progressively, over the duration of the project. Continuous support to capacity development was inherent to the project design. In addition, the Bank team provided ongoing mentoring in human resource development of education officials in regard to planning, budgeting, administration and monitoring (to capitalize on feedback from M&E functions) to strengthen quality assurance and outcomes. 7 Effective partnership with the Swiss Development Cooperation was manifest in US$6 million co-financing and in the joint DP “forum” created for technical supervision and support along with the Bank. 20. Factors causing implementation difficulties were: Human resource constraints. ROSC was launched during the early stage of PEDPII and this had some impact on ROSC because the two operations competed for similar expertise (e.g. fiduciary) in a fairly limited domestic market. This was a factor in delays experienced recruiting ROSCU personnel, which in turn resulted in uneven attention to community mobilization and contributed to slow start-up of LCs. The slow start ultimately led to rescheduling the ROSC MTR from July 2007 to February 2008. Further, both PEDPII and ROSC had the same implementing agency (MoPME/DPE), and the authorities, especially at field level, were initially preoccupied with PEDPII and were less sure about their responsibilities for ROSC. Steep learning curve engaging communities. The very newness of the project’s concept was its greatest challenge as well as the reason for its ultimate acclaim. The original design relied on an untested package of financial incentives channeled directly to beneficiaries (including LC grants sent to CMC bank accounts), training to build up community members to take on powers vested in CMCs, and regular third party monitoring as a constant check on performance. Over time, implementation was adjusted to achieve a satisfactory balance between incentives and sanctions to overcome a greater than anticipated dominance of ESPs vis-à-vis CMCs (i.e. “elite capture”), and tendencies for some NGOs to seek the lucrative financing grants without providing adequate service. Changes in national government which occurred three times over the project lifetime slowed implementation in the field as a result of changes in community leaders, ESPs and local officials who had been closely aligned with the previous government(s). Significantly, the original concept of the project was not disturbed. 21. The greatest challenge at the start of implementation was to secure the people’s participation in community-based clientele mapping, resource mobilization, program implementation and monitoring. The arrangements (PAD, Operation Manual, templates for MoUs and contracts) were methodically designed during project preparation, but they were untested. Creating CMCs in remote, marginalized areas had never been done before. Nor had working with local NGOs by putting the burden of their selection on the communities. The idea that LCs would operate with oversight support from their CMCs, in which a sizable portion of the membership was illiterate, was far more challenging than even the efforts of PEDPII that was breaking new ground trying to launch a program of small grants for School Management Committees, transferred via Upazila Education Offices (UEOs), for small-scale “school improvement plans” on the order of procuring sports and recreational equipment. The ROSC was far more ambitious assigning CMCs the responsibility of selecting teachers from among candidates in the local community and entering into contracts with NGOs (ESPs and ERPs) for services to help manage LCs and train the teachers. During ROSC preparations, it was not feasible to inventory all such NGOs’ capacities nor assess their provisions for receiving funds (and accounting for how the money would be spent) since the NGOs could not be identified in advance; indeed, many were created expressly upon hearing that ROSC was launched. As for the DPE, its experience lay in managing schools in the formal sector. For ROSC, DPE’s Upazila level administration was expected to provide support for management and capacity building to LCs and visit them regularly, but these functions were not traditionally in the UEO mandate (they are not otherwise involved in literacy 8 programs managed through MoPME’s Bureau of Non-formal Education which uses a different network of local literacy centers). 22. Ultimately, the ROSC took hold and entered a strongly productive phase especially in its last three years. Evidence points to this achievement having resulted mostly from: (i) stronger program coordination from a fully staffed ROSCU; (ii) more frequent communication initiated by the Project Director vis-a-vis the ROSC Committee to ensure political backing for remedial actions (e.g. initiatives from the then MoPME Secretary to fully engage local administration in ROSC implementation, and the decision to restrict the duration of ESP services to LCs to a maximum of two years); (iii) stepped-up publicity on the project, aided by the various evaluations which were identifying ROSC results on the ground; and (iv) capitalizing on the opportunity to use the Grade 5 exam (primary school leaving exam that MoPME introduced on a national level in 2009) as a new yardstick for gauging LC students’ performance. Linking ROSC to the Grade 5 exam was a significant step in mainstreaming the project into the formal education system and providing graduates clear avenues for moving into secondary education. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 23. Third-party monitoring and evaluation which ROSC brought directly into the GOB system was novel and impressive because of its scale and development impact. 24. M&E Design. The PAD (Annex 3) defines a project “Results Framework and Monitoring” comprising 8 PDO level indicators and 17 intermediate outcome indicators related to the four components. The rationale for selecting the eight PDO indicators was clear: (i) they had contribution linkages to EFA outcomes in terms of universalizing primary education; and (ii) they were a sub-set with most need of improvement to address entrenched challenges of getting the last 15 percent of primary school-aged children into school, while also paying attention to the quality of education. Given that the intermediate outcome indicators enabled progress tracking in various operational dimensions of project implementation, the preparation team’s choice of indicators was justified. Most are quantifiable, in principle, facilitating data collection. Some deficiencies worthy of notice include: (a) lack of precision about expected results for 5 out of 17 intermediate outcome indicators (e.g. “innovative” marketing strategies, “adequate” budget statement); and (b) failure to disaggregate project performance by income and ethnic population strata. 25. The project relied on two main data sources for tracking progress: (i) routine LC-reported data synthesized by LGED (external agency), and (ii) special purpose studies and surveys (large household-based baseline study, tracer study, ESP performance studies, Public Expenditures Tracking Survey, etc.) contracted out for more rigorous analysis than the LGED could perform. The MIS Cell that was created for the ROSC was the bedrock of collecting, hosting and processing critical project data. In connection with the LC-reported data, the MIS Cell introduced a sample-based monitoring mechanism in 2005/06 to validate information provided by LCs, Master Trainers (MTs), and AUEOs. It was also intended that UEOs/AUEOs would perform field-based monitoring and that their reports would be supervised by the ROSCU Assistant Director Monitoring. 26. M&E Implementation. The self-reported data, collected twice a year, provided a steady stream of information on LC operation, teachers, students, CMC activities and training which was regularly summarized by the LGED ROSC MIS Cell in Semi-annual and Annual Statistical Reports for each year of project operations. Throughout, the MIS Cell maintained a complete Database Management system. Local LGED staff (in Upazilas) physically visited schools to monitor 20 percent of the LCs in a year, using a monitoring checklist. This innovative mechanism 9 that allowed third-party validation of the self-reported data was instrumental in reducing data discrepancies over time. The routine functions of ROSCU internal monitoring, supervised by the Assistant Director Monitoring, relied on consultants and were slow to start up, mainly due to staffing deficiencies; consultants were appointed just before the MTR.7 The internal monitoring took a stronger, positive trend in the latter years of project implementation. Upgrades were made, including introduction of a format for ROSC officials’ field visits, and efforts were made to enter and maintain data on a similar data platform as LGED to facilitate triangulation/comparison across various streams of data. 27. Impact Evaluation: With technical assistance from the World Bank, a rigorous impact evaluation (IE) – sub-component of M&E component – was carried out (Sarr et. al., 2010) to assess the impact of the ROSC project. The IE design involved baseline (pre-project) and follow- up (post-project) surveys of children, households and schools from treatment and non-treatment areas, and allowed the evaluation to estimate the causal impact of ROSC intervention on schooling outcomes. The ROSC LCs were found to increase enrolment probability between 9 and 18 percent and perform as well as non-ROSC schools in raising test scores. In particular, better performing students attending grant-plus-allowance LCs improved their test scores by around 0.2- 0.4 standard deviations compared to other schools. These results pointed to the effectiveness of the ROSC schooling model as a comprehensive “package” whose salient characteristics are incentives to increase access, quality assurance (especially teacher training), and a spirit of constant self-learning and self-improvement in managing the operations. 28. M&E Utilization. Good use was made of the M&E at three levels: in day-to-day project operations, in cyclical programmatic assessments, and in the broader policy arena. In regard to the first two levels, the well-established review system supported by regular monitoring reports was the main basis for the ROSCU to decide on the number of annual student allowances (based on enrolment reports from the LCs). Evidence from regular monitoring was also used in management decisions on sanctions against improperly performing Upazilas, ESPs and LCs; and decisions for improvements to the project operational design during the course of implementation. Among these were actions which strengthened the M&E system itself. These included a full validation scheme that was introduced to check whether all LCs were meeting the basic educational compliance criteria to receive grants and student allowances, and the creation of profiles for all LCs and enrolled students which allowed the monitors (including Master Trainers from ERPs, DPE, ROSCU staff, consultants and IDA task team) easily to check on the status of LCs and their students, especially as a precaution against misrepresentation of the actual number of students enrolled. Together, these two improvements ultimately enabled 100 percent compliance monitoring of LCs every year, compared to the 20 percent monitoring of the self- reported data. For cyclical program assessments, results of the M&E were utilized for the Implementation Supervision Reports (ISRs) to IDA management and the DPE Director General. 29. The in-depth studies and special purpose surveys also contributed strongly to programmatic assessments. To illustrate, the evaluation of ESP performance over 2005-2007 (AcNielson, 2007) joined by investigations at the project Mid-Term Review generated an empirical base on which the GOB and Bank decided to take action against non-performing ESPs and further influenced the decision at the time of Additional Financing to limit the range and duration of ESPs’ contracts with CMCs. Feedback on teachers’ and students’ performance through various channels of field- 7 Only 5 of the originally intended 14 formal Internal Monitoring Reports were available for review by the ICR team. 10 based monitoring yielded information that was steadily used by the GOB and Bank teams to readjust and ramp up teacher training and development and initiate performance incentives to improve quality. Findings of the impact evaluation informed decisions by the GOB and the Bank to scale up the project at Additional Financing and to select the Grants-plus-Allowance approach as the sole method for use in the additional 30 Upazilas. The M&E reports, impact evaluation and Public Expenditures Tracking Survey contributed to the ROSCU’s ability to publicize the project’s success with credibility. The consequence at a broader policy level was government’s decision to extend use of the ROSC model for second chance education through ROSC-II, and even more extensively in forthcoming operations to cover rural areas across the entire country. 30. There were some weaknesses in the M&E system. These included considerable delay in procuring the firms for two key studies as well as limitations of the MIS Cell in being able to analyze the vast quantity of descriptive data collected. Regarding the former, the baseline study was delayed nearly two years, which complicated the impact evaluation because the baseline field surveys were carried out after the project had already started implementation. This study still has value, owing to its rigorous performance assessment of the project at a relatively early stage of implementation. In contrast, the analytical study on Non-Formal Education (NFE) which was also delayed (not carried out until 2010 because of a protracted lack of clarity over its objectives followed by delays in contracting the consultants) could have helped bring NFE much earlier into an overall programmatic framework for primary education if this work had been done on time. With regard to the MIS Cell, the ICR team noted that, by nature of its contract, the LGED could – and did – regularly recommend to ROSCU on actions to take based on assessment of field data. There was occasional tension vis-à-vis the ROSCU when LGED felt its recommendations were not adequately considered. As the M&E system keeps improving its capability for the data collection to capture ground realities (e.g. strengthening school visits and monitoring) under ROSC-II, so, too, management of the chain of various procedures for data interpretation, arbitration (if discrepancies arise across different data streams), assessment of appropriate follow- up, and ROSCU/DPE decision making on recommended actions needs to bring all these parts more effectively together. This is essential to cross-check feedback from multiple sources, avoid inadvertently overlooking important evidence from the field, and generally to draw the best collective use out of these positive features of the M&E program. 2.4 Safeguard and Fiduciary Compliance Safeguard Compliance 31. The project was Environmental Category “B” and triggered two safeguard policies: environmental assessment and Indigenous Peoples. The project preparation used the environmental guidelines just developed and disclosed for the PEDPII; these were based on a detailed Social and Environmental Assessment. The Borrower expanded the PEDPII guidelines for ROSC after carrying out a rapid assessment on non-formal education institutions. The ROSC- specific guidelines were disclosed publicly. 32. The LCs and SKT schools were located in existing premises so the project did not involve construction. The main concern was to ensure that communities provided safe drinking water and appropriate sanitation facilities for the Learning Centers. These aspects were clearly included on the monitoring checklists used systematically throughout project implementation and reported mainly through the Semi-annual and Annual Reports issued by LGED (ROSC MIS Cell). The ICR team found the MIS Cell database contained a record of compliance, including evidence of steps taken to improve facilities for safe drinking water and toilets in cases where they had initially been observed as inadequate. For these reasons, the ICR team rates compliance on environmental assessment “Satisfactory,” which is consistent with the ratings in the ISRs. 11 33. For the safeguard on Indigenous Peoples (IP), the project was to implement the Tribal Action Plan of PEDPII which covered all tribal areas of Bangladesh and was disclosed publically. The ROSC Integrated Safeguards Data Sheet (ISDS) approved at Appraisal noted that the tribal population comprised approximately one percent of the country’s total population but gave no indication of specific actions the project would take and report on during implementation. The ICR team found implementation support mission Aides-memoire occasionally referred to due consideration given to the potential enrollment of ethnic minorities. While project files document numerous LCs were implemented by tribal communities (for example, 23 LCs in Sylhet managed by the Monipuri community), the ROSC MIS Cell database has no record of ROSC students by ethnic background or explicit mention of follow-up on the IP safeguard. Nor is this specifically referred to in missions’ Aides-memoire. For these reasons, and considering the PDO focus on “disadvantaged” children, the ICR team rates compliance on IP “Moderately Unsatisfactory.” Financial Management 34. The overall performance of financial management (FM) is rated Moderately Satisfactory. 35. The project was implemented in a country fiduciary environment of high risk which did not improve over the project lifetime. To mitigate risk to the project, a number of initiatives were taken: (i) dedicated FM staff at the ROSCU (supplementing DPE’s Finance Unit); (ii) FM responsibilities articulated for central, intermediate, and the most local levels of project activity and systematically refined on the basis of implementation experience; (iii) contractual agreements between the nationalized commercial bank (Sonali Bank) and CMCs, and grant and education allowance criteria specifying eligibility and forming the basis for accounting; (iv) annually renewable Cooperation Agreements to be signed by DPE with each CMC indicating fiduciary and statutory obligations of both parties (as a legal covenant of the DGA); (v) IDA review of project quarterly Financial Management Reports (FMRs) and periodic desk review and field visits to monitor Statement of Expenditure (SOE) submission by LCs; (vi) annual auditing of project financial statements by the Comptroller and Auditor General of Bangladesh; and (vii) independent audits of educational allowances and grants on an annual basis. 36. The main challenges which became evident during implementation were: (i) vacancies in the core financial management and procurement positions of the ROSCU up until the MTR; (ii) CMCs’ weak financial management (FM) capacity (notably, in book-keeping); (iii) low quality financial management reports (FMRs) in early years of the project, displaying inconsistencies with financial statements from the LCs and Sonali Bank; (iv) Sonali Bank’s lapses in production of fund utilization reports and reconciliation statements of actual amounts disbursed to students and LCs; (v) over-stating of financial statements and in some cases payment for services which were not rendered; and (vi) delays in settling audit observations. 37. The impact of these issues was successfully mitigated through active follow-up by joint Bank-GOB reviews and implementation of remedial action plans. The MTR recommended preparation and implementation of an “accountability plan FM section” for the ROSC Committee, MoPME, DPE, ROSCU, UEO, partners, ERPs, ESPs, CMCs, teachers and parents to: (i) ensure the veracity of project expenditures at all levels by strengthening accountability and adherence to agreed procedures; (ii) promote transparency through public disclosure of LC activities (through social audits involving Parents’ Assemblies on utilization of grants and education allowances, display of funds receipts and expenditures in each LC, and use of checkbooks for LC transactions); and (iii) embed a fully functional funds reconciliation system in the regular monitoring system. The MTR further recommended a Public Expenditures Tracking Study 12 (PETS) to be carried out on a sample basis, and recruitment of a full-time FM consultant to support the ROSCU. 38. Evidence of success in building up a satisfactory level of financial management in the post- MTR period is chiefly the following: (i) FM positions at the ROSCU fully staffed; (ii) substantial improvements in on-time disbursement of education allowances; (iii) improvements (after 2010) in Sonali Bank fund utilization reports and reconciliation statements; (iv) results of the PETS which showed service delivery was effective in that over 95 percent of intended student beneficiaries reported to have received educational allowances, and close to 95 percent of them received both installments; and (v) reviews showing LCs maintained reasonable fund documentation. Additional measures adopted were random performance audits by the ROSCU as well as its examination of the respective separate reports of ESPs, UEOs, and Sonali Bank in parallel to reconcile LC/SKT financial transactions. Independent audits were also conducted on a contract basis to review overall LC/SKT performance. 39. Counterpart funding. There were no major issues with the Government’s project financing (US$ 6.8 million, including AF) which was nearly eight percent of the total and mainly covered salaries of the project staff and consultants as well as administrative costs and logistics. Total counterpart funds were over 85 percent disbursed.8 40. SDC co-financing grant of US$6 million. As agreed by the GOB, SDC and the Bank, the Swiss co-financing was deployed to co-finance all four components in the IDA DGA (Grant No. H-1020-BD). The Bank managed the Swiss grant through the Cofinancing Technical Assistance and Consultant Trust Fund Framework Agreement between The Government of the Swiss Confederation, IBRD and IDA. An initial amount of US$ 1.48 million was provided to the Trust Fund (TF053577) for a first phase of 18 months ending February 28, 2006. Following a satisfactory performance review (which included an external performance Audit of the initial 18 months of the project), the Administration and Management Agreement for the Trust Fund was amended to provide US$4.52 million for a second phase, effective July 1, 2006 until June 30, 2010. Beyond this date, there was no additional financing due to a shift in SDC’s strategy, to focus on income generation activities. Disbursements 41. Disbursement estimates were formally revised on August 16, 2010 because of the project restructuring (Additional Financing and extension of the closing date). Of the original Grant of SDR 35.2 million (equivalent to US$ 51 million), the undisbursed balance after the final disbursement on October 31, 2013 was SDR 0.12 million (equivalent to US$ 0.18 million). Of the Additional Financing Credit of SDR 23.1 million (equivalent to US$ 35 million), the undisbursed balance after the final disbursement on October 31, 2013 was SDR 7.05 million (equivalent to US$ 10.79 million). The total undisbursed amount in the ROSC project was SDR 7.17 million (equivalent to US$ 10.97 million). The project was 88 percent disbursed. 42. At the time of writing the ICR, the undisbursed amounts include balances in the Designated Accounts for the Original Grant (approximately SDR 0.118 million) and the Additional Financing Credit (SDR 4.6 million) which are in process of being refunded to IDA. Since IDA balances are in BDT, they are converted to SDR and US$ amounts based on current exchange rates. The main 8 In order to deal with a small discrepancy in the amounts between the approved Development Project Proforma (DPP) and the DGA, revisions to the DPP were made in January 2006. 13 reason for the less than full disbursement (12 percent undisbursed) in ROSC-1 is due to the fact that ROSC-II and ROSC-I overlapped for six months. Because of the GOB guidelines on funds release on projects with the same names (ROSC-I and ROSC-II), the project chose to use ROSC- II funds because the latter were made effective six months before ROSC-I was closed. Procurement 43. The overall procurement performance is rated Moderately Satisfactory. 44. Under the initial IDA grant (H102-BD) and TF053577, there was no procurement subject to international competitive bidding. The major portion (91 percent) of Grant proceeds comprised allowances for out-of-school children and grants transferred to CMCs, who had the primary responsibility for procuring materials and hiring teachers. Remaining procurement consisted of small value goods and services. This was managed by the ROSCU. There was no additional procurement planned under the Additional Financing as the procurement plan under the original project was adequately updated to meet the needs during the period of the AF and financed fully from the original project funds. 45. The project design and readiness steps sought to ensure adequate procurement capability by effectiveness. During implementation, procurement moved slowly up to the time of the MTR, and the quality was variable. This stemmed mostly from capacity constraints at the ROSCU; in particular, protracted vacancies in the procurement function. Deficiencies were partly compensated by the proactive recruitment of procurement consultants, but the joint reviews noted that capacity often diminished following a consultant’s departure. Several large services packages (including for Communication and Social Awareness, Network Coordinating Body for ERPs to assist LC teacher training, and Analytical work on Non-formal Education) were delayed inordinately. Up to Mid-Term, procurement post-reviews revealed some inconsistencies in the procurement process (including repackaging goods without IDA no-objection, deficient bidding practices, and limited screening of quotations). However, misprocurement was declared only once (on November 8, 2008). This was relatively minor, involving five packages amounting to US$4,662; the amount was canceled from the IDA Grant. 46. Procurement capacity picked up significantly over the second half of implementation, with weaknesses actively addressed through stabilizing staffing at the ROSCU and intensifying training (especially related to the new public procurement regulations), performance monitoring, and procurement auditing. Post reviews of FY10 and FY11 indicated satisfactory procurement compliance. 2.5 Post-completion Operation/Next Phase 47. The follow-on Reaching Out of School Children Project II (ROSC-II), which was approved by the Board in October 2012, was designed to improve equitable access, retention and completion for out-of-school children in Bangladesh. It built upon lessons derived from the implementation of ROSC-I. The ROSC-II is a US$ 130 million investment credit, and the design aims to reach a larger target group than ROSC-I (including – for the first time – urban slum areas) as well as incorporate design refinements into the program; in particular, a significantly enhanced emphasis on education quality with interventions focused on improving teacher effectiveness at the level of the classroom, streamlined provisioning of grants and allowances, an elaborate system of monitoring, and a more defined and time-bound role for supporting NGOs. Over time, the mechanism is expected to be streamlined into the overall Primary Education Development Program of the Government. 14 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 48. The overall relevance is Substantial. 49. Alignment with development strategy, objectives and design (Rating: Satisfactory). The objectives and design of ROSC were and are still fully relevant to the country strategy. The core features of ROSC to achieve its objectives were to expand second chance access to primary education through testing an innovative community-led approach for the most disadvantaged segment of the society, improve the quality of teaching and learning, and strengthen the institutional foundation of the delivery system for formal primary education for the children left out of the system. These actions directly supported GOB’s efforts toward EFA goals as outlined in the Interim Poverty Reduction Support Program (I-PRSP, 2003) and the National Plan for Education for All (EFA, 2003) which stated that access to education for the poor needed to be increased to ensure an efficient and sustainable approach to equitable, quality education. The project was also consistent with the Bank’s CAS dated February 2001, one of whose pillars was consolidating gains in human development, and remains consistent with the current CAS (2010) focus on more effective service delivery at the local level to enhance access to the marginalized, and better education quality and learning outcomes. Moreover, ROSC objectives are in line with the Third Primary Education Development Program (PEDP3, 2011-2016) whose key policy directions and expected results emphasize enhanced access and quality education for all. Building on core actions and capabilities of ROSC-I is an ongoing agenda in ROSC-II, with further expansion in terms of geographic coverage. In a recent policy decision, the Government indicates it will take this even further, to use the ROSC model (in rural areas) to provide second chance education to the disadvantaged children in the entire country. 50. Implementation (Rating: Satisfactory). While the project took some time to gain momentum, ROSC clearly performed well expanding access and made significant advances on education quality and developing appropriate institutions. The innovative approach of community-led learning centers, no-cost schooling incentives for the poor and former drop-outs; and incentives for performance of teachers, LCs, and ESPs worked effectively. Quality of education is a generic concern in Bangladesh. ROSC students, who were mostly from the most disadvantaged segments of society, performed reasonably well in the nationally administered Grade 5 (primary completion) examinations. ROSC arranged formal trainings for teachers, teachers’ group meetings for sharing experiences and innovative ideas, refresher trainings for quality enhancement of teaching and learning. Finally, ROSC built up a credible system of M&E and third party verification which is now being used across the sector. Contributing to project success was the proactivity shown in adapting design to accumulated experience, in line with the overall aim to test accountable, transparent, alternative procedures for delivering education services to the poorest and most disadvantaged children. 3.2 Achievement of Project Development Objectives 51. Based on the achievement of all core PDO indicators and significant surpassing of targets on five of them, the project fully achieved its development objectives. There has been a significant increase in access to primary education for disadvantaged children and in the quality of education in non-formal learning centers, both of which are advances toward the country’s EFA goals. These are satisfactory results. Detailed justification for this rating is given below. 52. The PDO formulation in the PAD consists of six elements, for which the first four have corresponding KPIs. The ICR team assessed achievements against all six of these elements by bringing in intermediate outcome indicators (PAD, Annex 3) and specific outputs from the 15 components (Annex 2). The ICR team’s assessment is based on project reported data and external sources. Results are summarized below, grouped into six parts corresponding to the respective PDO elements. Each part – “PDO1,” “PDO2,” PDO3,” PDO4,” PDO 5, and “PDO6” – is rated. 53. PDO 1: Reduce the number of out-of-school children. Achievement is assessed in the areas of: i) Total number of out-of-school children enrolled in ROSC. The 0.84 million children the project reports to have reached surpassed the (AF) EOP target of 0.75 million. The number of children who benefited from ROSC roughly corresponds to 40 percent of the 2.2 million decrease (from 7.7 million to about 5.5 million) in number of out-of-school 6-14 year olds reported by other sources for the 2001-2010 period (Education Sector Review 2013). Even though the reduction in number of out-of-school children prior to 2004 cannot be attributed solely to ROSC, this comparison helps put the magnitude of ROSC’s contribution in context. ii) Evidence of grants and grants-plus-allowances on increasing access. Independent in-depth studies carried out under ROSC show the incentive of grant-plus-allowance was particularly successful attracting children to LCs: the prevalence of primary school age out-of-school children was found to be considerably lower in grant-plus-allowance areas compared with the grant and control (non-ROSC Upazilas) areas, especially among the poorest segments of the population (Uniconsult, 2011). A study carried out in 2007 shows a statistically significant average net increase in primary school enrollment of 9 percentage points for children aged 6- 14 in grant-plus-allowance areas. This is an important indication of the reduction in number of never-enrolled. However, there was some qualitative evidence based on informant interviews that suggested some of the students had left other schools to join the LCs. While the empirical evidence on this issue was limited, it is possible that this would have compromised the project’s achievement of reducing the number of children who had never previously attended school. A rigorous impact evaluation carried out in 2010 (using data from 2006 baseline and 2009 follow-up surveys) shows that: (i) ROSC intervention is associated with as high as 20 percentage points increase in enrolment of primary-aged children, (ii) ROSC is reaching the poorest segment of the communities, and benefit incidence would be even more pro-poor if the program were to be scaled up; and iii) these results are notable given much lower per student unit cost in ROSC schools relative to formal primary schools. 54. Table 1 provides a summary of PDO 1. The overall rating is Satisfactory. Table 1: Ratings for PDO 1 PDO Indicators Highly Satisfactory Satisfactory 1.Number of out-of-school children enrolled in ROSC X 2. Evidence of grants/ grants-plus-allowances on increasing access X Total/average X 55. PDO 2: Improved access in primary education. Achievement is assessed in the areas of: i) Innovative grassroots marketing strategies and community outreach efforts. “Innovative” was not defined in the project Results Framework, making it difficult to assess outcomes against that part of the objective. However, in terms of numbers reached, it is clear that efforts were initially focused at Division level and only shifted to community level from 2009 (see Annex 2, Table 4). The intermediate outcome indicator for community outreach efforts originally specified a minimum of two workshops, two dissemination campaigns and two stakeholder training sessions annually. Data available to the ICR team confirms that the number of “social awareness” workshops over the 2004-2006 period exceeded these targets (see Annex 2, Table 16 1). The total number of participants in basic and refresher training (nearly 390,000) also indicates these targets were reached (see Annex 2, Table 5). A noteworthy “innovation” in the community outreach efforts was the production of videos prepared for courtyard training to assist government officials and community members on the ground, as they were not able to carry through the communications activities of a previously contracted firm (BCCP) at the same level of professionalism. ii) Number of LCs. The total number of new LCs and SKT schools (23,768) created over the project lifetime is shown in Table 2 of Annex 2. To illustrate the exemplary magnitude of this effort, these numbers can be compared with the numbers of Government Primary Schools (GPS) and Registered non-government primary schools (RNGPS) in Bangladesh which are 37,672 and 20,061, respectively (BANBEIS 2011). iii) Student enrollment in LCs and number of student years supported for new students. The total 0.84 million students who attended the LCs and SKT schools (778,147 students, if replaced students are not included) is reported in PDO1, above. The number of student years the project had supported by December 2012 is 2.94 million (Annex 2, Table 3), which met the (AF) EOP target of 2.9 million. iv) Student average annual attendance rate exceeded the EOP targets. Early on, the Baseline Survey (2006) found remarkably high attendance rates (85-91 percent) overall, and that rates in grant-plus-allowance LCs were higher (91 percent) than for students in other non-ROSC Upazilas (86 percent). Attendance rates remained high during implementation and were close to 90 percent by project closing, exceeding the EOP target of not less than 75 percent. This may have been due to the demand-side incentives which encouraged children to go to school, close monitoring and supervision by communities, as well as an improving quality. v) Bi-annual disbursement of allowances and grants. In the first two years, there were significant delays in disbursing funds which may have weakened incentives (allowances) for children to go to LCs. The slow disbursement of grants to LCs was a factor in their slow start-up and contributed to the decision to reschedule the MTR. As late as 2008, the record of planned vs. actual date of disbursement by Upazila by semester was not maintained, so it is not possible to assess the full extent of the delay. However, in the last few years of implementation, disbursement proceeded regularly and the effects of the allowances and grants appear to have been instrumental for access and quality outcomes, as shown by independent evaluations. 56. Table 2 provides a summary of PDO 2. The overall rating is Satisfactory. Table 2: Ratings for PDO 2 Highly Moderately PDO Indicators Satisfactory Satisfactory Satisfactory 1.Innovative grassroots marketing strategies and community outreach X 2. Number of LCs X 3. Student enrollment in LCs and number student years supported X 4. Student average annual attendance rate X 5. Bi-annual disbursement of allowances and grants X Total/average X 57. PDO 3: Improved quality in primary education. Achievement is assessed in the areas of: i) Percentage of students reaching expected competency level based on teachers’ assessment in Bangla and Mathematics. This is reported as 80 percent, exceeding the EOP target of not less than 65 percent (see DataSheet). The LGED database shows the percentage of students with 17 qualifying marks was well above 90 percent in most cases (Annex 2, Table 7).9 In 2008, the LC self-reported data started to be supplemented with that from LGED field monitoring. The ICR team’s review of the data revealed largely consistent results from these two sources, with the exception of 2009 when the field monitoring recorded considerably lower competencies, due to a higher competency threshold used by the monitors for the first time. In subsequent years, the same threshold was used in both the self-reported data and field monitoring data, and data between the two sources showed similar competencies. ii) Student academic achievement assessed by independent evaluations and performance on primary school completion (Grade 5) exam. When MoPME introduced the Grade 5 terminal exam at national level in 2009, the project made a concerted effort for ROSC students to sit for it. Their pass rates in 2009 and 2010 were 43 percent and 49 percent, respectively. By 2011, the pass rate was a remarkable 73 percent. While these rates are still lower than the national average and the government schools (which have been around 85-90 percent), it should be noted that ROSC students are first generation learners in their households, come from the most marginalized communities, and attend shorter LC class times as they are designed flexibly to suit for working children. Since the project introduced bonuses for teachers based on student performance as early as 2007, it is highly likely the strong improvement in Grade 5 pass rates by 2011 reflects the additional measures the project took to improve the quality of teaching and learning, e.g., scaled up teacher training (see vii, below), additional classes for Grade 5 students, and preparatory exams before the terminal exams. It is also noteworthy that findings from the independent impact evaluation (Sarr et. Al., 2010) suggest that performance of ROSC students in terms of gains in test scores between the two rounds of the evaluation’s surveys are similar to non-ROSC students, after controlling for socio-economic characteristics, and impacts on learning levels are stronger for girls in ROSC intervention areas. iii) Average teacher attendance rate. At 90 percent, this met the EOP target (see DataSheet), and it is high compared with average attendance rates in the formal primary sector (around 80 percent). Significantly, independent evaluation dating back to 2008 (follow-up to the Baseline Study) found that among teachers present in school during the survey visits, over 80 percent were actually teaching. iv) Number of textbooks ROSC students have in each grade compared with corresponding grades in the formal primary education system. The project reports that students in each grade had on average the same number of textbooks as those in corresponding grades of the formal primary education system, thus meeting the EOP target (see DataSheet). v) Basic facilities for conducive learning environment. The Results Framework (PAD Annex 3) did not itemize the specific facilities expected, but the detailed project description (PAD Annex 4) identified “books with supplementary materials.” These items were subsequently incorporated into the Monitoring Checklist and systematically tracked. The basic facilities markedly improved over the project lifetime. By 2010, across all LCs in the project, almost all were observed to have received textbooks on time, 76 percent had supplementary reading materials, and over 95 percent had blackboards and chalk. In 2009, the seating arrangement was mats on floor in only 4 percent of schools; by 2012 this was 96 percent. 9 The qualifying threshold was defined as the minimum level of marks for slow learners and this, along with the average attendance (minimum 75 percent) of all learners in general, was the yardstick for “re-qualification” namely for continuing students to receive education allowances in the next academic year. The threshold evolved in the project’s lifetime: It was “above 40 marks” for 2005-2006 and 2010 and “50% and above marks” for 2008-2009. From 2011 this became more complex, as follows: at least 33 percent marks for promotion to the next grade, and a more stringent threshold for “requalifying” marks that increases by 10 percent for each year with the age of the respective cohort. 18 vi) Student-teacher ratio. The intermediate outcome indicator specified a student-teacher ratio of 25-35:1 which is consistent with the criteria for creating LCs and continuing to receive grants in the subsequent year (PAD, Annex 4). Third party evaluations reported the average to be 27. vii) Number of teachers trained in instructional methods and classroom management. Statistics LGED provided on the project’s overall achievement in training show these efforts progressively became more robust over the project lifetime (Annex 2, Table 6). One of the indicators of success is seen in LGED reports documenting an increase in the percentage of teachers observed to have teaching plans: 76 percent in 2012 compared to 58 percent in 2009. viii) Teacher-based assessments available monthly. During implementation, the teachers’ assessments were provided to the LGED regularly on a semi-annual basis and are documented in the LGED Semi-annual and Annual Monitoring Reports. While provided with regularity, it is not clear whether these assessments were available monthly. 58. Table 3 provides a summary of PDO 3. The overall rating is Satisfactory. Table 3: Ratings for PDO 3 Highly PDO Indicators Satisfactory Satisfactory 1.Percentage of students reaching expected competency level (teachers’ assessment) X 2. Student academic achievement assessed by Grade 5 exam performance X 3. Average teacher attendance rate X 4. Number of textbooks for students in each grade X 5. Basic facilities for conducive learning environment X 6. Student-teacher ratio X 7. Number of teachers received training in instructional methods, classroom mgmt. X 8. Teacher-based assessments of student achievement available monthly X Total/average X 59. PDO 4: Improved efficiency in primary education. This section focuses on the internal efficiency of the ROSC and comparable indicators for formal primary education; the project’s technical efficiency (keeping the cost per unit of the outcome as low as possible), and the allocative efficiency (spending on the “right things” most likely to achieve the desired outcomes) particularly in regard to student learning achievement. Efficiency is further addressed in Section 3.3 (Economic Analysis) of the ICR. 60. PDO4 assesses ROSC achievements in the following areas: i) Student grade completion and transition rates. The project reported a 90 percent student grade completion rate, surpassing the (AF) EOP target of 85 percent (see DataSheet). Based on project data, the ICR team calculated the mean survival rate to Grade 5 was 70 percent for the initial four cohorts (2005-2008), with somewhat lower rates (60-65 percent) for the latter two ROSC cohorts as a result of more stringent re-qualifying criteria as mentioned above. By EOP, 70 percent of ROSC students were also making the transition from Grade 5 to Grade 6 (see DataSheet). ii) Benefit incidence analysis of ROSC relative to government schools reveals that ROSC spent less per child across all income groups (Education Sector Review 2013) compared with government spending per child, by income group, in formal primary education. Moreover, in government spending on formal primary education, the share of the non-poor is higher and that of the poor is lower than the corresponding spending in ROSC. For spending on children 19 from the poorest households, ROSC spends less than half the amount spent in government schools: in 2005, the government spent about BDT 2,991 annually for each enrolled child from the poorest households; for the same income group, ROSC spent BDT 1,307.10 iii) Level of transparency and accountability in service delivery through ROSC. The project’s success in progressively strengthening internal control systems is highly commendable and has potential for broader use within government systems. As late as for the AY2006 cohort, the Award Confirmation Form (ACF) for student allowances still had no signature column to confirm receipt of funds, so there was no actual proof of disbursement. As noted elsewhere in the ICR, the MTR recommended the development and implementation of an “accountability plan for financial management.” Among other actions, this recommended that the grants be posted on the wall of each LC to show an exact breakdown of the grant and how it was used. These measures were joined by additional safeguards including the introduction of a new “student unique ID,” creation of a new ESP supervisor position (in place for the Additional Financing), and improvements in Sonali Bank fund utilization reports and reconciliation statements. The Public Expenditures Tracking Survey captured some of the early benefits of these enhanced internal controls, as reported earlier in the ICR iv) Role of accountability in education outcomes. Independent analysis (Education Sector Review 2013) finds accountability in reducing inefficiency strongly correlated with generating a quality education (measured in student scores on the National Student Assessment 2011). Results also suggest that displaying information at LCs on the grants they received was strongly associated with improved (technical) efficiency, and schools in which students’ guardians elected the CMC chairperson tended to be more efficient than schools in which the local elite initiated the CMC’s formation. These findings similarly appear in a third-party review conducted under ROSC (Uniconsult 2011) as well as performance surveys carried out by SDC. The latter noted students’ learning outcomes were best where the CMC was strong. The strong accountability measures implemented, especially in the latter half of the project and the improving learning outcomes over time (Grade 5 examination results) suggest that improved accountability positively contributed to better learning outcomes. 61. Table 4 provides a summary of PDO 4. The overall rating is Satisfactory. Table 4: Ratings for PDO 4 Highly Moderately PDO Indicators Satisfactory Satisfactory Satisfactory 1.Student grade completion and transition rate to Grade 6 X 2. Benefit incidence of ROSC relative to government schools X 3. Transparency/accountability in ROSC service delivery X 4. Role of accountability in education outcomes X Total/average X 62. PDO 5: Especially [benefit] the disadvantaged children. Achievement is assessed in the areas of: i) The ROSC objective to target the poor was met consistently, as shown by external evaluations. Early on, among the LC graduates surveyed in the Tracer Study (2007), all came from hardcore poverty (defined as below subsistence level). A more recent study based on 10 This is due in part to the amount of the ROSC allowance being less than the stipend for the qualifying students enrolled in government schools. 20 panel data similarly found that over 90 percent of ROSC students were from families of “extreme poor” (Dang et. al., 2011). Furthermore, among all education incentive programs in rural Bangladesh, the ROSC school allowances appeared to reach the poorest most effectively (Uniconsult 2011). Marginal benefit analysis using data from the impact evaluation surveys (2006, 2009) indicates that ROSC project scale-up would disproportionally benefit the poor (58 percent of the increased program benefits would go to the poorest quintile). ii) Percentage of ethnic and other disadvantaged children enrolled in LCs. While ROSC had an explicit priority to reach the poorest segments of the out-of-school population, the PAD rounded out the definition of “disadvantaged” to consider gender aspects as well as disadvantages associated with socially stigmatized groups, children living in backward or inaccessible areas (with no mainstream school within two kilometers), children from households living on floating/khas land, and working children. ROSC MIS data show that at least half of the student beneficiaries in ROSC were girls, and education allowances were provided to students including those children from tribal populations who happen to live in the area of the LC location. However, within a community, there was no explicit targeting of these groups since any eligible child in the community would have access to ROSC intervention. In the case of SKT schools, ROSC benefits were extended to eligible students who also worked part-time to provide for themselves or families. 63. Table 5 provides a summary of PDO 5. The overall rating is Satisfactory. Table 5: Ratings for PDO 5 Highly Moderately PDO Indicators Satisfactory Satisfactory Satisfactory 1. Percentage of extreme poor enrolled in LCs X 2. Percentage of ethnic, other disadvantaged children enrolled in LCs X Total/average X 64. PDO 6: Support of GOB’s national EFA goals. The overall PDO has an implied causal chain leading from the first four PDO elements to imputed progress on the national EFA goals. What specifically has ROSC achieved in meeting the program related challenges which are outlined in the National EFA Plan (2003) for 2003-2015? This is assessed in terms of: i) Closing the gap on universalizing primary education (UPE). ROSC on the ground embodies the EFA principle of ensuring close cooperation between GOB, NGOs, and broader civil society to share responsibility for program development and management particularly to serve segments of the population which are not served by formal primary schools, and to give high priority to the constraints of the poor in terms of enrollment, attendance, retention, completion, and quality achievement. ROSC achieved both increased access and increasingly effective measures to tackle quality, which the project defined more broadly than simply mastery of the curriculum. 11 The increasing success in infusing the ROSC “package” of quality interventions with mainstream features of formal primary education (using the GOB textbooks, actively involving UEOs) and focusing a PDO level KPI on the goal of primary school completion (as well as lateral transfers of LC students to secondary schools) means that ROSC took concrete steps toward closing the gap on UPE. ROSC-II is adding to this. 11 For example, the role and performance of teachers, conducive learning environment, and capitalizing on the new Grade 5 exam to ensure basic levels of competencies for children graduating and entering secondary school also figured prominently in an inter-related “package” of project interventions to improve and maintain quality. 21 The uniquely significant achievement of ROSC-I is in the headway it made facing its greatest challenge: getting the people’s participation in community-based clientele mapping, resource mobilization, program implementation and monitoring. ii) Contribution to national framework policy on non-formal education. The analytical work on non-formal education was completed (in 2010) and additionally, the project files and other third party evaluations contain reflections on the role of ROSC within the national context of non-formal education. However, it is unclear how the project may have had a proactive role in helping develop an explicit national framework policy on NFE. 65. Table 6 provides a summary of PDO 6. The overall rating is Moderately Satisfactory. Table 6: Ratings for PDO 6 Moderately Moderately PDO Indicator Satisfactory Satisfactory Unsatisfactory 1.Closing gap universalizing primary education X 2. Contribution to national framework policy NFE X Total/average X 3.3 Efficiency (Economic Analysis) 66. The ICR economic analysis (Annex 3) presents an assessment of benefits and costs associated with the project over eight years (2005-2012). Compared to the economic analysis from project appraisal, the ICR economic analysis (EA) shows IRR and NPV. Retrospective data on cost and benefit stream used for the ICR EA are now different than the assumptions and projections made at appraisal because: (i) project cost increased due to additional financing, and (ii) the number of students reaching Grade 5 increased due to project expansion (across more areas) and extension (more years). The increase in the cost stream has been more than offset by a larger increase in benefit stream measured in terms of both internal efficiency under PDO4 above and external efficiency in terms of life-time earnings. Finally, the ICR analysis uses better quality data on both physical (enrollment and completion rates) and financial aspects (actual data on costs, wage premiums, etc.) of the completed ROSC project. The estimated internal rate of return is 27 percent and the net present value of economic benefits is US$ 328 million. 67. In sum: Benefits and costs (Rating: Satisfactory). Internal Efficiency (Rating: Satisfactory). 3.4 Justification of Overall Outcome Rating 68. Table 7 provides a summary of scores of different aspects of project outcomes, using a six- point scale and weights assigned to reflect the relative importance of different factors. The end result is an Outcome rating of Satisfactory. The weight given to “implementation” in the “Relevance” category reflects, in addition to implementation progress in a technical sense, the success in changing attitudes in favor of the new concept for a direct partnership between the GOB and communities to achieve more effective delivery of non-formal primary education at the local level, as well as the ROSC achievement implementing a previously untested package of financial incentives channeled directly to beneficiaries to extend primary education access to the most marginalized. The “Efficacy” of the project is well founded in the responsiveness of the project development objective to the GOB’s priorities for education under the Education for All (EFA) action plan to make education accessible and all-inclusive. All PDO level indicator targets have been achieved or surpassed, including those revised at the time of Additional Financing when the project scope was expanded. Since the targets were achieved, there was no impact of the undisbursed balance on achieving the PDO (a credit amount of approximately SDR 7.2 million was unused at the end of the project). In the “Efficacy” category of Table 7, PDO 6 is weighted 22 slightly less than the other PDOs to avoid over-weighting the EFA objectives, since these are also assessed in the “Relevance” category. PDO4 is added to the category on “Efficiency” in view of its responsiveness to the CAS higher level objective to ensure transparency and accountability in service delivery. 69. It should be noted that if one were to use the metric on project rating envisaged at Appraisal, ROSC-1 can be rated as “Highly Satisfactory,” since it has achieved the core KPIs (1-5) and the remaining ones as well. Table 7: Project outcome evaluation scores Weight Score Weighted score Relevance 0.33 Alignment with development strategy/ objectives/ design 0.15 5 0.75 Implementation 0.18 5 0.90 Efficacy 0.34 PDO 1: Reduce number of out-of-school children 0.06 6 0.36 PDO 2: Improve access 0.06 5 0.30 PDO 3: Improve quality 0.06 5 0.30 PDO 4: Improved efficiency 0.06 5 0.30 PDO 5: Benefit to disadvantaged 0.06 5 0.30 PDO 6: Support EFA goals 0.04 4 0.16 Efficiency 0.33 PDO4: Technical/allocative efficiency 0.17 5 0.85 Benefits to costs (Economic Analysis) 0.08 5 0.40 Internal efficiency (Economic Analysis) 0.08 5 0.40 Total 1.00 5.02 1 = Highly Unsatisfactory, 2 = Unsatisfactory, 3 = Moderately Unsatisfactory, 4 = Moderately Satisfactory, 5 = Satisfactory, 6 = Highly Satisfactory 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 70. Poverty impacts are discussed in PDO 5, above. 71. Gender aspects. Female empowerment is one of the most marked achievements of ROSC. In addition to the equity achieved in female and male participation rates in the LCs – and greater proportion of females than males enrolled in SKTs – over 80 percent of the LC teachers recruited were female. Many among the latter went on to establish new LCs, without requiring assistance from ESPs. Close to 90 percent of all CMC heads were female, together representing significant female empowerment at the local level. Consequently, through the CMC, they had decision- making power and bore major responsibilities in managing funds, employing teachers, monitoring school attendance, arranging training of teachers, and other logistics, all of which was highly unusual practice for these women in the context of their social status. ROSC contributed to the creation of new job opportunities for women in rural areas; this happened directly, through employing teachers for LCs, and in the longer term, by affording females the possibility to complete their primary education, go on to secondary school and then to remunerative employment. 72. Social Development. The multiplier effects of educating poor children and especially poor girls, leading to a positive impact on their career opportunities could have a far-reaching impact on Bangladesh’s economic and social development, especially in terms of poverty reduction. The novel idea of rural communities contracting NGOs to provide them with educational services attempted to recast the NGO-community relationship, which for a long time has been viewed as a 23 patron-client relationship. ROSC attempted to take community empowerment to a new level. In the given political environment where many of the local NGOs are inextricably linked to the political power bases, this was a daunting task, more so for a small-scale education project. Still, within the lifetime of the project, symptoms were visible that these relationships were being redefined in favor of the rural communities. (b) Institutional Change/Strengthening 73. Implementing the project via government systems, in particular through DPE which has overall responsibility for formal primary education, had significant influence on key government functions by bringing them directly into supporting non-formal alternatives to complement formal primary schools. The effects are expected to be sustained in future education sector-wide planning. The institutional strengthening under ROSC also has implications beyond the education sector. Mainly by the capacity development of local communities, NGOs, and financial institutions (Sonali Bank), the project has helped realize sets of accountability measures to reinforce governance for service delivery through decentralized channels of the public sector and through community participation. Besides, ROSC was instrumental in bringing a synergy between the government and NGOs as partners in provision of education, making use of a blend between formal education and a much acclaimed non-formal mode of delivery for which Bangladeshi NGOs are known worldwide. The now-launched third-party monitoring within the education sector of the government has considerably strengthened monitoring and reporting systems. (c) Other Unintended Outcomes and Impacts (positive or negative) 74. Interviews with key stakeholders pointed to the positive, catalytic effect of the GOB introduction of the Grade 5 exam in 2009 on ROSC to orient the assessment of student learning achievement around a nationally recognized benchmark, thereby advancing a mainstreaming of ROSC within the overall primary education system overseen by DPE. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops Not Applicable 4. Assessment of Risk to Development Outcome 75. The development outcomes of ROSC-I are being maintained in the GOB commitment to implement ROSC-II through 2017. A further indicator of sustainability of the benefits of ROSC-I is the fundamentally important policy decision by GOB to adopt the ROSC model as the national platform for reaching at-risk children in rural areas. The proposed implementation plan is being discussed within the context of the PEDP3, the ongoing successor SWAp to PEDPII. For these reasons, the Risk to Development Outcome at the time of ICR evaluation is assessed as “Negligible to Low.” 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry (Rating: Moderately Satisfactory) 76. The fast-tracked preparation period of six months may have left too little time for more development partners to explore co-financing of ROSC (so soon after PEDPII), but the schedule was partly dictated by the need to meet the deadline for the IDA Grant funding. Despite the short planning period, preparation of the basic building blocks for the implementation arrangements was thorough. They were technically sound and, together with the successful testing of comparative advantages of a grant-only vs. grant-plus-allowance approach, proved a solid base for extending the project’s development impact through Additional Financing for ROSC-I and next in ROSC-II. 24 77. Quality at entry could have been strengthened in three respects. First, despite the ground- level mapping of remote areas and precautions taken at Appraisal to avoid political interference in the selection of beneficiaries, project preparation could have benefited from a more comprehensive analysis of risks inherent in implementing an ambitious new concept that expected vulnerable population communities to lead in collaboration with NGOs. The Bank team sought advice on this at the QER; in retrospect, the guidance received, that “CMC membership should be limited to community members to empower them fully,” may have been most salient. A different tack was taken during implementation, i.e. the CMC membership included ESPs (NGOs), and this increased their leverage to dominate over local populations. Furthermore, the Bank team had studied the BRAC NFE program but could have gone farther distilling lessons from that experience, which clearly showed how direct links between BRAC regional administration and the villages/LCs were critical to “protect” beneficiaries from local elite capture. A second factor that affected project quality at entry was the timing of the baseline study. In an apparent trade-off for fast-tracking the preparation period, the project design called for the study to be conducted in the first six months of implementation. The baseline study was delayed considerably beyond this period due to difficulties procuring the firm and lost the opportunity to reinforce quality at entry since the project had already started up in the majority of Upazilas. Third, the interpretation of the safeguard policy related to Indigenous Peoples as requiring a Tribal Action Plan seemed more designed to satisfy the letter of the law than to tackle the many complex cultural and linguistic issues which would have been needed for LCs to expand enrollments of tribal children. (b) Quality of Supervision (Rating: Satisfactory) 78. The Bank team focused on the project outcomes and their impacts on the PDO during implementation. This required intensive supervision given the extensive geographic scope, range of activities, and multiple stakeholders. Supervision was made more challenging with the changes in national government and their repercussions at decentralized levels, as explained earlier in this ICR. The Aides-memoire and ISRs produced over the project’s lifetime reflect attention to detail with precise recommendations and follow-up actions for all aspects of the program, combined with strategic reminders of the overall development goal. Interviews conducted for the ICR pointed to the catalytic value of the Bank’s site visits for identifying problems as they occurred and using the flexibility of the project’s pilot nature to make changes which improved implementation. Team members notably helped in the early years to delve into the ground realities of ESPs who were not meeting the criteria for service delivery and to suggest solutions. Studies embedded in the project design to obtain data and analytical perspectives from credible third party experts produced reliable information and led to improvements in implementation. 79. Overall, the ICR team received positive feedback from government counterparts and the SDC concerning the Bank’s responsiveness to emerging situations. Some examples where the Bank team together with the Borrower’s project team showed proactivity include the following: successful transition from the two ROSC interventions (Grants-and-Allowances, and Grants- Only) towards a single scheme of Grants-plus-Allowances; introduction of a validation mechanism as a compliance confirmation tool to process and disburse grants and allowances; introduction of cluster-based master trainers to enhance quality of teaching/learning; limiting the duration of ESP services to foster community empowerment and sustainability; AF processing to address the funding gap and to provide existing ROSC students with an opportunity to complete Grade 5; project restructuring/Financing Agreement amendment(s) to address changing disbursement needs; strengthening the ROSC MIS Cell at LGED continually improving on the relevance of M&E; and capacity building of key stakeholders, including the local education offices, to enhance ownership of implementation on the Borrower’s side. The task leadership on 25 the Bank side changed six times. This did not negatively impact the team’s ability to ensure diligent implementation support as each incoming TTL had an overlap with the previous one through prior direct involvement in the project team, thereby making hand-over arrangements smooth and successful. Through close monitoring of the project progress, the Bank and GOB anticipated the need of Additional Financing and prepared this well in advance. There was no impact of the undisbursed balance on ROSC-I results, since the targets were achieved, and even surpassed in several areas. As part of the project transition arrangement, the ROSC-I findings were fed into the design of ROSC-II which the Bank team processed in a timely manner. The Bank team also notified the GOB on time and persisted with several reminders that the balance of funds from ROSC-I must be refunded; the GOB was processing this at the time of the ICR. 80. Three shortcomings were noted: (i) the project was slow to gain momentum due mainly to human resource constraints in the implementing agency, which resulted in slow disbursements the first three years and delayed the Mid-Term Review where crucial adjustments were made to project design and implementation arrangements; earlier start-up of Bank implementation support missions could have helped offset weaknesses in initial implementation (the first such mission occurred ten months after effectiveness); (ii) ISR ratings were too positive in certain aspects of the project performance (supervision of effects in terms of the participation of indigenous peoples did not receive the attention required, so the ratings for safeguard compliance do not reflect the limited progress made); and (iii) FM and safeguards experts familiar with field realities were not always accessible on a full-time basis in Dhaka. (c) Justification of Rating for Overall Bank Performance (Rating: Satisfactory) 81. With the rating of Moderately Satisfactory for preparation and Satisfactory for supervision, overall Bank performance is rated Satisfactory. 5.2 Borrower Performance (a) Government Performance (Rating: Satisfactory) 82. An important gauge of GOB commitment lies in how the ROSC PDO remained unchanged despite multiple changes of government. From the outset, GOB counterparts emphasized the strong potential of the proposed project and underpinned this commitment by vesting project implementation with DPE, the government body that has overall responsibility for primary education in Bangladesh. Detailed operational arrangements were clearly agreed by Appraisal: the MoPME technical team worked alongside the Bank at every step. These precautions were challenged in the initial years of implementation by chronic delays in recruiting qualified personnel for the ROSCU. At different points in time, there were also divergences among Project Directors (PDs) in their views about program implementation and what they would support. However, given that two markedly effective PDs (the project had a total of five in its lifetime) were in place for a combined total of six years, this balanced out the differences. The relative continuity in ROSCU leadership stands in contrast to most other projects in Bangladesh. Particularly from 2009, the GOB exercised its leadership through the ROSC Committee and this was joined by active liaison with the parliamentary standing committee on education, which helped ensure broad-based support and ownership to the project at all levels within the country. The government’s openness to third party M&E was also a key action, as was the regular annual auditing of project accounts by the Comptroller and Auditor General of Bangladesh. Both were fundamental for drawing credible lessons from ROSC-I. 26 (b) Implementing Agency or Agencies Performance (Rating: Satisfactory) 83. Continued improvement in human resources at the ROSCU significantly supported the project implementation. The human resource deficiencies at ROSCU negatively affected initial performance of the project; this is reflected in the disbursements which fell off schedule in the first year and lagged until midterm. The situation improved as the project moved forward, particularly profiting from the vigorous PD management from 2007/2008. The ROSC was fully staffed with qualified personnel over its last few years, and even with expansion of the project scope through the Additional Financing, the Implementing Agency was able to complete the project, meeting or exceeding most targets, and played a key role in the seamless preparation and approval of ROSC-II. A high level of engagement from DPE’s Director General (DG), who made it a priority to participate in all implementation support missions, delivered strong benefits especially through the progressively successful mobilization of ROSCU and field level staff (UEOs, AUEOs) to maintain active field presence in support of LC service delivery. 84. Program decisions made good use of the M&E data produced under the project. The ROSC was able to collect data even from the remotest LCs and regularly issued Semi-annual and Annual Reports. The LGED, contracted for the ROSC MIS Cell, was a consistently strong performer and took many initiatives to hone the reliability of data collection techniques (for example, through use of mobile phone technology). In the project’s later years, beneficiaries were consulted in increasingly meaningful ways on various aspects of the project (the Upazila level LC teacher conferences for 2008, 2010 and 2011 cohorts is one example) to identify, build on, and broadly publicize what was working well. (c) Justification of Rating for Overall Borrower Performance (Rating: Satisfactory) 85. With the rating of Satisfactory for preparation and supervision, overall Borrower performance is rated Satisfactory. 6. Lessons Learned 86. Key lessons are:  Ambitions and expectations for innovation need to be realistic. Changing approaches to service delivery, putting communities which have long remained disempowered at the center of the whole intervention, is difficult to implement in any country setting, and especially so in the context of Bangladesh. Analysis of the political economy of vulnerable populations would help to mitigate risks of such a proposed innovation agenda.  The model of government and community partnership can work. Lessons from ROSC are: (i) community awareness is crucial, as is the assimilation of roles intended for the respective stakeholders; both require vigorous and sustained communication efforts; (ii) if selected properly, local NGOs play a constructive role supporting the first year of LC operations; the dividend of using NGOs beyond the first year is marginal; (iii) clear performance criteria with initial and ongoing evaluations and action to terminate poor performers or to help them improve can ensure strong performance; and (iv) active engagement of UEOs with back-up from their central administration (MoPME/DPE/ROSCU) is decisive for vigilance of LC issues. Leaving these activities only to local NGOs (who are sometimes politically influential) without adequate government oversight was not effective. 27  Systematic strengthening of governance arrangements has potential for broader impact. This ICR discusses numerous governance issues in the context of public-private sector partnerships and how they were built up. Among them is the strengthening of financial management capacity, transparently publishing financial information, and third party monitoring, which improved accountability.  Combining educational allowances with higher level grants – the level received by LC schools in Grant-only areas – is likely the most effective way to reach the dual goal of attracting out-of-school children to school and improving their learning.  From the very outset, quality must be targeted simultaneously with access issues. Teachers are a key determinant of LC quality. Lessons learned from the changes made successively over the project lifetime are that more successful strategies: (i) actively manage the teacher selection process (recruit LC teachers from a pre-enlisted pool with appropriate qualifications); (ii) contract out teacher development (training and on-site support) to professional agencies with track records in areas LC teachers find most challenging (e.g. multi-grade teaching); and (iii) use incentives to attract and retain teachers and reward for performance.  A robust M&E program needs multiple systems of checks and balances. Monitoring all activities taking place simultaneously at ground level is impossible; even more so when such activities take place in the remotest and the most disadvantaged areas of the country. The monitoring framework needs to involve various stakeholders for monitoring at the local level and procedures for cross-checking feedback from multiple sources and resolution of data discrepancies. To offset risk of inflating student numbers, it is indispensable to have a mechanism that uniquely identifies each student in the project and is coupled with the MIS. Mainstreaming select survey instruments into the overall monitoring arrangements and obtaining necessary information of other cross-sectoral household surveys is necessary to ensure informed decision making. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies 87. Comments from the Ministry of Primary and Mass Education on the Bank Implementation and Completion Results Report (ICCR) and the Government’s own completion report on the project are attached at Annex 5. The Bank ICR team agrees with the assessment of achievements and strengths of ROSC-I highlighted in Annex 5, as well as the challenges to be addressed in the programs directly following on ROSC-I. The ICR team endorses the importance of building on the ROSC-I achievements in “mainstreaming” its approach as a second chance alternative in the primary education sector. In contrast to many other non-formal education programs, the ROSC has equivalencies with formal primary education because ROSC students use the same textbooks (prepared by the National Curriculum and Textbook Board) as the formal system and participate in the same Grade 5 terminal exams after completion of a five-year primary cycle; the ROSC design also involves the regular DPE administration at central and decentralized levels. Under the present (ROSC-I) design, students can complete the primary education cycle in five years and enter the mainstream for secondary education. (b) Co-financiers 88. The Swiss Agency for Development and Cooperation (SDC) provided comments in an interview with the Bank ICR team. The Bank concurs with the SDC’s synopsis of the longer term development impact of ROSC which was described as follows: “The ROSC design – NFE 28 channeled through the government system (i.e. DPE) – was its most compelling feature, and it was important to support this so as not to miss a generation of children.” The SDC highlighted the main success of ROSC as the combination of: (i) its social change process, (ii) reaching the under-served; (iii) establishing the LGED monitoring system; and (iv) achieving commendable learning outcomes especially under conditions of a strong CMC and good collaboration between parents and LC management. (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society) 89. No comments were received from other stakeholders. 29 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Appraisal Appraisal Appraisal Estimates for Estimates for Actual/Latest Percentage of Estimates for Additional both Original and Components Estimate Both Original Project Financing Additional (USD millions) Appraisals (USD millions) (USD financing millions) (USD millions) Improving Access to Quality Education for Out-of-School 53.5 35 88.5 79.3 90% Children Communications and Social 2.3 2.3 1.2 52% Awareness Project Management and 2.1 1 3.1 5.2 168% Institutional Strengthening Monitoring, Evaluation and 3.5 3.5 2.6 74% Research Total Baseline Cost 61.4 36 97.4 88.3 91% Physical Contingencies Price Contingencies 1.4 Total Project Costs 62.8 36 98.8 88.3 89% Front-end fee PPF Front-end fee IBRD Total Financing Required (b) Financing Appraisal Add. Fin. Actual/Latest Type of Estimate Estimate Percentage of Source of Funds Cofinancing (USD (USD Appraisal millions) millions) Borrower 5.8 1 4.5 66% International Development 35 25 71% Association (IDA) IDA GRANT FOR POOREST 51 53.9 106% COUNTRY SWITZERLAND: Swiss Agency 6 4.9 82% for Dev. & Coop. (SDC) Note: The main reason for less than full disbursement of the Additional Financing is due to the fact that ROSC-II and ROSC I overlapped 6 months. Because of Government of Bangladesh guidelines on funds release to projects with the same names (ROSC I and ROSC II) the ROSC I chose to use ROSC-II funds since the latter were made effective 6 months before ROSC I closed. 30 Annex 2. Outputs by Component The ROSC completed most planned activities, and exceeded targets for many. The matrix in Table 1 below summarizes the outputs delivered within the project period (including additional financing), by component. Tables 2-7 provide additional detail on selected outputs: Table 1: ROSC Main Outputs by Component Component wise outputs 1 Component 1: Improving Access to Quality Education for Out-of-School Children DO1 Objective 1: Education allowances for eligible students DO1 Objective 2: Grants to LCs where the students enroll 1.1 Sub-component 1.1: Education allowances to Out-of-School Children Total initial enrollees over AY2005-2012: 778,147 students (replaced students not included) By gender: 382,830 females; 395,317 males Contracting system established with Nationalized Commercial Bank (Sonali Bank) for disbursing grants Total amount education allowances disbursed to students (ROSC LC + SKT students): US$ 26.21 million 1.2 Sub-component 1.2: Grants to Learning Centers/schools - Total new LCs established: 23,695 LCs over AY2005-2012 - Total operating LCs AY2005-2012: 177,576 LCs (Note: Most schools = 1 teacher, 1 class cohort type enrolling all students in Grade 1 and moving them through to higher grades.) Total number LC teachers AY2005-2012: 90,861 teachers 1.3 Sub-component 1.3: Shishu Kallyan Trust (SKT) Schools) Total initial enrollees over AY2005-2012: 23,456 students (replaced students not included) By gender: 12,473 females; 10,983 males Total number SKT institutions supported AY2005-2008: 73 (Note: Grant support to schools was based on School Improvement Plans submitted.) Construction: 1 Technical School (Fatullah) built with ROSC support See also in this Annex: Table 2: Number of Learning Centers and SKT schools supported by ROSC, AY2005-2012 Table 3: Number of student years supported by ROSC, AY2005-2012 2 Component 2:Communications and Social Awareness DO2 Objective 1: Raise awareness about primary education in general and the Project in particular DO2 Objective 2: Mobilize stakeholders to open new LCs or expand existing ones DO2 Objective 3: Disseminate information on qualification criteria, resource support & other operational guidelines to stakeholders DO2 Objective 4:Assess the effectiveness and “reach” of the communications and social awareness activities 2.1 Sub-component 2.1: Social Awareness Social Awareness Workshops: - 60 Upazila workshops, 34 District workshops in 2004-2005 to disseminate information on ROSC - 40 workshops in 2005-2006 31 - See also in this Annex: Table 4 Participants in Community Mobilizations & Social Awareness, AY2005-2012 Posters etc.: 11 posters printed by ROSCU (some on contract to BITOPI, BCCP) Television & radio spots 2.2 Sub-component 2.2: Community Mobilization Total ESPs recruited: 1726 NGOs (Note: The longer CMCs were in operation, the more ESPs could be phased out. 28 ESPs remained in 2012.) Information/Training for frontline workers: - Pictorial CMC Manuals distributed to 60 Upazilas for new LCs - Videos: * ROSC goals, processes & achievements; video screened 5280 times in 59 Upazilas - * Video for “courtyard meetings” for use by Government & community members on the ground to inform CMCs, to supplement other awareness/mobilization outreach efforts - Circulars: * Roles, responsibilities of ERPs in cluster-based teacher support program * Roles, responsibilities of field level partners & CMC financial management aspects in context of larger responsibilities entrusted to UEOs, and the need to empower local communities * Operational mechanisms of Ananda schools (LCs) - See also in this Annex: Table 5Participants in CMC and ESP training, AY2005-2012 3 Component 3:Project Management and Institutional Development DO3 Objective 1: Establish a sound structure for managing and implementing the Project DO3 Objective 2: Develop and strengthen the capacity to deliver quality primary education to out-of- school children 3.1 Sub-component 3.1: Project Management ROSC Committee established within GOB ROSC Unit established within DPE Operations Manual (& English translation of shorter version) produced and updated progressively 3.2 Sub-component 3.2: Institutional Capacity Building Training established for CMCs, ESPS across participating Upazilas (Table 5, this Annex) Established system of teacher education/training (initial, refresher, continuous training, content & pedagogical knowledge). - Basic instructional package of teaching-learning materials produced and disseminated to LCs (Teacher Support Manual, teachers’ guides) - Sequenced lesson plans developed for Master Trainers (MTs) - Established system of supervisory support (involves evaluation forms used by supervisors for observations of Master Trainers training LC teachers and observation of LC teachers in classrooms) - See also in this Annex: Table 6 Participants in Teacher training, AY2005-2012 Staff training: - In-country training (management, educational development, financial management): 80 staff - Overseas training: 60 staff (i.e. training at National Institute of Education, Singapore; International Institute of Educational Planning, UNESCO Paris) 3.3 Sub-component 3.3: Networking to Improve Quality Established ERP Networking Group: technical advisory services to ERPs 32 (Note: This network also strengthened the professional partnership between MoPME/DPE and NGOs specialized in education, i.e. ERPs.) 4 Component 4: Monitoring, Evaluation and Research DO4 Objective 1:Monitor the use of grants support DO4 Objective 2:Monitor the use of education allowances DO4 Objective 3: Evaluate the two approaches (grants, grants-plus-allowances) through action research, thematic studies, and impact assessments 4.1 Sub-component 4.1: Monitoring the use of Grant Support by LCs ROSC reporting system established linking DPE with Upazilas (includes use of standardized monitoring templates, checklists) MIS Cell established (by contract to LGED, independent agency): tracking/verifying LC & SKT student enrollments. MIS Cell has facilities for school mapping/ monitoring /educational data collection & processing Enhanced surveillance system established: monitoring in 20 percent of LCs in a year; conducted monitoring in 21,609 LCs out of 23,695 MIS Cell publications: 24 evidence based reports providing better data and analysis for decision making on program implementation and quality assurance; includes 8 Annual Reports Computerized database & financial management system set up for monitoring student enrollments and grants across Upazilas Comprehensive database set up on MTs, UEOs, teachers, ERPs etc.; introduced mobile phone technology for data collection/monitoring Comprehensive database set up on MTs, UEOs, teachers, ERPs etc.; introduced mobile phone technology for data collection/monitoring 4.2 Sub-component 4.2: Monitoring and Assessing the use of Education Allowances As above for 4.1 4.3 Sub-component 4.3: Evaluating the Impact of Grants and Education Allowances (and additional studies/surveys for making program decisions) - Developed numerous thematic monographs and publications on program activities and innovations, including LGED’s 8 Annual Statistical Reports, financial and performance audit reports, ROSC Mid-term Review evaluation - Produced 7 third party evaluations (see also Annex 6 List of Supporting Documents): * Baseline Study (2006) * Evaluation of ESP Performance 2005-2007 (2007) * Evaluation of SKT Schools (20__) * Tracer Study (2008) * Qualitative study on performance of ROSC implementing agents (2010) * Quantitative Study evaluating ROSC students learning achievement & transition (2010) * Public Expenditures Tracking Survey (2010) 4.4 Sub-component 4.4: analytical Studies of Non-formal Education Non-formal education sector analysis (2010) 33 Table 2: Number of Learning Centers and SKT schools supported by ROSC, AY2005-2012 Category Detail AY AY AY AY AY AY AY AY AY 2005 2006 2007 2008 2009 2010 2011 2012 2005- 2012 No. of LC 727 7,184 3,027 4,066 - 4,700 3,991 - 23,695 institutions SKT 44 19 8 2 - - 73 (initial year) Total 771 7,203 3,035 4,068 - 4,700 3,991 - 23,768 Total No. 1st - 7,910 10,907 14,936 14,816 13,455 13,269 11,422 86,715 operating Sem. LCs 2nd 727 7,893 10,884 14,867 14,685 17,097 13,795 10,913 90,861 Sem. Source: LGED ROSC MIS Database Table 3: Number of student years supported by ROSC, AY2005-2012 AY AY AY AY AY AY AY AY Total 2005 2006 2007 2008 2009 2010 2011 2012 LC + SKT 30,145 254,054 357,153 495,063 480,427 533,615 428,985 355,744 2,935,186 SKT only 6,766 9,581 12,272 14,441 15,207 11,173 7,081 76,521 Source: LGED ROSC MIS Database Table 4: Participants in Community Mobilizations and Social Awareness, AY2005-2012 AY AY AY AY AY AY AY AY AY 2005 2006 2007 2008 2009 2010 2011 2012 2005- 2012 Community Division 8,552 1,002 9,554 mobilization Advocacy & social Upazila - - - - 711 980 - - 1,691 awareness: Advocacy number of Union - - - - - 3,536 - - 3,536 participants advocacy Community - - - - 41,365 33,557 10,780 - 85,702 level Source: LGED ROSC MIS Database Observations on Table 4: The late start-up of social awareness campaigns targeted at community level coincides with the period of a particularly large number of cases of local NGO dominance reported by other sources, including the Aides-memoires of the Bank’s implementation support missions. Table 5: Participants in CMC and ESP training, AY2005-2012 AY AY AY AY AY AY AY AY AY 2005 2006 2007 2008 2009 2010 2011 2012 2005- 2012 CMC Basic 5,805 54,456 23,888 28,105 - 4,400 8,013 - 122,667 training Refresher - - - 50,426 216,642 - - - 267,068 ESP Foundation - 300 108 - - - - - 408 training Refresher - - 300 - - - - - 300 Source: LGED ROSC MIS Database Observations on Table 5: Throughout the project’s lifetime, the assimilation of information on roles and responsibilities of relevant parties was a recurrent challenge. Acceleration of the refresher training from AY2008 is striking and, joined by the much more prominent targeting of social awareness campaigns at community level from AY2009, undoubtedly reinforced the base for the project’s strong performance after the MTR. 34 Table 6: Participants in Teacher training, AY2005-2012 AY AY AY AY AY AY AY AY AY 2005 2006 2007 2008 2009 2010 2011 2012 2005- 2012 Teacher Basic 727 7,184 3,027 4,066 - 6,325 3,899 - 25,228 training Refresher - 7,400 3,027 4,066 - 6,828 - - 21,321 (participants) Subject - - - 727 9,205 6,325 3,645 - 19,902 based Classroom TGM - - 3,027 7,093 10,120 6,168 - - 26,408 support *classroom Monthly - - - - - - 11,193 9,981 21,174 teacher training Intensive - - - - - - 1,604 6,175 7,779 teacher training Total 727 7,893 10,884 14,867 14,685 17,097 13,795 10,913 90,861 number of LC teachers (based on 2nd semester) *TGM = “Teacher Group Meeting” Source: LGED ROSC MIS Database; information in 2012 is based on newly constructed ROSC MIS-Cell teacher database. Note to Table 6: Initially, the ERPs were to provide a 10-day foundational and 3-day refresher training, plus “Teacher Group Meetings” (TGM). Until project midterm, there were weaknesses mainly due to an insufficient number of experienced Master Trainers (less than half of MTs were experienced in primary education and the MT dropout rate was high). The Aides-memoire from implementation support missions observed that CMCs were unable to hold all ERPs accountable for their services; furthermore, there was a great need to add measures to support the weakest LCs who required intensive, practice-based support (e.g. for multi-grade teaching). The Bank and GOB team response is evident in the number of participants in intensive teacher training in the latter two years of the project. Shifting the training from Upazila to cluster level helped to increase the contact hours between MTs and LC teachers. Supporting the increased field presence of ERP representatives were: progressively more active oversight and field-level engagement by ROSCU staff and consultants, sequenced lesson plans prepared by ROSCU, and introduction of quality bonuses to reward MTs and teachers for better student results in grade completion and Grade 5 exam pass rates. 35 Table 7: Percentage of students reaching expected competency levels Cohort No. students % <40 marks % 41-59 marks %60-79 marks % > 80 marks B* M E B M E B M E B M E 2005 23,379 7 7 8 25 19 28 50 47 49 18 27 15 2006 245,165 2 1 3 24 18 30 57 55 53 17 26 14 2010 311,450 4 3 5 19 16 20 60 54 61 17 27 14 50% & above marks in Bangla 50% & above marks in Math LC self-reported LGED Field LC self-reported LGED Field (ROSC Form 10) Monitoring (ROSC Form 10) Monitoring 2008 97% 94% 99% 92% 2009 97% 69% 99% 64% % students with qualifying marks in Bangla & Math LC self-reported (ROSC Form 10) LGED Field Monitoring 2011 5645** 91% 89% *B = Bangla, M=Math, E=English ** Number of students interviewed Note to Table 7: The table on student performance is based on LC teachers’ assessments, reported to the ROSC MIS Cell. Third party evaluations under the project also provide perspectives on trends in students’ performance over time and compare across school types. A standard achievement test administered to 5063 Grade 2 students in 2006 found LC students in grant areas consistently outperformed students in other formal primary schools in grant areas, formal primary schools in control areas, and LC students in grant-plus-allowance areas (Baseline Survey 2006). A follow-up survey of 2306 of the original baseline students in 2008 found that test scores in Bangla and Math for grant-only LCs once again had a very similar distribution to those for non-ROSC schools, but that both grant and grant-plus-allowance LCs had performed as well as non-ROSC schools in boosting student test scores from 2006 to 2008 (Dang, Sarr, Asadullah, 2001). 36 Annex 3. Economic and Financial Analysis 1. The economic analysis presented is an assessment of benefits and costs associated with the Bangladesh ROSC-I program. Costs include program costs occurred during project period and private costs that comprise direct household outlays as well as opportunity costs for schooling. The benefits are calculated based on the wage premium for Grade 5 completers of ROSC-I. The economic analysis presented here focuses on the eight-year project period, 2005-2012. 2. Source of benefits are from improved access to primary education resulting in increased number of primary school completers (Table 1 below) who earn higher wage relative to non-completers. The primary completers will earn the wage-premium. Wage levels for primary completers with some junior secondary education and non-completers is estimated from HIES 2010 and then projected to account for inflation (12% annually) for future years (the earning differential in 2010 is estimated to be over BDT7,200 or US$87 per person annually). The benefit stream accruing from lifetime earnings for the four cohorts is assumed to continue for 21 years (even though a typical primary completer will earn beyond 20 years, “discounting” will make the values insignificant beyond this time). Table 1: Impact of ROSC-I on the number of Primary Cycle Completers 2005 2006 2007 2008 2009 2010 2011 2012 Grade 5 completers 895 2,695 7,715 19,694 55,440 224,137 317,395 444,717 (cumulative) Total 30,145 254,054 357,153 495,063 477,760 533,615 428,985 359,617 Enrollees 3. ROSC-I program costs are estimated based on project cost occurred during the period 2005-2012 and estimated private cost of schooling including direct and indirect (opportunity cost) costs (Table 2). Per student private household cost (of attending primary school annually among bottom 40 % poor households) is about US$4 in 2010 as estimated using HIES 2010 data, and it is discounted to each corresponding project year; similarly, opportunity cost (forgone earnings) per student per year while attending school is estimated to be US$21 per student aged 6-12 in 2010 and discounted to each corresponding project year. Table 2: ROSC Project Costs and Private costs, US$ million 2005 2006 2007 2008 2009 2010 2011 2012 3.3 6.2 9.8 8.9 14.3 15.4 30.8 10.0 Total Project Cost Private Household 0.1 0.6 1.0 1.6 1.7 2.1 1.9 1.8 Cost 0.4 3.3 5.2 8.1 8.8 11.0 9.9 9.3 Opportunity Cost 3.7 10.2 16.0 18.5 24.7 28.4 42.6 21.0 Total 4. Based on a discount rate of 12 percent for the benefit and cost streams described above, the present discounted value of benefits (as of 2005) for the base-case scenario is estimated to be US$478 million while the present discounted value of costs is estimated to be US$91 million, and therefore the net present value (NPV) of program benefits is $328 million. The internal rate of return (IRR) associated with this NPV is 27 percent (Table 3 below). 37 Table 3: Discounted Present Values of Benefits and Costs streams, ROSC 2005-2012, US$ million Cost (NPV) 90.75 Benefit(NPV) 478.52 Net Benefit(NPV) 328.22 Internal Rate of Return (IRR) 27% Note: Discount Rate of 12% is used in deriving the DPVs, Discounted as of 2005 38 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Nilufar Ahmad Senior Social Scientist SASES Safeguards Mahmudul Alam Senior Education Economist SASHD Economic analysis Learning Center Maria Elena Andersen Consultant SASED grants Irajen Appasamy Senior Operations Officer AFTEW Operations Beatrice Avalos Consultant SASED Education quality Monitoring & Hon-Chan Chai Consultant SASED Evaluation Amit Dar Lead Education Economist SASED Co-Team Leader Communications & Maria Eitel Consultant SASED social awareness Zafrul Islam Lead Procurement Specialist SARPS Procurement Scherezad Monami Latif Senior Education Specialist SASED Management Education allowances Shiva Raj Lohani Consultant SASED & grants Hena G. Mukherjee Consultant, Lead Education Specialist SASED Co-Team Leader Yoko Nagashima Senior Education Specialist SASED Education quality Rajat Narula Senior Finance Officer LOAG2 Disbursements Management, Md. Mokhlesur Rahman Senior Operations Officer SASED procurement & costing Shobhana Sosale Senior Operations Officer AFTEW Education Kishor Uprety Senior Counsel LEGES Legal Social mobilization & Carolyn Winter Senior Social Development Spec MNSSU communications Senior Financial Management Financial Suraiya Zannath SARFM Specialist management Supervision/ICR Nilufar Ahmad Consultant SASDU Safeguards Senior Financial Management Financial Burhanuddin Ahmed SARFM Specialist management SDC Program Tahsinah Ahmed Gender specialist SDC Manager Irajen Appasamy Senior Operations Officer AFTEW Co-Team Leader Senior Financial Management Financial Papia Battacharya Specialist management Hon-Chan Chai Consultant SASED M&E Nazmul Chaudhury Lead Economist HDNCE Research & analysis Consultant, Financial Management Financial Aminur Rahman Chowdhury SARFM Specialist management Helen J. Craig Lead Human Development Special AFTHD Education Amit Dar Lead Education Economist SASED Co-Team Leader 39 Subrata S. Dhar Senior Operations Officer SASED Co-Team Leader Marghoob Bin Hussein Senior Procurement Specialist SARPS Procurement Syed Rashed Al-Zayed Josh Research Associate SASED ICR Co-Team Leader Qaiser M. Khan Sector Lead Economist AFTHD Economic analysis Fabienne Lagier Education Advisor SDC Education Pema Lhazom Senior Operations Officer EASHH Co-Team Leader Education allowances Shiva Raj Lohani Consultant SASED & grants Administrative Bertha M. Mburugu Program Assistant LCSPS support Marta Elena Molares- Lead Counsel LEGES Legal Halberg Hena G. Mukherjee Consultant, Sr. Education Specialist SASED Co-Team Leader Yoko Nagashima Senior Education Specialist SASED Education Senior Financial Management Financial Rajat Narula EASFM Specialist management Susan Opper Consultant, Senior Education Specialist SASED ICR Co-Team Leader Dilip Parajuli Senior Economist SASED Co-Team Leader Capacity building & Md. Mokhlesur Rahman Sr. Operations Officer SASED environment Administrative Nazma Sultana Program Assistant SASHD support Harvinder Singh Suri Consultant SARPS Procurement Financial Amandeep Trana Sr. Financial Management Specialist SARFM management (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY04 53.30 301.71 FY05 0.00 -6.47 TOTAL: 53.30 295.24 Supervision/ICR FY04 0.00 0.00 FY05 43.90 170.30 FY06 56.35 216.61 FY07 35.31 126.10 FY08 29.20 97.86 TOTAL: 164.56 610.87 40 Annex 5. Borrower's Comments on Draft ICCR and Summary of Government’s ICR 5.1 Comments on Draft ICCR Government of the People's Republic of Bangladesh Ministry of Primary and Mass Education Bangladesh Secretariat, Dhaka Planning Section-2 www.mopme.gov.bd No. 38.013.014.57.01.070.2012-189 Date: 19 December 2013 Subject: Comments on Implementation Completion and Results Report of ROSC-I. The undersigned is directed to convey the following comments on the Implementation Completion and Results Report of ROSC-I project prepared by the World Bank: 1. ROSC Project is one of the most challenging projects of the Government attempting to reach the primary school dropouts and the un- enrolled children in the remotest and the most inaccessible rural areas and those from the most disadvantaged families. While the Government’s sincere efforts to enroll and retain all children of school going age in the primary schools has been considerably successful as evident in the enrolment rates and gradual decrease in dropout rates, still there are a significant number of children from the disadvantaged families in the rural Bangladesh who, for various reasons, couldn’t be enrolled or even if enrolled, couldn’t complete the primary education cycle and move to the next level of education. ROSC Project has given them the opportunity to complete primary education and many of them have moved to secondary education paving their way for better opportunities in life. The fact that more than 750 thousand children have received education in the Ananda Schools is no simple achievement, more so when we take into cognizance the odds against which this has been achieved. These are the children coming from the poorest of the poor families, many of them are the first generation learners and many of them have to contribute to the family earnings for a living. 2. In the last 10 years, ROSC had to overcome many obstacles in implementation. Gathering un-enrolled and dropped-out children, particularly in the inaccessible areas, bundling them up, and grooming them at a center for a 5 year cycle, was not an easy task to perform. 3. As documented in the ICR, the ROSC Project, being a pilot project, went through a steep learning process, because of its unique features tried for the first time in Bangladesh. The attempt to blend formal education with the non-formal mode of delivery, trying to bring synergy between the Government’s sincere efforts and the much acclaimed experiences of the Bangladesh NGOs in non-formal education, making the government officials on the ground work with the local NGOs in a complex web of power relations in the rural areas, attempting to empower the rural school communities with considerable decision making authority vested in them were daunting challenges, and might have been considered by some too much for an education project to take on by itself. It demands a change in work culture, a change in the mindset of the people both at the center and on the ground. Yet, ROSC Project has started the process and set an example which the Government intends to pursue further with confidence based on the experiences of implementing this Project. While ROSC Project was implemented only in 90 upazilas of the country, the Government has now decided to follow the ROSC model in the rural areas through 41 ROSC II Project in 148 upazilas and through second chance education program under PEDP III in the rest of the country knowing that that there are still many out-of-school children in Bangladesh. This is a testimony of the fact how important ROSC model is to the government. 4. In spite of all the narrations sketched above, it would also be important to record the experience that had been achieved during implementation of ROSC-II. It is to be noted that at the end of ROSC-I the ongoing LCs of 48 upazilas were transferred to ROSC-II Project. A validation of LCs in 48 upazilas in ROSC-II shows a considerable number of students enrolled in 2010 and 2011 are missing. While this could have happened due to a number of factors during transition or before, we need to be careful in addressing this issue in the new project. 5. Moreover, one significant achievement of ROSC Project also documented in the ICR, is the elaborate M&E system the ROSC-1 had in place, which the report reveals, is a good example to be followed in other project as well. The M&E system in terms of technicality was excellent but it had also some limitations in capturing the ground scenario of the project results. 6. The experience of implementing ROSC project with all its innovations, the challenges that it incited and faced are a tremendous asset for the Government. While we can’t claim success in every aspect of the project, we certainly learned what the issues are and how to tackle them in our subsequent efforts in providing education to the most disadvantaged children both under ROSC II Project and the second chance education program being designed for the rest of the country following the ROSC model. 7. We always appreciate your contribution for the primary education sector of Bangladesh and hope that this excellent partnership between the World Bank and MoPME will continue in the future. Kindest Regards, (Md. Motasim Billah) Senior Assistant Chief Phone: 9568829 mopme.plan2@gmail.com Country Director The World Bank Dhaka Office E/8-A Rokeya Sharani Sher-e-Bangla Nagar Dhaka-1207. (Attention: Mr. Subrata S Dhar, Senior Operations Officer) cc: 1. Director General, Directorate of Primary Education, Mirpur-2, Dhaka-1216. 2. Project Director, Reaching Out of School Children (ROSC) Project-Phase-II, Directorate of Primary Education, Mirpur-2, Dhaka-1216 3. P. S to Secretary, Ministry of Primary and Mass Education. 4. P. O to Deputy Chief, Ministry of Primary and Mass Education. 42 5.2 Borrower’s ICR 1. Location of the Project 90 remote and disadvantaged upazilas of Bangladesh and 74 Shishu Kalyan Trust School 2. Implementing Agency Directorate of Primary Education 3. Administrative Ministry Ministry of Primary and Mass Education 4. Background Over the last two decades Bangladesh has achieved remarkable social and economic development in the areas of poverty alleviation, human development and education, especially primary education. The pass rate has increased at the primary level in the past few years. Gender parity in primary education has reached a significantly high level. But there are still 35 million hardcore poor people in Bangladesh and half of the total population is still illiterate. The country is yet to ensure equal educational opportunities across all regions. Many school-age children are still deprived of education. Besides, many of those who get enrolled cannot complete the primary education cycle. In order to cope with these problems, the government has initiated some programs in light of the National Education Policy 2010. In spite of such interventions, a huge number of 6-10 year old children (particularly from poor sections of the population) continue to remain out of school. To bring these backward and disadvantaged children into the mainstream of education, the government is now implementing some special schemes such as Reaching Out-of-School Children (ROSC) project which was aimed at providing primary education opportunities to the dropouts and out-of-school children from the remote and backward areas of the country. The Directorate of Primary Education (DPE) of the Ministry of Primary and Mass Education (MoPME) was responsible for implementing the project. ROSC created an opportunity for 7, 80,000 children to complete their primary education cycle through 23,000 learning centers (LCs) which are known as Ananda Schools (Schools of Joy). Every LC is meant to provide non-formal education to 35 children through one teacher. 5. Objectives of the Project a) To create primary education opportunities for disadvantaged, out-of-school children by setting up LCs; b) To improve the quality and efficiency of primary education; and c) To strengthen the relevant institutions and increase their efficiency. 6. Approval of the Project The ROSC project was approved on 6 May, 2004 in order to ensure educational opportunities for the disadvantaged, out-of-school children as part of the Education for All initiative. The estimated cost of the project was Tk. 3907.2 million and its duration was from July 2004 to June 2010. In order to deal with a discrepancy between the approved Project Proposal (PP) and the Development Grant Agreement (DGA) and the Project Appraisal Document (PAD), the project was revised on 19 January, 2006 through a Departmental Project Evaluation Committee (DPEC) meeting held in the MoPME. The cost of the revised project stood at Tk. 3830.2 million of which the GoB share was Tk. 236.8 million and the amount of project aid Tk. 3593.4 million. Based on the recommendations of a Mid-Term Review and those of various missions carried out by the development partners, the project was revised for the second time on 6 November, 2008 at a total 43 cost of Tk. 3240.9 million. The GoB share was Tk. 183.5 million and the amount of project aid Tk. 6057.4 million. A total of 4,91,416 children were enrolled in the LCs under four cohorts of the project up to 2005-8, the duration of every cohort being 5 years. At the end of the project duration only 11% learners could complete their primary education cycle, with the remaining 89% learners continuing their education in the LCs. On the other hand, the expenditure up to December, 2009 was only Tk. 3058.0 million which was not up to the expectation. Based on the recommendations of the 7th and 8th IDA Supervision Missions, the MoPME proposed to expand the project activities in 30 new upazilas and open some additional LCs in the old project upazilas in order to bring 2,50,000 more children under the project intervention. In line with this proposal the project was revised for the third time on 29 April, 2010 at an estimated cost of Tk. 6843.2 million ( GoB: Tk. 367.0 million and project aid: Tk. 6476.2 million). 7. Objective of the Project and Achievements Objective Actual Achievements The key project development 1. Under the Project, some 780,000 children who objective is to improve equitable either dropped out of school or were never access, retention and completion in enrolled got a second chance to education in quality primary education for out- some 23,000 Learning Centers (Ananda of-school children in selected Schools) in the remote areas of Bangladesh. under-served areas. 2. Under the Project, some 23,000 unemployed young males and females got job opportunities as teachers and were empowered to participate in the decision making processes in their respective families. 3. Some 98% of the chairpersons of the Ananda Schools have been elected from the female parents and they have run the schools efficiently. 4. Financial assistance has been provided to 74 schools run by Shishu Kalyan Trust and to some 20,000 students in those schools. 5. A total of 252 students have been provided vocational training in vocational education centers established with help from UCEP in Dhaka 8. Areas Visited An Inter-ministerial Evaluation Committee formed to evaluate the ROSC Project visited 19 schools selected through random sampling from five divisions of the country. They included seven schools in Kamalgonj Upazila of Moulovibazar district under Sylhet division, three schools in Ukhia Upazila of Cox’s Bazar district under Chittagong division, two schools in Sadar Upazila of Bogra district under Rajshahi division, five schools of Sadar Upazila of Thakurgaon district under Rangpur division, and two schools in Char Fashion Upazila of Barishal division. 44 9. Strengths of the Ananda Schools under ROSC Project  Ananda School students have completed five-year primary education cycle. Some of these students were never enrolled in schools and ROSC Project has given them the opportunity which they wouldn’t have had otherwise.  Ananda Schools have created employment for teachers which have given them the opportunity to earn their livelihood.  Schools are being run with direct participation of the local communities. Women constitute 75% of the teachers of Ananda Schools, who play an important role also in the society and their respective families.  Ananda Schools have been particularly effective in the socio-economic contexts of the poorer areas, among the poor people in the tea estates and the remote areas of the country.  The Project has played an important role in reducing child marriage, dowry system and child labor. 10. Challenges faced in implementation of ROSC As evident from the school visits, Ananda Schools were facing a number of challenges that need to be addressed with due attention in any new intervention. One significant challenge was dropouts from Ananda Schools before completion of primary cycle. For example, some 36% of the students in Kamalgonj Upazila did not participate in the Grade V terminal examinations. In some schools, the names of the replaced students were not properly recorded resulting in mismatch between the ID profiles and some of the current students. Some schools did not strictly follow the criteria regarding age of children admitted to the Ananda Schools. This has also affected the school performance. Besides, one school in Ukhia Upazila had seven students allegedly enrolled also in the nearest primary school. Monitoring from the master trainers and head teachers from the nearest primary schools was reportedly weak. In general, monitoring under ROSC has been a significant challenge. Although CMCs are at the heart of ROSC project, they have not been found equally active in all schools. This might have resulted in lack of supervision with regard to class room and school environment. Some schools did not have quality teaching and learning materials. Students in some cases did not receive school uniform. There was an instance of classes being held in the drawing room of a teacher. Some schools did not have proper toilet facilities. While some of these problems emanated from practical constraints, many of them could be resolved with active intervention from CMCs, government officials and ESPs. School performance has also been hindered because of the quality of teachers recruited in some upazilas. Training of teachers has also been an issue. Teachers were found particularly weak in Mathematics and English which might have also been reflected in the performance of the schools they ran. A major focus should be on recruitment of qualified teachers and providing them with extensive training under any new intervention. Some administrative weaknesses also need to be corrected. For example, in Ukhia upazila, students and teachers of 11 schools did not receive any salary and allowance for 16 months. This affects the morale of teachers, school communities and students. 11. Overall observation Under the ROSC Approach, students participate in the Grade V terminal examinations after completion of a five-year primary education cycle. ROSC Learning Centers use the textbooks prepared by NCTB. Timing of school is flexible and can be decided on the basis of convenience 45 of the children who might have dropped out of formal schools because of the problem of school timing. The approach is premised upon the idea of provision of formal education through nonformal approach. Under the present design, student can complete primary education cycle in five years and enter the mainstream for secondary education. There is no problem of equivalencies as in the case of other nonformal education systems. 12. Recommendations  In order to ensure attendance and participation of students in terminal examinations, it is important that adequate measures be taken to create awareness among parents and school communities through appropriate communication activities.  To cleanse the anomalies in student record, ID profiles need to be updated. While it is important to have a central database of the student, it would also be useful to have local data base accessible to the actors in the upazilas.  It is extremely important that the quality of the teachers of Ananda Schools be enhanced. Since ROSC follows the one-teacher-one school model, it is not unlikely that a teacher may not be proficient in all subjects. Therefore, it is imperative that the teachers undergo high- quality intensive training over an extended time, particularly on subjects like mathematics and English. It would also be useful to provide the teachers with better salaries and allowance so that better teachers can be attracted to the job.  The project authority needs to put in place strict monitoring to ensure that under-aged children are not admitted to Ananda Schools. There should be an effort to bring the children back to the mainstream education. The project authority needs to create this awareness among Upazila Education Officers and parents.  The Upazila Education Officers need to take appropriate actions to ensure the resources allocated for teaching learning materials are properly used. It would be helpful for students to have a six-inch high small table, particularly when they write something. The jute mats used in the class room can be replaced with plastic mats which are more durable. The project authority can consider having an allocation of these items.  The project or the Upazila Education Office should directly deliver the school uniforms. There should also be some allocations for maintaining cleanliness of the schools.  It would be useful to decide on exact criteria for school house and strictly follow them. Proper water and sanitation facilities should also be ensured.  Measures need to be taken to ensure disbursement of salaries of teachers and student allowance in due time.  Regular CMC meetings and other important meetings should be held and participation of the head teacher of the nearest primary school, union council member, and upazila education officer should be ensured.  Strict monitoring should be in place in conformity with the ROSC criteria to ensure the student allowances reach the right students. 46  School visits and monitoring need to be considerably strengthened involving the head teachers of the nearest primary schools, Upazila Education Officers and other relevant officials.  School hours under ROSC Project are different in different areas. It might be necessary to reorganize the schedules in keeping with the school hours in the primary schools of the respective areas.  In the light of the experiences in implementing this Project, it can be said that ROSC has proved to be a tested and acceptable model in bringing the children from disadvantaged background back to school and to the mainstream and providing them with a second chance to education. In view of this, it is recommended that the ROSC model be followed in providing second chance education to the children of other areas of the country with similar characteristics. 47 Annex 6. List of Supporting Documents Government of Bangladesh Ministry of Primary and Mass Education: (1) Education for All: National Plan of Action II: 2003-2015. Ministry of Primary and Mass Education, Government of the People’s Republic of Bangladesh, May 2003. (2) Primary Education Situational analysis, Strategies and Action Plan for Mainstreaming Tribal Children. Second Primary Education Development Program PEDP II, August 2006. Ministry of Finance: (1) Interim Poverty Reduction Strategy Paper, March 2003. (2) “Minutes of the Appraisal Wrap-up Meeting for Additional Financing of the ‘Reaching Out-of-School Children Project,’ Ministry of Finance, Economic Relations Division, No. ERD/IDA-2/ROSCP/03(Part-5)/09/27, dated February 22, 2010. (3) “Amendment of the Development Grant Agreement (DGA) of Reaching out of School Children Project (IDA Grant # H 102-BD),” Ministry of Finance, Economic Relations Division, No. 09.222.024.02.08.015.2009-10 dated 30 January 2011. Bangladesh Bureau of Educational Information and Statistics: (1) Bangladesh Education Statistics. BANBEIS Publication No. 404. Dhaka, 2011. The World Bank (1) Country Assistance Strategy for the People’s Republic of Bangladesh, dated February 8, 2001. Report No. 21326-DB. (2) Interim Poverty Reduction Support Program 2003. (3) Project Appraisal Document for a Reaching Out-of-School Children Project, dated May 19, 2004. Report No. 29019-BD. (4) Development Grant Agreement (Reaching Out-of-School Children Project) between People’s Republic of Bangladesh and International Development Association, dated June 30, 2004. Grant Number H102-BD. (5) Project Paper on a Proposed Additional Credit in the Amount of SDR 23.1 Million (US$35 million equivalent) to The People’s Republic of Bangladesh for the Reaching Out-of-School Children Project, Report No. 53926-BD, April 8, 2010. (6) Financing Agreement (Additional Financing for the Reaching Out-of-School Children Project) between People’s Republic of Bangladesh and International Development Association, dated June 7, 2010. Grant Number 4719-BD. (7) “Bangladesh: Reaching Out-of-School Children Project (Grant H102-BD) Extension of Closing Date and Amendment to the Development Grant Agreement, letter from the World Bank Acting Country Director dated June 16, 2010. (8) Restructuring Paper on a Proposed Project Restructuring of Reaching Out-of- School Children Project (Grant H-1020-BS) June 17, 2004 to the People’s Republic of Bangladesh, February 22, 2011. 48 Additional Project Documents (1) “Minutes of Negotiations between the People’s Republic of Bangladesh and the International Development Association, “Additional Financing for the Reaching Out-of- School Children Project, April 7, 2010. (2) Report on Post Review of Procurement Contracts [2004-2007] Reaching Out-of-School Children (ROSC) Project prepared by A. Qasem & Col. Chartered Accountants, August 2007. Research & Evaluation Reports (1) AcNeilson report on ESP performance 2005-2007 (2) Ahmed, A. U. (2006) Evaluating the Reaching Out-of-School Children Project in Bangladesh: A Baseline Study, International Food Policy Research Institute, Washington, D.C. in collaboration with the Data Analysis and Technical Assistance, Ltd. , Dhaka, Bangladesh. (3) Dang, H-A, Sarr, L. and Asadullah, N. (2011). “School Access, Resources, and Learning Outcomes: Evidence from a Non-formal School Program in Bangladesh. IZA DP No. 5659, Discussion Paper Series, Institute for the Study of Labor, Bonn, Germany. (4) HB Consultants, Ltd. (2007) Tracer Study for the Graduate Children of ROSC Learning Centers (LCs) and Shishu Kallyan Trust (SKT) Under the Reaching Out-of-School Children (ROSC) Project. Dhaka, Bangladesh. (5) Uniconsult International, Ltd. (2011). Final Report on ‘Quantitative Evaluation of Reaching Out-of-School Children (ROSC) Project (Package: S-8-2)’. Dhaka, Bangladesh. (6) Pathmark. Quality Study on ROSC, 2010. (7) RESPONSE. Non-formal Primary Education Performance Study. (8) Sarr, L.; Dang, H-A; Chaudhury, N.; Parajuli, D.; and Asadullah, N. Reaching Out-f- School Children Project (ROSC): Evaluation Report. Version 3.4 November 30, 2010. Human Development Sector, South Asia Region, The World Bank, Washington, D.C. 49 IBRD 33368R To Gangtok 88ºE 89ºE BHUTAN 90ºE 91ºE B A N GL A D ESH DISTRICT CAPITALS DIVISION CAPITALS To Dispur Panchagar NATIONAL CAPITAL To PANCHAGAR RIVERS Patna BANGLADESH MAIN ROADS LA 26ºN Thakurgaon LM NILPHAMARI RAILROADS O N THAKURGAON Nilphamari Lalmonirhat IR HA DISTRICT BOUNDARIES T Kurigram Rangpur DIVISION BOUNDARIES To KURIGRAM Katihar Dinajpur RANGPUR INTERNATIONAL BOUNDARIES DINAJPUR To Goalpara To I N DI A Katihar Gaibandha To 92ºE To Dispur Dispur GAIBANDHA JOYPURHAT SERPUR Joypurhat Sunamganj SYLHET 25ºN Serpur 25ºN NAOGAON BOGRA Jamalpur NETROKONA SUNAMGANJ Sylhet Naogaon Bogra JAMALPUR Netrokona RAJSHAHI Mymensingh SYLHET NOWABGANJ To Silchar MYMENSINGH J Jaamuna Nowabganj RAJSHAHI DHAKA Kishorganj MOULVI BAZAR na Serajganj HABIGANJ Moulvi Bazar G an Rajshahi Natore SERAJGANJ KISHORGANJ ge NATORE TANGAIL s Habiganj Tangail GAZIPUR PABNA Pabna Gazipur NARSINGDI I N D IA 24ºN a 24ºN gh n Narsingdi Brahmanbaria Me I NDI A KUSHTIA Kushtia Manikanj DHAKA BRAHMAN BARIA Meherpur Rajbari MANIKGANJ NARAYNGANJ MEHERPUR Chuadanga RAJBARI DHAKA Naraynganj Faridpur COMILLA CHUADANGA Jhenaidah Magura MUNSHIGANJ FARIDPUR G a n geMunshiganj JHENAIDAH s MAGURA Comilla HARI To SARIATPUR CHANDPUR RACH Calcutta Sariatpur Jessore Narail Madaripur Chandpur Khagrachhari KHAG MADARIPUR NARAIL JESSORE GOPALGANJ Gopalganj Feni 23ºN 23ºN KHULNA LUXMIPUR Luxmipur NOAKHALI FENI Khulna BARISAL Noakhali RANGAMATI Satkhira KHULNA Bagerhat PEROJPUR Jhalukathi Barisal Bhola C H I T TA G O N G SATKHIRA JHALUKATHI Rangamati Kaptai To Calcutta BAGERHAT Perojpur Lake CHITTAGONG BARISAL Patuakhali BHOLA a D e l t PATUAKHALI Chittagong s G a n g e Barguna BARGUNA Bandarban es 22ºN 22ºN Mt. Mowdok ns ng (957 m) a r ba G a d Sun the BANDARBAN of COX’S ths BAZAR Mou Cox's Bazar 0 10 20 30 40 50 Kilometers Bay of Bengal 0 10 20 30 40 50 Miles MYANMAR This map was produced by the Map Design Unit of The World Bank. 21ºN The boundaries, colors, denominations and any other information 21ºN shown on this map do not imply, on the part of The World Bank Group, any judgment on the legal status of any territory, or any endorsement or acceptance of such boundaries. To 89ºE 90ºE 91ºE 92ºE Sittwe MARCH 2008