Document of The World Bank Report No: ICR1552 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-38860 IDA-H0840 IDA-H5410 TF-92092) ON A CREDIT IN THE AMOUNT OF SDR 3.9 MILLION (US$5.3 MILLION EQUIVALENT) AND GRANT IN THE AMOUNT OF SDR 5.6 MILLION (US$7.7 MILLION EQUIVALENT) ON A TRUST FUND FINANCING BY THE AUSTRALIAN GOVERNMENT IN THE AMOUNT OF AUD 3,276,639 (US$2,880,657 EQUIVALENT) AND AN ADDITIONAL FINANCING GRANT IN THE AMOUNT OF SDR 9.7 MILLION (US$15.5 MILLION EQUIVALENT) TO THE LAO PEOPLE’S DEMOCRATIC REPUBLIC FOR A SECOND EDUCATION DEVELOPMENT PROJECT February 28, 2014 East Asia and Pacific/Education EASHE East Asia and Pacific Region CURRENCY EQUIVALENTS (Exchange Rate Effective February 28, 2014) Currency Unit = Kip Kip 1,000.00 = US$0.125 US$1.00 = Kip 8,022 FISCAL YEAR October 1 – September 30 ABBREVIATIONS AND ACRONYMS APL Adaptable Program Loan IMF International Monetary Fund AusAI Australian Agency for International INT World Banks Institutional Integrity D Development Unit AUS Australian Dollar MDGs Millennium Development Goals ASLO Assessment of Student Learning MoES Ministry of Education and Sports BEGP Basic Education for Girls’ Project MoF Ministry of Finance CAS Country Assistance Strategy MTR Mid-term Review CBC Community Based Contracting NPEP National Poverty Eradication Program CG Community Grants NSEDP National Social and Economic Development Plan CPS Country Partnership Strategy NUOL National University of Laos DESB District Education and Sports Bureau PDO Project Development Objective DPC Department of Planning and PES Provincial Education Service Cooperation ECU ESDF Coordination Unit PPF Project Preparation Facility EGDP Ethnic Group Development Plan PTA Parent Teacher Association EGRA Early Grade Reading Assessment QER Quality Enhancement Review EMIS Education Management Information QSA Quality of Supervision Assessment System FM Financial Management REIS Research Institute for Education Science GDP Gross Domestic Product SBM School Based Management IDA International Development VEDC Village Education Development Association Committee Vice President: Axel van Trotsenburg Country Director: Ulrich Zachau Sector Manager: Luis Benveniste Project Team Leader: Boun Oum Inthaxoum ICR Team Leader: Boun Oum Inthaxoum ii LAO PEOPLE’S DEMOCRATIC REPUBLIC Second Education Development Project CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 5 3. Assessment of Outcomes .......................................................................................... 13 4. Assessment of Risk to Development Outcome......................................................... 19 5. Assessment of Bank and Borrower Performance ..................................................... 19 6. Lessons Learned ....................................................................................................... 21 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 22 Annex 1. Project Costs and Financing .......................................................................... 23 Annex 2. Outputs by Component ................................................................................. 24 Annex 3. Economic and Financial Analysis ................................................................. 31 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 40 Annex 5. Beneficiary Survey Results ........................................................................... 42 Annex 6. Stakeholder Workshop Report and Results................................................... 43 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 44 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 56 Annex 9. List of Supporting Documents ...................................................................... 57 Map ............................................................................................................................... 58 iii A. Basic Information Lao People's Second Education Country: Project Name: Democratic Republic Development Project IDA-38860,IDA- Project ID: P078113 L/C/TF Number(s): H0840,IDA-H5410,TF- 92092 ICR Date: 02/27/2014 ICR Type: Core ICR Lending Instrument: SIL Borrower: LAO PDR Original Total XDR 9.50M Disbursed Amount: XDR 17.83M Commitment: Revised Amount: XDR 19.20M Environmental Category: C Implementing Agencies: Ministry of Education and Sports Cofinanciers and Other External Partners: Australian Agency for International Development (AusAID) B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 05/23/2002 Effectiveness: 01/24/2005 01/24/2005 05/15/2008 Appraisal: 05/05/2003 Restructuring(s): 07/09/2009 01/12/2010 Approval: 04/29/2004 Mid-term Review: 04/30/2007 04/06/2007 Closing: 02/28/2010 08/31/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Moderate Bank Performance: Satisfactory Borrower Performance: Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Satisfactory Government: Satisfactory Implementing Quality of Supervision: Satisfactory Satisfactory Agency/Agencies: Overall Bank Overall Borrower Satisfactory Satisfactory Performance: Performance: iv C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any Quality of No Satisfactory time (Yes/No): Supervision (QSA): DO rating before Moderately Closing/Inactive status: Satisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 35 35 Primary education 54 54 Tertiary education 11 11 Theme Code (as % of total Bank financing) Education for all 67 67 Participation and civic engagement 33 33 E. Bank Staff Positions At ICR At Approval Vice President: Axel van Trotsenburg Jemal-ud-din Kassum Country Director: Ulrich Zachau Ian C. Porter Sector Manager: Xiaoqing Yu Emmanuel Y. Jimenez Project Team Leader: Boun Oum Inthaxoum Rosemary T. Bellew ICR Team Leader: Boun Oum Inthaxoum ICR Primary Author: Sandra F. Beemer Franco Russo F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The objective of the project is primary enrollment and completion increased in the nineteen poorest districts in the six poorest provinces, quality improved, and capacities strengthened to develop policies and strategies to monitor and manage primary education. v Revised Project Development Objectives (as approved by original approving authority) The objective was not revised during the life of the project. (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Indicator 1 : Primary completion rate for entire country (Core Indicator) Value 92.8% quantitative or 64% 76% Souce: EMIS Qualitative) Date achieved 12/09/2009 02/28/2010 08/31/2013 Comments (incl. % Target exceeded. EMIS data for academic year 2012 -2013 achievement) Indicator 2 : Gender parity index for entire country (Core Indicator) Value 0.95 quantitative or 0.88 0.96 Source: EMIS Qualitative) Date achieved 12/09/2009 02/28/2010 08/31/2013 Comments Target met. The entire country target missed by .01 percent however, the gender (incl. % parity index for project districts is 0.96 which are the poorest districts in Lao achievement) PDR. This is a substantial achievement. Indicator 3 : System for learning assessment at the primary level (IDA rating scale) Value 3 quantitative or 2 3 Source: ECU Qualitative) Date achieved 02/28/2004 02/28/2010 08/31/2013 Comments Target met. The MoES has a learning assessment system in place had has (incl. % completed 2 grade 5 assessments. In addition, MoES is planning a grade 3 achievement) assessment. Increased number of classroom built and rehabilitated under the project as a Indicator 4 : percentage of total number of required classrooms at primary level in target districts. (Core Indicator) Value 102% quantitative or 55% 100% Source: ECU Qualitative) Date achieved 12/09/2009 02/28/2010 08/31/2013 Comments Target met. Although the total number of required classroom is not known, the (incl. % project did build 2 percent more classrooms than planned at the time of the achievement) additional financing. Increased number of teachers received training in multi-grade teaching as a Indicator 5 : percentage of the total number of primary teachers in the target districts and schools. (Core Indicator) Value 100% 25% 47% quantitative or Source: ECU vi Qualitative) Date achieved 12/09/2009 02/28/2010 08/31/2013 Comments (incl. % Target met. achievement) Indicator 6 : Lower Construction Costs (cost benchmarked against other projects) Value US$144/m2 quantitative or US$54/m2 US$110/m2 Source: ECU Qualitative) Date achieved 02/28/2004 02/28/2010 08/31/2013 Comments Target met. EDPII construction costs compared to other costs in Laos are (incl. % between ¼ and ½ less than the cost using traditional, centralized construction achievement) contracts. See Table 4 in Annex 3 for details. Indicator 7 : Increased enrollment in target areas Value 104,205 quantitative or 72,700 95,000 F 47.5% Qualitative) Source: EMIS Date achieved 02/28/2004 02/28/2010 08/31/2013 Comments (incl. % Target met. achievement) Indicator 8 : All teachers in project schools trained in multi-grade methods Value 843 quantitative or 0 540 Source: ECU Qualitative) Date achieved 02/28/2004 02/28/2010 08/31/2013 Comments (incl. % Target met. achievement) Indicator 9 : Improved learning outcomes in rural and ethnic groups areas. Second round results published ASLO results Value including published quantitative or N/A breakdown by Source: ASLO Qualitative) ethnic groups and report gender. Date achieved 02/28/2004 02/28/2010 08/31/2013 Comments Two rounds of grade 5 assessments completed (2006 and 2009) and ASLO (incl. % published the results. Learning improvement is mixed for skills levels. However, achievement) assessment institutionalized in MoES. Indicator 10 : Policies and strategies increasingly made on basis of information Many policy papers Value Published policy completed and quantitative or N/A studies and MoES results published. Qualitative) Strategy Papers Source: ECU Date achieved 02/28/2004 02/28/2010 08/31/2013 Comments Target met. Ethnic Group Development Plan, research reports on education (incl. % financing, grade repetition, economic relevance, impact evaluation of community vii achievement) based intervention, and learning assessment. All being used to inform MoES policy decisions. Indicator 11 : Increased education spending of total government expenditure Value 18.6% quantitative or 10% 15% Source: EMIS Qualitative) Date achieved 02/28/2004 02/28/2010 08/31/2013 Comments (incl. % Target met. achievement) Indicator 12 : Increased recurrent capital expenditure Value 74:26 quantitative or 40:60 70:30 Source: EMIS Qualitative) Date achieved 02/28/2004 02/28/2010 08/31/2013 Comments (incl. % Target met. achievement) Indicator 13 : Net enrollment ratio Value quantitative or 50% 70% Qualitative) Date achieved 02/28/2004 02/28/2010 Comments Indicator was dropped during 2007 mid-term review and formalized during the (incl. % 2009 restructuring. achievement) Indicator 14 : Increased primary completion rate in target districts. Value quantitative or N/A N/A Qualitative) Date achieved 02/28/2004 02/28/2010 Comments Indicator was dropped during 2007 mid-term review and formalized during the (incl. % 2009 restructuring. achievement) Indicator 15 : Reduced repetition in grade 1 and 2 in target districts Value quantitative or N/A N/A Qualitative) Date achieved 02/28/2004 02/28/2010 Comments Indicator was dropped during 2007 mid-term review and formalized during the (incl. % 2009 restructuring. achievement) Reduced gap in new enrollment rates between target districts and national Indicator 16 : average. Value quantitative or N/A N/A Qualitative) viii Date achieved 02/28/2004 02/28/2010 Comments Indicator was dropped during 2007 mid-term review and formalized during the (incl. % 2009 restructuring. achievement) Indicator 17 : Contract teachers upgraded to qualified status in project villages. Value quantitative or N/A N/A Qualitative) Date achieved 02/28/2004 02/28/2010 Comments Indicator was dropped during 2007 mid-term review and formalized during the (incl. % 2009 restructuring. achievement) Indicator 18 : Student:Textbook ratio at 1:1 in rural areas. Value quantitative or N/A N/A Qualitative) Date achieved 02/28/2004 02/28/2010 Comments Indicator was dropped during 2007 mid-term review and formalized during the (incl. % 2009 restructuring achievement) Indicator 19 : Increased expenditure on primary education. Value quantitative or N/A N/A Qualitative) Date achieved 02/28/2004 02/28/2010 Comments Indicator was dropped during 2007 mid-term review and formalized during the (incl. % 2009 restructuring achievement) (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Indicator 1 : Incomplete dilapidated primary schools replaced to accommodate 5 grades. Value 781 (quantitative 0 400 714 Source: ECU or Qualitative) Date achieved 01/24/2004 02/28/2010 08/31/2013 08/31/2013 Comments Target exceeded. The target number of schools increased from a target of 714 (incl. % schools (2,409 classrooms) to 781 schools (2,462 classrooms). achievement) Indicator 2 : Community Based Contracting (CBC) grants distributed equitably 82.4% in minority Value 50% minority 0% in minority villages villages; (quantitative villages 0% off the road 60.6% off road or Qualitative) 50% off road Source: ECU ix Date achieved 01/24/2004 02/28/2010 08/31/2013 Comments (incl. % Target exceeded. achievement) Indicator 3 : Community Grants (CG) awarded to CBC villages annually (cumulative) Value 714 villages 781 villages and 400 villages and (quantitative N/A and 1,893 grant 1,469 grants or Qualitative) 2,142 grants Source: ECU Date achieved 02/28/2004 02/28/2010 08/31/2013 08/31/2013 Target not met. A total of 1,893 CGs to 781 villages were provided by project Comments closing. The intended target of 2,142 CGs was missed due to some schools only (incl. % receiving 2 rounds of CGs by project closing because of initial construction achievement) delays. Indicator 4 : Education Management Information System strengthened Value Statistics Statistics available (quantitative EMIS Database generated Yearbook online or Qualitative) published Source: ECU Date achieved 02/28/2010 02/28/2010 08/31/2013 Comments Target met. A yearbook was published for 2008-2009 academic year. All (incl. % subsequent years are now available online. achievement) Indicator 5 : Primary Textbooks procured and delivered to schools Value 3,087,000 (quantitative N/A 2,240,000 Source: ECU or Qualitative) Date achieved 02/28/2004 02/28/2010 08/31/2013 Comments (incl. % Target exceeded. achievement) Indicator 6 : Teachers guides procured and delivered to schools Value 300,000 (quantitative N/A 196,000 Source: ECU or Qualitative) Date achieved 02/28/2004 02/28/2010 08/31/2013 Comments (incl. % Target exceeded. achievement) Indicator 7 : Program on Education Administration and Management established Lecturers trained Value Lecturers trained 7 intakes with 295 (quantitative No program existed and first year of graduates or Qualitative) second class Source: ECU Date achieved 02/28/2004 02/28/2010 08/31/2013 Comments Target met. Program was established, lecturers trained and 7 intakes of students (incl. % completed. Program continues and project conducted a tracer study of the first achievement) intake group. Indicator 8 : Participatory Beneficiary Assessment x Assessment was Value Final Beneficiary completed in 2009 (quantitative N/A Assessment and published. or Qualitative) Source: ECU Date achieved 02/28/2004 02/28/2010 08/31/2013 Comments (incl. % Target met. achievement) Indicator 9 : Village Development Committees (VEDCs) Established Value 781 (quantitative 0 400 714 Source: ECU or Qualitative) Date achieved 02/28/2004 02/28/2010 08/31/2013 08/31/2013 Comments (incl. % Target exceeded. achievement) Indicator 10 : Village committees include representation of all minority groups and women Value 100% (quantitative N/A 100% Source: ECU or Qualitative) Date achieved 02/28/2004 02/28/2010 08/31/2013 Comments Target met. All VEDCs have at least one women and a representative from each (incl. % ethnic group. achievement) Indicator 11 : Cost of education of parents decreased Value (quantitative N/A N/A or Qualitative) Date achieved 02/28/2004 02/28/2010 Comments Indicator was dropped during 2007 mid-term review and formalized during the (incl. % 2009 restructuring achievement) Indicator 12 : Each class has a qualified teacher in project schools Value (quantitative N/A N/A or Qualitative) Date achieved 02/28/2004 02/28/2010 Comments Indicator was dropped during 2007 mid-term review and formalized during the (incl. % 2009 restructuring achievement) Indicator 13 : Textbooks evaluated for ethnic inclusion & revised Value (quantitative N/A N/A or Qualitative) Date achieved 02/28/2004 02/28/2010 Comments Indicator was dropped during 2007 mid-term review and formalized during the (incl. % 2009 restructuring achievement) xi Indicator 14 : Outside expertise utilized for book evaluation and revision Value (quantitative N/A N/A or Qualitative) Date achieved 02/28/2004 02/28/2010 Comments Indicator was dropped during 2007 mid-term review and formalized during the (incl. % 2009 restructuring achievement) Indicator 15 : Policy Analysis Division established and fully operational Value (quantitative N/A N/A or Qualitative) Date achieved 02/28/2004 02/28/2010 Comments Indicator was dropped during 2007 mid-term review and formalized during the (incl. % 2009 restructuring achievement) Indicator 16 : Increased participation of experts in ethnic minority affairs Value (quantitative N/A N/A or Qualitative) Date achieved 02/28/2004 02/28/2010 Comments Indicator was dropped during 2007 mid-term review and formalized during the (incl. % 2009 restructuring achievement) Indicator 17 : GIS established and producing reports by districts Value (quantitative N/A N/A or Qualitative) Date achieved 02/28/2004 02/28/2010 Comments Indicator was dropped during 2007 mid-term review and formalized during the (incl. % 2009 restructuring achievement) Indicator 18 : Evaluation of transparency and equity of CBC and CG procedures Value (quantitative N/A N/A or Qualitative) Date achieved 02/28/2004 02/28/2010 Comments Indicator was dropped during 2007 mid-term review and formalized during the (incl. % 2009 restructuring achievement) Indicator 19 : Complaints mechanism effective Value (quantitative N/A N/A or Qualitative) Date achieved 02/28/2004 02/28/2010 Comments Indicator was dropped during 2007 mid-term review and formalized during the (incl. % 2009 restructuring xii achievement) Indicator 20 : VEDCs meet on a regular basis Value (quantitative N/A N/A or Qualitative) Date achieved 02/28/2004 02/28/2010 Comments Indicator was dropped during 2007 mid-term review and formalized during the (incl. % 2009 restructuring achievement) Indicator 21 : Villages participating in CBC receive support visits from district engineers Value (quantitative N/A N/A or Qualitative) Date achieved 02/28/2004 02/28/2010 Comments Indicator was dropped during 2007 mid-term review and formalized during the (incl. % 2009 restructuring achievement) G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (US$ millions) 1 06/22/2004 Satisfactory Satisfactory 0.00 2 10/20/2004 Satisfactory Satisfactory 0.00 3 01/31/2005 Satisfactory Satisfactory 0.00 4 08/11/2005 Satisfactory Satisfactory 0.63 5 03/15/2006 Satisfactory Satisfactory 1.61 6 10/06/2006 Satisfactory Satisfactory 2.99 7 04/23/2007 Satisfactory Satisfactory 5.26 8 11/29/2007 Satisfactory Satisfactory 7.94 9 05/20/2008 Satisfactory Satisfactory 9.62 10 11/25/2008 Satisfactory Satisfactory 11.54 11 05/21/2009 Satisfactory Satisfactory 12.40 12 03/25/2010 Satisfactory Satisfactory 13.23 13 04/04/2011 Satisfactory Moderately Satisfactory 14.08 14 01/29/2012 Satisfactory Moderately Satisfactory 19.32 15 06/17/2012 Satisfactory Satisfactory 21.68 16 12/19/2012 Satisfactory Satisfactory 23.94 17 06/13/2013 Moderately Satisfactory Moderately Satisfactory 25.17 18 08/26/2013 Moderately Satisfactory Moderately Satisfactory 27.37 xiii H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in US$ millions AusAID provided addtional 05/15/2008 S S 9.62 funding for EDPII project 07/09/2009 N S S 12.49 AusAID to up funding provided 01/12/2010 N S S 13.16 IDA additional financing grant I. Disbursement Profile 1 1 A glitch in the system did not capture the AusAID contribution as part of the additional financing or actual disbursements. Therefore, the data sheet and disbursement graph only reflect IDA financing. Furthermore, the formally revised disbursement estimates were entered into the Portal but not recorded by the system in the disbursement graph. xiv xv 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. Country Context. In 1997, the Lao People's Democratic Republic (Lao PDR) had an estimated per capita income of US$400 and was one of the poorest and least developed countries in the East Asia region. Although public expenditures on education had increased from 1.6 percent of Gross Domestic Product (GDP) in 1991 to 2.8 percent in 1997, they still remained among the lowest in Asia, were heavily dependent upon donor financing, and their impact was often limited. At the time of appraisal, a main goal of the Government of Lao PDR (GoL PDR) was to graduate from least developed country status by 2020. To achieve this goal, the government developed the National Poverty Eradication Program (NPEP), which aimed to stabilize the economy, support improvements in basic services, maintenance and capacity, while strengthening partnerships. The NPEP identified 72 districts as “poor”, of which 47 were identified for priority investments. The NPEP goals were also the goals of the World Bank Country Assistance Strategy (CAS) 2000-2002. At the time, the CAS linked poverty alleviation with economic growth, social and cultural development, and participation through decentralization. It aimed to reduce income gaps through more equitable access to resources, public goods and services. In education, the CAS highlighted inequitable access and a high degree of inefficiency due to lack of complete schools, trained teachers, and materials as hindering factors to growth. 2. Sector Context. The Ministry of Education and Sports (MoES) Strategic Vision up to 2020 was expressed in the Five Year Development Plan (2000-2005), as well as in the NPEP. Priorities in the Vision included universalization of compulsory primary education, abolition of illiteracy, expansion of vocational and higher education, development of science and technology, gradually raising standards to international levels, and improving management. Cross cutting issues (and strategies) include equitable access (infrastructure, private sector development, programs targeting disadvantaged groups), quality and relevance (learning materials, teacher training, curricula, linkages with private sector for vocational and technical education), and decentralization (training). In education, the NPEP prioritizes achievement of universal completion of quality primary education for the poorest. It planned to increase education’s share of the public budget to 15 percent by 2005/06 and the share for primary education to 50 percent, and to improve the predictability of salary payments. A Joint International Development Agency (IDA) – International Monetary Fund (IMF) Staff Assessment supported the GoL PDR's strategy, but stressed the need to further prioritize and cost the programs, link the programs to public expenditure policies, and make greater efforts to improve transparency in the management of public resources. In order to support Lao PDR achieve the NPEP priorities and strategies for primary education, the government requested IDA funding for the development of the Second Education Development Project (EDPII). 1 1.2 Original Project Development Objectives (PDO) and Key Indicators 3. The objective of the project was primary enrollment and completion increased in the nineteen poorest districts in the six poorest provinces, quality improved, and capacities strengthened to develop policies and strategies to monitor and manage primary education. The original key indicators selected to monitor progress toward Project Development Objective (PDO) achievement were: (i) lower construction costs; (ii) increased enrollments in target districts (number enrolled and net enrollment percentage); (iii) increased primary completion rates in target areas; (iv) reduced repetition rates in grades 1 and 2 target districts; (v) reduced gap in net enrollment rates between target districts and national average; (vi) reduced national enrollment gap between poor and non-poor households; (vii) contract teachers upgraded to qualified status in project villages; (viii) all teachers in project schools trained in multi-grade methods; (ix) student:textbook ratio at 1:1 in rural areas; (x) improved learning outcomes in rural and ethnic minority areas; (xi) policies and strategies increasingly made on the basis of information; (xii) increased education expenditure; (xiii) increased recurrent capital expenditure; and (xiv) increased expenditure on primary education. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 4. The PDO was not revised during project implementation. However, at the time of the 2010 additional financing, the PDO level indicators were modified. The project paper states that the original results framework would be modified based on the 2007 mid-term review (MTR) recommendations to reduce the number of overall indicators to 23 and that the additional financing results framework would focus on the newly introduced IDA Core Indicators. Therefore, the additional financing results framework combined the agreed 2007 MTR PDO level indicators with the new additional financing indicators for a total of 12 PDO level indicators. The combined list of PDO level indicators included: (i) primary completion rate for entire country; (ii) gender parity index for entire country; (iii) system of learning assessment; (iv) increased number of classrooms built and rehabilitate under the project as a percentage of total number of required classrooms at primary level in target districts; (v) increased number of teachers received training in multi-grade teaching as a percentage of the total number of primary teachers in the targeted districts and schools; (vi) lower construction costs; (vii) increased enrollments in target districts; (viii) all teachers in project schools trained in multi-grade methods; (ix) improved learning outcomes in rural and ethnic minority areas 2 ; (x) policies and strategies increasingly made on the basis of information; (xi) increased education expenditure; and (xii) increased recurrent capital expenditure. 2 There are approximately 130 different ethnic groups with four basically different language groups. Approximately 50.2 percent of the overall population are Lao. The remainder can broadly be categorized as Mon-Khmer, Hmong-Yao, Hmong-Mien, Tai-Kadai, and Sino-Tibetan and approximately 80 percent of the population in the project villages were in these four categories. 2 1.4 Main Beneficiaries 5. The main target population was villages and school aged children in the 19 poorest districts (out of the 72 NPEP designated poor districts) in six provinces (Phongsaly, Oudomxay, Louangnamtha, Houaphanh, Attapeu and Xekong) with incomplete schools and sufficient population to provide five years of education using multi-grade teaching. All school aged children nationwide benefitted from improved educational materials. Other target groups included education managers within the District Education and Sports Bureau (DESB) and Provincial Education Service (PES) as well as staff at the Ministry of Education and Sports (MoES). 1.5 Original Components 6. Component A. Increase Access and Completion of Primary Education in the Poorest Districts (US$6.5 million). This component supported three interventions to raise primary enrolment and completion rates, and reduce repetition and dropout. These were: (i) community based contracting for classroom construction (CBC) that would allow incomplete schools to gradually offer a complete five grades of primary education 3; (ii) community grants for schooling (CG) program accompanied the CBC program to lower the cost of education to communities. The CG provided resources for operations, maintenance and subsidizing the poor and scaled up the experience of the IDA supported Basic Education for Girls Project (BEGP), which targeted villages without schools in the same districts; and (iii) in-service teacher training that provided multi-grade teacher training to all teachers in participating villages and an in-service upgrading program for all untrained contract teachers to enable the expansion of access through multi-grade teaching. 7. Component B. Improve the Quality of Primary Education (US$2.10 million). This component: (i) provided textbooks and teachers’ guides aimed at ensuring that teachers and students in grades 1-5 had the required textbooks, and built greater capacity to manage the cycle of textbook production and distribution; and (ii) supported the Assessment of Student Learning Outcomes (ASLO) which aimed to establish a permanent national assessment system that would develop student assessment instruments to monitor achievement of basic competencies. 8. Component C. Strengthen Capacities for Policy Analysis and Management (US$4.2 million IDA and US$0.2 million PPF). This component had four sub- components: (i) strengthening policy and strategy development to support capacity 3 At the time of preparation, almost half of the schools in the 19 target districts operated multi-grade teaching with 24 percent of the schools having one classroom and 78 percent of the schools having two classrooms. Given the population density, the most common type of complete school would a three classroom/five grade multi-grade school. 3 building of the MoES to make information-based decisions, policy studies and dissemination on key education issues, the Ethnic Group Development Plan (EGDP) and project impact evaluation; (ii) strengthening information systems to support improved collection, analysis, reporting, filing, storing and maintenance of data and information; (iii) capacity building to strengthen capacity in management and administration; and (iv) project management and strengthened capacities in coordination, procurement and financial management. 1.6 Revised Components 9. On January 12, 2010, the Bank approved an additional financing grant in the amount of SDR 9.7 million (US$15.5 million equivalent) to scale up successful aspects of the CBC and CGs under the project. The project components were not revised but the additional financing supported Components A and C. 1.7 Other significant changes 10. On July 7, 2005 there was an agreement between IDA and GoL PDR that the financing parameters for all expenditure categories were to be set at 100 percent IDA. This meant that the US$500,000 counterpart funding included in the original project design was eliminated from the overall costs of the project, reducing the total project cost to US$13.0 million equivalent. 11. On June 15, 2007, IDA agreed to GoL PDR’s request to reallocate funds in part due to the elimination of counterpart funding (i.e. to cover the costs of activities initially financed by the government). 12. The Bank and the Government of Australia entered into an administrative agreement, signed and counter-signed on April 23, 2008 and April 29, 2008 respectively, for an AUD 2.5 million 4 contribution to EDPII. This led to the signing of a trust fund agreement between the Bank and GoL PDR on May 15, 2008 in the amount of AUD 2,336,139 to provide support for EDPII’s Component A package of school construction (127 additional schools 5), community grant scheme and teacher training. 13. On November 18, 2008, the Bank agreed to re-allocate funds of the original grant and credit to respond to adjustments in implementation progress. 14. On July 9, 2009, the Government of Australia provided additional funds to the existing trust fund in the amount of AUD 940,500 for a total grant of AUD 3,276,639 due to a depreciation of the Australian dollar to the Lao Kip. At the same time, the Letter 4 AUD 163,861 were set aside to cover Bank-related administrative fees and supervision costs. 5 After the AusAID funding was provided, the number of schools was reduced to 125 because the villages where the schools were to be constructed were resettled away from the designated construction sites. Therefore, the ICR analysis is based on AusAID target of 125 schools and villages. 4 Agreement between Lao PDR and the Bank was modified, an additional expenditure category for consultant services was added, and funds were reallocated between the three existing categories and the new category. 15. As indicated above, on January 12, 2010, the Bank approved an additional financing grant in the amount of SDR 9.7 million (US$15.5 equivalent) to continue support of the successful activities under Components A and C and extended the project closing date to August 31, 2013. The additional Financing Agreement was signed February 8, 2010 and the Grant became effective on May 8, 2010. 16. On August 10, 2010, the Bank approved the government’s request to reallocate proceeds under the original credit and additional financing grant per routine adjustments of tasks during implementation. The notification letter to GoL PDR was sent on August 20, 2010. 17. On January 23, 2013, the Bank approved the government request to reallocate proceeds to increase the amount available for sub-grants and training activities. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 18. Project Preparation. In 2002, the Lao government and IDA decided to support the EDPII with a combined IDA credit and IDA grant under the eligibility category of “IDA grant for debt vulnerable”. The original CAS proposal was to provide a 10-year Adaptable Program Loan (APL) that would deal with the NPEP issues of inequitable access and inefficiencies due to the lack of complete schools, trained teachers and materials. The 10-year APL was replaced with the EDPII design due to limited MoES capacity, knowledge base and donor coordination required to prepare an APL. In addition, it was decided to shift strategy and target IDA funding toward primary education and policy development in light of substantial needs in those areas. A choice was also made to target investments to the six poorest provinces and the 19 poorest districts within those provinces in order to concentrate resources and raise the provinces closer to the average in a shorter period of time than would be possible otherwise. Therefore, the EDPII continued to address the CAS priorities of improving primary education for the poorest. 19. The Bank preparation and appraisal teams consisted of technical experts that were appropriate for the development of the project. The preparation process included peer reviewers that provided sound guidance on the project design and there was a Quality Enhancement Review (QER) during project preparation. Project preparation was also spread over 20 months, which allowed for: (i) the full development of the project with the MoES, (ii) an agreement with MoES on the use of a project preparation facility (PPF) grant; and (iii) a dialog with other donors on project design. Once agreed, the PPF grant was particularly appropriate for this project because it allowed the MoES to help meet the project conditions for negotiations, conditions for effectiveness and to lay the foundation 5 for the full implementation of the project. The PPF grant supported the: (i) preparation of the CBC operations manual; (ii) development of CBC questionnaires aimed at collecting school data on all villages in the target districts in Phongsaly and Luangnamtha, which were the first two provinces to begin implementation of CBC and CG activities; (iii) development of a teacher upgrading program; (iv) development of procedures for setting up accounts and authorizing transfers and payments; (v) development of plans for the revision and production of textbooks and teaching materials beginning with grade 1; and (vi) hiring of consultants to assist in the implementation of the CBC, CGs, procurement and financial management, and provincial program advisors for Phongsaly and Luangnamtha. The project preparation process was thoughtful, collaborative, and technically sound. In addition, and with the support of the PPF, the preparation process gave the MoES the opportunity to be ready for a quick start-up once the project became effective. This was especially important since this was the first IDA-supported education project in Lao PDR that mainstreamed project implementation within MoES. The preparation took approximately two years during which there was a deliberative process between MoES and the Bank on how best to deliver education services at the local level. This process was important because education services have been decentralized to the local level and CBC and CG activities have been institutionalized within the education sector. 20. Project Design. The EDPII project design fit squarely into the government’s NPEP, the MoES's Strategic Vision up to 2020, as expressed in the Five Year Development Plan (2000-2005), and the Bank’s CAS 2000-2002. They shared the common objectives of providing equitable access, quality and relevance of basic education for the poorest and most disadvantaged children in Lao PDR. The project design took into consideration comments from peer reviewers during the concept and appraisal review meetings as well as the QER. In addition, the design team drew lessons from previous Bank-supported projects in Lao PDR, which informed the development of the CBC, CG and multi-grade teaching package of pilot activities. 21. The overall project design was simple; focused on access and quality in the poorest areas, and supported the government’s efforts to decentralize services to the local communities. The design also included the mainstreaming of project management within the different MoES departments both at the central, provincial and district levels. As mentioned, this was the first education project to include mainstreaming of project management and implementation, which supported the government’s move toward decentralization. The structure included: (i) Provincial Education Service and Sports (PESS) and District Education Bureaus and Sports (DEBS) implementing units (IUs) for CBC and CGs; (ii) a project management unit (ECU) in the Department of Planning (DP), which provided overall project coordination; (iii) MoES IUs that allocated staff time to implementation of project activities; and (iv) a project steering committee chaired by the Vice Minister of Education with department. This was an appropriate structure and was ultimately effective in implementing the project. 22. The project PDO was appropriate for the project design. Originally, there were 14 key indicators selected for measuring progress toward achieving the PDO and 19 6 intermediate output indicators. The intermediate indicators were included to assist the MoES with monitoring implementation of the project components since this was the first Bank-supported project where implementation was mainstreamed within the MoES. However, the results framework could have been more focused, and in some instances, could have included more realistic targets as well as details on the institutional responsibilities for updating indicators. In September 2006, a Quality of Supervision Assessment (QSA) highlighted the number of indicators and recommended modifications at the mid-term review (MTR). The project design also piloted a package of interventions (CBC, CGs and multi-grade teaching) at the local level in an effort to empower communities and improve the quality of education service delivery as well as foster ownership of the project. This package was a new approach in Lao PDR, which proved to be highly appropriate and is now being used for most education projects in Lao PDR. The design also included an impact evaluation of this package of interventions that was used by MoES and donors. The preparation team identified project risks and proposed adequate mitigation measures. 23. The 2009 additional financing (AF) did not modify the original design. The additional financing added US$15.5 million equivalent to the project to: (i) scale up the successful implementation of the CBC and CGs under Component A by adding 270 more schools with four classrooms to allow for the possible expansion of the school to cover pre-school; (ii) provide CG grants to these additional schools; (iii) offer multi-grade teaching to 700 more teacher; (iv) train 800 more VEDC members; and (v) provide continued support for capacity building within MoES for service delivery and project management under Component C. The capacity building support was particularly important for the MoES because they had made good progress with the mainstreaming of project management but also recognized the need to continue capacity building support. At the time of the AF, the Bank team took the opportunity to formally modify the results framework in line with the recommendations of the 2006 QSA and the stated modifications to the framework in the mid-term review (MTR) aide-memoire which reduced the list of indicators to 23, on the basis of relevancy, availability of data, and measurability. This action reduced the PDO level indicators and intermediate indicators and modified some of the input targets. The task team also used the opportunity to incorporate Tier 1 Bank “IDA core indicators” into the results framework. These indicators were national in scope and were to be used to monitor progress toward achievement of the Millennium Development Goals (MDGs), which the Bank team judged to be also appropriate for monitoring PDO progress of the AF. In retrospect, these indicators, while appropriate for monitoring the MDGs, were not focused on the six project provinces, making project attribution difficult. However, they proved to be useful for comparing provincial progress vis-à-vis national level progress. 2.2 Implementation 24. As mentioned above, implementation began with the PPF grant that allowed MoES to have access to an advance, which was used to lay the foundation for the full implementation of the project. The PPF was instrumental for ensuring that the government met the conditions of effectiveness and that project implementation staff and 7 consultants were in place. There was a project launch workshop that included participants from the project provinces, MoES, MoF, and the National Assembly. During the workshop, each IU identified risks for implementation of their activities with suggested mitigation measures. This project readiness led to early project disbursements of 4.4 percent, which were slightly ahead of the original projections. By 2006, implementation was fully underway with: (i) construction of the planned classrooms in the targeted villages completed; (ii) completion of the first full round of testing for ASLO; (iii) completion of the data gathering on the number of teachers to be trained and training of trainers underway; (iv) work underway on preparing for the publication and distribution of revised textbooks and teachers guides for Grade 1; (v) the launch of technical assistance contracts to support the design of the National University of Laos (NUOL) new Bachelor of Education Administration and Management Project; and (vi) the establishment of a baseline for the evaluation of the impact of the community-based development component. Financial Management (FM) was emerging as a constraint to project implementation because of slow replenishment of the special account and slow payments to village development committees (VDC, later renamed Village Education Development Committees - VEDC), national and international consultants and living allowances to long-term trainees. However, at the suggestion of the task team, a separate project accounting system was put in place to mitigate the risks, and the cumbersome government processes were streamlined, which led the Bank task team to rate FM risk as substantial even though the country risk was high. This was considerable progress in one year considering the MoES had never mainstreamed implementation of a Bank-supported project prior to the EDPII. In May 2007 there was an additional financial management issue. A national consultant (a district engineer) misused funds, which was discovered by the ECU manager who promptly informed the Bank. There was a follow up with a communication to the Bank’s Institutional Integrity (INT) unit, which resulted in the ECU’s termination of the consultant's contract and recovery of the funds. INT responded by noting this to be a minor matter and suggested possible administrative sanctions that the government could impose on the district engineer. These sanctions were carried out and by October 1, 2007, INT declared the case closed. 25. The 2007 MTR determined that the project was still relevant, the development objectives remained valid, the PDO was achievable, and disbursements were on track with 43 percent of the implementation period elapsed and 34 percent of the IDA funds disbursed. The MTR did agree to modify some project activities as follows: (i) reduce the number of schools to be constructed from the original target of 400 to 319 due to a 17 percent higher unit cost 6 than originally projected and to build more classrooms per school; (ii) reduce the number of indicators as suggested by the 2006 QSA; (iii) complete the purchase of textbooks and teacher’s guides at quantities that were 36 percent higher than planned; (iv) train all teachers in the target districts on the use of the revised textbooks and teachers guides; and (v) seek additional funds to expand the community- based targeted programs. In 2008, AusAID added additional funds to the EDPII project to 6 It should be noted that the MTR did indicate that US$62/m2 was still significantly lower than the unit cost of comparable donor programs. 8 expand the CBC, CG and teacher training activities to more schools. These activities were relevant to the support of the government’s decentralization program and AusAID wanted to support the program. 26. Since EDPII implementation was progressing well, the Lao government, in December 2009, requested US$15.5 million additional financing to scale-up community based targeted programs, which was approved by the Bank. The additional financing was to support the scaling up of the successful CBC and CG programs, expand the multi- grade teacher training and continue the capacity building of the VEDCs, DEB and PES. At that time, the project had disbursed 95 percent (approximately US$12.4 million) of the IDA funds, all time bound actions had been met, there had been full compliance will all covenants, and the public reception of the project was very positive as indicated based on an independent beneficiary assessment. 27. By the end of the project, the MoES had been able to implement all original and additionally financed activities in the project despite the remoteness of the locations. This finding was supported by a review that showed: (i) all project classrooms were completed using the new CBC; (ii) community grants were enabling communities through the VEDCs to purchase teaching and learning materials for children; (iii) teachers were trained in multi-grade teaching and the new grade 1-5 curriculum and textbooks, (iv) textbooks and teachers guides were provided for project schools and were being used; (v) two rounds of grade 5 learning assessments had been administered with results published and the experience gained being incorporated into the new grade 3 assessment; (vi) the Education Management Information System (EMIS) was strengthened; and (vii) project management was successfully mainstreamed into the MoES. 28. There were also challenges during implementation. In 2010 the project conducted a quality audit of the EDPII schools because the CBC was a new concept and the project schools were in the most remote and hardest to reach areas. The audit identified weaknesses in the system that led to construction errors or weaknesses in building quality. To resolve these issues, the Bank team hired a consultant to do an analysis of all levels of the CBC production system. The consultants made recommendations for quality assurance, quality control, and process improvements. These recommendations were taken seriously by the MoES and all have been incorporated into the CBC operation manual. In addition, the MoES put together a central quality control team that is responsible for visiting all CBC project schools and providing training for PESs and DEBs on quality control. The other challenge for the project was financial management. The delays in liquidating advances were the main factor in rating implementation progress as moderately satisfactory four times during the life of the project. The ECU has taken steps to improve this process and continues the capacity building efforts at the village, district and provincial levels. These challenges, however, did not undermine the project’s ability to complete all activities. 29. There were 18 Bank review and implementation support mission from March 2004 through August 2013. The makeup of the Bank teams was well balanced with financial management, procurement, construction, safeguards, and monitoring and 9 evaluation specialists as necessary. There was a cancellation of US$2,054,254 equivalent (approximately SDR 1,340,000) due to SDR:US$ exchange rate gains. The additional financing grant activities were all completed by August 31, 2013. There was a cancellation from the Australian trust fund of approximately US$360,800 equivalent (AUD 410,400). This was due to delays in construction of the last group of schools as indicated above. However, all the other AusAID supported activities were completed as designed. The total project disbursements in US$ equivalent were approximately US$29.87 or 92.52 percent of the combined total of approximately US$32.28 million7 equivalent. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 30. Design. The M&E design was twofold. The monitoring of the results framework and project outputs was to be done by the ECU while the impact of the community-based interventions would be assessed through an impact evaluation, with the help of technical assistance that would evaluate any changes at the school and village level of the project interventions. The impact evaluation was to utilize a control group design to estimate the effects of the CBC and CGs. As stated previously, the original results framework could have been more focused, and in some instances, could have included more realistic targets as well as details on the institutional responsibilities for updating indicators. However, the Bank team took the opportunity to modify the results framework at the time of the 2010 additional financing, taking into consideration the QSA recommendations, which was appropriate. The intermediate indicators were linked to the components and were appropriate to measure intermediate level achievements. The inclusion of the original impact evaluation was highly relevant and an important design feature for understanding the effectiveness of project interventions. 31. Implementation and Utilization. The ECU and MoES/EMIS regularly provided Bank missions with data and information that allowed teams to make judgments on progress toward achieving the PDO and the project’s overall implementation. This information was consistently reported on by the Bank in aide-memoires and Implementation Status and Results (ISRs) reports. The ECU also monitored the project outputs and used the information to make the appropriate adjustments in its annual plans and progress reports. In addition, the project supported the EMIS system, which has had an impact on monitoring and managing the education system. EMIS data is being used at the national level as well as the district level. Workshops, using EMIS data, are being carried out in every district to facilitate school plans. As a result, all districts have completed plans that identify needs based on enrollment and school data, which was extracted from EMIS and GIS. This can have long lasting effects in improving the efficiency in the use of resources, especially schools and teachers, by promoting the use of data for planning at the district level. 7 This amount includes exchange rate fluctuations as the total funds at the time of their approval amount to US$30.7 million equivalent. 10 32. The project impact evaluation was carried out during the project period as originally designed. During the analysis of the data it became evident that, as a result of the community grant and new school buildings, families sent their children to school. The value of education was increased and many village chiefs encouraged parents themselves to obtain a certificate equivalent to level five through non-formal education routes. Comments made during focus group meetings connected to the assessment also showed that parent teacher associations (PTAs) became more active. The report considered the project activities at local levels to have led to significant changes. An independent beneficiary assessment also showed that with the community grants and the construction of the new primary schools, there was increased demand for provision of a pre-school to accommodate younger siblings of primary school pupils and lower secondary school students. 2.4 Safeguard and Fiduciary Compliance 33. Procurement. Procurement was consistently rated satisfactory. Initial delays due to quality of bidding documents were addressed so that the project was able to carry out its activities. Procurement plans were prepared and routinely submitted for review by the Bank, found to be acceptable and used as a basis to judge overall implementation progress. There was initial confusion regarding which sources of funding to use for a given procurement package during the time that the project unit increased the number of projects it managed. However, this confusion was quickly dealt with and the proper resources were used to process bidding packages and award contracts. Bank supervision missions carried out ex-post reviews and also confirmed that the project was in conformity with the project’s and Bank’s procurement rules and guidelines. 34. Financial Management. Financial management remained moderately satisfactory throughout the life of the project mainly due to delays in liquidating outstanding advances, initial delays in submitting unaudited interim financial reports (IFRs), record keeping challenges and delays in flow of funds to decentralized structures. However, these challenges did not significantly impact the project’s ability to complete its activities. Furthermore, the project’s accounting system “ACCPAC” was deemed satisfactory and able to generate accurate reports, and audit reports consistently issued a clean opinion throughout the life of the project. Steps taken to increase the flow of funds and reporting included: (i) increasing the Special Account ceiling in 2007 and again in 2011; (ii) establishing an operating account at the ECU level for small expenditures; and (iii) intensive technical assistance by the Bank’s financial management team. As a result, IFR submission significantly improved, disbursements accelerated and outstanding advances being on track to be fully cleared before the end of the grace period. Going forward, ACCPAC should be expanded to allow for separate tracking of different sources of funding and IU expenditures, which would facilitate ECU implementation progress monitoring and reporting. 35. Safeguards. Environmental safeguards were rated satisfactory. The project was rated a category “C”, which did not require a separate Environmental Impact Assessment to be carried out. Instead, an environmental safeguards analysis was conducted and 11 mitigation measures included in the Operations Manual. The project provided training to district and provincial level staff in charge of ensuring adherence to safeguards guidelines, and the project was found to be in compliance. Suggestions for improvements in terms of school designs were related to reducing risk factors. There is a possibility that some building material used for classroom ceilings sold to VEDCs might contain asbestos. This was reported by the ECU based on the realization that such materials resurfaced on the market and could therefore have been sold to villages for school construction. Given that many of the building materials were unmarked, the MoES has undertaken a review of the materials used on each individual school to determine if there is documentary proof that the material is asbestos free. Thus far, no asbestos cement materials have been found. There are 68 schools where no photographic proof has been documented to show that the materials used are asbestos-free. However, MoES has launched the process of testing the ceiling materials of these schools for asbestos. Should the tests confirm the presence of asbestos, MoES will either replace the ceiling material or train the community on asbestos management. This has been deemed as a safe and practical approach by the safeguards team and the architect assisting the government with quality control. 36. No major social safeguards issues were encountered during the life of the project. Rare cases of minor land acquisition were adequately documented and compensation provided. 8 The project also developed an Ethnic Group Development Plan (EGDP) to support non-Lao speaking grade 1 students from ethnic villages, which showed positive results and led to an expansion of the number of schools being covered. Further accomplishments by the EGDP are discussed under Annex 2. The project also confirmed that female representation was adhered to with one or two women being part of each VEDC. 2.5 Post-completion Operation/Next Phase 37. The community based targeted programs have become the cornerstone of basic education for the government. The current Global Partnership for Education (GPE) project (formerly EFA-FTI) incorporated the EDPII community activities. The new IDA- supported Early Childhood Education (ECE) project, and new GPE project, both currently under preparation, have also included the community-based programs in the design. These next operations, which the Bank is working on with MoES, will build on the success of the EDPII project and will: (i) continue to rely on the cost effective CBC method for construction of project classrooms in the most disadvantaged districts; (ii) provide community grants that enable communities, through the VEDCs, to purchase teaching and learning materials for children; (iii) provide scholarships to minority girls to become teachers in their villages; (iv) provide health and disability screening for children ages 3-5 in the poorest villages; and (v) pilot school-based management (SBM) in the designated 56 disadvantaged districts. The Second Global Partnership for Education project currently under preparation will include: (i) school based management which 8 Although the last aide-memoire makes reference to a household potentially not having been compensated adequately, it has become apparent that this particular household was not a beneficiary of this project. The MoES has, however, confirmed that the land was given to the government for the construction of the school. 12 includes additional training for VEDCs; (ii) reading development interventions for Grades 1 and 2; (iii) strengthening the capacity within MoES to conduct research activities and assessment; and (iv) develop school-based financial reporting. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 38. Relevance of Objectives. The project development objectives were highly relevant to the country’s sectoral needs when the project was developed. The objectives fit squarely into the government’s 2020 strategic vision, NPEP, and the Bank’s 2000- 2002 CAS. They shared many common objectives, the most relevant for this project being universal completion of quality primary education for the poorest. The project objectives continue to fit into the FY2012-16 Country Partnership Strategy (CPS) for Lao PDR, which in turn is closely aligned with the 7th National Social and Economic Development Plan (NSEDP) 2011–2016, in which each have indicated that education is a national priority and core part of the Government’s overall development strategy, which has centered on the main objectives of achieving the Millennium Development Goals (MDGs) by 2015 and graduating from Least Developed Country status by 2020. 39. Relevance of Design. The project design was highly relevant and appropriately aligned with the MoES’s 2020 Strategic Vision and focused on 19 poorest districts, which was highly relevant for the NPEP. In addition, the project design was simple to allow for mainstreaming implementation into the MoES’ daily work. The PDO was appropriate and, again, focused on the poorest districts. The original PDO indicators were appropriate to measure project impact; however, a sub-set or fewer indicators, as suggested by the QSA, would have been more appropriate. The PPF grant was important for ensuring the smooth start-up of project activities for which the design team should be commended. The inclusion of the assessments, tracer studies of the NUOL graduates and quality audits of the CBC, again, was a relevant design feature and provided valuable information on the impact of project activities. The additional financing also used the opportunity to formally modify the results framework based on the MTR and QSA advice. The project risks and mitigation measures were also appropriate. The additional financing used to scale up the project activities was highly relevant. The impact of continuing the community activities at a pivotal time proved to be important for the institutionalization of the CBC and CGs. These ideas are now being built on with the piloting of SBM in the new GPE support education project. 40. Relevance of Implementation. The relevance of project implementation is judged to be high. As indicated above, implementation of the majority of project activities was successful. All project activities for both the original grant and the credit and the additional financing grant were implemented. Where the MoES and ECU encountered implementation challenges, such as with the financial management and quality of construction, they were able to work with Bank staff to find solutions to the problems and resolve the issues. The lessons learned are now being incorporated into the ongoing work of the MoES. This is commendable considering the limited experience and 13 knowledge MoES and ECU had with Bank projects and procedures. The project was consistently rated satisfactory, except in June 2013 and August 2013, when implementation and progress towards achieving project objectives were rated moderately satisfactory. The decision to lower the ratings in 2013 was based on the need for current data for the PDO indicators which was not yet available at the time of project closing. The data was available during the preparation of the ICR. The last ISR rated overall implementation as moderately satisfactory because of the difficulties with financial management, and the Bank team’s cautionary approach regarding the ECU being able to collect and analyze the final data in time for the ICR. Latest information available at the time of ICR writing, however, revealed that the ECU: (i) had been able to clear the project liquidation issues; and (ii) provide the needed data which recorded current evidence of the project’s final achievements. These were positive developments and an indication of increased capacity. 3.2 Achievement of Project Development Objectives 41. The project development objective was primary enrollment and completion increased in the 19 poorest districts in the 6 poorest provinces, quality improved, and capacities strengthened to develop policies and strategies to monitor and manage primary education. This section evaluates the outcomes against the PAD results framework that was modified at the MTR and formalized at the time of the additional financing. In addition, the analysis includes the three PDO level indicators that were added at the time of the additional financing. More details on project outputs can be found in Annex 2. 42. Increased enrollment and completion increased was to be measured by the following PDO indicators: (i) increased enrollment in target districts, (ii) primary completion rate for the entire country; (iii) gender parity index for the entire country, and (iv) lowering construction cost to be at par or lower than other schools built with other sources of funding. There was considerable progress in these PDO level indicators. In target districts, actual enrollment went from 72,700 in 2005 to 104,205 in 2013 surpassing the target of 95,000. In addition to the increase in actual enrollment, the net enrollment rate increased from 50 percent in 2005 to 94.4 percent in 2013 with a female net enrollment rate of 93.6 percent by 2013. 9 Looking at differences in enrollment between targeted schools and non-targeted schools within targeted districts, the effect of the rehabilitation/ completion of schools is clear. Enrollment in grade 4 and, especially grade 5, grew much faster in beneficiary schools than other schools in the same districts (13 percent vs. 9 percent). Since much of the construction that the project financed went to complete existing schools that did not have enough classrooms to provide a full cycle 9 The net enrollment rate indicator (NER) was dropped during the 2010 restructuring because EMIS could not accurately calculate these enrollment rates due to the fact that, at that time, population numbers were only projections. Beginning in 2010, the EMIS data collection system was modified to collect population at the village and school levels, thereby making the NER indicator a valid measure for the purposes of ICR analysis. 14 of primary education, it is unlikely that the village children would have been able to attend schooling in the absence of the project. (See Annex 3 for details.) 43. The primary completion rate for the entire country increased from 64 percent in 2009 to 92.8 percent in 2013, exceeding the project target of 76 percent. The gender parity index for the entire country improved from 0.88 in 2009 to 0.95 in 2013 just missing the target of 0.96. Interestingly, the project districts have a gender parity index of 0.96 in 2013. During the project period, the construction cost for the project schools and classrooms were consistently lower than those of schools built with other sources of funding. For example, in 2009 EDPII schools were being constructed for an average cost of US$94/m2 while schools built with other sources of funds had an average cost of between US$100 and US$250/m2. (See Table 4 in Annex 3 for the full comparison of costs.) The other PDO level indicator was increased number of teachers received training in multi-grade teaching as a percentage of the total number of primary teachers in the target districts. The percentage of teachers increased from 25 percent to 100 percent surpassing the target of 47 percent. The MoES was able to accomplish a full 100 percent by combining support from EDPII and other donor operations. The actual number of teachers trained was 843, exceeding the original target of 540. In addition, the number of qualified teachers (i.e. those with 8+3 years of education) in target schools increased from 67 percent to 80 percent between 2005 and 2013. 44. Quality improvement was measured by the PDO indicators: (i) System for Learning Assessment and (ii) improved learning outcomes in rural and ethnic group areas. The project had a substantial impact on developing a system for learning assessment. The project supported two rounds of the grade 5 assessments on Lao language, mathematics, and the “world around us” (i.e. general science studies). The government, and particularly the Research Institute for Educational Science (RIES), has gained experience in managing the assessment process, including data gathering, analyzing and reporting. Experience from the grade 5 assessments also helped inform the development and administration of the grade 3 assessments under the EFA-FTI project in April 2012, further improving the relevant assessment tools and strengthening capacity in data processing. Lastly, experience from the assessments highlighted the opportunities for closer collaboration between the policy and research centers of the MoES, which the ministry will take into account when administering the next round of grade 3 assessment. The experience for EDPII is being used to inform the further expansion of RIES under the new GPE-supported education project. RIES is looking to refine the purpose and scope of ASLO Grade 3 and Grade 5 in the context of the early grade reading assessment (EGRA). RIES is keen to introduce ASLO in a formal way in Grade 3 potentially to help teachers identify and support students that are falling behind their peers. In the future, RIES wants ASLO to evolve into a competence-based assessment to monitor the acquisition of key skills in literacy and numeracy. 45. Improved learning outcomes in rural and ethnic group’s areas was measured by conducting a 2006 and 2009 national sample assessment of students outcomes in grade 5 against six skill levels identified in each of the subject areas of Lao language, mathematics, and world around us. The results from the 2009 assessment were mixed 15 showing that the Lao-Thai ethnic group performed better in all three areas compared to the other ethnic groups. The Hmong-Mien were second best, especially in mathematics were the scores were almost identical to the Lao-Thai group. A comparison of the 2006 and 2009 assessments again, show mixed results. In the three areas tested, the results for the six basic criteria do not necessarily show much improvement. However, a major contribution of the project has been the considerable capacity building in assessment and institutionalization of assessment within MoES. (See Annex 3 for details.) 46. Capacities strengthened to develop policies and strategies to monitor and manage primary education was measured by the PDO level indicators: (i) increased education spending of total government expenditure; (ii) increased ratio of recurrent to capital expenditure; and (iii) policies and strategies increasingly made on the basis of information. All of these PDO level indictors were met. Lao PDR increased education spending of total government expenditure from 10 percent in 2005 to 18.6 percent in 2013, surpassing the original target of 15 percent. In addition, the government increased the ratio of recurrent to capital expenditure from 40:60 in 2005 to 74:26 in 2013, again, surpassing the target of 70:30. The project supported the establishment of a policy analysis division within MoES, the Education and Sport Research Center (ESRC), to manage the ministry’s policy analysis program. This division carried out three research studies on: (i) education financing; (ii) grade repetition; and (iii) economic relevance of education, supporting informed policy and sector decision making. In addition, the project supported the ESRC by training MoES central, district and provincial staff in action research and research methodologies and conducted rapid assessments on textbooks. 3.3 Efficiency 47. The economic analysis shows that the project has achieved its objectives. The project was also financially sustainable. Enrollment and survival rates in target districts and schools have increased, more than the national average. The analysis also shows that this was done in a cost-effective manner, especially the construction through community- based contracting. The project also supported important reforms and improvements in capacity: supporting SBM, institutionalizing student learning measurements, improving the information system and promoting its use for policy making, as well as establishing an Education Management program at the National University. An ex-post cost-benefit analysis done for the project shows that the benefits of the project greatly outweigh its costs. Considering the patterns of enrollment in grades 4 and 5 as shown in Annex 3 and using a simple assumption that: (i) the project increased educational attainment of beneficiaries by 1 year; and (ii) the returns to education in these districts are half of the country average (which is about 10 percent 10), the project investments will bring in more than US$28 million in income for beneficiaries. (See Annex 3 for full details.) 10 STeP survey 16 3.4 Justification of Overall Outcome Rating Rating: Satisfactory 48. The overall outcome rating of the project is moderately satisfactory. The relevance of design was high. The design was simple and focused on the government’s decentralization of education with a recognition of the need to support local communities for the delivery of education in remote schools. It shifted the orientation of the MoES and sector to focus on the community and provided a good entry point for a dialogue with government on the best delivery system of quality education. While the project preparation dialogue with government took approximately two years, it proved to be highly beneficial for setting the overall framework for the delivery of education services at the local level. In that the project was implemented at the community level, it empowered VEDCs and teachers and presented a marked and positive shift from the centralized structure of delivery. This community empowerment approach with CBC and CGs has also become a key feature of ongoing reforms within MoES. The design also was instrumental in mainstreaming implementation of donor supported projects within MoES, which has now been institutionalized. The relevance of implementation was high. While there were challenges during implementation with the quality of the CBC and financial management, the MoES and ECU, working with the Bank team, have made considerable modifications to the implementation manuals. They also continued to focus considerable attention on strengthening capacity at the local levels to ensure quality of construction and improvements in the financial management system. Efficacy is rated moderately satisfactory. The PDO indicator achievement along with the recorded improvements in local VEDC participation in the delivery of education, the institutionalization of data collection through the EMIS system, and institutionalization of the ASLO all contribute to the efficacy rating. Finally, the project was implemented efficiently. Table 1: Project rating Original Project – 4/29/2004-4/30/2010 – 44.58 percent disbursement-net project funds1 Project Relevance Achievement of PDO (Efficacy) Efficiency Overall Rating High Moderately Satisfactory Substantial Moderately Satisfactory Project Restructuring - 1/2011-9/30/2012 – 55.42 percent disbursement-net project funds Project Relevance Achievement of PDO (Efficacy) Efficiency Overall Rating High Moderately Satisfactory Substantial Moderately Satisfactory Overall Project Ratings – 92.52 percent disbursement of total grant Project Relevance Achievement of PDO (Efficacy) Efficiency Overall Rating High Moderately Satisfactory Substantial Moderately Satisfactory2 1 The net amount for the total project is US$29.87 million. 2 Overall rating = (4)(.4458) +(4)(.5542) =4.00 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 49. The project has played an important role in increasing female participation with the VEDCs. Upon completion of the project 26 percent of VEDCs included female representatives compared to approximately 13.8 percent at the beginning of the project. 17 This increase is a reflection of the focus of the VEDC training, which was supported by the project, on the importance of improving female participation in the local communities. During the implementation of EDPII the enrollment gap between poor and non-poor districts were improved from 50:80 to 70:85, and for girls in particular the enrollment gap between target districts and the national average is now 86:91. With regard to social development, the same team that developed the Ethnic Group Development Plan EGDP also, with project support, initiated a pilot to measure the effect on learning competency of three pedagogical methods of Lao language teaching among ethic grade 1 students in 12 schools in three districts. Based on observations in the pilot areas, the project developed a teacher upgrading program for grade 1 teachers to improve their Lao language teaching. The pilot proved so successful that MoES, with project support, expanded the new methods to six provinces and began to use the methods to teach grade 1 mathematics. The MoES continues to expand the use of the methods developed by the project. (b) Institutional Change/Strengthening 50. The government has fully embraced the CBC which has now become the norm for the construction of pre-primary and primary classrooms in rural Lao. The project helped to institutionalize CGs, which has strengthened the capacity of VEDCs to assist in the delivery of education services. The success of the CBC and CG activities has now evolved into implementation of SBM, which will bring together all village level activities and further strengthen capacity of villages to implement the government’s decentralization program. The project was also responsible for the establishment of the education and management bachelor degree program at the National University of Laos. Since the start of the program in 2006, seven intakes produced a combined total of 295 graduates and a tracer study showed that graduates were entering the civil service as planned. The advances in the EMIS have increased the amount of data available as well as the processes for data collection, data entry and verification. Reporting has greatly improved. Data are collected yearly at the beginning of the school year, and are available before the end of the natural year. The data are also reported through a variety of formats including a Statistical Book, a CD, a website and, notably, mobile apps. Another long- lasting contribution of the project to the quality of the system was the institutionalization of the ASLO. The availability of learning outcomes data will help MoES and schools identify strengths and weakness of student competence against curriculum standards. (c) Other Unintended Outcomes and Impacts (positive or negative) 51. An unintended outcome of the distribution of community grants and construction of primary schools was the demand for provision of pre-schools to accommodate younger siblings of primary school pupils. This demand has led to an increased focus by MoES on early childhood education. The challenge for MoES has been that teachers and principals were not prepared for the number of children wanting to enter pre-school nor did they have the proper early childhood training. In response to this, and as stated above, the next IDA-supported project will focus specifically on early childhood education with a focus on access and training for teachers, principals, parents, and VEDCs. 18 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops Not applicable 4. Assessment of Risk to Development Outcome Rating: Moderate 52. There were four issues identified as risks during the original project preparation with an overall rating of moderate (formerly referred to as modest). The ratings were primarily related to barriers to access due to poverty, ethnicity, limited government resources to hire the needed number of teachers and capacity constraints. The mitigation measures incorporated in the project, with the relevant targeting, capacity building in every component and development of the EGDP, were appropriate and ultimately responsible for ensuring smooth project implementation. The additional financing identified ten risks that maintained the moderate risk for the project. Again, the ratings were primarily related to capacity constraints, which were mitigated through continued training. The additional financing specifically identified financial management as a substantial risk. However, the introduction of the mitigation measures of having one bank account, upgrading the accounting software, and increasing staff, the financial management risk was judged to be moderate. Going forward, the risks identified during preparation of the new operation should remain moderate given there is increased capacity within MoES to implement donor supported projects. However, the financial management risk is likely to be substantial given the increased decentralization of financial responsibility that will require considerable capacity building at the local level. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Satisfactory 53. The project preparation team ensured that the project design was closely aligned with the 2020 strategic vision, NPEP, and the Bank’s 2000-2002 CAS. The design was simple, the PDO and the results framework were appropriate, albeit lengthy, for measuring progress toward achieving the PDO. The design took into consideration the recommendations of the QER, comments from peer reviewers and lessons learned from other Bank-supported projects. The design team ensured that project management was structured to operate through the MoES and not an external management unit. This was a new concept for donor-supported projects in Laos. The design team’s utilization of a PPF grant helped to ensure a smooth implementation start-up and continued ownership of the project. The preparation team identified the appropriate risks and incorporated design features to mitigate them and included the relevant technical specialists to develop the project. The additional financing phase of the project continued to be closely aligned with the FY 2012–2016 CPS and the 7th NSEDP 2011–2016. The design team used the additional financing phase to formally revise the MTR modifications to the results 19 framework, which was based on the Bank QSA. The additional financing did not modify the original design but added US$15.5 million to scale up activities, which was appropriate given the growing implementation capacity exhibited by MoES. All-in-all, the quality at entry for the EDPII project was satisfactory. (b) Quality of Supervision Rating: Satisfactory 54. As previously mentioned, there were 18 Bank review and implementation support missions which included a mid-term review. The project team was actively engaged in supporting the government in its efforts to implement the project. Whenever implementation challenges arose, the Bank team worked with government to find solutions that would not compromise the integrity of the design. As indicated earlier, the Bank team worked quickly to resolve the INT case, restructured the results framework at the time of the additional financing, provided ongoing support for financial management, and quickly dealt with the quality issues raised with school construction. The supervision teams consistently reported on financial management and procurement progress during supervision missions and worked with the ECU team to build their capacity in these areas. As already indicated, they also systematically documented project progress in aide- memoires, back-to-office reports and ISRs, all of which kept Bank management informed of progress and provided the foundation for the ICR analysis. (c) Justification of Rating for Overall Bank Performance Rating: Satisfactory 55. Based on the implementation analysis above, overall Bank performance is rated satisfactory 5.2 Borrower Performance (a) Government Performance Rating: Satisfactory 56. At the time of preparation, the government was fully committed to the project. MoF and MoES were responsive to the utilization of the PPF grant and sent the formal request to the Bank with adequate time to utilize the PPF grant during preparation. As stated previously, this is viewed as being of importance for ensuring the smooth start-up of the project. MoES agreed and worked with the Bank to design a simple project which would be relevant to the needs of Lao PDR. All parties actively participated in the project launch workshop and remained committed to the overall goals of the project. As mentioned above, MoES has embraced the CBC and CG of the EDPII design and institutionalized it as the government’s way of delivering pre-primary and primary education in the most disadvantaged districts. 20 (b) Implementing Agency or Agencies Performance Rating: Satisfactory 57. The ECU was responsible for coordinating all aspects of implementation. If faced with implementation challenges, the ECU was able to work effectively with the other MoES departments and the Bank to resolve them. In addition, the ECU, very early during implementation, quickly identified the misuse of funds by one district engineer, reported it to the Bank and resolved the entire issue quickly and to the satisfaction of the Bank’s INT group. The liquidation of advances was a consistent problem but this was mainly due to weak capacity at the local level. The ECU continues to work to build local capacity and improve the liquidation of funds. The project’s procurement performance was also consistently rated satisfactory due to fact that procurement files were in proper order and post-reviews revealed that procurement guidelines were followed. Based on these aspects, the implementation agency performance is rated satisfactory. (c) Justification of Rating for Overall Borrower Performance Rating: Satisfactory 58. Based on the implementation analysis above, overall borrower performance is rated satisfactory. 6. Lessons Learned 59. Lesson 1. Decentralization of school construction to remote rural communities increases ownership and commitment to education. The EDPII project introduced CBC and CGs which substantially lowered the cost of classroom construction and led to increased interest in education in the project areas. A beneficiary assessment showed that parents were more willing to send their children to school when they participated in the CBC and when CGs were provided for teaching and learning materials. It also showed that CBC was more effective in the project villages rather than in villages where the community based activities were mandated by the authorities, donors or development partners. 60. Lesson 2. Mainstreaming implementation within government agencies requires ongoing capacity building. EDPII was the first Bank-supported project that mainstreamed implementation within MoES. The project supported capacity building which has led to substantial skills upgrading within MoES for project management. There are still implementing units within MoES where capacity needs to be strengthened, particularly in financial management. However, mainstreaming of project management was institutionalized within MoES during the life of EDPII and is the now the norm for all donor supported education projects in Lao PDR. 61. Lesson 3. Strong financial management systems are critical for ensuring the efficient use and accounting of resources. Lack of a fully functioning financial management system led to delays in liquidation of advances throughout the project period 21 and reporting delays during the early implementation period. There was substantial improvement in reporting during the implementation of the project. However, there is still a need for improvement in the liquidation of advances. The MoES continues to use technical assistance to support capacity building in this area and follow-on projects continue to focus on improving the financial management system. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies See Annex 7 for Borrower ICR. (b) Cofinanciers Not Applicable (c) Other partners and stakeholders Not Applicable 22 Annex 1. Project Costs and Financing (a) Project Cost by Component (in US$ Million equivalent) Actual/Latest Appraisal Estimate Percentage of Components Estimate (US$ millions) Appraisal (US$ millions) Component A: Increase Access 6.50 18.98 291.94 and Completion of Primary Education in the Poorest Districts Component B: Improve the 2.10 1.77 84.06 Quality of Primary Education Component C: Strengthen 4.20 8.92 212.47 Capacities of Policy Analysis and Management PPF 0.20 0.20 100.00 Total Baseline Cost 13.00 29.87 230 Physical Contingencies 0.00 0.00 0.00 Price Contingencies 0.00 0.00 0.00 Total Project Costs 13.00 29.87 Front-end fee PPF 0.00 0.00 .00 Front-end fee IBRD 0.00 0.00 .00 Total Financing Required 13.00 29.87 230 (b) Financing Appraisal Actual/Latest Type of Estimate Estimate Percentage of Source of Funds Cofinancing (US$ millions (US$ millions Appraisal ) ) Borrower 0.50 0.00 0 International Development 5.56 104.9 5.30 Association (IDA) IDA Grant for Debt Vulnerable 7.70 8.58 111.4 IDA Grant 13.21 .00 Australian Trust Fund 2.52 .00 23 Annex 2. Outputs by Component 62. Initial progress towards achieving project outcome and meeting output targets was measured by 14 PDO indicators and 19 intermediate indicators. Following the 2007 mid- term review and the additional financing, the revised results framework included 12 PDO and 10 intermediate indicators. 11 This annex uses the combined indicators as listed in the revised results framework to assess output achievements. Component 1: Increase Access and Completion of Primary Education in the Poorest Districts (Original grant: US$6.60 million; AusAID funding: approx. AUD 3.28 million 12; and Additional Financing grant: US$10.565 million). 63. Original component activities: The aim of the original component was to support three interventions in the 19 poorest districts in six provinces 13 to raise primary enrollment and completion rates, and reduce repetition and drop out by: (i) piloting a community-based contracting (CBC) and maintenance program for classroom construction to provide the physical access to allow incomplete schools to gradually offer all 5 grades of primary education; (ii) providing community grants (CG) for schooling to accompany the CBC program to lower the education cost for communities; and (iii) conducting in-service teacher training to all teachers in participating villages in multi- grade teaching, and upgrading untrained contract teachers through an in-service upgrading program. Additional Financing component activities included the scaling up of the successful CBC and CG programs, expand the multi-grade teacher training as well as capacity building of VEDC, DEB and PES. 64. Sub-component 1.1: Community-based Contracting (CBC) for School Construction. Progress of this sub-component was to be measured by the following indicators: (i) incomplete dilapidated primary schools replaced to accommodate 5 grades; (ii) CBC grants distributed equitably (50 percent of schools in ethnic minority villages; and 50 percent off the road 14 ); and (iii) increased number of classrooms built and rehabilitated under the project as a percentage of total number of required classrooms at the primary level in target districts. 65. The following results have been reached: (i) the end-target for completing 714 dilapidated schools with 2,409 classrooms 15 was exceeded, reaching a combined total of 11 The indicator “communities receive school grants” was listed as a new indicator in the AF results framework. This indicator will not be listed separately in this ICR because progress is already being reported under the original indicator “CG grants awarded to CBC villages”. 12 The actual amount is AUD 3,276,639. 13 The target provinces were Phongsaly, Oudomxay, Louangnamtha, Houaphanh, Attapeu and Xekong. 14 “Off road” is defined as schools not connected to a numbered national road but accessible via informal roads such as feeder roads to measure school remoteness. 15 The end-project target of completing dilapidated schools was modified per the April 2007 mid-term review aide-memoire: MOE decided to reduce the number of schools from 400 to 319 but increase the number of classrooms from 800 to 957 classrooms. The new overall targets are comprised of 319 schools 24 781 schools 16 with 2,462 new classrooms; (ii) 82.4 percent of all schools having received CBC grants are located in ethnic minority villages and 60.6 percent are located “off road,” exceeding the 50 percent target; and (iii) 102 percent of the required classrooms were constructed over the life of the project, thereby meeting the end target. 66. In addition to these achievements, the project piloted the community-based approach for school construction, the success of which led to additional financing for the project from AusAID in the amount of AUD 3,276,639 and IDA in the amount of US$15.5 million equivalent. There is also evidence that CBC was more cost efficient than classrooms financed by other projects (see Annex 3 for details) as well as effective in building schools in very remote areas. Net enrollment rates in the target districts have risen, suggesting a possible positive link between project interventions and net enrollment. Although school construction activities encountered initial delays due to the novelty of the CBC approach, measures were taken in 2010 to strengthen local capacity and improve quality assurance. As a result, weaknesses could and were addressed during construction 17. The project also conducted a Safety and Quality Audit that was able to identify schools needing retrofitting or rehabilitation given that some structures dated back to 2005. This audit was an important aspect of the project implementation for ensuring the ongoing quality of the school buildings. 67. School construction under the original project included toilets and water tanks, which were challenging for communities to build and at times required improvements. Due to these challenges it was agreed that the UNICEF WASH program would be responsible for construction of toilets and water tanks under the additional financing phase of the project. There were delays in the construction of the toilets and water tanks. However, UNICEF has confirmed that all project schools will receive the required toilets and water tanks and they are in the process of retrofitting these schools. Capacity building of PES’ and DEBs was carried out and basic office equipment procured to facilitate their respective roles in the CBC and CG activities which, in addition to VEDC training, helped strengthen the foundation for the government’s decentralization agenda. 68. This component also supported achievement of the project’s objective through two PDO indicators: (i) lowering construction cost to be at par or lower than other schools built with other sources of funding, which was met; and (ii) increased enrollment in target districts, which rose from 72,700 to 104,205. In addition to the increase in actual enrollments in absolute terms, the net enrollment rate increased from 50 percent in 2004 to 94.4 percent in 2013 with a female net enrollment rate of 93.6 percent by 2013. (957 classrooms – original project), 125 schools (372 classrooms – AusAID TF) and 270 schools (1,080 classrooms - AF), reaching a new target of 714 schools (2,409 classrooms). 16 318 schools (953 classrooms) were financed under the original grant, 125 (372 classrooms) with AusAID funds and 338 (1,137 classrooms) with resources provided under the additional financing. 17 Steps taken to strengthen the quality assurance process included: (i) updating/clarifying the CBC operations manual and roles/responsibilities of all actors involved in the quality assurance process; (ii) increasing the original 3-member VEDC to 5 or 7 members depending on the CBC and CG program demands; and (iii) strengthening district and provincial education officials in supporting villages during the construction process and carrying out regular supervision missions. 25 69. Sub-component 1.2: Community Grants for Primary Education. Progress of this sub-component was to be measured by the following indicators: (i) CGs awarded to CBC villages annually (number of villages; and cumulative grants); (ii) number of VEDCs established; (iii) percentage of village committees that include representation of all ethnic groups and women; and (iv) participatory beneficiary assessment carried out. 70. Achievements under this sub-component included: (i) a total of 1,893 CGs were provided by project closing, missing the intended target of 2,142 CGs 18 due to initial construction delays; (ii) 781 VEDCs were established, exceeding the combined target of 714; (iii) 100 percent of VEDCs include one member from all ethnic groups present in a given village and at least one woman, meeting the end-target; and (iv) an impact evaluation, which includes a beneficiary assessment, of CG and CBC recipients was carried out in 2009 and showed an increased interest in pre- and secondary school spaces as a result of project interventions. The CG program helped reduce the cost of schooling especially for the poorest by financing school uniforms, school bags, and teaching and learning materials. A small percentage was set aside to help with school maintenance. The project also helped train 5,467 VEDC members in the use of CGs and application of the CBC approach, exceeding the combined target of 2,007. 19 Efforts are being made to further strengthen VEDC’s participatory approach especially to encourage members not accustomed to speaking up and ensure their active participation. 71. An additional outcome was the use of bank accounts to transfer CBC and CG funds down to the school and village level, which was an innovative project design feature. This has been repeatedly highlighted by beneficiaries as a major improvement to previous practices of traveling long distances to collect funds. 72. Sub-component 1.3: Teacher In-service Upgrading. Progress of this sub- component was to be measured by the indicator increased number of teachers received training in multi-grade teaching as a percentage of the total number of primary teachers in the targeted districts and schools, 20 which was exceeded: 100 percent of the 3,832 teachers in the project areas were trained in multi-grade teaching of which 843 were supported through the project, surpassing the initial target of 540 teachers. The project also trained 116 master trainers (11 females) on the use of the new textbooks. The master trainers trained 4,764 teachers (1,447 females) on how to use the new textbooks and curriculum for Grades 1-5. 18 This figure is based on the stated goal to provide 3 rounds of grants to 319 schools under the original project (i.e. 957 CGs instead of the 1,469 as stated in the original results framework), 270 schools under the AF (i.e. 810 CGs), and 125 schools constructed with AusAID funds (i.e. 375 CGs). 19 1,207 villagers under the original project and 800 villagers per the additional financing project paper. 20 This AF indicator also reports on the original PDO indicator “All teachers in project schools trained in multi-grade methods” to avoid reporting duplication. 26 Component 2: Improve the Quality of Primary Education (Original credit: US$2.30 million). 73. The objectives of this component were to: (i) provide textbooks and teachers’ guides to ensure that teachers and students in grades 1-5 had the required textbooks and guides, and building MoES capacity to manage the cycle of textbook production and distribution and (ii) conduct the ASLO to establish a permanent national assessment system that would develop student assessment instruments to monitor achievement of basic competencies. 74. Sub-component 2.1: Textbooks and Teaching Materials. Progress of this sub- component was to be measured by: (i) primary textbooks procured and delivered to schools and (ii) teachers’ guides procured and delivered to schools, which were both exceeded. The project procured and distributed 3,087,000 grade 1–5 textbooks to schools compared to the target of 2,240,000 textbooks and 300,000 teachers’ guides compared to the 196,600 target. As mentioned above, the project also supported training of teachers in the application of these revised learning materials. To ensure sustainability and replacement of textbooks, the government has used revenues from the Nam Teun II dam to periodically replace approximately 11 percent of the textbooks to ensure that schools have an adequate supply. 75. Sub-component 2.2: Assessment of Student Learning Outcomes. Progress of this sub-component was to be measured by the PDO indicator System for Learning Assessment, 21 which was met. The project supported two rounds of the grade 5 assessments on Lao language, mathematics and “world around us” (i.e. general science studies). The government, and particularly RIES, has gained experience in managing the assessment process, including data gathering, analyzing and reporting. Experience from the grade 5 assessments also helped inform the administration of the grade 3 assessments under the EFA-FTI project in April 2012, further improving the relevant assessment tools and strengthening capacity in data processing. Lastly, experience from the assessments highlighted the opportunities for closer collaboration between the policy and research centers of MoES, which the ministry will capitalize from when administering the next round of grade 3 assessments. Component 3: Strengthen Capacities for Policy Analysis and Management (original credit: US$2 million; original grant: US$1.3 million; additional financing grant: US$3.575 million 22) 76. Original component activities: Component 3 aimed to: (i) strengthen policy and strategy development capacity of MoES to make information-based decisions; (ii) strengthen information systems for improved collection, analysis, reporting, filing, storing and maintenance of data and information; (iii) build capacity for better 21 The original project PDO indicator “improving learning outcomes in rural and ethnic minority areas” will be reported on under this AF PDO indicator. 22 US$1.36 million remained unallocated. 27 management in a broader effort to strengthen capacity in management and administration; and (iv) finance project management and strengthen capacities in coordination, procurement and financial management. Additional financing activities supported the next phase of implementation of these policy reforms to help implement the ESDF, and modification of project management activities to help MoES move in a systematic and phased manner towards SWAp implementation. 77. Sub-component 3.1: Strengthen Policy and Strategy Development. This sub- component was to be measured by the PDO indicator policies and strategies increasingly made on the basis of information, which was met. The project supported the establishment of a policy analysis division within MoES, the Education and Sport Research Center (ESRC), to manage the ministry’s policy analysis program. This division has carried out three research studies on: (i) education financing; (ii) grade repetition; and (iii) economic relevance of education, which supported informed policy and sector decision making. In addition, the project supported ESRC by training MoES central, district and provincial staff in action research and research methodologies and conducted rapid assessments on textbooks. The project also hired a firm to conduct an impact evaluation of the CBC and CG interventions. The evaluation showed that parents were more inclined to enroll their children in schools and sought to educate themselves. ESRC’s capacity to manage education sector issues has been strengthened with five staff members having pursued Master’s degrees in social science, economics, education, finance and education management. ESRC was also supporting the development of the ESDP, which was endorsed by the local education working group, and helped establish a harmonized approach to investments in the education sector. The endorsement of the ESDP also helped attract education sector funding, including a US$30 million from EFA- FTI and AUD 20.26 million in support of the Education for All-Fast Track Initiative project. 78. In 2004, MoES established the EGDP to support awareness raising and teacher training on how to better teach grade 1 ethnic minority students. With project support, EGDP initially developed teaching and learning materials for teaching Lao language to non-Lao pupils in 12 pilot schools and trained 221 teachers. Based on positive results observed, the program was expanded to ultimately cover 48 schools in 6 provinces, train an additional 253 teachers and develop materials to teach grade 1 mathematics to ethnic minority students. The project also helped produce and distribute a total of 6,652 manuals and books, which benefitted a total of 814 students. By project closing, the EGDP team trained 56 teachers from all 8 teacher colleges, mainstreaming these training modules into the pre-service teacher training curriculum. 79. Although GoL PDR did not complete the transition towards a SWAp, it has made important progress towards a sector-wide support to education. In addition to developing an ESDF (which ultimately resulted in the ESDP mentioned above), the ESRC and Faculty of Education initiated a multi-departmental education research support network, which currently functions on an informal basis but with the intention of being institutionalized. The government also developed a Medium-Term Strategy and Performance Improvement Plan (MTSPIP) in 2011 to facilitate using the country’s 28 financial management systems. A manual covering financial management, accounting and procurement was developed and is currently being revised. Approval of the manual’s final version is expected soon to allow for its application starting fiscal year 2013-14. Furthermore, MoES is in process of developing an education sector results-based monitoring and evaluation matrix. A number of workshops to discuss respective draft documents have been conducted with an initial focus on developing the matrix for the pre-primary and primary sub-sector. This matrix is expected to serve as a model for the expansion to other education sub-sectors. The first draft of the primary and pre-primary sub-sector matrix was piloted in August 2013 and this activity is being continued under the EFA-FTI project. 80. Sub-component 3.2: Strengthen Information Systems. This sub-component was to be measured by the intermediate indicator, education management information systems strengthened as evidenced by the publishing of statistics yearbooks, which was met. The project supported the strengthening of its EMIS system, including the installation of improved software. Although there were initial delays in setting up the EMIS, the ministry and international technical assistance were able to overcome these challenges and improve overall data collection and statistics reporting. Some of the main achievements include: (i) publishing statistical yearbooks and other education information online and making them accessible via the web or mobile applications; (ii) providing education information disaggregated by gender and ethnicity; (iii) incorporating personnel and financial information down to school levels; and (iv) linking EMIS data collection and analysis with GIS. The EMIS team has also provided extensive training at the central, provincial, district and school levels on a range of information collection, analysis and management aspects, including the use of GIS. 81. Although requests for ad-hoc information by other MoES divisions have faced delays in the past, the EMIS team continued to improve data gathering, generation and dissemination. The activities related to the gathering of GIS data were also delayed. This was due to the government’s re-drafting of village parameters, which has now been completed and will allow the EMIS team to start collecting the relevant information. The project supported the establishment of a “Documents Center” to establish norms and standards for filing and storing MoES information. Although the EMIS team, which is in charge of these centers, has routinely uploaded documents since 2006, MoES still needs to ensure that the appropriate human and financial resources are made available so that the EMIS team can continue to carry out its mandate. 82. Sub-component 3.3: Capacity Building for Improved Management. This sub- component was to be measured by the intermediate indicator, program on education administration and management established, which was met. The project successfully supported the FOE at the National University of Laos in establishing a department of education administration and management (EAM) and developing an EAM bachelor’s degree program. Since the start of the program in 2006, seven intakes produced a combined total of 295 graduates. A tracer study on first intake graduates conducted in 2013 showed that of the 41 graduates, 40 were employed as civil servants in the MoES and one pursued a Master’s degree. EAM has also developed a master’s degree syllabus 29 and started offering weekend classes in 2008. In 2013, NUOL prepared the curriculum for a full-time master’s degree course, which is currently being considered for approval. The project also supported 8 NUOL staff pursuing a higher education degree, leading to three PhD and five master’s degree holders, who were instrumental in helping, develop the EAM bachelor’s degree program and continue to work closely with the NUOL. 83. Project Management. The Project Management Unit (PMU), under the guidance of a steering committee, was tasked with managing project implementation, coordination of activities with implementing units (IUs), and all fiduciary and safeguards aspects of project activities. In 2010, the PMU changed its name to “Education Sector Development Program Coordination Unit (ECU)”. The IUs implemented the aspects of the project for which they were responsible and the ECU successfully carried out its mandate of overseeing and monitoring overall project implementation, coordinating with internal departments such as the departments of finance, planning and inspection, and organizing joint donor review missions. ECU, along with the IUs, was effective in identifying implementation challenges and addressing them, and strengthening CBC and CG implementation oversight despite the remoteness of many project sights. Initial weak capacity of IUs to carry out advanced planning led to some delays, which were addressed with targeted and ongoing capacity building. The ECU also regularly collected performance indicator data and reported on overall progress, and the procurement management process was consistently rated satisfactory. Although financial management challenges related to the advance liquidation process persisted, no major irregularities were identified and all audits were unqualified. 30 Annex 3. Economic and Financial Analysis 85. The economic rationale for this project as stated in the PAD was based on four objectives: access, quality, efficiency and equity. Based on these objectives, the PAD reported an approximate Economic Rate of Return greater than 10 percent. However, no details were given on how it was calculated or under which assumptions due to the fact that no data were available at the time of preparation to carry out a thorough cost-benefit analysis. A simple ex-post calculation using recently available data on returns to education shows that the benefits did indeed outweigh its costs. However, even without providing a cost-benefit analysis, the economic justification for the project was straightforward. The standard arguments for investing in education all applied in this case, such as externalities and constraints in access to capital. Furthermore, the expected benefits from expanding access and improving quality of education in the poorest districts in the country (higher productivity, higher wages, and better health) were likely to outweigh the cost of the project. 86. The project components were aligned with the PDO, and the target districts were well defined and justified. The PDO of the project was to increase enrollment and completion in the 19 poorest districts in the six poorest provinces of the country, as well as strengthening capacities to plan, budget, manage and monitor results. The six poorest provinces and, within these, the 19 poorest districts were identified based on the results of the most recent Lao PDR’s Poverty Assessment at the time. The project 23 deliberately focused all resources on these 19 poorest districts to address gaps in access to primary education between these districts and those in other parts of the country. Citing the need increase survival rates in target districts, the construction component was focused on completing and rehabilitating existing incomplete schools instead of building new ones. The quality component focused on providing these schools with improved inputs such as trained teachers, operational resources, textbooks and community participation. The quality and capacity building components sought to introduce important long-lasting reforms. 87. This economic analysis shows that the project has achieved its objectives. The project was also financially sustainable. Enrollment in target districts and schools have increased, more than the national average. The analysis also shows that this was done in a cost-effective manner, especially the construction through community-based contracting. The project also supported important reforms and improvements in capacity: supporting school-based management, institutionalizing student learning measurements (ASLO), improving the information system and promoting its use for policy making, or establishing an Education Management program at National University. 88. This annex explores first the contribution of the project to the PDO, then discusses the cost-effectiveness of some of the project interventions before looking at its financial sustainability, including a simple cost-benefit analysis. 23 See “sector issues to be addressed by the project and strategic choices” in the original PAD 31 89. Contribution to the PDO. The project contributed to improving access to education in the targeted districts through a combination of school construction and rehabilitation, school grants for materials and the provision of textbooks. The project increased the number of classrooms available in targeted districts by 2,462 classrooms in 781 schools, building or rehabilitating classrooms in almost half of all the schools in the targeted districts. Table 2: School construction targets and achievements Source Total of EDP II EDP II AF Description funding Achievement Target Achievement Target Target Achievement 8/31/2013 IDA 319 318 270 338 Schools 714 781 AusAID 125 125 IDA 957 953 1080 1137 Classrooms 2,409 2,462 AusAID 372 372 Source: Borrower Implementation Completion Report, February 2014 90. This increased access to education has translated into better educational attainment in these districts. Both enrollment rates and survival rate to grade 5, which were two of the main objectives of the project, have improved in target districts and have done so slightly faster than the national average. In EDPII districts, net enrollment rates increased from 84 to 93 percent, whereas the national average went from 89 to 95 percent. The difference in the increase in the completion rate was similar. However, while the differences were small, taking into account that target districts were disadvantaged to start, converging to the national average should be considered an achievement. Figure 1: Enrollment and survival rates target districts, targeted provinces and all country, 2005-2012 32 91. Perhaps a cleaner estimate of the impact of the school construction is to look at differences in enrollment between targeted schools and non-targeted schools within targeted districts, especially in grades 4 and 5. When comparing these two groups, the effect of the rehabilitation/completion of schools is clear. Enrollment in grade 4 and, especially grade 5 grew much faster in beneficiary schools than other schools in the same districts, indicating that the completion of the school had an effect on enrollments. Since much of the construction that the project financed went to complete existing schools that did not have enough classrooms to provide a full cycle of primary education, it is unlikely that these children would have been able to attend schooling in the absence of the project. Figure 2: Enrollment growth by grade, in beneficiary and non-beneficiary schools from construction in targeted districts 14% EDP schools Non-EDP schools in EDP districts 12% 10% % icnrease in enrollment by grade 8% 6% 4% 2% 0% -2% -4% Grade 1 Grade 2 Grade 3 Grade 4 Grade 5 Source: EMIS 92. Construction was not the only input the project provided. In line with existing evidence of the importance of school-based management and community involvement, the project provided community grants of 3,000,000 Lak per year for three years to a number of these schools. As previously mentioned, these grants were provided to the VEDC’s to buy teaching and learning materials for the school as well as uniforms, pens, school bags, sports equipment, school maintenance etc. In addition, the project provided in-service teacher training for teachers in these areas. Overall, the interventions were designed and implemented as a package of physical infrastructure, materials and teacher training, which was complemented by institutional changes aimed at increasing community participation and school management of resources. 33 Table 3: Cumulative total of Community Grants Plan EDP II EDP II AF Source of & 2005 2006 2007 2008 2009 2010 2011 2012 2013 Total* fund Actual (Y1) (Y2) (Y3) (Y4) (Y5) (Y6) (Y7) (Y8) (Y9) Plan IDA 19 200 319 319 100 270 270 270 1,767 AusAID 125 125 125 375 Actual IDA 267 319 318 51 12 271 283 1,521 AusAID 124 **99 **25 **124 372 Actual Total 267 319 318 175 111 296 407 1,893 Source: Borrower Implementation Completion Report, February 2014 *Totals are cumulative **Schools constructed using AusAid funding near the end of EDPII. One round of grants was funded under EDPII. The final two rounds were paid from the AF budget. 93. The provision of all this inputs contributed to reducing the cost of attending school for parents in beneficiary districts. While no data exists on average expenditure for households in project districts before or after the project, the likelihood that the project reduced the cost of attending school and therefore facilitating enrollment and attendance, is high. The construction of new schools and classrooms in existing schools reduced the transportation cost. The community grants of 3,000,000 LAK per year for three years were used to offset the previously-mentioned schooling, which parents/community would normally have to pay. The grants were designed to reduce the costs to parents and in this they were fundamentally successful. The publishing and distribution of textbooks and teachers guides also reduced the cost to parents as they did not have to purchase textbooks. 94. Beyond the quality improvements and cost reductions for parents that the provision of inputs brought, the introduction of community grants and community involvement in managing education funds contributed to setting the foundation for transforming the way the education system is managed. In line with the decentralization and community development efforts of the GoL PDR, this project introduced these community grants and was a precursor to the existing school block grants later introduced by the government. While still in the early stages, considering existing evidence on school-based management and the importance of strong community involvement in school-based management, these institutional changes could potentially have very large effects in the quality of education in the longer term. 95. Community-based construction: As shown in Table 4, the average cost of building per square meter is between a quarter and half less than the cost using traditional, centralized construction contracts. While the initial limited capacity to supervise construction resulted in some quality shortcomings, these processes have been greatly improved. By the end of the project, the quality of the construction was at least as good as that in other projects that used contractors for construction. Given the considerable school infrastructure deficits remaining in Laos, the introduction and improvement of this method will have very significant effects on the capacity of Lao PDR to attain its ambitious goal of increasing access to education. 34 Table 4: Comparison of construction costs under different projects Cost per Implementation Rate per m² Project classroom Notes year (US$) (US$) EDP II* Baseline 2004 54 3,500 1 IDA 2005 77 5000 2 AusAID 2009 86 5600 3 EDP II Baseline 2009 94 6,000 4 AF* 291 2010 110 7,100 5 schools 47 schools 2013 144 9,330 6 EQIP2** 2009 110-120 7 PRF** 2009 100 8 2013 250 BEGP** 2005 85-95 JICA** 2009 250 Source: Borrower Implementation Completion Report, February 2014 Notes: All m² rates are arrived at by taking total contracted cost and dividing by total m² of building. 1 EDP II notional cost is 54 per m². 2 The average school construction cost financed by IDA is US$77 per m², these inclusive of furniture. 3 The average school construction cost financed by AusAID is US$86 per m², these inclusive of furniture. 4 EDP II AF notional cost is 94 per m². 5 The average school construction cost of 291 EDP II AF initially schools is US$110 per m², these inclusive of furniture. 6 The average school construction cost of 47 EDP II AF additional schools is US$144 per m², these inclusive of furniture. 7 Comparative cost from ECDM. 8 Comparison is for PRF schools in the district in which it overlaps with EDP II. * built by villages. ** built by contractors employed by the projects. 96. EMIS. The advances in the EMIS with the support of the project were impressive. Not only has the amount of data available through EMIS been increased, but the processes for data collection, data entry and verification, and reporting have been greatly improved. Data are collected yearly in the beginning of the school year, and are available before the end of the natural year. This is an impressive turnover. The data are also reported through a variety of formats including a Statistical Book, a CD, a website and, notably, mobile apps. In addition to publishing data, the project supported using EMIS data at the district level by carrying out workshops in every district to facilitate the development of school plans at the district level. As mentioned, all districts now have a completed plan that identifies needs based on enrollment and school data from EMIS (including GIS). This can have long lasting effects in improving the efficiency in the use of resources, especially schools and teachers, by promoting the use of data for planning at the district level. 35 97. ASLO. A second long-lasting contribution of the project to the quality of the system was the institutionalization of the ASLO. During the 2005-10, the ASLO team in RIES developed the instruments for monitoring achievement of basic competencies, carrying out a baseline test in Grade 5 in school year 2006/7. A further ASLO test for Grade 5 was carried out in 2008/9. Used mainly as a diagnostic tool, the use of the results to design interventions is still a challenge. However, the mere availability of learning outcome data can potentially bring tremendous benefits to the MoES and to schools, helping to identify strengths and weakness of student competence against curriculum standards and designing interventions specific to the learning challenges of different students. Figure 3: Lao Language, 2006 and 2009 60 50 40 Percent Achieving 30 2006 2009 20 10 0 1 2 3 4 5 6 Skill Level Source: Borrower Implementation Completion Report, February 2014 36 Figure 4: Mathematics, 2006 and 2009 60 50 40 Percent Achieving 30 2006 2009 20 10 0 1 2 3 4 5 6 Skill Level Source: Borrower Implementation Completion Report, February 2014 Figure 5. World Around Us, 2006 and 2009 40 35 30 25 Percent Achieving 20 2006 2009 15 10 5 0 1 2 3 4 5 6 Skill Level Source: Borrower Implementation Completion Report, February 2014 37 98. Quantifying the impact of these institutional changes is not possible, but the contributions of this project to systemic change are clear. The improvements in data availability and use, institutionalization of learning outcome assessments, the increase in community involvement in managing education resources are all characteristics of well- performing systems. 24 99. Cost-Benefit. The original economic analysis estimated at approximately 10 percent the IRR of the investment, but no data or methodology were provided for the calculation. While the institutional investments of the project are hard to quantify, an attempt is made to provide an ex-post estimate of the cost-benefit analysis of project interventions. Because of the limitations in analyzing the benefits of institutional reforms, the focus of this analysis on the effect of increasing access to education will mainly focused on construction. In terms of benefits, the analysis only considers the income associated with the increased educational attainment associated with increased enrollment and completion brought about by the project-supported investments. As for cost, the total amount of the project is being used to estimate a per student cost based on the number of beneficiaries in target schools. As a consequence, estimates are probably lower-bound estimates of the actual benefit of the project, since the social returns to education have been shown to outweigh private returns. In addition, the cost estimates are likely to be higher of the per student cost, since we allocate all the cost of the project to the beneficiaries in schools, whereas institutional reforms affect all students in the country. 100. Even with these conservative assumptions, the ex-post cost-benefit analysis shows that the benefits of the project greatly outweigh its costs. Considering the patterns of enrollment in grades 4 and 5 shown in the previous section and using a simple assumption that: (i) the project increased educational attainment of beneficiaries by 1 year; and (ii) the returns to education in these districts are half of the country average (which is about 10 percent 25 ), the project will bring in more than US$28 million in income for beneficiaries which is a very positive return. 101. The project investments were largely financially sustainable and the current budget trends are a small threat to its sustainability. The financial sustainability of new projects is an important concern in Lao PDR, considering that recent budget trends have left very little space for non-wage recurrent costs to be absorbed by the government budget. Between 2010 and 2012, capital spending has remained relatively constant in real terms, while salary spending has increased by 150 percent. As a result, by 2012, only 8 percent of the budget was for non-wage recurrent spending. In this context, the construction of new classrooms invariably leads to increased needs for teachers, and even though the evidence on enrollment indicates that the new classrooms were well targeted to schools with enough demand, the current budget situation raises some questions about the overall sustainability of salary spending. 24 SABER Overview Report, World Bank. 25 STeP survey 38 Figure 6: Trends in education spending 2,500 25% Education Spending (Real, 2012 Kip) As a % of Total Govt Spending 2,000 20% 1,500 15% Billion Kip 1,000 10% 500 5% - 0% 100% 90% 80% 70% Capital 60% 50% 40% 30% Recurrent 20% 10% 0% 100% 90% 80% 70% Capital 60% 50% 40% 30% Recurrent 20% 10% 0% Source: Borrower Implementation Completion Report, February 2014 39 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Rosemary Bellew Lead Education Specialist EASHD Task Team Leader Serge Theunynck Sr. Implementation Specialist Implementation Chinnakorn Chantra Procurement Specialist Procurement Rasario Aristorenas Program Assistant EASHD Preparation Support Kannathee Danaisawat Financial Management Specialist Financial Management Mark Bardini Monitoring & Evaluation Specialist M&E Luis Benveniste Education Specialist EASHD Ed. Specialist Alfonso de Guzman Sr. Education Specialist EASHD Textbook Specialist Peter Moock Lead HD Economist EASHD Economic Analysis Nina Eejima Counsel Lawyer Rosa Muleta Finance Officer Disbursements Sr. Financial Management Behdad Nowroozi Financial. Management Specialist Nipa Sirbuddhamas Financial Management Specialist Financial Management Eliezer Orbach Consultant EASHD Capacity Building David Week Consultant EASHD Architect Danile Schwitter Consultant EASHD Architect Svend Jensby Consultant EASHD Safeguards Robert Krech Consultant Human Dev. Specialist Birger Fredriksen Peer Reviewer Project Dev. Guidance Christopher Shaw Peer Reviewer Project Dev. Guidance Mourad Ezzine Peer Reviewer Project Dev. Guidance Robert Prouty QER Panel Chair David Harding QER Panel Panel Member Albert Aime QER Panel Panel Member Anne Dykstra QER Panel Panel Member Supervision/ICR Boun Oum Inthaxoum Operations Officer EASHE Task Team Leader Jeffrey Waite Lead Education Specialist MNSHE Task Team Leader Rosemary Bellew Lead Education Specialist EASHD Task Team Leader Luis Benveniste Lead Education Specialist AFTED Primary Education Margaret M. Clarke Sr. Education Specialist HDNED Primary Education Kannathee Danaisawat Financial Management Specialist EAPFM Financial Management Rie Kijima Consultant EASHD Primary Education Robert Krech Operations Officer SECPO Implementation Jennica Larrison Consultant EASHD William Anthony Lorie Sr. Education Specialist HDNED Primary Education 40 Oithip Mongkolsawat Senior Procurement Specialist EAPPR Procurement Donald Herrings Sr. Financial Management AFTFM Financial Management. Mphande Specialist Emiko Naka Consultant AFTAR Hoi-Chan Nguyen Consultant LEGEM Suhas D. Parandekar Senior Education Economist EASHE Education Sybounheung Social Development Specialist EASTS Safeguards Phandanouvong Viengkeo Environmental Specialist EASTS Safeguards Phetnavongxay Somphone Simmalavong ET Consultant EAPCO Education Nipa Siribuddhamas Financial Management Specialist EAPFM Financial Management. Sirirat Sirijaratwong Procurement Analyst EAPPR Procurement Souphanthachak Procurement Analyst EAPPR Procurement Sisaleumsak Malarak Souksavat Resource Management Assistant EACLF Binh Thanh Vu Sr. Education Specialist EASHE Primary Education Sandra Beemer Consultant EASHE ICR, Author Franco Russo Operations Officer EASHE ICR, Author Pedro Cerdan-Infantes Education Economist EASHE ICR Economic Analysis (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle US$ Thousands (including No. of staff weeks travel and consultant costs) Lending FY02 7.29 37.98 FY03 42.30 367.60 FY04 42.16 122.31 Total: 91.75 527.89 Supervision/ICR FY05 57.47 115.23 FY06 31.16 71.35 FY07 34.47 72.21 FY08 37.14 80.92 FY09 49.45 87.16 FY10 36.90 26.48 FY11 45.11 93.60 FY12 37.95 31.22 FY13 60.63 153.10 FY14 30.80 74.73 Total: 421.08 806.00 41 Annex 5. Beneficiary Survey Results Not Applicable 42 Annex 6. Stakeholder Workshop Report and Results Not Applicable 43 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR I. Main Beneficiaries and Outputs under EDPII 102. The Grant Agreement clearly identified the expected beneficiaries through the project development objective which was “primary school enrollment and completion increased in the 19 poorest districts of the six poorest provinces.” The original EDPII objective and project components as shown below remained unchanged through the EDPII Additional Funding and reflected the main strategic goals of the Education Strategic Plan. The project development objective was achieved through the implementation of three components each of which had their own objective. 103. The project will benefit thousands of primary school students, teachers and Education staff in the future through a range of interventions at all levels to improve quality and effectiveness of the education service. In addition to this, the community construction sub-component has made a significant contribution to enabling equitable access to education provision at local level and improving school facilities as well as strengthening responsibility, ownership and understanding of local people for their school and the community it serves. Component A: Increase Access and Completion of Primary Education in the Poorest (19) Districts in six provinces. 104. Community Based Construction. Students, teachers and the community were the beneficiaries. The activities were: a) construction of schools: from 2005 to February 2010 under component A the project constructed 443 schools (125 funded by AusAID) comprising 1,325 classrooms (372 funded by AusAID) from a target of 400 school (800 classrooms). The Additional Funding from February 2010 – August 2013 (EDPII- AF) increased the total number of school constructions by 338 (1137 classrooms) against a further target of 270 (1080 classrooms). 105. The EDPII +AF combined total number of 781 schools (2,462 classrooms) against a combined total target of 714 schools (2,409 classrooms) will benefit primary school children for generations to come by providing the facilities for them to receive a full five years of primary education and thus enabling them to move on to lower secondary and at least complete the full nine years of basic education as stated in the MoES education strategic plan. Regular access to education will in turn give the opportunity for the community to improve health, business and the resulting reduction in poverty. 106. Another benefit was that by using community based construction, the project kept construction costs to a minimum when compared with other methods, while at the same time encouraging ownership and developing capacity of the community. 107. Community Grants. Students, teachers and the community were beneficiaries of this activity. Further benefits to the school, teachers and students were provided in the form of community grants of 3,000,000 Lak per year for three years. These grants 44 enabled the community, through Village Education Development Committees (VEDCs) to buy teaching and learning materials for the school (e.g. uniforms/ pens/ school bags / sports equipment, school maintenance). A total of 1,079 community grants were disbursed during the original EDPII period of 2005 – 10. An additional 814 grant disbursed through EDPII-AF increased the total number of grants to 1,893 by 2013. The additional 47 schools constructed at the end of EDPII-AF did not receive a community grant. 108. Teacher Training. Students and teachers benefited from this activity. Teachers (and students) also benefited from in service teacher training. During the life of the EDPII project, 48 Master trainers (4 females) were trained on multi-grade methods. This benefited 843 (165 female) teachers who, in turn were trained in multi grade teaching, methods. Training in multi grade methods continued to be funded through EFA-FTI after 2010. It continues to benefit countless teachers and students. A further 116 (11 females) Master Trainers were trained on use of the new textbooks. This directly benefited 4,764 teachers (1,447 females) who were subsequently trained on how to use the new textbook and curriculum for Grades 1 – 5. Teacher upgrading during the AF period was funded through EFA-FTI. Component B: Improve the Quality of Primary Education. 109. Under component B students and teachers further benefited from delivery of new textbooks at Grades 1 – 5 together with the corresponding teachers guides. Table 4 below shows that 3,078,000 textbooks were printed and distributed against a target of 2.24million. 300,000 teacher’s guides were printed and distributed against a target of 196,600. Table 5: Distribution of Textbooks Date of Grade No. of Textbooks No. of Teachers Guides Statistics 2008 – 2009 Distribution Subject Total Subject Total Students Classes 1.9.2008 1 318,000 945,000 10,900 65,400 276,658 9,810 1.9.2008 2 219,000 657,000 10,900 65,400 192,980 9,401 1.9.2009 3 186,000 558,000 10,900 65,400 166,995 7,540 1.9.2009 4 164,000 492,000 8,650 51,900 144,316 5,874 28.2.2010 5 142,000 426,000 8,650 51,900 128,003 5,260 Total 3,087,000 300,000 110. Instruments for monitoring achievement of basic competencies were developed under this component by the Assessment of Student Outcomes (ASLO) team during 2005 – 10. The baseline test in Grade 5 was carried out in school year 2006/7 and the results were disseminated in school year 2007/8. A further ASLO test for Grade 5 was carried out in 2008/9. The results of these tests will be of great benefit to the MoES and to schools when the strengths and weakness of student competence can be objectively measured against curriculum standards. The information will lead to more interventions which can be put in place to increase performance. 45 Component C. Strengthen Capacities for Policy Analysis and Management. 111. The MoES was the main beneficiary of Component C. The main objective of this component and its sub-components was to support capacity building of the MoES staff and leaders to make decisions based on reliable information. 112. EMIS. This designed to strengthen the EMIS function in order to benefit all departments of the MoES and all Implementing Units (IUs) of the program by providing timely, accurate statistics and data for effective education management. By 2013 the EMIS office had supported the production of EduInfo, an easily available database of all education statistics. It was also engaged in training provincial and district staff on how to use SPSS for data analysis and how to use the GIS mapping program. These activities help to strengthen capacity for analysis and management of the MoES staff at all levels and help them to gather clear information on which to base future planning. Building on the work funded through EDPII and AF the department is using EFA-FTI funding to further strengthen the planning process by hiring a consulting firm, “Network Solutions Services (NSS) Co. Ltd. to make revisions to the EMIS system which will link it to personnel and finance management information systems 113. ESRC. This unit was originally called the Policy Analysis Division. It then went on to be called Strategic Research Education Analysis Centre and is currently known as the Education Strategic Research Centre (ESRC). It was established during EDPII and conducted three important research studies. These were: (i) financing, (ii) grade repetition, and (iii) economic relevance. These key research reports and other related studies were produced as a basis on which to make sector decisions and inform policy. Research skills were further strengthened by sending five MoES staff on long-term training to gain Masters Degrees in the following subjects: social science, economics, education finance, and education management. Increased capacity in research and education management also has a longer term benefit to the MoES as a whole. 114. NUOL: Staff from the National University of Lao and ultimately their students have benefited greatly from the Education Administration and Management Program (sub-component C3). The Department of Education Management and Administration was established in 2006 funded by EDPII. The new Department offered a Bachelor Degree in Education Management and Administration. The first intake of students in 2006 included 41 participants of which 13 had just left secondary school. The remaining participants were serving teachers or administrators. There were three intakes during the life of EDPII. These were intake (i) 42 trainees; intake (ii) 41 trainees; intake (iii) 50 trainees. The program was continued under EDPII-AF and intake 7 brought the total number of lecturers trained to 295. In addition to this, at the start of the program eight NUOL staff were offered scholarships to study for higher degrees. All of these scholarship beneficiaries are still working in close association with NUOL and have helped to build capacity of many others over the years. 115. In 2013, under EDPII-AF the Department of Education Management and Administration prepared the syllabus for a full time Master’s Degree in Education 46 Management and Administration. It is currently ready for presentation to the academic board. 116. Ethnic Group Development Plan. More than 117 central, district and provincial education staff as well as teachers and by association, their countless students in Oudomxay and Luang Namtha benefited from cultural and gender awareness training in a workshop to disseminate the positive findings of three different pedagogical methods of teaching Lao language among ethnic group Grade 1 pupils in Oudomxay province. This work was continued during EDPII – AF when teaching materials and teacher guides were piloted and then expanded to other districts. 117. Project management. Under the sub-component on Project Management the MoES and project staff benefitted from training in procurement, accounting and financial management. II. Key Factors Affecting Performance. 118. On the whole the project experienced very few barriers to achievement of original goals. However there were areas where the project did not meet the planned levels of performance. On 2 June 2008, the government accepted a trust fund granted by AusAID. This included funding to build 127 schools in Component A: Community Based School Construction. These schools were not in the original target numbers. However two schools planned to be constructed with AusAID funding were removed from the 127 because the villages where the schools were to be constructed were resettled away from the designated construction sites. The remaining 125 were all constructed successfully and were in full use before EDPII was scheduled to finish. Overall the project exceeded the original target of 670 schools (1880 classrooms) by constructing 781 schools and 2642 classrooms. 119. Delays in reporting in general at central, provincial, district and local level affected clearance of advances and delays in withdrawal applications. In addition, planned development of the GIS system was delayed due to the fact that newly established district borders were not clearly defined. However work started again in May 2013 and was completed by the end of the project. It will eventually be mainstreamed into the main ESIC work. The Department of Inspection was able to prepare the monitoring tools without assistance from an international TA and these tools are currently in the process of gaining MoES approval. 47 III. Achievement of Overall Project Objectives. 120. This section of the report will summarize the extent to which the project achieved the objectives. 121. Increased access to 5 years of primary education. Access was increased during EDPII through construction of 443 schools with 1,325 classrooms. This exceeded the target of 400 schools with 800 classrooms. EDPII – AF had a target of 270 school constructions with 1080 classrooms. This too exceeded the target with a total of 338 schools and 1,019 classrooms. This resulted in a combined total of 781 schools with 2,409 classrooms against a combined target of 670 schools with 1,880 classrooms. Increased access was also evidenced through increases in enrollment which exceeded the targets. The net enrollment rate in target districts rose from 50 percent to 88.3 percent (86 percent girls) during EDPII and again to 94.4 percent (93.6 percent girls) during EDPII- AF. 122. Lower costs to parents. Activities carried out by the project aimed at reducing costs to parents were as follows: (i) construction of additional schools potentially reduced travel costs; (ii) community grants of 3,000,000 LAK per year for three years were used to offset various costs of schooling such as school uniforms, books, school bags, teaching and learning materials for the school, which parents/ community would have been expected to pay for. The independent beneficiary assessment (Evaluation of Impact of Community-based Interventions by Madecor Career Systems Inc.) found that in EDPII, as a result of the community grant and the new school building, every family was obliged to send their child to school. The value of education was increased and many village chiefs encourage parents themselves to obtain a certificate equivalent to level Grade through non- formal education routes. The report considered the project activities at local level during EDPII to have led to “significant change.” The independent study concluded that the community grants were “successful and effective. 123. Improved school quality. School quality was improved in a number of ways. In addition to the replacement/refurbishment of the school buildings and classrooms themselves, the use of the community grants to buy teaching and learning materials and the distribution of new textbooks together with teachers guide books, the following activities also helped to improve the quality of teaching and learning. 124. Teacher Upgrading. (i) 100 percent of the 3832 teachers in the project area received multi-grade teacher training. 843 of these teachers (165 females) were supported by EDPII; (ii) 4,764 (1,447 female) teachers were trained in use of the new text books and teachers guides; (iii) over the life of the project the number of qualified teachers (those with 8+3 or higher) in target schools rose from 67 percent to 80 percent and finally to 100 percent by the end of EDPII-AF; (iv) a national assessment of student learning outcomes was developed and piloted. The learning assessment tests were developed by teams of specialists from the Assessment of Student Learning Outcomes team (ASLO), supported by an international consultant, to ensure that the test items measured 48 achievement against the learning outcomes in the curriculum. They were designed to measure achievement in three subject areas: literacy in Lao language, Mathematics, and World Around Us. Six skill levels were identified in each of the subject areas under consideration. 125. The Ethnic Group Development Plan (EGDP). The Ethnic Group development plan team was formed to carry out research on the quality of teaching Lao language to Grade 1 children from ethnic groups. The final report completed in 2008/9 showed that pupils in the pilot schools performed better than the pupils in the non-pilot schools. Overall, boys scored higher than girls. A survey of the teachers and school principals in the pilot schools showed that they appreciated the training. They agreed that the new methods were effective and could be scaled up for use with any ethnic group. The outcome of the research was used as a basis for expansion of the pilot to 6 provinces and ultimately 48 schools in EDPII-AF as well as training on teaching and learning in Grade 1 mathematics for ethnic pupils. 126. The main lessons learned from the EGDP team during EDPII were: (i) frequent follow up was essential to maintain momentum, encourage the teachers and to check on the learning outcomes; (ii) teacher exchange visits between pilot schools was a cost effective way of sharing lessons learned and examples of good practice; (iii) the most effective teaching and learning materials were those which were based on the real life ethnic diversity of the students; (iv) gender balance was a cross cutting theme in all training and materials and (v) more research was needed to identify the key issues which affect the learning outcomes of ethnic pupils. 127. The EGDP main objectives for the work under EDPII-AF were: (i) to expand the work on teaching Grade 1 language for ethnic group children to five further provinces covering 840 (416 girls); and (ii) to research the best way to improve learning outcomes in Grade 1 mathematics for children from ethnic groups. The extension period offered the team an opportunity to prepare a comprehensive set of teaching and learning materials and to train teachers on how to use them effectively. 253 teachers (53 female) were trained during EDPII-AF. By 2012, under EDPII-AF the team had completed a teacher’s guide for teaching Lao language for Grade 1 for ethnic students as well as games for teaching Lao language, exercises for beginners in Lao language, a teacher’s guide for teaching mathematics for Grade 1 and exercises in mathematics. In 2012/13 the team extended the teacher training of Lao Language at Grade 1 from 12 schools in each of six provinces (36 schools) to 48 schools in the same 6 provinces, extending the number of children to 814. During that year 6,652 manuals and books were distributed to teachers and students in the 48 target schools (36 original schools plus 12 extended schools). 128. In January 2013, the EGDP team shared the assessment results on the pilot teaching Lao language and mathematics at Grade 1 to 814 ethnic students in 36 schools. Participants were schools principals, provincial and district PAs, and Grade 1 teachers. An agreement was made during the workshop that the schools should continue to use this teaching methodology and the program should expand to the other schools with proper training and refresher training for the teachers. In July 2013 the EDGP trained a total of 49 56 teachers from the 8 teacher training colleges on how to train pre service teachers to teach Grade 1 Lao Language and Mathematics to children from Ethnic groups. A further 20 staff from the MoES, and provincial offices also attended (total 76 participants, 38 female). This will assure the sustainability of the program. 129. Key things which contributed to the success of the team activities during EDPII- AF included: (i) a clear monitoring plan with sufficient budget; (ii) good cooperation between the staff of the 6 target provinces and staff of the 36 target schools; and (iii) the increased capacity and experience of the EDGP central team. 130. The main lesson learned by the team under EDPII-AF was that more teacher training is needed nation-wide to improve the quality of teaching and learning for children from ethnic groups. The program for teaching Lao language at Grade 1 for ethnic pupils is now ready for a nation-wide rollout. 131. Lower repetition and higher completion rates. Technically the MoES operates a system of progressive promotion. This means that only the very poorest performing students will be required to repeat. These repetition rates are collected by the EMIS team. At the start of EDPII in 2005 the repetition rate in Grades 1 and 2 in target districts was 40 percent. The target was to reduce repetition in the target districts to 20 percent. At the end of EDPII repetition was 32 percent in Grade 1 (girls 32.2 percent) and 13.4 percent (girls 13 percent) in Grade 2. By 2013 under EDPII-AF, repetition had reduced to 22.5 percent (girls 22 percent) in Grade 1 and 9.6 percent (girls 8.8 percent) in Grade 2. Primary completion rate in target districts was 50 percent in 2005 at the start of the project. The target for improvement was 70 percent. By the end of EDPII in 2010 actual achievement was a small increased 54.7 percent overall with 53.9 percent for girls. This had slightly decreased by 2013 to a total of 51 percent (girls 50.3 percent). These figures should be seen in a context of increasing enrollment rates and where the gap in net enrollment rate in target districts compared with national enrolment figures was steadily reducing. See below for further discussion under “Reduced enrollment gap between poor and non-poor districts and for ethnic groups”. 132. Village empowerment and support. The project constructed new schools where VEDCs were established in villages The VEDCs comprised 7 members of the community, including women, representatives of local ethnic groups, teachers and school principals. Each project related VEDC had three sub committees. One was responsible for the management and coordination of all project funded activities in their village. The second sub-committee was responsible for making an annual development plan for their school and managing the community grant in accordance with the plan. A third sub- committee, with support from a district engineer appointed by the project, assisted with procurement of construction materials and a contractor to build the school. The head of the VEDC was the legally recognized representative of the village with regard to school construction. The VEDCs received regular visits from their district engineer during the school construction phase. From a target to establish 400 VEDCs between 2005 and 2010, the EDPII team established 402 new VEDCs to assist the project implementation at local level and trained 1,207 villagers to become effective VEDC members. Under 50 EDPII-AF 338 new VEDCs were established and 3,718 villagers were trained to support the project activities between March 2010 and August 2013 when the project closed. 133. Reduced enrollment gap between poor and non-poor districts and for ethnic groups. Both EDPII and EDPII-AF were successful in achieving this indicator. In 2005 the baseline was 50:80 with a target of 70:85. The gap had reduced to 88: (girls 86) in target districts compared with a national average of 93 (girls 91). The gap in enrollment rates between target districts and national average continued to decrease to 94:96. The gender gap also reduced. 93:94 in target districts compared with a national average of 96.8:96. 134. Increased public expenditure on education. Under EDPII the target for increased public expenditure on education was 15 percent against a baseline of 10 percent. However, during EDPII- AF, by school year 2012/13 public expenditure on education had risen to 18.6 percent. 135. Improved policy analysis. During the life of EDPII a number of key activities were implemented to improve policy analysis. These included commissioning reports on financing, grade repetition, economic relevance and research on development of teaching methodologies to help students from ethnic groups to learn Lao language effectively. A beneficiary assessment (Evaluation of Impact of Community –based Interventions by Madecor Career Systems Inc.) and work on the assessment of learning outcomes were also started during EDPII. The results of these reports and research have been used to improve policy analysis. A summary of the findings from the work of the EGDP is included in the section on improving school quality. 136. The Education Statistics and Information Centre (ESITC) was established in 2007. It was originally known as the Statistics and Information Technology Division (SITD) but later changed its name to the Education Statistics Technology and Information Centre. (ESTIC) A documentation centre was established in 2008. The Centre was originally designed to be a resource for MoES staff and others to access published information on the work of the MoES. However, development of the MoES website and further strengthening of the EMIS means that the role of the documentation Centre is becoming integrated into the work of the ESIC. The EMIS IU was divided into two main activities. One was tasked with implementing improvements and extensions to the function of EMIS, the second was responsible for developing the Geographic Information System (GIS) as well as establishment and maintenance of the Documentation Centre. In 2006, the Bank used its own budget to carry out a stocktaking and diagnostics study of the EMIS system, supported by an international consultant. The study was designed to identify the strengths and weaknesses of the EMIS system. It was completed by April 2007 and the MoES agreed to work towards integrating EMIS information throughout the MoES to enable it to produce sector wide reports. This was further strengthened during EDPII and to date the EMIS team has strong capacity to provide a wide range of basic information and statistics. It is currently providing valuable training to provincial and district staff on using GIS and analysis of SPSS data to identify needs and solve problems. This together with the EMIS system expansion is detailed in 51 the section above on component C beneficiaries. Despite these crucial achievements there is still a need for further development to enable IUs to obtain timely information as a basis for monitoring, evaluation and decision making. 137. Project Management. Key Activities carried out across EDPII and EDPII-AF carried out were: (i) established a project working group and monthly meeting schedule comprising IU heads, chaired by the Project Director; (ii) maintained positive and productive relations with the World Bank, and other development partners; (iii) developed capacity of MoES central staff in the IU’s to clearly manage and implement their EDPII and AF activities in accordance with the agreed development objectives; (iv) maintained a clear overview of disbursement and encouraged IU’s to complete their planned activities and budgets in a timely manner while paying attention to potential impact on achievement of project goals; (v) operated within a framework of good governance to ensure transparency and accountability; (vi) encouraged and where possible, facilitated and enabled synergy between different areas of the project, especially with Community Construction and Community Grants; (vii) held regular meetings with the Steering Committee to monitor progress and find solutions to project issues; (ix) monitored timely implementation of the procurement plan (x) created a small but effective management team with clearly delegated responsibilities within the ECU to support and monitor timely implementation of IU activities across EDPII-AF and the EFA- FTI program; and (xi) ensured that Project Audit reports were submitted on time and were unqualified. 138. Procurement. Throughout the life of both EDPII and EDPII-AF .the project has encountered no significant procurement issues and procurement has been rated as satisfactory throughout the project. 139. Financial Management. A project financial management manual was approved by the Bank at the start of EDPII and is the foundation of financial management for the project. The first quarterly Financial Management Report (FMR’s) using the standard tables set out in the Financial Management manual was approved as satisfactory by the Bank and subsequent FMR’s have all been submitted on time. The reports have encountered issues of variable quality throughout the project lifetime, particularly in the area of matching the physical progress report with expenditure There is still more work needed to ensure that correct financial information is provided to the ECU on a timely basis In part the problems are due to working within the government system which can be time consuming given the number of departments involved in the process. Disbursement was slow at the start of the project but has increased steadily over the years. 140. Female Representation. By the end of EDPII 26 percent of VEDCs (against a baseline of 13.8 percent) included female representatives. Learning achievement and completion rates of girls in target districts were similar to those of boys. 141. Representation from Ethnic Groups. There are approximately 49 different ethnic groups with four basically different language groups, living in Lao PDR. These are Lao-Thai aprox. 66 percent; Mon-Khmer aprox. 24 percent; Hmomg-Mien aprox, 8 52 percent; Sino-Tibetan including Chinese Ho and Tibeto-Burnam aprox. 3 percent. Approximately 80 percent of the population of Project target villages were from at least four of the country’s ethnic groups. These were Mon-Khmer, Hmong-Mien, Tai-Kadai and Sino-Tibetan. Therefore, all project activities and results can be seen within the framework of supporting capacity development and encouraging demand- led initiatives from non –Lao ethnic groups. Sustainability. 142. Component A1. School construction is a totally sustainable activity which will benefit countless numbers of children over the coming years. 143. Component A2.1. Community Grants in themselves are not sustainable as villages receive only 3 rounds of community grant. However the community grants have been moved in to the government “Block Grants” program and are therefore sustainable. The training received by the VEDC’s and the experience they have gained in managing the Grants mean that they will be better equipped to manage the government Block Grants in the future. 144. Component C1. The work of the Ethnic Development Group on development of teaching materials and strategies for teaching Laos Language Grade 1 children from Ethnic Groups is sustainable since it is becoming mainstreamed. 145. Component C2.1. Training in the use of GIS for school mapping at District and Province level will be sustainable in that, over time, it will result in clear information on the location of schools and their distances from local landmarks. This will enable the MoES to more accurately plan the national school construction strategy in the coming years. 146. Component C2.2. Data analysis using SPSS training for technical staff from MoES is a sustainable activity as it will develop the MoES staff capacity to analyze the data collected from the provinces and elsewhere. This will enable more effective results based management and decision making. 147. Component C3. The establishment of a full-time Master’s Degree in Education Management and Administration at NUOL will be a valuable addition to the mainstream portfolio of the Education Management and Administration Department within the NUOL and contribute to long term quality improvement of Education Management and Administration nation -wide over the coming years. 148. Long Term training of seven MoES staff. This will result in sustainable long term benefit to the MoES. In addition to increasing the number of internationally trained staff in the MoES, these staff will share their knowledge and skills, thus contributing to overall institutional development. 53 Lessons learned 149. Quality improvement and community based interventions have resulted in a significant rise in enrollment and completion over the life of EDPII and EDPII- AF. This has led to demand for a program to expand access to lower secondary education and number of trained teachers. 150. Community based construction and incentives are used more effectively in villages where social cohesion is already strong rather than in villages where the community based activities are mandated by the authorities, donors or development partners. 151. Monitoring and Evaluation capacity within the IU’s was adequate. More focus and training on M+E is needed in the future. 152. Given the difficulties in ensuring quality of construction, especially when using the community construction procurement method, the CBC team, ECDM, ECU and other stakeholders should work together to further strengthen reporting on quality monitoring. Key enabling factors 153. The majority of the target villages which received the community grants had previous experience of working with International projects through e.g.; Unicef, WFP, EU, GTZ, UNDP. This helped them to more quickly understand the processes they needed to follow for CBC and community grants. 154. The final Independent Impact Evaluation of Community Based Interventions (by Madecor Career Systems Inc.) stated that achievement of project objectives at local level was strengthened because of the close link with government and the support of the District Education staff who were regularly monitoring progress. 155. At central level there was a strong, permanent management/ coordination team with oversight of all activities. Central level staff from the PMU/ECU and the IU’s made frequent visits to the project sites to gain a full picture of the real situation on which to base management / implementation decisions. 156. The World Bank, AusAID and other development partner staff provided advice and support where needed. They made regular visits to the project sites in the provinces to clearly understand the issues. 157. Conclusion. The EDPII and EDPII – AF successfully achieved the majority of the performance indicators. Teacher training and school construction exceeded targets. 158. The key factors which contributed to the success of EDPII and EDPII-AF were: (i) excellent leadership and management both from the MoES Management Team 54 and the World Bank Management Team; (ii) a country context where the development partners and MoES work positively to create a clear strategic plan, clear development goals and a policy monitoring framework against which to monitor achievements; (iii) an environment where communities support and embrace the development of education provision; and (iv) finally, of these factors the first is the most important--a good leader makes sure that everyone understands the issues, works towards achievement of common goals, is not afraid to admit to problems and motivates staff to find positive ways to solve them. The program was very fortunate to have these in both the Program Director and the Task Team Leader. 55 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Not Applicable 56 Annex 9. List of Supporting Documents Project Appraisal Document, World Bank, 2004 Aides-memoire and Implementation Status Reports 2002 – 2013 World Bank Country Assistance Strategy 2000-2002. Project Paper for a proposed restructuring, World Bank, 2009. World Bank Country Partnership Strategy 2012-2016. 57 IBRD 33431R1 100°E 102°E 104°E 106°E 108°E This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and any other information shown on this map do not imply, on the part of The World Bank Group, any judgment on the legal status of any territory, or any endorsement or acceptance of such boundaries. CHINA LAO P.D.R. To Lincang To Gejiu To Daluo Gnot-Ou Ou 22°N 22°N Phongsali Boun-Nua PHONGSALI MYANMAR VIET NA M Muang Luangnamtha Khoa To LUANGNAMTHA Hanoi g kon Meung Xai Xiangkho Me Nambak BOKEO g u Xam-Neua O Houyxay T a Viangxai n Be OUDOMXAY LUANGPRABANG HUAPHANH To Chiang Rai X i a n g k h oa n g Pl a t eau 20°N Pakbeng Luangprabang Kham Nan Plain o f J a r s Phokhoun Pek Xayabuly XIENGKHUANG Kasi XAYABULY VIENTIANE Phou Bia Gulf of Vangviang (2,817 m) Ban Nalé Xaisomboun Tonkin Nam Ngum Reservoir BORIKHAMXAY Viangkham g Pakxan on ek Khamkeut Paklay M Kadi VIENTIANE ng Me 18°N Xanakham 18°N ko ng PREFECTURE OF KHAMMUANE VIENTIANE MUN. Cammon Plateau Ban Na Phan To Xe Thakhek Vinh an B To g fai Khon Kaen N oy To Khon Kaen Xebangfai 0 50 100 Kilometers Xéno SAVANNAKHET Xepon 0 25 50 75 100 Miles THA ILA ND Kaysone Phin To Qui Nhon g Se Banghion Samouay LAO PEOPLE'S 16°N 16°N DEMOCRATIC SARAVANE n Saravane Do REPUBLIC Khongxedon SEKONG Lamam CITIES AND TOWNS Pakxong To Pakxe PROVINCE CAPITALS Ubon Ratchathani CHAMPASAK B o l o v e ns P l a t e a u ATTAPEU NATIONAL CAPITAL Samakhixai Sanamxai RIVERS MAIN ROADS RAILROADS Khong 14N 14°N PROVINCE BOUNDARIES CA MBO DIA INTERNATIONAL BOUNDARIES 104°E 106°E OCTOBER 2013