Report No. 101692 PROJECT PERFORMANCE ASSESSMENT REPORT NEPAL EDUCATION FOR ALL (IDA-39560 IDA-H3400) December 22, 2015 IEG Public Sector Evaluation Independent Evaluation Group ii Currency Equivalents (annual averages) Currency Unit = Nepalese rupee (NPR) 2003 US$1.00 NPR76.10 2004 US$1.00 NPR73.70 2005 US$1.00 NPR71.40 2006 US$1.00 NPR72.80 2007 US$1.00 NPR66.40 2008 US$1.00 NPR69.80 Abbreviations and Acronyms ASIP Annual Strategic Implementation Plan AWPB Annual Work Program and Budget BPEP Basic and Primary Education Project CERID Research Center for Education Innovation and Development CSSP Community School Sector Project DANIDA Danish International Development Agency DEO District Education Office(r) DFID Department for International Development DOE Department of Education ECD Early Childhood Development EFA Education for All EMIS Education Management Information System FM Financial Management FMIS Financial Management Information System GON Government of Nepal HT Head Teacher ICR Implementation Completion Report IDA International Development Association ADB Asian Development Bank IEG Independent Evaluation Group IEGPS IEG Public Sector Evaluation INGO/NGO International non-governmental organization ISR Implementation Status and Results Report JICA Japan International Cooperation Agency MDG Millennium Development Goals MoE Ministry of Education MoES Ministry of Education and Sports NASA National Assessment of Student Achievement NLSS National Living Standards Survey NORAD Norwegian Agency for Development Cooperation AusAid Australian Agency for International Development OP Operational Policy PAD Project Appraisal Document SSRP School Sector Reform Project PCF Per Capita Financing PDO Project Development Objective PETS Public Expenditure Tracking Survey PPAR Project Performance Assessment Report PPE preprimary education PTA Parent Teacher Association SBG school block grants SIP Strategic Implementation Plan SMC School Management Committee SSRP School Sector Reform Project SWAp Sector Wide Approach TEP Teacher Education Project TSC Teachers Service Commission’s UNESCO United Nation’s Educational, Scientific and Cultural Organization UNICEF United Nations Children’s Fund VEC Village Education Committee WFP World Food Programme Fiscal Year Government: July 15 – July 14 Director-General, Independent Evaluation : Ms. Caroline Heider Director, Human Development and Economic Management : Mr. Nick York Manager, Corporate and Human Development Unit : Ms. Marie Gaarder Task Manager : Ms. Ann Elizabeth Flanagan v Contents Principal Ratings ................................................................................................................ ix Key Staff Responsible........................................................................................................ ix Preface................................................................................................................................ xi Summary .......................................................................................................................... xiii 1. Background and Context................................................................................................. 1 Background ..................................................................................................................... 1 Socio-Economic Context ................................................................................................ 1 Country characteristics................................................................................................ 1 Economics ................................................................................................................... 2 Poverty ........................................................................................................................ 3 Education in Nepal .......................................................................................................... 3 Evolution of Education in Nepal................................................................................. 4 Trends in Education 1999 to 2003 .............................................................................. 4 Regulation and Provision of Primary Education ........................................................ 5 Remaining Challenges in Primary Education in 2003 ................................................ 7 2. Objectives, Design, and their Relevance ........................................................................ 7 Objectives ....................................................................................................................... 7 Project Institutional Framework ................................................................................. 9 Relevance of the Objectives and Design ...................................................................... 10 Relevance of Objectives ........................................................................................... 10 Relevance of Design ................................................................................................. 11 Implementation ............................................................................................................. 12 3. Achievement of the Objectives ..................................................................................... 14 Ensuring Access and Equity in Primary Education ...................................................... 15 Outputs ...................................................................................................................... 16 Outcomes: was there Achievement in Access and Equity? ...................................... 18 Improving Efficiency and Institutional Capacity .......................................................... 21 Outputs ...................................................................................................................... 22 Outcomes: Did efficiency and institutional capacity improve? ................................ 24 Enhancing the Quality and Relevance of Basic Primary Education to Children and Illiterate Adults ............................................................................................................. 26 This report was prepared by Ann Elizabeth Flanagan who assessed the project in May, 2014. The report was peer reviewed by Tara Béteille and panel reviewed by Monika Huppi. Viktoriya Yevsyeyeva and Yezena Yimer provided administrative support. vi Outputs ...................................................................................................................... 26 Outcomes: Was the quality and Relevance of Basic Primary Education to children and Illiterate Adults enhanced? ................................................................................. 29 4. Efficiency ...................................................................................................................... 31 5. Ratings .......................................................................................................................... 32 Outcome ........................................................................................................................ 32 Risk to Development Outcome ..................................................................................... 32 Bank Performance ......................................................................................................... 33 Borrower Performance .................................................................................................. 34 Monitoring and Evaluation ........................................................................................... 35 6. Lessons .......................................................................................................................... 37 References ......................................................................................................................... 39 Annex A. Basic Data Sheet ............................................................................................... 43 Annex B. List of Persons Met ........................................................................................... 45 Annex C. Borrower Comments......................................................................................... 46 Tables Table 1.1. Selected Macro-Economic Indicators, 2000-2013............................................. 2 Table 1.2. Poverty Indicators, 1995-2010 ........................................................................... 3 Table 1.3. Evolution of Education Provision and Structure in Nepal ................................. 4 Table 1.4. Educational Outputs 1999-2003 ........................................................................ 5 Table 1.5. Grant Funding of Schools in Nepal ................................................................... 6 Table 2.1. Education for All Program Components ........................................................... 8 Table 2.2. Appraised and Actual Costs of the Education For All Program 2004-2009 ... 13 Table 3.1. Results Chain for Obtaining EFA Access and Equity Project Goals .............. 15 Table 3.2. Outputs Associated with Increased Access and Equity in Basic and Primary Education .......................................................................................................................... 17 Table 3.3. Number of Scholarships Distributed, 2005-2009 ............................................ 17 Table 3.4. Primary Education Access and Equity Outcome Indicators ............................ 18 Table 3.5. Results Chain for Obtaining EFA Efficiency and Institutional Capacity Project Goals ................................................................................................................................. 22 Table 3.6. Repetition Rate by Grade ................................................................................. 24 Table 3.7. Drop-out Rate by Grade ................................................................................... 25 Table 3.8. Results Chain for Obtaining EFA Quality Project Goals ................................ 27 Table 3.9. Outputs and Intermediate Outcomes Associated with Education Quality ....... 29 Table 3.10. Survival Rate to Grade 5 ................................................................................ 29 Table 3.11. Comparison of Literacy Rates over Time ...................................................... 30 vii Figures Figure 1.1. Population Age 6-24 Who Have Never Attended School, 2003-2010 ............. 3 Figure 3.1. Trend in the number of students enrolled in primary school.......................... 19 Figure 3.2. Improvement in net enrollment rate, primary education ............................... 19 Figure 3.3. Number of Dalit/Janajati students enrolled in primary education .................. 20 Figure 3.4. Share of Dalit/Janajati enrolled in primary education .................................... 20 Figure 3.5. Net Attendance Rate in Primary School, 2011 ............................................... 21 Boxes Box 2.1. The EFA Sector-Wide Approach ....................................................................... 10 ix Principal Ratings ICR* ICR Review* PPAR Outcome Moderately Moderately Satisfactory Moderately Satisfactory Unsatisfactory Risk to Development Moderate Moderate Moderate Outcome Bank Performance Moderately Moderately Satisfactory Moderately Satisfactory Unsatisfactory Borrower Moderately Moderately Satisfactory Moderately Satisfactory Performance Unsatisfactory * The Implementation Completion Report (ICR) is a self-evaluation by the responsible Bank department. The ICR Review is an intermediate IEGWB product that seeks to independently verify the findings of the ICR. Key Staff Responsible Division Chief/ Project Task Manager/Leader Country Director Sector Director Appraisal Chingboon Lee Julian F. Schweitzer Kenichi Ohashi Rajendra Dhoj Joshi Michelle Riboud Completion Rajendra Dhoj Joshi Amit Dar Susan G. Goldmark Venkatesh Sundararaman x IEG Mission: Improving World Bank Group development results through excellence in evaluation. About this Report The Independent Evaluation Group assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the Bank’s self-evaluation process and to verify that the Bank’s work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, IEG annually assesses 20-25 percent of the Bank’s lending operations through field work. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which Executive Directors or Bank management have requested assessments; and those that are likely to generate important lessons. To prepare a Project Performance Assessment Report (PPAR), IEG staff examine project files and other documents, visit the borrowing country to discuss the operation with the government, and other in-country stakeholders, and interview Bank staff and other donor agency staff both at headquarters and in local offices as appropriate. Each PPAR is subject to internal IEG peer review, Panel review, and management approval. Once cleared internally, the PPAR is commented on by the responsible Bank department. The PPAR is also sent to the borrower for review. IEG incorporates both Bank and borrower comments as appropriate, and the borrowers' comments are attached to the document that is sent to the Bank's Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public. About the IEG Rating System for Public Sector Evaluations IEG’s use of multiple evaluation methods offers both rigor and a necessary level of flexibility to ad apt to lending instrument, project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (additional information is available on the IEG website: http://worldbank.org/ieg). Outcome: The extent to which the operation’s major relevant objectives were achieved, or are expected to be achieved, efficiently. The rating has three dimensions: relevance, efficacy, and efficiency. Relevance includes relevance of objectives and relevance of design. Relevance of objectives is the extent to which the project’s objectives are consistent with the country’s current development priorities and with current Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, Country Assistance Strategies, Sector Strategy Papers, Operational Policies). Relevance of design is the extent to which the project’s design is consistent with the stated objectives. Efficacy is the extent to which the project’s objectives were achieved, or are expected to be achieved, taking into account their relative importance. Efficiency is the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared to alternatives. The efficiency dimension generally is not applied to adjustment operations. Possible ratings for Outcome: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Risk to Development Outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized). Possible ratings for Risk to Development Outcome: High, Significant, Moderate, Negligible to Low, Not Evaluable. Bank Performance: The extent to which services provided by the Bank ensured quality at entry of the operation and supported effective implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of supported activities after loan/credit closing, toward the achievement of development outcomes. The rating has two dimensions: quality at entry and quality of supervision. Possible ratings for Bank Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Borrower Performance: The extent to which the borrower (including the government and implementing agency or agencies) ensured quality of preparation and implementation, and complied with covenants and agreements, toward the achievement of development outcomes. The rating has two dimensions: government performance and implementing agency(ies) performance. Possible ratings for Borrower Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. xi Preface This Project Performance Assessment Report (PPAR) assesses a primary education project in Nepal, the Education for All Project 2004-2009 (EFA). The project was implemented through a Sector Wide Approach (SWAp), Nepal’s first SWAp in the education sector. The World Bank Group was one of five original pooling donors. Total project cost was estimated at US$664 million. The Government of Nepal was to finance US$479 million. By project closing, the Government’s commitments had risen to US$703 million. The World Bank approved its contribution on July 8, 2004 with an IDA credit of US$50 million. Due to substantial cost increases Additional Financing was granted in 2007. With the addition of an IDA grant of US$60 million, the Bank’s final contribution to the project was US$110 million. The report was prepared by Ann Elizabeth Flanagan, Economist, IEG. The findings are based on a three week field mission to Nepal from May 15, 2014 to June 8, 2014 which was conducted concurrently with an evaluation mission which contributed to IEG’s evaluation of World Bank support to Early Childhood Development (ECD). The mission visited the districts of Dhanusa, Kaski, Kathmandu, Kavre, and Mahotari and met with a range of stakeholders: government, donors, civil society, School Management Committees, Parent Teacher Associations, Head Teachers, Teachers, ECD Facilitators, parents, and community members. Ms. Sangeeta Rana helped facilitate meetings in the field. IEG is grateful for the cooperation and assistance provided by all the concerned stakeholders and the support provided by the World Bank country office staff in Kathmandu. Following standard IEG procedures, a copy of the draft PPAR was sent to the relevant government officials and agencies for their review and feedback, and comments received from the Government have been included in Annex C. xiii Summary This Project Performance Assessment Report evaluates the contributions to educational achievements made in Nepal as part of the World Bank’s Education for All Project 2004- 2009. The Education For All Project represents the World Bank’s financial support to the Government’s Education For All Program. This is the first time the Independent Evaluation Group has conducted a PPAR on an EFA funded project financed through a Sector-wide Approach (SWAp). Context Nepal is a demographically and geographically diverse country. The Central Bureau of Statistics counted 125 castes and 123 mother tongue languages spoken among its 26.5 million people (Government of Nepal, 2011). Among the country’s four ecological zones and five development regions, the geography ranges from rugged mountainous terrain to forests to fertile plains. Nepal’s topography, traditions, ethnic, and linguistic diversity have contributed to isolating certain population segments from access to basic services. Girls, Dalits, and disadvantaged Janajatis are among the most disadvantaged groups in terms of access to education services. Trends in education improvement prior to the EFA project had been mixed. Significant gains had been achieved in net primary enrollment rates, which increased from 69 percent in 1999 to 84 percent in 2003. Drop-out and survival rates had improved, yet repetition rates remained high. Little progress had been made on eliminating the gender gap in access to primary education. Results of student learning as measured by tests of student achievement indicated low performance at the primary school level. In terms of subgroups, boys and girls performed equally well. Results were not disaggregated by caste, ethnicity, or income group making differential performance among these groups and more advantaged students impossible. Data from performance on the School Leaving Certificate (SLC) examination taken after completing 10th grade suggested private school students – those whose families have the means to pay for an education – were learning more. The EFA project was designed to address gaps in educational achievement in access, equity, quality, and efficiency. Project Design and Objectives The Project’s Development Objectives were to (i) ensure access and equity in primary education; (ii) improve the efficiency and institutional capacity of primary education; and (iii) enhance the quality and relevance of basic primary education for children and illiterate adults. The Education for All project continued and expanded the processes and systems put in place under previous Bank and donor support through the Basic and Primary School Project II and the Community School Support Project. Donors had been coordinating in Nepal’s education sector for more than two decades. The EFA SWAp helped to further coordinate their efforts. xiv Project activities were designed to be complementary and synergistic and many activities supported more than one intended outcome. Taking into consideration the relatively high overall level of enrollment, the project rightly emphasized increasing the demand for education among historically excluded and hardest to reach populations while continuing to expand and rehabilitate classrooms to accommodate increasing enrollments. Activities to improve school-level efficiency emphasized decentralized provision of education and empowerment of School Management Committees (SMC) and local communities, in particular decision-making authority and control over flexible resources. At the time of project approval, there was, however, little evidence of the effectiveness of school-based management. As well, there were ambiguities in the project’s definition of autonomy and the precise levels, ways, and means to achieve participation in decision- making. In particular, roles and responsibilities of SMCs were inconsistent between the Acts and Regulations and relevant Operational Directives and Guidelines (CERID, 2007). Accountability mechanisms were weak and support for capacity building limited. Capacity building activities for district and school level stakeholders incorporated into project design lacked a clearly defined and strategically prioritized capacity development plan. Low capacity at all levels and weak internal controls limited the SWAp’s appropriateness as a funding vehicle. Activities to address the quality of education included teacher training, privatization of textbook delivery (to improve the timely delivery of learning materials), early childhood development (to improve readiness to learn), and support to literacy programs (to achieve basic literacy for all). Improvements in the quality of education were also assumed to follow from activities supporting decentralization and improved accountability. A theoretical link between decentralization, accountability, and improved student learning had been established (World Bank, 2004a), but little empirical evidence of the effectiveness of school-based management existed at project approval. Overall, insufficient emphasis was placed on improving the quality of education under EFA. The project’s objectives were relevant at approval and remained relevant at project closing. The project’s objectives and activities supported the Government of Nepal’s strategy to achieve broad based economic growth, improvement of social services delivery, social inclusion of historically excluded social groups, and good governance. They were relevant in terms of Government priorities and commitments to global initiatives such as the Millennium Development Goals (MDG) and Education for All, Nepal’s educational context, and conflict mitigation. Democratic rule and demand for equitable access to public services were major concerns arising out of the “Peoples Movements.” While the EFA project was approved during the conflict, discussions in the literature are mixed on the net impact the conflict had on the provision of education and EFA’s implementation. The project was aligned with the World Bank’s strategies. At project approval, the Bank was supporting broad based economic growth and social sector development in Nepal; universal access to an inclusive, quality primary education system was emphasized. When the conflict ended and rebuilding began, in particular the writing of a new Constitution, the Bank strategies focused on foundation laying activities, such as education, upon which to build sustainable inclusive development. At project closure, xv education remained a priority as it served the dual purpose of fostering both economic development and peace-building. Project Results At project closing, the project had achieved its access and equity targets, obtaining gender parity and representative shares of historically disadvantaged students enrolling in primary education. A combination of new and improved learning environments, scholarships, increased parental awareness, and community involvement likely contributed to achieving improved access to education. On April 25, 2015, a 7.8 magnitude earthquake struck Nepal. The earthquake destroyed 30 percent of the basic primary subsector schools in the country. The Region reports that classes were resumed in temporary learning centers within a month after the earthquake. Student attendance remains an issue, especially for poorer students. There is little evidence of improved learning outcomes, which points to the need for more focused attention on the determinants of learning and improving the quality of education. Gains were made in repetition rates and drop-out rates in primary education. A number of factors have contributed to the improvement of the efficiency of the primary education system. Increased awareness, community participation, remittances, and a liberal promotion policy were all contributing factors. Other systemic inefficiencies such as the misallocation of teachers persisted. Evidence to show that institutional capacity has improved is scant. Capacity building activities were undertaken, but overall the project paid insufficient attention to systematically strengthening capacity, particularly at local levels. Capacity constraints continue to be a substantial concern. Efficiency is rated modest. The benefits derive primarily from the number of students completing fifth grade. Estimates may be overstated to the extent that calculations rely on Government reported enrollment and completion figures. Based on high relevance of objectives, modest relevance of design, modest achievement on project objectives, and modest efficiency, the project’s outcome is rated moderately unsatisfactory. Risk to Development Outcome is rated moderate. Political instability and fiscal constraints continue, however, the Government was quickly able to mobilize US$300 million in additional funding from donors for post-disaster reconstruction efforts. Bank supervision was strong and the Bank made a useful contribution to the project’s access and equity objectives (later affected by the earthquake), but the Bank underestimated local capacity and the consequent capacity building effort required to implement a complex project which rested on decentralized decision-making. Bank Performance is rated moderately unsatisfactory. xvi The Government of Nepal remains committed to education reform yet, recurrent implementation issues and a weak internal control system persisted throughout project implementation. Borrower Performance is rated moderately unsatisfactory. Lessons Based on the experience of this project, lessons for future education projects in Nepal and elsewhere can be drawn:  To improve the quality of education, in particular learning outcomes, it is important to emphasize the quality of learning inputs such as teachers and instructional and learning materials. There is little evidence to suggest inputs to the teaching learning process improved due to the project: the quality of ECD suffered from limited Government support; teachers received limited in-service training; and children still received textbooks after the start of the school year.  In a low capacity environment, a strategy to build local level capacity is critical in a decentralized service provision model. Low capacity limits the potential of various instruments under communities’ control to improve the efficiency and quality of schools. Under EFA, distribution of grants was often delayed due to overly complex rules and procedures which were not well understood at the school-level. Low capacity limited the impact of School Improvement Plans which were meant to focus on teaching and learning activities.  It is important that central, district, and local level roles and responsibilities are clearly defined when education provision has been decentralized. In Nepal, ambiguities existed as to where ultimate responsibilities for education decision- making lie. These ambiguities contributed to the failure of decentralization to bring about increases expected in the quality of public education.  Quality assurance in M&E data is essential when funding is linked to school-level data. Verification of the current system suggests irregularities in reporting data: scholarship distribution has not followed guidelines and the incentive to over- report enrollment data has increased with the introduction of per capita financing. Nick York Director Human Development and Economic Management 1 1. Background and Context Background 1.1 This Project Performance Assessment Report (PPAR) reviews the implementation experience and outcomes of World Bank support to the Government of Nepal’s Education For All (EFA) program (2004 – 2009).1 The World Bank has supported Nepal’s basic education sector for over two decades. Support for EFA is a continuation of the assistance provided through the Basic and Primary Education projects I and II and the Community School Support Project. This project was selected for a PPAR as part of a series of assessments of the Bank’s long-term engagement in Nepal’s education sector including the Basic and Primary Education Project II and Community School Support Project prepared by IEG in 2009 and 2010, respectively. This is the first time the Independent Evaluation Group has conducted a PPAR on an EFA funded project financed through a Sector-Wide Approach (SWAp). Socio-Economic Context COUNTRY CHARACTERISTICS 1.2 Nepal is a demographically and geographically diverse country. The Central Bureau of Statistics counted 125 castes and 123 mother tongue languages spoken among its 26.5 million people (Government of Nepal, 2011). The country is divided into five development regions (Eastern, Central, Western, Mid-Western, and Far Western) and four ecological zones (Mountains, Hills, Valley, and Terai). Among these divisions the geography ranges from rugged mountainous terrain to forests to fertile plains. The country is divided into 75 districts, comprised of 3,915 villages, 58 municipalities, and one metropolitan area. Nepal’s topography, traditions, ethnic, and linguistic diversity have had a hand in isolating certain population segments from access to basic services. Girls, Dalits and disadvantaged Janajatis are among the most disadvantaged groups in terms of access to education services. 1.3 Caste and ethnicity are not only highly correlated with access to services but also with resultant development outcomes, including poverty. Social inclusion, which requires equitable delivery of basic social services, formed the cornerstone of the “People’s Movement” in Nepal and a conflict that lasted for ten years (1996-2006).2 1 The Government of Nepal’s strategy in the primary education subsector was outlined in its Education for All Action Plan. The EFA Program is the financing provided by the SWAp donors to coordinate their activities in support of the Government’s strategy. The share of financing and activities undertaken by the World Bank in support of the EFA Plan will be referred to as either the EFA project (or simply the project). 2 The EFA project was approved during the conflict. Discussions in the literature are mixed on the net impact of the conflict on the provision of education. 2 ECONOMICS 1.4 Education and quality investments in human capital augment the growth process (Hanushek and Woessmann, 2012). Currently, Nepal is one of the world’s poorest countries (Table 1.1). Over the past decade, economic growth was fairly stable but low, averaging four percent per year. The economy depends primarily on agriculture and services (e.g., agro-processing and tourism). Low educational attainment, in addition to lack of electricity, and political instability constrain higher economic growth. Table 1.1. Selected Macro-Economic Indicators, 2000-2013 Region 2000 2003 2006 2009 2013 GDP per capita (2005 US$) 297 306 328 363 409 GDP growth rate (%) 6.2 3.9 3.4 4.5 3.8 Personal remittances, (% of GDP) 2.0 12.2 16.1 23.1 28.8 Agriculture, value added (% of GDP) 40.8 37.5 34.6 34.0 35.1 Manufacturing, value added (% of GDP) 9.4 8.4 7.8 7.2 6.6 Services, etc., value added (% of GDP) 37.0 44.3 48.2 49.6 49.2 Source: World Development Indicators. 1.5 The majority of the population works in the agriculture sector (approximately 64 percent). An estimated 500,000 youth enter the labor force each year facing limited job openings at home. As a result of low opportunities at home, migration for work is on the rise and remittances have increased substantially over time. Approximately 1.92 million young Nepalis were not living in Nepal at the time of the last Census. The majority of Nepal’s migrant workers abroad are employed in low skilled jobs. 3 POVERTY 1.6 Nepal has made significant progress in reducing poverty (Table 1.2). Progress on social inclusion in schooling was made over time and in every income quintile. The poorest benefitted the most but are still most likely to remain out of school. (Figure 1.1). Table 1.2. Poverty Indicators, 1995-2010 Figure 1.1. Population Age 6-24 Who Have Never Attended School, 2003-2010 1995 2003 2010 Extreme poverty 50 never attending school % of population 6-24 headcount ratio at $1.25 40 68.0 53.1 23.7 30 a day (PPP) (% of 20 population) 10 0 Poverty headcount ratio at $2 a day (PPP) (% of population) 89.0 77.3 56.0 2003 2010 Source: World Development Indicators. Source: National Living Standards Survey, 2003/04 and 2010/11. Education in Nepal 1.7 Nepal’s vision for education is to instill social values and equip its citizens with the skills and knowledge needed to compete in the global economy (Government of Nepal, 2005a). 1.8 At project appraisal in 2004, the primary and secondary education sector in Nepal was organized as follows: early childhood development and preprimary classes (ECD/PPE), primary school (grade 1-5), lower secondary school (grade 6-8), secondary school (grade 9-10) and higher secondary school (grades 11-12). ECD/PPE received little financial or technical support from the Government of Nepal. Basic education was considered grades 1-8. The Government provided universal free primary education. As of 2011 the mean years of schooling for the population 15 and above was 8.2 years (Government of Nepal, 2011). 1.9 At the time of EFA implementation, there were four broad categories of schools in Nepal: (i) institutional schools, which are privately funded (ii) community schools funded by the Government of Nepal; (iii) community managed schools, which are funded by the Government of Nepal but have chosen to formally transfer management of the school to the community; and (iv) unaided community schools, which may receive partial funding from the Government of Nepal. In this document we refer to these schools as private schools, community schools, community managed schools and unaided community schools. The last three types of schools received support under the EFA project. 4 EVOLUTION OF EDUCATION IN NEPAL 1.10 From 1846 to 1960, the education system in Nepal was limited to the ruling elites (Table 1.3). The Government and donors maintain that the decentralization reforms implemented under EFA were an extension of Nepal’s long history and tradition of community schooling. The tradition of community schooling began in 1951 when schools were opened by communities, however, these schools were restricted to the upper castes. Nepal’s experience with an “open” and officially decentralized public education system (e.g., open to all castes and communities) is relatively short. Given the remoteness of some schools, even under a centralized system, the day-to-day operation of schools remained with communities. 1.11 In 1971, with donor support, the Government centralized control over the education system and began funding public education; communities could establish schools and would receive Government support. Table 1.3. Evolution of Education Provision and Structure in Nepal Date Description Rule of the Rana party. Education is highly centralized and limited to the ruling 1846-1950 party and high caste elites. Popular democracy emerges out of the Rana period. The provision of education is 1951-1960 decentralized and School Management Committees are established. Community schools are established and managed at the local level with centralized control. 1960 Return to the Panchyat system. The community school system remains in place. Panchyat System introduces a centralized system of education, promulgates the Education Act of 1971 and introduces the National Education System Plan (with 1971-2001 the help of donors). The Act defines “community schools” and “institutional schools". All schools require government approval for establishment and operation. Only community schools receive regular funding from the Government. The 7th Amendment to the Education Act of 1971 decentralizes education and formally establishes School Management Committees (SMCs) in every community 2001 school “…for its operation, supervision and management…” Primary level education in community schools is legally free, including the provision of free textbooks. Source: Khanal (2011). TRENDS IN EDUCATION 1999 TO 2003 1.12 The Government, the World Bank, and other Development Partners have supported reforms to achieve access, equity, and efficiency in Nepal’s basic and primary public education system for two decades. The Bank has supported these objectives since 1999 through four projects: Basic and Primary Education Project (BPEP I); BPEP II; Community School Sector Project (CSSP), and EFA. Support continues with the School Sector Reform Project, the follow-on project to EFA. 1.13 Trends in education indicators between 1999 and 2003 were mixed. Few children were enrolled in preprimary school in 2003 indicating a late start to implementing early 5 childhood education in Nepal. Significant gains were made in net primary enrollment, which increased from 69 percent in 1999 to 84 percent in 2003. By 2003, Nepal performed well against other South Asian countries in terms of enrollments. Drop-out and survival rates had improved, yet repetition rates remained high. Little progress was made on eliminating the gender gap in access to primary education (Table 1.4). Table 1.4. Educational Outputs 1999-2003 Region 1999 2003 Preprimary enrollment (%) - 7.7 Gross enrollment rate 127.7 126.7 Net enrollment rate 69.0 83.5 Gender parity index 0.81 0.86 Repetition rate, primary 23.0 22.3 Drop-out rate, primary 42.0 10.1 Survival rate, grade 5 49.7 76.2 Source: World Development Indicators and Government of Nepal, 2012a. Note: Gender parity index calculated on gross enrollment. 1.14 On 2003 national tests of student achievement, fifth grade students answered half of all questions (52 percent) correctly, on average. At that time, boys and girls performed equally. Results were not disaggregated by caste, ethnicity, or income group making differential performance measurement among these groups and more advantaged students impossible. In 2004, performance on the School Leaving Certificate (SLC) examination taken after completing 10th grade indicated extremely low knowledge gained. The pass rate for public school 10th graders on the SLC was 38 percent compared to an 85 percent pass rate for their private school peers (Bhatta, 2005). This result would suggest students whose families have the means to pay for an education were learning more. 1.15 Achievements in primary education coincided with liberalization of private school policies and a surge in demand for private education – fueled by perceived higher quality as compared to public primary education (Carney and Bista, 2009). Currently, remittances are also driving private school enrollment. Fees to attend private schools exclude those who cannot afford to pay for schooling; students attending private schools tend to be from higher income families. REGULATION AND PROVISION OF PRIMARY EDUCATION 1.16 The central Government continues to make policies, fund, regulate, and supervise the education system in Nepal. At the local level, districts, schools, and local communities act as the implementing agencies of national policies and are expected to ensure local priorities for education are addressed. 1.17 Decentralization has been the driving force in education reform in Nepal. In 1999, Nepal passed the Local Self-Governance Act (LSGA) which effectively reinstituted community management of government services. LGSA devolved decision-making authority for education service delivery from the central government to District Development Committees (DDC) and Municipalities. 6 1.18 The 2001 7th Amendment to the Education Act of 1971 formalized the relationship between the central government and local bodies by renaming all schools “community schools” and re-establishing SMCs. Certain ambiguities exist between the LSGA and the Amendment to the Education Act. In the former, ultimate responsibility for education decision-making lies with the DDC while in the latter, more power is vested with the Village Education Committee (VEC) and decision-making authority was handed to SMCs. 1.19 Under both Acts, education planning was envisioned as a bottom up process. The 7th Amendment outlined a process of school-level education planning through school improvement plans (SIP) upon which higher level plans (i.e., village, district, national) were built. Since the dissolution of local bodies in 2002 (elections have yet to be held), Village Development Committees (VDC) and VECs have not been as instrumental in school planning as envisioned. Locally elected officials at the district level were replaced by government appointed civil servants. District education planning goes from the schools to the District Education Officer (DEO) who is essentially a central level employee. In effect, the system remains highly centralized. 1.20 Per EFA project documentation, SMCs have been given the authority to mobilize resources for schools, participate in the SIP process; approve the school’s annual budget; provide day-to-day operating, monitoring, and managing support to schools; arrange for the auditing of the annual school budget; present yearly financial statements to community stakeholders; and report on educational progress and planning. The community, SMCs, parents, and teachers are responsible for ensuring that block grants disbursed to schools are properly allocated and efficiently used (World Bank, 2004b; Independent Evaluation Group, 2010a; CERID, 2004; CERID, 2005). Both community schools and community managed schools continue to receive earmarked grant funding for teachers’ salaries as before but now also receive block grant funding which includes both the earmarked component and a non-earmarked component. The latter consists of a general block grant awarded to community managed schools (incentive grant for transferring), a SIP grant awarded to all community schools, and performance-based incentive grants for which all schools are eligible (Table 1.5). Table 1.5. Grant Funding of Schools in Nepal Earmarked Grants Block Grants REGULAR GENERAL SIP PERFORMANCE Teacher salaries, Grants to community scholarships, infrastructure managed schools; Per student development – new incentive grants; school allocation for Reward classroom construction and administrative school grants furniture, school external expenditures; stationary; improvement, free based on environment improvement bilingual education; textbooks and other performance (drinking water, fencing, primary school learning materials indicators and toilets, including improvement; and teacher support separate toilets for girls) educational materials and maintenance Source: CERID (2005a); Draft report on Financing Education in Federal Nepal prepared for the Education Financing Reference Group under Education and Federalism Support Group supported by UNESCO Kathmandu. 7 REMAINING CHALLENGES IN PRIMARY EDUCATION IN 2003 1.21 Since 1992, the Bank, in coordination with other donors, has supported the government in reforms focusing on three primary objectives: building institutional capacity in education at all levels of decision-making (e.g., central, district, and school); improving the efficiency and quality of education services; and improving equity in access to education, especially for girls and students from disadvantaged communities. 1.22 Despite continuous donor support and the improvements achieved toward universal access to primary education, at the time of project approval challenges remained in all four areas – education access, equity, efficiency, and quality. Nepal’s promise of free primary education had not come to pass; public schools continued to charge fees to students, reducing the demand for education especially among the poor. More than one-quarter of school age children remained out of school. Out-of-school children were disproportionately girls, Dalits, and other disadvantaged children. Repetition rates remained relatively high at the primary level and completion rates remained low, indicating inefficiencies in the system. The quality of public primary education remained low causing an outflow of children from families who could afford to send their children to private school, where quality was perceived to be higher (Government of Nepal, 2003a). Finally, central recruitment of teachers through the Teacher Service Commission (TSC) reduced the accountability of teachers to schools, contributed to an inefficient deployment of teachers (in favor of urban and semi-urban posts to remote rural areas), and created a mismatch between teacher subject knowledge demand and supply (World Bank, 2004b). 1.23 Donor support to the Government’s Education for All program was designed to address these challenges and to support the Government’s commitment to the global Education for All initiative. Nepal had committed to attain Education For All as a signatory to the Jomtien Declaration on Education for All (1990). Nepal recommitted itself to achieving its goal of universal access to basic and primary education by adopting the six goals introduced in the Dakar Framework for Action in 2000. In collaboration with the United Nation’s Educational, Scientific and Cultural Organization (UNESCO), the Ministry of Education3 (MOE) designed its National EFA Plan of Action which outlined the strategic framework to achieve each of the EFA goals by 2015 and forms Nepal’s primary education strategy to 2015. 2. Objectives, Design, and their Relevance Objectives 2.1 The objectives of the project as stated in the Development Credit Agreement (DCA) and the Financing Agreement were to ensure access and equity to primary education, improve efficiency and institutional capacity; and contribute to sustainable 3 Originally the Ministry of Education and Sports (MOES). 8 socioeconomic development and equity through the enhancement of quality and relevance of basic primary education to children and illiterate adults (in Nepal). 2.2 The project’s objectives encompass all six EFA goals: expanding and improving early childhood education, ensuring access to education for all children, meeting the learning needs of all children including indigenous peoples and linguistic minorities, improving adult literacy, eliminating gender disparity, and improving all aspects of quality education. Emphasis on linguistic minorities and mother tongue education was an additional requirement added by the Government of Nepal. 2.3 The project had three components – improving access and equity; enhancing quality and relevance; and improving efficiency and institutional capacity.4 The first component, improving access and equity supported infrastructure development, financial aid to disadvantaged students, and non-formal education to out-of-school children. The component was comprehensive, targeting physical and financial barriers to education for poor girls, disadvantaged students, and the hard to reach school-age population (e.g., in remote areas such as the Karnali region). The second component, enhancing quality and relevance, involved overall improvements in children’s learning environments through facilities up-grades and the expansion of programs to prepare teachers and students for quality classroom interactions. Regularly scheduled, school-based, in-service teacher training for primary school teachers was supposed to assist teachers transfer knowledge to students, while early childhood development programs in remote and disadvantaged areas and mother-tongue language support was supposed to improve children’s readiness to learn. The third component, improving efficiency and institutional capacity, aimed to decentralize the provision of education in Nepal through an expanded block grant program and community school management to improve all aspects of schooling at the local level (i.e. access, equity, and quality) through gains in efficiency. Table 2.1 describes the project’s components and activities. Table 2.1. Education for All Program Components 3. Improving Efficiency Components 1. Improving Access 2. Enhancing Quality and Institutional and Equity and Relevance Capacity Constructing and Establishing 11,000 new Decentralizing education rehabilitating 8,000 early childhood service provision, directly classrooms, including development centers funding recurrent and temporary classrooms targeting remote and non-recurrent costs nationwide disadvantaged areas through block grants to schools Activities Constructing toilets, Strengthening recurrent Training and capacity separate toilets for girls, in-service teacher building for school and safe drinking water training management committees, in schoolsa head teachers and community organizations to enable effective grant 4 The PAD, Annex 4, presents the six EFA goals as the project components. 9 use Providing flexible Supporting participatory Providing one time block schooling options for school planning and grants to schools remote areas direct funding to schools voluntarily transferring to community school management Providing scholarships Supporting mother Providing additional to 50% of poor girls, tongue learning and home block grants to accredited Dalits, indigenous to school transition for schools children, and other non-Nepali speaking typically excluded children (half of children to attend predominantly minority primary school schools) Expanding literacy Conducting audits of programs for 108,000 block grants use (by women and skills parents) training for 10,800 women from traditionally excluded castes and ethnic minorities Expanding school-based non-formal education programs a Listedas an activity under Component 1 and Component 2. Source: PAD PROJECT INSTITUTIONAL FRAMEWORK 2.4 The project was implemented as a Sector Wide Approach (SWAp) – the first in Nepal’s education sector (Box 2.1). The project disbursed against the Government’s Annual Work Program and Budget (AWPB) and Annual Strategic Implementation Plan (ASIP) for the primary education sector, jointly determined by Development Partners and the Government. 2.5 The SWAp arrangement included pooling funds for regular and development expenditures; resources were pooled with government budgetary resources. The SWAp had five original pooling members: the Department for International Development (DFID), Danish International Development Assistance (DANIDA), Norwegian Agency for Development Cooperation (NORAD), Finland Ministry of Foreign Affairs, and Australian Agency for International Development (AusAid). The Asian Development Bank (ADB) and the United Nations Children’s Fund (UNICEF) joined as pooling donors bringing the total to seven by project closing. Parallel funding was provided by non-pooling donor agencies including the Japan International Cooperation Agency (JICA), World Food Program (WFP), and UNCESCO. 2.6 The nature of the SWAp financing arrangement made it impossible to distinguish IDA expenditures by activity or to breakdown IDA commitments and disbursements by component. 10 Box 2.1. The EFA Sector-Wide Approach Donors supported the Government of Nepal’s EFA National Plan of Action (NPA) (2001-2015) through a Sector-wide Approach (SWAp). The Education for All (EFA) 2004-2009 SWAp was introduced to support an integrated, holistic framework for education reform versus a piecemeal approach focusing on either school infrastructure, teacher development, or curricular development in isolation of the primary school system (Asia South Pacific Association for Basic and Adult Education, 2010). The approach was developed in response to perceived ineffectiveness and inefficiency of project-based development assistance due to fragmentation of effort and heavy managerial burden placed on the Government of Nepal. The EFA SWAp was seen as a means not only to move away from a traditional project approach but also to generate more support for recurrent costs (e.g., teachers’ salaries); facilitate donor coordination and harmonization; and lower transaction costs. In studies of health and education SWAps, Vaillencort (2009 and 2012) found evidence supporting the claim of improved donor harmonization along the dimensions of a country-led partnership and use of national systems but little evidence to support reduced transaction costs. Only modest evidence of improvements in the efficiency of resource allocation (improved sector stewardship) was found. A realistic, evidence-based strategy, national and local capacity, inclusive partnerships, and predictable funding were associated with the effectiveness of the approach, some of which were missing in Nepal (e.g., evidence base, capacity). Relevance of the Objectives and Design RELEVANCE OF OBJECTIVES 2.7 The relevance of the project's objectives was rated high. The objectives were relevant at the time of approval and remain relevant today. Significant progress in access to primary education had been made in the period prior to this project – primary net enrollment rates had increased but the hardest to reach student populations remained out of school. For example, both enrollment and retention of girls, Dalits, and other disadvantaged students remained lower than their more advantaged peers (Government of Nepal, 2011). High repetition and drop-out rates were symptomatic of the inefficiency and low quality of the primary education system (see Section 1). The project’s objectives sought to address the remaining challenges in the primary education sector and to attain universal primary education, defined as a quality primary education for all relevant children and an equivalent level of literacy for adults. 2.8 The objective was fully aligned with the Government of Nepal’s national priorities. The objective was consistent with the goals of the Government’s Tenth Plan (2002-2007) to improve broad-based economic growth, social sector development, social inclusion, and good governance. The objectives supported the Tenth Plan’s over-arching goal to reduce poverty through inclusive and equitable growth and improved processes for service delivery (HMGN, 2003c). Post-conflict, the project remained consistent with the Government’s objectives as outlined in its National Development Strategy Paper (2008) and successive Three Year Interim Plans, the most recent of which (2011-2013) remained committed to improving the living standard of all Nepalese people through equitable and sustainable economic growth. Education remained a priority sector to achieve poverty alleviation, sustainable peace through employment-centric, inclusive, 11 and equitable economic growth (Government of Nepal, 2010a). The objective remains highly relevant in the context of the Government’s basic and primary education strategy and the National EFA Plan to 2015. 2.9 At approval, the objective was fully aligned with the Bank’s Country Assistance Strategy (CAS) in Nepal (2004-2007), which supported broad based economic growth and social sector development. Universal access to an inclusive, quality primary education system was emphasized in the CAS. The Interim Strategy Note in 2009 adopted more flexibility and selectivity given the post-conflict situation. The Bank committed to the over-arching goal to consolidate peace and promote development. The project’s objective supported the Bank’s objectives to promote capable state structures and systems, foster accountable institutions, and to lay the foundation for sustainable inclusive development (World Bank, 2009). Education served the dual purpose of fostering both economic development and peace-building. At project closure, the objective remained relevant to the Bank’s strategy in Nepal. Equitable access to education remains a priority for the Bank’s goal to increase inclusive growth and opportunities for shared prosperity (World Bank, 2013). RELEVANCE OF DESIGN 2.10 The relevance of project design is rated modest. Project activities were designed to be complementary and synergistic and many activities supported more than one intended outcome. There was a theoretical link between project inputs and intended outcomes. However, project design was ambitious and complex in a low capacity environment like Nepal. 2.11 Project activities were relevant to achieve the first objective of increased access and equity to primary education. Given the improvements in net enrollment rates between 1995 and 2003 expanding and rehabilitating classrooms to improve the availability of education was highly appropriate. Taking into consideration the relatively high overall level of enrollment, the project rightly emphasized increasing the demand for education among historically excluded and hardest to reach populations. 2.12 The design for the objective to improve efficiency and institutional capacity had shortcomings. Activities to improve school-level efficiency emphasized decentralized provision of education and empowerment of school management committees and local communities, in particular decision-making authority and control over flexible resources. The necessary conditions for successful school-based or site-based management were not fully met (CERID, 2007; Barrera-Osoria, Felipe, et al. 2009). There were ambiguities in the definition of autonomy and the precise levels, ways, and means to achieve participation in decision-making. In particular, roles and responsibilities of SMCs were inconsistent between the Acts and Regulations and relevant Operational Directives and Guidelines (CERID, 2007). The accountability mechanisms were weak and support for capacity building limited. There was some ambiguity in the need for incentive grants when all schools were required to establish SMCs. The project scaled-up school improvement planning tied to non-earmarked Government grants disbursing directly to schools however, the amount of flexible funding was small limiting the potential benefits of community control over resources. The ambiguities between formally transferred 12 community schools and regular community schools have been noted (Independent Evaluation Group, 2010). Finally, little evidence of the effectiveness of school-based management existed at project approval, as the project was designed to scale-up the model before the pilot project was completed and evaluated. 2.13 Capacity building activities for district and school level stakeholders were incorporated into project design, however, the project lacked a clearly defined and strategically prioritized capacity development plan. Institutional capacity was poorly defined and no outcomes were provided to measure attainment toward the objective. This was a significant shortcoming in design in a low capacity environment like Nepal. 2.14 Low capacity at all levels and weak internal controls limited the SWAp’s appropriateness as a funding vehicle. Concerns remain about central, district, and local (e.g. school management committees) capacity to implement the SWAp. There are also concerns about sustainability, as these processes are significantly donor driven (Asia South Pacific Association for Basic and Adult Education, 2010). 2.15 Project activities were relevant to achieve the objective to enhance the quality and relevance of basic primary education to children and illiterate adults, with the above qualification on school based management. The link between project inputs and intended outcomes was generally clear. Activities addressed quality from several different angles such as teacher training, privatization of textbook delivery (to improve the timely delivery of learning materials), early childhood development (to improve readiness to learn), and support to literacy programs (to achieve basic literacy for all). Given the slow progress in improving the quality of education with continued donor support, insufficient emphasis was placed on improving the quality of education. Activities supporting early childhood development focused on establishing of centers with little emphasis on quality of provision. Activities to support adult literacy were insufficient for the attainment of the objective. 2.16 Improvements in the quality of education were also assumed to follow from activities supporting decentralization and improved accountability. While a theoretical link between decentralization, accountability, and improved student learning had been established (World Bank, 2004a), little empirical evidence of the effectiveness of school- based management existed at project approval. Implementation 2.17 The project was approved on July 8, 2004, became effective on August 27, 2004, and closed on January 31, 2010. There were no extensions and the project closed on time but at significantly higher cost than at appraisal. 2.18 Project Costs. At closing, total project cost amounted to US$989.3 million. The Government financed the majority of the project – US$703 million. Donor support totaled US$286 million. Initial funding for the project was through an International Development Association (IDA) credit of US$50 million. Additional Financing of US$60 million in the form of an IDA grant was approved in November, 2007. Additional grant financing was approved to accommodate the Government's commitment to provide free 13 primary education which required: more classroom space to accommodate increasing enrollment; an increase in the number and amount of scholarships for girls and Dalits; additional teachers to reduce high student-teacher ratios in some regions; an unexpected 27 percent increase in permanent teachers’ salaries; and an 11 percent fluctuation in exchange rates (World Bank, 2007a). 2.19 Project costs exceeded appraisal costs. Table 2.2 presents the appraisal and actual financing from IDA, pooled donors, and the Government. Table 2.2. Appraised and Actual Costs of the Education For All Program 2004-2009 Appraisal Additional Actual (US$ millions) (US$ million) (US$ millions) IDA 50 60 111.3 Other Pooled Donors 100 75 175 Government 479 224 703 Total 629 359 989.3 Percent IDA 8 17 11 Source: PAD and ICR 2.20 Implementation Experience. The project’s primary Implementing Agency was the Department of Education (DOE) working in close consultation with local education offices. Most of the project’s activities were implemented by District Education Officers (DEO) and schools. 2.21 The main implementation challenge during EFA was insufficient capacity at all levels. Local level capacity building activities did not provide enough support to DEOs, Head Teachers, teachers, parents, or communities to fulfill the new requirements imposed under EFA. The Mid-Term Review concluded that simplification of the processes for grant allocations, clear guidance on school improvement plans, and measures to improve accountability were needed (DANIDA, 2006). Activities to address capacity constraints were proposed post-Mid Term Review but the project continued to concentrate resources on improving access, equity, and quality without sufficiently addressing capacity development. 2.22 Fiduciary. Financial management was considered high risk at project appraisal. IDA and other SWAp partners jointly financed (along with Government finance) the Government’s EFA Program, including both development and regular expenditures. Each year, a mutually agreed upon Annual Strategic Implementation Plan (ASIP) and accompanying Annual Work Program and Budget (AWPB) for the sub-sector were determined by donors and the Government of Nepal. IDA funds financed a percentage of all eligible expenditures covered under the AWPB. IDA funds were disbursed into a Foreign Exchange account with Nepal Rastra Bank. 2.23 The World Bank took the lead among donors on financial management in this SWAp. Financial Management remained a concern throughout implementation. Weak internal controls and low capacity were major shortcomings in financial management. The Office of the Auditor General consistently highlighted shortcomings in financial 14 reporting. Audited financial reports highlighted areas of improvement including: integration of financial reporting with the Financial Comptroller General’s Office (FCGO); capacity for financial reporting at the school level (often done by head teachers or teachers with no financial or accounting training); alignment of planned donor allocation and expenditures and actual allocations and expenditures; elimination of ineligible expenditures, such as over-disbursements of school grants for teachers’ salaries; timeliness of late audits and financial reporting; and compliance with social audits. 2.24 The Bank commissioned a detailed Financial Management and Procurement Review at the Mid-Term Review and DFID undertook another assessment in 2007. The Government of Nepal produced a Financial Management Action Plan which was accepted by the Bank and donors. The DOE made slow progress toward implementing the Action Plan. 2.25 Safeguards. The project was classified as a Category B project. Two safeguards were triggered: environment (OP 4.01) and indigenous peoples (OP 4.10). 2.26 Environmental impacts were assumed to be minimal given that the majority of construction would entail rehabilitation of classrooms on existing school properties. An Environmental Management Plan and National Environmental Guidelines (His Majesty’s Government of Nepal, 2004b) were prepared for school improvements. An Environment Assessment was not conducted although there was some concern that proposed new construction would be located in forests or other protected areas. Concerns were also raised that construction prove resilient to natural catastrophe e.g., to ensure construction in earthquake prone areas was compliant with safety guidelines. 2.27 A Vulnerable Communities Development Plan (VCDP) was prepared to address possible negative impacts on the learning needs of indigenous peoples which recommended home-to-school transitional support programs, recruiting bilingual female teachers in areas with a majority of linguistic minority students and families, disaggregation of education statistics by gender and ethnicity, and efforts to enhance partnerships between SMCs and the communities. The VCDP was consistent with the Government’s Indigenous People’s Development Plan. All VDCP recommendations were incorporated into project design. 2.28 The project in-and-of itself was considered a mitigating factor for OP 4.10 given its strong focus on removing barriers to educational opportunities and emphasis on social inclusion especially of individuals who had historically been excluded from education based on their gender, caste, ethnicity, or other considerations. 3. Achievement of the Objectives 3.1 The project’s design relied on a set of mutually reinforcing activities to achieve three outcomes: ensuring access and equity in primary education; improving efficiency and institutional capacity; and enhancing the quality and relevance of basic primary education to children and illiterate adults. 15 Ensuring Access and Equity in Primary Education 3.2 The project’s results framework was designed to measure access using net enrollment rates by gender, caste, and ethnicity (Table 3.1). The two primary activities under increasing access and equity were construction, rehabilitation, and improvement of physical sites and provision of scholarships to disadvantaged students. 3.3 Prior to the earthquake, output targets were met or exceeded (Table 3.2). National survey data support the evidence of increased enrollment and achievements in gender parity. National survey data also indicated progress toward equity using outcome measures by income quintile (Figure 3.1 and Table 3.4). 3.4 However, the 2015 earthquake destroyed approximately 30 percent of the 2011 stock of community schools, and destroyed or damaged 40 percent of the stock of classrooms in the basic primary subsector (including facility improvements).5 In light of the Region’s comments that “In most parts of the country, classes were resumed in newly constructed transitional/temporary learning centers within a month after the earthquake…” the achievement of this objective is rated substantial. Table 3.1. Results Chain for Obtaining EFA Access and Equity Project Goals  Higher net enrollment rates in primary education Final Outcomes  Gender, case, and ethnic parity in primary enrollment Intermediate  Reduction in the share of out-of-school girls, Dalits, and children from Outcomes disadvantaged Janajati groups  Number of new, rehabilitated classrooms (permanent and temporary designed appropriately to local conditions (including temporary), meeting minimum standards of quality for example, providing toilet facilities, safe Outputs drinking water, fences, desks, etc.  Number of scholarships distributed to poor girls, Dalits, disadvantaged Janajatis, and other marginalized groups. Inputs  Government, donor, and community resources and support to education (Access/Equity)  Government policies in the education sector Source: PAD. 5 These figures were calculated based on damage estimates in the Nepal Earthquake Post Disaster Needs Assessment Report (World Bank, 2015) and the stock of existing schools and classrooms as of 2010-11 (Government of Nepal, 2012). On April 25, 2015, a 7.8 magnitude earthquake struck Nepal along with several aftershocks. Fourteen districts in the Kathmandu Valley were heavily impacted by the earthquakes. In total 8,242 community schools were impacted by the earthquake: 26,080 classrooms were completely destroyed and another 26,090 classrooms partially destroyed. Further damage to toilets, sanitation, and compound infrastructure assets was estimated to be around 6,200 in number. Recovery and reconstruction estimates in the basic education sector are estimated at US$361 million. The number of EFA funded schools destroyed or damaged in the earthquake was requested but not available as of November 2015. 16 OUTPUTS 3.5 Prior to the earthquake, construction and rehabilitation of classrooms targets had been exceeded. Overall, the project supported the construction of classrooms throughout the country on a cost-sharing basis with communities. Construction grants were provided to schools through DEOs; schools were responsible for contracting and monitoring civil works under overall supervision of District Education Officers. The project prioritized building and rehabilitating classrooms and providing essential services to enhance the learning environment of students and to accommodate enrollment and attendance (i.e., improving school sites through additions of drinking water, toilets and separate toilet facilities for girls). 3.6 The project targeted building 8,000 new classrooms. An unforeseen shortfall of classrooms was identified during implementation, stemming from the lack of primary schools in some areas and rising enrollments in others. This led to significant increases in expenditures related to construction. The project constructed over 17,000 classrooms, exceeding its target. 3.7 Similarly, the targeted number of 6,000 rehabilitated classrooms was exceeded with 9,379 rehabilitated prior to the earthquake. The project exceeded its target to improve 5,000 school sites by providing fencing, water, and toilet facilities to 7,934 schools (Table 3.2). Although the baseline and achievement data are not strictly comparable, in 2010, 94 percent of classrooms were useable and in need of no repairs. No data were available on classrooms meeting the minimum conditions outlined at project approval. 3.8 Scholarship distribution for girls was close to targets but fell short of targets for Dalits. Table 3.3 presents the annual number of scholarships reported. Scholarships were the primary vehicle to incentivize poor girls and disadvantaged students to enroll in and stay in school. Caste and ethnicity are highly correlated with income; scholarships to these groups were also used to improve equity in access to education. The project document set an annual target of 650,000 scholarships per year (World Bank, 2004b). However, actual targets set by the ASIP varied by year. For example, the target for girls’ scholarships ranged from 560,000 in 2005 to 829,000 in 2009. 3.9 There were shortcomings in the use of scholarship to increase enrollment. The scholarship incentive mechanism was complex. Scholarship money was disbursed as part of earmarked grants to schools, however, the rules for allocation and distribution were not well understood causing delays in disbursement at the school level. Overall responsibility for identifying scholarship recipients and distributing and monitoring scholarships was assigned to the SMCs. In practice, the amount disbursed was determined by the District Education Officer based on enrollment data provided by the school. 17 Table 3.2. Outputs Associated with Increased Access and Equity in Basic and Primary Education Output baseline target† achievement 2003 2009 2010 Construction of new classrooms 0 8,000 17,954 Rehabilitation of classrooms 0 6,000 9,379 Improvement of school sites 0 5,000 7,934 Share of schools with required number of classrooms of acceptable standards 52.5a 82.0 93.6b Share of schools with safe drinking water 59.4a 90.0 77.0 Share of schools with girls’ toilets 37.5a 80.0 65.0 Source: PAD and Government of Nepal (2012) †No revision to the targets was made even though additional financing was granted. The flexible nature of developing ASIPs and setting the AWPB along with the shortfall of classrooms reported in Implementation Status Reports suggests the these targets may have changed. For example, for the 2007/08 school year, the Government’s targeted number of new classroom construction was 11,568 of which 4,670 were built (Government of Nepal. Flash II 2007-08). a Based on 2001 EMIS data. b Not strictly comparable to the baseline figure. The share refers to classrooms that are usable, adequate, and not in need of repair or renovation. 3.11 Reports, supervision documents, and discussions with stakeholders both in Kathmandu and in the field suggested that scholarships were insufficient to cover the opportunity cost of attending school. Originally the scholarship was set at 250 Nepalese Rupees (NPR) per year, later the amount was raised to 350 NPR per year. A 2006 study reported the direct costs to parents of a primary education were NPR 819 per year (books, fees, etc.) and that the value of household chores done by primary school aged child was as much as NPR 4,607 per year (Acharya and Luitel, 2006). Table 3.3. Number of Scholarships Distributed, 2005-2009 Cumulative Scholarships 2005 2006 2007 2008 2009 Target Total Girls 574,196 656,018 n/a 512,851 761,939 2,649,100 2,622,978 Dalits 521,064 624,896 n/a 630,825 818,317 2,636,766 2,477,128 Note: Data for 2007 were not available (n/a). Source: Government of Nepal, Implementation Progress Reports and Flash Reports. 3.12 During field visits, some stakeholders told IEG that the actual number of scholarships distributed to students were of smaller amounts and likely greater in number than the officially reported numbers (Table 3.3). Scholarships for girls and Dalits were distributed based on criteria other than those outlined by the project. Scholarships to promote enrollment, attendance, and retention of excluded groups were sometimes distributed based on students’ academic merit or distributed to all girls, since distribution to only 50 percent of girls seemed unfair, or distributed to students outside targeted groups based on teachers’ selection criteria (Acharya and Luitel, 2006). Receiving school uniforms or stationary instead of money was also found. 18 OUTCOMES: WAS THERE ACHIEVEMENT IN ACCESS AND EQUITY? 3.13 Nepal made substantial progress toward enrollment and inclusion. By 2010, access to primary education had substantially increased, gender parity was achieved in primary education, and improvements were made in access to primary education for otherwise disadvantaged students. Equity in educational outcomes by income improved (Table 3.4). Table 3.4. Primary Education Access and Equity Outcome Indicators (Percent) baseline target achievement Access 2003 2009 2010 Net enrollment, total 83.5 96 94.5 Net enrollment, girls 77.5 96 93.6 Net enrollment, boys 89.4 96 95.3 Share of out-of-school girls (%) 22.0 4 6.4 Gender parity indexa 0.87 n/a 0.98 Equity Income parity indexb .59 n/a 0.91 b Income parity index, girls .49 n/a 0.91 a Basedon Net Enrollment Rate. b Basedon the Net Enrollment Rates of the poorest and richest income quintiles, NLSS 2003/04 and 2010/11. Source: Government of Nepal, Ministry of Education, Department of Education, 2011. 3.14 Reliability of education data remains a major concern. Net enrollments rates as reported by the Education Management Information System (EMIS or Flash Reports) differ substantially from those reported by National Living Standards Survey (NLSS) data (Figure 3.2). The NLSS reports enrollment rates approximately 15 percentage points lower than those in Flash reports. By either source, net enrollment for girls increased substantially over the project period. NLSS data show a slight decrease in the net enrollment rate for boys over the project period. Net enrollment in 2010 according to NLSS was 78 percent; higher for girls (80 percent) than boys (77 percent). Flash data report 95 percent net enrollment for both boys and girls. These data are not strictly comparable but the difference is something to note especially given concerns over the quality of Flash data.6 6 Recent evidence has shown that there are issues with the validity of the reported data. Per capita financing creates an incentive to inflate enrollment figures. Technical reviews, CERID research, commissioned studies, and stakeholder interviews indicated a tendency to over-report student numbers. Recent data from a Public Expenditure Tracking Survey (PETS) conducted as part of the School Sector Reform Project suggests enrollments are inflated by 8 percent, confirming reports of “ghost” students – students who do not exist but still figure into textbook and PCF grants (World Bank, 2014). 19 Figure 3.1. Trend in the number of Figure 3.2. Improvement in net students enrolled in primary school enrollment rate, primary education 2.5 100 Number of students (millions) Net enrollment rate 2 90 80 1.5 70 1 60 0.5 50 40 0 1995/96 2003/04 2010/11 2003 2004 2005 2006 2007 2008 2009 2010 2011 Boys, NLSS Girls, NLSS Girl Boy Total, NLSS Total, EMIS Girls, EMIS Boys, EMIS Source: Government of Nepal, 2011. Note: Enrollment data may be inflated. Source: National Living Standards Survey 1995/96, 2003/04, and 2010/11; Government of Nepal, 2011. Note: EMIS enrollment data may be inflated. 3.15 Figure 3.1 also shows that the number of students enrolling in community schools rose and fell over the project period, especially for boys. The number of students was higher overall at project closure. 3.16 Gender parity is widely recognized as having been achieved. By any data source, the enrollment of girls has increased substantially (Figure 3.2). The share of out- of-school girls has decreased substantially from 22 percent to 6 percent (Table 3.4).7 Enrollment of boys has stabilized or decreased slightly between 2003 and 2010. The stabilization in boys’ enrollment may stem from already high rates of enrollment. 3.17 Parity in enrollment across income groups was also achieved. (Table 3.4). In 2003, fewer poor children were enrolled in primary school than rich children. By 2011, students in the lowest income quintile had almost identical enrollment rates as those in the highest income quintile. The pattern of enrollment rates by income quintile suggests the project was successfully pro-poor and improved equity in enrollment. 7 Over the same period the proportion of out-of-school boys was reduced from 10 percent to six percent. 20 Figure 3.3. Number of Dalit/Janajati Figure 3.4. Share of Dalit/Janajati enrolled students enrolled in primary education in primary education 2,500,000 Dalit, girl Dalit, boy Number of Dalit, Janajati students 2,000,000 Janajati, girl Janajati, boy 50% 1,500,000 45% % Minority enrollment 40% 1,000,000 35% 500,000 30% 25% - 20% 15% 10% 5% Total Dalit enrollment, primary 0% Total Janajati enrollment, primary Source: IEG calculations based on Flash Consolidated Report Source: IEG calculations based on Flash Consolidated Report 2011. 2011. Note: Enrollment data may be inflated. Note: Enrollment data may be inflated. 3.18 Primary education had become more inclusive as of 2009. Good progress was made toward achieving overall population representative enrollments by caste and ethnicity (Figures 3.3 and 3.4). However, the extent to which Dalit and Janajati students were included in primary education varied by district (Norad, 2009). 3.19 Net attendance rates were considerably lower than enrollment. This raises concerns about the achievement of a larger access-related objective, especially among the poor where attendance is lower still, which has implications for equity (Figure 3.5). The Midterm Evaluation for the School Sector Reform Program reports a net attendance rate of 78 percent in 2010 (Flash data). Survey data suggest net attendance rates between 66 and 69 percent (World Bank, 2014). Low attendance points to an area that Nepal will need to address in order to boost low student achievement. The pattern of net attendance rates suggests an area to focus on within attendance is equity. 3.20 There are factors outside the project that may have impacted access and equity in primary education. Education is a normal good and increases in income increase the demand for education. Over the project period Nepal experienced increases in remittance inflows. While remittances are largely believed to increase expenditures on private schools, the increase in income has likely impacted public school enrollment and retention as well, especially for poorer families. There is some evidence that suggests much of Nepal’s success in improving its human development outcomes, at least during the conflict period, was a result of remittances (World Bank, 2006). 21 Figure 3.5. Net Attendance Rate in Primary School, 2011 Net enrollment by income quintile 100 90 80 76.4 72.9 65.7 67.8 66.5 68.8 70 60 50 40 30 20 10 0 Total Poorest Second Third Fourth Richest Total Source: IEG calculations based on Flash Consolidated Report 2011. Improving Efficiency and Institutional Capacity 3.21 Achievement of improving efficiency and institutional capacity is rated modest. Institutional capacity received little attention. Gains were made in improving internal efficiency, however, it is difficult to connect these gains with project activities. 3.22 Under the Education For All project, the Government of Nepal introduced and strengthened a number of reforms to improve efficiency and institutional capacity. The kernel of this objective was to decentralize decision-making powers to the local level, in particular to the SMCs. This was to be achieved through two channels: (i) control over education resources and (ii) control over school management. The results chain is presented in Table 3.5. 3.23 In both cases, the underlying assumption was that education resources would be used more efficiently when individuals closest to the schools decided how to improve educational performance. Due to the diversity of community characteristics in Nepal, local solutions were assumed to be the best way to boost efficiency at the school level through more efficient allocation of resources, increased accountability of resource use (e.g., teachers), and improved transparency. Improvement in institutional capacity was critically linked to the achievement improved efficiency through decentralized decision- making. 3.24 Project activities also focused on improving efficiency in the early grades through expansion of ECD to reduce the number of children under the age of six attending first grade classes (see discussion under quality below, for achievement of ECD outcomes). 3.25 Capacity building was not well-defined in project documents. The institutional capacity objective was pitched at the output level; no outcome measures were defined. 22 Table 3.5. Results Chain for Obtaining EFA Efficiency and Institutional Capacity Project Goals Final Outcomes  Reduced repetition and drop-out rates  More students enter grade one with ECD/PPE experience, are more likely to complete grade one, and less likely to repeat at the grade one Intermediate level Outcomes  School Management Committees actively engage in SIP process  Community participation increases  Number of ECD/PPE classes established and additional number of children completing an ECD program  Number of schools with School Management Committees  Number School Management Committees, Head Teachers and community organizations trained to enable effective grant use  Share of funding allocated to schools through block grants Outputs  Number of schools that prepared School Improvement Plans  Number of schools voluntarily transferred to community school management Inputs  Government, donor, and community resources and support to education (Access/ Equity)  Government policies in the education sector Source: PAD. OUTPUTS 3.26 Most schools have a School Management Committee. The Department of Education reports that 93 percent of schools in Nepal have SMCs8 (Government of Nepal, 2012a). Formation of SMCs was not a new concept introduced under EFA. Community managed schools have a long history in Nepal. And, as noted earlier, the provisions for SMCs were formally laid out in the 7th and 8th Amendments to the Education Act of 2001 and included teacher recruitment authorities such as the authority to hire and fire community-recruited teachers, allocate grants monies to implement SIPs, and oversee the use of earmarked grants e.g. distributions for textbooks and scholarships (His Majesty’s Government of Nepal 2001, 2004; CERID, 2004; CERID, 2009; World Bank, 2014). In theory, SMCs in community managed schools have more authority over Government-funded teacher positions than SMCs in community schools. In reality, the DEO has retained significant power over teacher appointments, placement, and transfers. Community managed schools and community schools have the same authority over the teaching cadre – SMCs in both schools only recruit community-funded teachers (Khanal, 2011).9 High performing schools as well as high-functioning SMCs are still a function of dynamic Head Teachers. 8 SMCs are required in all schools, including private schools. 9 Currently there are five categories of teachers in Nepal: permanent, temporary, fixed-term government teachers (rahat teachers), Per Capita Financed, and community teachers. Permanent teachers have passed the TSC’s selection criteria and are appointed by the Commission. Permanent teachers are government employees who enjoy government salaries, benefits, and job security. Temporary teachers are teachers in government approved positions who have yet to sit 23 3.27 Limited Government support was provided through SIP grants. By project closure, 77 percent of funding was allocated through block grants surpassing the target of 60 percent (Table 3.6). These flexible resources were intended for teacher learning materials, capacity development, library development, etc. (UNESCO, 2014) to be determined by the local community. Earmarked grants for recurrent expenditures (e.g., teacher salaries and textbooks) accounted for 80 percent of grant funding. Grants were disbursed from the DEO directly to schools. The expanded block grants were contingent on SMC preparation of SIPs. 3.28 The Ministry of Education produced guidelines for the provision of block grants, conducted communication and outreach programs, and training for district staff, SMCs, teachers, and communities on the grant program. However, reports indicate that the rules and procedures for distributing grants were not well understood and a clear understanding at the school level was lacking often causing delays in the distribution of grant money. The Mid-Term Review suggested simplifying the entire block grant facility. 3.29 SIPs are prepared in the majority of schools in Nepal. By project closure, 18,000 schools report preparing SIPs. Although envisioned as vehicles for needs-based educational planning to improve access and quality through an efficient allocation of resources, SIPs were generally numbers-based reports based on previous year enrollment figures. The MTR found the SIP ineffective (DANIDA, 2006). 3.30 The targeted number of schools was transferred to community school management. By July 2011, there were 28,057 community schools in operation in Nepal. Of these, 11,902 were community managed schools (Government of Nepal, 2012a). Approximately 9,867 schools were transferred to community management under EFA, just shy of the target of 10,000 schools. The number is not broken down between primary and secondary school levels. Over the EFA time period, including activities under CSSP, 7,944 schools offering primary education had transferred to community management. The rigidity of the teacher management system hindered successful implementation of school autonomy (CERID, 2009). Interviews with staff confirmed no significant difference between community schools and community managed schools. 3.31 No systematic capacity building efforts were undertaken at the local level (CERID, 2009). Capacity building activities generally cited as part of EFA included orientation for SMCs, PTAs, and HTs on aspects of access; school management, school grants, and parental awareness; and social mobilization. Successful capacity building measures to improve participation of SMCs, school-level education planning, allocation of SIP resources, etc. were intended to contribute to both efficiency and quality outcomes. 3.32 Capacity building at the central level was assumed through the use of Government systems to implement EFA. The Ministry of Education produced guidelines for the for the licensing test. In an attempt to rectify the unequal distribution of teachers, the DOE began to fund rahat and contract teachers (per capita financing or “PCF” teachers). Finally, communities have the authority to hire community-funded teachers. 24 provision of block grants, conducted communication and outreach programs, and training for district staff, SMCs, teachers, and communities on the grant program. OUTCOMES: DID EFFICIENCY AND INSTITUTIONAL CAPACITY IMPROVE? 3.33 Overall the internal efficiency10 of primary education has improved. The project likely contributed to the reductions in repetition and drop-out rates, although drop-out rates remain high in grade 1. Output targets were generally met. 3.34 Repetition rates began to improve during the project period. A major source of internal inefficiency is grade repetition. As discussed above, net attendance in Nepal is low which can result in the need to repeat grades. Repetition can also result from poor quality of education. The project did not set targets for reductions in repetition rates. It did, however, expect lower repetition in grade one due to a reduction in under-age students and improved quality of entry through the increased ECD experience. 3.35 There is some evidence to suggest that the reductions in repetition and drop-out rates are due to the transfer of schools to community school management. Still, net attendance rates remained low over the project period, the quality of ECD was low, and the quality of education did not improve over the project period (see discussion below). The liberal promotion policy introduced in 2003 could also have contributed to reductions in repetition rates – moving students through the system regardless of achievement of learning. 3.36 Repetition rates declined over all grades but remained high in grade one. Table 3.6 shows the repetition rates by grade. Table 3.6. Repetition Rate by Grade Girls Boys Total Repetition Percentage 2003 2010 2003 2010 2003 2010 Grade 1 34.3 22.3 34.8 22.8 34.6 22.6 Grade 2 19.8 8.5 19.1 8.7 19.4 8.6 Grade 3 15.6 8 15.1 7.7 15.3 7.9 Grade 4 15.8 7.7 15.5 7.9 15.7 7.8 Grade 5 13.8 5.9 13.3 5.6 13.5 5.7 Source: Government of Nepal, Ministry of Education, Department of Education, 2012, 2013a and 2013b. 3.37 Drop-out rates have improved over the project period. No data are available to compare the trend prior to the project. Low student retention especially in the early 10 In the education sector, internal efficiency measures the number of years it takes a student to graduate a particular grade, for example grade five (the end of primary education in Nepal) relative to the number of years it should take a child to pass through grade five (i.e., the optimal number of years). Ideally, the coefficient of efficiency would be one (1). Common measurements of internal efficiency include the repetition rate and the dropout rate, as well as aggregate indicators of enrollment (UNESCO Institute for Statistics, 2009). 25 grades was a significant problem in Nepal. Drop-out rates decrease after grade one but increase again at grade five indicating many students never graduate from primary education. Table 3.7 shows the specific dropout rate per grade in 2003 and 2010. A substantial reduction in the grade one drop-out rate was achieved – from roughly 15 percent of students to eight percent – and again at grade five (from 14 percent to six percent), indicating a greater share of students remaining in primary education to the end grades. The trend is reflected in the survival rate to grade five which improved over the project period (see discussion below). Table 3.7. Drop-out Rate by Grade Girls Boys Total Drop out Percentage 2003 2010 2003 2010 2003 2010 Grade 1 14.9 8.3 14.5 8.2 14.7 8.3 Grade 2 5.5 4.9 6 5.6 5.8 5.2 Grade 3 6.6 4.6 7.3 4.4 7 4.5 Grade 4 7.1 3.6 7.8 3.9 7.5 3.8 Grade 5 13.6 6.2 13.4 6.8 13.5 6.5 Source: Consolidated Flash Report 2010-2011. 3.38 Community managed schools have made greater strides in reducing repetition rates as compared to community schools. Chaudhury and Parajuli (2010) conducted an impact evaluation comparing access, equity, and quality indicators of community managed schools transferred during the EFA period, with those obtained in community schools. The authors’ estimates found positive impacts associated with community school management with respect to efficiency outcomes – a 12.3 percentage point reduction in primary school repetition rates which was significantly greater than the reduction in repetition rates in community schools. Qualitative evidence supports these findings. Full Bright (2011) reports lower drop-out rates were associated with community managed schools (as reported by stakeholders randomly selected for the study). 3.39 While the project targeted reduction in repetition and dropout rates improved, other systemic efficiencies persist. Approximately two-thirds of the Government’s spending on its EFA Program was allocated to teachers’ salaries and pensions. In 2006, the MTR reported that the allocation of teachers was not based on need, having identified an unequal and inefficient allocation of teachers across Nepalese schools. In one set of schools, a surplus of 12,000 teachers was identified while in another set of schools, a shortage of 37,000 teachers was found. 3.40 Implementation Status Reports indicate actions to rectify or mitigate the situation were not implemented. In order to deal with the shortages of teachers in some schools, additional per capita funding was allocated to hire teachers where needed. Additional funding financed salaries for additional teachers to compensate for the imbalance of teachers. At the time of IEG’s mission, the Government had still not proposed a plan of action of redeploying teachers. 26 3.41 The capacity of SMCs has remained low. Limited improvements in capacity at the local level played had a hand in limiting the scope and effectiveness of project activities. For example, effective use of the SIP as a tool for school improvement requires additional capacity at the local level but trainings were short-term, generally two-three days (Norad, 2009) and insufficient for successful implementation. Insufficient school-level capacity also hindered successful implementation of school autonomy (CERID, 2009). 3.42 SMCs have not focused on teaching and learning. Early evidence suggested that participation in school activities increased with the presence of an SMC but SMCs tended to focus on activities related to physical improvement of school sites. The most recent qualitative evidence found that SMCs still focus on improving physical facilities but have also begun to focus on student and teacher attendance (CERID, 2009; Full Bright, 2011; and New Era, 2014). Enhancing the Quality and Relevance of Basic Primary Education to Children and Illiterate Adults 3.43 Achievement of this objective was rated modest. The results chain is presented in Table 3.8. Overall, the project fell short of meeting output and intermediate outcome targets (Table 3.9) and there is little evidence that the quality and relevance of education improved. 3.44 Two major shortcomings under the quality objective were (i) the failure to measure outcomes associated with learning achievement and (ii) the absence of the planned performance-based accreditation system for schools. OUTPUTS 3.45 The quality of Early Childhood Development is low. Limited Government support for early childhood development (ECD) reduced the quality of ECD offered in Nepal. The number of ECD centers established was exceeded but fewer grade one entrants than expected had ECD experience. As part of the EFA program in Nepal, early childhood development was supported as a means to improve quality of primary education focusing on marginalized communities. The project’s goal was to establish 11,000 new ECD centers and provide remuneration and short-term training for two ECD facilitators in new and existing centers. The project surpassed its goal and by 2009, 24,773 community-based ECD/PPE centers had been established from a baseline of 7,200 (Government of Nepal, 2012a). At project closure, 50 percent of new entrants into primary school had ECD experience, falling short of the project’s goal to achieve a 60 percent rate of new grade 1 entrants with ECD experience (i.e., having attended an early childhood development center or preprimary school program). 3.46 Cohort analysis of primary school progression prior to EFA implementation found a significant relationship between ECD and the progression of students through grade 5. For cohorts entering post project implementation, ECD experience had no measurable effect (was uncorrelated with) on progression rates. The rapid expansion in ECD under 27 EFA, which lowered the overall quality of the ECD experience is one plausible explanation for this effect (CERID, 2009). Table 3.8. Results Chain for Obtaining EFA Quality Project Goals  More students graduate from primary school (grade 5) Final  Learning outcomes at the grade five level improve Outcomes  A more literate adult population  More students enter grade one with ECD/PPE experience, are more likely to complete grade one, and less likely to repeat at the grade one level  More primary school level teachers with improved skills  More students receive their textbooks on time Intermediate  More active participation of SMCs in teaching and learning activities, Outcomes including teacher management  Early Childhood Development Centers and Preprimary classes established  Teachers provided recurrent in-service training to strengthen their skills  Textbook printing and distribution privatized  Transitional language support is provided to students whose mother tongue is not Nepali Outputs  Number of adults provided with literacy training Inputs  Government, donor, and community resources and support to education (Quality)  Government policies in the education sector Source: PAD. 3.47 Interviews in the field and interviews with Bank staff and donors confirm the low quality of ECD. Communities were responsible for the lion’s share of resources to open and operate ECD centers and were also responsible for establishing partnerships with I/NGOs to enhance the quality ECD provided to students. Anecdotal evidence from the field suggests that, where NGO partnerships were established and outside support and technical assistance provided, the quality of ECD was higher than in centers without NGO support. Quality enhancements included supplemental training for ECD facilitators, classrooms equipped to introduce the main developmental activities associated with ECD, and more materials on the walls, including students’ work. 3.48 Little recurrent, in-service teacher training was provided by the project. The project expected to strengthen the existing model of in-service teacher training by providing recurrent school-based in-service training for primary school teachers. Documentation and interviews with teachers suggests there was limited training, which was short in duration and generally offered at district headquarters. In more remote areas it is often difficult for teachers to travel to district headquarters to receive training. 3.49 A teacher is considered trained if s/he has completed and passed a ten month certification program or has majored in education at the higher secondary or university level. During EFA implementation, parallel financing from the Asian Development Bank’s Teacher Education Project (TEP) provided a 10 month teacher certification program. The increase in the percentage of trained teachers from 31 percent to 79 percent (Table 3.10) is likely due to TEP and not EFA. 28 3.50 The official statistics on teacher training do not reflect the objective of providing recurrent, school-based, in-service training. Resource Persons (RP) envisioned as leading both school-based trainings and monitoring teachers’ classroom activities were unable to support the number of schools assigned to each RP and were reportedly overburdened with administrative duties (DANIDA, 2006). 3.51 Knowledge transfer due to training remains low. One study suggests that less than half of what teachers learned in training translated into changes in classroom behavior (in terms of knowledge transfer) in Nepal’s schools (as cited in Norad, 2009) reducing any likely impact on the quality of education. Observations from the field and reported by New Era (2014) suggest that the vast majority of teachers continue to use traditional modes of instruction as opposed to interactive methods, at least at the basic and primary level. Only 18 percent of basic and primary school teachers incorporate high student involvement in teaching-learning practices. 3.52 Delivery of textbooks improved but the target was not reached. Improving access to textbooks is a quality enhancing measure that can have a positive impact on education under certain circumstances – depending on the quality and relevance of the materials delivered to the students not simply the timeliness. The privatization of textbook printing and distribution was expected to reduce the number of students who were not receiving textbooks at the start of each academic year. Delays in piloting and expanding the program created delays in progress on this indicator. Privatization began in 2007 in the Eastern region with grade five textbooks, expanding in 2008 to all primary grades in the Eastern and Western regions covering 47 percent of the market. In 2009, 57.6 percent of students received a full set of textbooks within two weeks of the beginning of the school year (Government of Nepal, 2012a). The percentage is in line with interviews with teachers and SMCs in the field. While a significant increase from the percentage at the start of the project (30 percent), the achievement was well below the targeted 72 percent. 3.53 Limited resources were devoted to activities supporting adult literacy, focusing on female literacy. The project expected to provide basic literacy to 60,000 women and assist another 48,000 become functionally literate (PAD, p. 34). World Bank documents report 387,257 women had enrolled in literacy programs by project closure. Most project activities were contingent on successful coordination between the project and on-going national efforts such the Women’s Literacy Program and National Literacy Campaign. A 2006 estimate of EFA expenditures reported two percent of program funds supported adult literacy activities. Bank documents confirm resource allocation toward female literacy was minimal compared to other project supported activities (World Bank, 2010). 29 Table 3.9. Outputs and Intermediate Outcomes Associated with Education Quality baseline target achievement (Percent) 2003 2009 2010 % new entrants to grade 1 with ECD experience 10.9 60.0 52.1 % fully trained teachers a 30.5 99.0 82.9 % textbooks received within the first week of the academic 30.0 72.0 66.3b session a Datafrom 2004 as reported in the 2004 Flash I Report. b Shareof textbooks received within the second week of the academic session. Source: Consolidated Flash Report 2010-2011 and Implementation Status Reports. OUTCOMES: WAS THE QUALITY AND RELEVANCE OF BASIC PRIMARY EDUCATION TO CHILDREN AND ILLITERATE ADULTS ENHANCED? 3.54 There is little evidence supporting improvement in the quality of primary education as a result of the project. Project outputs associated with quality were not of sufficient quantity or quality to improve overall educational quality. 3.55 Since no outcome measures were provided for quality and relevance, survival rates are used. The survival rate to grade five measures the percentage of a cohort enrolled in grade 1 which is expected to reach grade five. Using a promotion indicator as a measure of quality assumes children learn the longer they are retained in school. Remaining in school is also commonly considered a requirement for sustainable literacy. 3.56 Increases in survival rates over the project period were driven by boys. The target of 85 percent survival rate for both boys and girls was not achieved; for girls the improvement was minimal (Table 3.10). The extent to which improvement in the survival rate reflects the Government’s liberal promotion policy for grades 1-3 or student learning is not known. Table 3.10. Survival Rate to Grade 5 baseline Target achievement 2003 2009 2010 Survival rate to grade 5, total 76.2a 85 80.6 Survival rate to grade 5, male 72.4 a n/a 80.4 Survival rate to grade 5, female 80.6 a n/a 81.2 aData are for 2004 as reported in the Consolidated Flash Report 2010-11. Source: Consolidated Flash Report 2010-2011. 3.57 There is no evidence supporting a change in student achievement. There are no comparable test score data to judge improvement over time. However, on a nationally representative test of student achievement administered in 2012, students in grade five community schools answered 49 percent of math questions, 54 percent of Nepali questions, and 44 percent of English questions correctly. The corresponding scores for 30 students in grade five private schools were 67, 78, and 79 percent, respectively (Government of Nepal, 2014). 3.58 Community managed schools have not out-performed community schools. Chaudhury and Parajuli’s (2010) impact evaluation compared learning outcomes between community managed schools transferred during the EFA period with those obtained in community schools. The evaluation found no impact associated with community managed schools on grade three and five tests of student achievement conducted by the impact evaluation. The interval between pre-transfer (baseline test) and post-transfer to community management could have been too short for significant impacts to materialize. International evidence suggests it takes at least five years to institutionalize decentralized reforms and even then the evidence from developing countries on the effectiveness of decentralized reforms is mixed (Bruns, et. al, 2011). 3.59 Nepal has become more literate over time. Adult literacy increased from 48 percent at project approval to 79 percent in 2006, exceeding the project’s target of 66 percent. A comparison of data from successive Demographic Household Surveys (DHS) shows improvements in literacy rates among both men and women ages 15-24 and 15-49 years of age (Table 3.11). By 2011, 67 percent of women age 15-49 were literate. Still, significant variation remains by income quintile. Less than half of Nepal’s poorest women achieved literacy by 2011 while more than 90 percent of the wealthiest were literate. Table 3.11. Comparison of Literacy Rates over Time Source Male Female Total Adult Population 15-24* DHS 2006 91 75 79 DHS 2011 95 83 89 * Adult Population 15-49 DHS 2011, Overall 87 67 77a DHS 2011, Poorest 72 44 58 a DHS 2011, Richest 99 91 95 a Source: Demographic Household Survey, 2006 and 2011. * Literacy is determined by (i) attendance at the secondary level of schooling or higher and/or (ii) the ability to read a whole sentence or part of a sentence. a The total is calculated as the simple arithmetic mean of the percentages in the columns for males and females as is the practice in the 2011 Demographic Health Survey. 3.60 Adult literacy measures the cumulative effect of long term changes in education. Few resources were devoted to this particular goal under the project. Improvements in adult literacy are unlikely to be associated with project activities but with likely associated with previous achievements in education reform in Nepal. 31 4. Efficiency 4.1 The PPAR rates the project’s efficiency as modest. 4.2 Economic analysis. The PAD (pp. 60-64) presented a cost-benefit analysis of the project at appraisal. The ex-ante internal rate of return (IRR) to the EFA project was estimated at 16.1 percent using a discount rate of 12 percent. Net present value was estimated to be US$235 million. The rational for the analysis was based on a set of widely cited public and private benefits of primary education. The analysis does not consider benefits beyond primary education, for example, the increased benefits of more children attending and graduating from lower secondary education. The ex ante rate of return was conservative given this omission. 4.3 Sensitivity analyses were conducted using different rates of internal and external efficiency such as (i) an increased number of students passing more efficiently through the primary education sector (e.g., repetition and drop-out rates) and (ii) improved productivity resulting in wage premia for primary school graduates (as compared to individuals with less than five years of schooling). The latter assumption was based on improved education quality as a result of the project; the individual impacts of years of schooling versus quality of education were not distinguished. At the time, increased labor market returns to quality primary education were assumed to be eight percent (PAD, p. 62). Public and private costs, both direct and indirect were taken into consideration in calculating EFA Project costs. This resulted in an estimated range of returns to the project between the low case scenario (8 percent) and the high case scenario (24 percent). 4.4 The ICR recalculated the IRR with realized data at project closing (most recent available was 2009). The new estimation clearly established a set of assumptions and data sources used for the calculations including: gross enrollments in publically funded primary schools only, the cumulative number of students completing grade 5, cohort- based completer versus non completer wage differentials, full EFA Program costs (Government cost and pooling and non-pooling donor costs), and the public and private savings from a reduction in the average number of years it takes a student to complete grade five (ICR, p. 26-27). 4.5 The ex post IRR to the EFA Program was estimated at 21 percent using a discount rate of 12 percent and a 10 percent unemployment rate of primary school completers. The net present value was estimated to be US$992 million. 4.6 Sensitivity analyses with varying quality premia and unemployment rates were presented and resulted in a range of rates of return between 17 and 21 percent. The net present value ranged from US$574 to US$992. 4.7 The ex post estimates may be overstated given (i) the accuracy of enrollment and completion data (noted in Section 3) and (ii) the assumption of an unemployment rate of 10 or even 20 percent. Benefits to the project accrue primarily through completer versus non completer wage differentials. The World Bank (2013b) reports an average 32 unemployment rate of 46 percent. Efficiency would also be impacted by the extent that EFA-funded schools suffered damage from the earthquake. 4.8 The Government and donors generally regarded the SWAp as a cost effective approach to implementing the Government’s primary education sector strategy. The Government reported reductions in transaction costs. 4.9 Although overly complex rules and procedures for the distribution of grants resulted in these resources arriving at schools with delays, the project was completed on time with no extension of the closing date. 5. Ratings Outcome 5.1 This PPAR rates the project’s outcome as moderately unsatisfactory. Relevance of the Objectives is rated high. Relevance of Design is rated modest due to shortcomings in project design, in particular insufficient attention to capacity building at all levels and inadequate emphasis on appropriate activities to improve the quality of education. Efficacy in the achievement of the objective to improve access and equity is rated substantial. Efficacy in the achievement of the remaining two objectives is rated modest. Efficiency is rated modest. Together these ratings result in a moderately unsatisfactory outcome rating. Risk to Development Outcome 5.2 Risk to development outcomes is rated moderate. 5.3 Political instability remains a reality in Nepal – the Peace Accord was signed in 2006 but a Constitution was only introduced in September 2015. The new Constitution extends free and compulsory education to the basic level. Even without a Constitution, the Government has remained committed to promoting education for all as a lever for social inclusion, peace building, and economic growth. The final phase of the EFA Action Plan began in 2009 with the follow-on project (SSRP), which wraps up Nepal’s 15 year EFA strategy. 5.4 Commitment to goals does not ensure commitment to existing reform strategies and this has been a concern. The Bank has played a key role ensuring reforms stay the course. Political uncertainties remain at the highest levels of government. Continued dialogue may be required between the Bank, other donor agencies, and the DEO to stem reversals in reform policies depending on the results of the next election. 5.5 Fiscal constraints are a reality in Nepal and while Government support to education has remained high so has its reliance on donor and local support for education. Fiscal concerns are increasingly important in light of the strains imposed by reconstruction and recovery efforts post-disaster. Prior to the earthquake, the share of education spending in the national budget had fallen slightly. Given these realities, gains made during EFA will remain heavily reliant on donor support. Estimates for the 33 education sector alone are upwards of US$400 million (World Bank, 2015). According to the Region’s comments, the Government was quickly able to mobilize US$300 million in additional funding from donors for post-disaster reconstruction efforts. 5.6 A follow-on project continues to support the EFA reforms. The number of pooling partners has increased from seven at the end of EFA to nine during SSRP implementation. A strategic plan for capacity at all levels – central, district, municipality, village, and school – has not yet been established. Bank Performance 5.7 Overall Bank Performance is rated moderately unsatisfactory. 5.8 Quality at entry is rated moderately unsatisfactory. 5.9 The Bank and other donors had been coordinating their efforts in Nepal for many years. Adopting a SWAp approach reduced the administrative burden on the Government and harmonized donor support – directly funding the Government’s sector strategy. 5.10 During preparation, the Bank team correctly identified improving service provision as the means to attain access, equity, and quality in education however there was little evidence from either Nepal or other countries to support large scale implementation of decentralized provision of education to attain these goals. This is especially true in the case of quality. The Bank team incorporated lessons learned from experience during Basic and Primary Education Project II into program design but it was too soon to incorporate lessons from the on-going pilot of transferring schools to community school management under the Bank’s Community School Support Program. The Bank prematurely supported the scaling-up of both SIP funding and community school management. 5.11 The project was designed and implemented at a time of conflict, political instability, and nascent decentralization. The team identified the significant risks associated with the project and incorporated appropriate design features to these mitigate risks. However the Bank underestimated the risk associated with local level capacity. The Bank did not ensure that sufficient activities were supported such that necessary conditions for successful decentralization were in place, in particular capacity at the local level. 5.12 Fiduciary concerns were noted at appraisal but were not sufficiently addressed and remained troublesome throughout project implementation with late reporting and ineligible expenditures – requiring significant Bank oversight. 5.13 Quality of Supervision is rated moderately satisfactory. The formal arrangements for annual reviews and consultations between the donors and the Government of Nepal provided well-coordinated oversight of project implementation. At the annual reviews, shortcomings were identified and plans to mitigate the shortcomings were collectively developed. The Government’s Implementation Progress Reports initially provided little information on targets achieved, reporting only the percentage of 34 physical achievement. Reporting improved with donor oversight, however, the arrangements for more timely supervision reports remained weak. 5.14 Donors reported strong Bank leadership throughout project implementation whether or not the Bank was the Contact Point for the Pooling Donors Working Group at the time. 5.15 Throughout implementation, the Bank’s Implementation Status Reports rated implementation progress Satisfactory based on progress toward output and outcome targets. This rating seems optimistic given the data irregularities and significant shortcomings in implementation progress identified in annual progress reviews, Technical Reviews of School Education, and the Mid-Term Review, and discussed by the Government and Donors’ at annual consultative sessions. 5.16 The Bank’s strength in the area of Financial Management was of considerable importance and greatly appreciated by the other donors given that throughout project implementation, financial management and reporting remained a concern. The Bank’s fiduciary oversight was regular and follow up was consistent. 5.17 The Bank supported increased monitoring to accompany validation of progress reports; these reports were supplemented with validation of program implementation relative to the agreed upon program. However, the Bank was slow to respond to findings in the technical reviews produced, which included concerns over data quality, the need for capacity building, and the appropriateness of the decentralization policy in Nepal (DANIDA, 2006). 5.18 Throughout project implementation, political and policy instability threatened progress in key education reforms. The Bank played a key role in protecting reforms in the face of changing political circumstances, including, but not limited to various aspects central to decentralization. For example, revoking the teacher licensing scheme. Borrower Performance 5.19 Overall Borrower Performance is rated moderately unsatisfactory. 5.20 Government Performance is rated moderately satisfactory. Over the project period, regardless of political party, successive Governments in Nepal have reiterated their commitment to education reform. During project implementation, the government of Nepal changed three times. Nepal was an absolute monarchy (2002-2005), a constitutional monarchy (2006-2007), and a multiparty democracy (2008-present). 5.21 At the time of the PPAR mission, the current Ministry of Education was less reform minded than in previous years which was providing an element of uncertainty, especially concerning the up-coming Amendment to the Education Act. The Government’s commitment to specific reforms being implemented varied. At various times the Government sought to revoke teacher licensing and reverse the gains made in privatizing textbook printing and distribution. Concerted effort on the part of the Bank and other donors was required to ensure reforms stayed the course. 35 5.22 Coordination between the Ministry of Education and the SWAp partners worked well. The process followed a twice yearly review and consultation cycle between Donors and the Government (which chaired the meetings) where mutually agreed upon actions were determined based on strategic areas in need of additional support as identified in Aide Memoires. A majority of actions were taken up in annual plans. 5.23 Implementing Agency Performance is rated moderately unsatisfactory. The Department of Education (the technical arm of the Implementing Agency) was committed to achieving the project’s objectives. However, the new school grant program was overly complex and the DOE was slow to provide the necessary guidelines and capacity building required for smooth implementation of the program. Information flows from the central level to the district and school level was weak. 5.24 The same issues recurred throughout implementation, such as those surrounding the need for better communication, capacity building, simplification of the fund flow process, low attention to ECD, and lack of attention to quality aspects of education. 5.25 The Government reported on both financial and physical progress toward implementation on a trimester basis. Implementation Progress Reports provided information on several input activities associated with the key performance indicators for example, the number of scholarships distributed, orientations for SMCs, inputs on non- formal education and adult literacy. For the most part, Government reporting consisted of the percentage of physical and financial progress under each EFA goal. The MTR highlighted the issue of late, inconsistent, unreliable, and non-transparent reporting. 5.26 The Education Management Information System (EMIS) was the primary vehicle to collect and disseminate education indicators. Fund allocations were based on EMIS (or Flash) data. Verification of self-reported data proved inadequate, especially given the reliance on the data for resource allocation. 5.27 As noted earlier, a weak internal control system in the Department hampered effective implementation. Financial reporting was delayed; not all shortcomings identified in audit reports were addressed in a timely fashion. Progress on implementing the Financial Management Action Plan to remedy these shortcomings was slow. Monitoring and Evaluation 5.28 This PPAR rates overall Monitoring and Evaluation of the project modest. 5.29 M&E Design. A range of indicators were proposed and to a large extent collected to provide inference of changes along the causal chain. These indicators, for example the share of trained teachers, the share of schools receiving textbooks on time, the number of schools transferred to community management, or the number of schools preparing SIPs, are output indicators. Measures of teacher skills, use of class planning/preparation or learning materials, participation and engagement of SMCs, parents, and communities, would have provided more information leading to final outcomes. 5.30 The key performance indicators were (i) net enrollment rates; (ii) grade five survival rates; and (iii) learning achievement outcomes in grade five. Baselines were 36 provided for all indicators in the aggregate and broken down by sex; targets were not set for boys versus girls. Neither baselines nor targets were provided for Dalits or Janajati. 5.31 There was a clear plan and timetable for collecting most of the monitoring data. No clear plan was presented for collecting data on the quality of education. Although Nepal had not systematically tested student learning in the past, it had tested students at grades 3, 5, and 8 in several subjects at various points in time. The project planned on conducting three tests over the duration of the project. Baseline data on student test scores presented percentages (correct answers) with no reference or benchmark for pass, fail, or proficiency levels. 5.32 A common monitoring and evaluation framework under the SWAp streamlined data collection and analysis systems. The project relied on the EMIS data collection system initiated under BPEP II – a bottom-up reporting system from the school to the district to the central level. All schools were to report data on key indicators. 5.33 The comprehensive M&E system emphasized local reporting for more efficient resource allocation. Participation of communities was crucial to generate the data required for the project monitoring. An integral component was output-based expenditure reporting and decision-making based on collection of EMIS data in conjunction with SIPs. 5.34 Other monitoring arrangements were built into design such as social audits, national survey data validation studies (e.g., NLSS and DHS data), and an impact evaluation on girls' scholarships (PAD, p. 11). Government Implementation Progress Reports collected additional data on all EFA goals. A Mid-Term Evaluation and Final Evaluation were planned to asses overall project performance. 5.35 M&E Implementation. The expanded and upgraded EMIS provided twice yearly school-level reporting on relevant project indicators, once at the beginning of the academic year (Flash I) and again at the end of the academic year (Flash II). School-level data were collected by Head Teachers and reported to the District Education Office for validation before submission to the Department of Education for aggregation and publication. 5.36 The system encountered significant shortcomings in the reporting of data at the school- and district-levels. Schools reported data to the districts; DEOs were responsible for verification and correction of the data. Capacity at both the school level and DEO level was lacking. Verification systems established by the project in the initial years of implementation concluded the Flash and EMIS system were accurate. These technical reviews of data quality were suspended in 2006, perhaps prematurely, having determined that data reporting was accurate. The issue arose again in 2014 when the data on enrollments were found to be inflated. A recent study estimated that 15 percent of school level report cards were improperly filled in (Aasaman Nepal and ISPC, 2014). Newly published results prepared for the World Bank provide similar evidence to that cited above with differences in many of the key performance indicators (New Era, 2014; World Bank, 2104). 37 5.37 Certain indicators were not systematically collected or reported. There were no indicators defined to measure progress toward the objective to improve institutional capacity. Not all data were disaggregated as stated in the project’s results framework. 5.38 The Mid-Term Evaluation and Final Evaluation were undertaken as planned as was a large amount of formative research during project implementation. The impact evaluation on girls’ scholarships was not undertaken. 5.39 Social audits were introduced as a tool to increase accountability, participation, and efficiency at the school level. The Mid-Term Review reported significant issues with the process and reporting through social audits (DANIDA, 2006). Schools seemed to have limited understanding of the role of social audits, often confusing them with financial reporting. 5.40 M&E Utilization. In practice, DEOs used EMIS data to allocate grants to schools based on the previous year’s reporting, i.e., on last year’s enrollment figures. Issues with the reliability of the data suggest major issues regarding utilization of the EMIS data. 5.41 A vast amount of research was generated in conjunction with the project. Donors funded technical assistance to support learning and accountability throughout implementation. Issues and areas of improvement were noted along the way, including at the Mid-Term Review, however few course corrections were made over the life of the project. For example, simplifying the grant process, increasing the emphasis on quality, or preparing a strategic plan for capacity building. 6. Lessons 6.1 Since 1992, the Bank, in coordination with other donors, has supported the Government of Nepal build institutional capacity; improve efficiency and quality of education services; and improve equity in access to education, especially for girls and students from disadvantaged communities. Since that time, IEG has conducted three PPARs, including the current one, to evaluate achievement toward these objectives. The findings from each evaluation are similar: increased funding and decentralization can improve access but direct attention to educational inputs is required to improve quality. 6.2 Based on the experience of this project, lessons for future education projects in Nepal and elsewhere can be drawn: 6.3 To improve the quality of education, in particular learning outcomes, it is important to emphasize the quality of learning inputs such as teachers and instructional and learning materials. There is little evidence to suggest inputs to the teaching learning process improved due to the project: the quality of ECD suffered from limited Government support; teachers received limited training; and children still received textbooks late. 6.4 In a low capacity environment, a strategy to build local level capacity is critical in a decentralized service provision model. Low capacity limits the potential of various instruments under communities’ control to improve the efficiency and quality of schools. 38 Under EFA, distribution of grants was often delayed due to overly complex rules and procedures which were not well understood at the school-level. Low capacity limited the impact of School Improvement Plans which were meant to focus on teaching and learning activities. 6.5 It is important that central, district, and local level roles and responsibilities are clearly defined when education provision has been decentralized. In Nepal, ambiguities existed as to where ultimate responsibilities for education decision-making lie. These ambiguities contributed to the failure of decentralization to bring about increases expected in the quality of public education. 6.6 Quality assurance in M&E data is essential when funding is linked to school-level data. Verification of the current system suggests irregularities in reporting data: scholarship distribution has not followed guidelines and the incentive to over-report enrollment data has increased with the introduction of per capita financing. 39 References Acharya, Sushan and Bal Chandra Luitel. 2006. The Functioning and Effectiveness of Scholarship and Incentive Schemes in Nepal. UNESCO Kathmandu Series of Monographs and Working Papers No. 9. Assam Nepal and International Service Providers Company Pvt. Ltd. 2011. Final Report on A Study on Reliability of Educational Data of Schools. Sanothimi: Nepal. Asia South Pacific Association for Basic and Adult Education. 2010. Exploring Partnerships for Education For All: A Study on The Sector-Wide Approach (SWAp) in Nepal. Kathmandu: Nepal. Barrera-Osorio, Felipe, Tazeen Fasih, and Harry Anthony Patrinos with Lucrecia Santibáñez. 2009. “Decision-Making in School: The Theory and Evidence on School-Based Management.” Washington, DC: World Bank. Bhata, Saurav Dev. 2005. Study on Student Performance in SLC: Determinants of Student Performance in the SLC Examinations. Prepared for the Mnistry of Education and Sports/ESAT. Bruns, et. al, 2011. Making Schools Work New Evidence on Accountability Reforms. Washington, DC: World Bank. Carney, Stephen and Min Bista. 2009. Community Schooling in Nepal: A Genealogy of Education Reform since 1990. Comparataive Education Review. vol. 53 no 2. Pp. 189-211. CERID. 2004. Management Transfer of Community Schools. Balkhu, Kathmandu, Nepal. _____. 2005. Disbursement of Block Grants. Balkhu, Kathmandu, Nepal. _____. 2007. Understanding School Autonomy. Balkhu, Kathmandu, Nepal. _____. 2009. Community Managed School: An Innovative Approach to School Management. Balkhu, Kathmandu, Nepal. Chaudhury, Nazmul and D. Parajuli. 2010. "Giving It Back: Evaluating the impact of Devolution of School Management to Communities in Nepal." Unpublished manuscript. Washington, DC: World Bank. Danish International Development Agency. 2007. Mid-Term Review of Nepal Education for All (EFA) Programme 2004-2009. Draft Report. Copenhagen: Denmark. Full Bright Consultancy. 2011. Final Report: A Study on Effectiveness of Community Managed Schools. Kathmandu, Nepal. Government of Nepal. 2007. Nepal Demographic and Health Survey 2006. Kathmandu: Nepal _____. 2008. Flash II 2007-2008 Report. Kathmandu: Department of Education. _____. 2010a. Flash II 2009-2010 Report. Kathmandu: Department of Education. _____. 2010b. Flash I 2010 Report. Kathmandu: Department of Education. _____. 2011. Nepal Living Standards Survey 2010/11: Kathmandu: Central Bureau of Statistics. _____. 2012a. Consolidated Flash Report 2010-2011. Kathmandu: Department of Education. _____. 2012b. Nepal Demographic and Health Survey 2011. Kathmandu: Nepal _____. 2013a. Draft Flash I 2012- 2013. Kathmandu: Department of Education. _____.2013b. Status Report 2013. Kathmandu: Department of Education. _____. 2014. “Where are the grade 3 and grade 5 students now? National Assessment of Student Achievement: Student achievement in Mathematics, Nepali, and English at grade 3 and 5 in 2012.” Kathmandu: Education Review Office. Hanushek, Erik and Woessmann. 2012. Do better schools lead to more growth? Cognitive skills, economic outcomes, and causation. Journal of Economic Growth 17:267–321. 40 His Majesty’s Government of Nepal. 1996. Nepal Living Standards Survey 1995/96: Kathmandu: Central Bureau of Statistics. _____. 2001. Education (Seventh Amendment) Act, 2058. http://planipolis.iiep.unesco.org/upload/Nepal/Nepal_Education_Act_1971.pdf. _____. 2004. Education (Eight Amendment) Ordinance, 2060. http://planipolis.iiep.unesco.org/upload/Nepal/Nepal_Education_Act_1971.pdf. _____. 2003a. Education for All: National Plan of Action. Kathmandu, Nepal: National Planning Commission. _____. 2003b. The Tenth Plan, 2002-2007. Kathmandu, Nepal: National Planning Commission. _____. 2004a. Nepal Living Standards Survey 2003/04: Kathmandu: Central Bureau of Statistics. _____. 2004b. National Environmental Guidelines for School Improvement and Facility Management in Nepal. _____. 2005a. Education for All: Status Report 2005. Sanothimi: Department of Education. _____. 2005b. Flash I Report. Kathmandu: Department of Education. _____. 2005c. Flash II 2004-2005 Report. Kathmandu: Department of Education. _____. 2005d. Flash I 2005 Report. Kathmandu: Department of Education. Independent Evaluation Group. 2009. Project Performance Assessment Report. Basic and Primary Education Project II. Washington, DC: World Bank. _____. 2010. Project Performance Assessment Report. Community School Support Project. Washington, DC: World Bank. Khanal, Peshal. 2011. Teacher management in a decentralised school context in Nepal: Fuelling tension and dissent?, Compare: A Journal of Comparative and International Education, 41:6, 769-784 Khaniya, Tirth Raj and Arbind Bhomi. 2005. Education for All Supplementary Technical Review of Education. New Era. 2014. Nepal Education Studies 2012/13 School and Household Survey. Submitted to the World Bank. Kathmandu, Nepal. Norwegian Agency for Development Cooperation. 2009. Joint Evaluation of Nepal´s Education for All 2004-2009 Sector Programme Evaluation Report 1/2009. Oslo, Norway: Norad. Technical Review of School Education. 2006. Kathmandu, Nepal United Nations Educational, Scientific, and Cultural Organization, Institute for Statistics, 2009. Education Indicators Technical guidelines. UNESCO. 2009. Education Indicators Technical guidelines. UNESCO: Institute of Statistics. _____. 2014. A Resource Material on Education and Federalism in Nepal. UNESCO/UNPFN Project: Planning Effective Delivery of Education in Future Federal Nepal. Lalitpur: Nepal. Vaillancourt, D. 2009. Do Health Sector-Wide Approaches Achieve Results? Emerging Evidence and Lessons from Six Countries. IEG Working Paper 2009/4. Washington, DC: World Bank. _____. 2012. In Sweet Harmony? A Review of Health and Education Sectorwide Approaches (SWAps) in the South Pacific. Washington, DC: World Bank. World Bank. 2003. Nepal Country Assistance Strategy 2004-2007. Washington, DC: World Bank. Report No. 27783. _____. 2004a. World Development Report: Making Services Work for the Poor. Washington, DC: World Bank. _____. 2004b. Project Appraisal Document on a Proposed Credit to the Kingdom Of Nepal For An Education For All Project. Washington, DC: World Bank. Report No. 27890. 41 World Bank. 2006. Nepal Resilience amidst Conflict: An Assessment of Poverty in Nepal, 1995–96 and 2003–04. Washington, DC: World Bank. _____. 2007a. Project Paper on a Proposed Additional Financing (Grant) to Nepal for the Education For All Project. Washington, DC: World Bank _____. 2007b. Nepal Interim Strategy Note. Washington, DC: World Bank. _____. 2007c. Findings of Financial Management Review of Education for All (EFA). FY 2004/05 & 2005/06. _____. 2009. Nepal Interim Strategy Note FY2010-2011. Washington, DC: World Bank. _____. 2011. Nepal Interim Strategy Note FY2012-13. Washington, DC: World Bank. _____. 2013. Nepal Country Partnership Strategy 2014-2017. Washington, DC: World Bank. _____. 2014. Public Expenditure Tracking and Quantitative Service Delivery Surveys in Nepal’s Education Sector. Washington, DC: World Bank. Report No. ACS8283. _____. 2015. Sector Reports. Nepal Earthquake Post Disaster Needs Assessment. Washington, DC: World Bank. World Health Organization. (2006). A guide to WHO's Role in Sector Wide Approaches to Health Development. Geneva, Switzerland: World Health Organization http://www.epdc.org/sites/default/files/documents/EPDC%20NEP_Nepal.pdf http://www.undp.org/content/dam/nepal/docs/reports/governance/UNDP_NP_Local%20Self Governance%20Act%201999,%20MoLJ,HMG.pdf 43 Annex A. Basic Data Sheet NEPAL EDUCATION FOR ALL PROJECT (P074633) Key Project Data (amounts in US$ million) Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Total project costs 664.00 989.34 148.99 Loan amount 50.00 110.94 221.88 Cofinancing 100.00 175.04 175.04 Cancellation 0.00 0.32 0.00 Cumulative Estimated and Actual Disbursements FY05 FY06 FY07 FY08 FY09 FY10 FY11 Appraisal estimate (US$M) 6.99 14.99 24.99 36.99 49.99 49.99 49.99 Actual (US$M) 5.60 21.94 40.43 51.71 102.60 111.26 110.94 Actual as % of appraisal 80.11 146.36 161.78 139.79 205.24 222.56 221.94 Date of final disbursement: 09/10/2012 Project Dates Original Actual Initiating memorandum 03/20/2003 08/15/2003 Negotiations 02/05/2004 05/19/2004 Board approval 03/22/2004 07/08/2004 Signing 08/12/2004 08/12/2004 Effectiveness 07/01/2004 08/27/2004 Closing date 01/31/2010 01/31/2010 Task Team Members Names Title Unit Lending Julie-Anne M. Graitge Program Assistant SASHD Susan E. Hirshberg Sr Education Specialist AFTED Rajendra Dhoj Joshi Sr Education Specialist AFTED Chingboon Lee Sector Manager, Education LCSHE John Middleton Consultant AFTH1 - HIS Sushila Rai Program Assistant SASHD ANNEX A 44 Names Title Unit Supervision/ICR Mohan Prasad Aryal E T Consultant SASHD Kiran R. Baral Sr Procurement Officer SARPS Lynn Bennett Consultant SASDS Drona Raj Ghimire E T Consultant SASDI Julie-Anne M. Graitge Program Assistant SASHD Yuka Makino Natural Resources Mgmt. Specialist SASDI Nagendra Nakarmi Senior Program Assistant SARFM Bigyan B. Pradhan Sr Financial Management Specialist SARFM Sushila Rai Program Assistant SASHD Neena Shrestha Procurement Assistant SARPS Gajendra Man Shrestha Consultant SASHD Laxmi Prasad Subedi Consultant SASDA Venkatesh Sundararaman Senior Economist SASED Staff Time and Cost Stage of Project Cycle Staff Time and Cost (Bank Budget Only) US$ Thousands (including No. of staff weeks Lending travel and consultants costs) FY03 105.55 FY04 327.04 FY05 3.68 Total: 436.27 Supervision/ICR FY05 96.14 FY06 30.29 FY07 35.56 FY08 38.58 Total: 200.57 45 Annex B. List of Persons Met Mr. Mahashram Sharma Joint Secretary Ministry of Education Mr. Tekraj Niraula Under Secretary Ministry of Federal Affairs and Local Development Dr. Lava Deo Awasthi Director General Department of Education Mr. Janardhan Director General, former Department of Education Mr. Hari Lamsal Under Secretary Education Review Office Mr. Bhola Dahal Embassy of Norway Prof. Dr. Krishna Chandra Executive Director Research Center for Education Sharma Innovation and Development (CERID) Prof. Kishor Shrestha ECD Specialist Research Center for Education Innovation and Development (CERID) Ms. Smita Gyawali Associate Project Officer Asian Development Bank Mr. Bhuvan Consultant Asian Development Bank Mr. Krishna Pandey Consultant Asian Development Bank Mr. Krishna Lamsal Program Manager, Education Australian Embassy Ms. Tara Gurung Director, Development Policy Australian Embassy and Programs Ms. Indra Gurung Programme Manager Embassy of Finland Ms. Marianne Kujala-Garcia Counsellor, Development Embassy of Finland Mr. Udhav Karki Programme Manager, Education European Union Lena Hasle Counsellor Royal Norwegian Embassy Mr. Shiva L. Bhushal ECD Program Specialist UNICEF Ms. Jayanti Subba Education Specialist, Education USAID Mr. Ram K. Rijal Junior Professional Associate World Bank Mr. Rajendra Joshi Consultant, former TTL World Bank Mr. Saurav Dev Bhatta Senior Economist World Bank Mr. Mohan Prasad Aryal Operations Officer World Bank Ms. Shraddha Shah Consultant World Bank Mr. Bigyan Pradhan Senior Operations Officer World Bank Mr. Albertus Voetberg Lead Health Specialist World Bank Ms. Pushpa Sharma Kavre School District Mr. Upendra Mahato, District Education Officer Dhanusa School District Mr. Dilli Luitel, District Education Officer Kaski School District Reiny de Wit Managing Director Early Childhood Education Centre 46 Annex C. Borrower Comments 47 ANNEX C ANNEX C 48 49 ANNEX C