Document of The T;lTorIdBank Report No.: 40160-IN PROJECT PERFORMANCE ASSESSMENT REPORT U'ITARPRADESI.IBASICED~JCATIONPROJEC'I' (CREDIT-2509) SECONDUT'TARPRADESH BASIC EDUCATIONPROJECT(CREDIT-3013) DIS'~R~CT PRIMARYF,I)LJCATIONPROJEC'I' (CREDIT-2661) DISTRICTPRIMARY EDUCA'TTON PKOJEC'I-(CREDIT-2876) I1 June 30,2007 ,secdor Thematic and Global Evuluatio~Dil.ii.~io~2 IlzrlependerrfE\:aluation Groztp - Currency Equivalents (annual averages) C'rtn-erzcl: Ur?it Irreii~lnRupees (Rs) Rs. 31.37 Rs. 3 1.39 R.s. 33.44 Rs. 35.50 Rs. 37.16 R.s. 4 1.30 Rs. 43.10 Rs. 46.37 District Primary Educatiol~Project Exchange rate effective March 1994 IJS$I Rs, 32 At the end of the project: US$1 Rs. 45 District Primary Education Project I1 Exchange rate effective average over life of project US$l Rs. 43.7 Abbreviatio~lsand Acronyms ASER Annual Status of Education Report AWPB Annual Work Plan and Budget BAS Raseliile Assessment Study BRC Bloclc Resource Center CRC Cluster Resource Center DIET District Institute of Eclucation and Training DISE District Ii~fomationSysten~for Education DO Dropout DPEP District Prinlary Education Project EdCi1 Education Consultai~tsIndia Limited EFA Education for All EFd4PB Educational for All Project Board EMIS Education Managemei~tinformation System GER Gross enrollment ratio GO1 Government of India GOUP Government of LJttar Pradesh ICDS Iutegrated Child Development Services ICR Implementation Completion Report IDA international Developrnei~tAssociation IDF Institutional Developnlent Fund IEG hdependent Evaluation Group JRM Joint Review Mission MAS Mid-term Assessment Survey MHRD Ministry of Human Resource Development MIS Management Infom~ationSyste~n MLL Minimum Levels of Learning NCERT National Council of EducationalResearch and Training NEP National Education Policy NER Net enrollment ratio NFE Nonformal Education NIEPA National Instituteof Educational Planning and Administration PAD Project Appraisal Document PPAR Project Perfomance Assessment Report PRI Panchayat Raj Institutions (local government) SC Scheduled Caste SCERT State Councilof EducationalResearch and Training SIEMAT State institute of Educational Management and Training SSA Sarva Shiksha Abhiyan (Elementary EducationProject) ST Scheduled Tribe SWAP Sector Wide Approach TAS Terminal Assessment Survey TSG Technical Support Group UP Uttar Pradesh UPBEP Uttar Pradesh Basic Education Project VEC Village Education Committee Fiscal Year April 1 - March 31 Director-General, Evaluation : Mr. Vinod Thomas Director, Independent Evaluation Group, World Bank : Mr. Ajay Chhibber Manager, Sector, Thematic, and Global Evaluation Division: Mr. Alain Barbu Task Manager : Mr. Dean Nielsen IEGWB Mission: Enhancing development effectiveness through excellence and independence in evaluation. About this Report The IndependentEvaluationGroupassessesthe programs and activities of theWorld Bank for two purposes:first, to ensure the integrity of the Bank'sself-evaluationprocessand to verify that the Bank'swork is producingthe expected results, and second, to helpdevelop improveddirections, policies,and proceduresthrough the disseminationof lessons drawnfrom experience.As part of this work, IEGWBannuallyassessesabout 25 percent of the Bank's lendingoperations throughfield work. Inselecting operationsfor assessment, preferenceis givento those that are innovative, large, or complex; those that are relevantto upcomingstudies or countryevaluations; those for which ExecutiveDirectorsor Bank management have requested assessments; and those that are likely to generateimportantlessons. To prepare a Project PerformanceAssessment Report(PPAR), IEGWB staff examine projectfiles and other documents, interviewoperationalstaff, visit the borrowing country to discussthe operationwith the government,and other in-countrystakeholders, and interview Bankstaff and other donor agency staff bothat headquarters and in localoffices as appropriate. Each PPAR is subject to internal IEGWB peer review,Panel review, and management approval. Once cleared internally, the PPAR is commented on bythe responsibleBank department. IEGWB incorporates the comments as relevant. The completed PPAR is then sent to the borrowerfor review; the borrowers'comments are attached to the document that is sent to the Bank's Boardof ExecutiveDirectors.After an assessment report has been sent to the Board, it is disclosedto the public. About the IEGWB Rating System IEGWB's use of multipleevaluationmethods offers both rigor and a necessary levelof flexibilityto adapt to lending instrument,projectdesign, or sectoral approach. IEGWBevaluatorsall apply the same basic methodto arrive at their project ratings. Following is the definition and ratingscale used for each evaluationcriterion (additionalinformation is available on the IEGWB website: http://worldbank.orglieg). Outcome: The extent to which the operation's major relevantobjectiveswere achieved, or are expected to be achieved, efficiently.The rating has three dimensions: relevance, efficacy, and efficiency. Relevanceincludes relevanceof objectives and relevanceof design. Relevanceof objectives is the extent to which the project's objectives are consistentwith the country's current development priorities and with current Bank countryand sectoralassistance strategies and corporate goals (expressedin PovertyReductionStrategy Papers, CountryAssistance Strategies, Sector Strategy Papers, Operational Policies). Relevance of design is the extent to which the project's design is consistent with the stated objectives. Efficacy is the extent to which the project's objectives were achieved, or are expectedto be achieved, taking into account their relativeimportance.Efficiency is the extent to which the projectachieved, or is expected to achieve, a return higher than the opportunitycost of capitaland benefitsat least cost compared to alternatives. The efficiencydimensiongenerally is not applied to adjustment operations. Possibleratingsfor Outcome: Highly Satisfactory, Satisfactory, Moderately Satisfactory, ModeratelyUnsatisfactory,Unsatisfactory,Highly Unsatisfactory. Risk to Development Outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained(or realized). Possibleratingsfor Risk to Developmenf Outcome: High Significant, Moderate, Negligibleto Low, Not Evaluable. Bank Performance: The extent to which services provided by the Bank ensured quality at entry of the operationand supported effective implementationthrough appropriatesupervision(includingensuring adequatetransition arrangements for regular operationof supportedactivities after loanlcredit closing, towardthe achievement of developmentoutcomes.The rating has two dimensions:quality at entry and quality of supervision. Possibleratings for Bank Performance: Highly Satisfactory, Satisfactory,ModeratelySatisfactory,ModeratelyUnsatisfactory,Unsatisfactory, Highly Unsatisfactory. Borrower Performance: The extent to which the borrower (includingthe government and implementingagencyor agencies)ensured qualityof preparationand implementation, and compliedwith covenants and agreements,toward the achievement of development outcomes. The rating has two dimensions:governmentperformanceand implementing agency(ies) performance. Possibleratingsfor BorrowerPerformance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory,Highly Unsatisfactory. Contents PRINCIPAL RATINGS ..............................................................................................................V KEY STAFF RESPONSIBLE ...................................................................................................VI PREFACE ...............................................................................................................................VII SUMMARY ...............................................................................................................................IX I. BACKGROUNDAND CONTEXT ...................................................................................I 2. PROJECT DESIGN AND IMPLEMENTATION ...............................................................2 Project Objectives.......................................................................................................................... 2 Project Components ...................................................................................................................... 5 3. IMPLEMENTATIONAND COSTS .................................................................................. 6 4. MONITORINGAND EVALUATION ................................................................................9 Monitoring and EvaluationDesign ...............................................................................................9 ImplementingMonitoring and Evaluation in Projects ..............................................................I 0 Utilization of Monitoring and EvaluationResults ................................................................... II 5. ACHIEVEMENTS OF PROJECT OBJECTIVES ...........................................................12 Improving Access to PrimaryEducation ................................................................................... 12 Reducing Dropout....................................................................................................................... 5 Improving LearningAchievement ..............................................................................................16 Building Institutional Capacity .................................................................................................... 18 EnhancingCommunity Participation in Primary Education ....................................................22 UPBEPand DPEP .........................................................................................................................22 6 . RATINGS ......................................................................................................................22 Outcomes...................................................................................................................................... 22 Institutional DevelopmentImpact............................................................................................... 25 Sustainability ................................................................................................................................ 26 Bank Performance ........................................................................................................................ 27 Borrower Performance ................................................................................................................ 28 7. REALIZING EDUCATION FOR ALL .............................................................................29 Current EFA Status ......................................................................................................................30 Student LearningOutcomes ....................................................................................................... 32 FundingSupport for EFA ............................................................................................................33 ExternalAgency Support ............................................................................................................ 33 8. LESSONS LEARNED ...................................................................................................34 REFERENCES ....................................................................................................................... 37 ANNEX A.BASIC DATA SHEET .............................................................................................39 ANNEX B PROJECT AND NATIONAL EDUCATION DATA . ..................................................49 ANNEX C.PROJECT OUTCOMES AND OUTPUTS ...............................................................59 ANNEX D: PERSONSAND ORGANIZATIONS CONSULTED ...............................................65 This report was prepared by Dean Nielsen who assessed the project in August 2005 The report was edited by Bill . Hurlbut. Pilar Barquero and Marie-Jeanne Ndiaye provided adrmnistrative support. PRINCIPAL RATINGS UttarPradesh Basic Education Project (UPBEPI) ICR* ICR Review* PPAR Outcome Highly Satisfactory Satisfactory Moderately Satisfactory Sustainability Likely Likely Highly Likely InstitutionalDevelopmentImpact Substantial Substantial Substantial Bank Performance Highly Satisfactory Highly Satisfactory Highly Satisfactory Borrower Performance HighlySatisfactory HighlySatisfactory Highly Satisfactory Second UttarPradesh Basic Education Project fUPBEP II) ICR* ICR Review* PPAR -- Outcome Highly Satisfactory Satisfactory ModeratelySatisfactory Sustainability Likely Likely Highly Likely InstitutionalDevelopmentImpact Substantial Substantial Substantial Bank Performance HighlySatisfactory Highly Satisfactory Highly Satisfactory Borrower Performance HighlySatisfactory Highly Satisfactory Highly Satisfactory District Primary Education Project (DPEPI) ICR* ICR Review* PPAR Outcome Satisfactory Moderately Satisfactory ModeratelySatisfactory Sustainability Likely Likely Highly Likely InstitutionalDevelopment Impact Substantial Substantial Substantial Bank Performance Satisfactory Satisfactory Satisfactory Borrower Performance Satisfactory Satisfactory Satisfactory District Primary Education Project I1 (DPEP 11) ICR* ICR Review* PPAR Outcome Satisfactory ModeratelySatisfactory Moderately Satisfactory Sustainability Likely Likely Highly Likely InstitutionalDevelopment Impact Substantial Substantial Substantial Bank Performance Satisfactory Satisfactory Satisfactory Borrower Performance Satisfactory Satisfactory Satisfactory *The ImplementationCompletionReport(ICR) isaself-evaluation by the responsibleBank department.The ICR Reviewis an intermediateIEGWBproductthat seeks to independentlyverify the findings of the ICR. KEY STAFF RESPONSIBLE Project Task Manager/Leader Division Chief/ Country Director Sector Director Uttar Pradesh Basic EducationProject Appraisal John Middleton Richard Lee Skolnik HeinzVergin Completion Edward H. Heneveld EmmanuelY. Jimenez Edwin Lim Second Uttar Pradesh Basic Education Project Appraisal Adrian Verspoor Richard Lee Skolnik HeinzVergin Cornoletion Edward H. Heneveld EmmanuelY. Jimenez Edwin Lim District Primary Education Project Appraisal John Middleton Richard Skolnik HeinzVergin Completion Prema Clarke Charles C. Griffin Michael F. Carter District Primary Education Project I1 -- - -- Appraisal Adrian Verspoor Richard Lee Skolnik HeinzVergin Completion Susan E. Hirshberg Charles C. Griffin Michael F. Carter PREFACE This is a Project Performance Assessment Report (PPAR) on four interrelated education projects in India: Uttar Pradesh Basic Education Project (UPBEP I; Credit 2509) for US$165 million, which was approved in FY93 and compelled on schedule on September30, 2000; Second Uttar Pradesh Basic Education Project (UPBEP 11; Credit 3013) for US$59.4 million, which was approved in FY98 as a complement to UPBEP I, and completed on scheduletogetherwith UPBEP I on September 30,2000; District Primary Education Project (DPEP I; Credit 2661) for US$206.3 million, which was approved FY95 and completed after a 15mo. extension on June 30, 2003; District Primary Education Project (DPEP 11;Credit 2876) for US$425.2 million, which was approval FY96 as complement to DPEP I and was completed on time, together with DPEP I, on June 30,2003. The PPAR findings are based on the following sources: a review of the projects7files and the Implementation Completion Reports (ICRs);review of published and unpublished literature on the projects and related educationaltopics in India; interviewswith current and previous task managers,project staff and technical advisors,central,provisional, and district officials in the Ministry of Human Resource Development and Project Offices; nongovernment organizations (NGOs); field visits to New Delhi and two states (Uttar Pradesh and Karnataka) during an IEG mission held in June and July of 2005; and a reanalysis of selected data on learning outcomes. Data was collected from two states, the northern state of Uttar Pradesh, where the UPBEP was implemented, and the southern state of Karnataka, one of the six original DPEP I/II states. In each statetwo districts were visited,one near the capital and one more remote. And in each district 2-3 schoolswere selected at random for spontaneousvisits (schools were not informed in advance). In the end, ten schoolswere visited, all fiom relativelypoor areas (given the UPBEPIDPEP focus on low literacydistricts),but covering a wide variety of social and cultural contexts. (see Annex D for a list of schools.) The original Uttar Pradesh Basic Education Project (UPBEP) was not numbered,but to distinguish it from UPBEP I1it will be referredto in this review as UPBEP I. Likewise, the first DPEP project was unnumbered, but to avoid confusion,it will be referred to herein as DPEP I. When the terms UPBEP and DPEP are used, they refer to the pair of related projects or to their concepts. This PPAR was one of a series which was undertaken during 2004-2005 to provide in inputs into a larger IEG evaluationon World Bank support for primary education. In light of that purpose, relativelymore material has been presented in ths "enhanced" PPAR than is the IEG standard (in particular, the Section 7, "Realizing Education for All.") As a contributorto the largerprimary education study, this PPAR will appear on that study's webpage (www.worldbank.or~/ie~/education). The author was assisted during the field work phase by a local consultant from New Delhi, Suman Sachdeva, Ph.D., whose important contribution is gratefully acknowledged. The author also expresses his appreciationto all of those who made time for interviews and provided information. Following standard IEG procedures, copies of the draft PPAR were sent to government officials and agencies for their review and comments. However, no commentswere received. SUMMARY India was a majorparticipant in the 1990World Conference of Education for All held in Jomtien, Thailandwhich was organized to heighten national and international commitment to improved basic education enrollments and outcomes. At that time India harbored the world's largest number of out-of-schoolchildren(around 25 million) and witnessed huge disparities between males and females and between the mainstream and lower castes in enrollments, completion and learning outcomes. Four years before (1986) the country had already reoriented its national education policy towards primary education. Jomtien and a 1992Program of Action added urgency to the agenda and opened the country to large scale financial assistance from major donor agencies. Prior to this period the World Bank had not provided any financial support to Basic Education in India. The initial breakthrough came in 1992when the Bank agreed to a Social Safety Net Adjustment Credit to help India recover from a balance of payment crisis which had forced reductions in government expenditures on social services. The credit included planning for a "centrally sponsored (subsidized) scheme" that was eventually called the District Primary Education Program (DPEP). Totally planned by Indian educators and managers, based on small- scale efforts at innovation and reform, the scheme was to channel funding and technical assistanceto state and district governments for both expanding access to primary education and improvingits quality. Increases in external financialsupport for primary education also began at this time, with the provision that it all be coordinated by the GO1under the DPEP banner. The first Bank-supported project under this program was mounted in the state of Uttar Pradesh, the country's most populous state (population 165million) and an underachiever in education (see paragraph 6.I), but blessed with dynamic educationalleadership. The Uttar Pradesh Basic Education Project (UPBEP I; IDA credit of US$165 million) was launched in 1993and was followed in 1997by a companionproject (UPBEP 11),which added about US$60 in IDA credits to cover increased infrastructure,teachers and textbooks in the original UPBEP I districts,given much higher than expected new enrollmentsthere (see paragraph 2.2). Reassured by the viability of UPBEP I, the government launched (in 1994)the large District Primary Education Project (DPEP I; IDA credit of US$260.4 million) in another six states, This was also known as DPEP Phase I, since other phases were expected to follow covering additional districts and states, the preparation of which was covered in Phase I. In 1996,Phase I1was launched (DPEP 11; IDA credit of US$425.2 million), which expanded both the number of districts in the original DPEP states but also the number of states covered (from 6 to 9). These four primary educationprojects were eventually followed by four additionalBank-supportedDPEP projects in other states,two state-based education adjustment projects, and finally, the country-wide 2004 Elementary Education Project (SSA), using a sector-wide,which brought DPEP to scale in the entire country. The design features of the four projects (two pairs) were similar, given they all followed the general DPEP model, constructed in the early 1990s.Their objectivesincluded: a) improving access to primary education; b) improving school retention (or, in other words, reducing dropout); c) improving student learning outcomes (achievement test scores); d) improving institutionalcapacityto manage basiclprimaryeducation at the national (DPEP YII only), state and district levels; and e) enhancingcommunityparticipationin primary education. The two pairs of projects were organized somewhat hfferently, but both included the followingmain components: a) improving school accessby building schools and creating alternative education programs for the difficult to reach children;b) improving school retention and learning outcomes by creating new primary school curricula and learning materials,providing inservice training and professional support to teachers, rehabilitating school facilities, and encouraging community participationin primary education; and c) building institutionalcapacity through strengthening institutionalstructures and improvinginformation flows, research and evaluation, and planning. In both pairs of projects the project design was consideredby the mission to be have generally relevant for the attainment of objectives, except that school dropout needed more specific and focused interventions related to root causes of dropout (both supply- and demand-side), and learning outcomes improvement needed a stronger focus on changes at the classroom level (intermediate outcomes,such as improved teacher attendance, more time devoted to learning, and improved teaching methods), and improved institutionaldevelopment capacity needed more of a focus on outcomes(sustainableinstitutional change instead ofjust output). Attainment of project objectives were generally evaluated in relation to key performance indicators (where they existed), and these varied across the two pairs of projects. UPBEP VII aspired to improve access in terms of district capacityto enroll 100percent of 6-10 year olds and 75 percent of 11-13year olds, levelswhich were not quitereached, especially if net enrollment ratios-the appropriatemeasure in this case -are used. The goal for student retention was a 50 percent reduction in student dropout,which turned out to be difficult to assess, given limitations in data, but the best indicator (a real cohort analysisin a sample of districts) revealed actual changes were closer to 35 percent. Learning outcomes were to have been improved by 50 percent over baseline levels, and although this was alleged in the ICR to have been accomplished,the validity of the test was called into question by the same ICR. Institutional capacity building did not have good outcomemeasures, but there was evidence of positive changes in a number of important management domains. On communityparticipation, the project activated village education committees in most locations, but it was unclear how many became "fully hnctional" or even what that meant. In DPEP UII improved access was assessed in terms of the reduction in enrollment gaps between girls and boys and disadvantaged groups and the mainstream. Results show this was accomplishedin 6 of 7 states in DPEP I, but only 6 of 10in DPEP 11. Enrollment of children of scheduled castes came basically into line with their shareof the population but not that of the scheduledtribes, which remained underserved. Reducing dropout to below 10percent was only accomplishedin the DPEP I districtsof Kerala state (where DO was already low); in DPEP I1 districts this was accomplished in only 16percent of districts. The target of improving learning outcomes by 25 percent appears to have been met in the aggregate, but between 13and 39 percent of districts(dependingon grade level and subject) did not reach the goal. Also there are serious questions about the validity of the DPEP achievement tests. As in UPBEP I/II there were no good outcomes measures to use in assessing the fulfillment of capacity building at the national, state, and district level, but a body of evidence exits to show that meaningful improvement was attained at all levels. On community participation, all states activated village education committees (in various formats),but as in UPBEP VII but a large proportion of them could not be assessed as having become fully functional. Outcomesratings for the projects are based on their efficacy (see above), relevance and efficiency. The two projects in eachpair had the same ratings. Outcomes ratings for both UPBEP and DPEP projects are moderately satisfactory, even though the pattern of achievement across the different sub-objectives is different. The sustainabilityof both pairs of project is rated highly likely, given the fact that most of the projects' design features have been spread to all districts in the country where they are taking root. Institutionaldevelopment is rated substantial forboth pairs of projects. Bank and borrower performance are rated highly satisfactoryfor UPBEP and satisfactory for DPEP, with the differencecoming largelyin the extent to which project components were fullyimplemented. Many lessons have been learned in the course of UPBEP UII and DPEP 1/11 implementation, the main ones being: More concerted effort is needed to provide access and better learning outcomes among tribal children and the disabled; The information-based planning and decision making approachespromoted by UPBEPDPEP are only as good as the data availableto them; there is an urgent need for higher levels of qualitycontrol in the government's education information and assessment systems. The main constraint to the GOI's reaching its goal of universal primary school completionby 2010 is no longer school accessbut high student dropout; stronger commitment to dropoutreduction and more effective interventions (based on local research on causes) are needed, targeting locations where the problem is particularly seI1ous. There is an urgent need for strategicthinking and decision making concerning the deployment of "para teachers," taking into consideration equity issues, cost-effectiveness, sustainabilityand its longterm impact on the teaching profession; There is a need for a clear understanding of the reasons and consequencesof the rapid growth of unrecognized private schools, and sharpened governmentresponses; Improving student learning outcomesneeds more thanjust setting goals and mobilizing inputs; it needs coherent changes in intermediateoutcomes at the classroom level. Vinod Thomas Director-General Evaluation 1. BACKGROUNDAND CONTEXT 1.1 The Uttar Pradesh Basic Education Projects and the District Primary EducationProjects were prepared in the early 1990sin the context of a major Indian reform movement in basic education. In 1986the country enacted its New Education Policy (NEP), renewing government commitment -both financialand technical -- to pr~maryeducation. In 1992the nation's Central Advisory Board on Education revised the W E and added a Program of Action, which included a decentralizedapproach to the planning and execution of Education for All. The World Conferenceon Education for All (EFA) held in 1990added to the Government's sense of urgency to attain universal primary education and opened the possibility of large scale financial assistance from major donor agencies. The sense of urgencywas well founded: at the time India harbored the world's largest number of out-of-schoolchildren (around25 million), and huge disparities existedbetween males and females and between mainstream and lower castes in enrollments,completion, and learning outcomes. 1.2 During the 1980sto early 1990sthe central and state educationprograms had already benefited from small-scale programs of external support for innovation and reform in primary education (e.g., that provided by DfID, Netherlands, SIDA, and UNICEF). In 1992 a balance of payments crisis in India forced the government to reduce expenditures for social services, prompting the World Bank to provide a US$500 million Social SafetyNet Adjustment Credit, which included planning for a large "centrally sponsored scheme" called the District Primary EducationProgram (DPEP), which would become the focal point of expanded externalfinancial support. Totallyplanned by Indian educators and managers and built on the earlier small-scale programs, the scheme was to channel funding and technical assistance to state and district governmentsfor both expanding accessto primary education and improving its quality. 1.3 The first Bank-supportedproject under this program was mounted in the state of Uttar Pradesh, the country's most populous state (population 165million) and an underachieverin education (seeparagraph 6.I), but endowedwith dynamic educational leadership. The Uttar Pradesh Basic Education Project (UPBEP I; IDA credit of US$165 million) was launched in 1993and was followed in 1997by a companion project (UPBEP 11),which added about US$60 in IDA creditsto cover increased infrastructure, teachers and textbooks in the originalUPBEP I districts, given much higher than expected new enrollmentsthere (see paragraph 2.2). Reassured by the viability of UPBEP I, the government launched (in 1994)the large District Primary EducationProject (DPEP I; IDA credit of US$260.4 million) in another six states.' This was also known as DPEP Phase I, since otherphases were expected to follow covering additional districts and states,the preparation of which was covered in Phase I. In 1996,Phase I1was launched (DPEP 11;IDA credit of US$425.2 million),which expanded both the number of districts in the original DPEP states but also the number of states covered (from 6 to 9).2 The design features of DPEP I1were also similarto DPEP I. By 1997UPBEP I and I1 and DPEP I/II were supervisedthrough "joint review missions," which meant that feedback, reflection, and program adjustmentinvolving GO1actors and major donor agencies were mainly undertaken at the national level. 1. The states were Assam, Haryana, Karnataka, Kerala, Maharastra, and Tamil Nadu. A seventh state, Madhya Pradesh is included in government descriptions, but its external was from the European Commission not IDA. 2. The new IDA supported states under DPEP I1 were Himachal Pradesh, Orissa, and Uttar Pradesh. West Bengal and Gujarat were also involved in DPEP I1 but their external funding came from DfID and Netherlands, respectively. 1.4 The two groups of projects were completed in 2000 and 2003 respectively,but Bank support for universal primary educationin India continued under additionalDistrict Primary Education Projects, two related developmentpolicy projects (Andhra Pradesh), an IDF Grant (Karnataka), and nation-wideproject, taking a sector wide approach, the Elementary Education Project (Sarva ShiksaAbhiyan or SSA), approved in 2004. The full set of Bank supported projects coveringprimary education is as follows: Table I.I.Bank-FinancedProjectsCovering Primary Education in India Project Name States Districts Approval Date Closing Date US$ millions Uttar Pradesh Basic EducationProject 1 17 0611011993 09/30/2000 165 Second Uttar PradeshBasic Ed Project 1 17 (same) 12/04/1997 09/30/2000 59.4 District Primary Education Project 7' 42 11/22/1994 06/30/2003 260.3 District Primary EducationProject II 7"+5 92 06/06/1996 06/30/2003 425.2 District Primary EducationProject Ill 2 17 12/04/1997 09/30/2006 152 (Biharand Jharkhand) RajasthanDistrict Primary Ed Project I 1 10 06/08/1999 12/31/2005 85.7 Uttar Pradesh DistrictPrimary Ed Proj Ill 1 42 12/16/1999 03/31/2006 182.2 Rajasthan District Primary Ed Project II 1 9 06/21/2001 12/31/2006 74.4 ImprovingPublic Expenditurein Education 1 01/21/2004 05/24/2007 0.5 in Karnataka Andra Pradesh Economic Reform Proj II 1 14 0211012004 08/14/2004 6.6 Elementary EducationProject(SSA) 28 + 7 union 600 04/20/2004 12/31/2007 500 territories Includesthe original6 states which received IDA funding plus a 7thfunded by the EC *'Covers additionaldistricts in the original DPEP states 2. PROJECT DESIGNAND IMPLEMENTATION 2.1 The two pairs of primary educationprojects, UBBEP and DPEP, were created in the early 1990sby Indian educators and government officials as vehicles for realizing the country's Education for All (EFA) agenda. The objectives of the four projects (two BEP and two DPEP) are shown in Box 2.1. Box 2.1. Project Objectives(as per Development CreditAgreement) -- Uttar Pradesh Basic EducationProject(UPBEP I): To assist UttarPradesh in increasing enrollment in, and completion oJ;basic education and to improve its quality. To this end, the Project would also aim to strengthen institutional capacity in UttarPradeshforplanning and management of basic education. Second Uttar PradeshBasic EducationProject (UPBEPII): To assist UttarPradesh in increasing enrollment in, and completion oJ basic education and to improve its quality. District Primary Education Project @PEP I): To build national, state, and district level managerial andprofessional capacityfor sustainable primary education development, decrease drop out rates, increase learning achievement, and improve access of children toprimay education, inparticular for female students as well as scheduled caste and tribal students, and enhance community participation in primaly education inprimary education in the Project States and Project Districts. . SecondDistrictPrimary EducationProject @PEP 11): Tobuild national,state, district and sub-district level managerial andprofessional capacityfor sustainable primaly education development in the Project States, and, with respect toProject States and Project Districts, to decrease drop out rates of childrenfrom primary education, increaselearning achievement of childrenreceivingprimary education, improveaccess of children toprimay education, inparticular for female students as well as scheduled caste and tribal students and children with disabilities, and enhance community participation in primary education. (Underlined phrases are those unique to DPEP 11.) UPBEP 2.2 The two UPBE Projects had the same main objectives,given that the secondwas designed as a way to expanding resources for the first.3 The second,however, did not include the objectiveof strengthening institutionalcapacity at the state and district levels, since this was already a goal in the first and no new states and districtswere added. The two projects were supervised as one (once the UPBEP I1 came on line in 1998)and used the same outcomes measures to determine fulfillmentof their shared objectives. The UPBEP I developmentcredit agreement4contained the followingdelineationof terms used in the objectives: "completion" was defined as reduction in school dropout and repetition, and "improvement of quality" as changes in learning achievement. Basic educationcovered both lower (grades 1-5) and upper primary schooling (grades 5-8). 3. Given the fact that the UPBEPI was carefully planned, including projections of future student enrollments and beneficiary assessments of demand, it is curious that enrolment increases were seriously underestimated. This must be considered a design flaw. However, the strategy taken to overcome this flaw appears reasonable: since UPBEP I was already at its limit of proportion of funds to be allocated for infrastructure and new personnel, the creation of a expansion project provided a way to add such resources. Not having done so would have created much more overcrowding than actually occurred and would have seriously undermined the quality improvement objectives. 4.World Bank, Development Credit Agreement, Uttar Pradesh Basic Education Project (CR 2509-IN), July 7, 1993. DPEP 2.3 DPEP I and DPEP I1had basically the same objectives,but served different districts and states. Therewere minor differences:DPEP I1added "children with disabilities" to the list of disadvantaged groups to be servedby the project and it explicitly included the sub-district level in its capacity building agenda (but this was implicit in DPEP 1's goal to build block and cluster resource centers and village education committees). Also, DPEP VII were different fiom UPBEP VII in their objective to build capacity at the national level, given the fact that they included more than one state. Also, the DPEP projects included enhancing community participation among their objectiveswhile the UPBEP projects did not, but again this goal was included in the projects' performance indicators.Finally, while UPBEP covered both lower and upper primary education,DPEP covered only lower primary ed~cation.~ Both UPBEP and DPEP 2.4 The convergenceof the four projects on objectivesis largelybased on the fact that design features and experiencesin the first UPBEP project fed into the design of the DPEP projects; in addition,by 1997 seventeen Uttar Pradesh districts became a part of DPEP 11, meaning that UPBEP and DPEP were being implemented sideby side in the state, leadingto considerable cross-learning. Also, by 1997,UPBEP and DPEP projects were all supervised together using a common set of project guidelines.6 This and the explanations above make it possible to assess the four projects in this PPAR in relation to the same five objectives: Improving access to primary education, especiallyfor female children and those from scheduled castes and tribal areas (DPEPI1added "children with disabilities'); Reduce primary school dropout;7 Improve student learning outcomes; Improve institutionalcapacity (DPEP covers national in addition to state, district and sub- district); and Enhance communityparticipation in primary education(only explicitly mentioned in DPEP, but implicit in UPBEP). Performance Indicators 2.5 Performance indicators for these objectivesvaried somewhat across the two sets of projects (see Annex C, Figures C.1 and C.2 for a full listing). UPBEP initially defined increased access as a 50% reduction in the proportion of school girls not in school and the increasein both school and nonforrnal educationplaces, but after mid-term review redefined it as capacityto enroll all 6-10year olds and 75% of 11-13year olds. DPEP defined increased access as a five percent reduction of dsparities across gender and social groups. Concerning drop-out,UPBEP 5. This ternnology can be confusing. In UPBEP IIII basic education is defined as lower and upper primary education. Thus, the term primary education could also be used to refer to it. Jn DPEP, the use of the term primary education referred only to the lower prlmary cycle (grades 1-4or 1-5,depending on the state). 6, In at least of one DPEPs (DPEPII) IDA credit negotiations included an agreement that the Project would not "make any changes to the DPEP or DPEP guidelines" (World Bank, 1996). 7. UPBEP also committed to reducing student repetition of grades. However, repetition was not seriously tracked (one ICR indicated there was no baseline data for this) and can be subject to teacher manipulation (automatic promotion), so this indicator is not considered here (see also Section 4: Monitoring and Evaluation). called for a 50% reduction, whereas DPEP specifiedreduction to less than 10%.On learning outcomes, UPBEP projects aspired to a 50% improvementin grade 5 test scores (over baseline) and DPEP called for a 25% improvement. On institutional capacity,neither set of project provided outcomes indicatorsfor the objective; instead they committed to specific outputs under the capacitybuilding component of the projects (e.g., the establishment of offices, the production of materials, the provision of so much training, and the conduct of so many evaluationand research studies). To evaluate this objective, this PPAR drew selectivelyfrom the outputs indicators as well as other evidencesof improved institutional capacity. Finally, on community participation, the two pairs of projects called for fully functional schooVcommunity organizations for at least 50% of schools in Project districts. 2.6 To reach their objectives, project designers developed a set of components. components. The projects in the two sets had basically the same componentsand subcomponents,except as noted. Components costs, both estimated and actual, are also noted. UPBEP o Improving Access (estimated and actual costs -UPBEP I: US91.38 million, US$91.16 million; UPBEP 11:US$23.8 million, US$23.38 million). This was to be accomplished through: a) a community school construction and maintenanceprogram; and b) (for UPBEP I only) expandingand improving non-formal education. The latter was phased out after the mid-term review showed little demand for it. It was not included in UPBEP 11. Improving Quality and Completion(estimated and actual costs UPBEP I: US$83.68 - million; US$80.73 million; UPBEP 11: US$51.9million; US$50.83million). The projects were designed to support UP government in intensifyingand expandingthe followingprogram in project districts: a) strengtheningcommunityparticipation (through Village Education Committees);b) improving readiness to learn (through early childhood education and development activities);c) improving teacher and staffperformance (through training, and additional teachers and other personnel); d) improving curriculum,textbook and teaching materials; e) improving school management, and f) rehabilitation of school facilities. Beginning with UPBEP I1 Village Education Committeeswere to receive grants to cover school improvementsand teachers grants for the purchase or construction of teaching aids. Building Institutional Capacity(UPBEP I only) (estimated and actual costs: US$18.71 million; US$20.12 million). This covered: a) improving institutional structures(state EFA office, State Institution for Educational Management and Training (SIEMAT);District Institute for Educationand Training(DIET); and Block Resource Centers); b) improving information flows, research, and planning (including school mapping and micro-planning (based on household surveys); management information system development;assessmentof learning achievement (for project districtsand state-wide), and research and evaluation studies. DPEP Improving quality and access inprimary education (estimated and actual costs - DPEP I: US$225.1 million, US$226.84 million; DPEP 11:US$483 million; US471.2 million). This was to include: a)building district institutional capacities through construction of district project offices,strentheningDIETs, carrying out pilots studies to strengthenschool supervision);b) reducing dropout by strengthening school-communityorganizations,expanding or improving facilities, carrying out awareness campaigns, c) improving learning achievement through enhanced inserviceteacher training, use of improved learning materials and teaching aids, targeted interventionsfor girls, SCIST students and (in DPEP 11)children with mild to moderate disabilities);and piloting earlychildhood education programs; and d) improving access thought the provision of approximately 470,000 new studentplaces (DPEP I; no target for DPEP 11)in new schoolsand classrooms, and support for alternatives to formal school for isolated small communities and for working children. Building National Institutional Capacity (estimated and actual costs - DPEP I: US$20.5 million, US$20.41 million; DPEP 11: US$1.4 million;US$1.3 million). This included two subcomponents:a) national management, and b) national technical assistanceprograms. Building State Institutional Capacity (estimated and actual costs DPEP I: US$24.8 - million, US$24.0 million; DPEP 11:US$7.3 million,US$6.7 million. This was designed for: a) strengthening state program management structures;b) improving instructional materials development and production; c) improvement in teaching and inservicetraining (through state level institutionssuch as State Councilsfor Research and Training (SCERT)); d) improvement of educational,planning, management and monitoring (through state level institutions such as SIEMAT;and e) improvement of educationalresearch and evaluation. Distance Education Program (DPEP I1only) (estimateand actual costs: US$4.3 million, US$4.1 million). This was to cover the development of state-basedservices to support ongoing inservice and other training activities. 3. IMPLEMENTATIONAND COSTS UPBEP 3.1 Costs and Financing. Despite detailed preparation and planning, school construction and front-loaded activitiesstarted slowly, leading to a disbursement rate of 6 percent during the first two years (compared to a planned 30 percent). This was attributedto confusion over unfamiliar building designs, insufficient technical support for community-based civil works, delays in staffingstate and district support agencies(e.g., SIEMAT and DIETs), and the unexpected bifurcation of districts (from 10to 17),requiring additionalpreparation efforts. 3.2 Despite slow disbursement at the beginning, both UPBEP I IDA and GO1funds were fully disbursed. UPBEP I1also disbursed near one hundred percent of IDA commitments (99.6) and 92.4 percent of Government funds. IDA accounted showed less than estimated spendingfor "improving quality and completion" (87.3 percent) and more for "improving access" (110 percent). In fact, higher than anticipatedenrollment expansion led in turn to the hiring of more teachers than expected and higher the anticipatedexpendituresfor teacher salaries (54 percent of total comparedto SAR estimate of 39 percent). 3.3 Project Restructuring. UPBEP 1's mid-term review in late 1996revealed some needs for restructuring. As mentioned above,the higher than expected increase in enrollmentsled to the overcrowdingof schools (in many places pupil teacher ratios in excess of 60:1) and the need forUPBEP 11. This allowed for the quick expansion of school numbers and the hiring of 17,000 new teachers, almost 6000 of whom were locally-recruited contract teachers over half of whom were placed in one-teacher schools(an UPBEP innovation during its final year).8 When UPBEP 11was launched the two projectsrevised its access targets to "capacity to enroll" all 6-10 year old project district children and 75% of 11-13 year olds. In addition, a new feature was added which was expected to motivate better teacher performance: annual grants (Rs 500) to them for purchasing and making learningmaterials. 3.4 The mid-term review also noted that planned expansion of conventionalnon-formal education centers (catering to youth) was well below target, and thus decided to phase this out, in favor of more innovativeprograms (e.g., NGO-run bridging programs to fonnal schooling) that were being developed under the recently launched District Primary Education Project (DPEP). Just prior to the mid-tern review (October 1996),Uttar Pradesh launched its own participationin DPEP (DPEP II), involving 18districts. This, in turn, led to the creation of a third district primary education project (UPDPEP 111),covering42 districts, in 1999. By the close of UPBEP VII most of the state's 84 districtswere involved in eitherUPBEP or DPEP, which accordingto one knowledgeable informant (Ayyar, 2005), created a condition for more effective implementation in the state compared to most other states involved in the DPEP. 3.5 Changes in UPBEP Sewice Delivev. UPBEP was designed to support the piloting of 1000Early ChildhoodCase and Education centers attached to schools. During the course of implementation a decisionwas made to begin with strengtheningexisting Integrated Child Development Services(ICDS),providing staff training and educationalmaterials to enhance school readiness of participatingchildren, and changing location and hours of ICDS so that older girls could attend school instead of needing to mind younger siblings (who could be left at ICDS centers). Positive experiencewith this led to a statepolicy to locate all new ICDS centers on schoolpremises. Concerningregular teacher professionalsupport, the creation of a chain of services, from the state resource (SIEMAT) to the districttraining institutes (DIETS)to Block and Cluster Resource Centers to the school was a long (yet to be optimized) trial and error process. Early on it became clear that the pre-planned inservice curriculum that was to be deliveredwas not appropriate;instead professional support was provided on the basis of need (e.g., teaching for basic skills acquisition -- reading and math, and the use of multigrade teaching).g 3.6 With respect to project-generated learningmaterials, the project began with its conventional approach to textbook revision, but soon, at the urging of some participating teachers, moved to the creationof more attractive,user-friendly, teacher-influencedmaterials, 8. Theuse of contract or "para" teachers has become a controversial issue. Although it was originally set up to allow communities having teachers shortages to recruit local secondary school graduates as teachers (after minimal training), it has expanded to the point that some districts are hiring contract teachers for mainstream schools as a way to reduce teacher costs. Govinda and Josephine (2005) claim that this is eroding professional standatds for teachers and creating a second tier relegated mostly to the poorer areas. 9. It is not clear how needs-based the revised approach was, since virtually all block centers visited during the PPAR were covering the same topics. including supplementsconveying regional and cultural themes. This set the pace for both UPBEP UII and DPEP VII. Likewisethe Projectmobilized community resources, such as Woman's Collectives,to create incentives, curricularrevision, and teacher sensitizationwith respect to improved girls' participation and treatment in schools. A special project component supportingexperimentationwith innovations and pilot projects led to somenew initiatives for community libraries, double shifting and work-experienceprograms for girls, but no major reform effort or movement came from it. DPEP 3.7 Costs andfinancing. Both DPEP I and DPEP I1disbursed all IDA funds in terms of Special Drawing Rights (SDR); in terns of US dollars, there is an appearance of underspending, but this is due to the devaluation of the India Rupee against the US dollar. In both projects spendingfor books and learning materials was far below budget, due to overestimationof costs at appraisal. Also, spending for consultants was below expectations,especially for international consultants,which were sparingly used. Finally, procurement of civil works was designed to be mostlythrough national competitivebidding, but since in reality constructionwas largely managed by Village Education Committees, local shopping (by communities)became the dominant means of procurement for construction. Although an effectivemeans of encouraging the developmentof local capacity, such reforms were often accompanied by weaknesses in financialmanagement. A review of fiduciaryresponsibility in DPEP I1 shows numerous cases in which the state project officerswere unable to reconcile the amountsof outstanding advancesto local entities (blocks)with their statements of expenses. Numerous audit reports brought out such weaknesses,but some stateswere slow in responding to such issues. Overall, it seems that trainingof local staff in financial management (accounting,reporting and internal controls) was inadequate. 3.8 Already in the first year of DPEP I, preparation for DPEP I1began. As with DPEP I preparation for DPEP I1 was detailed, involvingthe use of beneficiary and design studies in 40 new districts in the originalPhase I states and in 3 new states with their 30 project districts. DPEP I1became effectivein October of 1996. Eventually,DPEP I was extended for an additionalyear, allowing it to close at the same time as DPEP 11, which closed on time. 3.9 Implementation Progress. The Projects were implementedaccordingtheir designs; there was no restructuringof either objectives or components. Relatively smooth implementationwas attributed to the fact of little turn over among Project managers at the national, state and district levels, the strong sense of national and regional ownership of the Project, and the availabilityof skilled and committedpersonnel at all levels in most states. Unexpected interruptionscame in 1996, 1998and 1999when general elections were held (normallyonly once in five years), drawing school personnel into election duty. State counterpart funding was generallyprovided at agree amounts and in a timely fashion, except in Assam and Orissa,where it caused delays. Funding flows from the center governmentwas also delayed especiallyin the early years, causing some turmoil. 4. MONITORINGAND EVALUATION 4.1 This sectionwill focus on three aspectsmonitoring and evaluation in and by the UPBEP and DPEP projects: a) the monitoring and evaluationdesign; b) the implementationof monitoring and evaluation in the projects; and c) the utilization of monitoring and evaluation results. MONITORING EVALUATION AND DESIGN UBBEP 4.2 Sinceno logical framework was provided in the UPBEP project document^,^^ this review gathered information from various sectionsof the two Staff Appraisal Reports, e.g., for UPBEP I, "Goals and Objectives" and Annex 21, and for UPBEP 11,Block 1 and Annex 1. In the PPAR these are presented in a single matrix (see Annex C, Figure C.1) and include project objectives,performance indicators and targets, data collectionmethods, and end of project status. 4.3 For the first three objectives, those concerning access, dropout and learning outcomes of students, the M&E design appropriately includesoutcomesindicatorswith measurabletargets. A weakness in the assessment of access was the use of both gross enrollment ratios (all children enrolled, irrespective of age, in relation to the school-agepopulation) and the creation of new studentplaces. (The latter is more appropriateas an output target). Also, when under UPBEP 11 the enrollment goals changed to the "capacity to enroll 100percent of 6-10 year-olds and 75 percent of 11-13year olds" (as indicated by gross enrollment ratios of 100 and 75, respectively), a more appropriate indicatorwould have been net enrollment ratios (which removes over- and under-age children from the ratio calculation). Also, the revised enrollment goals did not specify targets for the disadvantaged groups,underminingthe possibility of showing equity improvement. '' Improving student learning outcomeswas appropriatelyaddressed by conducting a sample-based mid-term and final assessment and comparing the outcomeswith baseline results. However, it would also have been useful to have indicators of intermediate outcomes, such as the time spent by students on academictasks, and teacher attendance and performance (pedagogical skills), all essentialimmediate conditions for improved learning (see discussion in Section 6 on relevance). One common weakness in the assessment of all three objectives was the lack of a counterfactual,namely control group data that could be used in attributing results to project interventions. Without them, it is not clear whether the positive changes are being influenced by the Projects or other factors also influencing non project schools (see Section 6, relevance). 4.4 On the capacity building objective, indicatorswere created for each UPBEP capacity building sub-objective at the state and district levels; however, almost were output indicators (number of teachers trained, etc.). A more robust design would have been to use more outcomes. indicators, suchpercent of schools in a districtmaking certain improvements or specificchanges in trainer/ teacher/manager/VECperformance.Also, as with the other objectives, there was no use of the counterfactual (whether similarmanagement changes were taking place in non-project 10.UPBEP I was designed before logframeanalysiswas required in project documents. 11.A GER of 100%does not mean full access for all, since the GER can go above 100%. For example, one hundred percent could break down into 110%for boys and 90% for girls, a 20% enrollment gap. districts), nor (aspointed out by Ayyar, 2005) any comparison acrossproject districts on improved institutionalcapacity (i.e., which districtswere improving the most/least and why). Given a lack of outcomes measures, it would have been more appropriate for capacitybuilding to have been specified a means to project objectives(e.g., as a component)rather than an objective itself. 4.5 Finally, concerningenhancing communityparticipation inprimary education (mentioned as a goal under "improving qualityand completion"), the design was to assess the percent of villages havingfully functional Village Education Committees,with a target of 50 percent. Although this appears to be a reasonableindicator, creating a good definitionand measure of "fully functional" was challenging. DPEP 4.6 The main M&E design features of the DPEP I/II were articulated in Annex 5 of the SAR in both projects and in passages of SAR text. These design features are summarizedby this review in ANNEX C (Figure C.2). Improving access in both DPEPs is operationalizedin terms of reduced enrollmentdisparities across gender and social groups (SC/ST) -to below 5 percent. While this treats the equity aspectsof access, it fails to treat the expansion part (disparitiescould be reduced by boys' enrollmentsgoing down). For this DPEP would have needed to show if enrollmentswere increasing, and notjust in absolute terms but in relation to population (generallyindicated by gross [orperhaps better neJ enrollmentratios). On reducingprimary school dropout DPEP set an absolute standard (reductionto below 10percent). Given the wide variations in baseline status on this across states and districts, it would have been more appropriateto put this in relative terms (an x%decline in dropout), as was done in UPBEP ~11." And on improving student learning outcomes, the comment about the lack of intermediate outcomesrelated to learning (e.g., improved teaching behavior) mentioned for UPBEP 1/11is also relevant here. 4.7 The evaluationof institutional capacity building at the national, state, and district levels was constrained by the projects7reliance on a long list of output indicators (around 45). As with UPBEP 1/11,this was not an appropriate design for documentingthe fulfillment of the capacity building objective, for outputs such a "functional" office or hours of training delivered do not necessarily lead to improved capacity. Also, many of the indicators targets were vague (not involving hard evidence). Finally, for the fifth objective, enhancing comrnunityparticipation in primary education, the indicatorwas the same as for UPBEP UII and had the same limitations. IMPLEMENTING MONITORING EVALUATIONPROJECTS AND IN UPBEP 4.8 UPBEP VII saw the pioneering of the new computerizededucational management information system (EMIS), and thus an improvement in the accuracy and timelinessof student access and completion data, at least for lower primary education (upper primary was not covered 12.This is not to denythe importance of eventuallyeliminating dropout, a necessary condition if the school system is to reach the millennium development goal of universal primary school completion. However, the time required for this will vary from district to district, and in the short run this could be accommodated by the more flexible target. by EMIS during project years). Nevertheless, there were still problems with data quality at the source(data entry) and in the calculationof change scores, since most baseline data was badly flawed. Measuring changes in student dropout was constrainedby the faulty baseline data and also by EMIS7slack of coverage of the growing number of unrecognizedprivate schools (shifts to which were often counted as dropouts). Uttar Pradesh's SIEMAT eventually addressed this weakness by doing a sample survey of dropout using a real cohort method in six districts (SIEMAT),but it is not clear how representative the outcomeswere. For learning outcomes assessment, baseline and time seriesdata were apparentlycollected in all project districts, but serious questions were raised in the ICR about data reliability (scores were skewed in unrealistic ways) and the final results were not reported in that review. DPEP 4.9 Themonitoringof expanded access in DPEP the project included tracking changes in enrollments in project districtsbut did not relate this to population change, so it was not clear whether access was improving or not. The main explanation for this was that age disaggregated population data from the 2001 census were not available at the time of Project closing (and the writing of the ICR). It might have been possible to compute enrollmentratios from the National Family Health Surveyof 1998199which had household data on education,but the project did not do this. This gap will filled by the external studyby Jalan and Glinskaya (2002) using household data from 93/94 and 99100. Concerning gender and social group disparities, this was tracked for gender and for scheduled castes, but data for tribals was incomplete. On learning outcomes, there were problems comparingbaseline and end-of-projectresults since they used different instruments.13 4.10 For the objective of building institutional capacity,progress on many of the 46 outputs indicatorswas not reported out in the ICRs and, accordingto PPAR informants,was not fully tracked. Allegedly, the complex M&E design in the StaffAppraisal Report (SAR) was not fully "bought into" by Projectmanagement,which shaped Joint Review missions. Management generallyused the government's DPEP guidelines to shape its monitoring efforts and not all of the SAR indicatorswere in those guidelines. For example, the commitmentto "a multivariate analysis of achievement data using school effectivenessmodels" did not appear on the GO1 agenda nor did "improved classroom teaching processes for 50% of trained teachers." Likewise, many features of "establishing programs for Reading Skills and ComprehensionEnhancement and for MathematicsActivities" were not documented, includingoutcomes assessmentsfor both using an experimentaldesign. UTILIZATION MONITORING EVALUATION OF AND RESULTS 4.11 The two main "products" of the monitoring and evaluationprograms in both pairs of projectswere the management information system (EMIS), used mostly for monitoringprogress 13.This problem arose at mid-term when the test was revised based on the new curriculum. The new test was also administered at the termination of the project. This allowed for a direct mid-term to terminal comparison (MAS to TAS). To check on changes in the early years (start-up to mid-term) the project drew a 5 percent sample of schools at the midpoint of the project and re- administered the original baseline test to them: thus creating a BAS1 to BAS2 comparison. Strictly speaking there is no capacity in the test data to show changes in test scores over the entire length of the project: BAS to TAS. Nevertheless, the Project has, inappropriately, added BAS1-BAS2 and MAS-TAS change scores to determine whether the overall 25 percent increase in learning achievement was achieved. in access and retention, and the student learning assessments, used in evaluating learning outcomes. In both pairs these productswere invaluable to policy makers and program managers in their ability to document progress. In the case of EMIS, timely and user-friendly "reports cards" (stateand district) were ultimatelyproduced containing400 variables (summarizedon one sheet!) and made available within the year of data c~llection.'~These have been sent to all states and districts in hard copy and digital form (CDs) and meetings are held to solicit feedback. In addition, a thousand copies are sent to universities and research institutions each year, which are encouragedto use the reports and the raw data for research. Demand for the reports by school system managers are acknowledged to be rather modest, although growing. Also, reports fiom the districtshave shown that the use of the data by schools and Village Education Committeesin their annual plans is not widespread (see outcomes section on "management and planning" for evidence of this). 4.12 It is not clear how widespread the use of the learning outcomes measures has been. It is clear, however, that they have not been extensively mined for trends and insights. For example, The DPEP report on Phase 1student achievement devotes only one page to describing the differences among states and districts. In the analysis there is no attempt to connect performance (e.g., high or low) with the characteristicsof the location or Project Management, nor any attempt to connect it to DPEP inputs, service delivery, or intermediate outcomes such as teacher or student classroom behaviors (e.g., active learning). Moreover, what is analyzed is simplythe average districttest score and notpercent change, the main learningprogress indicator of the Project. 5. ACHIEVEMENTS OF PROJECT OBJECTIVES 5.1 This section examinesthe outcomes of the UPBEP and DPEP projects by Project Objective. The objectivesare those which were found to be common to the two sets of projects (see section2), namely: a) improving access to primary education,b) reducing primary school dropout, c) improving student learning outcomes, d) improvinginstitutional capacity, and e) enhancing communityparticipation in primary education. The examination weighs reports in self-evaluations (ICRs and attached "national reflectionpapers" by the GOI), aide-memoires from supervision missions, informationin subsequentproject design papers (e.g., new phases of DPEP or of SSA),reflections in papers and publicationswritten about the projects, and direct observationsby the PPAR mission. Outcomes on each objective will be discussed by project. They are also presented in Annex C, in the two figures(C.I and C.2) portraying the project M&E design. UPBEP 5.2 In original UPBEP I outcomesindicatorsfor improvement in access were gross enrollment ratios (overall and SCIST,boys and girls) and a decreasethe percent of girls found to be out of school. The overall targets were for primary: 85 and 71 percent for boys and girls, 14.The technical name for this information created for UPBEPIDPEPby NIEPA is the District Information System for Education (DISE). respectively; and for upper primary: 76 and 64 percent. The targets for SC were 109 and 71 percent for boys and girls (primary) and 43 and 22 percent for upper primary. (None were specified for scheduledtribes)." These targets were reached or exceeded by the end of the project, especiallythose for scheduled castes, but caution is needed especiallywith respect to the upper primaryoutcomes, given the fact that they are based on flawed administrativedata and not the EMIS. Also, with UPBEP I1came a reframing of the outcomes to the "capacity to enroll all 6-10 year-olds and 75 percent of 11-13 year-olds." Indicators for this were set as 100percent gross enrollmentratio for lower primary in project districts and 75 percent for upper primary. These were not reached: GER for lower, based on EMIS data was 93 percent in 2000; for upper primary Directorate data shows 71percent, based on administrativedata which is most likely inflated. Actually, since the goal is enrollment of an age group, net enrollments would have been a more appropriate measure and these were not reported (but by definition would be lower than the GER), further reinforcing the conclusionthat the target capacity was not fully reached. Further evidence is the fact that during appraisal of the Uttar Pradesh 3rdDistrict Primary Education Project (UPDPEP111),prepared a year before the completion of the Basic Education Projects (2000), it was estimated that there were "about 3,102 unserved habitations in all project districts entitled to have a school building as per the state norm, but were still unserved" (World Bank, 1999). 5.3 The reduction in percent of girls out of school (by 50 percent) could not be accurately tracked, because baseline enrollment data was highly flawed. Nevertheless, relatively reliable EMIS data, collected in UPBEPDPEP districtssince 1997, showed a girls' gross enrollment rate of about 91.4 in the final year of the Projects (2000). Assuming a 10percent error in the baseline data in one direction or the other would still make the percent reduction well above the target. Also, given that the GER for boys was 94.4 in that year, it is clear that the gender gap had narrowed to below 5 percent, an accomplishmentperhaps even more important than the original target of a 50 percent reduction in those out of school. 5.4 The output goals for this objective were creatingnew primary school and upper primary school places. The combined goals for UPBEP I and UPBEP I1 for primary were 900,000 new places and for upper primary 350,000 new places. These goals were exceeded at both levels (estimated final status were 981,000 and 365,000,respectively),with overfulfillmentbeing especiallynoticeable during UPBEP I. The increasein places in UPBEP I was largelydue to lower than estimatedunit costs of the new schools, which were built to new, economical BEP design speczfications. Underhlfilment of the NFE target of providing 150,000new places (at their peak in 1997NFE centers enrolled only about 5000) led to Project restructuring and the redesign of this subcomponentunder DPEP, in which the state developed a wide range of "alternative school" programs, for example, the demand-dnven Education Guarantee Scheme for remote areas which had been pioneered in Madhya pradesh.I6 5.5 The connectionbetween increasing school places and improvementsin enrollment ratios is still a tenuous one. Within the project there was no way to document whether the 15.The neglect of scheduled tribes in Uttar Pradesh is explained by the fact that there is only a small pocket of them (around 35,000 people) in a state having a total population of over 150million. This being the case, the specification of scheduled tribes as a project target group was a design flaw. 16.In the ldhProgress Overview of DPEP, Aggarwal(2001) shows that by then Uttar Pradesh had about 7000 Alternative School centers, serving more than 250,000 children. increases in enrollmentswere due to increased number of places or some other reasons (e.g., mid-day meals which were provided in all schools). This is because there were no control districtsto be used to demonstrate the counterfactual(what would have occurredwithout the project). There are some recent figures showinghigher increases in primary school enrollments over 6 years (91192to 97/98) in UPBEP districts than in non-Project districts: 32 percent versus 22 percent (PAD, UPDPEP 111, 1999). SinceUPBEP districts were selected on the basis of their relatively low literacylevels, it is not clear whether the comparisondistricts are, in fact comparable. Also, the baseline data for both groups of districts were of questionablevalue. Thus, although there is a hint that enrollmentsincreased in UPBEP districtsbecause of the intervention, the data need to be used with caution. (In DPEP the attribution issue was eventually sorted out through an impact evaluation in which districtswere match by a computer model (see next section)). Overall, achievementof this objectivewas substantial. DPEP 5.6 The sole DPEP outcomeindicator for improved accesswas "the reduction in access disparitiesamong gender and social groups to less than 5 percent."17 Figures fiom the DISE database show that in DPEP I the average gender gap across all project districts, after 7 years of implementation,was below 5 percent in 6 of 7 states; in DPEP 11,after 5 years, it was below 5 percent in 6 of 10 states,'"ndicating that the goal was not reached in many locations. 5.7 Enrollment of children from scheduled castes has come basically into line with their share of the population (Ayyar and Bashir, 2004; Pandey, 2000), but not those among children of scheduled tribes, who are still underserved. Concerningscheduledtribes, DPEP capacity building plans included a study at the national level on ST education development,but this is not listed as a Project output. Also, this PPAR shows little follow-throughon the development of textbooks in tribal languages.I9 It has also been noted that ST parents are under-representedin Village Education Committees or, when present, generally silentparticipants (EdCil, 2003); that micro-planning for primary education sometimes overlooks ST communities(Vasavi and Chamaraj (2001));that textbooks still contain biases against lower castes and tribals, and that teachers sometimesperpetuate biased perceptions about them, even to the point of seatingthem separately(Jha and Jhingran,2002). 5.8 DPEP I1also included children with mild to moderate disabilities as a social group to be provided improved access by the Project. Although the strategy for doing this was never clearly articulated,there is evidence that improved access for this group did occur during DPEP implementation. According to the "National Reflection Paper" appended to the DPEP I1 implementationcompletion report (ICR), about 276,000 children (in eight of 12DPEP states) were identified as being disabled,and over the Project about 212,000(77 percent) were mainstreamed in primary school classrooms. In this there was wide variation across states: in Assam the proportion mainstreamed was reportedly 25 percent, while in Haryana, Himachal 17.The District Primary Education Program (to which the Project contributed) had a broader set of access objectives, including "universal access." 18.It is not clear why DPEP 11appears to have been less successful than DPEP in closing the gender gap. However GO1(2003) data shows that the gap in DPEPII states (12 percent) was higher at entry than in DPEP states (8 percent). Also, in the end the average gap in DPEP I1was 8 percent, not too far from the 5 percent goal. 19. In an interview with a senior manager of NIEPA, the mission was told that despite DPEP goals those using minority languages are still at a disadvantage. Pradesh, and Kerala the proportion was at or near 100percent.) This is encouraging, but is should be noted that one third of the DPEP states are silent on the matter. Also, it is not clear what kind of support is available to those who are mainstreamed. The PPAR mission observed severalmainstreamed disabled children in schools who appeared to be marginalized, i.e., having no special programs or supportto help them overcometheir learning disability. 5.9 The impact of DPEP I on enrollment rates (alongwith other outcomes) has been investigated (Jalan and Glinskaya, 2003), through the use of propensity score matching techniques to create an appropriate control group (neededbecause DPEP districts were not selected at random, but were those having the lowest literacylevels). Their conclusion is that DPEP I had a small positive impact on the enrolment rate of 6-10 year olds and a moderate-sized one on 11-13 year olds, once other factors are accounted for. This is not the resounding impact that the robust increases in enrollment figures had led stakeholders to expect, but it must be acknowledgedthat besides DPEP many other government interventionsaffecting enrollments had been implemented (which were not controlled for in the Jalan and Glinskaya study); and conversely,the catalytic effects of DPEP on educational improvementswent far beyond the narrow confines of participatingdistricts. Overall, achievement of this objective was substantial. UPBEP 5.10 In UPBEP the Project outcomesindicatorwas a 50 percent reduction in primary and upper primary dropout (DO). Concerning DO at the primary level, EMIS data revealed rates of 27.4 and 27.9 for boys and girls at the end of the Projects, compared to a baseline of 60 and 40 percent, yieldingpercent change estimates of 54 and 30 percent, respectively). However, since the baseline data is based on faulty school (administrative)records (the EMIS was not operational yet), the change scores cannotbe considered reliable. Even the EMIS end-of-project DO data should be considered flawed, since they were based on "reconstructed" (not true) cohort studymethods, and since EMIS did not track enrollmentsin unrecognized primary schools to which substantial numbers of studentshad been moving. SIEMAT conducted a household survey in 1999in 24 blocks in 6 of 17UPBEP districts, and in that the DO rate was determined to be about 32 percent (SIEMAT, 1999). Although this still does not include a computationof percent reduction (again, due to poor baseline data), it is clear that dropout is still a major problem in project districts. One of the issues identifiedby the mission was that the problem had not been fully researched prior to project start-up (e.g., reasons for drop out, both supply and demand side) and thus no specific evidence-based solutionshad been put into place. The assumption appeared to have been that if the schoolswere improved and teachers were trained, children would be much less likely to drop out, which did not turn out to be the case.20Overall, achievementof this objectivewas modest. DPEP 5.11 In DPEP an absolute level of dropout (less than 10percent) was set as the outcomes indicator. In none of the Project districts, except for those in Kerala (where these levels already 20. Recently a study of DO in UP (SIEMAT 2005) was completed and it has led to intensified efforts to mobilize parents and the community in child attendance tracking. existed at the outset), was this goal met. In DPEP I end of project rates were between 20 and 40 percent in the states of Haryana, Maharastra, Karnataka, and Tamil Nadu; in Assam they were over 40 percent. In DPEP I1only 16percent of districts ended up with DO rates below 10 percent (state averagesbeing between 10and 40). Evaluators and commentators have suggested that it was inappropriate to apply the same standard to all states and districts, irrespective of startingpoints (pointingto the fact that Rerala, which already had DOs below 10percent, easily reached the goal). However, valid data on DO did not yet exist at the beginning of the projects, so percent change would have been hard to track. Perhaps more realistic would have been differentiatedtime lines for each state, depending on baseline conditions, with all stateshaving similar low drop-out aspirationsin the long run. Again in DPEP there was little in the way of diagnosis and rigorous measurement of DO, and few strategiesbeyond vague notions of improved quality and communitysupport for pressing drastic reduction^.^' Overall, achievement of this objective was modest. UPBEP 5.12 The main UPBEP performance indicatorfor its quality improvement objectivewas a 50 percent improvement in grade 5 learning achievement over baseline achievement levels. Accordingto the ICR this goal was surpassed (based on a comparison of Terminal Assessment Scores to Baseline and Mid-term scores). However, some questions have been raised about the methodologyof the assessment,particularly with respect to the extremely skewed distribution of outcome scores.22Mid-term learning outcomes compared to baseline were presented in the UPBEP I1PAD and whereas these showed some modest improvementsin language learning (but not math) the differenceswere not statisticallysignificant. Reasons cited for little changewas the fact that project inputs had been delivered only shortlybefore mid-term testing. Overall, achievementof this objectivewas modest. DPEP 5.13 The main student learning indicator in DPEP VII was an "improvement of learning achievementby 25 percent over the baseline." As mentioned in Section 4, change over the life of the project could not be directly determined given the fact that the achievement test was changed at midterm. Table 5.1, based on data from the ICR, gives the percent changein student scores over all project districts for the baseline to midterm (using one test) and midterm to end- of-project (called the terminal assessment)using the other. These results show strongpositive changes in both subjects and at both grade levels in DPEP I and DPEP 11. Whereas technically the scores from the two phases (baseline to midterm and midterm to terminal) cannot be added, there is a strong impressionthat the 25 percent improvement target was met. Also notable was the disappearance of significant differencesbetween girlsllowercaste children and the others on achievementscores,but this could not be said for children of scheduled tribes whose scores remained significantlylower. 21. At the launching for the country-wide SSA project (2004) DO rates were still very high across districts and states, but curiously DO reduction did not become one of the Project's objectives. 22. Data from this achievement testing could not be obtained by the mission, so these findings could not be validated. Table 5.1. PercentChange inAverage Test Scores for DPEP I and DPEP Ila Grade Subject Baseline to Midterm to Baselineto Midterm to idt term^ TerminalC idt term^ TerminalC 1 Language 11 20 23 12 Math 28 20 40 13 3 or 4 I Language 1 19 15 16 Math 13 23 I I I alncludesall DPEP 1/11districts, even those not covered by IDA projects. b~ased baseline test results given to a full sample priorto project start up on and to a small sample at midterm. 'Based on a new test given both at midterm and at end of project (Terminal) 5.14 When the results are disaggregatedto the district level, a somewhat differentpicture emerges (see Table 5.2).23 As can be seen, the percent of districtswhere results reach the 25 percent improvement goal was between 61 and 87, depending on the subject and grade level. The average was about 73 percent; thus in about %of DPEP 1/11districtsthe target was not reached. Why this discrepancybetween the student level analysis and the one based on district averages? One explanationis that the student level analysis includes data from all states implementing DPEP, includingthose with relatively good results like Madha Pradesh, whereas the district level analysisonly covered those states in DPEP I or I1covered by Project fimds. Table 5.2. Percent of Districts Reaching Learning Improvement Goal Grade Subject DPEP I* DPEP I/** 1 Language Math 3 or 4 Language Math * Excludes Madhya Pradeshand Chhatisgarh (which split from MP) ** ExcludesAndhra Pradeshand West Bengal, missing Kerala Adapted from: S S.K.S Gautarn, StudentAchievement UnderTAS: An Appraisal in Phase I States, NCERT,2002 and S.K.S Gautam, Synthesis Reporton Student Achievement UnderTAS: An Appraisal in DPEP States, NCERT, 2003. 5.15 As impressive as the changes in student learning outcomes appear, they can also be challenged on the basis of attribution. TheDPEP assessment included no control group data, so it is not possible to establishempiricallywhether the changes are attributableto the projects. One way to overcomethis would be to look at performance over time. If DPEP treatmentswere influencing the direction of achievement scores, one might have expected consistency across the 23. This table was created by IEG based on GO1synthesis reports. It takes district averages for the two time periods (baseline to midterm and midterm to terminal) and estimates whether there was a 25 percent improvement over the project period. It includes only the districts covered by the DPEP I and 11. two testing periods, baseline to midterm and midterm to terminal.However, this is far from the case in many districts. In DPEP I districts about in only about half of the districts are is there consistency(near the goal in both time periods), and in DPEP I1districtsthere is consistencyin only 113of districts in language and only 54 in mathematics.In many of these districts the change during one period is negative and in the other positive. It is not clear why there is so little correlationin the changes across the two periods, but one possible explanation is the validity of the tests. 5.16 The basically positive slant of the DPEP I/II assessmentshas been questionedby some independentevaluators. The Bangalore-based Azim Premji Foundationdid a review of the "Status of Learning Achievement in India" (2004) and found many small-scale,in-depth studies in DPEP states that showed low mastery levels in language and math among school children. In most cases outcome showed findings nowhere near DPEP's terminal assessment averages in grade 1 language and math of around 80 percent. A recent survey in rural areas by the NGO Pratham of simplereading and math skills revealed pervasiveweaknesses, for example, in the proportion of rural children who could read a simple paragraph.2425It is important that the country reconcile these findings with those coming out of DPEP (which also mostly covered rural, low literacydi~tricts).~~In the mean time, Azim Premji and others are questioningwhether the DPEP's paper and pencil approachto assessment is the most appropriateway to measure student learning outcomes. At this point, the main conclusionthat can be drawn is that while there has been some improvement in average learning outcomesunder DPEP I and 11,despite rapid increases in enrollments, the absolute levels of basic knowledge and skills in project (mostlyrural) districts(as determined by criterionreferenced tests) is still very low in most locations. Overall, achievement of this objective was substantial. UPBEP 5.17 The improvement of institutionalcapacity for the planning and management of basic educationwas a major objective of UPBEP I. Prior to the state supported schemes of UPBEP and DPEP the management of basic educationhad been of marginal interest to state and district educationofficials. Big changes in this were envisionedby the projects,but in their designs capacitybuilding was never clearly defined, and, as mentioned in Section4, no outcomes indicatorswere specified. Incontrast, many outputs indicatorswere specified, but these often just showed the formal changes (establishmentof an office) or the delivery of inputs (such as hours of training) and not improvements in institutionalor managerialperformance. Nevertheless, there is some evidence of new institutionalcapacity having been built and this will be presented briefly in relationto five main management domains:planning and change management; curricularreform and materials development;professional development of teachers 24. Pratham's Annual Status of Education Report (2005). The report was based on a national random sample in 485 rural districts (9000 households) in which volunteers had 7-14 year olds read simple paragraphs at the first and second grade level and solve simple math problems during a home visit. It was found that 48 percent of 7-10 year-olds could not read the first-grade level paragraph fluently. 25. Similar observations were made by the PPAR mission: in six rural schools visited in Karnataka and Uttar Pradesh most of the second graders tested could not read a first grade textbook passage fluently; somewhat over half of the 5thgraders could. 26. One insight is that the DPEP 1/11reading tests were about 50 percent word recognition, which could be considered less demanding than reading a passage (the Pratham test). and local managers; management of educational information;and research and evaluationrelated to basic education. Planning and change management. Before UPBEP therewas no systematic planning for basic education at the state and district levels. The UP EFA Society, which managed UPBEP VII at the state level, initiated a program of annual work plan and budget planning (based on existing data) at the state level and helped to build capacity for this at the district level. Not all plans are carefullyconstructed upon the existing data (some became a bit ritual is ti^),^' but plans are used as a basis for annual financial allocations. Village Education Committees have also are involved in school level planning, includingplanning for the use of an annual grant, The EFA Societyalso completed a state-wideEFA plan, which became the basis for expansion of UPBEPDPEP to 60 of its 84 districts,and by the end of the century four "sustainability studies." One concern is the fact that the EFA Society is an autonomousbody not embedded in the educational bureaucracy and it is not clear how much of its planning and leadership expertise is being transferred to the state government offices. Curricularreform and materials development. With the help of consultants the project built a new curriculum for basic education and created a new generation of student fiendly textbooks and supplementarymaterials. During the course of this effort, state curricular developers learned how to draw upon teachers and try out materials in the field (new skills). The StateCouncil for Educational Research and TrainingCapacitybecame engaged in these activities, but was still not fully capable of leading it and moving it ahead. Professional development of teachers and local managers. UPBEP I/II created a new infrastructure to teacher development, firstby establishing the State Institute of Educational Management and Training (SIEMAT) and, at the district level supporting a more proactive role for the District Institutesof Education and Training(DIETs),and setting up teacher resource centers at the block and cluster levels. Whereas prior to the project teachers rarely received any inservice training at all, under the project most received 6 to 10days a year and attended a meeting at the cluster center and/or are visited by a resource person from the cluster center once a month. Block and cluster resource centers spread to non-project districts even before they were sponsored state-wideby the 2001 Elementary Education SWAP (SSA). Teachers also started receiving modest annual grants for use in purchasing or making teaching aids, a pattern which is expected to continue. SIEMAThas become a productive agency, directing training and managing studies;the DIETs have a more checkered record; and the resource centers have generallybeen active, although their effectiveness varies from district to district. Evidence of improved teaching has not been systematically documented, but some qualitative studies (supportedby PPAR observations)have shown an increasein student-centered learning and student-friendly classrooms.Old fashionedpedagogy and rote learning are still evident everywhere,however, so there is still a long way to go. Management of educational information. With the help of the National Institute of Educational Planning and Administration the state set up a new Educational Management Information System (described in this report's Section4). It became computerized and operational at the state level and all project districts and has provided the basis for improved planning. Data is still flawed in many places, given low-level skills at the subdistrict level, 27. Based on PPAR mission observations. where data entry takes place, and the rapid turnover of district managers (requiring some functionsto be out-sourced). Its use in planning at the school level is still quite rare, but the culture of data-basedplanning and management is beginning to be established. Research and evaluationfor basis education. One of the first tasks to be undertaken by SIEMAT was to establish a student learning assessment system for the state, but this did not come to hition. SIEMAT has undertaken some useful research (such as the real cohort study of student dropout)but much of its output is of uneven quality. The state EFA society sponsored a total of 63 studies, but these too were of uneven quality and relevance (World Bank, 2003a). 5.18 Achievement of objective:even though the evaluation design for this objectivewas flawed there was enough evidence for improved institutional capacity to be assessed as substantial. DPEP 5.19 One of the main objectivesof DPEP VII was to "build national, state, and district capacityto plan for and manage primary education." Determining whether this objectivewas fulfilled was not a straightforward task, given the lack of outcomes indicators. There is some evidenceof change, however, which will be compiled around the same themes as in the UPBEP assessment,namely: planning and sub-sectormanagement; curricularreform and materials development;professionaldevelopment of teachers and local managers; management of educationalinformation; and research and evaluation related to primary education. Planning and change management. The national BPEP Bureau and its technical support group were highly motivated and proactive managers of the many changes introducedby DPEP. Continuity in its leadership and senior staff was one key to this. State and district project offices were also generally fully staffed and motivated, but there was some turnover. The main planning vehicle at all levels was the Annual Work Plan and Budget, a bottom-up planning system that has taken root and changed the way needs are determined and resources allocated. The capacity for needs- and performance-based planning varied widely from state to state and districtto district.*' However, there has been no systematic evaluation of district planning capacity,or even a "comparative descriptiveaccount" of planning and management performance across districts (Ayyar, 2005). Curricular reform and materials development. Curriculumrevision and learning materials production was successfbllyimplemented in all DPEP 1/11states in the same innovative manner employed by Uttar Pradesh, largelya result of the strengtheningof state agencies, support from the center, and cross-fertili~ation.~~Statesvaried in the extent to which their resource institutionswere fully capable of driving this such pedagogicalrenewal, but at least one 28. Evaluation studies reviewed by the World Bank (DPEP Review, 2003) show that AWPBs were rarely based on assessed needs and performance but more on simple lists of activities and their funding requirements (DPEP Review, 2003). This was confirmed by the PPAR mission in its visit to Hardoi district in Uttar Pradesh, where the district personnel admitted that planning exercises are mostly reduced to a "game of numbers" with not much thought given to the current year's strategies, which resemble those of other districts. In the early years of DPEP, when the Project was fresh, there was alleged to be more meaningful and context-specific planning. 29. According to Ayyar and Bashir (2004), "renewing pedagogical content and processes could be the single most important contribution of DPEP, shifting from a teacher to a learner-centered pedagogy through revised curricula, curricular materials and a huge teacher training network." institution, the DSERT of Kamataka(visitedby the mission), had continuedto innovate in curricular and materials development." One disappointmentwas in the area of tribal language use. There was no evidenceof DPEP I/II having created textbooks in tribal languages as en~isioned.~' Professional development of teachers and local managers. One of the important legacies of DPEP 1/11has been the establishmentof an operationalin-serviceteacher training network. During DPEP this network was able to provide 6-10full days to each teacher per year, plus regular teacher meetings at clusterresource and school visits by a mentor teacher. During the Project training content became increasinglyneeds oriented (althoughthe PPAR mission observed only a limited number of subjectsbeing covered, such as multigrade teaching and English). The PPAR mission confirmedthat, at least in Karnataka,this system was still operational two years after the close of DPEP 1/11. The quality and impact of the training has not been well documented;however, as in Uttar Pradesh, there were small-scale qualitative studies showing improved teaching skills in some 10cations.~~The only evidence of local management capacitybuilding was amount of management training delivered. Management of educational information. By project completionthe educational management information system (EMIS)was operational on a computer format in all Project districts(and by extension to many non-DPEP districts,too). A host of district level database managers were trained to start up and maintain the system, neverthelessthere are still quality control issues and low demand for the data in decisionmaking. Research and evaluation related toprimary education. Most issues have already been covered in Section4, Monitoring and Evaluation. DPEP 1/11have clearly mobilized much more interest in studyingand evaluatingprimary education than existed in the past. At least 180 studies were undertaken under the two projects.33However, interest is not the same as capacity. Given the poor qualityof much of the research, capacitybuilding could still be considered a work in progress. Overall, achievement of the objective was substantial. 30. Seenew "trimester curricular system" developed by DSERT in 2005. 31.According to a NIEPA administrator this is one of the reasons that tribal studentswere still disadvantaged by the end of DPEP 1/11. 32. During unannounced school visits to 9 schools in Karnataka, PPAR mission members observed one school not using any project emphasized "active learning" approaches, five using some (especially in lower grades), and three using an abundance. Concerning more specific skills, there was less evidence of training impact: for example, of the nine schools visited eight used multigrade teaching, but only four used any kind of multigrade teaching strategy, despite the fact that all teachers had received training in it. Discussions with teachers and trainers revealed that in their districts there is little connection between training provided and follow-up at school, both by school management (head teachers) and by visiting mentors. The latter rarely check to see if training was understood or implemented, nor do they provide hands-on assistance to make sure that teachers master new innovative instructional methods. 33. According to Ayyar (2005) no other program in India has generated the volume of research and evaluation in education that DPEP has. A conference on the impact of DPEP on Primary Education in 2003 saw the compilation of 491 study abstracts, 152 on evaluation studies (p. 49). UPBEPAND DPEP 5.20 Both UPBEP and DPEP had a single outcomeindicator for this, namely that filly functional Village EducationCommittees would be establishedin 50percent of schools in project districts. As mentioned in the monitoring and evaluationsection, this indicatorpresents a rather limited view of communityparticipationin primary education,and even for the Village Education Committees the failureto spell out in detailwhat "fully functional" meant, leaves the evaluator without much hard evidence of fulfillment. Nevertheless,for UPBEP it was shown that in most project villages VECs were establishedand given two rounds of three day trainings. The extent to which they were filly functionalwas not well established,but a high proportion was at least partly functional,supporting construction and enrollment drives. A PanchayatiRaj Act passed at the state level eventually reduced members to 5 (including the mayor and the school head, ex officio), limiting the extend of communityparticipation in the committee. 5.21 The community empowerment concept is powerful and its potential to improve school accountability and local planning is high; however, there is still not much evidence that the VECs made effective contributionsto school quality, and where so, under what circumstances, thus, achievement of objectivesis assessed as modest forboth WBEP and DPEP. RATINGS 6.1 Relevance: UPBEP I/II. UPBEP I/II support for universal primary educationwas built upon the country's 1986National Education Policy which mandated a renewed focus on access to basic education, especiallyfor the underserved, and the related Plan of Action of 1992,which called for central government support to state and district offices for improved primary education, buttressed by external financing. The core educationalproject objectives of improving access and reducing dropout among girls and the poor were relevant to that those policy positions. National concernsabout building a solid basic education foundation and baseline assessments during preparation showing alarmingly low student achievement scores, led to the additionof an objectiveon improving learning outcomes. The three core objectives were particularly relevant to the state of Uttar Pradesh, an underachiever in basic education. At the time of project preparation the statehad showed a gross enrollment ratio of below 65 percent compared to near 90 percent in other parts of the country, with those for girls and scheduled castes even lower. Nearly half of those enrolled were droppingout before completing grade 5. Baseline surveys during preparation found learning levels in participatingUP districts to be among the lowest in the country. Concerningthe capacity building objective, relative neglect of primary education by local government officials had resulted in weak managerial capacity and commitment in the educational bureaucracy, which the project aimed to overcome. It is questionable, however, whether capacity building was appropriateas a project objective, especiallysincethere were no good outcomes measures for it in the design (see Monitoring and Evaluation Section). The way it was treated was more as a means to the core primary education objectives (and their related outcomes) than as an objectiveitself. 6.2 In 1993,the year of project approval,the IDA strategy for India was to support investmentsin the social sectors, including public education, and to provide access to them for the poor. The 1998CountryAssistance Strategyfor India in place at project completion continued that theme, supportinginvestments in primary education,especially for girls, as a key to poverty alleviation and sustainable economicgrowth. 6.3 The design of UPBEP LfIIwas distinguishedby high levels of government ownership and extensivepreparation activities,which were summarized in 18project document annexes and cited in 1994by IEG (then OED) as best practice. Nevertheless,there were some weaknesses in the design. A main concern was in the single component related to two objectives of improved learning and increased school completionrates (called Improved Quality and Completion). The assumptionwas that improved school inputs would help studentslearn better and stay in school through completion. However, this overlooked some important considerations. For example, for improved school completion (or reduced school dropout)both supplyfactors (what the schoolprovides) and demandfactors (what the familyneeds andlor can afford) are at play. The design (project subcomponents and indicators) did not focusvery heavily on the immediate causes of student dropout, especiallythose from the demand side.'j Likewise for student learning improvement: in UPBEP I there was little explicit emphasis the classroom (immediate)conditions for improved learning (such teacher attendance, effectiveuse of classroom time, and improved instructional methods), all of which are links between inputs (textbooks, training, and supervision) and student learning outcome^.'^ UPBEP I1did add one such a feature,an intermediateoutcomes indicatorof "improved methods introduced and impact learning" to be assessed through a "special study," but this came four years after UPBEP I started and does not appear to have been emphasized. Finally, having no control groups or other ways of establishinga counterfactual the projects were limited in their capacity to demonstrate project impact. Nevertheless, given the high relevance of the project objectives, overallrelevance for UPBEP projects is rated as substantial. 6.4 Relevance: DPEP 1/11. DPEP I became effectivejust 17months after UPBEP I did. Because of the time proximity to UPBEP 1and similarityof its design, its objectiveswere relevant to countrypolicies and states needs in the same ways. Also, given the equity featuresof governmentpolicies and UPBEPDPEP objectives(reachingthe poor and socially disadvantaged), it is notable that the states and districts covered are those among the poorest in ~ n d i a .It~is~also relevant to the 1993IDA strategyof increasinginvestment in the social sectors and extendingthem to the poor, and the more recent CAS of moving towards universal primary education,including among girls and the underserved. 34. This is a matter of emphasis and scope. Whereas the supply side interventions (increases in school places, teachers and books) were everywhere, the demand-side interventions were often more localized and less visible. Nevertheless, there were some good examples of such, for example: a) the local uselupgrading of madrasahs or maqtabs for those reluctant to enter convent~onalschools; b) flexible school timing catering to the needs of children needing to help parents at certain times of the year; c) and expanded hours for early childhood programs, allowing female sibling caregivers to stay in school longer. 35. The design appears to have kept what happens in the classroom in the "black box," resting on the assumption that if classrooms received improved textbooks and teachers were provided with a modest amount of inservice training and technical support, then classroom instruction would improve. The small number of observational studies that have been conducted seem to show that old classroom routines are more difficult to break than expected. 36. An exception to this was the inclusion of districts of some of India's southern states that were selected on the basis of their strong participation in a total literacy campaign. 6.5 In retrospect the project design for DPEP also contained some weaknesses. Like UPBEP it included few, if any, evidence-basedinterventions for bringing down student dropout, and there were there no design elements for directly tracking and improvingteacher classroom behavior and time on task (leavingthem in the "black box" - see above). Other weaknesses are in the specificationof indicators(see also the Monitoring and Evaluation section): the failure to use enrollment ratios (enrollment comparedto population) for tracking access and the problems with using a single standard for improved dropout in all states and districts (to below 10percent) despitevast differenceacross districts in startingconditions. Like UPBEP VII, the DPEP projects were also limited by having no controls groups in their designs. Given the high relevance of objectives, however, the overall relevance is rated as substantial. 6.6 Efficacy: UPBEP VII. In the UPBEP projects the outcomes ratings on the five objectives are summarized as follows: Improving access to primary education: substantial; Reducingprimary school dropout: modest; Improving student learningoutcomes:modest; Improvinginstitutional capacity: substantial; Improving community participation in education:modest. Overall, efficacy is assessed as modest. 6.7 Efficacy:DPEP VII. For DPEP VII outcomes ratings on the five objectiveswere: Improving access to primary education: substantial; Reducing primary school dropout: modest; Improving student learning outcomes: substantial; Improving institutional capacity: substantial; Improving communityparticipation in education:modest; Overall, efficacy is assessed as substantial. 6.8 Efficiency:UPBEP VII. The Projectsdid not track efficiencyin the use of resources or any cost effectivenessratios. However, in the Project many examples of improved efficiency can be identified.For example, innovations in school design led to a reduction in the unit costs of classrooms, meaning that up to 75 percent more classrooms could be constructedwithin the original classroomconstructionbudget. In addition, the hiring of 5700 "para teachers" in the last year of the project, at average salaries of around 115 of those received by appointedteachers and with relatively low teacher education expense, has brought economies.37 In a perverse way, the rapid expansionof student numbers and the resulting increase in the pupil-teacher ratio to around 60:1 is an economy, but one to back away from, since it works against student learning gains. Expected savings from a 50 percent reduction in student dropouts did not come to full fruition, since reductions did not reach that level, but even with more realistic reductions of 30-35%, there 37. This is at least in the short-run. It is still not clear how long these salary conditions will affect teacher morale and perseverance in the longer term. were savings. The one areawhere cost-effectiveness is still unclear is in inservice teacher training. Large expenditureswere made to set up block and clusterresource centers and to provide 6-10 days of training per year to each teacher, but the outcomes, in terms of improved instruction,have so far not been strong. The rating for efficiencyis substantial. 6.9 Efficiency: DPEP 1/11. In DPEP efficienciesare not as obvious.There were no new savings on unit costs of construction but the project stillbenefited fiom UPBEP's more efficient school designs.Also, planned efficiencies fiom reducing student dropoutsbelow 10percent were not realized in most locations - in fact, the extent of the reduction, if any, was never computed, since baselines were unclear. Therewere likelyto have been some efficienciesbased on the use of contract teachers, but as above, it is not clear how sustainablethese might be. Finally, there are the same concerns as above about the cost-effectivenessof teacher training, which has yet to borne as much fruit as hoped for in changed teacherbehavior. Efficiency or cost-effectiveness were not analyzed at appraisal nor were they treated in the ICR. The efficiencyrating for DPEP is modest. 6.10 In both UPBEP and DPEP institutionaldevelopment was to occur through two channels: direct implementation of capacity buildingproject components,and the more indirect process of diffusion,from the Projects to the educationbureaucracy.38The section on capacity building above revealed project accomplishments(and a few weaknesses) in establishing innovativeorganizational structures and procedures, and in strengtheningmanagement capacities (planning, supervising, monitoring and evaluating, etc.) of state and district project offices. Beyond capacitybuilding, the Projectsmanaged incentivesso as to motivate key personal (like state and districtproject officer managers)to do earnest work and to motivate others.39The Projectswere seen as holistic in scope, but in fact there were some awkward gaps in them. For one thing, UPBEPIDPEP never dealt with schools as units (and teachers as a team); never focused on school supervisionor quality assurance, and stayed out of teacher management, all potential contributorsto the Projects' instructionalimprovement agenda. Sticking to their Project agendas and operating through a separate administrativenetwork, UPBEPIDPEP were seen by many as settingup a dual management structure,which limited capacitybuilding in the education system mainstream. 6.11 But then there was the diffusion channel. It was assumed that by exemplifying good practice that UPBEP/DPEP would influenceregular educationaldepartments and offices by example or by diffusion. How successful was this? There are two ways for framing the answer: structural and behavioral. In terms of structure,the mainstream ended up adopting many of UPBEPIDPEP structures and processes.Project pioneered approaches to curriculum and curricular materials developed were generally adopted for state-wideuse; the EMIS system created for the Projectswas diffused to all districts;block and cluster resource centers have spread to virtuallyall districts, and, through SSA the processes of decentralized planning have 38. Accordingto the originaldesignerof DPEP, "DPEP is not an enclaveproject, it seeks to restructure and improve the system of primary education as a whole" (see Ayyar (2005). 39. One crucial move was to staff project office management positions with members of the elite Indian Administrative Service, persons whose upward mobility depended on goodjob performance. been adopted nation-wide.Also under SSA, at least at the district level, Project Offices have been merged with the Department of Education offices, eliminating dual management. 6.12 Project influence on behavior change is harder to document.The case for diffusion of pedagogical processes is quite strong. Project pioneered pedagogicalrenewal is everywhere, to the point of becoming what some consider to beparadigm shift. Transformationof management behavior, however, appearsto be more limited. 40 In fact, in some cases the process of reverse osmosis seems to be occurring: as structures like Block Resource Centers become mainstream, they begin to take on routine bureaucratic functions,such as school "inspection"and data collectors. "Goingto scale" oftenpresents this risk: as innovationsmove into the mainstream they lose their cutting edge." Perhaps the greatest challengeto SSA Project managers is to transform the system before the system transforms them. Overall, institutional development impact for both UPBEP and DPEP is rated as substantial. 6.13 Project sustainabilityis more securenow than could have been expected in the beginning. UPBEP I led quickly to UPBEP I1and these fed into DPEP, phase 1,covering seven states. Eventually six more DPEP projects were created, covering 18states and 271 low literacy districts(around45% of the all districts).Before the last DPEP projects had closed the GO1 initiated the ElementaryEducationProject (SSA), which embraced all of India's stateslterritories and their districts,and, as a System Wide Approach (SWAP),brought together all donors and centrallyfunded projects in a singlenational effort to achieve universal elementaryeducation and acceptable levels of learning. Throughoutthis expansion the original UPBEPIDPEP objectives of increasing access, especially for the disadvantaged, and improving student learning outcomes were maintained;remarkably also, most of the original Project structures and processes were retained and brought to a national scale. 6.14 Domestic financing in support of this efforthas also been forthcoming:over 1993to 2002 public expenditureson elementary education rose from 1.7to 2.1 percent of GDP, accounting for more than 60 percent of the growth in public expenditure on education for this period (Wu, Kaul and Sankar, 2005). Average per pupil expendituresincreased fiom $25 to $44, despitedramatic increases in enrollment. In 2004 domestic financial support of SSApoured another $3.5 billion into basic education improvement, and this has been met by external agency contributions of more than one billion dollars (WB $500million). In 2005 a new 2 percent "cess" for basic education was introduced into the nationalbudget, a major source of new financingfor SSA. The transfer of federal funds to states for DPEP and SSA (covering 85% and 75% of Project costs, respectively) has been predicated upon a state's commitment to maintain its own spending for elementary educationin real terms. This condition was included to prevent 40. Ayyar (2005) writes, "The efforts to build policy planning capacities at the state level and to locate district plans within an overall framework for the development of primary education within the state appear to be relatively limited" (p. 56). 41. In one district visited by the mission, it appears that the process of annual planning has become totally mechanical and routine. In another, now under SSA management, the number of schools covered by one of the clusters visited was 51, compared to the ideal of 8-10 under DPEP. The district is creating some new clusters, but the plan is to make the average about 20 schools per cluster, well above the number in the original concept, and probably more than a single cluster coordinator can serve effectively. substitutionof funds and ensure that Central grants were truly additional. During UPBEP and DPEP states did, in general, adhere to these conditions, at least for recurrent costs.42 6.15 Program ownership is also firmlyin the hands of domestic actors, who have done all of the major planning and designingfor program expansion. In addition, the influence of other stakeholders is significant and growing, including that of major NGOs, and most importantly, parents and community members through their involvement in village education ~ornmittees.~~ Legal support for sustainability,at least until the governmenthas succeeded in achieving universal completionof elementaryeducation, is assured, given the passage in 2001 of an amendment to the national constitution making the attainment of a quality elementaryeducation a fundamental right of every child. Given this course of expansionand institutionalizationsince the close of the two set of projects, sustainabilityis rated as highly likely. 6.16 UPBEP. For UPBEP the Bank designedan experienceTask Team Leader who was resident in India, who wasjoined in Project preparation by a professional from inside and outside the Bank. The Bank team was first invited to brainstorm on best practice in primary education development across the world and then in the latter part of 1992to appraise a proposal prepared by the government of Uttar Pradesh (GOUP) and the government of India (GOT). The Bank team appreciated this fully-owned and well developed proposal, which was subsequently strengthened throughjoint analytical work and workshops. Throughout, the borrower took the lead in framing the Project's design, and the Bank team was flexible and responsive to the government's positions. Ultimately, project preparation was rated as best practice in OED's annual review of project economic analysisof 1994. 6.17 Bank supervisionof the Project was also carried out from its New Delhi office where the Task Team was stationed.This strengthenedthe partnership between the GOUP implementation team and the Bank's task management during the life of the project. Supervision missions included regular biannual supervisions as well as interim ones focused on specific themes. It was said that the Bank's presence in the Joint Review Missions added status to the event and made officials pay attention to them and the UPBEP enterprise. In 1997when the DPEP was launched cover 17districtsin UP, the Task Team agreed to fold the supervisionof UPBEP into thejoint supervisionmechanism for DPEP. This reduced the frequencyof visits to the state somewhat,but enriched certain quality aspects through cross sharing. When in UPBEP an unanticipated surge in enrollmentscreated a surge in demand for new classrooms and schools, the Bank team responded with a sense of urgency and supportedthe preparation of UPBEP 11. 42. There was some difference in the way the GO1and the Bank interpreted this agreement. In its DPEP guidelines the GO1 specified that total state govemment expenditures (development plus recurrent) were to have been maintained in real terms, but in the World Bank's SAR it is recorded that the states should maintain real levels in development ("plan") expenditures. Total costs mostly covered teacher salaries and these were, in fact, generally maintained or increased by states. Development costs, however -those covering system expansion andlor quality improvement -were, in actual practice, subject to considerable fluctuation from year to year. (See Ayyar and Bashir, 2004). 43. In recent years the GO1has devolved many government functions, includingthe provision ofbasic education, to local government or Panchayat Raj institutions (PRI). Most states under DPEP (and now SSA) have created statutory connections between village education committees (VEC) and PRI; in other words, the VEC has become a legalized subcommittee under local government institutions. Of course, there is variability in the extent to which parents and other community members have actually been empoweredthrough VECs to influence the direction and quality of primary schools, Such responsivenesscontributed to communityconfidencein GOUP'Snew educational programs. Supervisionof financialmanagement brought up some irregularitiesin construction qualityand tendering, but these were all resolved satisfactorily;routine aspects of financial management were monitored regularly. Overall Bank performance is rated as highly satisfactory. 6.18 DPEP. Bank support for preparation of DPEP was similar to that for UPBEP, including assuring GO1ownership and initiative on design features and workingjointly to supplement the proposal with relevant analytical and policy developmentwork. Continuity in Bank team members assured that there was a smooth transition from UPBEP to DPEP. Strengthsof the Bank team during preparation included: (i) the continuity of staff and consultantswho had worked on the program over time; (ii) the ability of Bank staff to recognize the strengthsand capacities of the GO1counterparts, GO1policy, and lessons learned; (iii) the good balance and high level of technical expertiseprovided; and (iv) the simultaneousin-depth sector work to learn fiom early interventions, improve sectorknowledge, and adapt the program accordingly. 6.19 Early in DPEP'shistory, the GOI, the Bank and other contributing development agencies agreed to ajoint supervision strategy, with rotating leadershipresponsibilities among partners. Subsequently(in 1998)Joint SupervisionMissions were renamed Joint Review Missions in 1998in recognition of the collegial, supportive nature of these visits to the states and to differentiatethe roles of GO1and its fundingpartners: supervisionwas the role of the former, while the latter participated to confirm and verify the supervisionresults.44National, state and districtofficials universally praise the process followed by the JRMs with its clear objectives. The Bank's presence in the event again appears to have brought some stature to the event and the Projects. Some concerns were expressed, however, that the impact of the JRM weakened over time, for example in depth of coverage and appropriatenessof the terms of reference. Difficulties also emerged fiom the Bank's perspective in meeting its fiduciaryrequirements since the JRM did not allow for the review of procurement, disbursementsand financialmanagement.This may have contributedto some issuesraised in project audits. Overall the Bank performance is rated as satisfactory. 6.20 UPBEP. The Government of Uttar Pradesh (GOUP) took the lead in preparing UPBEP with technical assistance from the GOI, private consultants (e.g., a new concept for carrying out baseline beneficiary assessment surveys and focus group discussionswith the stakeholders)and the Bank's team. As this was the first district-based project which followed participatory methodology, project preparation took about two years. The capacity for undertaking such an exercise with stakeholder's participation was low, especially at the district level, but during the preparation process it improved. During the length of the Projects, the GOUP provided an enabling policy environment for effective project implementation, includingthe timely release of Eunds to the Project and the posting of senior and committed official to staff positions. In many ways, the pioneering efforts of this group began the process of putting primary education "on the map," where before it had been neglected 44. Government officials acknowledged to the PPAR mission that it was the involvement of external agencies like the Bank in the joint review missions that gave the Project special statusand commanded attention. 6.21 The StateProject Office, EFAPB, was established even before Board's approval of the project. It executed its responsibilities with a high level of effectiveness, largely due to its capacity to respond flexiblyto feedback from the field, to network with capable individuals and institutionswithin and outside the state, follow through on plans and promises in a visible manner, and effectivelyuse reporting systems and open communications. Perhaps its most significantcontribution was the close working relationshipsit established with the regular structuresand institutionsof the state, which ultimatelyresulted in improving chances of sustainabilityof the programs and inputs afterproject completion.Overall the perfomance of the borrower is rated highly satisfactory. 6.22 DPEP. Full and proactive country ownership of DPEP was one of the main ingredients of its positive influence on the course of primary educationin the country. This was not just a central government effort, but involved activeengagement by all participating states and districts. This participatoryprocess, plus the added prestige brought by the buy-in of major development agencies (including the Bank), had the effect of lifting the stature of primary education in the country and created widespread commitmentto the Project's dual goals of improving access and improving learning outcomes. The central DPEP Bureau and its Project Board gave strong and consistent leadership to this effort and solidtechnical assistancewas provided by Project's Technical Support Group. The central government was also skillfulat channelingmost external assistanceto primary educationthrough the DPEP program (a precursor to a sector-wideapproach or SWAP),and created a systemof harmonized project supervision (review) epitomized by the semi-annualjoint review missions. Two negative consequencesof this system were that DPEP did not become as open to fresh ideas from the outside as originallyenvisioned, and opportunitiesfor Bank capacitybuilding at the state level (throughpolicy dialogue and mentoring) were limited. 6.23 At the state level, in fact,performance in managingthe Project varied considerably. In some states (e.g.,Karnataka and Madhya Pradesh) there was strong commitment and impressive institutionalstrengthening,but in others recurrentproblems were found, including frequent staff turnover, vacancies left unfilled, and delays in transferringfunds. In the early years of DPEP 1 there were also someproblems with financialmanagement and accountability(mostly overcome). On the deliveryof services, some states and districtsprovided training and technical assistanceconsistent with DPEP's high expectations,whereas others did not. Given this uneven terrain, it would have been useful if DPEP had provided targeted support to states and districts found to be underperforming, but this rarely happened. Nevertheless, given strong national leadership and the way that the projects changed the face of primary education in the country, the overall borrower performance is rated as satisfactory. 7. REALIZING EDUCATION FOR ALL 7.1 During the time that ths PPAR was being prepared, IEG was also undertaking a global evaluation of World Bank supportto primary education.45 Since one purpose of a PPAR is to contributeto such studies,it was decided that all 2004-06 PPARs on projects featuring primary 44. The finishedproduct is From School Access to Student Learning Outcomes:An UnfinishedAgenda. An Evaluation of World Bank Support to Primary Education, World Bank, IEG, 2006. education shouldbe enhanced to address countryprogress towards EFA. This section was included for that purpose and an early draft used as an input to the larger study. 7.2 Whereas UPBEP VII and DPEP VII were launched to set the country on the road to Education for All, more recent projects, especially Elementary Education Project (SSA), supportedby India's first Sector-WideApproach (SWAp),were set up to finish thejob. The GOI's goals for SSA, which began in 2001, were fox: a) all 6-11 year olds to complete five years of primary schoolingby 2007; and b) all 6-14year-olds to complete eight years of elementary schoolingby 2010. The SWAp,which featuredjoint support by UK's DfID, the European Union, and IDA (together committingmore the $1 billion over the period 2004 to 2007) had as its overall objective to assist the Governmentof India in its implementationof the SSA Program, by: Reducingout of school children by at least 9 million, with an increase in enrollmentthus moving towards universal elementaryeducation; Narrowing existing gender and social gaps, so that enrollment of girls will reach near parity with boys; enrollment of children of scheduled castes and scheduledtribes will be near parity with that of other groups; and enrollment of children with disability will increase; Enhancing the quality of education for all students so that transition rates from primary (Grades 1-5)to upper primary stages of education (Grades 6-8) and correspondinglearning levels will be improved. CURRENT EFA STATUS 7.3 Given continued problems with data reliability, it is not fully clear how much progress the country has made in school access. Ministry records summarized in Figure 7.1 above show the changes in Gross Enrollment Ratios over the past ten years. Figure 7.1 Gross Enrollment Ratio, Lower and Upper Primary All All Boys Girls Boys Girls Lower Primary(6-I0) Upper Primary (II- 14) 7.4 As seen in the figure, government figures, as far as they are reliable, show significant improvementin the national GER, to near 100percent for boys and well above 90 for girls. Girls' rates improved more than boys over the ten-year period, such that the male-femalegap went from 17points in 1993to 5 points in 2003. GER for upper primary was shown to increase less rapidly (from 54 to 61percent), but the increasewas faster for girls than for boys. The latest Joint Review Mission for SSA (January-February2006) shows primary school gross and net enrollmentratios for boys and girls as of 2004/2005(see Figure 7.2). 7.5 On the positive side, the figurereveals a male-female enrollmentgaps even narrower than in 2002, but on the negative, net enrollmentratios that are around 15percentage points below the GER. One contributorto this differential is student dropout, which the JRM found to be still unacceptablyhigh (in excessof 50 percent in some states).46 If the NER was only around 80 percent in 2005, it is highly unlikely that the GO1will reach its SSA target of universal completionby 2007. The GO1goal for SSA of reaching 100completions among 6-14 year-olds by 2010 seems even more remote, since as of 2002 the GER for upper primary was only about 61 percent. Figure 7.2 Lower Primary School Gross and Net Enrollment Ratios, 2005 GER NER Percent 7.6 The Pratharn sponsoredAnnual Status of Education Report 2005, mentioned above, covering the results of a national household surveyof rural districts, reported more positive findings: 93.4 percent of 6-14 year-olds were found to be in school as of late 2005 (95.4 percent of 6-10 year-olds and 90.8 percent of those 11-14). Unlike government statistics,ASER takes into account enrollment in all kinds of private schools (includingunrecognized), and does not concern itself about which level of schooling the child is enrolled in.47 Of course, it is not clear how many of the more than 90 percent of 6-11year olds who are currently enrolled will complete lower and upper primary school. Good household surveys, includinginformationon school completion,will ultimately be needed in order to determine for sure whether the country is nearing or has reached its universal completion goals.48 7.7 Current projections are that the SSA Project will be successful in reaching its enrollment targets (a reduction of 9 million out of school children -a benchmark already attained),but that 46. It might be confusing to consider such high dropout rates in light of near 100percent gross enrollment ratios. One explanation lies in the fact that student recruitment efforts in the past decade have brought in many formerly out-of-school, overage children (the presence of whom would increase the GER). If there had been low dropout during the period, the GER would have been well over 100 percent, as it is in many Latin American countries. 47. A NER of below 60 percent in upper primary does not necessarily mean that over 40 percent of the age group are out of school. In today's India a substantial number of the age group for upper primary (11-14) are "overage" lower primary school enrollees. The current government MIS format makes it hard to sort this out. 48. During the 3rdSSA Joint Review Mission the MHRD revealed that it had commissioned an outside body to conduct a household survey to study out-of-school children (475,000 households) and the findings reportedly "confirmed the government's estimation of OOSC" (Aide-Memoire, 3'* Joint Review). However, for some reason the survey numbers were not reported. It would be important to know which results they reflected more: the Government's GERs/NERs or Pratham's more positive findings. the country will fall far short of its goal of universal primary school completionby 2007 and 2010 for lower and upper primary, respectively. The main barrier to attainment of the completion goal is high student dropout. Dropout reduction was not even an objective in the SSA project and has, accordingto JRM reports, not received the urgent attention it deser~es.~' Until that is the case, universal primary completion will be an elusive goal. 7.8 In the course of SSA implementation,a nation-wide survey of sthgrade student achievement in mathematics and language was undertaken in 2002 to serve as a baseline for measuringimproved learningoutcomes under the Project. (This was the first nation-wide assessment of this sort; those conducted before and reported on above were for only DPEP districts.) The results, shown in Table 7.1 below, reveal low levels of learning in government schools. The results are norm referenced on a hundred point scale.Results were better for language than for mathematics, and slightlybetter for boys than for girls, although the differences were not significant. Rural-urbandifferenceswere found in language outcomes. Large differences were found across states and districts,as indicated by the high standard deviations. Table 7.1 Gender wise and Area wise Achievementof Grade 5 Students, 2002 Subject Gender Rural Urban Total Mean SD Mean SD Mean SD Mathematics Boys 46.72 21.I1 47.36 21.53 46.9 21.24 Girls 45.54 21.21 47.29 21.61 46.09 21.35 Total 46.15 21.17 47.32 21.57 46.51 21.3 Language Boys 57.95 18.00 61.36 18.43 58.94 18.19 Girls 57.37 18.18 61.89 18.51 58.79 18.41 Total 57.67 18.09 61.63 18.47 58.87 18.3 Source: "Learning Achievement of Students at the End of Class V"; Department of Educational Measurement and Evaluation, National Council of Educational Research and Training, 2003. 7.9 Across the states (see Annex B, Table B.5), average languagescores above 70 were found in Manipur, Tamil Nadu and West Bengal; below 50 in Assam, Chhatisgarh,Himarchal Pradesh, and Jammu and Kashmir; on average math scores, above 60 were Bihar, Manipur, and West Bengal, and below 38 were Goa, HimachalPradesh, Jarnmu and Kashmir, Kerala, Uttar Pradesh, and Pondicherry. Some of the poor performers, like Goa and Kerala, are states with high rates of enrollmentsand low standard deviationson the achievement tests, suggesting a policy of spreading learning opportunitieswidely. Conversely, states like Bihar show relatively good outcomes, but enroll a relatively low percent of the primary school cohort. 49. Reason for high dropout (50percent or more) in many places have not been well researched and there have been few specific remedies developed. The third SSA Joint Review Mission reported: "Given that the First JRM recommended that greater emphasis should be given in future to attendance rates rather than merely enrollment rates in monitoring performance and setting targets, it is disappointing to note there has been only limited follow-up action in this area. The mission recommends ...steps... to intensify efforts aimed at retaining more children in the education system and thus reducing dropout," p. 15 7.10 Thesenorm referenced tests do not provide a clear insight into how well students are performing against a standard. For this, Pratharn7sAnnual Status of Education Report 2005,has provided sobering insights, reported on section 5 above (for more complete data, see Annex B, Table B.6). These findings reveal that a majority of rural students in India are not acquiring basic reading and math skills at an early age. This reveals another questionable design feature of SSA: it's established of a very vague learning outcomes target ("learning will be improved"). There is a growing awareness in the World Bank and other bodies that teachers and managers need specific learninggoals to work towards (see Abadzi et al, 2005), such as xx percent of second graders will be read passages fiom primary 1textbooks fluently. FUNDING SUPPORT EFA FOR 7.11 Over the past two decades India has seen consistent increases in funding for education in general, and basic education in particular (see Figure B.1in Annex B). Education expenditures as a percentage of GDP have risen from below 1percent in the early fifties to above 4 percent in 2000-2001. Also, as seen in Annex B (Figure B.2), there has been a doubling in primary education's share of government allocationsin five year plans from the mid-1980sto the current Ninth Plan. Increased support for primary educationhas been particularly pronounced at the central government level, given a change in policy in the mid-1980s favoring federal subsidies through pro-poor "centrally sponsored schemes" (see Annex B, Figure B.3). Most states have also increased their funding for primary educationduring the past ten years.50 In addition, average fundingper student has increased across the nation from about $25 in 1993to $44 in 2002, a period of rapid expansion of enrollments, showing a determination not to expand at the expense of per capita spending." With more than 10million children still out of school in 2006 and consistently high dropout and low achievement outcomes, spending for educationwill still need to increase. A new tax assessment ("cess") in 2005 of 2% allocated to basic education is expectedto provide an additionalboost. 7.12 India has taken advantageof donor agency interest in supporting Education for All over the past three decades. Prior the beginning of Bank support, small scale projects in various states were being supported by agencies like UNICEF, SIDA, and ODA (now DfID). With UPBEPDPEP the governmentpressed for coordinated donor support, requesting that all donor contributions to basis educationbe made through these two programs, either through parallel projects (EU support for Madhya Pradesh's participationin DPEP; UNICEF's support of EMIS), or co-funding (Netherland's fundingof Gujarat under DPEP 11). In these Projects the donors and the GO1agreed onjoint supervisionarrangements. SSAwas negotiated as a SWAp, in which major co-financing agreementshave been made between DfID, the EC and IDA. Besides using - 50. Ayyar and Bashir (2004) point out that state increases in funding under DPEP were influenced by formal agreements to maintain real expenditures on elementary education at levels extant in the first year ofjoining the Project (instead of using Project funds to substitute for their own funding). This has generally been done, but, as mentioned in Section 6, this was mostly for maintaining recurrent cost spending on salaries and not for development expenditures. There has been a risk that DPEP programs and funding would displace some state-level quality improvement features, a problem found by Clarke and Jha (2006) in Rajasthan (not among the original DPEP VII states). 5 1. Per student expenditures in primary education (2002) varied widely from state to state, from $16 in West Bengal to $68 in Himachal Pradesh, depending on conditions (terrain), commitment to spending, and the proportion of the cohort enrolled (lower proportion often resulting in higher expenditure per capita, as in Bihar). thejoint supervisionsystemlaid down in DPEP, this has also entailed harmonization of procurement and other donor requirements. Given the recent increase in domestic fundingfor primary education (through tax assessment), it is not clear whether the government will continue to seek external funding. However, the government has found that having internationalpartners like IDA in its review missions adds stature and clout to mission events and their agreements. 8. LESSONS LEARNED 8.1 Many lessons have been learned in the course of UPBEP 1/11and DPEP I/II implementation,which apply to both pairs of projects, the main ones being: More concerted effort is needed toprovide access and better learning outcomes among tribal children and the disabled. One problem has been the frequent groupingof scheduled tribes (ST) with scheduled castes (SC), groups whose needs and conditions are vastly different. The use of tribal languages in textbooks and classroominstruction (including recruitingteachers who can use the languages) are still unfulfilled promises. For the disabled,practitioners have noted the need to move beyond the current medical model to a socialmodel of disability, which would include help to teachers on how to assist those with special needs. The information-basedplanning and decision making approachespromoted by UPBEP/DPEPare only as good as the data available to them. Both the education management informationsystem created and the learning assessmentsundertaken during UPBEP UII and DPEP 1/11produced data of questionable quality, creating a crisis of confidencein any conclusions based on them. There is an urgent needfor higher levels of quality control in thegovernment's education information and assessment systems. Also, any evaluationprogram set up to showprogram effectiveness or impact needs appropriate control group information. The main constraint to the GOl's reaching its goal of universalprimary school completion by 2010 is no longer school access but high student dropout. In many project areas the student dropout rate is still above 50 percent, despite goals for drastic reduction. Curiously,reducing dropout was not an objective in the ongoing SSA project. Stronger commitment to dropout reduction and more effective interventions (based on local research on causes) are needed, targeting locations where theproblem isparticularly serious. There is an urgent needfor strategic thinking and decision making concerning the deployment of 'para teachers," taking into consideration equity issues, cost-effectiveness, sustainability, and theprogram's long term impact on the teachingprofession. Para teachers have been recruited in many statesunder DPEP/SSA for various reasons (difficulty finding teachers for remote areas; budgetary constraints;need for speed in hiring) and under different conditions. In the short-term this has solved many teacher shortageproblems, but there are questions about sustainabilityand equity (is there a growing second tier of teachers mostly assigned to low income areas?). Is the use of para teachers a temporary stop-gapmeasure or it is here to stay? How effective is their teaching, compared to fullytrained and certified teachers? What are the career track possibilities for them? Cadshould they be "regularized," and under what conditions? These are questionsthat need to be addressed at the state and national level through additionalpolicy research and dialogue. There is a needfor a clear understanding of the reasons and consequences of the rapid growth of unrecognizedprivate schools, and sharpened government responses. An increasing number of parents, urban and rural, appear to be sending their children - for qualityreasons -to private schools, but this is happening below the education system's radar, since unrecognized schools are not covered by EMIS. This leaves it unclear as to how many of the alleged dropouts are actuallyjust transfers to such private schools. Also, the PPAR mission observed that a high proportion of studentsin such schools are boys. Is it possible that the femaleparity that has been achieved in primary education enrollments is in part an artifact of more girls being lefi in the lower status government schools? There is a need to investigate the possibility that gender inequity, once a factor in state schools, has now migrated to higher-status and higher-priced private schools. Also, given the growing popularityof such schools,it essentialthat this movement be carefully studied: are such schoolsreallyproviding better instruction, are they biased against girls, do they have characteristics which could/should be brought into state schools, and how can state schools "win back" studentsthat have been shiftedto these schools?52 Improving student learning outcomes needs more thanjust setting goals and mobilizing inputs; it needs coherent changes in intermediate outcomes at the classroom level. This implies openingup the black box of learning improvement efforts and a focus on change at the classroom level. More specifically,it means not only recording the amount of training teachers receive but the impact of it on teaching and learningprocesses. UPBEPIDPEP have created a solid structurefor increasedteacher professional developmentopportunities,but so far changes in classroom behavior have been below expectations. One factor in this could be the lack of coherence in the various parts of the system: teachers receive training in certain active learning methods (e.g., multigradeteaching),but the meetings they attend at the cluster centers do not follow up on this, nor do the coaching sessions that they receive from visitingmentor teachers; thus, they do not master or internalizethe new techniques (as evidencedby classroom performance,when it is observed). More coherence among its parts could go along way towards improving the system's impact. 52. It is not that state schools are to be preferred over private schools for ideological reasons, but that families have been promised free public education of good quality. The PPAR team visited some unrecognized fee-bearing private schools which were bursting at the seams, but it did not appear that they were as advanced as the DPEP schools in improving teaching and learning processes. Some of the features that appeared to make them attractive to parents (at least in Karnataka) were relatively good teacher attendance, the provision of "free" school uniforms, and the teaching of English from grade 1. REFERENCES Abadzi, H., Crouch, L., Echegaray,M., Pasco, C., and Sampe, J., 2005. "Monitoring Basic Skills Acquisition through Rapid Learning Assessments: A Case Study From Peru." In Prospects, vol. XXXV, no. 2. Agganval, Y.,2001.16thProgress Overview of DPEP,New Delhi, National Institute of Educational Planning and Administration. Ayyar, R.V.V., 2005. "What Lessons Can DPEP Offer?" In Journal of Educational Planning and Administration, Volume XIX, no. 1,pp. 49-65. Ayyar,R.V.V. and Bashir, S., 2004. "District Primary Education Programme." In Encvclopaedia of Indian Education,edited by J.S. Rajput, New Delhi, National Council of Educational Research and Training. Azim Premji Foundation, 2004. Status of Learning Achievement in India. Bangalore: Azim Premji Foundation. Clarke, P., and Jha, J., 2006. "Rajasthan's Experience in Improving Service Delivery in Education." In Reinventing Public Service Delivery in India: Selected Case Studies, edited by V. Chand, Thousand Oaks, CA: SAGE Publications. EducationWorld, 2006. "Taxes Down the Drain: Little Learning in Government Schools." In Education World,March 2006. EdCil, 2003. External Evaluation of DPEP, Phase I - Summary of Main Findings based on the State Reportsprepared by the Indian Institutes of Management, Ahmedabad, Bangalore, Kolkata, and Lucknow. New Delhi, Research, Evaluation and StudiesUnit, Technical Support Group of DPEP. Gautam, S.K.S.,2002. Student Achievement Under TAS: An Appraisal in Phase IStates, New Delhi, National Center for Educational Research and Training. ,2003. SynthesisReport onStudentAchievement UnderTAS:An Appraisal inDPEP States, New Delhi, National Center for Educational Research and Training. Govinda, R. and Y. Joshephine, 2005. "Para Teachers in India: A Review. " Contemporary Educational Dialogue. 2 (2). Independent Evaluation Group (Formerly Operations Evaluation Department, OED), 2000. India: CountryAssistance Evaluation. Washington, DC: The World Bank. Jalan, J. and Glinskaya, E., 2003. Improving Primay School Education in India: An Impact Assessment of DPEP-Phase I. New Delhi: India Statistical Institute, and Washington, D.C.: The World Bank. Jha J., and Jhingran, D., 2002. Elementary Educationfor the Poorest and other Deprived Groups: The Real Challenges of Universalization. New Delhi: Centre for Policy Research. NCERT, 2003. Learning Achievement of Students at the End of Class V.New Delhi: Department of Educational Measurement and Evaluation, National Council of Educational Research and Training. Pandey, R., 2000. Going to Scale with Education Reform: India's District Primary Education Program, 1995-1999. New Delhi: Education Reform and Management Publication Series, The World Bank. Pratham, 2005. Annual Status of Education Report (Rural). Mumbai, India: Pratham. Rampal, A., 2001. Curriculum Changefor QualityEducation: A Study of Schools in DPEP and non- DPEP Districts in Kerala. Trivandrum, Kerala: Primary EducationDevelopment Society. SIEMAT, 1999.A Study of Drop Outs in Six BEP Districts of UP. Allahabad,U.P.: State Institute of EducationalManagement and Training. SIEMAT, 2005. A Study of Dropout Rates in 5 Districts of Dzflerent Regions in UttarPradesh. Allahabad,U.P.: State Institute of EducationalManagementand Training. Vasavi, A.R., and Chamaraj, K., 2001. Community-SchoolInterlinks: Report of a Socio-Anthropological Study of Primary Education in Five Districts in Karnataka. Bangalore: National Institute of Advanced Studies. World Bank, 1999. Project Appraisal Document: UttarPradesh Third District Primary Education Project, Washington, D.C.: Education SectorUnit, South Asia Region,The World Bank. World Bank, 2003.A Review of Educational Progress and Reform in the District Primary Education Program (PhaseI and II). Washington, D.C., South Asia Human Development Sector,The World Bank. World Bank, 2003. Implementation CompletionReport: District Primary Education Project, Washington,D.C., South Asia Human Development Sector,The World Bank. World Bank, 2004.Project Appraisal Document: Elementary Education Project (SSA), Washington, D.C.: Education Sector Unit, South Asia Region, The World Bank. Wu, K.B., Kaul, V., and Sankar,D., 2005. "The Quiet Revolution," In Finance and Development, Vol. 42, no. 2. Annex A ANNEX A. BASIC DATA SHEET UTTARPRADESH BASICEDUCATION PROJECT (CREDIT-2509) Key Project Data (amountsin US$million) Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Total projectcost 193.77 192.02 99.09 Loan amount 165 163.5 99.09 Cancellation Project Dates Original Actual Board approval Signing Effectiveness Closingdate Staff Inputs Stage of Project Cycle Actual/Latest Estimate Stage of Project Cycle W' Staff weeks IdentificationlPreparation NA ldent:04/15/90 - Prep: 02/21/93 AppraisalINegotiations NA Apprai: 02/22/93- Neg:05/10/93 Supervision 19.09 US$297,400 (06110193 -09/30/2000) ICR 13.79 US$27,612.76 (ICR SECBO 04/12/2001) Total 32.88 Total ect, ~twa retrieved from FACT, but this database IS not used any long at the Bank. SAP IS not pulling the informationfor FY prior to 2000. SW listedfor supervlslon cover FY2000and part of FY2001only (SW cost for those two FY is US$32,143.06),as for supervisionbudgetit covers FY98, FY99, FYOO, F01. SAP could not providespecific on BBfor FY prior to 98. Overall BB (SW and VC) actual as shown in SAP is US$1,066,689.73 brokendownas follows: LEN: US$571,910.15and SPN: US$494,779.22. 40 Annex A Mission Data Date No. of Specializationsrepresented Performance rating (month/year) persons Implementation Development Progress Objective Identification1 November- 11 1 EducationPlanner,2 Economists, Preparation December 1 Social DevelopmentSpecialist, 3 1992 Educationists, 1Architect, 2 ManagementSpecialists, 1 FinancialAnalyst AppraisallNegotiation February- 10 2 EducationPlanners,2 March 1993 Economists, 1 Social Development Specialist, 1 Educationist, 1 Architect.2 Management Specialists, 1 FinancialAnalyst Supervision January 1994 6 1EducationPlanner, 1 Economist, 1Educationist,1Architect, 2 ManagementSpecialists June 1994 7 2 EducationPlanners, 2 Educationists,2 Architects. 1 ManagementSpecialist February 1995 5 IEducationPlanner, 3 Educationists,1Architect July 1995 5 1EducationPlanner, 3 Educationists,1Architect, March 1996 6 1 EducationPlanner. 2 Educationists,1Architect, 2 ManagementSpecialists December 9 1EducationPlanner. 3 1996 Mid Educationists,5 Architects Term Review September 8 2 EducationPlanners, 1 1997 Economist, 3 Educationists, 1 Architect, 1 Management Specialist March 1998 3 1EducationPlanner, 1 Social Development,l Educationist October 1998 7 1 EducationPlanner. 6 Educationists April 1999 7 1 EducationPlanner, 6 Educationists November 8 1 EducationPlanner, 1 Economist, 1999 6 Educationists March2000 8 2 Economists, 6 Educationist ICR December 10 2 Economists, 1 Social 2000 DevelopmentSpecialists, 1 EducationPlanner, 3 Educationists,1Architect, 1 FinancialManagementSpecialist. 1 ProcurementSpecialist Annex A Second Uttar Pradesh Basic Education Project (Credit-3013) Key Project Data (amounts in US$million) Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Total projectcost Credit amount Cancellation Proiect Dates Original Actual Board approval Signing Effectiveness Closing date Actual/Latest Estimate Stage of Project Cycle Af'Staff weeks US$ ('000) Supervision 14.16 US$43,813.42 ICR 12.56 US$29,909.08 Total 26.72 Note: Given the age of the project, it was very difficult to get data on SW. Data used to be retrieved from FACT, but this database is not used any long at the Bank. SAP is not pulling the information for FY prior to 2000. SW listed for supervision cover FY2000 and part of FY2001 only (SW cost for those two FY is US$23,372.42), as for supervision budget listed above it covers FY98, FY99, FYOO, F01. SAP could not prov~despec~ficon BB for FY prior to 98. Overall BB (SW and VC) actual as shown in SAP is US$160,689.58 brokendown as follows: LEN: 50,235.62 and SPN: US$110,453.96. Annex A Mission Data Date No.of Specializationsrepresented Performancerating (month/year) persons Implementation Development Progress Objective - Identification1 March 1996 3 1 Educationist, 1 Educational Preparation Planner, 1 Architect AppraisallNegotiation June 1997 3 3 Educationists Supervision September 8 2 EducationPlanners, 1 S 1997 Economist,3 Educationists,1 Architect, 1 Management Specialist March 1998 3 1 EducationPlanner, 1 Social S Development, 1Educationist October 1998 7 6Educationists,1 Education S Planner April 1999 7 6 Educationists,1 Education S Planner November 8 6Educationists,1 Education S 1999 Planner, 1 Economist March 2000 8 6 Educationists,2 Economists S ICR December 10 2 Economists, 1 Social HS 2000 Development, 1Education Planner,3 Educationists,1 Architect, 1 Financial Management Specialist, 1 ProcurementSpecialist -- April 1996 5 ResearchSpecialist (I); S Management (1);Staff development& Training (1); Civil Works (1);Implementation Specialist (1) District Primary Education Project (Credit-2661) Key Project Data (amounts in US$million) Appraisal Actual or Actual as % of Estimate current estimate appraisal estimate Total project cost Credit amount Cancellation IThe differencebetween the figures can beexplainedby adevaluationof the Indian Rupee from Rs.45 to Rs. 32 that occurredbetweenproject Appraisal and project closing. 43 Annex A Project Dates Original Actual Board approval 11/22/1994 11/22/1994 Signing 12/22/1994 12/22/1994 Effectiveness 03/22/1995 03128/1995 Closingdate 03/31I2002 06/3012003 Staff Inputs (staff weeks) ActuaVLatest Estimate Stage of Project Cycle lf Staff weeks USS('OO0) AppraisallNegotiations NA Supervision 62.69 ICR 13.34 Total Giventhe age of the project,it was very difficult to get data on SW. Data priorto 2000 used to be retrieved from FACT, but this database is not used any long at the Bank. SAP is not pullingthe informationfor FY prior to 2000. SW listed for supervision cover FY2000 and part of FY2003only (SW cost for those two FY is US$98,788.93), as for supervision budget listed above it covers FY98-F03. SAP could not providespecific on BBfor FY prior to 98. Overall BB (SW and VC) actual as shown in SAP is US$1,270,355.67 brokendownas follows: LEN: 520,973.33 and SPN: US$749,382.34. Mission Data Date No. of Specializationsrepresented Performance rating (month/year) persons Implementation Development Progress Objective Identification1 08/06/1993 12 Mission Leader(I), Preparation ImplementationOfficer (I), Architects (2), Principal Sociologist (1), Social DevelopmentSpecialist (1), EducationAdvisor ( I) Management (2); Economist (3) EducationAdvisor (1); Management (2); Gendernribal (1); Economist(3); CivilWorks Specialists(2), Social DevelopmentSpecialist (1). Sr. ImplementationSpecialist (I), Textbook Specialist (I), Education Specialists(2),Distance Education Specialist(I) Supervision April 1995 5 Management (1); Gendernrial S specialist (1); Staff development & Training (1); Civil Works (2) November 5 Management (I); Gendermribal S 1995 specialist (1); Staff Development& Annex A -- Date No, of Specializations represented Performancerating (month/year) persons Implementation Development Progress Objective Training (1); CivilWorks (2) April 1996 5 ResearchSpecialist (1); Management(1); Staff development & Training (1); Civil Works (1); Implementation Specialist (1) November 5 Management(1); Gendermribal 1996 specialist (1); Staff Development& Training (1); CivilWorks (2) April 1997 5 Management(1); Gendernribal specialist (1); Staff Development& Training (1); CivilWorks (2) November 5 ResearchSpecialist (1); 1997 Management(I); Staff development & Training (1); Civil Works (1); lmplementation Specialist (1) April 1998 5 Management(1); Gendermribal specialist (1); Staff Development& Training (1); CivilWorks (2) November 5 Research Specialist (1); 1998 Management(1); Staff development& Training(1); Civil Works (1); lmplementation Specialist April 1999 5 Management(1); Educational Planning (1); Education(2); Staff development& Training(1) November 6 Management(1); Curriculum- 1999 Instruct(2); Sociology(1); Civil Works (1); Inst. Capacity Bldg (1) April 2000 6 School Based management( I); Planning and management(1); School Effectiveness(1); Early Childhood Edu (I); Curriculum& lnstruc(1); EducationalResearch (1) November 7 School.Based management( I); 2000 Planning & management(1); Teaching and Learning (1); Early Childhood Edu (1); School Effectiveness (1); School improvement(2) April 2001 8 School Improvement(3); Planning & Management(1); Special Education(1); Economicsof Education(1); Economicsof Education(1); School Administration (1) November 7 School Improvement(2); Planning 2001 & Management(1); Early Childhood (1); Special Education (1); EducationEconomist(2) 45 Annex A Date No. of Specializationsrepresented Performancerating (month/year) persons Implementation Development Progress Objective April 2002 8 School Improvement (2); Early S S Childhood(1); Special Education (1); Economics(2); Gender Issues (1); Assessment (1) November 8 Teaching and Learning (1); Early S 2002 Childhood(1); Disabilities (1); Economics(2); School Improvement(3) ICR 9 Pedagogy(1); School 05/05/2003 Improvement(2); Special Education(1): Education Economist(3); Civil Works (I); InstitutionalAnalysis (1); CommunityProcesses(1) District Primary Education Project I1 (Credit-2876) Key Project Data (amourzts in US$ million) - -- - Appraisal Actual or Actual as % of esfimafe current estimate appraisal estimate Total project cost Credit amount Cancellation Project Dates Original Actual Board approval 06/06/1996 06/06/1996 Signing 0711511996 0711511996 Effectiveness 1011311996 loll 311996 Closing date 0613012003 0613012003 P, 2 The original principal amount of the Credit was fully disbursed(SDR 291.7 million). Difference between appraisal and actual project costs 1s due to the devaluation ofthe Rupee and the fluctuations in the exchange rate of the SDR vis-a-vis the USdollar between the time of appraisalof the project and the current date. Annex A Staff Inputs (staff weeks) -- Stage of Project Cycle Actual/Latest Estimate NO Staff weeks US$ ('000) IdentificationlPreparation NA NA Supervision 82.785 297,810.25 ICR 13.342 16,096.39 Total Note: Giventhe age of the project, it was very difficultto get data on SW. Data prior to 2000 used to be retrievedfrom FACT, but this database is not used any long at the Bank. SAP is not pullingthe informationfor FY priorto 2000. SW listed for supervision cover FY2000 and partof FY2003only (SW cost for those two FY is US$140,620.25),as for supervision budget listed above it covers FY98-F03. SAP could not providespecificon BBfor FY prior to 98. Overall BB (SW and VC) actual as shown in SAP is US$1,078,486.85 brokendown as follows: LEN: 474,776.50 and SPN: US$603,710.35. Mission Data Performancerating Date No. of Specializationsrepresented (month/year) persons Implementation Development Progress Objective Identification1 05/0611995- 5 1MissionLeader. 1WID Officer, 2 S S Preparation 0511511995 EducationSpecialists,l CivilWorks Engineer 1 EducationSpecialist, S S 2 Architects, 1 Principal Sociologist, 1 Social DevelopmentSpecialist 1Team Leader, 3 CivilWorks S S Specialists, 1Sr. Social Development Specialist; 1Sr. lmplementation Specialist, ITextbook Specialist, 1 Sr. Economist,1Sr. Education Specialist, 2 EducationSpecialists, 1 Distance EducationSpecialist, 1 Operations Assistant 1Team LeaderIEducationAdviser. 2 S S Civil Works Specialists, 1Social DevelopmentSpecialist, 1 Sr. lmplementationSpecialist, 1Textbook Specialist, 1Sr. Economist,2 Education Specialist, 1Sr. EducationSpecialists, 1 DistanceEducationSpecialist 14membersof Governmentdelegation, S S 6 Bank Staff team Supervision 11/14/1996 6 1 Management Specialist, 3 Education S S Specialists, 1 Education Planning, 1 Civil Engineer 1 Management, 1 Educational Planning. S S 2 EducationSpecialists, 1 Staff Developmentand TrainingSpecialist 3 Management Specialists, 1 Education 47 Annex A Performancerating Date No. of Specializations represented (rnonth/year) persons Implementation Development Progress Objective -- Specialist, IEducational Planning, I Staff Developmentand Training Specialist, 1Civil Engineer, 1 Curriculum Renewal 1 ManagementSpecialist, 1Curriculumand Instructional development Specialist, 1Civil Engineer, 1InstitutionalCapacity BuildingSpecialist, 1 Educational ManagementlSocial, 1Education Economist 1 School BasedManagement Specialist, 1MultigradeTeaching, 1 School Effectiveness, 1 Planningand Management,1 Early Childhood Education, 1Educational Research, 1 Curriculum and Instructional DevelopmentSpecialist 1 School Effectiveness. 1School Based Management,2 School Improvement Specialist, 1 Education Management and Planning Specialist, 1 Early Childhood Education, 1 Learning and Teaching Specialist 1School Effectiveness, 1 Team Leader, 1General Educator, 1Economist. 1 EducationalPlanning, 1 Early Childhood Education, 1 PedagogyIResearch,1Project Management,1 Community participation, 1 Management Capacity Building 1Team Leader, 1Educator/Pedagogy, 1Education Management, 1Early childhood Education, 1 Economist, 1 OperationsAnalyst, 1 Research,1 community parkipation 1Team Leader, 1School Improvement, 1Teacher Development, 1Education Management& Planning, 1 Education Finance, 1 Pedagogy, 1 Economist, 1 EducationAdministration 1School Improvement,:! Pedagogy Specialists, 1 EducationManagement Specialist, 1Child Development Specialist, 2 EducationEconomists 1General Educator,l Education Management, 1 Early Child DevelopmentSpecialist, 1Gender Specialist, 1EducationEconomist, 1 Teaching and LearningSpecialist I Education Finance Specialist, 1Student Assessment Specialist 2 School lmprovementSpecialists, 1 Special Education, 1Student Assessment Specialist, 2 Economists, 1 Early childhood Educationist, 1Gender 48 Annex A Performancerating Date No. of Specializationsrepresented (month/yearJ persons Implementation Development Progress Objective Specialist 05/06/2002 8 2 Learningand CompletionSpecialists, S S 2 ProgramImplementationSpecialists, 2 SustainabilitySpecialists, 1 Education Management and PlanningSpecialist, 1 EquitySpecialist 12/23/2002 7 1 General Educator, 1 EarlyChildhood S EducationSpecialist, 1 Institutional Capacity Building, 1 Pedagogy, 1 School Improvement, 1 Education Economist,1 Project Management Specialist ICR 05/05/2003 10 1 Team Leader, 1 Pedagogy. 1 S S Community Development, 1Evaluation Expert, 1 Management& Planning, 2 Economists, IArchitect, 1 FMS, 1 Procurement Annual FinancialManagementand Procurementreviews by Bank specialists took place outside the JRM. 49 Annex B ANNEX B. PROJECT AND NATIONAL EDUCATION DATA Table B.1. District Means scoreson DPEP tests, Grade 1 (Phase 1 or DPEP I) 1 Phase 1 Language 1 Math 1 I BAS1- IMAS- IBAS1- IMAS- StateDPEP District BAS2 1TAS Target BAS2 ITAS Target Assam: Darrang 44.5% 11.5% YES 23.6% 6.0% YES Dhubri 2.5% 20.4% NO 0.1% 28.3% YES Morigaon -9.8% 9.0% NO -5.0% 5.1% NO Karnat: Belgaum 14.5% 2.5% NO 27.1% 6.1% YES Kolar 1.8% 10.9% NO 36.5% 10.2% YES Mandya 26.4% -0.7% YES 37.6% -1.7% YES Raichur 32.6% -3.6% YES 53.0% -1.3% NO Kerala: Kasargod 42.1% -1.8% YES 43.0% -2.2% YES Malappuram 76.2% 24.1% YES 106.0% 14.5% YES Wayanad 56.7% 8.8% YES 81.2% 1.6% NO Mahar: Aurangabad 53.0% -11.5% YES 63.5% -11.3% YES Latur -4.8% 4.6% NO 7.2% 5.0% NO Nanded 11.4% -1.3% NO 39.8% 0.6% YES Osmanabad -27.8% 11.O% NO -18.5% 10.0% NO Parbhani 22.5% -3.6% NO 41.2% -4.3% YES Haryana: Hissar -20.7% 34.2% NO 41.O% 26.8% NO Jind 69.3% 48.8% YES 72.3% 69.2% NO Kaithal 25.2% 6.0% YES 48.7% 7.1% NO Sirsa -18.3% 26.3% NO -6.7% 19.2% NO TN: Cuddalore 1.7% 35.0% YES 65.5% 46.7% YES Dharampuri 13.0% 40.7% YES 61.3% 55.3% YES Thiruvannamalai 19.0% 61.9% YES 103.6% 78.2% YES Source:SS.K.SGautam,Student Achievement Under TAS: An Appraisal in Phase I States,NCERT, 2002 50 Annex B Table B.2.District Means scores on DPEP tests, Grade 3 or 4 (Phase 1 or DPEP I) Phase 1 Language 314 I Math 314 I MAS- I BAS1- IMAS- I StateIDPEP District --BAS1-BAS2 TAS Target I BAS2 ITAS Target I Assarn:Darrang 37.3% 13.8% 5.7% 14.1% NO Dhubri 30.2% 2.0% YES -9.9% Morigaon 8.2% 1 21.1% YES -18.1% 20.3% NO Karnat: Belgaum YES 62.3% -2.1% YES Kolar YES 48.1% 9.9% YES Mandya YES 3.4% 34.7% YES 1 Raichur YES 46.1% -41.9% NO Kerala: Kasargod YES YES Malappuram NO YES Wayanad NO NO Mahar: Aurangabad ij YES Latur YES Nanded YES Osmanabad YES Parbhani YES Haryana: Hissar Jind Kaithal Sirsa YES TN: Cuddalore YES Dharampuri YES YES Villupuram 57.0% 32.3% Source: S S.K.S Gautam, Student Achievement Under TAS: An Appraisal in Phase I States,NCERT, 2002 5 1 Annex B Table B.3.District Means scores on DPEP tests, Grade 1 (Phase 2 or DPEP 11) Phase 2 Lang 1 Math 1 BAS1- MAS- BAS1- MAS- State District BAS2 TAS Target BAS2 TAS Assarn Barpeta 104.5% -22.55 YES -5.31 109.16 YES Bongaigaon 59.3% 17.32 YES 13.83 29.98 YES Goalpara 105.0% -7.17 YES -13.68 119.57 YES Karbi-Anglong 69.3% -31.16 YES 5.37 21.27 YES Kokrajhar 10.1% 1.01 NO -0.08 16.25 NO Sonitpur 100.5% 0.75 YES -12.55 82.06 YES Chhatisgarh Bastar 4.2% -3.08 NO 9.97 14.99 NO Dantewada 4.2% 25.80 YES 23.99 14.99 YES Dhamtari 24.7% -10.19 NO -10.62 43.57 YES Kanker 4.2% 0.20 NO -3.24 14.99 NO Mahasamund 24.7% -14.40 NO -11.15 43.57 YES Raipur 24.7% -59.01 YES -6.44 43.57 YES Gujarat Banaskantha 13.1% 35.53 YES 41.46 16.04 YES Dangs 27.4% 16.23 YES 12.12 21.89 YES Panchmahal 21.9% 11.41----YES 11.56 21.12 YES Haryana Bhiwani 38.4% 26.95 YES 39.74 51.41 YES Gurgaon 18.5% -12.55 NO -3.37 42.09 YES Mahendergarh 69.1% 12.34 YES 16.80 81.30 YES Hirnachal Pradesh Chamba 15.6% -6.68 NO -12.57 29.76 NO Kullu 17.4% 8.57 YES -9.35 19.55 NO Lahauil-Spiti 9.8% -0.55 NO -12.52 23.91 NO Sirmour 8.3% 7.56 NO -0.94 17.48 NO Bangalore Karnataka (rural) 2.2% 47.10 YES 21.86 66.69 YES Bellary 46.1% 35.12 YES 15.71 78.40 YES Bidar 40.7% 22.10 YES 32.18 14.19 YES Bijapur -1 1.7% 60.78 YES 26.33 38.51 YES Dharwad 4.1% 60.94 YES 27.27 38.68 YES Gulbarga 1.1% 40.06 YES 25.12 12.78 YES Mysore 46.4% 25.63 YES 11.57 44.96 YES Madhya Pradesh Barwani 57.9% 4.18 YES 8.88 114.15 YES Bhind 67.6% 6.36 YES 8.18 104.90 YES Darnoh 55.1% 27.18 YES 19.26 131.36 YES Datia 61.2% 46.32 YES 43.05 44.50 YES Dewas 47.6% 2.32 YES 12.90 53.97 YES Dindori 70.1% 51.41 YES 23.83 96.42 YES Jhabua 52.4% 32.84 YES 31.67 63.91 YES Khandwa 50.7% 17.01 YES 14.40 63.98 YES Khargone 57.9% 36.46 YES 32.92 114.15 YES Mandla 70.1% 51.54 YES 40.30 96.42 YES Morena 46.9% 16.26 YES 7.50 64.88 YES Seoni 72.5% 13.85 YES 8.19 98.00 YES Shajapur 50.5% 37.24 YES 32.16 109.15 YES 52 Annex B t Table B.3. District Means scores on DPEP tests, Grade 1 (Phase 2 or DPEP 11) Phase 2 Lang 1 Math 1 BAS1- MAS- BASI- MAS- State District BAS2 TAS Target BAS2 TAS Seopur 46.9% 30.06 YES 26.92 64.88 YES Shivpuri 154.4% 7.38 YES 6.64 163.45 YES Vidisha 52.8% 9.35 YES 9.98 27.56 YES Maharashtra Beed 21.1% 12.27 YES 12.05 28.62 YES Dhule 91.6% 2.28 YES 9.98 65.36 YES Gadchiroli 35.1% 7.44 YES 9.42 65.68 YES Jalana 11.6% 20.15 YES 16.61 3.29 NO - Orissa Bolangir 43.1% 4.76 YES 9.54 20.58 YES Baragarh 71.5% 9.61 YES 15.87 97.78 YES Dhenkanal 29.0% -6.28 NO -3.67 26.01 NO Gajapati 7.5% 1.13 NO 77.21 -24.78 YES Kalahandi 67.9% -1.66 YES 12.39 53.21 YES Keonjhar 29.2% -4.58 NO -5.38 33.52 YES Rayagada -2.9% 10.32 NO 13.26 12.93 YES Sambalpur 12.3% 7.99 NO 3.15 24.08 YES Tamil Nadu Perambalur 69.6% 1.36 YES 8.86 136.48 YES Puddukottai 32.9% 15.48 YES 17.54 105.01 YES Rarnnathpuram 25.9% 6.72 YES 10.28 83.21 YES Uttar Pradesh Badaun 77.9% 5.02 YES 3.48 109.09 YES Balrampur 29.0% 27.30 YES 0.68 72.41 YES Bareilly 44.8% 54.80 YES 57.51 118.88 YES Basti 4.9% 20.36 YES 14.88 41.34 YES Deoria -10.5% 29.62 NO 11.47 4.82 NO Firozabad 25.9% 37.31 YES 19.97 100.33 YES Gonda 29.0% 34.29 YES 5.04 72.41 YES Hardoi 23.6% 47.63 YES 27.03 99.56 YES JP Nagar 37.1% 4.32 YES 3.39 118.04 YES Lakhmipur Source: S.K.S Gautam, Synthesis Report on Student Achievement Under TAS: An Appraisal in DPEP States, NCERT, 2003 53 Annex B TableB.4 District Means scores on DPEP tests, Grade 3 or 4 (Phase 2 or DPEP 11) Phase 2 Language 314 Math 314 I State District BAS1-BAS2 MAS-TAS I Target BAS1-BAS2 MAS-TAS Target Assam Barpeta 29.6% -14.4% NO 34.2% -13.5% NO Bongaigaon 11.7% -0.4% NO 20.4% 19.1% YES Goalpara 54.5% -29.5% YES 58.6% -19.4% YES Karbi-Anglong 40.3% -16.0% NO 38.8% -17.8% NO Kokrajhar 1.6% -11.1% NO -8.4% 15.2% NO Sonitpur 30.1% -4.3% YES 48.1% -16.9% YES Chhatisgarh Bastar -21.1% 86.4% YES -30.6% 92.7% YES Dantewada 8.4% 3.8% NO 7.9% 4.0% NO Dhamtari 9.6% 12.4% NO 2.1% 23.0% YES Kanker 0.6% 27.1% YES 5.7% 41.4% YES Mahasamund -20.7% 23.4% NO 7.1% 39.3% YES Raipur 17.6% -2.1% NO 62.6% -13.9% YES Gujarat Banaskantha -2.6% 69.5% YES -21.1% 88.3% YES Dangs 1.4% 46.5% YES -12.1% 66.8% YES Panchmahal -19.8% 42.0% NO 9.2% 65.9% YES Haryana Bhiwani -6.6% 17.2% NO 116.1% -29.6% YES Gurgaon 31.8% -1.2% YES 50.0% 6.3% YES Mahendergarh 54.2% -0.9% YES 108.3% 2.2% YES Hirnachal Pradesh Chamba 22.5% 7.7% YES 9.7% 16.9% YES Kullu 4.0% 32.6% YES 4.7% 63.4% YES Lahauil-Spiti 7.1% -5.2% NO 18.8% -6.3% NO Sirmour 7.1% 11.0% NO 18.8% -0.6% NO Bangalore Karnataka (rural) 31.1% -9.4% NO 24.0% 0.6% YES Bellary 7.1% -8.5% NO 18.8% -7.1% YES Bidar 31.1% -2.5% YES 24.0% -1.9% YES Bijapur 31.1% 1.5% YES 24.0% -22.4% YES Dharwad 25.6% 11.5% YES 10.3% 20.2% YES Gulbarga 17.6% -6.6% NO 11.7% -22.4% NO Mysore 52.9% 13.0% YES 39.5% 21.8% YES Madhya Pradesh Barwani 40.7% -31.3% NO 39.4% -22.0% NO Bhind 37.8% -19.5% NO 11.4% 22.2% YES Damoh 28.3% -23.3% NO 24.9% 4.2% YES Datia 34.7% -23.8% NO 24.4% 9.9% YES Dewas 15.1% 37.0% YES -3.7% 43.1% YES Dindori 76.6% 0.9% YES 95.4% 1.3% YES Jhabua 110.3% 34.1% YES 80.6% 69.3% YES Khandwa 42.6% 47.6% YES -0.7% 89.0% YES Khargone 14.3% 46.0% YES 16.8% 85.1% YES Mandla 47.8% 39.6% YES 14.9% 77.1% YES Morena 47.8% 9.7% YES 6.2% 30.5% YES Seoni 43.2% 18.6% YES 39.1% 28.5% YES Shajapur 15.1% 59.7% YES 11.3% 136.4% YES Seopur 47.8% 26.1% YES 14.9% 56.9% YES 54 Annex B TableB.4 District Means scores on DPEP tests, Grade3 or 4 (Phase 2 or DPEP II) Phase 2 Language 314 Math 314 I State District BAS1-BAS2 MAS-TAS ITarget BAS1-BAS2 MAS-TAS Target Shivpuri -7.9% 103.1% 1 YES -24.7% 1 81.0% YES Vidisha 63.5% 9.0% YES 53.0% 28.4% YES Maharashtra Beed 15.3% 33.1% YES 2.5% 73.1% YES Dhule -7.9% 159.5% YES -26.3% 156.7% YES Gadchiroli 106.6% 24.8% YES 160.3% 0.3% YES Jalana 57.6% 19.6% YES 53.6% NO Orissa Bolangir 6.6% -19.2% NO -2.2% NO Baragarh YES 35.1% YES Dhenkanal NO 37.8% YES Gajapati NO 25.0% NO Kalahandi NO 7.0% YES Keonjhar YES 32.3% YES Rayagada NO -0.2% NO Sambalpur YES 9.7% NO Tamil Nadu Perambalur YES 24.5% YES Puddukottai YES 55.4% YES Rarnnathpuram YES 5.8% YES Uttar Pradesh Badaun YES YES Balrampur YES YES Bareilly YES YES Basti YES YES Deoria NO NO Firozabad YES YES Gonda YES YES Hardoi YES YES JP Nagar YES YES Lakhrmpur Kheri YES YES Lalitpur YES YES Maharajgani YES YES Moradabad YES YES Pilihbhit YES YES SK Nagar YES YES Shahjahanpur YES YES Siddharthnagar YES NO raisal in DPEP States, NCERT, 20 55 Annex B TableB.5. AveragePercentCorrectof BaselineAssessmentof Grade5MathematicsandLanguage, 2002 Arunachal Pradesh nd Evaluation, National Council of Educational Research and Training, 2003 56 Annex B Table B.6 Rural Student Learning Outcomes from Annual Status of Education Report, 2005 % Children who CANNOT read ... % Children who CANNOT solve numerical written sums of ... Subtraction or Level 1" Level 2** Divisioil Divisio~i Age: 7-14 ALL 34.9 51.9 41.1 65.5 Age: 7-10 ALL 48.2 67.7 53.8 79.5 Age:ll-14 ALL 17.2 31.0 24.3 47.0 Govt: Std 11-V 43.9 65.3 49.7 77.9 Pvt: Std 11-V 32.1 52.4 38.1 65.9 Govt: Std VI-VIII 9.3 22.2 17.0 40.0 Pvt: Std IVI-VIII 6.7 16.8 14.6 33.4 Source: Prathanz: Annual Status of Education Report, 2005. "Level 1: Ability to read a small paragraph with short Subtraction: 2 digit subtraction w/ sentences of std 1 level difficulty borrowing **Level2:Ability to read a "short"text with some long Division: 3 digits divided by one sentences of std 2 level difficulty digit. Figure B.1. Expenditure on Education 1951-2000 Annex B Figure B.2. Share of Expenditure on Elementary, Secondary and Higher as a % Of the Total EducationExpenditurein the Five Year Plans Figure B.3. Central Government Spending on ElementaryEducation and SSA (US $ millions), 1991192-2003104 Figure 1.1$2 Cemlralf;ovcramc.nf Spmtiiagaa ElemerttitvEdawtian ara4 %A IF2 SS M311ion), 199IA2 2M3fQ1 - ---- 0 0 ~8 ~ SSA 1 DPEP OMDM ' p E EfiP, QUBB Source:Ar~alysisaFBadgeadDxpendaturre an Wucatina,wcrrin~symmlpqy; GavcmmerlX uf India, 3f~nis%y~f Humm Resource Dt)e!~s.lopn.tenz,(Dcpmnlcnt of Secondmyand Highermucntlan),Plmnsng Monxsotirtg; Netv Dclhi Annex C ANNEX C. PROJECT OUTCOMES AND OUTPUTS Figure C.1. IEG Summary of UPBEP MI Project Objectives, PerformanceIndicators and Targets,Methods and End of Project Status Project Objective Outcomes/Outputs Incomes Target Method End of Project Status 1. ImorovineAccess to . a Enrollments(Male/Female/SC/ST' L. Primarv Educanon Total) 50% decrease girls out-of-school (00s), . EMIS/ GER 91% vs 50% baseline (82% decline) Primary 900,000new piaces (no breakdowns)' Proj data 981,000 Upper Primary 350,000new places (no breakdowns Ditto 365,000 b. Gr Enroll Ratio M/F/Tot) Primary UPBEP I: 85/71/?;UPBEPII:100( ~ o t a l ) ~ UPBEP VII: M: 94.4; F: 91.4; Tot. 93.0 Upper Ptimary UPBEP I: 76/64/?; UPBEPII: 75 (Total) WBEP VII: M: 78; F: 65; Tot: 71 c. SCISTGr Enroll Ratio (MIF) Primary UPBEP I: 109/71/?;(No targets for ST) EMIStDir SC: M: 126;F: 118.6;Tot:-121 Upper Primary UPBEP I: 43/22/?; (No targets for ST) Data SC: M: 61; F: 53; Tot: -58. d. NFE Enrollments 150,000new recruits NFE At least 20,000 new students enrolledbut eventually (dropped after Project mid-tern review) records phased out. 2. Reducingprimary a. Dropout rate -50% reduction compared to baseline estimates -Reduction by 45% (male:30%; female: school dropout -UPBEPII: 90% of IS'graderscomplete 1' year 54%) of primary -No data -No target b. SCIST dropoutrate. DO rate: M: 32.5%; F: 31.8%(not chgrate) 3. Improving student a. Learningachievement -50% improvementover baseline (primary and Sample -Allegedlytargets surpassedbut methods questioned learning outcomes upper) survey -No data b. SC/STlearning achievement -No target -Improvementin student-teacherinteractionand socio- c. Teachingmethods4 -UPBEPII:improved methods introduced and Special emotionalclimate of classrooms(more student friendly) impact learning study compared to baseline 60 Annex C 4. Improving Institutional State Level Capacity a. Status of EFA Society Fully Functional Fully staffed and equipped, appraisedlapproved annual work plans, monitored and evaluated WBEP implementationand prepared state EFA plan. -Established and functional:provided 1600training per b. Statusof SIEMAT -Fully operational,providing 1000person wks of week per district; completed23 research studies (but of trng per yr and completing5 major studies per variable quality). year. -State plan was prepared and is manifest in subsequent c. Status of State EFA Plan -Completed,manifest in subsequentprojects projects (DPEP I1and 111) -Improved procedures -Teachers included in textbookpreparation;textbook d. Statusof MLL Cunic Revision supplementsproduced having local content -Cumculum revised;new generationof textbooks& -Curriculumrevised; textbooks in schools supplementsproduced/ distributed. e. Status of SCERTCunic Devl. Proc -Computerized;fully functionalat the state level -A computerized system is functionalat the state level; f. Statusof MIS system -Established data quality is uneven -No target specified -Not developed at SIEMAT as planned g. Statusof AssessmentSystm -Committee supported a total of 63 research studies and h. Research/Evaluation Studies findings shared (no quality indicator) -Fullyoperational;providing 1000pers wks of -DIETSstrengthened;BRCs provide 4300 wks of training District Level trainingper year per year per district; some DIETSdo not reach desired a. Statusof DIETS and BRCs -Computerized;fully functional in proj level of effectiveness Districts -A computerized system is functionalin all b. Statusof MIS system -Planningmanagement teams trained Districts;data quality in uneven -Districtplanning teams in place in initial districtsbut not c. Status of District Planning in reserve districts (given lack of funds Project Objective Indicator Target Method End of Project Status 5. Enhancing Percentof villages having 50 percentof villages Community fully functionalVillage Participationin EducationCommittees PrimaryEducation 1. SC = Scheduled castes; ST = Scheduledtribes 2. These actuallyoutput (notoutcomes) indicators. Actuals were computed from school/classroomconstruction data on assumption of 3 classroomsper school and 40 places per classroom. 3. UPBEPII: Capacity to enroll all 6-10 year olds; 75% of all 11-13year olds. 4. Takenfrom UPBEPll ProjectAppraisal Document, 1997,"Monitoring and EvaluationArrangements." 61 Annex C Figure C.2. IEG Summary of DPEP H I Project Objectives,Outcomes/OutputsIndicators, Targets and End of Project Status Project Objective Outcomes/Output Indicator Target End of Project Status 1.Improving access Reduction of enrollmentdisparitiesby gender To less than 5 percent - In DPEP (phase 1) for gender:below 5% in 6 of 7 states; in and casteltribe DPEP I1below 5% in 6 of 10 states; for SC, Pandey (2000) reports social equityindex to be 100in all DPEP districts and for DPEP I1 above 95% in 80% of districts;for ST data incompletebut considered unmet. 2. Reducingprimary school Dropout rate DO rate reduced to less than 10%. - in DPEP dropout fell below 10%in Kerala districtsonly; Dropout Districtsin Haryana,Maharastra, Kamataka, and Tamil Nadu have DO ranged from20-40%; in Asam it was over 40%; in DPEP I1only 16%of districtsended up with DO below 10%. State averageswere 10%to 40% for project districts; overall average for project districtswas 30%. 3. improving student learning a. Learningachievement 25% increase over baseline estimatesin learning - In DPEP grade 1:5% reach target in language; 61% in outcomes achievementin language and mathematicsin math; grade 3-4 8% in language an d65% in math; in DPEP final year of primary school (grade4 or 5) I1 grade 1:78% reached target in language and 87% in math; grade 3-4 68% in language and 81% in math.55 4. Improving Institutional National Level Capacity a. DPEP bureau will be fully functional - 140sub-projectsappraised annually - No data on spccific indicators - 140sub-projectssupe~isedtwice annually - Ditto (ICR indicates "appraisal" (progress)reports provided 2x annually) - DPEP Bureau becomes a strongforce in program managementand primary education reform b. School StatisticalMIS computerizedto - Softwareplatform adopted and installed in all - Completed district level DPEP states - MIS facilitiesestablishedand equipped - Completed (NationalMIS cell, 6 states,23 districts) - Stafftrainingfor 50 systemmanagers, 60 - training undertaken but numbers not reported computer operators, 100 disb' block supervisors, 10,000school headmasters - Reportstwice annually - Regular reports provided,but not clear if twice annually. Releasingannual "report card" (stateand district)becomes the pattern. - MIS system effectivenessand efficiency: -No infol~nationabout this: accuracyof school statistics is 85%accuracyof school statistics problematic. c. National Training ResourceGroup - Number and quality of training designs, -No data on these five specific indicators;data from established to support state and district training prototype trainingmodules and master trainer BRCs/schools show teachers receiving 3 to 20 days of activitiesfor: teachers,head teachers;cluster program. insewice training per year; most locationsalso have outreach resource teams; and master trainers. (teacher meetingsandlor school visits) from cluster resource centers (SRCs) around oncelmonth. - Impact of trainingon teacher performance: - Somequalitative studies undertaken by participatingstates improvedclassroomteachingprocesses for more show active teachingand leaming in various locations,but than 50% of trained teachers given no baseline data attainmentof target could not be determined. d. DPEP Program EvaluationResearchand -24 completed research trainingprograms - No data on indicator Studiesestablished to assess DPEP program - Learningachievementstudies of high quality - Leaming achievementassessed in grade 1& grades 3 or 4 strategies and impact and build evaluation completed in 3rd & 6th year of project in 23 (penultimateprim grade) in all districts (baseline,midterm & 1. During implementation DPEP management decided to assess language and math skills in grade 1 and in the penultimate year of primary, which is in some states is grade 3 and in others grade 4. Annex C Project Objective Outcomes/Output Indicator Target End of Project Status research capacityin state institutions districts; in additionaldistrictsjoining the terminal); qualityunclear program - Multivariateanalyns of achievementdata using - No evidenceof multivariateanalysishavingbeen school effectivenessmodels undertaken - 50 research studies of good quality - 180research studies supportedby DPEP completed; commissionedand completed in first, fourth and quality uneven seventh year of project -researchernewsletterissued twice annually -No data on researchernewsletter - 6researcher conferences completedand results -No data on reseacherconferences published e. Program of Technical Assistanceto States - 10prototype training modules completed and in - No specific data on output; in Education Planningand Management regional languages(includingmicro-planning, establishedto develop capacityin the State stateldistrprog management,schl supenision) (Topdown planning of training generally abandoned in favor Institutes of Educational Managementand -24 trainer training programs completed with of needs-basedtraining covering areas of difficulty(hard Training to train state and districtoficials. counterpart state agencies spots) in the schools in the states and districts.PPAR mission observes training events in districtsbut in limited numbers of topics.) f. Program for Reading Skills and -Micro-analysisof baseline achievementdata Workshops were held at thenational and state levels to ComprehensionEnhancementestablishedto: completed discuss difficultiesin teaching languageand mathematics assess reading readiness for grade I, identify -Readingreadmessstudy completed and to identifyand share strategiesbeing used by different best practice in India and internationally, -Reading difficultiesstudy completed states. The textbooks in all states were revised. developreading research capacity,develop -Prototype materials of package completed and of field tested exemplary reading materials; good quality No mention of studies on readiness and difficulties, encourage exchangeof materialsamong states; -Developmentteams trained in each state developmentteams, and evaluation studies at grade 2. build capacityfor reading prog dev't in state -Regional language adaptationsfield tested and in institutions use on 6 states -Evaluationstudiesusing baseline assess- ment tests will show at least a 25% increase in reading comprehensionand word knowledge in Grade2. g. MathematicsActivity Program (MAP) -Best practicespaper published Workshops were held at the national and state levels to establishedto: assessbarriers to math -Complete MAP package of good quality field discuss difficultiesin teaching languageand mathematics achievement,identifybest practices,develop tested and published and to identifyand share shategies being used by different prototype activitiesand materials, and build -Developmentteam functioningin 6 states states. The textbooks in all states were revised. capacity in stateresource institu- tions to adapt -MAP adaptationsin use in regional languagein and further develop MAP activitiesand 6 states No mention of best practices paper, development teams, and materials -Evaluation studies using experimental design evaluation studies at grade 2 will show at least 25% increase in mathematics achievementin Grade 2 h. Program for Instructional Materials -Curricularevised in 6 states - Cumcula revised in all DPEP states Renewal establishedto review existing -New textbooks, workbooks and teachers' gulides - No data on use of textbooksand inservicetrainingre curricula,design anew MLL curriculum; in use in schoolsin 23 dist., in service to prepare textbooks develop and test prototypetextbooks, teachers workbooksand teacher guides, and build -Each state textbook writingprocess includes - Textbookcreation includes involvementof teachers and capacity at statelevel. - teachers as textbook writers field testing in all states - field trials followed by revisions -New materials have impact on learning -No explicit informationon impact of new materials on achievement learning achievement Annex C Project Objective OutcomeslOutput Indicator Target End of Project Status i. Program of special emphasisfor ST - Increasein the number of tribal languagesin -No evidenceof textbooks being produced in mbal Education established. which primary instructionalmaterials available languages but some supplementarymaterialsproduced - Increase in the number of ST persons in training - nospecificevidence given as teachers - Availabilityof evaluationsand documentation - Noinformation of effective interventionstrategiesfor ST learners State Level (e.g.,Kamataka) - Average 90% of annualphysical targets & - Nospecificreports from States in ICR a. State Implementation Societiesestablished expendit across state activitiesand districts (KPSVY) - Supervisionmissionsto districts4 times annually - ReportssubmittedQuarterlyfrom second year of project - Annual Work Plan and Budgets (AWPB) 100% -Annual reportsand Work Plans largelyon time on tlme b. State School StatisticsMIS -Softwareplatform adopted and installed established by end of project year I -MIS facilitiesestablishedand equipped by end of project year 1 - Staff baining, initial and refresher (6admin; 6 programers/operators;3 systemsanalysts - Quarterlyreports submitted - 100percent conformancewith GO1data requirements. c. DSERT capacityto develop and publish - Textbooktrialling; training completed for cumc -Completed as planned new MLL textbooks and related materials dev-pas, authors,editors, illustrators,evaluators, strengthened des~gners& prod/ distrib staff - Desktop publishing capacityinstalled & fully -Completed utilized;prod, stock and dist control systems computerized - New high quality MLL textbooks and auxiliary -Completed;schoolsvisited during PPAR missionsall had materialspublished & in use in all proj districts new MLL textbooks and some auxiliary materials (relative qualitynot discernable) d. DSERT capacityto improve pre-service -Conferencehall constructedand equipped' -DSERTbuilding upgraded teacher training strengthened documentationcenter and library upgraded; computerunit estiequipped - NCERT materials adapted and tested; additional - Materialsproduced (not clear if minority language state-specificmaterials developedincluding those materialsproduced). in minoritylang - 25 master trainers trained and working in -Master trainers trained and working in Diets; numbersnot DIETSand BRCs clear e. Strengthen DSERT capacityfor educational - At least 15research/evaluationstudies -No data research and evaluation by creating a network commissioned by KPSVY completed involving of research agencies at least 5 institutions and supportresearch in universities. - DSERT research coordinationcapacity - No data strengthened:Databaseof research studies functioningin DSERT -Universityeducationalresearch output - No data expanded: 10doctoral studies completed 64 Annex C Project Objective Outcomes/Output indicator Target End of Project Status f. Establishnetwork of institutionsunder the -Materialsdeveloped, tested and revised of good -Trainingmaterials developed, tested and in use. DIET and coordinationof DSERRT to developand quality and in use. Five initial &refresher trng BRCICRCpersonnel trained by SCERT implementmanagement training progs for 25 master trainerscompleted:for admin staff, headteachersand VEC members - Tmgprgms provided to district and block education staff and trainers,heads of DIETSand pnmary tchrs tmg instit.,headtchrs & VEC members (tot 2000 persons, 50 courses - Ten research studiescompleted;3 presented at national conferences -Research studiescover new cunicular materials, school developmentand monitoringcommittees,capacity building needs, and DIET evaluation; not clear whether number of - Leaming achievementstudies completed in 3rd studiesreached target &6thyear ofprojectin 5 districts;studiesin -Statecoordinationof DPEP learning achievementstudies in additional districtsjoining theprogram project districts; Dishict Level a. District Proj Sub-Teams established - Established in all districts b. Supportfor district/school level -Planningand management - Teams establishedand trainingprovided -Teacher insewice training - Districtnetworks set up by state agencies,consistingof DIET, BRC and CRC: varying degrees of readiness and effectiveness -Developmentof teaching aidss6 - Beginningwith DPEP 11,teachers have received an annual grant of Rs 500, which most have used for buying or making teachingaids. -Programevaluation - EMIS built and student achievementsurveys administered, but little attention to institutionaldevelopmentindicators 5. Enhancing Community Percent of villageshaving fully functional 50percentof project villages have fully Participationin Primary Education Village Education Committees functionalVillage Education Committees 3. Project document says "learning materials" 65 Annex D ANNEX D: PERSONS AND ORGANIZATIONSCONSULTED Ministrv of Human Resource Development Vrinda Sarup, Joint Secretary,Department of ElementaryEducation and Literacy Dhir Jhingrn, Director R. Meena, Deputy Secretary R. S. Pandey, Former Head of DPEP Bureau (1995-1999) National Center of Education Research and Training (NCERT] Krishna Kumar, Director Ved Prakash, Former Director (now Secretary for University Grants Commission) National Institute of EducationalPlanning and Administration (NIEPA) M. Mukhopadhyay,Head Arun Mehta, Fellow, ORSM Unit SMIA Zaide, Fellow and In Charge,Educational Planning Unit Prancila Menon, Fellow and In Charge, Subnational SystemsUnit R. Govinda, Senior Fellow and Head, School and Nonforrnal Education Unit World Bank Office Venita Kaul, Senior Program Officer N.K. Jangira, Consultant(previously Head, Department of Teacher Education and Special Education, NCERT Non-Government Organizations Rukmini Bannerji, Pratham KARNATAKA (State) T.M. Vijay Bhashar, Secretaryto Government,Education Department (Primary and Secondary Education) Sanjiv Kumar, Commissionerfor Public Instruction Raj Kumar Khatri, StateProject Director, SSA D. Jagannatha Rao, Director,Departmentof State Education Research and Training (DSERT) K. Gurumurthy,Team Leader, Policy Planning Unit, DSEERT Azim Premji Foundation R.V. Vaidyanatha Ayyar, VisitingProfessor, Indian Institute of Management Bangalore; Formerly Joint Secretary,Department of Elementary Education and Literacy Annex D Field Visits Kolar District - DistrictProject Office - District Institute of Education and Training - Block Resource Center in Kolar town - GovernmentHigher Primary School: Patna - Head teacher, teachers, students,and School Developmentand Monitoring Committee - GovernmentLower Pnmary School: Pakarahally - Head teacher, teachers, students,and School Development and Monitoring Committee TurnkurDistrict - DistrictProject Office - District Institute of Education and Training - Block Resource Center in Gubbi - Cluster Resource Center in Hebbur - Govemment Lower Primary School:Heggere - Met with head teacher, teachers, students, and School Development and Monitoring Committee - GovernmentLower Primary School:Mudigere - Met with head teacher, teachers, students - Govemment Higher Primary School: Gulur - Met with head teacher, teachers, students,and SchoolDevelopment and Monitoring Committee UTTARPRADESH (State) J.S. Deepak, State Project Director, SSA and SecretaryUPEFA Partha Sarthi Sen Sharma, Additional State Project Director, UPEFA S.C. Srivastava,Additional Project Director Sri Binod Singh,Administrative Officer Krishna Mohan Tripathi, Director, State Institute of Education Management and Training Najma Saxena, Researcher, SIEMAT Neeta Mathur, Principal, English Language Teaching Institute, State Council of Educational Researchand Training Field Visits Saharanpur and Muzafarnagar Districts - District Project Officers - DIET (Saharanpurand Muzafarnagar) - Visit to GovernmentHigher Primary School:Manaharpur (Saharanpur District) - Met with head teacher, teachers, studentsand Village Education Committee - Visit to GovernmentLower Primary School: Barta (MuzafarnagarDistrict) - Mete with head teacher, teachers, students and Village Education Committee Annex D Hardoi District - District Project Office - District Institute of Education and Training - Block Resource Center at Sursa - ClusterResource Center in Ahirohi Block - ClusterResource Center in Sandila Block - Government Lower Primary School:Begumgunj - Met W/head teacher, teachers, students (Village Education Com'tee not active) - Government Lower Primary School:Khausikheda - Met W/head teacher, teachers, students (Village Educ. Com'tee not available) WASHINGTON, D.C. John Middleton, former World Bank Task Team Leader, UPBEP Ward Heneveld, former World Bank Task Team Leader, DPEP Sajitha Bashir, Senior Education Economist,World Bank and former Technical Support Group Researcher (GOI) Prema Clarke, former World Bank Task Team Leader, DPEP Susan Hirshberg, former World Bank Task Team Leader, DPEP Kin Bing Wu, Lead Education Specialist (World Bank), and former SSA Task Team Leader Robert Prouty, Lead Education Specialist,World Bank