Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Secondary Education Program (P118445) Report Number : ICRR0021084 1. Project Data Project ID Project Name P118445 IN: Secondary Education Program Country Practice Area(Lead) India Education L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-50880,IDA_-50880 30-Jun-2017 12,896,000,000.00 Bank Approval Date Closing Date (Actual) 22-Mar-2012 30-Jun-2017 IBRD/IDA (USD) Grants (USD) Original Commitment 500,000,000.00 0.00 Revised Commitment 500,000,000.00 0.00 Actual 475,874,310.87 0.00 Prepared by Reviewed by ICR Review Coordinator Group Judith Hahn Gaubatz Judyth L. Twigg Joy Behrens IEGHC (Unit 2) 2. Project Objectives and Components a. Objectives According to the Project Appraisal Document (PAD, page 3) and the Financing Agreement (page 5), the project objective was as follows: • To help India to achieve increased and more equitable access to good quality secondary education through support of the Government‘s ongoing program for secondary education as delineated in the Rashtriya Madhyamik Shiksha Abhiyan (RMSA) Framework. Page 1 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Secondary Education Program (P118445) b. Were the project objectives/key associated outcome targets revised during implementation? No PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? No d. Components The project was designed to provide support to the government's entire RMSA Framework through a Sector-Wide Approach (SWAp), and therefore there were no explicit project components (and no separate cost estimates). However, the RMSA Framework included the following parts: Part 1: Improving the Quality of Secondary Education: Activities included: recruitment of additional teachers, in-service teacher training, capacity development of local institutions to support quality improvements, provision of core infrastructure and learning materials for secondary schools, and development of a student assessment system. Part 2: Improving Equitable Access to Secondary Education: Activities included: Construction and upgrading of secondary schools, girls' hostels in educationally backward blocks, and separate toilet facilities for girls and boys. Part 3: Improving Accountability of Secondary Education: Activities included: Strengthening of School Management and Development Committees. Part 4: Developing Innovative Approaches to Secondary Education: Activities included: Sub-projects to pilot or scale up innovations to improve secondary education. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost • The estimated project cost was US$ 12,896.0 million. The actual project cost was US$ 5,029.9 million. See below for reasons for the significant decrease in total project cost. • Part 4 (Developing Innovative Approaches) was not implemented due to low counterpart fund availability. Financing • The Bank provided support by way of a Sector-Wide Approach (SWAp) and therefore supported the Page 2 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Secondary Education Program (P118445) government's sector program as a whole, rather than specifying Bank support for certain activities or components. The estimated International Development Association financing was US$ 500.0 million, of which US$ 475.9 million disbursed. This difference was in part due to exchange rate fluctuations and the loss in US dollars. • The UK Department for International Development was expected to provide US$ 90.0 million in co- financing; US$ 94.0 million was actually disbursed. Borrower Contribution • The Government was expected to provide US$ 12,306.0 million for the RMSA Framework. The actual contribution was US$ 4,460.0 million. The ICR (page 9) noted the following reasons for the lower than appraised contribution: the appraised amount included a plan to extend the RMSA Framework to grades 11-12 and to cover all government and government-aided schools, but this expansion did not take place; the appraised amount was based on the Ministry of Human Resource Development budget request, but the actual budget amount approved by the Ministry of Finance was lower (62% of requested budget amount); the fund-sharing ratio between central and state governments was reduced from 75:25 to 60:40. Dates • June 2014: The project underwent a Level 2 restructuring to modify financial management arrangements, in order to facilitate withdrawals from the Credit. Changes included: (a) eliminating the requirement of a consolidated report on audits summarizing all the state reports in the national program supported by the project; and (b) eliminating the requirement of a consolidated report for the previous financial year before a withdrawal is made for a given financial year. • June 2017: The project closed as planned in June 2017. 3. Relevance of Objectives Rationale Recent economic growth in India has been driven by sectors that are skills-intensive, and thus completion of secondary education was becoming an essential requirement for productive employment. As noted in the PAD (page 1), employers were increasingly seeking skilled workers, and demand for relevant education and training at the secondary level were also increasing; however, supply was not keping up. The strides made in elementary level enrollment (net enrollment had reached 96%) also increased pressure on the secondary level. However, the secondary gross enrollment rate was 50%, with inequities in access for poor and rural populations, girls, and socially disadvantaged groups (i.e. Scheduled Caste and Scheduled Tribe). Physical access was a significant barrier, as one-third of villages were located more than 5 km from a lower Page 3 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Secondary Education Program (P118445) secondary school. Large class size (above the norm of 40 students per class) due to lack of available infrastructure was also a significant barrier to quality. Quality of education was also a challenge, with recent small-scale assessments indicating low learning outcomes. The government of India, in recognition of these multiple challenges to secondary education, launched a centrally sponsored program in 2009, called the Rashtriya Madhyamik Shiksha Abhiyan (RMSA) Framework. The RMSA aimed to provide universal access to secondary education by 2017, with universal retention/completion by 2020. The project objective was directly relevant to and supportive of this overarching government program. It was also highly consistent with the Bank’s Country Partnership Strategy for FY2013-17. Key outcomes included improved market-driven skills for productive employment, and improved access and quality of education, including for excluded groups. Rating High 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective To help India to achieve increased and more equitable access to good quality secondary education through support of the Government‘s ongoing program for secondary education as delineated in the Rashtriya Madhyamik Shiksha Abhiyan (RMSA) Framework - ACCESS Rationale The theory of change and results framework are overall sound, with planned outputs (as outlined by the elements of the RMSA Framework) likely to lead to the intended outcomes of increased access, equity, and quality of secondary education. The addition of school infrastructure and teachers was likely to increase access; interventions to reach girls and disadvantaged populations were likely to increase equity; improvements in learning materials and qualifications of teachers and leaders, as well as measures to increase accountability, were likely to improve quality. However, attribution to Bank support was unclear given the sector-wide approach and the relatively small contribution of Bank funding. Also, a notable shortcoming in evidence to support the story of achievement in this project was the lack of accurate baseline data for net enrollment rate vs. gross enrollment rate, as noted by the ICR (page 33): although the net enrollment rate is generally understood to be lower than gross enrollment rates in countries where access and repetition are significant issues, both net enrollment and gross enrollment rates are reported as 58.5% in 2009/10 (based on the 2001 Census), whereas project supervision reports suggest that, according to updated Census data, the net enrollment rate baseline was 45.6%. Outputs •Construction of 12,682 new secondary school facilities, of which 11,310 had become functional as of the end of the project and had a total enrollment of 1.47 million students. According to the PAD (page 25), Page 4 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Secondary Education Program (P118445) these schools were to be mainly in underserved areas, based on school mapping exercises and data from the education management information system. • Construction of 50,713 additional classrooms in existing secondary schools, of which 31,315 had been completed. • The ICR did not report the number of teachers who were recruited; however, the project team subsequently reported that 76,297 teachers were recruited from 2009-2017. (The PAD, page 3, noted that about 500,000 additional teachers were needed, and that the project expected to support hiring of 100,000.) Outcomes • Total enrollment in secondary education increased from 28.3 million in 2009 to 39.0 million in 2017, falling short of the target of 40.3 million. • The gross enrollment rate for secondary school increased from 58.5% in 2010 to 80% in 2017, falling short of the target of 87.4%. 16 states/territories, including some low-income states, saw an average increase of 25%. Of note, the RMSA aimed at milestones of 75% gross enrollment by 2013 and 100% by 2017. • The net enrollment rate for secondary education increased from 45.6% in 2013 to 51.6% in 2015/16. The ICR reported that the target (as well as the original baseline figure) was inaccurate. • The gross access ratio (percentage of habitations having a secondary school or section within 5 km) increased from 68.2% in 2011/12 to 87.5% in 2016/17. Given that the outcome-level targets for this objective were almost fully met, achievement of this objective is considered Substantial. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 2 Objective To help India to achieve increased and more equitable access to good quality secondary education through support of the Government‘s ongoing program for secondary education as delineated in the Rashtriya Madhyamik Shiksha Abhiyan (RMSA) Framework - EQUITY Rationale Outputs • Construction of 5,028 new schools and upgrading of 10,421 existing schools in Special Focus Districts Page 5 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Secondary Education Program (P118445) (those with students from disadvantaged social categories). • Construction of 1,148 girls’ hostels, with a total enrollment of 103,969 girls. • Provision of school activities to support increased participation and performance of girls, including stipends, provision of toilet facilities, self-defense training in martial arts, adolescent well-being programs, career guidance, and motivational camps. • Development of gender sensitization modules for teacher training and gender awareness guidelines for education officials. • The project team noted that central and state governments provided scholarships for students from disadvantaged groups and with special needs, in order to address affordability barriers. Outcomes • The Gender Parity Index for gross enrollment increased from 0.94 in 2010 to 1.03 in 2016/17, surpassing the target of 0.98. • The number of girls per 100 boys completing grade 10 improved from 79% in 2010 to 91.2% in 2017, falling short of the target of 93%. • The proportion of secondary completers from Scheduled Caste (SC) groups increased from 16.8% in 2010 to 18.4% in 2017, achieving the target of 18.3%. The gross enrollment rate for SC girls was 86.9%, and 83.9% for SC boys. • The proportion of secondary completers from Scheduled Tribe (ST) groups increased from 6.0% in 2010 to 7.7% in 2017, achieving the target of 7.4%. The gross enrollment rate for ST girls was 75.38%, and 73.7% for ST boys. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 3 Objective To help India to achieve increased and more equitable access to good quality secondary education through support of the Government‘s ongoing program for secondary education as delineated in the Rashtriya Madhyamik Shiksha Abhiyan (RMSA) Framework - QUALITY Rationale Outputs • Development of national system of student assessment at the secondary level, capable of tracking progress in learning achievements over time. • Development of national curriculum standards. Page 6 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Secondary Education Program (P118445) • Remedial education for 2,241,800 students to enable them to pass into 9th grade. • Construction of core infrastructure in secondary schools, including 14,424 toilet blocks, 8,647 drinking water facilities, 16,328 science labs, 11,295 computer labs, 15,989 libraries, and 17,900 art/craft/culture rooms. • Development of in-service training modules for core subjects. The ICR did not report the number of teachers receiving training; however, the project team subsequently reported that over 2.0 million teachers, cumulatively, participated in in-service training from 2009-2017. • Training for 79% of all School Development Management Committees to improve local accountability. • Training of 3,526 State Resource Group members and 26,332 secondary school principals in leadership. The RMSA component to fund innovative activities was dropped. Outcomes Instruction and school environment • The proportion of teachers with professional qualification (B.Ed. or equivalent or above) increased from 83% in 2010 to 87% in 2017, achieving the target of 87%. • The proportion of schools with specific teachers available to teach the five core subjects (math, science, social studies, English, and Hindi) increased from 62.2%in 2009 to 87.7% in 2017 (no target provided). However, the proportion of schools with all subject-specific teachers available increased only from 21% to 24.6%, falling far short of the target of 40%. • The teacher vacancy rate decreased from 26.0% in 2012/13 to 14.8% in 2016/17. • The student-classroom ratio improved from 56 in 2010/11 to 46 in 2015/16. Learning outcomes • The gross secondary completion rate increased from 55.4% in 2009/10 to 64.5% in 2016/17. There was no target provided. • The graduation rate (the proportion of students appearing for grade 10 exams in relation to the number of students enrolled in grade 9 in the previous year) decreased from 74% in 2010 to 69% in 2017, falling short of the target of 88%. The ICR (page 16) suggested that a decline could indicate more students dropping out or leaving the public school system for other options (for example, open schools/vocational training) or more students being unprepared for taking the grade 10 board examination or being perceived to be unprepared by teachers. It is also possible that a large number of students from disadvantaged groups entered the secondary education system, which may have negatively affected the graduation rate. • Prior to the project, the country had no reliable national learning assessment available at the secondary level. With the project’s contribution, learning achievements in 10th grade were measured for the first time in 2014/15, with a sample of 277,000 students in 7,216 schools across 34 states/territories. The national report and state summary reports were disseminated, providing information on various remediation Page 7 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Secondary Education Program (P118445) strategies and leading to development of learning improvement plans in several states. The second round of assessment was not conducted during the project period as planned, but preparation is underway and is likely to be conducted in 2018. Rating Modest PHREVDELTBL PHOVRLEFFRATTBL Rationale There were substantial achievements in (i) increasing access, as reflected by increased total enrollment and gross access ratio (although increases in enrollment rates are unclear due to some inaccurate baseline and target figures); and (ii) increasing equity among girls and socially disadvantaged groups. However, there were shortcomings in improving quality, as reflected by outcomes on graduation rate and availability of teachers for all subjects. Attribution of the achievements observed to Bank financing is not clear, as the Bank's financial contribution comprised approximately 9.5% of total secondary education spending during this time period. On balance, however, achievement of objectives is considered substantial. Overall Efficacy Rating Substantial 5. Efficiency The PAD (Annex 7) presented a cost-benefit analysis for the project activities. Costs were calculated as the project costs, the direct costs of schooling incurred by households, and the opportunity costs of earnings forgone by children going to school rather than working. Benefits were calculated as increased additional earnings (for 15 years after project closing) for those who enter the labor market with secondary education, and increased employment rates. The stated assumptions included an increase in the lower secondary gross enrollment rate to 72% and in the lower secondary gross completion rate to 76%, private cost of education at INR 3,465, and a labor force participation rate ranging from 65% to 75%. The analysis resulted in an internal rate of return (IRR) of 24.25% and a net present value (NPV) of benefits of $14 billion (assuming a discount rate of 12 percent). A sensitivity analysis using the most conservative estimates of costs and benefits indicated an IRR in the range of 18.1% to 36.5%. According to the ICR (Annex 4), the cost-benefit analysis using the same categories of cost and benefits but actual figures and more conservative assumptions (secondary gross enrollment rate of 60.5%, secondary gross completion rate of 56.5%, private costs of education at INR 6,788, and a discount rate of 8%) found an IRR of 16.1% and a NPV of US$3.5 billion in the base case scenario. These may be considered positive returns, although lower than the estimates at appraisal. Page 8 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Secondary Education Program (P118445) The ICR (page 17) pointed to various efficiency gains in implementation, such as the level of project achievement despite lower counterpart support, the use of information technology to streamline administrative and monitoring processes, improved fiduciary procedures to benefit fund flows and budgeting, leveraging of local expertise, and knowledge sharing to promote best practices. In addition, the joint review approach by donors likely contributed to reduced transaction costs. However, there were some initial bottlenecks in fiduciary processes, and another notable shortcoming was a shortage of management personnel at the state and district levels (ICR, page 21) that contributed to implementation and program quality challenges. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 100.00 Appraisal  24.25 Not Applicable 100.00 ICR Estimate  16.10 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of the project objective is rated High. Achievement of the objective to achieve increased and more equitable access to good quality secondary education (Efficacy) is rated Substantial due to significant achievements in (i) increasing access, as reflected by increased total enrollment and gross access ratio (although increases in enrollment rates are unclear due to some inaccurate baseline and target figures), and (ii) increasing equity among girls and socially disadvantaged groups. However, there were shortcomings in improving quality, as reflected by outcomes on graduation rate and availability of teachers for all subjects. Efficiency is rated Substantial due to positive estimated returns (as reflected by the IRR and NPV analysis) as well as other indications of implementation efficiency. Taken together, these ratings indicate that there were only minor shortcomings in the project's preparation and implementation, producing an Outcome rating of Satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome Page 9 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Secondary Education Program (P118445) The project was strongly aligned with the national government’s priorities and program for the secondary education sector, and therefore government commitment at the central level is likely to continue. The ICR (page 19) reported that the project has generated substantial international interest in India’s secondary education sector and a large amount of high-quality research to provide inputs into the sector program. Institutional capacity has also been bolstered, including in planning and budget management processes, the establishment of decentralized project offices, and training for school management committees. However, commitment and capacity at the state level is more varied and could pose a challenge to sustaining results. 8. Assessment of Bank Performance a. Quality-at-Entry The strong alignment of the project objective with government priorities and the central-level sponsorship of the program are noted as significant positive factors in ensuring strong government commitment at multiple levels. The SWAp instrument contributed to harmonization of donor processes, including monitoring, and likely reduction in transaction costs; the ICR (page 20) also noted that it facilitated a focus on sector-wide results among partners and the government. Lessons drawn from prior experience were discussed in the PAD; however, two salient lessons (the importance of increasing effectiveness of teachers in improving educational quality; the importance of strengthening the role of local bodies in school management) were not given due attention in the project design. The overall risk was assessed as Substantial, due to potential for corruption in fiduciary matters and low fiduciary capacity among several states. Stringent fiduciary monitoring, including the use of third parties, was an effective mitigation measure for the former risk, though not for the latter as financial management remained a challenge throughout the project period. The indicators for the results framework were overall sound; however, inaccurate/unidentified baseline and target figures became apparent due to data limitations in the government’s existing monitoring system. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision Joint Review Missions (JRMs) were a key feature of supervision, given the SWAp modality. The ICR (page 27) noted that JRMs were well planned, with experts in specific areas providing reviews and recommendations, facilitating the sharing of good practices, and maximizing the impact of supervision. In particular, donor participation expanded the range of just-in-time technical inputs into the supervision process. The field-based location of the task team leader and core team members also ensured ongoing dialogue. Initial fiduciary bottlenecks were addressed by project restructuring, although financial management has remained challenging, with overall performance rated Moderately Unsatisfactory (see below). In addition, shortcomings in the results framework (inaccurate baseline and lack of target figures) were not Page 10 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Secondary Education Program (P118445) rectified, with persistent underlying issues in the accuracy and timeliness of data from the government's education management information system.. Discussions on restructuring with the government were initiated by the Bank following the midterm review, but the restructuring did not ultimately occur due to delays in government processes. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The results framework and project indicators were overall sound. The choice of indicators was appropriate and wide-ranging, given the broad nature of the project objective. The M&E arrangements were embedded in the government's existing monitoring systems, which aided in strengthening the government's focus on M&E. The M&E design also included some measures to ensure quality of M&E data, such as regular review of the data at all levels and support to the implementing agencies to strengthen data collection. However, one apparent drawback of relying on the government's system was the insufficient quality of the baseline data. This led to inaccurate baseline and target figures for several indicators, a shortcoming that was not adequately addressed during the project period. b. M&E Implementation Data collection and monitoring were carried out as planned. An official database for key education indicators and an online project management system were established. The government also instituted an external monitoring system by contracting various institutions (non-governmental organizations, state universities, and foundations) to visit a sample of schools and monitor program implementation. The key activity of the National Assessment for Grade 10 was conducted for one round, with the second round expected to be conducted in the near term. Several research studies were also carried out, including classroom process studies in Meghalaya, Uttar Pradesh, and Rajasthan; curriculum studies; time-on-task studies in Tamil Nadu and Madhya Pradesh; a teacher study on teachers in nine states; a study on the impact of teacher training on the quality of teaching in secondary education; a study on the dropout factors in secondary education; a study on the impact of self-defense training in schools; and a study on the impact of vocational education. The midterm review, which was conducted in September 2015, highlighted the weaknesses of the Results Framework and recommended updating and/or modifying some targets through restructuring. Specifically, it recommended: (a) dropping the two intermediate indicators on innovations as there was no progress on activities (‘number of innovative activities approved’ and ‘25 percent of the completed innovation component projects that meet at least 75 percent of their agreed targets’); (b) modifying the target values for the Page 11 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Secondary Education Program (P118445) indicators that were far behind and not on track to be achieved by the end of the project (‘gross graduation rate’, ‘NER’, and ‘proportion of schools with all subject-specific teachers available in required number’); (c) setting the baseline values for indicators that were not available at appraisal (‘gross secondary completion rate’, ‘proportion of schools with all the main infrastructure available’, ‘proportion of schools with all subject- specific teachers available in the required number’, ‘proportion of SMDCs receiving capacity/management training’, and ‘all staff positions at the state, district, and sub-district levels filled’); and (d) setting the target values for indicators that were not determined at appraisal (‘proportion of SMDCs receiving capacity/management training’ and ‘all staff positions at the state, district, and sub-district levels filled’). However, as noted previously, the restructuring did not ultimately occur, and the shortcomings in the results framework remained. c. M&E Utilization The ICR (page 24) reported several examples of M&E utilization. The online project management system was used for enhanced efficiency in the management and implementation of the RMSA, as comprehensive information from states regarding their physical and financial progress on all the RMSA components was captured and progress was monitored. The official education indicators database was used to monitor key school-, teacher-, and student-related indicators and led to remedial actions being taken by the states. The results of the first round of the National Assessment were analyzed, and the summary report and state report cards were made available online. M&E Quality Rating Modest 10. Other Issues a. Safeguards The project was classified as an Environmental Category “B” project due to potential school construction/rehabilitation and triggered the safeguard policy on Environmental Assessment (OP/BP 4.01). An Environmental Management Framework , which laid out principles of conservation of energy and natural resources, improving air quality, employing sustainable purchasing and green cleaning practices, as well as decreasing the burden on municipal water and wastewater treatment, was under development during project preparation. However, according to the ICR (page 25), due to significant delays in updating the environmental assessment, the Framework could not be completed, and therefore environmental management was initially assessed as Moderately Unsatisfactory. The rating was subsequently changed to Moderately Satisfactory when the assessment was updated and the Framework was finalized. The safeguard policy on Indigenous Peoples (OP/BP 4.10) was also triggered, due to project implementation in tribal areas. To ensure that program benefits reached tribal peoples, an Equity Action Plan was prepared, and the project management information system was expected to collect data on all disadvantaged groups. Page 12 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Secondary Education Program (P118445) The ICR (page 25) repored that a Bank safeguards assessment was conducted, and no issues were raised during the project period regarding compliance. b. Fiduciary Compliance Financial management: The ICR (page 19) reported that the project contributed to the development of better fiduciary practices, including financial management (FM) processes, FM manuals, and FM training programs. These practices were implemented for the whole RMSA program, not just for Bank activities. FM performance was rated Moderately Unsatisfactory from September 2014 onwards due to delayed fund releases and inadequate quality of audit reports. Following a joint review, recommendations were made to strengthen capacity of auditors and state accountants, particularly with regards to the quality of audit reports; however, sufficient actions were not taken during project implementation, and the FM rating at project closing remained at Moderately Unsatisfactory. Procurement: Procurement responsibilities was handled at all three levels:national, state and district. Performance was initially rated Moderately Satisfactory due to slow progress of civil works and delays in the post-procurement review of contracts; however, the procurement rating was subsequently upgraded to Satisfactory after the efficiency of procurement implementation and management was improved. The ICR (page 26) noted the following procurement measures to support implementation: in most of the states, the e- tendering process was followed for all major procurements above a certain threshold; the central Ministry cancelled civil works pending in 2015–2016 and sanctioned fresh works based on a revised schedule of rates in the states, which improved the progress rate of new school construction and promoted better use of the limited resources; and the Ministry made the decision to release funds to states in advance, which helped states to finalize civil works contracts expeditiously. c. Unintended impacts (Positive or Negative) None reported. d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory --- Page 13 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Secondary Education Program (P118445) Moderately Moderately Bank Performance --- Satisfactory Satisfactory Shortcomings in the results framework (inaccurate Quality of M&E Substantial Modest baseline and target figures) were not sufficiently addressed. Quality of ICR Substantial --- 12. Lessons Lessons drawn from the ICR and adapted by IEG: • The sector-wide approach (SWAp) can be an effective modality, assuming that strong central government commitment trickles down to decentralized levels (and low fiduciary capacity is mitigated at the decentralized level). In this case, the SWAp instrument contributed to harmonization of donor processes, including monitoring, and likely reduction in transaction costs, as well as a focus on sector-wide results among partners and the government. • Embedding M&E arrangements in the government's existing systems can bolster government's focus on robust M&E. However, the quality of the data produced by those systems needs to be sufficient to ensure accurate baseline and target data. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR was thorough, given the wide-ranging nature of the project. The quality of the evidence was overall sufficient for assessing the multiple dimensions of the project objective (i.e. access, equity, quality), although the analysis minimally addresses the issue of attribution to Bank support and lacks some candor in reporting achievements. [For example, the Annex 1 table of results indicators (ICR, p. 34) states that the proportion of schools with all subject-specific teachers available in required number reached 24.6% at completion, compared with 21% at baseline, and compared with a target of 40%, with the "Comments" associated with this indicator stating "Not achieved (62% achieved)" (ICR, 35). The "not achieved" portion of the comment is factual, but the parenthetical "62% achieved" might imply that the proportion achieved was 32.78% (because 32.78% sits at a point that is 62% of the distance between the baseline of 21% and the target of 40%), when the actual proportion achieved was 24.6%.] The Lessons Learned section would have more impact if the information were more clearly framed as "lessons" to be applied to future operations, rather than as a summary of project experiences. Page 14 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Secondary Education Program (P118445) a. Quality of ICR Rating Substantial Page 15 of 15