ICRR 11790 Report Number : ICRR11790 ICR Review Operations Evaluation Department 1. Project Data: Date Posted : 06/24/2004 PROJ ID : P036952 Appraisal Actual Project Name : Cn-basic Ed. IV Project Costs 165.2 174.81 US$M ) (US$M) Country : China Loan/ US$M ) 85 Loan /Credit (US$M) 82.68 Sector (s): Board: ED - Primary Cofinancing education (44%), US$M ) (US$M) Secondary education (43%), Tertiary education (12%), Central government administration (1%) L/C Number : C2954 Board Approval 97 FY ) (FY) Partners involved : Closing Date 06/30/2003 09/30/2003 Prepared by : Reviewed by : Group Manager : Group : H. Dean Nielsen Ridley Nelson Alain A. Barbu OEDST 2. Project Objectives and Components a. Objectives The main project objectives were : 1) to increase access/equity in primary and junior secondary education for the absolute poor in poor and minority rural areas to achieve universal primary education and expand junior secondary education, with particular attention to girls education and minority nationalities; 2) to enhance quality in primary and junior secondary schooling and corresponding teacher education and training of trainers' programs; and 3) to improve the efficiency in the delivery of education by improving educational management at the national, provincial, county and project institutional levels . b. Components 1. Access to nine-year compulsory education (SAR: $114.26 m, excluding contingencies; actual : $154.05 m); 2. Enhancement of the quality of education (SAR: $14.74 m, excluding contingencies; actual $ 19.21 m); 3. Improvements in education management (SAR: $1.27 m, excluding contingencies ; actual $1.56 m) c. Comments on Project Cost, Financing and Dates The overall cost of the project at appraisal was $ 165.2 m of which $85 m was to be covered by IDA credits . The project disbursed all of the original IDA special drawing rights (SDR), but due to exchange rate fluctuations dollar amounts disbursed were $82.68, about $2.3 m below the dollar amount calculated at appraisal . Nevertheless, total disbursement in the project ($174.81 m ) was well above the appraisal amount, owing to counterpart contributions almost $12 m above appraisal amounts. IDA credit disbursement was generally ahead of schedule, with 80% being disbursed by the mid-term review, but there were significant delays in reimbursements within the project to the provincial and county levels . Also, some discrepancies in the ways of accounting for reimbursements across parts of the government (MOE and MOF) needed to be reconciled at the end of the project leading to a closing date extension of three months. 3. Achievement of Relevant Objectives: 1. Concerning increased access and equity (achieving universal primary education and expanded junior secondary education; improved access for the absolute poor, girls and minorities ), the government reported that primary school enrollment rates (presumably "net enrollment ratios") increased from 92 to 98 percent; junior secondary increased from 75 to 90 percent. With enrollment rates near 100% for primary education it is assumed that access for the absolute poor increased, but this was not clearly documented, neither did the ICR text show enrollment increases for girls and minorities, however, Annex 1 data made it clear that both groups reached near full enrollment in primary education (average 98% over the 9 provinces) and achieved substantial gains (over 20%) in junior secondary education in the six provinces where this was a goal . 2. Quality improvement was demonstrated by the proportion of students passing the 6th and 9th grade exams (leaving exams?), which in all cases showed significant increases on the average (at the primary level: 82 to 93 percent in mathematics and 78 to 88 percent in Chinese; and at the junior secondary : 75 to 89 mathematics and 68 to 80 Chinese); and the proportion of teachers who are "qualified," which also increased over the life of the project from 78 to 94 percent for primary and 65 to 88 for junior secondary. It is assumed that the learning improvements are attributable to better school facilities, increased supplies of books (including those in minority languages ) and teaching equipment, and the improved qualifications of teachers (as above) in project schools; however, lacking any control data the possibility of system -wide improvement in learning outcomes (perhaps due to easier examinations or score inflation), cannot be ruled out. 3. Efficiency improvement was only briefly mentioned in the ICR outcomes section : namely that an effective management system for the delivery of educational services at the local level had been institutionalized and that the results of provincial-based results had been disseminated to project schools . The ICR project component review covering the management improvement provides a more in -depth assessment of efficiency outcomes . It notes significant improvements in staff and institutional capacity (to the point that Project staff were used in Province -wide compulsory education programs and even outside the education sector ), the establishment of productive Education Management Information Systems at the provincial, regional and municipal levels, and the constructive contributions of the "Chinese Experts Panel" to policy development and implementation effectiveness . 4. Significant Outcomes/Impacts: 1. The Project accelerated the attainment of universal primary education (in some cases by two years ) in 115 of 117 counties and 9 years of compulsory education in most project countries, while at the same time increasing student learning outcomes; 2. The delivery of project inputs (school construction, books and teaching materials, training, and management improvements) were according to plan, in fact, in most cased exceeded planned amounts . 3. A demand-side program of providing financial aid to students from poor families in support of their educational participation exceeded target numbers and appeared to be successful . When project funding proved to be unequal to the demand, local government allocated funds from recurrent budgets and social donations to cover it, making the sustainability of the program more likely . 4. The Project set a new standard for capacity building at the provincial level through continuous training and reforming management procedures. These standards have been emulated by other units in and outside of the Ministry of Education. 5. The Project developed school maintenance arrangements for its new buildings such that maintenance will be assured during at least a five year post -project period. 6. The Project demonstrated how effective school access and quality improvement efforts can be embedded within a more general regional poverty reduction program (the Qinba Mountains Poverty Reduction Project, the context for three of the Projects nine provinces ). 5. Significant Shortcomings (including non-compliance with safeguard policies): 1. There was a tendency for this Project to focus most of its attention on the delivery of inputs instead of the attainment of project outcomes; outcomes (such as student learning gains ) were still positive, but might have been even more so if, for example, teacher training had been more connected to the real needs of teachers and students instead of adopting conventional output targets . 2. The Project supported the establishment of a well functioning management information system, but, as in many similar cases world-wide, there was very limited use of the results produced in local decision -making and system improvement. It was therefore not a very efficient investment . 3. The M&E design of the Project did not allow it to demonstrate development impact, which might have been done had it measured access and quality improvement outcomes in comparisons schools and counties; a related issue is the limited description of the student leaving examination and apparent nonapplication of measures to rule out biases in results due to possible manipulation of test scores or grade inflation . 4. The Project had the potential to demonstrate the effectiveness and desirability of cross -sector programming, since service delivery in three of the nine provinces was embedded in a regional poverty alleviation program . For example, Project evaluation could have demonstrated the benefits and drawbacks of such cross -sectoral programs in reaching the poor, compared to those under the more conventional, single sector approach . The ICR is mostly silent on how implementation and results differed under the two approaches . 6. Ratings : ICR OED Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Institutional Dev .: Substantial Substantial Sustainability : Likely Likely Bank Performance : Satisfactory Satisfactory Borrower Perf .: Satisfactory Satisfactory Quality of ICR : Satisfactory NOTE: NOTE ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness. 7. Lessons of Broad Applicability: 1. Demand-side approaches (targeted subsidies) to increased basic education participation among the poor can be effective; they are particularly effective when local commitment to them is strong enough to generate funding from recurrent budgets or social contributions. 2. In poverty reduction schemes which include basic education participation, it is good to rely on the local educational infrastructure for the planning and execution of the education part, given sector's professional resources and village level connections, but this does not mean that the sector should work in isolation . Connections to the other parts (sectors) in the scheme should be maintained and evaluated. 3. When programs of increasing access are planned and managed well, there need not be a trade off between access and quality. 4. A mix of solid policies, management capacity building, flexibility in using resources and political will can accelerate a country's attainment of universal primary and lower secondary education . 5. Ministries of Education having bold programs of educational attainment and quality improvement would do well to build in measures that would allow for impact assessment . This would entail either setting up a controlled experiment or a quasi-experimental design (in which project schools are compared to those of a control group ). 8. Assessment Recommended? Yes No Why? The project is a good example of expanding access without sacrificing quality and in the use of targeted student subsidies for reaching the very poor . There are also potential lessons to be learned about the relative strengths and weaknesses of reaching the poor through an integrated, cross -sectoral program compared to a single sector one. 9. Comments on Quality of ICR: The ICR was generally well organized and effective in marshalling evidence concerning project effectiveness, institution building and sustainability; however, it could have provided a more organized and focused treatment of outcomes in relation to objectives; some explanations of discrepancies between baseline and target data in the SAR (annex 8) and Annex 1 of the ICR; some comments as to the quality and potential biases in the learning outcomes data; better description of enrollment data (are the figures net enrollment rates? ) and comments on biases in these data (a concern brought on by the passing reference to a possibility of enrolment "exaggeration"); some confusion in the discussion of student assistance numbers and amounts; and the listing of lessons learned (the first 4 bullet points) for which there is inadequate background information in the ICR .