ICRR 11256 Report Number : ICRR11256 ICR Review Operations Evaluation Department 1. Project Data: Date Posted : 08/19/2002 PROJ ID : P005743 Appraisal Actual Project Name : Secondary Education Project Costs 184.30 142.20 Support Project US$M ) (US$M) Country : Tunisia Loan/ Loan US$M ) 98.30 /Credit (US$M) 87.60 Sector (s): Board: ED - Primary Cofinancing education (48%), US$M ) (US$M) Secondary education (48%), Central government administration (2%), Tertiary education (1%), Sub-national government administration (1%) L/C Number : L3786 Board Approval 95 FY ) (FY) Partners involved : Closing Date 06/30/2000 12/31/2001 Prepared by : Reviewed by : Group Manager : Group : Gillian M. Perkins Martha Ainsworth Alain A. Barbu OEDST 2. Project Objectives and Components a. Objectives To assist the government in implementing its secondary education reform program, aimed at : (i) achieving improved quality and efficiency; (ii) improving access to accommodate the projected increase in upper basic and secondary school enrolments . b. Components A. Improve educational delivery mechanisms ($7.3 m.) Strengthen the education performance evaluation and assessment system : training, TA and equipment for the National Institute for Educational Research (NIER), and training for selected central and provincial MoE staff . Promote more effective teaching and training practices : training for inspectors, counselors, MoE staff, school directors and teachers, and TA for evaluation of the in -service training program. Strengthen the MoE's administration and planning services at central and regional levels, improving the existing information and feedback mechanisms : TA, training, and equipment to develop, implement and test a management information system (MIS). B. Improve access ($151.9 m.) Construction of 30 upper-basic schools and 44 secondary schools in under -served areas. Rehabilitation of schools in priority areas and establishment of a preventive maintenance system . Provision of instructional equipment for about 380 secondary schools, with training and TA to improve the use of the equipment and institutional support for the National Maintenance Center . c. Comments on Project Cost, Financing and Dates Actual project costs fell short of estimates by 23%: by 66% for the first component and 8% for the second. Disbursements were slow until after mid-term and the project closed 18 months later than scheduled with 89% of the Bank loan disbursed. Expenditure on works and goods reached 78% of appraisal estimates. Expenditure on services and training, at only 18% of estimates, is understated due to incomplete provision in project accounting for government contributions. The shortfalls in expenditure also reflect US$ savings from depreciation of the dinar . The Loan Agreement was amended in 1/99 to allow (i) use of loan funds to finance all costs of overseas training for NIER and (ii) consolidation of the Training and Technical Assistance expenditure categories . 3. Achievement of Relevant Objectives: The project partially achieved its objective of supporting reforms to improve quality and efficiency in the education system, although the exact nature and extent of the activities carried out is unclear in the ICR . The planned assessment system and MIS were not implemented, but the project did have some impact in enhancing capacity for evaluation and information management . The subcomponent for promoting more effective teaching and teacher-training practices was successfuly implemented : some two thirds of teachers in the second cycle of basic education were trained in the core skills approach; training received by officers in the In -Service Training Division and regional centers was found to be effective and the capacity of this Division was strengthened . The project succeeded fully in its objective of expanding enrolment capacity, meeting its targets for school construction, rehabilitation and equipment . Support was given throughout the life of the project to the center charged with maintaining school equipment . 4. Significant Outcomes/Impacts: With regard to the impact on quality and efficiency, the ICR reports substantial improvements in relevant Key Performance Indicators at project closing (2000/01) against the baseline (1993/94]: promotion rates increased from 60% to 87% (grade 6) and 77% to 82% (grade 9), while repetition rates declined in all the grades reported . The ICR also mentions reduction in dropout rates but does not quantify this . It is reasonable to assume that the training carried out under the project, together with related Bank activities (Bank-funded studies on repetition and dropout, effective collaboration with UNICEF on reform of teacher training, policy dialogue with the government on quality improvements ) contributed to these improvements . Other factors, such as other interventions, or changes in policy or practice related to promotion and repetition, are not reported . The increase in promotion rates could presumably be related to the impact of the second project component in expanding capacity. The project succeeded in meeting the country's needs for expansion of upper basic and secondary enrolments, increasing enrolment capacity at both levels by 37% while slightly reducing student -classroom ratios to 34 in upper basic and 32 in secondary, and substantially improving equipment levels . By the end of the project, male and female enrolments throughout the secondary level were on a par . Institutional development impacts include improvement in in -service teacher training capacity and in government capacity for evaluation, research, and information management . The legal status of the NIER was redefined, allowing for more effective future operation . The MoE's Institute for Information Technology was strengthened through training in the use of IT in teaching, contributing to creation of an EduNet network . The Ministry's capacity for project management was enhanced through project implementation by the MoE's core units instead of by a stand-alone PMU. 5. Significant Shortcomings (including non-compliance with safeguard policies): Implementation of the assessment system could not be achieved, due in large part to weak capacity and difficulties in the legal status of the NIER . Now that the NIER has been granted semi -autonomous legal status, implementation of the assessment system is to be carried out under the follow -on project. Failure to develop and implement the MIS under the project may have been related to the lack of an adequate communications strategy to generate the necessary interest and demand of stakeholders . A follow-on project is expected to support some elements of a system that is now being developed through other means . Early delays in implementation of civil works were attributed to unsuccessful bids, problems with the availability of land, and lack of capacity at the regional level especially in procurement procedures . 6. Ratings : ICR OED Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Institutional Dev .: Substantial Substantial Although intended impacts on the education evaluation and information systems were not fully achieved, the project still had a substantial impact in strengthening institutions and building the skills needed for more efficient and effective use of resources in the sector . Sustainability : Likely Likely Bank Performance : Satisfactory Satisfactory Borrower Perf .: Satisfactory Satisfactory Quality of ICR : Unsatisfactory NOTE: NOTE ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness. 7. Lessons of Broad Applicability: Strong government ownership was a key factor in project success . The "cascade" approach adopted for in-service teacher training, consisting of overseas training of trainers (a core group of inspectors, school directors, and teachers ), followed by a roll-out phase in the country, proved to be an efficient and effective means of introducing Core Skills methodology, supported by strong collaboration between the government, UNICEF and the Bank . The constraints encountered in efforts to implement the MIS and the assessment system might have been anticipated through more rigorous institutional assessment in project preparation . 8. Assessment Recommended? Yes No 9. Comments on Quality of ICR: The ICR presents insufficient information and explanation to allow project rating (it was supplemented by follow-up enquiries for the purpose of this Summary ). More information is needed on the quantity, type, and effectiveness of training and TA that was carried out under the project, and the "impact assessment study of the training activities ." Better explanation is needed for the radical shortfall in expenditure on training and services, and for the failure to implement the MIS subcomponent. Other issues not addressed adequately in the ICR include : the quality and cost effectiveness of the civil works; the costs and efficiency of maintenance operations; fiscal impact of the project; the equity and gender impacts of the expansion in enrolments; and monitoring and evaluation activities under the project . The project monitoring plan (para 49 and Annex 2 of the SAR), includes 26 impact indicators. Were these indicators monitored? If not, we should know that . If so, the results should be reported .