ICRR 11629 Report Number : ICRR11629 ICR Review Operations Evaluation Department 1. Project Data: Date Posted : 03/09/2004 PROJ ID : P000975 Appraisal Actual Project Name : Basic Education Sector Project Costs 241.6 47.8 Improvement Project US$M ) (US$M) Country : Ghana Loan /Credit (US$M) Loan/ US$M ) 50.0 45.0 Sector (s): Board: ED - Primary Cofinancing 191.6 0 education (88%), Tertiary US$M ) (US$M) education (9%), Central government administration (3%) L/C Number : C2885 Board Approval 96 FY ) (FY) Partners involved : None Closing Date 06/30/2001 12/31/2002 Prepared by : Reviewed by : Group Manager : Group : Howard Nial White Ronald S. Parker Alain A. Barbu OEDST 2. Project Objectives and Components a. Objectives The Basic Education Sector Improvement Program (BESIP) was designed to support the Government of Ghana (GoG) in implementing its policy to achieve free, compulsory, and universal basic education (FCUBE). FCUBE had three broad objectives: (a) improve the teaching process and learning outcomes; (b) strengthen management of the basic education system; and (c) improve access to basic education, especially of girls and the poor and other disadvantaged segments of the population . Because it was designed to support FCUBE, the BESIP had these same objectives. There was a fourth objective in the Staff Appraisal Report which was stated as follows : "ensure financial sustainability of the government program for basic education over the longer term ." However, this fourth objective was not mentioned in any of the other project documents such as the Development Credit Agreement and the Project Status Reports, and there were not corresponding project components addressing this objective in the SAR . For the reasons just given, the fourth objective is not discussed in the ICR . b. Components The three components correspond to the three objectives : (1) Enhanced quality of teaching and learning, through (a) in-service training of education personnel; (b) pre-service teachers’ training; (c) assessment and evaluation of student performance; (d) provision of instructional materials; and (e) curriculum review and development . (2) Management for efficiency, through (a) building capacity for institutional /organizational analysis and change; (b) staffing and personnel management; (c) performance management; (d) budget and financing management; and (e) district capacity-building. (3) Improving access and participation, through (a) infrastructure development; (b) school facilities planning and maintenance; (c) increasing girls’ participation; (d) fostering community involvement; (e) implementing School Improvement Fund (SIF); and (f) promoting Information, Education, and Communication (IEC). c. Comments on Project Cost, Financing and Dates The original financing plan was based on the assumption that other donors would co -finance BESIP in support of FCUBE. The appraisal estimate (SAR) did not clearly separate the contribution between the government and donors . In the event donors undertook their own stand -alone projects. BESIP itself was also restructured at mid -term review as a stand-alone project. The project got off to a slow start . By April 1999 (mid-term review) disbursement was 80% behind schedule, with unsatisfactory ratings for both Implementation Progress and Development Objectives . The project was restructured to focus on a smaller number of components, but the objectives not altered . In light of these considerations the postponed closing of the project was reasonable . 3. Achievement of Relevant Objectives: Improve Teaching Process and Learning Outcomes : The project financed the publishing and distribution of 10.6 million textbooks to primary and junior secondary schools . In addition 96,200 syllabi and 6,500 globes were distributed. The quality of teaching and learning has been improved with the availability of textbooks and other instructional materials and better trained teachers, as reflected by the results of two tests, namely the Criterion Referenced Test (CRT) administered at grade 6 (started in 1992) and the Performance Monitoring Test (PMT) administered from grade 1 to 6 (started in 1998). OED's impact evaluation of support to basic education in Ghana confirms the importance of textbook supply in improving learning outcomes . Support was given to curriculum development, but new textbooks are only beginning to be developed at the end of the project, so there is a mismatch between syllabi and textbooks . In-service training was provided, but on a limited basis . Strengthen Management: Progress has been made in management efficiency : (1) the pupil-teacher ratio has increased, and (2) the establishment of the Education Management Information System (EMIS) has been a success. In terms of building capacity for institutional /organizational analysis and change, there were five training courses in Accra for senior management of MOE and GES, and participation in a Washington -based course and at Harvard for EMIS staff. In addition capacity building was provided to the Supplies and Logistics Unit . The following areas were not included in the project after the MTR : staffing and personnel management, and budgeting and financial management; in addition district capacity -building was reduced. The ICR claims the project supported improved management efficiency through the enhanced role of circuit supervisors and expanded role of districts in civil works . These effects would have been strongest prior to the MTR . Following restructuring these components were reduced, party in consideration of district level capacity building efforts which had been launched by DFID . Improve Access: Access and participation have been increased : there has been an increase in both the Gross Enrolment Rate (from 76.5 to 80% for primary and 58.7 to 63.9 for JSS) and a narrowing of the gender gap in enrolments. The sub-components were: (i) infrastructure development - (a)construction of 344 four unit teacher accommodation blocks, 151 classroom blocks and 151 dry sanitation facilities, (b) provision of school furniture including 17,250 dual desks, and (c) rehabilitation of 2,300 classrooms; (ii) a manual and guidelines on school maintenance; (iii) activities to promote girls' participation, such as training workshops and review of curricula and textbooks, (iv) fostering community involvement through the creation of School Management Committees, although the role of BESIP in this is not stated, (v) School Improvement Fund, a demand -driven fund for school expenditures which was discontinued in 1999; and (vi) the creation of national, regional and district Information, Education and Communication (IEC) teams. 4. Significant Outcomes/Impacts: The most significant outcomes are the construction and rehabilitation of classrooms and the distribution of school equipment and textbooks. These activities supported the rise in enrolments and modest improvement in learning outcomes recorded in the CRT. The most significant outcome under institutional development has been the successful establishment of EMIS, though its sustainability is dependent on continued external support, and there are questions as to the adequacy of capacity development in the technical aspects of managing the system . There has also been decentralization of education management, though the role of BESIP in that is not clear . The project also support ID in funds and procurement and the Supplies and Logistics Unit . BESIP jointly supported (with GoG and USAID) the Criterion Reference Test and BESIP supported MOE's participation in the Trends in International Mathematics and Science Study (TIMSS). 5. Significant Shortcomings (including non-compliance with safeguard policies): The project design was over -ambitious and problems arose since the project was structured as a sector program in the absence of a firm commitment of co -financing from other donors. A related issue is the lack of clarity in project documents as to what the IDA credits would finance . To be fair to the staff involved in design, Bank staff made a strong attempt to mobilize external resources behind a clearly spelled out government strategy (FCUBE) and were putting their "money where their mouth was". Moreover, the emerging view on sector programs at the time was that the Bank should be willing to adopt a residual position, financing the parts of a program not picked up by other agencies. Once other donors failed to come on board the project design, which included the components to be financed by co-financiers, was clearly too ambitious for the Bank alone . Bank staff were right to scale down at mid-term review, and might perhaps have done so earlier . Another apparent short-coming was the continued emphasis on hardware after restructuring, following which the project focused on the three main sub -components: (a) construction of the new schools and teacher blocks and rehabilitation of schools; (b) textbook reprinting and distribution; and (c) EMIS development. The project largely focused on the supply side (building facilities) despite the evidence of the importance of addressing demand side considerations. However, OED's impact study shows that this is a case in which increased supply stimumates demand: buildings and textbooks have played an important role in improved enrolments and learning outcomes, so the nature of the restructuring was not inappropriate in relation to the first and third objectives, with the focus on EMIS covering the second objective . Sustainability will only be possible with the continued support from DPs . The MOE does not allocate sufficient funds to print new textbooks that comply with the new syllabi, to replace the damaged or missing textbooks or to increase the coverage with the increasing of enrollments; and it does not have sufficient funds to continue the new construction and rehabilitation of classrooms and teacher accommodation blocks . EMIS finally got a budget line in the MOE’s budget, but there was no budget line for the EMIS units at the district and regional levels at the time the project closed... Technical sustainability is mostly at stake in the area of EMIS . Despite heavy and expensive technical assistance by the Harvard consultants, the national EMIS staff are still burdened by a lack of technical skills to independently carry out various functions of EMIS and to further develop the system . In addition, it should be recalled that the project had a fourth objective to "ensure financial sustainability of the government program for basic education over the longer term." This objective has clearly not been met . The project design is to be faulted for not addressing this objective. 6. Ratings : ICR OED Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately Satisfactory [The ICR's 4-point scale does not allow for a "moderately sat." rating]. Performance in relation to the second objective (strengthen management) was weak. Many activities to support first objective were not carried out. Institutional Dev .: Modest Modest Sustainability : Likely Likely Financial sustainability is dependent upon continued donor financing. Bank Performance : Satisfactory Satisfactory Borrower Perf .: Satisfactory Satisfactory Quality of ICR : Satisfactory NOTE: NOTE ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness. 7. Lessons of Broad Applicability: (1) Avoid unrealistic project designs . In a context of weak institutional implementation capacity, a complex design with many components/sub-components and activities is bound to ensure failure . It is premature to design a sub-sector program without a full financing commitment from the government and development partners . (2) Ensure that core capacity is in place prior to implementation . It is very risky to move from preparation to implementation if the core capacity is not readily available in implementation agencies . (3) Carefully analyze counterpart funding capability . Low disbursements often occur among activities in which the proportion of counterpart funding is high . This makes such activities very risky in terms of achieving the outcomes . (4) Have an explicit and realistic plans for sustainability . (5) Involve all major stakeholders in the entire project cycle . It is important to include all major stakeholders (from the central level to the community level ) in the entire cycle of the project, from preparation to implementation and finally to completion. It may hurt the stakeholders' motivation, ownership, commitment, and accountability if some do not become involved until the middle of implementation. (6) Ensure accountability and continuity in project management . The Bank should try not to change Task Team Leaders frequently. (7) Better target the disadvantaged areas and groups . Access to education and quality of learning are uneven across regions and groups in Ghana. The design of the BESIP project failed to take enough actions to address these issues . . 8. Assessment Recommended? Yes No Why? (1) Cluster with preceding Primary School Development Project, (2) complement impact study and Education Sector Review. 9. Comments on Quality of ICR: The ICR is well-written. Its main shortcoming is a failure to clearly identify all the activities supported by the Bank, or the way in which they were supported . For example, what was the nature of support for in -service training? The ICR states that the SAR itself did not clearly identify what Bank funds would support . But the SAR states that: "The proposed IDA Credit of US$50.0 million equivalent will support in-service training of educational personnel, provision of instructional materials to schools, training and workshops for MOE /GES, DEOs and SMCs to improve efficiency in management, rehabilitation and construction of school facilities to improve access and the teaching /learning environment, and initiatives to encourage girls' participation and community involvement in education and schooling activities. The level of IDA financing of specific items and activities will be reviewed annually as part of the government/donor review process" (SAR, p.27). However, the information in project files does not, as the ICR correctly states, allow indentifcation of which of these activities were in fact implemented using BESIP resources . A second short-coming is the relative light treatment of the project's focus on hardware . Despite the ICR itself concluding that investments in software matter, it does not criticize the restructuring at MTR at which software investments were largely dropped . However, OED's own impact study suggests that the focus on hardware was, after all, not inappropriate. Finally, and linked to the first two points, the ICR does not provide evidence that the project supported the decentralization which took place, partly since information is not given on how the money was spent, but also because district capacity building was scaled down .