Document of The World Bank Report No: 31861 IMPLEMENTATION COMPLETION REPORT (IDA-33430 PPFI-Q2010) ON A CREDIT IN THE AMOUNT OF SDR 9 MILLION (US$12 MILLION EQUIVALENT) TO THE REPUBLIC OF ALBANIA FOR AN EDUCATION REFORM PROJECT April 22, 2005 Human Development Sector Unit South East Europe Country Unit Europe and Central Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective March 4, 2005) Currency Unit = Albania Lek Lek 96.55 = US$ 1 US$ 1 = Lek 96.55 FISCAL YEAR January 1 December 31 ABBREVIATIONS AND ACRONYMS CAS Country Assistance Strategy CIS Center for Information and Statistics DIS Department of Information and Statistics DO Development Objective EMIS Education Management Information System ERP Education Reform Project GOA Government of Albania ICB International Competitive Bidding IDA International Development Association IPS Institute for Pedagogical Studies M&E Monitoring and Evaluation MOES Ministry of Education and Sciences MTR Mid-term Review NAEC National Assessment and Examinations Center NGO Non-governmental Organization OECD Organization for Economic Cooperation and Development PAPU Policy Analysis and Planning Unit PCD Project Concept Document PCP Public Contact Point PIU Project Implementation Unit QAG Quality Assurance Group SDR Special Drawing Rights SRCAE Statistical Report Card on Albanian Education SRCBP School Rehabilitation and Capacity Building Project STD Textbook Distribution Enterprise STPH State Textbook Publishing House UNESCO United Nations Educational, Scientific and Cultural Organization WB World Bank Vice President: Shigeo Katsu Country Director Orsalia Kalantzopoulos Sector Director Charles Griffin Sector Manager Maureen McLaughlin Task Team Leader Peter Darvas REPUBLIC OF ALBANIA EDUCATION REFORM PROJECT CONTENTS Page No. 1. Project Data 1 2. Principal Performance Ratings 1 3. Assessment of Development Objective and Design, and of Quality at Entry 2 4. Achievement of Objective and Outputs 6 5. Major Factors Affecting Implementation and Outcome 12 6. Sustainability 12 7. Bank and Borrower Performance 14 8. Lessons Learned 15 9. Partner Comments 16 10. Additional Information 17 Annex 1. Key Performance Indicators/Log Frame Matrix 18 Annex 2. Project Costs and Financing 21 Annex 3. Economic Costs and Benefits 23 Annex 4. Bank Inputs 24 Annex 5. Ratings for Achievement of Objectives/Outputs of Components 26 Annex 6. Ratings of Bank and Borrower Performance 27 Annex 7. List of Supporting Documents 28 Annex 8. Ministry of Education and Science Comments on the Education Reform Project 30 Project ID: P069120 Project Name: Education Reform Project Team Leader: Peter Darvas TL Unit: ECSHD ICR Type: Core ICR Report Date: April 22, 2005 1. Project Data Name: Education Reform Project L/C/TF Number: IDA-33430; PPFI-Q2010 Country/Department: ALBANIA Region: Europe and Central Asia Region Sector/subsector: Primary education (45%); Central government administration (37%); Other industry (18%) Theme: Education for all (P); Administrative and civil service reform (P); Social analysis and monitoring (S); State enterprise/bank restructuring and privatization (S) KEY DATES Original Revised/Actual PCD: 11/08/1999 Effective: 08/01/2000 07/19/2000 Appraisal: 02/11/2000 MTR: 11/04/2002 11/04/2002 Approval: 05/11/2000 Closing: 07/31/2004 10/31/2004 Borrower/Implementing Agency: MINISTRY OF FINANCE/MINISTRY OF EDUCATION AND SCIENCES Other Partners: STAFF Current At Appraisal Vice President: Shigeo Katsu Johannes F. Linn Country Director: Orsalia Kalantzopoulos Arntraud Hartmann Sector Manager: Maureen McLaughlin James A. Socknat Team Leader at ICR: Peter Darvas Sue Ellen Berryman ICR Primary Author: Peter Darvas 2. Principal Performance Ratings (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible) Outcome: S Sustainability: L Institutional Development Impact: SU Bank Performance: S Borrower Performance: S QAG (if available) ICR Quality at Entry: U Project at Risk at Any Time: Yes The Project ratings reflect that, on balance, the major outcomes and the outputs were achieved and sustainability is likely. Given the fact that the implementation was hindered by design and by capacity problems especially at the beginning of the project implementation, a moderately satisfactory overall rating would more appropriately reflect the project outcomes but this option is not available to the ICR team. The project provides important lessons regarding designing and implementing future development programs and investment projects in the education sector. In the meantime, the project significantly increased management capacities and put a strategic agenda and reform program in motion. It established new structures and procedures for key areas in education management and service delivery. The outcomes and the fact that significant improvement was achieved by the Borrower should be recognized. 3. Assessment of Development Objective and Design, and of Quality at Entry 3.1 Original Objective: Original Objective: The development objective of the Education Reform Project was to assist the Ministry of Education and Sciences (MOES) in planning and managing the delivery of educational services and strengthening its accountability to stakeholders for that delivery. This objective was not changed during the project implementation. The outcome indicators were the following: (1) More effective Ministerial leadership and management of sector improvement; (2) Enhanced civic support for sectoral reform; (3) Statistical and information databases that are (a) routinely used for decision-making by policy makers at the central level, (b) routinely updated, and (c) routinely published in the form of education indicators for the broader public; (4) More rational and persuasive budgetary processes for the sector; and (5) Increased and better-used international aid resources. Context: Albania's economic transition in the 1990s was severely disrupted in 1997, when the collapse of pyramid schemes wiped out a large part of private savings in this poorest society of Europe and caused widespread civil unrest. This crisis also exposed an essentially weak State and underdeveloped public institutions. During the disturbances, people turned against both, causing widespread destruction of properties providing public services, such as schools. The Country Assistance Strategy (CAS) for the 1999-2002 period put a central emphasis on governance and institution-building, based on the recognition that a strong State and effective public institutions are essential for sustaining long-term development. The economic reform of the early 1990s did not stimulate a similar reform process in the education sector. Curriculum reform appeared to be a complex and overwhelming challenge and was deferred throughout the 1990s. Intensive migrations left some schools, mostly in Tirana and in other cities, overcrowded, and others in rural areas, underutilized. Public expenditures were inefficient, enrollments started to decline, especially at the secondary level -- primarily due to the closure of many vocational and technical schools -- and the Third International Mathematics and Science Survey in 1999 ranked the performance of 14-year old Albanian students the lowest in Europe. Typically, the more meaningful reforms were initiated in parallel to -- rather than from within -- the Ministry of Education and Science by an externally (Soros Foundation) funded development program that relied on non-governmental capacities and allocated resources to schools outside of the public routes. Between 1999 and 2000, reports by the Bank's Education Sector Work identified a large number of policy actions to be undertaken including reforming education management and administration, finances, curriculum, textbook-reform, training, decentralization and social participation. Under these circumstances, the CAS plans for the Education Reform Project (ERP) targeted systemic reforms that should start from an improved institutional and strategic capacity in the Ministry of Education and Sciences (MOES). A first education project, the School Rehabilitation and Capacity Building Project (SRCBP), was implemented between 1994 and 1999. The project targeted similar policy issues as the Education Reform Project, albeit with some different emphases. Given the serious deterioration of school infrastructure and - 2 - the constrained public finances, the Government could not significantly address the overall operation of the education system and the urgent school rehabilitation needs. The school infrastructure was the SRCBP's first and foremost priority. In addition, SRCBP, like ERP, attempted to address key sector capacity issues, including planning and management information, quality assurance and textbook quality improvement. Even though implementation of the SRCBP was rated satisfactory, sustainability of the project outcomes was rated low and there was a reasonable justification that many of the project activities needed to be repeated or continued during the new project due to insufficient capacity and high turnover of the Ministry staff. Relevance: The development objective of the ERP was generally in line with the 1998 CAS in that it focused on building Government and public institutions. The project's overall concept as well as the project components matched the country's needs at the time of the preparation and have also remained relevant to the present. The plan was also based on the lessons learned from the first education project in the country, which closed in 1998. Both the project objective and the specific indicators were harmonized with two other Bank-financed projects, the Public Administration Reform Project that started in 2000 and the Poverty Reduction Strategy Credit that started in 2001. There were links in terms of objectives, indicators and activities between these projects. The expected project outcomes were in line with the critical sector issues. The outputs delivered during implementation brought about significant and relevant outcomes. There were also problems with the project design, however. In particular, there was a disconnect between the development objective and indicators and the overall project design. While the indicators were mostly linked to capacity building through technical assistance, the Government's major efforts and the large majority of the project funds went to school rehabilitation and textbook production. These activities were significant and in line with key sector needs, but did not specifically relate to the development objective or to the designated indicators of the project. This disconnect presented problems in monitoring and in early project implementation. Realism: The capacity building for planning and managing educational service delivery and for strengthening accountability is a realistic goal in terms of what the project intended to achieve. However, the project concept remained ambiguous especially in terms of how the development objective would be measured. The development objective left it unclear under what circumstances would the assistance to the Ministry -- as the key objective -- be successful, how such success should be measured, and indeed, what improved service delivery would entail in the sector. This difficulty was reflected in monitoring of the key outcome indicators; measuring progress along the key objective and outcome indicators have remained persistently problematic, even if most of the outputs were delivered. Limited Government ownership also negatively affected the realism of the project design. Even prior to project effectiveness, high turnover among ministry staff from the senior leaders to the mid-level managers constrained the Ministry's ability to implement the first Bank-financed project or to actively participate in defining policy priorities, project activities or implementation arrangements. With the limited capacities, setting high standards in a large number of areas could be considered unrealistic. A so-called "Special Initiatives" subcomponent was a good example of this problem. Under this category, unidentified Ministry initiatives were to be supported once they met "specific" criteria. In absence of adequate understanding of these "specific" criteria in the Ministry, this activity was never implemented. Prioritization: Previous sector work defined a broad set of critical sector issues. There was a very broad scope of capacity building, ranging from Ministry reorganization to curriculum reform, to education assessments, to building management information system and to education strategy, making it difficult for - 3 - the Borrower to integrate all the new capacities. The general description of the project identified 13 broad outputs and further outputs were specified under each component. The lack of specificity in terms of priority areas in the project scope was detrimental to the project particularly given seriously limited Ministry capacities. The high expectations and the limited capacities haunted the project implementation. The project could have benefited from clearer and more narrowly defined set of policy priorities, better targeted technical assistance and investments and more measurable outputs. 3.2 Revised Objective: During the mid-term review of the project, which coincided with a change in the Minister's post, the Bank task team, together with the Government representatives concluded that the project objectives had not been fulfilled at that point and the implementation was unsatisfactory. However, the new Minister strengthened the Government's commitment towards the project, and -- consequently -- the review process did not propose the revision of the development objective. Implementation did improve after that mid-term review. 3.3 Original Components: 1. Policy Design, Management, and Improvement of the Education System ($3.59 Million). This component was an institution-building and organizational reform component. It was designed to evaluate and strengthen basic Ministry functions and structure and to carry out various strategic planning activities and policy analyses. The component supported the establishment of new units, new structures and more competitive hiring policy as well as accountability measures (evaluation of Ministry activities and standards for staff performance). The primary role was assigned to a to-be established Council of the Ministry of Education and Sciences to set sector strategy and priorities and to a technical unit to act as a technical secretariat to the Council. The technical secretariat was assigned to conduct expenditure review, strategic planning and analyses of various policies. As mentioned before, under the "Special Initiatives" subcomponent, the project was also designed to provide funds to start to implement programs that had not been identified at the design phase but were supposed to improve the quality and administration of education. Proposals for these programs would receive funding if and when they met specific criteria. These criteria included that the Government and the World Bank needed to agree that the proposed program was of high priority in accordance with the MOES strategy, that it had been costed, that its cost effectiveness was shown to be satisfactory, that a satisfactory action plan was showing how and by whom the program would be implemented, and that the other resources needed for program implementation were available. The project design planned to set aside US$2 million for these previously unidentified proposals. The key expected outputs of the component were: (a) the establishment of a Council of the MOES that functions as a priority-setting and program-designing policy body; (b) the establishment of a technical policy analysis and planning unit that, as technical secretariat, supports the Council; (c) the development of financial analysis skills and tools in the technical secretariat; (d) the completion of a reform plan for the sector that meets specified criteria; (e) the completion of at least one instance of a program design that meets specified criteria; (f) a first annual reform work plan that meets specified criteria; and (g) the establishment of a PIU to manage the implementation of the project. From the development objective's perspective, this component was absolutely critical for the success of the project. It was not only important to deliver the numerous outputs, strategies and standards but the component was supposed to provide an overall umbrella for the rest of the project. 2. Education Management Information System ($0.8 Million). This component was also an institution-strengthening component, built on the school mapping data base that had been completed under - 4 - the Bank's first education project in Albania and to be managed by a newly-established department within the MOES, the Center for Information and Statistics (CIS). It supported a management information system that was to link all branches of the Ministry, including school district offices and universities. The key outputs were planned to include: (a) a simple EMIS that can be easily extended in terms of data, applications, and users, introduced and tested in at least two pilot education district offices, and used routinely by policymakers and managers at the Ministerial and district levels; and (b) publication of the first "Statistical Report Card on Albanian Education" (SRCAE) for stakeholders. 3. Assessments and Examinations ($0.57 Million). This component was also planned as an institution-building and policy reform component and also built on activities begun under the first education project financed by the World Bank in Albania. It was planned to be managed by the National Assessment and Examinations Center (NAEC), an agency under the auspices of the MOES. The key expected outputs included: (a) creating and administering a university entrance examination that meets international standards for quality and fairness; (b) conducting and analyzing the results of a national grade 4 learning assessment in Albanian language, mathematics, and science and publishing these results; and (c) preparing for activities to be completed under a follow-on project, including administering national examinations for grades 8 and 12 and developing diagnostic assessment materials for teachers to use in the classroom. This component appeared at the start of the project the best understood in terms of purpose and implementation. 4. School Rehabilitation ($7.3 Million). This component included physical goods (civil works), policy reform, and debt reduction. The component had the following planned goals and outputs: l the development of building and design guidelines and specifications of bills of quantities for schools and furniture; l the rehabilitation of 44 schools with approximately 22,000 places at an average estimated cost of US$200 per school place (i.e. the maximum number of students that can potentially be enrolled in a school); l a transparent and rationalized model to allocate school rehabilitation financing among districts; l private architectural firms or NGOs used for design and quality assurance functions; and l communities participating in the rehabilitation of the schools and their grounds. School rehabilitation had persistently been identified as the most acute problem for the sector, requiring sustained support from Governmental, non-governmental and external sources alike. The major concern with the inclusion of this component is that linkages were not made between investment under the component and the project development objectives. 5. Textbooks ($2.62 Million). This component was a policy reform and physical goods component. Within the policy reform, two State-owned companies, the State Textbook Publishing House (STPH) and the Textbook Distribution Enterprise (TDE), were to be brought to the point of sale. The component funded textbook procurement using international competitive bidding to reduce the costs and increase the quality of the textbooks being purchased by Government and published textbooks for the new curricula in vocational/technical education as a trial for privatizing publishing. - 5 - The key outputs for this component were to be: (a) STPH and TDE brought to the point of sale; (b) technical assistance for the STPH and TDE and equipment provided for the STPH to create a level playing field during the privatization process; (c) contracts for publishing textbooks transparently provided; (d) textbooks procured through International Competitive Bidding (ICB); and (e) new financing policy based on the basis of savings realized through ICB (including revisiting the percent of total textbook costs paid by Government versus parents). The components, activities and outputs were by and large in line with the development objective, with the exception of an issue raised in the context of the fourth component. The key challenge was stemming from the complexity of the design, the number and scope of various activities and the mismatch between these plans and the absorption capacity of the Ministry. Careful sequencing (with capacity building preceding other outputs) and a more narrow focus would have made the achievement of some of the outputs more timely and realistic. However, the Project Operational Manual reflected no such concern with sequencing the implementation schedule in line with the limited capacities. 3.4 Revised Components: By the mid-term review of the project, there were some significant delays and a lag in disbursement. However, the new MOES leadership, appointed just before the mid-term review, made some progress in accelerating project implementation and no restructuring was initiated. Since the MOES had not been able to meet the criteria for the Special Initiatives under the first component, the Bank and MOES agreed to reallocate the US$2 million, originally planned for these special initiatives, to finance additional school rehabilitation activities with a minor portion of the funds earmarked for school furniture and for technical assistance for community participation. 3.5 Quality at Entry: There was no formal Quality at Entry Review. There was a simple and largely incomplete Quality at Entry Review, provided by a WB staff member, based only on the Project Appraisal Document. This feedback rated the project at entry highly satisfactory based on institutional, political economy, and participation dimensions. Overall, this ICR rates the quality of entry unsatisfactory given (1) the disconnect between the objective and the project design (activities and investments), (2) the difficulties with measuring progress, (3) the lack of prioritization and overly complex assignment, and (4) inadequate Government ownership and lack of understanding by the Government of the project objective and key activities. There has been considerable uncertainty at the MOES as to how to implement, what to expect from the project and, consequently, delays in disbursement and in implementation and persistent problems in monitoring. Many of the problems regarding the Borrower capacities had been foreseen during the project preparatory work, but no adequate measures were put in place to mitigate the risks stemming from the subsequent problems. In the absence of such measures, the implementation of the project was significantly constrained during the first half of project implementation. 4. Achievement of Objective and Outputs 4.1 Outcome/achievement of objective: While the development objective -- as defined in the original project concept -- was difficult to measure, the project successfully assisted the Ministry in planning and managing the delivery of educational services and strengthening its accountability. This is evidenced by (1) more transparent and accountable internal procedures for hiring, performance standards and assessments, (2) new capacities in the Ministry for - 6 - strategic planning and policy analysis, financial planning, statistical information, public information and textbook policy, (3) new structures and capacities established at central level for curriculum, teacher training, and education assessments, (4) competitive procedures and private capacities for textbook publishing and distribution, and (5) the completion and adoption of key strategies guiding education investment, expenditures, decentralization and education for all policy. Also, perhaps more importantly, most project activities brought about significant positive outcomes that may not have been planned explicitly in the development objectives, but helped to support better education by improving the conditions and capacities necessary to achieve measurable progress in learning outcomes and in the quality of instruction. Whereas the development objective did not go as far as targeting such improvements, the activities and project outputs clearly facilitated and targeted such achievements. For example, the new textbook policy resulted in efficiency gains by cutting average textbook prices by over 30 percent. The establishment of the Curriculum Center and accompanying technical assistance contributed to a curriculum reform, which in the next five years will establish the basis of significant improvement in the quality of education. Also, there is some evidence from Tirana municipality -- measured by schools' self reports -- that school rehabilitation and textbook provisions resulted in improvement in attendance and in student performance, even though such expectations had not been formed at the beginning. Planning some of these outcomes explicitly would have made the relevance of the project more meaningful and more attractive to a broader group of stakeholders and beneficiaries of the system. Relevance: The capacities and the increased accountability have remained relevant in terms of the current Bank priorities and country assistance strategy updates. The project supported the development of a new Government Education strategy, an education decentralization strategy and an Education for All strategy. These will be important guidelines both for the Bank and for other donors in the next 10 years. School rehabilitation, sponsored by the Government and various external donors, has so far covered about 20 percent of the total need. However, the standards, developed under the project, will guide civil works in the next 10 years during which, approximately US$180 million is needed from internal and external resources to complete the rehabilitation of schools in the country. The implementation of a curriculum reform program, which has been designed through the ERP, has started in 2004. This will be the most critical priority for the Government in the sector for the next five years and will require roughly US$4 million. During these years, the Government will need to use the new textbook policy to issue the new textbooks based on the new curriculum in the amount of US$4.5 million and also strengthen its assessment capacities to be able to better define and monitor learning outcomes. The capacity building is also in line with the Bank priorities as education remains a strategic sector for future investment. Given that the next education project -- scheduled for the 2006 Fiscal Year -- will require key Governmental institutions to be capable of implementing the Government Strategy and monitoring the results of the implementation. The project relevance is substantial. Efficacy: By the end of the project, significant indicators of institutional development have been achieved. There are new internal processes within the Ministry, including competitive hiring and the performance standards, which increased transparency. These standards were also supported by the Public Administration Reform Project. The strategic activities (10-year strategic plan and plan for Education for All) make both education management more accountable and sector developments more transparent. These plans were also in line with the benchmarks of the Poverty Reduction Strategy Credit. The transparency of the education system is also increased by the establishment of the Center for Educational Assessment and Examinations and the Center's assessment and examination activities. A more capable, better structured and more transparent Ministry is a key to achieve the Government's strategic objectives and to gain stronger commitment and civic support for education reform. Lastly, the Ministry developed an education decentralization agenda and is on track to implement this program. - 7 - The budgeting processes are more rational and persuasive as they are now based on a medium-term expenditure framework, more transparent expenditure mechanism and costed long-term education sector development agenda. As part of the budgeting and financial capabilities, the Ministry is able to monitor the economic and fiscal benefits of interventions, track expenditures, and show improved accountability of its and the sector's performance. There are also some key institutional developments that reflect improved budgeting and financial capabilities, including the monitoring capabilities of the cost-efficiency of the improved textbook production system and also the estimates and standards for school rehabilitation. The statistical information databases have also been established with some qualifications. The first statistical report has been published, improving significantly the transparency and accountability of the sector. However, the statistical information is not linked to the EMIS nor to school mapping and its local usability is also unknown. There is very little concrete information about civic support for sectoral reform or for the increased and better-used international aid resources, both being part of the original outcome indicators. In fact, international aid for the education sector has decreased from US$26.5 million in 2000 to US$17.5 million in 2004. However, this decline (nor an expected increase) is hardly an adequate measure of success for an education reform project and should not be used as a basis of project rating. While school infrastructure will become, in 2005, the responsibility of the municipal governments, new infrastructure standards and school rehabilitation standards were established in the Ministry. These were adopted by the Government as mandatory standards for the municipalities. These standards extend transparency measures to the municipalities, some of whom have already started to link investments in school rehabilitation and also improvements in textbook quality and distribution to improvements in key educational indicators such as attendance and also in performance. A key unexpected outcome is the impact of textbooks and school rehabilitation on dropouts and student performance (measured in terms of successful passing of year-end examinations) at primary and secondary levels. In the sample of five primary schools in Tirana municipality, the dropout rate at the primary level decreased by 15 percent, and in the sample of four lower secondary schools, the dropout rate decreased from 7.6 percent to 3.6 percent in one year, between 2003 and 2004. In the same sample of lower secondary schools, the passing rate (successful completion of year-end examination) improved from 53 percent to 59 percent for all the students and 59 percent to 66 percent among girls. While these improvements may not have been caused only by the project interventions, these interventions clearly contributed to the positive trends. In terms of finances, a number of indicators suggest improvements. For instance, the competitive textbook printing and procurement system is likely to contribute not only to improved conditions of quality of learning but also an impressively significant 33 percent reduction in the average price of textbooks. These savings allowed broader access to textbooks (97 percent of the 1st to 7th graders received new textbooks). An implementation plan for the education strategy includes detailed information on financing needs and a significantly broader spectrum of expenditure categories including curriculum development, teacher development and new textbooks covering the entire compulsory education. Also, as part of the Government's overall education strategy, compulsory education is being extended from eight to nine years which, in turn, will significantly improve education attainment (measured by the average number of school years completed). These measures of improving access and education participation will bring about solid improvements in the sector beyond the project's lifetime. Overall, with minor shortcomings, the objectives were met and the project efficacy is substantial. - 8 - Efficiency: The efficiency of investment in institutional capacity building and technical assistance in general cannot be measured against alternative investments. In the long-term, such capacities can assure more efficient spending of internal and external resources in the sector. Another way in which the project components involving Technical Assistance can have financial returns is through efficiency savings. This can increase the efficiency of the entire public budget spent on education in Albania. The benefits could be measured through improvements in learning as opposed to alternative investments. Such improvements were evidenced in the Tirana school district, where the district officials monitored -- through school reports -- the impact of the distribution of textbooks on learning and showed positive results even within a two-year period. Similarly, efficiency gains may be combined with education benefits through the curriculum reform program which is now based on larger knowledge areas rather than specialized subject fields. Such reform can make both pre-, and in-service training more efficient and more effective in terms of learning improvements. The one area where efficiency improvements could have been achieved with better planning is to better link school rehabilitation to EMIS and assure that the capital expenditures are targeted at schools with increasing student numbers and that investments follow the internal migration patterns. Overall, efficiency should be rated substantial. Overall, the strengthened Ministry capabilities and management processes, the reforms that have been started and the strategic plans reflect significant improvements in education outcomes. These improvements plus some additional outcomes that had not been foreseen at the beginning, make the achievement of the development objective satisfactory. 4.2 Outputs by components: 1. Policy Design, Management and Improvement of the Education System (US$1.56 Million):The project successfully completed the restructuring of the Ministry of Education and Sciences. It helped establish a Council of MOES in 2000 and a Policy Analysis and Policy Development Department in 2003. The latter contributed to the development of key documents, including the Education sector strategy for 2004-2015, an implementation and financial plan for the strategy, and an Education for All development proposal. A performance audit was carried out, as planned. Based on the audit outcomes and experience, new performance standards were developed, training provided and a competitive hiring process was established. These outputs were also in line with the Public Administration Reform Project objectives, indicating adequate synergy between sector investment and adjustment operations. The key outputs in the context of financial capabilities in the Ministry included the costing of the education strategy, the public expenditure analysis, the medium-term expenditure framework and the monitoring of the financial impact of the school rehabilitation and textbook production components. In addition to the changes in the Ministry, the Institute of Pedagogical Studies was divided into a curriculum and a teacher training center. The former became the central agency responsible for a curriculum reform, which from this year on, will introduce a new curriculum for general education. The MOES, in collaboration with the new Curriculum center, produced the National Framework Curriculum, including standards for assessment and evaluation. While at the end, the component achieved satisfactory outcomes, it was marred at the beginning by delays and, throughout, by problems of sequencing. The strategic documents were finalized only towards the end of the project implementation and a question remains whether the strategies are fully understood and owned by the staff. Also, the restructuring of the Ministry, new hirings and training were not adequately sequenced. It appears that the high turnover will necessitate repeated capacity building for new staff. In addition to the Public Administration Reform Project, several outputs of this component were used as conditions for the Poverty Reduction Strategy Credit. Overall, this component of the project is satisfactory, as most of the originally expected institutional and strategic outputs have all been achieved and the - 9 - implementation of a significant curriculum reform has started and synergy between different Bank-financed activities was established. 2. Education Management Information System (US$0.72 Million): The project developed the fundamentals necessary for a functioning education management information system, including the establishment of the Department of Information and Statistics, with a small staff. Also, the EMIS component funded the development of a data collection and query system and both general administrative and analytical softwares. However, these systems and softwares have never worked adequately. These were supposed to link the Ministry with two pilot school districts, but this link has not been functional either, and data has not been collected or analyzed. One positive measure is the statistical report card, which was first issued in 2004. The report is helpful for the policy makers and the public alike and is expected to be published on a yearly basis. Also, a contact point was established in the Ministry for the public to be able to access information about the education sector, however, this facility is rarely used and the data is somewhat outdated. The component was rated unsatisfactory throughout the project implementation. One of the key limitations is that there is no clarity among the decision makers as to how an EMIS should be used, i.e. whether it should collect student-based information (to monitor participation and student performance) or school-based information (to monitor supplies and efficiency). The lack of clarity is also stemming from the fact that the key strategic and policy documents, which would have been able to identify the key information necessary to monitor, were developed during the last third of the project implementation period. 3. Education Assessment (US$0.56 Million): The National Assessment and Examinations Center (NAEC) was established in 2001 and carried out the first two sample-based learning assessments at grade 4 in mathematics and language in May 2002 and 2003. The assessment results were disseminated to the public. A new 8th grade examination was developed and administered for the first time in June 2003. The examination was administered for a second time in June 2004. The examinations were scored, the results analyzed and reported by dissemination to stakeholders. The new matura examination was developed and administered for the first time in June 2004 in mathematics, literature, science and Albanian language. The NAEC also completed in 2004 the national university entrance examination. The establishment of the new education assessment and examination capacity, as well as the first assessments and examinations are highly satisfactory. They assure improved system accountability (assessments) and transparency (examinations). Even though the Center is independent, it has an important public purpose and needs to be continually financed from the public budget. 4. School Rehabilitation (US$9.53 Million): The project targeted and completed the rehabilitation of 44 schools with about 22,000 school places. Each school district participated in the project, and school selection was carried out based on a school mapping exercise. The school mapping was completed by the Ministry and considered as one of the key outputs of the project. The mapping software is used for all other possible investments, such as state budget and also by the donors involved in the education sector. However, the mapping software cannot and did not substitute for a properly functioning EMIS. The latter should have been able to monitor the significant internal migrations in the country and adjust the targeting of civil works to the changing need. In absence of a link with EMIS, school rehabilitation at times targeted schools with a declining student population, when priority should have been given to overcrowded schools. During the implementation of this component, allegations were made about possible mis-procurement or mishandling of funds. However, these allegations were not substantiated. - 10 - The Ministry also developed new standards for school construction and for school rehabilitation and has issued a decree that mandates these standards for the municipalities, which are taking over the responsibility of maintaining school infrastructure. Establishing and implementing the guidelines has strengthened the capacity at the Investment Unit of the Ministry, which, in turn, started to assist the local governments in their capacities of managing school infrastructure. Overall, the component output is satisfactory as it did achieve the targeted number of school rehabilitations, albeit with some delay, and set standards that are being used for continued rehabilitation to be supported by central, local and international funds. 5. Textbooks (US$3.04 Million): There are three significant outputs for the textbook component: First, a new, more liberal textbook policy was prepared and adopted by the Government. Accordingly, new training materials were developed and training provided to Ministry staff and to other stakeholders. Second, the privatization of the textbook publishing and distribution companies was nearly fully completed. Both companies were brought to the point of sale by April 2004 and the state textbook publishing company was sold in August 2004. Thirdly, the Ministry issued two tenders, one by using the IDA funds to publish 3,340,000 textbooks in 2003 through International Competitive Bidding for grades 3, 4, 5, 7 and 8 and one by using Government funds to publish textbooks in 2004 through National Competitive Bidding for grades 1, 2 and 6. Both were considered successful and both resulted in significant improvement of the physical quality of textbooks while the average price of textbooks declined from US$1.50 to US$1. Textbook Publishing Amount of Books Cost in Albanian Lek Cost in US$ Printed 2002 through State 3,760 thousand 150.33 Lek $1.50 monopoly 2003 through 3,340 thousand 99.12 Lek $1.00 Competitive system About 94 percent of students in the first seven grades of basic education received free textbooks, which are returned to the school to be used by three consecutive classes. The component is rated as satisfactory as a more competitive system resulted in improved quality and efficiency. 4.3 Net Present Value/Economic rate of return: No applicable. 4.4 Financial rate of return: Not applicable. 4.5 Institutional development impact: All the project components targeted and achieved significant institutional development impact. Given the weak public administration system, overly political selection and decision making, such a strong focus on institutional development is completely justified. Starting from 2004, education in Albania is being decentralized and that requires new roles and responsibilities at each level of education administration. Therefore, linking investment with institutional capacity building should continue. - 11 - At the central level, important steps were taken to make the MOES more accountable and performance, hiring and budgeting processes more transparent. While it cannot be expected that the institutionalized changes will be unequivocally accepted and the new procedures will all be functional and effective, the steps taken have been steps in the right direction. The new strategic documents and guidelines listed under 4.2 (1st component) will also guide development for the next 10-year period, including those that define practices for subnational levels, such as municipalities (school maintenance) and schools (curriculum standards). The other central agencies, including the Curriculum and Teacher Training Centers and the National Assessment and Examinations Center (NAEC) have all been given a new mandate and strengthened capabilities. Finally, the privatization of the textbook publishing and distribution business started to open up the sector for private providers. It is expected by the Ministry that the teacher training standards that are being developed in the near future, will also open up this area for the private sector. Overall, the institutional development impact of the project is substantial. 5. Major Factors Affecting Implementation and Outcome 5.1 Factors outside the control of government or implementing agency: The project implementation was not affected by any factors outside the control of the Government or implementing agency. This project did not have major disruptions, even though reforms and donor-funded project implementation in Albania are generally constrained by political instability. 5.2 Factors generally subject to government control: Government reshufflings were carried out at the beginning of the project and also after one year of the project implementation. Whereas the first reshuffling resulted in significant loss of continuity and institutional memory, the second reshuffling actually helped speed up implementation by bringing on board a new Minister who was strongly committed to the objectives and the project implementation. Other changes in the Ministry's key personnel also resulted in changes in strategic direction and lack of continuity. These changes caused some disruptions but did not have a significant impact. Finally, there were also several changes in the Project Implementation Unit, affecting the speed of project direction, sometimes negatively, sometimes positively. 5.3 Factors generally subject to implementing agency control: The PIU capacity was considered weak for a significant period of project implementation. Specifically, the procurement capacity was problematic and required special actions, aiming at its reinforcement. The PIU staff changes, which followed the project mid-term review (MTR), brought positive results in the overall project implementation pace. Following the mid-term review, the PIU hired an implementation specialist that helped speed up implementation. The project implementation was also affected by allegations of inappropriate handling of project funds, procurement or other funds used under the same Government development program. One PIU Director resigned and investigations were carried out, but these investigations did not substantiate the allegations until this report was completed. 5.4 Costs and financing: Disbursement was only 12 percent at the mid-term. After the mid-term review, the disbursement speeded up but an acceptable disbursement rate required intensive implementation support. Changes in the exchange rates resulted in 7.3 percent gains in the total available IDA credit (XDR9 Million, or US$13.66 Million during the implementation period). The disbursed IDA credit was 3.2 percent higher than the amount estimated at appraisal, whereas the Government reported a 5.2 percent increase of its contribution relative to the estimates. No problems were encountered with Borrower contributions. By the closing of the project, the total project costs -- including IDA disbursement and Government contribution -- increased by 3.6 percent compared to the estimates. Still, by project closing, about 9.3 percent of the IDA funds -- available - 12 - from the exchange rate gains -- were left unspent. 6. Sustainability 6.1 Rationale for sustainability rating: The project's overall sustainability rating is likely. Sustainability needs to be analyzed at four levels: (1) what are the systemic implications of the project for future Government activities in the sector; (2) what is the technical, institutional impact of the project; (3) financial issues: whether the efficiency gains can be sustained; and (4) economic feasibility: whether the development priorities set up by the project are likely to be provided. 1. The key issue for the Government is how the project affected the future conditions and performance of the education system and whether the strategic direction taken can be adequately financed. Presently, education is underfinanced relative to the country's income level and its overall public expenditures. However, given the absence of credible purpose -- i.e. how would the system's productivity change if more funding was provided -- increasing public funding had not been justified until recently. The new education strategy provides this needed direction. However, it is not enough to publish the strategy, it is also necessary that the strategy becomes a guidance for the key agencies responsible for service delivery. Also, increasing compulsory education to 9 years, addressing the deferred investments into the capital stock and establishing a more accountable system (via assessments and EMIS) are an adequate basis for increased public financing, especially in a country where the student-age population is stable or increasing for the next 10 years. The financial plans of the education strategy need to include the recurrent expenditures of an expanded system and these will have to be reflected in the medium-term expenditures. We need to emphasize that there is no assurance for increased funding of the sector but there is a stronger justification for it as a result of the project. The Government ownership altogether is substantial. 2. The question at technical/institutional level is how the institutional capacities that have been set up or strengthened through the project will have been sustained in the future. l A new Ministry structure has been introduced but the Ministry continues to suffer from high staff turnover. l A satisfactory capacity for education assessment and examinations is in place, however, it is unclear whether the Government will continue to finance the new center. l The new textbook policy, which introduced free textbooks for basic education, has already given some impact on improving access and quality of the system. This impact will only be sustained if the textbook publishing system remains transparent and competitive. l Another key area is the school rehabilitation. With the new decentralization policy in place, which transfers the implementation of capital investment to local government and also work carried out so far by the MOES on introducing the new guidelines and standards to the local government; even this activity is likely to be sustained. Nevertheless, this is tempered by the current capacity of local government and also the fact that the new decentralization policy is still at very early implementation stages. Overall, sustainability of the increased technical/institutional capacities is substantial. 3. The financial implications of the project are modest, even at some levels, some achievements are impressive. The project started some activities that could potentially bring efficiency gains. The cost of textbook production has been reduced by 33 percent. Standards and guidelines of rehabilitation will make it easier to plan continued school rehabilitation and minimize overspending. - 13 - 4. Albania will continue to rely on donor support if it intends to bring its education system to a level that is compatible with European standards. The implementation of the (pre-tertiary) education strategy will require about US$200 million over the next 10 years or about US$20 million per year. This is roughly 10 percent of the present and future public education expenditures spent from internal sources for investments and recurrent expenditures. While 10 percent for non-recurrent expenditures is more than what the present system can afford, the Government already made, for 2005, a commitment of US$10 million for continued school rehabilitation, which is 5 percent of its education budget and some 60 percent of the funds needed yearly for 10 years. The proper financial estimates and the Government commitment together constitute a good basis for a possible sectorwide approach with pooled donor funding. While the Government's commitment to continued school rehabilitation is impressive, its education expenditures at around 3 percent of GDP point to modest prospects for economic stability. 6.2 Transition arrangement to regular operations: There are few areas where transition arrangements are needed, since most of interventions took place within the context of pre-existing structures. 1. The education strategy, together with the activities envisioned for application for Education for All, funding will have to be transformed by the Government into an education development program with clear commitment from the Government for its own funding and reliable estimates for recurrent expenditures, plus a more detailed implementation plan. 2. Education decentralization needs significant institutional planning and strengthening. This requires that municipalities develop capacities for local school system planning, and standards are developed and approved for infrastructure maintenance. 3. One of the crucial issues is the future decision on the Center for Educational Assessments and Examinations (CEAE). The CEAE, created under the project, is an independent body, and provides valuable public service. However, funding from the budget is not yet assured. Therefore, an urgent decision needs to be made for its further financing through public funds. 4. A clear decision needs to be made on the purpose of EMIS use: monitoring of participation and performance of students or monitoring the supplies and efficiency. 5. Also, arrangements need to be considered for the newly established Curriculum and Teacher Training Centers. Both institutes, established under the project, would clearly need to be considerably strengthened. 7. Bank and Borrower Performance Bank 7.1 Lending: During the preparation phase, the Bank team carried out a number of important steps. It reviewed the lessons learned under the previous investment project and even considered a supplementary loan to complete all activities. It also carried out detailed economic sector work covering numerous key issues, i.e. economic analysis, institutional analysis, social assessment, and review of the vocational education sector, etc. Finally, it linked the project preparation to the overall Bank strategy in the education sector with the intention to bring Albanian education on par with its neighbors. As a result, it identified a relevant agenda for the project. However, this ICR has some reservations given the disconnect between the development objectives and outcome indicators, on the one hand, and project design, on the other. Together with the lack of measurable indicators and the lack of clear priorities, these problems contributed to confusion on the Government side, uncertainty about the objectives, slow implementation and disbursement, and persistent problems with project monitoring. For these reasons, the Bank's performance at lending was unsatisfactory. 7.2 Supervision: - 14 - The Bank carried out a mid-term review together with the Government, and decided that the project development objectives should not be changed. Instead, the Bank team reallocated the IDA funds between expenditure categories and components, started to provide a much more proactive assistance through more frequent supervision and daily contacts with the PIU and with the Ministry and extended the project closing date. As a result, implementation improved dramatically, due to (1) more extensive Bank assistance and (2) stronger Government ownership, which was also facilitated by the Bank team starting with a mid-term review workshop. The Bank performance was satisfactory during supervision. 7.3 Overall Bank performance: Overall, the Bank's performance was satisfactory since increased supervision compensated for the problems at the early stages and contributed to achieving the development objectives as well as to completing most assignments. Borrower 7.4 Preparation: At the preparation period, Government ownership was limited. The Ministry staff frequently changed, resulting in very little capacity or institutional memory to follow up on previous experience. Neither the Ministry nor the Government identified key policy priorities or an agenda that would be clear and realistic for all stakeholders. The limited ownership was also reflected in that the Borrower did complete in time all the preparatory work, required prior to Board approval. Therefore, the Borrower's role in designing the project was minimal and their performance at the lending phase was unsatisfactory. 7.5 Government implementation performance: The project status was not satisfactory at the beginning of the implementation, with all the components having problems and disbursement delays. The most serious delay -- from the MOES side -- was in the context of developing a priority list of schools for rehabilitation, based on the school mapping exercise. Since this was the precondition for starting the implementation of the school rehabilitation component, considerable delays occurred in the project implementation phase. Delays also happened in revisiting the strategy on textbook publishing houses. Frequent changes among senior MOES staff had a negative impact on its performance, especially at the early days of project implementation. However, starting from mid-term review, the MOES significantly improved its overall performance, especially after a Government reshuffling brought a new Minister on board. Efforts were made to strengthen the capacity of the staff, which was reflected in achieving positive results in all but one component. As a result of these significant improvements, the Government implementation performance is rated satisfactory. Now, that the capacities in the Ministry have increased, an education strategy is in place, there is a good reason to expect that the problems faced by the ERP will not present major constraints in the planning of new projects and during future interventions in the education sector. 7.6 Implementing Agency: The Project Implementation Unit performed well at the start, but at mid-term, both the director and the procurement officer were replaced and the unit had serious capacity problems until finally an international advisor and a senior procurement specialist were brought on board. Also, from mid-term, stronger Government oversight helped to improve project implementation. Altogether, the PIU's performance has to be rated satisfactory,given that it responded to all the emerging challenges and completed project implementation in good shape. 7.7 Overall Borrower performance: The overall Borrower performance is rated satisfactory mostly because by project closing, the project was able to achieve its intended goals, complete all activities, and reach the expected outputs. Whereas there were challenges at the beginning, the problems were addressed and eventually both the MOES and the PIU - 15 - appeared to be on track. 8. Lessons Learned Lessons for project planning: General: l Project activities and investments should be more closely linked to the development objective. Even with the expectation that institutional capacities increase overtime, investment projects in the education sector need to be focused, with realistic and measurable development indicators. l Key performance indicators need to be more thoroughly and realistically planned. Stronger Governmental ownership and adequate monitoring arrangements need to be carefully established at the lending phase. l Given the upcoming decentralization, institutional capacity building should strengthen its focus on sub-national levels, including schools and municipalities. Schools need capacities and incentives to strengthen service delivery and municipalities to manage the local school system. l The next project should be linked with the education strategy. The Government contribution to investment components within the project should increase as they need to provide their own investment into implementing a development program. Also, some activities that are necessary to be financed beyond the project's lifetime, should be financed by the Bank or other donors on a declining scale. l Ensure that TA, if required, is provided early and there are appropriate measures for the Ministry and other local agencies to absorb TA input. Education Sector specific: l Investment into infrastructure should continue but it needs to be linked to reallocation of resources between school districts to favor those where migration is oriented. Also, future projects should better target overcrowded schools within school districts. l The purpose and specification of EMIS need more focus. The first and foremost issue is student migration whereas other aspects (such as education inputs, processes and outcomes) could gradually be added to the information system. In the center, capacity building in finances has to be strengthened. Formula financing, sustained financial planning and accountability instruments (inspectorate functions) need stronger focus in the Ministry. l Community participation in managing school-based activities can be achieved through demand-driven interventions. These appear to be also effective instruments in achieving measurable improvements in key indicators such as decreasing absenteeism, increasing participation, and even improving learning performance. l Investment into curriculum needs to be continued for the next five to complete the reform for compulsory education, even if Bank financing into some aspects (i.e. teacher training, textbook publishing) should be on a declining scale. In this program, the curriculum of various knowledge groups should continue to be broadly based, consolidating narrowly defined subjects and continuing to integrate new pedagogical methods focusing on skills rather than factual information. Both pre-service and in-service training need to be adjusted to the new curriculum, as teachers' skills and qualifications need to be broader overarching several subject areas. In-service training and textbook production should follow suit. Lessons for project implementation: l Integration of project implementation in the MOES: Project activities should be implanted and implemented in the relevant departments in the MOES, while some fiduciary activities could be carried - 16 - out by an intergovernmental project unit (under the Ministry of Finance). l The investment projects should have clear implementation plans/operational manuals by the beginning of project implementation. Such manuals need to be fully integrated into the regular activity plans of the relevant MOES units. l Supervision missions need to give more emphasis to monitoring outputs and outcomes. 9. Partner Comments (a) Borrower/implementing agency: "The World Bank Project is considered to be successful by all measures, unparallel to other donor-funded projects. By contrast, its success cannot even be matched by the previous World Bank Project, which, by comparison, falls short of the size and success achieved with the current project. Admittedly, a huge number of both local and international experts were ideally involved with this project. A significant role was played by the education team attached to the World Bank Mission in the successful completion of the project. The project drew to an end within the set deadlines. In the first two years, the project got off to a bad start, with the real developments not becoming effective until 2002 and in the years 2003-2004, a time period that speeded up the attainment of the set targets." Lessons learned l "The conceiving stage of the project was rather complex and mixed: Therefore, issues have come up during its implementation, mainly related to legal, political, organizational, and management areas in component 1 as well as concrete matters technically oriented into components 2-5. l In view of the reform initiatives taken by the Government, the endless and consistent efforts in the education sector (such as EFA/FTI) as well as the outcomes in the implementation of the component, MOES will keep focusing on organizational and legal reforms and other initiatives to follow. In addition, there is a need for efforts to be made in terms of technical orientation in the curriculum development, teacher training and the preparation of school textbooks and other teaching materials, publishing, printing and distributing textbooks. Furthermore, the attempts will continue in rehabilitating schools and participating in national and international evaluations. These could well be considered as top priorities to be considered and supported in the future." See in Annex 8 the full text of the MOES comments. (b) Cofinanciers: (c) Other partners (NGOs/private sector): 10. Additional Information - 17 - Annex 1. Key Performance Indicators/Log Frame Matrix Outcome / Impact Indicators: 1 Indicator/Matrix Projected in last PSR Actual/Latest Estimate 1. Better ministerial leadership and Improved leadership and strengthened A stronger, more qualified and proactive management of sector reform. management capacity in the MOES. leadership and management team for the MOES is in place and is functioning more effectively. Significant organizational structural changes have been implemented at the central-MOES and its agencies/institutions. Also, there are new standards, audit and competitive hiring procedures. In conjunction with overall guidelines and legal/regulatory reforms of the GOA, steady progress has been made with regard to decentralization and devolution to the reorganized district-level. Education sector reform is driven by long-term education strategy, decentralization strategy and Education for All strategy. 2. More civic support for sectoral reform. Increased proportion of population satisfied The results of M&E study indicated that the with the education system. majority of the public was aware of main project activities and expressed high satisfaction. School rehabilitation and new basic school furniture are viewed positively, as is the increased availability of reprinted textbooks. School rehabilitation received local support, education performance in areas receiving support improved. 3. Statistical and information databases: (a) (i) Department of Information and Statistics Department of Information and Statistics was used for decision-making by policy established; (ii) data are available, routinely established and functioning. The First makers; (b) routinely updated; and (c) updated and used for decision-making; and Statistical Report was published; but will regularly published in the form of education (iii) Annual Statistical Report Card need further improvement since it was not indicators for the broader public in Statistical published. linked to the EMIS and the school mapping Report Card on Albanian Education. program. Investments in hardware and software have resulted in technically sound systems. However, full implementation and on-going use of the EMIS has not been realized at the MOES and within the pilot districts. 4. More rational and persuasive budgetary Improved budgetary process for the Public expenditure tracking analysis was processes for the sector. education sector. carried out, medium-term expenditure framework was established, financial estimates for long-term education strategy were prepared, financial implications of some reform steps (textbooks, assessments) were made. Education budget for 2005 includes line for continued school rehabilitation. 5. Increased and better-used international Increased and better used-international aid International aid resources can be linked to aid resources. resources. programmatic development agenda and some key reform steps have been budgeted. Meanwhile, the international aid resources have decreased, but this is independent from the project implementation. - 18 - Output Indicators: 1 Indicator/Matrix Projected in last PSR Actual/Latest Estimate 1. Strategy document and annual work plan MOES will complete the education sector The MOES completed the education strategy acceptable to IDA. strategy for sectoral reform, including the for 2004-2015 and a sector plan for the education sector plan for the EFA FTI Education for All application. application. 2. Establishment of Council of the MOES The Council of Ministry to be established and The Council was established and functioning. that functions as priority/setting program reviewing all strategic documents. The designing policy body. Council of Ministry to be established. 3. Establishment of a Policy Analysis and PAPU established and taking lead role in PAPU was established in 2003 and Planning Unit (PAPU) in the MOES: developing the strategic documents. functioning. 4. Institutional/Peer Review of the Institute The MOES approves the proposed new A review of the IPS completed. The IPS was for Pedagogical Studies (IPS); and a more structures of the IPS. restructured based on the recommendations effective organizational structure of the IPS is of the review, dividing it into Curriculum and in place to carry out reforms for curriculum Training Centers. and teacher education in primary and secondary education. 5. Curriculum review and the design of a A curriculum reform plan has been prepared (i) National curriculum concept was curriculum reform plan. and approved by the MOES; and the IPS was developed by the Curriculum center and restructured and strengthened to carry out adopted by MOES. Curriculum framework for the implementation of curriculum reform. Grades 1 - 8 was developed and published by July 2004; and Syllabi for Grades 1 and 6 was piloted/implemented during 2004-2005 6. Completion of teacher training for Curriculum for teacher training (information Training was not carried out before project informatics teachers in secondary education. technology) prepared; and 150 teachers are closing. trained in information technology. 7. Pilot project and data collection and Pilot project and data collection and querying The system was established; but it is not yet querying system in place 12 months after system. functioning due mainly to the shortage of project effectiveness. trained staff. 8. General administrative software (4 General administrative software (4 modules) The software was installed. modules) developed 24 months after project developed 24 months after project effectiveness. effectiveness. 9. Analytic and policy support software Analytic and policy support software Software was installed; and staff were developed and institutionalized by project developed and institutionalized in MOES. trained, but the software was not functioning end. by the end of the Project. 10. Public contact point (PCP) operational The PCP is operational and staffed; and PCP operational but the majority of the public 24 months after project effectiveness. used by the public to access information were not aware of PCP and did not use the provided by MOES. system. 11. Establishment of the Department of (i) DIS established and functioning; (ii) 6 staff DIS was established and functioning, but it Information and Statistics (DIS), which is hired and trained; and (iii) Publication of first lacked the capacity in terms of system adequately staffed and supported by project "Statistical Report Card". development, planning and implementation end. and management. The First Statistical Report was prepared and printed, but it was not linked to EMIS and Map Decision program. 12. Assessment and Examinations Center is Completion of (i) a pilot sample assessment NCAE established and operational. It carried established and internationally certified as of grade 4 students in 3 subjects; and (ii) a out a pilot sample assessment. In addition, professional, and press clips of results. nationwide sample based assessment of the Center carried out a national college grade 4 student achievement in 3 subjects. entrance examination; and a grade 8 (matura) examination. 13. New 8th grade examination conducted Completion of grade 8th examination and The Center completed the 8th grade national and graded. dissemination of the results. examination on math, literature, science and Albania language. 14. New matura pretested. New matura prepared and pretested by the The Center completed the 12th grade end of Project. national examination on math, literature, science and Albania language. 15. Development and administration of Preparation and completion of a new The Center completed a national university university entrance examinations by NCAE. university entrance examination by NCAE -- entrance examination. for a more objective, equitable and effective - 19 - implementation of student admission policy. 16. 35,000 school places (about 44 schools) Rehabilitation and refurbishment of 35,000 44 schools were rehabilitated and refurbished rehabilitated and refurbished. school places. by the end of the Project. 17. Minister of MOES publishes decree to Completion of standards for school School standards were completed under the adopt new design, construction, and construction and refurbishment; and adoption project; and used in the design of school quantities guidelines for all construction and of new design, construction and quantities rehabilitation projects. rehabilitation activities: end of year 1. guidelines for all construction and rehabilitation activities. 18. MOES and MOP bring STPH and TDE STPH and TDE to be brought to point of Both agencies were brought to the point of to point of sale according to timelines sale. sale. By the end of the Project, STPH was specified in detailed plan of action. privatized; and the privatization of TDE was postponed due to large debts. 19. Development and adoption of a new A new textbook policy for primary and A new textbook policy was prepared and textbook policy for primary and secondary secondary education completed and adopted. adopted. education. 20. (i) Provision of core textbooks in basic (i) Provision of textbooks for all basic (i) Textbooks for all basic education students education (grades 1-8); and (ii) 20% of basic education students in grades 3, 4, 5, 7, and 8 in grades 3, 4, 5, 7, 8 under the Project were education students (G1-8) with "free" core under the Project; and (ii) about 95% provided; and (ii) about 94% students in textbooks increased from 80% in 2000 to students in grades 1-8 will access to free grades 1-7 receive free textbooks (according 95% in 2004. core textbooks. to a survey of 186 household members). 1End of project - 20 - Annex 2. Project Costs and Financing Project Cost by Component (in US$ million equivalent) Appraisal Actual/Latest Percentage of Estimate Estimate Appraisal Component US$ million US$ million 1. Policy Design and Management 3.56 1.56 0.38 2. Education Management Information System 0.78 0.72 0.77 3. Assessment/Examinations 0.54 0.56 0.93 4. School Rehabilitation 6.52 9.53 1.16 5. Textbooks 2.44 3.04 1.05 Total Baseline Cost 13.84 15.41 Physical Contingencies 0.73 0.00 Price Contingencies 0.31 0.00 Total Project Costs 14.88 15.41 Total Financing Required 14.88 15.41 Comment: The 3.6 percent increase in total project costs is caused by exchange rate gains on the XDR 9 Million IDA during the implementation period. Project Costs by Procurement Arrangements (Appraisal Estimate) (US$ million equivalent) 1 Procurement Method Expenditure Category ICB NCB 2 N.B.F. Total Cost Other 1. Works 1.40 2.70 1.90 0.00 6.00 (1.10) (2.00) (1.40) (0.00) (4.50) 2. Goods 3.50 0.00 0.20 0.00 3.70 (3.00) (0.00) (0.10) (0.00) (3.10) 3. Services 0.00 0.00 4.30 0.00 4.30 (0.00) (0.00) (4.20) (0.00) (4.20) 4. Incremental Operating 0.00 0.00 0.50 0.40 0.90 Costs (0.00) (0.00) (0.20) (0.00) (0.20) 5. Miscellaneous 0.00 0.00 0.00 0.00 0.00 (0.00) (0.00) (0.00) (0.00) (0.00) 6. Miscellaneous 0.00 0.00 0.00 0.00 0.00 (0.00) (0.00) (0.00) (0.00) (0.00) Total 4.90 2.70 6.90 0.40 14.90 (4.10) (2.00) (5.90) (0.00) (12.00) 1/ Figures in parenthesis are the amounts to be financed by the Credit. All costs include contingencies. 2/ Includes Minor Works (aggregate US$1.9 million) of less than US$100,000; International Shopping (aggregate US$0.2 million) of less than US$100,000; National Shopping (aggregate US$0.04 million) of less than US$25,000; Direct Contracting (aggregate US$0.05 million) for purchase of proprietary items for reasons of compatibility and standardization. Inncremental Operating Costs (aggregate of US$0.9 million), financed by IDA on a declining basis, relating to managing the Project. Comment: The US$0.02 difference between the planned project costs by components and by procurement arrangements is based on the PAD estimates. - 21 - Project Costs by Procurement Arrangements (Actual/Latest Estimate) (US$ million equivalent) 1 Procurement Method Expenditure Category ICB NCB 2 N.B.F. Total Cost Other 1. Works 0.00 7.70 0.00 0.00 7.70 (0.00) (5.45) (0.00) (0.00) (5.45) 2. Goods 3.64 0.00 0.84 0.00 4.48 (3.05) (0.00) (0.75) (0.00) (3.80) 3. Services 0.00 0.00 3.00 0.00 3.00 (0.00) (0.00) (2.90) (0.00) (2.90) 4. Incremental Operating 0.00 0.00 0.05 0.00 0.05 Costs (0.00) (0.00) (0.05) (0.00) (0.05) 5. Miscellaneous 0.00 0.00 0.11 0.00 0.11 (0.00) (0.00) (0.11) (0.00) (0.11) 6. Miscellaneous 0.00 0.00 0.07 0.00 0.07 (0.00) (0.00) (0.07) (0.00) (0.07) Total 3.64 7.70 4.07 0.00 15.41 (3.05) (5.45) (3.88) (0.00) (12.38) 1/Figures in parenthesis are the amounts to be financed by the IDA Credit. All costs include contingencies. 2/Includes civil works and goods to be procured through national shopping, consulting services, services of contracted staff of the project management office, training, technical assistance services, and incremental operating costs related to (i) managing the project, and (ii) re-lending project funds to local government units. Project Financing by Component (project costs are inclusive of physical and price contingencies) (in US$ million equivalent) Percentage of Appraisal Component (project Appraisal Estimate Actual/Latest Estimate costs are inclusive of physical and price contingencies) IDA Govt. CoF. IDA Govt. CoF. IDA Govt. CoF. 1. Policy Design and 3.37 0.22 0.00 1.50 0.06 0.00 44.5 27.3 0.0 Management 2. Education Management 0.63 0.17 0.00 0.59 0.13 0.00 93.7 76.5 0.0 Information System 3. 0.43 0.14 0.00 0.44 0.12 0.00 102.3 85.7 0.0 Assessment/Examinations 4. School Rehabilitation 5.40 1.90 0.00 7.28 2.25 0.00 134.8 118.4 0.0 5. Textbooks 2.17 0.45 0.00 2.57 0.47 0.00 118.4 104.4 0.0 TOTAL 12.00 2.88 0.00 12.38 3.03 0.00 103.2 105.2 0.0 Comment: Based on the XDR9 Million IDA, USD13.66 Million was available for the ERP and USD1.27 Million was left undisbursed, which is 9.3 percent of the available total IDA credit. - 22 - Annex 3. Economic Costs and Benefits - 23 - Annex 4. Bank Inputs (a) Missions: Stage of Project Cycle No. of Persons and Specialty Performance Rating (e.g. 2 Economists, 1 FMS, etc.) Implementation Development Month/Year Count Specialty Progress Objective Identification/Preparation 11/08/1999 PTL (1); OPERATIONS SPECIALIST (1); CONSTRUCTION AND PROJECT IMPLEMENTATION SPECIALIST (1); PUBLIC ADMINISTRATION SPECIALIST (1); LEARNING AND ASSESSMENT AND EXAMINATION SPECIALIST (1); PROJECT OFFICER (1); PAS (1) FMS (1); TEAM ASSISTANT (1). Appraisal/Negotiation 02/18/2000 TTL (1); OPERATIONS OFFICER (1); PROJECT OFFICER (1); PAS (1); FMS (1); ARCHITECT, CONSULTANT (1); INFORMATION SPECIALIST, CONSULTANT ( 1). Supervision 10/02/2000 9 TTL (1); SR. OPERATIONS S S OFFICER (1); PROJECT OFFICER (1); SR. PSM SPECIALIST (1); CONSULTANTS (3); FMS (1); PAS (1) 05/09/2001 1 TTL (1) S S 10/08/2001 8 TTL & EDUC. SPECIALIST (1); S S TTL & EDUC. ECONOMIST (1); SR. OPERATION SPECIALIST (1); LEAD PUBLIC SEC MAN SP (1); PAS (1); CONSTRUC & IMPLEM SPC (1); ASSESSMENT SPECIALIST (1); PROJECT OFFICER (1) 02/11/2002 4 TEAM LEADER (1); S S OPERATION OFFICER (1); PROCUREMENT OFFICER (1); - 24 - SECTOR MANAGER (1) 05/17/2002 4 TEAM LEADER (1); S S OPERATIONS OFFICER (1); OPERATION OFFICER (1); PROCUREMENT SPECIALIST (1) 05/17/2003 5 TEAM LEADER (1); SR. U S OPERATIONS OFFICER (1); SR. PROCUR SPECIALIST (1); OPERATIONS OFFICER (1); SR. FINAN MANAG SPECIA (1) 07/19/2003 4 TEAM LEADER (1); SR. S S OPERATIONS OFFICER (1); SR. PROC./IT SPECIAL (1); OPERATIONS OFFICER (1) 05/04/2004 4 TEAM LEADER (1); U S PROCUREMENT/MIS SPEC (1); IMPLEMENTATION/ASSESS MENT CONSULTANT (1); OPERATIONS OFFICER (1); JPA (1) ICR 02/07/2005 2 TASK TEAM LEADER (1); OPERATIONS OFFICER (1) (b) Staff: Stage of Project Cycle Actual/Latest Estimate No. Staff weeks US$ ('000) Identification/Preparation Appraisal/Negotiation 53.74 157,648.74 Supervision 136.16 243,542.07 ICR 17.86 15,313.66 Total - 25 - Annex 5. Ratings for Achievement of Objectives/Outputs of Components (H=High, SU=Substantial, M=Modest, N=Negligible, NA=Not Applicable) Rating Macro policies H SU M N NA Sector Policies H SU M N NA Physical H SU M N NA Financial H SU M N NA Institutional Development H SU M N NA Environmental H SU M N NA Social Poverty Reduction H SU M N NA Gender H SU M N NA Other (Please specify) H SU M N NA Private sector development H SU M N NA Public sector management H SU M N NA Other (Please specify) H SU M N NA - 26 - Annex 6. Ratings of Bank and Borrower Performance (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HU=Highly Unsatisfactory) 6.1 Bank performance Rating Lending HS S U HU Supervision HS S U HU Overall HS S U HU 6.2 Borrower performance Rating Preparation HS S U HU Government implementation performance HS S U HU Implementation agency performance HS S U HU Overall HS S U HU - 27 - Annex 7. List of Supporting Documents · "Education Reform Project - Development Credit Agreement", Credit Number 3343, International Development Association, June 2000 · "Education Reform Project ­ Project Appraisal Document", The World Bank, April 2000 · "Education Reform Project - Supervision Mission Aide Memoire", World Bank, Tirana, October 8-12, 2001 · "Education Reform Project - Supervision Mission Aide Memoire", World Bank, Tirana, February 6-11, 2002 · "Education Reform Project - Supervision Mission Aide Memoire", World Bank, Tirana, July 1-7, 2002 · "Education Reform Project - Mid Term Review Supervision Mission Aide Memoire" ­World Bank, Tirana, November 4-12, 2002 · "Education Reform Project - Supervision Mission Aide Memoire", World Bank, Tirana, July 14-18, 2003 · "Education Reform Project - Supervision Mission Aide Memoire", World Bank, Tirana, April 13-16, 2004 · "Education Reform Project - Supervision Mission Aide Memoire", World Bank, Tirana, November 8, 2004 · "Final Consultant Report on the Curriculum Development and Implementation Component", MOES, Dr. Alexandru Crisan, February 2004 · "Final Draft Report on the Performance Standards of the MOES and Evaluation Plan", October 2004 · "Final Report on Technical Assistance to Develop and Strengthen Teacher Training Agency", MOES, Dr. Frank Schorn, October 2004 · "Public Expenditure Tracking Survey", MOES, C. Tibi, October 2004 · "Report on Performance Review", MOES , A. MacDonald, R. D'Aoust, C.Harison, May 2003 · "Report on EMIS Evaluation", MOES, PIU, October 2004 · "Assesment of the Implementation of the Education Management Information System Component", MOES, E. Sheshi, R. D'Aoust, April 2004 · "School-Leaving Examination of the Compulsory Education - Mathematics and Albanian Language" Academic year 2002-2003, MOES, NAEC, March 2004 - 28 - · "Literacy Skills for the World of Tomorrow ­ Further Results from PISA 2000", UNESCO Institute for Statistics, OECD · "Albania's National Assessment and Evaluation Center: Progress, Change, Challenge and Needs", Final Report, MOES, Dr. R. Altman, October 25, 2004 · "First National Assessment of Student's Achievement for the Fourth Grade", MOES, NAEC, Tirana- 2003 · "Report on the Consultant Services to Establish the NAEC in Albania", MOES, June-July 2002 · "Final School Project Rehabilitation Report", ILIRIDA SH.P.K, Tirana, May 2004 · "Report on a Visit to Albania to provide support to Establish a Textbook Approval Board for the MOES" , T. Horsler, May 2004 · "Final Report on School Textbook Provision", MOES, R. Boulton, October 2004 · "Annual Statistical Report 2002-2003", MOES, 2004 · "Strategy of Pre-University Education in Albania 2004 - 2015", MOES, 2004 - 29 - Additional Annex 8. Ministry of Education and Science Comments on the Education Reform Project I. Introduction This document is the completion report of "Education Reform" Project (No. 3343-ALB), prepared by the Ministry of Education and Science. The project became effective as of July 2000, and was completed October 31, 2004. (The initial completion date was July 31, 2004, which was later postponed to October 31, 2004.) The report includes information on: (i) project goals and component design; (ii) project activities and an evaluation of the performed activities; and (iii) the sustainability of the results, and learnt lessons from the project which can be used in future projects. Also, the Appendices give financial and procurement information. II. Project Development Objective The project development objective was to assist the MOES in planning and managing education services and strengthening its accountability towards the stakeholders in this sector. III. Project Components The project has five components: 1. Policy Design, Management and Improvement of Education System; 2. Education Management Information System; 3. Assessment and Evaluation; 4. School Rehabilitation; and 5. Textbooks. Component 1: Policy Design, Management and Improvement of Education System: This component supports the institutional strengthening and the organizational reform. The component was managed by MOES Council and funded the revised functions that are required by the Ministry to enhance the sector of education: policy analysis, strategic planning and management, human resources management, and implementation and monitoring of specific programs of education reform. The main expected results of this component were: (a) the establishment of the MOES Council functioning as priority-setting and policy-making body; (b) the establishment of a planning and policy analysis unit, which supports MOES Council as a technical secretariat; (c) development of capacity and tools for financial analysis; (d) completion of a reform plan for the education sector, based on specific criteria; (e) completion of at least on model which meets the specified criteria; (f) development of an annual action plan for the reform as the criteria; and (g) establishment of project implementation unit at the Ministry. Component 2: Education Management Information System: This component is based on the development of a database and institutional strengthening in this area. The component has been managed by a department recently established at MOES (Information and Statistics Department). This component would support an education management information system, which would link all the branches of the ministry, including regional education departments and universities. Main results of the component would be: (a) a simple MIS which could be easily expanded, and it would include the collected data, application and users as well as the completed test at, at least, two pilot - 30 - educational offices of two regions, to be used by Ministry decision-makers on ongoing basis, and (b) the publication of the statistical yearbook for education in Albania (the first report). Component 3: Assessment and Evaluation: This component was managed by the National Center for Assessment and Evaluation, under MOES and was expected to reach the objectives upon the completion of the Education Reform Project. Main results include (a) the development and management of university admission test, based on international standards, (b) management and evaluation of the result fourth-graders' achievement in the subject of Albanian language, mathematics, and sciences, and the publication of the results, and (c) preparation for the activities that would be completed under a project in the future, including national test management for eighth and twelfth grades. Component 4: School Rehabilitation: This component funds: (a) the development of building and design guidelines and specification of bills of quantities for schools and furniture; and (b) the rehabilitation of about 22,000 school places (about 44 schools). The selection of the schools was done based on the school mapping data-base to ensure an equitable distribution. Project funds were allocated within districts according to each district's rank-order. The key outputs for this component are: (a) building and design guidelines and specification of bills of quantities for schools and furniture developed; (b) 22,000 school places rehabilitated, using the new design guidelines and specifications; (c) a transparent and rationalized model to allocate school rehabilitation financing among districts applied; (d) private architectural firms or NGOs used for design and quality assurance functions; and (e) communities participation in the rehabilitation of all schools and their grounds in limited ways. Component 5: Textbooks: This component is a policy reform and physical goods component and was managed by the MOES. The component funds: (a) procuring textbooks, using international competitive bidding to reduce the costs and increase the quality of the textbooks being purchased by the Government; and (b) publishing textbooks for the new curricula in basic education as a trial for privatizing publishing. The key outputs for this component were: (a) STPH and TDE brought to the point of sale; supported also by appropriate technical assistance, (b) ensuring the transparent contracting procedures for textbooks publication, (c) textbooks procured through International Competitive Bidding (ICB); and (d) on the basis of savings realized through ICB, revisiting the percent of total textbook costs paid by the Government versus parents. IV. PROJECT ACTIVITIES IV.1. Policy development, education system management and improvement. This component had three major directions: (i) policy development, (ii) restructuring of educational institutions, and (iii) building of professional capacities. Policy development: 1. Curricula Reform. (02/2003 -06/2004). Under the assistance of an international consultant, basic concepts of a deeply reformed curriculum were elaborated, the action plan for the expected development of the curriculum was prepared, and in cooperation with the Institute for Curricula and Standards, the curriculum framework was developed. 2. EFA/FTI Draft. (05/2004). The MOES proposal for EFA was completed. The proposal states the - 31 - necessary reforms in the education system and the adequate financial resources to achieve, by 2015, a qualitative 9-year education for all. 3. Education Strategy. (05/2004 ). The development of the Strategy for Pre-University Education was completed, which was then approved by the Government in June 2004. This document outlines the governance reform for teaching quality enhancement, education financial reform, and the ways for building capacities and human resource development. The document includes an action plan for the implementation of the Strategy, and cost estimation. 4. Public Expenditure Track Survey. (02/2004 ­ 10/2004). A Public Expenditure Track Survey was performed for education expenditures in two stages: local government jurisdiction, and school under the selected jurisdiction. 5. Education Decentralization, (08-09/2004). A Government Decentralization Policy Paper, has been prepared and adopted. Also, a relevant piloting plan was developed and is under implementation. Restructuring of education institutions: 1. MOES Restructuring. (10/2001 ­ 05/2003). A new administrative and functional structure for MOES was prepared and adopted by the government. The structure comprises new units, such as the Minister's Council, the Sector of Analysis and Planning, the Inspection Directorate, and the re-composition of previous units, such as the Curriculum Directorate, Human Resources Directorate, etc. 2. Enhancement of the Pedagogical Studies Institute (PSI). (2002). PSI structure was assessed and divided into two institutions: the Institute for Curricula and Standards, and the Center for Teacher Training, which were approved by the Government in September 2003. 3. Establishment of education departments in regions and education offices in districts. (03/2004 ­ 08/2004). MOES has deeply reorganized the district educational departments, transferring them to regional education directorates, and sub regional education offices. BUILDING OF PROFESSIONAL CAPACITIES 1. Performance standards (09/2004-10/2004). MOES has developed performance standards for all its departments and directorates. Moreover, an action plan is developed aiming at a continuous performance evaluation of these departments. 2. Capacity building for the staff of Teacher's Training Center. (09/2004-10/2004). A number of seminars and workshops were organized, including also the representatives from the regional educational offices. IV-II. Component 2: Education Management Information System (EMIS): In this component, the ERP funded support for continued development of a management information system, that would ultimately link all branches of the Ministry, including district offices and universities. Also, testing of the system with basic software applications, linking on a pilot basis in 2-3 District Education Offices, and training of staff at both levels of government, were supported too. The MOES established a new Department for Information and Statistics in January 2003. With the support of a technical assistance a number of steps were undertaken, such as: build a simple EMIS, design and develop a data collection system and database query system, provide user-friendly software that can be used to ask analytic questions of databases and receive answers that fit their needs, provision of turnkey legislative document management system (LDMS) for MOES. The DIS produced its first Annual Report in May 2004. The report contained detailed and useful statistical - 32 - data and indicators. Another effort by the MOES to share data with their stakeholders was the Public Contact Points located in the Ministry in Tirana. Public Contact Points are kiosks containing electronic folders with information about such items as tests data, rules and regulations, school calendars, and university admission information. However, the evaluation of this component (completed in April 2004), shows that there are concerns between the declared strategic intentions of the MOES Departments as to the direction and sharing of information and information systems between Departments, the Regional District Offices and the Department of Information and Statistics (DIS). The system has not fully succeeded to manage, develop, implement and sustain a single software application. The data base piloted by the EMIS systems (Map Decision) was not sustained by MOES Departments, the Regional District Offices or the DIS personnel. IV-III. Component 3: Assessment and Examinations: 1. Following the decision of Council of Ministers (25 October 2003), a National Assessment and Evaluation Center was established. With technical assistance support, the Center has progressed in terms of: (A) developing clearly its mid-term strategy, including: (a) legal status of NAEC within legal framework in Albania, (b) definition of role of NAEC, (c) internal organizational structure of NAEC, and (d) relationship between NAEC and other educational bodies in Albania; (B) Establishment of: (a) Public Report of the Grade 4 Albanian Sample (2002), (b) Technical Report on Data Preparation and Multilevel Modeling Procedures for the Grade Four Albanian Sample Study (2002), and Attachment 4: Report on Grade Four Testing. These were essentially draft documents applicable to the 4th grade learning assessment in May 2004. The Education Reform Project has funded (a) improvement in the current examinations administered at grades 8 and 12; (b) continued development and administration of high quality and fair entrance examinations to university, and (c) carrying out a sample-based assessment for grade 4 in three subjects that will provide baseline data for monitoring performance trends. Overall, the activities completed under the component were as follows: · Administration of grade 4 learning assessments in May 2002 and in May 2003; · Administration of new 8th grade examination in June 2003 and for the second time in June 2004; · Administration of new Matura (12th grade examination in June 2004); · Planning and conduct of national conference on education assessment in July 2004. IV-IV. Component 4: School Rehabilitation: The activities supported under this component were: (a) the development of a national school mapping, (b) building and design guidelines and specification of bills of quantities for schools and furniture; and (c) the rehabilitation of about 22,000 school places (about 44 schools), including their furniture. The rehabilitation works were supervised by a local company (Iliriada). However, in February 2004, the MOES terminated the contract, based on their poor performance and appointed a civil engineer and an architect among Ministry staff to persuade the implementation of the rehabilitation schools process. A detailed procedure for completion and handover of the rehabilitation works was developed and followed. At the completion, a "certificate of hand over" was prepared and was signed by: MOES, PIU, District Education Department, School Director and Contractor. - 33 - IV-V. Component 5: Textbooks: This component funded: (a) procurement of textbooks, using international competitive bidding to reduce the costs and increase the quality of the textbooks being purchased by the Government; and (b) publishing textbooks under the new curricula for basic education. Furthermore, the MOES, as agreed since the project effectiveness, achieved bringing the STPH (Textbook Distribution enterprise) to the point of sale (April 2004) and also completed the sale procedures for the Textbooks Publishing Company (by August 2004). Among key project outputs, it has to be noted the establishment of the "textbook publication Unit" in MOES. The Unit is responsible for the textbooks policy at national level, including publishing, printing and distribution of textbooks. Also, the Unit leads all the competitive procedures related to new curricula introduction. V. MOES standing on the project Achievements The World Bank Project is considered to be successful by all measures, unparallel to other donor-funded projects. By contrast, its success cannot even be matched by the previous World Bank Project, which, by comparison, falls short of the size and success achieved with the current project. Admittedly, a huge number of both local and international experts were ideally involved with this project. A significant role was played by the education team attached to the World Bank Mission in the successful completion of the project. The project drew to an end within the set deadlines. In the first two years, the project got off to a bad start, with the real developments not becoming effective until 2002 and in the years 2003-2004, a time period that speeded up the attainment of the set targets. 1. Policy Development, Management and Improvement of the Educational system Among the main outputs of component 1, carried out in this project, which were conceived and designed in the special initiatives context, we could list the following: · Development of "National Strategy of Pre-university Education 2004-2015"; · Decentralization of functions, responsibilities and accountability in the education system; · Draft proposal regarding participation in EFA/FTI; · Structural arrangements within MOES; · Establishment of Curricula and Standards Institute; · Establishment of Training and Qualification Center; · Establishment of National Educational Center for Evaluations and Examinations; · Structuring and organizing the Regional Educational Authorities and the Educational Offices; · Reforming the education inspectorates. Under the assistance of World Bank the " National Strategy of Pre-university Education 2004-2015" was conceived and developed accordingly, a strategy which outlines an important generalization of the orientations and steps to be taken towards developments of pre-university education in the 10 next years. l Upon approval of Decision by Council of Ministers no. 538, dated 12.08.2004, this document becomes a work agenda awaiting implementation. It perpetuates the governance reform in boosting quality of teaching-learning process, the financial reform in education, and the means of building capacities and human resources. The document also contains an implementation plan of the strategy as well as the incurring costs. l The Decentralization Policy Paper was developed in conjunction with the Decentralization Policy Pilot - 34 - Project of the education services in Municipalities and Districts across the country. In this document the joint responsibilities and competences were split equally among the Local and Central governments in the field of pre-university education, aiming at transferring/devolving decision-making closer to the services, increasing quality of school as well as transferring responsibilities and authority in decision-making for the management and financing of pre-university education;. In the framework of decentralizing functions in pre-university education, in anticipation of 2005, the funds earmarked for investments in pre-university education have been devolved to the local government entities, in districts and center district municipalities. l With the assistance of consultants at the World Bank, the application template/formats in EFA/FTI for MOES were developed. This proposal sets out the necessary reforms to be taken in the educational system, as well as the financial resources badly needed in order to attain a quality 9-year Education for All by 2015. The proposal relies heavily on an overall analysis/scrutiny of the conditions in achieving the 2015 target as well as in estimating the costs on the basis of a very sophisticated financial model. l Establishing the Advisory Board next to the Ministry of Education and Science, which include representatives coming from the academic circles, education unions etc. This advisory board discusses and analyses the strategies and priorities in education at all times supported by the Policy Development Sector at MOES, which by assuming the attributes of a technical secretariat helps in developing the necessary policies, planning and financial analysis. l Organizational structuring of MOES allowing for new structures within MOES to be established such as the Inspectorate Department, the Curriculum Development Department, the Department of Information and Statistics, which were lacking previously at MOES, the Policy Development Unit, and the Textbook Publication Unit. l Establishing new structures outside of MOES, such as National Educational Center of Evaluation and Examinations, the Institute of Curriculum and Standards as well as the Teacher-Training Center: a. Institute of Curricula and Standards in Education (ICS), as an institution specialized in developing curriculum and setting standards; b. Center of Training and Qualification in Education, as a management institution on training and qualifying the education system employees. Thanks to the efforts of ICS the curriculum framework for compulsory education was developed. This framework takes into account the fulfillment of present-day demands by taking a significant step towards meeting the NSPU target concerning improvement of quality of teaching/learning process. c. The new reorganization of educational administration in regional structures, with the setting-up of 13 Regional Educational Departments, in which each department comprises the investment sector, the human resources sector, the teacher training and qualification for teachers, the inspection sector as well as the establishment of 24 educational offices, which are directly linked to Regional Educational Departments. d. Establishing the Competition System in the context of decentralizing the education system. At one point in time the Heads of RED and EO, were appointed from the leading official of the Central Office, whereas of now in due consideration of the Law on civil servants model and the relevant Decision of CM , the appointments of Heads of educational authorities are based on the competition pattern; e. Development and implementation of performance standards for each department at MOES. This process was initiated with the reforming of content within the educational system as well as with this project. At present, work is continuing in establishing performance standards at the central level and extending it to other levels, to all the positions/jobs within the system; f. Restructuring and establishing the Inspection Service at the entire pre-university education system by establishing the inspection sectors across 13 REDs and by making possible the organization of inspection at two levels: central and local ones. This service comes complete with Regulations and Inspection manuals. - 35 - Most of the initiated reforms are still into their beginning stage, but sustaining long-term implementation of these reforms would require an on-going support, by some bigger measure, in building and improving the existing capacities, as well as in terms of training and qualifying human resources, which are detrimental in endorsing and taking this reform forward. 2. Information System for Education Management This component did achieve a few goals, but it's worth pointing out that the objectives outlined within this project were not properly formulated in the initial stage of the project design, that's one strong reason that accounts for failure in falling short of expected results. Among the most significant achievements regarding this component we could list: · Internal network (grid) set ­up which allows for a quick exchange of information via the internet. The provision of internet hookup and email is up and running. The electronic communication and exchange of information between MOES and RED and EO has been successfully realized. This is the chart that shows the way how information flows between the concerned institutions · Contact point established within MOES. This contact point consists of booth PCs which boast folders containing electronic information with data on testing, regulations, school-year calendars, information on admission to universities etc. · First statistical education report released in 2003. On the basis of the previously gained experience, and in view of the wider vision for education sector, MOES has already developed and is ready to publish the Yearly Statistical Report for 2004. But, the lack of a central database has hindered the realization of projection-based analysis and the coordination of activities and decision-making among the different Departments within MOES and other dependent institutions. At present, work has been successful in developing software that facilitates the collecting, processing and communicating of the data output. In spite of their presence, it remains to be seen how we can put to better use the advantages of these software. Easy reference can be drawn to Map Decision which was originally thought as a tool in support of planning, projecting, monitoring, evaluating, and disseminating the resources and the decision-making. Thus, one of the key elements in getting the expected results is to make the best use of this program at the central and regional level and any other program that has been made available. 3. Evaluation and testing (exams) In the context of this component it was possible for the examination system to be placed beyond the reach of MOES competences, and be run and organized by a specialized Center under the name National Education Center for Evaluation and Exams. This center has carried out the following: · The first ever national evaluation on the achievements of the primary 4th graders in subjects such as Mathematics, Albanian Language and Literature. These results were made public at a National Conference held in May 2004. These results made their way to the public through the publication of the respective report. The survey "The National Evaluation of the Achievement of the 4th graders, in subjects such as mathematics, Albanian language and literature" is the first study of its kind ever to be conducted by the NECEE, according to the international best standards and a sample based evaluation. (This was technically and financially supported by a WB project. (Reform in education". · The first national school-leaving examination for 8th graders, by introducing for the first time elements of the external examination as well as standardized procedures (June 2004). · The admission competition to secondary vocational schools (September 2004). · Developing orientation programs on the national school-leaving and maturity examination for 2003-2004. · The maturity examination for the general secondary and profile-based school. - 36 - NECEE is making efforts toward taking part in the earliest international evaluations, such as in TIMSS 2007, an international program on achievements of pupils in mathematics and science, with the view to combining the national evaluations with the international practices. 4. School rehabilitation The 4th component was concerned with rehabilitating 44 schools in all. In this framework, it provided some 22.000 school places. A detailed School Map at national level was compiled also. The real value behind this component lies not only in the physical rehabilitation of the school buildings, but also in the development and utilization of rehabilitation and refurbishment standards. MOES is currently applying such standards to the on-going rehabilitation of schools with state funds as well as money coming from various donors. Moreover, there has been an increase in the awareness level on the part of the parent community, who now looks at the investments in schools as part of the formal education of their children. It has been possible to ensure a direct participation of the community in the realization of this project as well as a close cooperation between schools and community in better maintaining and safeguarding the investments made so far. But the funds made available from the project were sufficient only in ensuring the basic equipment for the 44 schools already rehabilitated. As a consequence, the special equipment for Albanian language, laboratories and other teaching equipments are still necessary in order to carry out a proper and smooth teaching/learning process. 5. School textbooks Privatization of Publishing Houses and of Distribution Enterprise of School Textbooks, constituting the 5th component of the project, put forward the need for immediate measures to be taken in terms of school publication management. The establishment of textbook management unit within MOES, at the urgent suggestion by the international experts, set up the right conditions for the implementation of European policies and standards with respect to liberalization of school publications. The new policy followed by MOES in terms of school textbooks was intended to massively favor the compulsory education pupils, especially the children in need. In so doing, MOES was intent on guaranteeing the best quality textbooks. Proceeding from the experience gained from the printing of textbooks in 2003 with the funds allotted by World Bank, it was possible to decrease the printing costs of any school textbook by around 50% comparing it with the cost of the previous years. With the savings accrued from the reduction in costs it was possible for MOES to implement a policy that was supportive of disseminating the school-textbooks. In close follow-up of this new policy, initiated and supported by component 1 and 5 of this project, the following were carried out: · Developing programs and new textbooks for the first graders in compulsory education; · First time selection on the basis of competition of textbooks that are currently in use for the school year 2004-2005, based on the relevant teaching plan and the subject programs for grade 1 of the compulsory 9-year education and a textbook title for grade 10. · A free competition of several alternative textbooks designed by a number of publishers for the second year of general secondary schools. · Printing all the new textbooks for the first grade. Reprinting all the textbooks for the second and six year. With this school year in mind, some 123 textbooks titles have been prepared in total. · Developing teaching programs for subjects of 2 and 6 grade ( 9-year education) as well as announcing the competition for their publication; - 37 - · Providing free textbooks for the second year in a row for the compulsory education as well as providing free text-books to all first graders without having to ask them to check in their books for the 2004-2005 periods. · Providing free textbooks to all pupils of general secondary education who live in 80 poverty-riddled communes, by helping increase the enrollment rates in these communes. In the context of liberalizing the school textbooks, publishing houses, upon receiving the right to publish the school textbook, request the right to printing and disseminating such texts without having to set any ceiling price. Such a thing was strongly suggested by the WB consultant on textbook policies, that is, liberalization of the school textbooks policies should apply equally to both printing and dissemination. We have to emphasize that the policies on printing and disseminating textbooks will continue to be under the care and responsibility of MOES, with the view to ensuring the provision of such textbooks on time, with the highest possible quality and a reasonably affordable price for the pupils. It's precisely through these textbook policies that MOES has managed to improve substantially the vital education of the population. Thus during the 2002 school year, only 60% of pupils that completed the compulsory education were drawn in to secondary education, while in 2004, by contrast, this figure shot up at 77%. Ministry approach towards lack of project achievements, in the following areas: · Lack of adequate responsibility and accountability on the part of MOES especially in the first two years. · The interested parties (stakeholders) were not fully acquainted with the general objectives of the project. They possessed information that was rather limited to the area they were responsible for. · The degree of information on the part of structures responsible to MOES was increasingly insufficient. There was a failure to make public the results achieved in every step of the way. · Since the project, at its core, was accompanied with other structural changes of MOES, with new openings and some old structures disappearing for good, some employees did not take as effective a part as they should in its implementation phase. · Since the curriculum reform was wide-ranging and the likely changes in the political staff could risk hampering its implementation, there was acceleration in the implementation pace of the project, but always intent on establishing a sustainable reform. · Since the implementation of the project was rather stretched out in the span of the last two years, we can not claim that its results are far immediate. What we can safely state, is that the groundwork has been laid for future results to be grown. · The consultants' reports were not consistently followed up with "objection" feedback from MOES. At times these reports were oversized and beyond comprehension, as was the case with the report prepared by the consultant of the school textbooks. · We have not yet determined the performance indicators at MOES in order to measure up or gauge the achievements and success to be expected in various fields. · The government was not generous in its financial support (state budget) in helping the reform undertaken within this project. VII. LESSONS LEARNT TO BE DULY CONSIDERED IN THE FUTURE PROJECTS Developing a new project · The conceiving stage of the project was rather complex and mixed: Therefore, issues have come up during its implementation, mainly related to legal, political, organizational, and management areas in component 1 as well as concrete matters technically oriented into component 2-5. - 38 - · In view of the reform initiatives taken by the government, the endless and consistent efforts in the education sector ( such as EFA/FTI) as well as the outcomes in the implementation of the component, MOES will keep focusing on organizational and legal reforms and other initiatives to follow. In addition, there is a need for efforts to be made in terms of technical orientation in the curriculum development, teacher training and the preparation of school textbooks and other teaching materials, publishing, printing and distributing textbooks. Furthermore, the attempts will continue in rehabilitating schools and participating in national and international evaluations. These could well be considered as top priorities to be considered and supported in the future. Component Report Title Date Component 1 Translation of IPS Education Strategy Document (Irena January, 2001 Pata) Component 1 Management and Implementation of Education System May, 2003 (IBF) (Note 1) Component 1 A Review of the Institute for Pedagogical Studies (IPS) December 2002 (Cameron Harrison) Component 1 EFA/FTI-Albania (Claude Tibi, Argon Kasapi) (Note 5) June 2003 Component 1 Curriculum Development and Implementation Component February 2004 (Alexandru Crisan--International Consultant) Component 1 Curriculum Development and Implementation Component September 2003 (Teuta Braco--Local Consultant) Component 1 Performance Review (Andy Mc'Donald, Raymond May 2003 D'Aoust, Cameron Harrison) (Note 3) Component 1 National Education Strategy (Hitendra Pillay) July 2004 Component 1 National Education Strategy: Implementation Plan October 2004 (Hitendra Pillay) Component 1 Monitoring and Evaluation Study (American Councils for October 2004 International Education) Component 1 Decentralization of the Education Sector in Albania May 2004 (Samuel Benalal) Component 1 Pilot Plan for Decentralization of the Education System in September 2004 Albania (Samuel Benalal) Component 1 Public Expenditure Tracking Survey (PETS) (Claude Tibi) October 2004 Component 1 PETS Survey --Data(Public and Private Finance Institute) September 2004 Component 1 Performance Standards of the Ministry of Education and October 2004 Evaluation Plan (Marmanet and Telos Group) Component 1 Technical Assistance to Develop and Strengthen Teacher October 2004 Training Agency (Frank Schorn) Component 2 Decision Software Evaluation (Betim Cico) April, 2000 Component 2 Internet Webpage of MOES (INTECH) December, 2001 Component 2 Center for Information and Statistics (CIS) Staff Evaluation June, 2001 (Nadem Muhamad) Component 2 Terms of Reference (TOR) for Four Administrative August, 2001 Modules (Ilir Beqja--Local Consultant) Component 2 TOR for Four Administrative Modules (Kamal August, 2001 - 39 - Siblini--International Consultant) Component 2 Development and Implementation of EMIS Pilot (EMI September 2002 Systems) Component 2 Consulting Services: Development and Implementation of April 2003 EMIS Systems (DATECH) Component 2 Assessment of the Implementation of the EMIS Component April 2004 (Raymond D'Aoust, Edmond Sheshi) (Note 4) Component 3 Technical Assistance (TA) to National Assessment and April, 2001 Examination Center (NAEC) (Nikoleta (Litoiu) Component 3 TA for Establishing NAEC (Nikoleta Mita) September, 2001 Component 3 Learning Assessment and Examinations (IDP) (Note 2) November 2003 Component 3 Advisor/Mentor to the NAEC (Robert Altman) October 2004 Component 4 Architectural/Engineering Services to Elaborate Building April, 2002 Material Specifications and School Furniture Specifications (iC Consulenten) Component 4 Final School Project Rehabilitation Report (ILIRIADA) May 2004 (Note 6) Component 5 Assessing Financial Documents for State Textbook October, 2000 Publishing House (STPH) and Textbook Distribution Enterprise (TDE) (Lindita Xhillari) Component 5 Assessing Legal Documents for STPH and TDE (Gent October, 2000 Ibrahimi) Component 5 Support to Establish a Textbook Approval Board (Tim May 2004 Horsler) International Training Completed Component Dates Number of Location Topic/Content Participants Component 1 2000 7 USA Strategic Choices for Education Reform Component 1 2000 1 USA Managing Development Projects Component 1 2000 2 Bosnia and Procurement Herzegovina Seminar for Implementing Agency Staff Component 1 2001 3 USA Strategic Choices for Education Reform - 40 - Component 1 2001 2 Italy Financial Management and Disbursement in World Bank Financed Projects Component 1 2002 2 USA International Procurement Component 1 2002 4 Hungary Strategic Choices for Education Reform Component 1 2002 4 Greece Education Investment Conference Component 1 2003 5 United Kingdom Study Visit of the Albanian Minister of Education and Science and Other Officials (Education Management and Education Standards in England) Component 1 2004 2 France Monitoring and Evaluating the Quality of Education Component 1 2004 1 USA Education Reform Core Course Component 1 2004 1 France Education Budgets Component 1 2004 6 Romania International Conference for Education Component 2 2001 2 Lithuania Workshop on Education Statistics and Indicators Component 2 2003 1 Luxembourg Systems of Education Statistics Component 3 2001 2 Romania NAES - 41 - Conference on Learning Assessment Component 3 2003 1 USA +United Multilevel Kingdom Analysis, Hierarchical Linear Modeling, Multivariate Analysis, Scale Analysis Component 3 2004 7 Netherlands Item Writing Test Preparation and Interpretation of Statistics for Assessments and Examinations Component 5 2000 3 Hungary Textbook Training Workshop Component 5 2000 7 Poland Training in WsiP Publishing House in Warsaw In-Country Training Workshops and Seminars Completed Component Dates Topic/Content Participants Number Category Component 1 December Education Finance and 10 MOES, Directorates of 11-16, 2002 Budgeting Economy and Investment Component 1 Not reported Development of Policy 14 MOES main organizational Document units Component 1 Not reported Development of 10 MOES main organizational Education Sector units Strategy Component 1 Not reported Development of Annual 25 MOES main organizational Education Performance unit Report Component 1 Not reported Educational Planning and 15 MOES main organizational Financing, Operational unit Planning and Budget Preparation Component 1 Not reported Pre-Service and 12 MOES main organizational In-Service Teacher unit Systems Component 1 Not reported Education Policy and 15 MOES main organizational - 42 - In-Service Training units Reforms Component 1 April 08, 2003 Organizing Working 25 IPS Groups for Curriculum Design Component 1 April 10-11, Conceptual Working 25 IPS 2003 Groups and Curriculum Framework Component 1 April 17, 2003 Curriculum Development 25 IPS and Curriculum Framework Component 1 April 17, 2003 Curriculum Management 10 MOES Process Component 1 July 03, 2003 Institutional Structures 25 IPS for Curriculum Development Component 1 July 04, 2003 Development of Plani 10 MOES Mesimor Component 1 September 7, In-service Training 12 IPS (teacher training 2004 Planning Seminar: personnel) Topics: a) capacity building, b) planning seminars, and c)training of trainer programs Component 1 October 19-24, Active and Interactive 110 totalDistrict and local 2004 Teaching/Learning education office inspectors and Methodologies staff (Four (4) one-day workshops in Shkodra, Tiana, Elbasan, Fier) Component 1 Week of Seminar: Evaluating 12 IPS (teacher training October 25, In-Service Programs and personnel) 2004 Developing National In-Service Plan Component 1 September 13, PETS Seminar 25 MOES , MOF , Districts 2004 Component 1 September Performance Standards 14 MOES organizational unit 22-23, 2004 directors and representatives from Training Institute for Civil Service Component 2 First round Two round of training in First round: 19 training May the framework of 2002 contract for consulting Second round: 35 Workshops services : Development for information specialists in and Implementation of MOES, CIS specialists and EMIS Modules (i) Data district staff Second round Collecting System , (ii) - 43 - training The Database Querying September-Octo System , (iii) The ber 2002 Analytical and Policy/Decision Support System. Component 2 July, 2003 Training for Four 20 user personnel and 2 staff for General Administrative each systemUser personnel Modules: (MOES and district), MOES Legislation Document staff Management System (LDMS) Document Management System (DMS) Accounting Management System (AMS) Payroll Management System (PMS) Component 3 April 09, 2001 Seminar for National Not reported NAEC Assessment Education Center Component 3 June 2002 Writing Multiple ­Choice Not reported NAEC Items for University Entrance Examination Component 3 June 2002 Use of Rasch Modeling 1 NAEC for Test Analysis Component 3 May 21, 2003 National Conference of 80 NAEC NAEC: Dissemination the first results of National Assessment for fourth grade of elementary school Component 3 July 09, 2004 National Conference 80 NAEC "New system of the National Assessments and Examinations, to ensure quality of education in Albanian Education System Component 5 October, 2000 Textbook Conference; 37 MOES / Publishers Liberalization Component 5 May 13-14, Subject Specialist Panels 15 Subject Specialist Panels 2004 for Textbook Manuscript (Teachers) Evaluation - 44 - - 45 -