Report No: ICR 3150 IMPLEMENTATION COMPLETION AND RESULTS REPORT ON A CREDIT IN THE AMOUNT OF SDR 28.2 MILLION (US$ 42 MILLION EQUIVALENT) TO THE UNITED REPUBLIC OF TANZANIA FOR A ZANZIBAR BASIC EDUCATION IMPROVEMENT PROJECT July 14, 2014 Education Global Practice East and Southern Africa Africa Region Currency Equivalents (Exchange Rate Effective July 14, 2014) Currency Unit = TZS US$ 1.00 = 1660 TZS Fiscal Year July 1 – June 30 Abbreviations and Acronyms BWMTTC Benjamin William Mkapa Teacher PDO Project Development Objective Training College CSEE Central Secondary Education Exam PMT Project Management Team DO Development Objective PPF Project Preparation Facility EMIS Education Management Information QaE Quality at Entry System QAG Quality Assurance Group ESMF Environmental and Social RGZ Revolutionary Government of Management Framework Zanzibar GDP Gross Domestic Product RP Restructuring Paper GER Gross Enrollment Ratio RPF Resettlement Policy Framework GPE Global Partnership for Education SEDP Secondary Education Development ICR Implementation Completion and Program Report SIDA Swedish International Development IOI Intermediate Outcome Indicator Agency IP Implementation Progress SIL Specific Investment Loan ISR Implementation Status Report SUZA State University of Zanzibar JAST Joint Assistance Strategy for Tanzania SWaP Sector-Wide Approach M&E Monitoring & Evaluation UNICEF United Nations International MKUZA Zanzibar Strategy for Growth and Children’s Emergency Fund Reduction of Poverty URT United Republic of Tanzania MoEVT Ministry of Education and Vocational USAID United States Agency for Training International Development MoF Ministry of Finance ZBEIP Zanzibar Basic Education MTR Mid-Term Review Improvement Project NECTA National Examinations Council of ZEDP Zanzibar Education Development Plan Tanzania OP/BP Operational Policies/Bank Procedures PAD Project Appraisal Document Vice President: Makhtar Diop Country Director: Philippe Dongier Practice Manager: Sajitha Bashir Project Team Leader: Arun R. Joshi ICR Team Leader: Barbara Weber The United Republic of Tanzania Zanzibar Basic Education Improvement Project CONTENTS DATA SHEET A. Basic Information.................................................................................................................. iii B. Key Dates .............................................................................................................................. iii C. Ratings Summary .................................................................................................................. iii D. Sector and Theme Codes....................................................................................................... iv E. Bank Staff .............................................................................................................................. iv F. Results Framework Analysis ................................................................................................. iv G. Ratings of Project Performance in ISRs .............................................................................. vii H. Restructuring (if any) ........................................................................................................... vii I. Disbursement Profile ........................................................................................................... viii 1. Project Context, Development Objectives and Design ....................................................... 1 1.1 Context at Appraisal ................................................................................................. 1 1.2 Original Project Development Objectives (PDO) and Key Indicators ..................... 3 1.3 Revised PDO and Key Indicators, and reasons/justification .................................... 3 1.4 Main Beneficiaries .................................................................................................... 3 1.5 Original Components ................................................................................................ 4 1.6 Revised Components ................................................................................................ 5 1.7 Other Significant Changes ........................................................................................ 5 2. Key Factors Affecting Implementation and Outcomes ...................................................... 5 2.1 Project Preparation, Design and Quality at Entry..................................................... 5 2.2 Project Design........................................................................................................... 6 2.3 Implementation ......................................................................................................... 7 2.4 Monitoring and Evaluation (M&E) Design, Implementation and Utilization.......... 7 2.5 Post-completion Operation/Next Phase .................................................................... 9 3. Assessment of Outcomes .................................................................................................... 9 3.1 Relevance of Objectives, Design and Implementation ............................................. 9 3.2 Efficacy: Achievement of Project Development Objectives .................................. 10 3.3 Efficiency................................................................................................................ 12 3.4 Justification of Overall Outcome Rating ................................................................ 12 3.5 Overarching Themes, Other Outcomes and Impacts .............................................. 12 4. Assessment of Risk to Development Outcome ................................................................. 13 5. Assessment of Bank and Borrower Performance ............................................................. 14 5.1 Bank Performance .................................................................................................. 14 5.2 Borrower Performance ........................................................................................... 16 i 6. Lessons Learned................................................................................................................ 17 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners................... 17 Annex 1. Project Costs and Financing ...................................................................................... 18 (a) Project Cost by Component .................................................................................... 18 (b) Financing ................................................................................................................ 18 Annex 2. Outputs by Component.............................................................................................. 19 Annex 3. Economic and Financial Analysis ............................................................................. 29 Annex 4. Bank Lending and Implementation Support/Supervision Processes......................... 32 (a) Task Team members ............................................................................................... 32 (b) Staff Time and Cost ................................................................................................ 33 (c) Formal Missions ..................................................................................................... 33 Annex 5: Results Across Implementation Period ..................................................................... 35 Annex 6: Graphs: Trends in Results Achievements over Time ................................................ 36 Annex 7. Zanzibar Ministry of Education and Vocational Training: Implementation Completion Report – shortened and summarized version ........................................................ 39 MAP .......................................................................................................................................... 49 ii DATA SHEET A. Basic Information Zanzibar Basic Country: Tanzania Project Name: Education Improvement Project Project ID: P102262 L/C/TF Number(s): IDA-42930 ICR Date: 07/14/2014 ICR Type: Core ICR UNITED REPUBLIC Lending Instrument: SIL Borrower: OF TANZANIA Original Total XDR 28.20M Disbursed Amount: XDR 28.18M Commitment: Revised Amount: XDR 28.18M Environmental Category: B Implementing Agencies: Minstry of Education and Vocational Training Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 10/31/2006 Effectiveness: 01/29/2008 01/29/2008 Appraisal: 01/29/2007 Restructuring(s): 04/22/2013 Approval: 04/24/2007 Mid-term Review: 05/05/2011 06/06/2011 Closing: 07/31/2013 11/30/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Development Outcome: Substantial Bank Performance: Moderately Unsatisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Moderately Quality at Entry: Government: Moderately Satisfactory Unsatisfactory Implementing Quality of Supervision: Moderately Satisfactory Satisfactory Agency/Agencies: Overall Bank Moderately Overall Borrower Moderately Satisfactory Performance: Unsatisfactory Performance: iii C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments (if Indicators Rating Performance any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any Quality of Supervision Yes None time (Yes/No): (QSA): PDO rating before Moderately Closing/Inactive status: Unsatisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 9 9 Primary education 2 2 Secondary education 66 66 Tertiary education 23 23 Theme Code (as % of total Bank financing) Education for all 40 40 Environmental policies and institutions 20 20 Gender 20 20 HIV/AIDS 20 20 E. Bank Staff Positions At ICR At Approval Vice President: Makhtar Diop Hartwig Schafer Country Director: Philippe Dongier Judy M. O'Connor Practice Manager: Sajitha Bashir Dzingai B. Mutumbuka Project Team Leader: Arun R. Joshi Ivar Strand ICR Team Leader: Barbara Weber ICR Primary Author: Barbara Weber F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The development objective is to improve completion of lower secondary education with successful performance among students. Revised Project Development Objectives (as approved by original approving authority) N/A iv (a) PDO Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Absolute numbers of pupils, disaggregated by gender, taking the examination at the Indicator 1 : end of the lower secondary education (CSEE). Value T: 5,389 T: 13,051 quantitative or M: 2,707 T: 24,455 T: 13,962 M: 5,651 Qualitative) F: 2,682 F: 7,400 Date achieved 01/01/2005 07/31/2013 04/22/2013 12/31/2012 Comments T: 93.5% achieved (revised target) (incl. % T: 53.4% achieved (orginal target) achievement) Indicator 2 : Absolute numbers of pupils, disaggregated by gender, passing the CSEE. Value T: 4,897 T: 6,935 quantitative or M: 2,442 T: 22,254 T: 12,705 M: 3,011 Qualitative) F: 2,455 F: 3,924 Date achieved 01/01/2005 07/31/2013 04/22/2013 12/31/2012 Comments T: 54.6% achieved (revised target) (incl. % T: 31.2% achieved (original target) achievement) Absolute numbers of pupils, disaggregated by gender, passing the CSEE with Div. Indicator 3 : III or better. Value T: 757 quantitative or 900 4,647 2,653 M: 384 Qualitative) F: 373 Date achieved 01/01/2005 07/31/2013 04/22/2013 12/31/2012 Comments T: 28.5% achieved (revised target) (incl. % T: 16.3% achieved (original target) achievement) Indicator 4 : The percentage of CSEE takers divided by the population aged 17 years. Value T: 27% T: 49% T: 46% quantitative or M: 26% M: 49% M: 40% Qualitative) F: 28% F: 49% F: 52% Date achieved 01/01/2007 04/22/2013 12/31/2012 Comments (incl. % T: 93.9% achieved achievement) (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Indicator 1 : Pupil/Classroom ratio Value (quantitative 83 92 58 or Qualitative) Date achieved 01/01/2006 07/31/2013 12/31/2012 v Comments Overachieved: 160%. Baseline and target values referred to pupil/classroom ratios in (incl. % a double shifting environment. W/o projct intervention, the pupil/classroom was achievement) projcted to raise to 120 by 2013. Percentage of teachers teaching at secondary level who are qualified to teach at the Indicator 2 : secondary level. Value 59% Qualified (Diploma (quantitative or higher education >65% 79% or Qualitative) qualification) Date achieved 01/01/2006 07/31/2013 12/31/2012 Comments (incl. % achieved, 121.5% achievement) Percentage of teachers of English (at secondary level) are qualified to teach Indicator 3 : secondary school English. Value 59% Qualified (Diploma (quantitative or higher education >65% 0 or Qualitative) qualification) Date achieved 01/01/2006 07/31/2013 11/30/2013 Comments (incl. % This IOI was not measured; data are not available. achievement) Percentage of teachers of mathematics (at secondary level) are qualified to teach Indicator 4 : secondary school mathematics. Value 59% Qualified (Diploma (quantitative or higher education >65% 47% or Qualitative) qualification) Date achieved 01/01/2006 07/31/2013 12/31/2012 Comments (incl. % 72.3% achieved achievement) Percentage of teachers of science (at secondary level) are qualified to teach Indicator 5 : secondary school science. Value 38% (Biology) 59% (Biology) (quantitative 43% (Chemistry) >65% 62% (Chemistry) or Qualitative) 39% (Physics) 57% (Physics) Date achieved 01/01/2006 07/31/2013 12/31/2012 not (fully) achieved. Comments 90.8% (Biology) (incl. % 95% (Chemistry) achievement) 87.7% (Physics) Indicator 6 : Pupil/Textbook ratio 1.4/1: Maths (Forms III-IV): 1.1/1: Physics Value (Forms III-IV) (quantitative 0 1/1 1.5/1: English or Qualitative) (Forms III-IV) 1.5/1: Kiswahili (Forms III-IV) Date achieved 01/01/2006 07/31/2013 12/31/2012 Comments not fully achieved, except for Physics vi (incl. % achievement) Percentage of (primary school) teachers successfully completing (and passing the Indicator 7 : examination) in the in-service (English) language course. Value (quantitative 0 90 0 or Qualitative) Date achieved 01/01/2006 07/31/2013 11/30/2013 Comments (incl. % Indicator only mentioned in PAD main text but not in PAD RF. Not measured. achievement) Percentage of teachers successfully completing (and passing the examination) in the Indicator 8 : in-service mathematics and science courses. Value (quantitative 0 90 0 or Qualitative) Date achieved 01/01/2006 07/31/2013 11/30/2013 Comments (incl. % Indicator only mentioned in PAD main text but not in PAD RF. Not measured. achievement) G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 12/20/2007 Moderately Satisfactory Moderately Unsatisfactory 0.00 2 02/28/2008 Moderately Satisfactory Moderately Unsatisfactory 0.00 3 09/18/2008 Moderately Satisfactory Moderately Unsatisfactory 2.01 4 11/24/2008 Moderately Satisfactory Moderately Satisfactory 2.06 5 06/09/2009 Moderately Satisfactory Moderately Satisfactory 2.23 6 12/03/2009 Satisfactory Satisfactory 2.47 7 06/27/2010 Satisfactory Satisfactory 3.01 8 03/26/2011 Satisfactory Satisfactory 9.93 9 10/23/2011 Satisfactory Satisfactory 15.89 10 06/22/2012 Moderately Unsatisfactory Moderately Satisfactory 26.54 11 12/25/2012 Moderately Unsatisfactory Satisfactory 32.60 12 05/17/2013 Moderately Satisfactory Moderately Satisfactory 37.67 13 11/27/2013 Moderately Unsatisfactory Moderately Satisfactory 43.49 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Restructuring Disbursed at Reason for Restructuring & Approved Date(s) Restructuring Key Changes Made PDO Change DO IP in USD millions A Restructuring Paper was 04/22/2013 MU S 36.04 approved on April 22, 2013 (change of target values of all vii ISR Ratings at Amount Board Restructuring Restructuring Disbursed at Reason for Restructuring & Approved Date(s) Restructuring Key Changes Made PDO Change DO IP in USD millions three PDO indicators, introduction of additional, fourth PDO indicator, reallocation of funds, and extension of the original closing date by four months to November 30, 2013). I. Disbursement Profile viii 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. Country context. Zanzibar comprises two main islands, Unguja and Pemba, and a number of smaller islets along the western rim of the Indian Ocean. The islands lie about 40 kilometers off the coast of mainland Tanzania. In 1964, the sultanate of Zanzibar – with its long history of Persian, Portuguese and Arab influence - joined with Tanganyika to form the United Republic of Tanzania (URT). However, Zanzibar retains considerable autonomy over her internal affairs and has her own legislature, judiciary and an executive. According to the Constitution of the United Republic of Tanzania, both, primary and secondary education are under the responsibility of the Revolutionary Government of Zanzibar (RGZ) while tertiary education is a “union matter”. 2. Even though the service sector, including tourism, represents a more important share of GDP than most other countries in Sub-Saharan Africa, at the time of appraisal the people of Zanzibar were not participating in tourism and other services in the numbers expected. This was largely due to lack of required skills and English language proficiency; specifically, skills in mathematics and science were inadequate to serve the needs of a modern economy. Sector context 3. At appraisal, education spending as a share of Government budget and of GDP was in the range of 4.0-4.5% of GDP, and equivalent to about 18.5% of government spending targeted to reach 22% over the next five years. Gross enrolment rates (GER) at primary schools were about 91% (with about 80% net enrollment) while GER at secondary school level was about 30%. 4. According to the URT Constitution, both, primary and secondary education are under the responsibility of the RGZ while tertiary education is a “union matter”. No matter these differences in responsibility, the education system in Zanzibar is closely aligned with the system on the mainland: While primary education curriculum and examinations are managed in Zanzibar, curriculum and examinations for secondary education are the same for Zanzibar and the mainland. The curriculum is under the auspices of the Tanzania Institute of Education and the examinations under the National Examinations Council of Tanzania. 5. The 2003 Zanzibar Education Sector Country Status Report identified the following major constraints and challenges: (i) low access rates, particularly at secondary levels, (ii) poor quality of education caused by inadequate facilities and limited access to textbooks, and (iii) the difficult transition experiences between primary and secondary school caused by the abrupt change in language of instruction (from Kiswahili to English). 6. The Project Appraisal Document (PAD) described the following key issues: (i) The rate of construction of new classroom space did not keep pace with the increased enrollments in lower secondary which nearly tripled over the previous 10 years. In order to cope, double- shifting was common practice in almost all secondary schools, but average class size was still over 60. Consequently, school infrastructure was considered a priority; (ii) Poor English language proficiency of secondary school students 1; (iii) Under-qualified and unqualified 1 Since the language of instruction in primary school is Kiswahili, many students were unable to cope with the transition to learning in English at secondary level. 1 teachers: In addition to lack of qualification to teach at secondary school level (based on their formal qualification, i.e. either diploma or degree 2), many were not sufficiently proficient in English to be able to teach in this language; (iv) Acute shortages of teachers in secondary level English, mathematics and science; (v) Inefficient deployment of secondary school teachers; and (vi) Shortage of secondary school teaching/learning materials, including textbooks, laboratory and ICT equipment. 7. Gender disparities were not a major issue though access problems increased in the final two years of secondary school and there were ongoing concerns about the interest of girls in mathematics and science. Project context 8. Following the decision by the URT Government not to include Zanzibar in the Tanzania Secondary Education Development Program (SEDP, approved by the Board in June, 2004) but to develop a separate program for Zanzibar, the Zanzibar MoEVT asked the World Bank to lead external partners in their analytical work and dialogue with RGZ on support to secondary education. The initial intention was to develop a Sector Wide Approach (SWaP) in education with involvement of the World Bank and other development partners, and preparatory studies for the Project were coordinated with those for the SWaP. This coincided with a considerable amount of analytical work globally and regionally by the Bank to determine approaches and best practice in secondary education. Higher level objectives to which the project contributes 9. The improvement of Secondary education, the Project’s core objective, contributes to all three pillars 3 of Zanzibar's Strategy for Growth and Reduction of Poverty (MKUZA), through increasing productivity, improving social well-being for those accessing secondary education and their families, and contributing to democracy, crime reduction and improved living conditions. Secondary Education is also embedded in the Tanzania Vision 2020, and the former Zanzibar Education Master Plan (1996 - 2006). The 2006 Zanzibar Education Policy emphasized the development of quality secondary education and the importance of boosting performance in English, mathematics and science. It directed for children being taught mathematics and science through English from Standard 5. A first Zanzibar Education Development Plan (ZEDP), prepared in 2007, was succeeded by a second, launched in 2012 and prepared in close collaboration between the MoEVT, World Bank, SIDA, USAID and UNICEF. 10. All of these are embraced by the Dakar Framework of Action on Education for All and the achievement of the Millennium Development Goals. RGZ has resolved to devote at least 20% of the total public expenditure to the education sector. Furthermore, the education sector was given top priority in the MKUZA covering the period 2007-2010. MoEVT has also strengthened aid management and coordination by setting up a Zanzibar Education Sector Committee which is a forum for policy dialogue between stakeholders involving Ministry officials, NGOs, private sector and development partners. 2 Following graduation from higher secondary school, prospective secondary school teachers have two possible options for formal professional qualification: For those with lower grades at graduation: a two years training program at a Teacher Training College, completed with a Diploma; for those with higher grades: a three-years university program at the State University of Zanzibar, leading to a Degree. 3 The MKUZA includes three pillars, or clusters of activities: (i) Growth and reduction of income poverty; (ii) Social services and well-being; and (iii) governance and national unity. 2 11. The Joint Assistance Strategy for Tanzania (JAST) 4 for the period of FY07 to FY10 was presented to the Board at the same time as this Project. The strategy described principles for coordination of development support in Tanzania, and included support for secondary education on Zanzibar. 1.2 Original Project Development Objectives (PDO) and Key Indicators 12. The development objective of the Zanzibar Basic Education Improvement Project (ZBEIP), as stated and approved in the PAD, was to improve completion in lower secondary education (the first four years) with successful performance among students. 5 13. Evidence of progress towards achieving the PDOs was to be assessed through the following (original) PDO Indicators: (i) Absolute numbers of pupils, disaggregated by gender, taking the examination at the end of lower secondary education (CSEE). (ii) Absolute numbers of pupils, disaggregated by gender passing the CSEE. (iii) Absolute numbers of pupils, disaggregated by gender passing the CSEE with Div. III or better. 1.3 Revised PDO and Key Indicators, and reasons/justification 14. The PDO was not revised during the life of the project. 15. A Restructuring approved in April 2013 (when disbursement stood at 77%), formally revised the targets for all three initial PDO indicators, and introduced a forth: (iv) Percentage of CSEE takers divided by the population aged 17 years. As the project was implemented, it became clear that the initial targets had become unrealistic since one of the key assumptions made during project preparation did not happen: The examinations determining the transition of students Form II to Form III were not abolished early into project implementation as initially assumed. As a result, fewer students than expected passed through to O-level (CSEE) exams at the end of Form IV 6, hence adding a percentage figure to the absolute numbers (of CSEE passers) seemed to make a lot of sense. 1.4 Main Beneficiaries 16. The main target groups to be reached by the project were primary and secondary school students and teachers. Secondary school students were to be reached directly through the provision of an improved learning environment, i.e. newly constructed or rehabilitated classrooms and other school facilities, adequate distribution of learning material, and, in the case of girls, availability of additional learning opportunities in science camps. Teachers benefitted directly from access to learning opportunities, and distribution of teaching material. Students were reached indirectly through, as was hoped, improved subject matter knowledge, 4 The JAST is a joint product of the Government of the URT, the Tanzania Development Partners Group, and the World Bank Group. 5 The formulation of the PDO in the PAD and the Financing Agreement (FA) differ slightly. The formulation in the FA is as follows: The objective of the Project is to support the Recipient’s efforts to improve the lower secondary education in the islands of Zanzibar within its territory, including raising the rates of students successfully completing the first four years of secondary education. This formulation does not imply any deviations in terms of substance matter from the PDO presented in the PAD and approved by the Board of Executive Directors. 6 It is now planned to abolish these examinations only by 2015, once the first cohort of pupils that started primary school under the revised curriculum, has reached Form II. 3 language and teaching skills of their teachers. Other direct beneficiaries were head teachers, school inspectors, school management committees, and last but not least MoEVTstaff. 1.5 Original Components 17. The project as approved consisted of two components: (i) Increase enrolment and improve quality in secondary education through provision of infrastructure; (ii) Improve quality of education through teacher training, curriculum reform, and provision of learning materials. Component 1: Increased enrolment and improve quality in secondary education through provision of infrastructure (US$35.1 million) 18. This component was designed to meet the demand for school facilities for the numbers of pupils enrolled in secondary schools at the time of appraisal, and to accommodate the projected increase in secondary school enrolment. It was estimated that 24,000 new school places would be created 7. The component covered the construction and rehabilitation of school infrastructure, including the construction of nineteen new schools, incorporating eight schools in rural areas with limited hostel facilities to serve students from remote rural communities, particularly girls, and three with facilities to meet the needs of students with disabilities. The rural schools were to be equipped with 10 staff houses each. All schools were to be constructed with appropriate sanitary facilities and science laboratories, and supplied with furniture and laboratory equipment. The component also included the rehabilitation of six existing schools. In order to create additional supply of qualified teachers to match the projected expansion of enrolment (the need was estimated with about 1,400 teachers), the project was also to support the construction of new facilities for Benjamin William Mkapa Teacher Training College (BWMTTC) on Pemba Island. Component 2: Improve quality of education through teacher training, curriculum reform, and provision of learning materials (US$12.1 million). 19. In complementarity to the expansion of physical infrastructure, the project included measures to support the quality of education, including provision of in-service teacher training, provision of textbooks and other learning resources, and provision of support for planned curriculum reforms. These interventions targeted both upper-primary and secondary education. They were to cover in-service-training of primary and lower secondary math and science teachers in subject matter, and upper primary math and science teachers in English as the language of instruction introduced with the primary education curriculum reform; in addition, in-service courses were to be provided to school head teachers, inspectors and subject advisors, as well as training and other support to the inspectorate and school management committees. Further, the quality component included the revision of the primary curriculum, and the provision of learning materials (textbooks) in five subjects for all upper primary students based on the revised curriculum; textbooks in four subjects for all lower secondary students, and in all subjects for forms V and VI; library books; learning materials for deaf and blind students in three schools; support for science education for girls through science camps; and the development of a new manual for education on life skills (including HIV/AIDS, gender and environmental awareness) and workshops to assist teachers of subjects such as biology and geography incorporate life skills into their teaching. 7 Please note that this was an estimate and not a target. 4 1.6 Revised Components 20. The components were not revised during the life of the project. However, some project activities were modified, as follows: 21. Component 2: The provision of new textbooks in five subjects for each pupil in upper primary school was revised to lower primary, Standards 1, 2, 3 and 4. It would not have made sense to provide new textbooks for higher grades in primary school, when the lower grades had not been provided with textbooks first, since the learning based on the new curriculum was expected to build up from the lower grades. The GPE project, approved in 2013, includes provisions for new textbooks for upper primary grades, i.e. Standard 5, 6, instead. 22. USAID provided secondary school math textbooks for Forms I-IV and V-VI which led to cost savings for the respective activity for textbook provision for secondary education, under Component 2. 1.7 Other Significant Changes 23. The initial approach towards teacher training was modified, in the course of project implementation: Whereas the PAD provided for a classroom-style center-based training, the World Bank and Government teams jointly develop an alternative approach that ended up being potentially more effective at improving teacher practice. (See more details in Annex 2, on Component 2). 24. The restructuring of April 2013, in addition to revisions in the Results Framework as described in Section 1.3 above, reallocated funds in the amount of US$2.5 million (equivalent to about 6% of credit proceeds), including the amount of US$1.4 million in exchange rate gains, basically in favor of the categories Works (Component 1), Consultants Services and Training. Also, in the same restructuring, the original closing date was extended by four months to November 30, 2013 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry Project Preparation 25. To address urgent needs in secondary education upfront, while preparing for an envisaged SWaP, the Government asked the World Bank to spearhead support for secondary education first of all through an investment credit that was supposed to contribute to the Program under development. The Bank's experience suggested that investments in infrastructure would need to be balanced with inputs to promote quality. In addition to the focus on secondary education, the project would strengthen the crucial link between the upper primary and lower secondary levels. However, it became apparent that the education sector policy environment and budgeting plan at the time of appraisal were not sufficiently developed, and the intended SWaP never materialized at the end, hence the Project did not form part of an overall program-oriented approach. 26. Political economy issues between the URT federal government and Zanzibar (which has been ruled by an opposition party to the party in power on the mainland) might have adversely affected Project preparation. For example, there was a perception of only a small 5 window of opportunity for the preparation of the project with the IDA allocation that was finally approved 8, which led to a short preparation time of approximately one year from the first Identification Mission in May 2006 to Approval in April 2007. Another issue in this context was that the MoF in Dar-es-Salam did not request a Project Preparation grant for the Zanzibar MoEVT which would have helped enormously assuring readiness for implementation of the Project upon approval. 2.2 Project Design Assumptions, results, risks 27. The concrete targets for the PDO indicators were modelled based on a comprehensive list of assumptions and projections related to both the behavior over time of relevant education indicators 9, and the implementation time-line 10. 28. Critical risks identified at preparation included (i) needs for cost savings on public service wage bill could constraint expansion of teacher workforce; (ii) additional resources needed for introduction of preschool education to limit resource available for secondary education; (iii) lack of teachers competencies to teach subjects in English in upper primary; (iv) disproportionate running costs for hostel facilities in eight upper secondary school to drain education budget; (v) inefficient use of funds and ineffective internal controls; (vi) inadequate legal and regulatory framework for procurement; and (vii) and inadequate institutional arrangements for procurement at MoEVT. 29. Quality at Entry. A Quality of Entry (QaE) Review, conducted in June 2010 by the Bank-internal Quality Assessment Group (QAG, now dissolved), rated the likelihood of this project to achieve its objectives as Moderate. QAG came to the conclusion that the Bank had 8 According to the TTL of the preparation team, there was a perception that the IDA amount allocated to Zanzibar (US$42 m for a population of roughly 1 million) was exceptionally high in comparison to the SEDP-1 (US$150 m for a population of 36.5 million at the time), and that there might be a risk of a change of mind with respect to the IDA allocation on the part of the federal government if this were not locked in quickly. 9 These assumptions and projection were the following: (i) Primary Education intake growth rate to be equivalent to population growth rate, given that primary education has reached full enrollment; (ii) Repetition and Drop-out rate assumptions to remain constant and gradually decline; (iii) Abolishment of the Orientation Secondary Class from 2008 creating a "double cohort" which would put stress on the system short term. (iv) Standard 7 to be abolished from year 2011, and hence another double cohort entry to secondary expected in 2012 - based upon a sequencing of curriculum reforms in primary which were expected to be implemented from year 2009 in Standard 4. (v) Transition rate to secondary to gradually increase but to flatten as the impact of the double cohort were absorbed. (vi) Transition rate from Form II to Form III to increase as demand increases and space becomes available due to new construction. (vii) Transition rate from Form II to Form III also to increase due to the envisaged (at the time) elimination of an exam at the end of Form II; (viii) Target graduation and examination ratios at "O-levels" (CSEE) were based on existing ratios where about 98% sit for the exam, 91% passes, and only about 19% passes with Division III or better. These ratios were projected to remain constant. (viii) Transition to upper secondary (Forms IV to Form V) to increase slightly. 10 The main assumption was that bidding for school construction would start between April and June 2008, projecting completion of construction work by mid-2010. This would have allowed students to benefit from the new, more conducive learning environment for about 3 years before project closing. Assuming that at the same time, teacher training and procurement of teaching materials would have progressed significantly, this could have indeed contributed to an increased perseverance of students in secondary schools (as intended to be measured by the increased number of students taking the CSEE) and to improved learning outcomes (as anticipated through the PDO indicator related to the passing of and the quality of CSEE grades) attributable to the Project’s interventions. 6 missed some important steps in the preparation process, and identified quality of entry as a core weakness of this project. QAG was specifically concerned about the PDO indicators, given the decline in the CSEE scores already observed at that time based on test results 11. Consequently, Quality of Design was rated Moderately Unsatisfactory. 2.3 Implementation 30. Implementation began slowly, and initial assumptions about the timing of project implementation were not realized. 31. The most significant challenge at the beginning of implementation was that school designs were not ready until early 2010 (AM 05/10), over 2.5 years into effectiveness, which held up the commencement of civil works. In the second semester of 2008, the MoEVT Procurement Specialist in charge of the Project resigned, and the position was never re-filled. Instead the Ministry’s procurement team took charge, reinforced by a consultant procurement specialist, and supported intensively by Bank procurement staff. 32. Most factors affecting implementation progress were indigenous, and related to capacity weaknesses of the Implementing Agency. QAG, in its QaE review, encouraged the team to reduce the apparent disconnect between rating of both DO achievement and IP, and the actual progress achieved on the ground. Yet, Bank Supervision by a new team which proactively addressed identified weaknesses was assessed as Moderately Satisfactory. 33. By the end of the implementation period, the initial delays had been mitigated thanks to re-planning and making use of efficiency gains. Most expected outputs were reached or exceeded. 2.4 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 34. Design. Project design and in particular the design of the results framework was based on assumptions (see above, para 27) that were, as such not monitored and/or revisited. Once the team finally acknowledged that the M&E design needed a revision, the restructuring of April 2013 was of negligible relevance. Even though PDO indicator values were modified to reflect a more realistic anticipation of project results, it came too late and did not take into consideration the fundamental disconnects in the results chain. 35. According to the PAD, MoEVT had developed a robust Education Management Information System (EMIS) with support from the UNESCO Institute of Statistics, and much of the basic information needed was supposed to be available from the system. In addition, information on O-level examination results were to be obtained from NECTA and this information analyzed and the results compared to those of all students in URT. Ministry staff was supposed to monitor project execution through a set of key performance indicators, enabling analysis of the data and dissemination among key line officials - supported by IDA supervision missions. In addition to these quantitative measures, the MoEVT was supposed to develop qualitative indicators of classroom behavior, to be used in classroom observation to monitor the changes in teachers' performance throughout the project. 11 The comment of the CD to ISR#6 indicated that latest test results had shown a significant reduction in the CSEE pass rate from 91% in 2006 to 77% in 2008. 7 36. Implementation and Utilization. As became apparent during implementation, assumptions made at appraisal on the functioning of the EMIS had been over-optimistic. Concerns about the weakness of the system were a recurrent theme in ISRs. The Mid- Term Review (MTR), conducted in June 2011, came to the conclusion that the system was not operational even after extensive delays. As a result, there was a serious lack of timely data to assess outcomes/results. Furthermore, data on project inputs and outputs had not been adequately entered, analyzed, and retrieved for the purposes of decision making. UNESCO was not able to provide continued support as envisaged, due to financial constraints. Hence, other sources of information were used, including for example a PhD thesis which provided relevant, recent and reliable data on the quality of education sector in Zanzibar for the MTR. The Bank team continued to provide informal support and technical assistance throughout the operation to collect reliable data and manage it for retrieval and analysis. The return of two M&E experts after completing their advanced studies had helped to strengthen the MoEVT capacity from later in 2011. At the end of the project implementation period, while still not high standard, the EMIS collected reasonably reliable data. 2.4 Safeguard and Fiduciary Compliance 37. Safeguards. The project triggered both the Environmental Assessment (OP/BO 4.01) and Involuntary Resettlement (OP/BP 4.12) policies due to its focus on construction and rehabilitation of schools. An Environmental and Social Management Framework (ESMF) was prepared to identify, assess and mitigate likely negative environmental and social impacts of the project. A Resettlement Policy Framework (RPF) was prepared to address potential resettlement issues. 38. As set out in the ESMF, the project benefits were expected to outweigh the adverse impacts from school construction which would pose no direct risks to biodiversity, natural habitats or wetlands, as it would not fund activities in protected areas, national parks or wetlands. Further, out of 18 possible adverse impacts, half were expected to be short-term and cleared with the completion of construction work. The bulk of other impacts were expected to have readily available means for total and effective mitigation. In order to fully comply with requirements of the World Convention on Heritage Sites and preserve the integrity of Stone Town, all construction work in Stone Town would adhere to guidelines set by the Stone Town Development and Conservation Authority who would undertake routine supervision. 39. Fiduciary compliance. Under oversight of the MoEVT Chief Accountant, the Project used the RGZ Financial Management system. Overall, sound financial management systems were maintained in place throughout implementation which enabled proper project data to be captured and store for generation of acceptable financial reports. The financial reports were prepared and submitted to the Bank on time in compliance with the financing agreement. These reports were reviewed and found to be satisfactory and therefore acceptable to the Bank. Audit reports were received on time and where they came in late the reasons for delay were fixed on advice from the Bank. The Controller and Auditor General issues unqualified (clean) opinion for the project. Any issues of internal control weaknesses as identified by the auditors were fixed by the project and verified by the Controller and Auditor General. Financial management implementation support missions were also carried out to provide support to the project during implementation. Overall the FM performance was noted to be satisfactory as the project always fixed issues as they were identified. 8 40. On the procurement side, a couple of mitigation measures had been put in place against deficiencies established in the 2006 Public Expenditure and Financial Accountability Review: i.e., it was agreed to follow World Bank procurement guidelines. Also, both a MoEVT tender board and an adequately staffed Procurement Management Unit in the MoEVT were established; a procurement specialist hired for the first 18 to 24 months of project implementation provided hands-on training to Project Management Team (PMT) staff and members of the tender board and helped in the preparation of an 18-month procurement plan. A case of allegations of misconduct related to textbook procurement in 2009 was settled with involvement of the World Bank Group Integrity Vice-Presidency 12. 2.5 Post-completion Operation/Next Phase 41. There will be no immediate World Bank follow-up project to the ZBEIP. Project achievements will be complemented by activities funded by other donors. Specifically, a US$5.2 million GPE grant was approved in May 2013, in support of the second Zanzibar Education Development Plan (ZEDP-2, 2012/13-2015/16) and supervised by the Swedish International Development Agency. One of its four components (Teaching and Learning Materials with Specific Focus on the Sciences and Mathematics) will build on the achieve- ments of the ZBEIP: It includes, amongst others, the provision of textbooks and learning materials for upper primary education which were, as indicated above, not covered by the ZBEIP in contrast to initial plans. Zanzibar was not included into the IDA-supported Program for Results towards the Tanzanian Big Results Initiative for education. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 42. The overall relevance of the Project is rated substantial. 43. Objectives. The relevance of project’s objectives was high, which is confirmed in the Borrower’s ICR: The objectives have responded to the identified needs and priorities in the country at the time of project preparation and up to now. It was and is fully in line with both the MKUZA I (2006-10) und MKUZA II (2010-15), and Zanzibar education strategies and policies, as well as with the 2007 JAST and the subsequent World Bank Group Country Assistance Strategy for the period of FY2012-15 13. 44. Design. The relevance of the original project design was modest: Even though the choice of lending instrument and implementing arrangements was appropriate, as was the credit/grant amount at Appraisal, the time-line was ambitious given the degree of inexperience of the implementing agency and the fact that the project was not fully ready for implementation when it was approved. In this light, the objectives related to learning outcomes, and their respective indicators were overly optimistic. The fact that two results indicators were in fact achieved without project activities possibly being attributable to these achievements shows the significant design flaw when it comes to the results chain. (See para 12 The Negotiated Resolution Agreement between Oxford University Press and the World Bank Group included the debarment of Oxford University Press Tanzania for the period of 3 years, and a payment to remedy part of the harm done by the misconduct. (see details in Press Release no. 2013/003/INT). 13 The CAS includes a specific outcome related to Improved Access to and Quality of Education. 9 48. below). Objectives and indicators made sense only under the risky assumptions that the tight time line could be met, which turned out not to be the case. 45. Implementation. Relevance of the project’s implementation was substantial: It is true that assumptions taken in the design phase were not revisited, and consequently, the design not formally revised even though challenges became obvious right from the beginning of project implementation, as is documented in the early ISRs. Also, the April 2013 restructuring did not have any significant impact on the final outcome achievements of the project. However, both the government and the Bank team did a great job in turning the project around with intensive hands-on support to the implementing agency and delivered the expected outputs without major extension. 46. Exit: Relevance at exit is substantial. The outputs put in place with support of the project will have a lasting positive impact on the learning environment for secondary education on Zanzibar. 3.2 Efficacy: Achievement of Project Development Objectives 47. Project efficacy is considered modest. 48. This project faces a unique tension between on the one side failing to fully achieve the intended outcomes on the PDO level, while on the other delivering the expected outputs on the level of the components in the satisfactory range, hence the difference between ratings for overall efficacy and outputs by components. The explanation for this disconnect is related to the design of the results chain, which included different levels of education goals: Whereas the outputs from activities like classroom construction/ rehabilitation, provision of teaching and learning materials, teacher training and others) aimed at improving the learning environment for students in primary (textbooks only) and lower secondary (all activities) schools, it was also expected that the improved learning environment would impact on improved learning outcomes at the end of the lower secondary in the same project implementation period of six years. In hindsight, this was clearly ambitious. 14 49. On the level of the PDO, only two (#1 and #4) out of the four PDO indicators show reasonably good progress against targets formally revised through the 2013 restructuring. 50. In particular, PDO indicator #1: the number of students taking the CSEE increased considerably since project inception from a PAD baseline value of 5,389 in 2005 to 13,051 in the year 2012 (see Figures, 1, 9 in Annex 6). Hence, actual values only slightly missed the target (13,962) as formally revised with the Restructuring of April 2013 (93.5% of the target value). When disaggregating numbers by gender, it must be recognized that while the gender ratio was always very close to 50:50 up to the year 2009, the share of girls has increased considerably since then (approx. 56.7% of CSEE takers were girls in the year 2012). This is certainly an interesting trend which will require MoEVT’s attention in the future. 51. Similarly, PDO Indicator #4 (introduced with the restructuring of April 2013): lower secondary completion rates 15 have increased significantly from the Restructuring Paper (RP) 14 The figures used for final target achievements in this Report were provided by the MoEVT during the last Implementation Support mission in November 2013; however, they are not consistent with the values in the Borrower ICR. An explanation for the divergence of data was not given by the Borrower. 15 Defined in the RP as “The percentage of CSEE takers divided by the population aged 17 years”. 10 baseline of 27% for 2007 to 46% in the years 2012, which is quite close to the end-of-project target value of 49% (see Figure 8 in Annex 6). 52. Even though the achievements in values for PDO indicators #1 and #4 are highly commendable, they can very likely not be attributed to project activities, given that most project activities were completed only in the course of 2013, and the figures refer to the exams taken in (late) 2012. This disconnect is also acknowledged in the Borrower’s ICR. It is obvious for component 1, where most newly constructed/ rehabilitated/ furnished and equipped school buildings were finalized and handed over only in early 2013, and to a significant degree also for component 2, where a large quantity of teacher training had not been completed as of January 2013. 53. The two remaining PDO indicators miss previously set target values by a large margin. In particular, the absolute number of students actually passing the CSEE (PDO Indicator #2) has increased at a much slower pace than expected from a PAD baseline value of 4,897 in 2005 to 6,935 in the year 2012, and even the (revised with April 2013 Restructuring) end-of- project target value of 12,705 students was missed (actual numbers are 54.6% of the target value). As a matter of fact, CSEE pass rates dropped massively, overall, both on the levels of Zanzibar, where the rate declined from approx. 91% in 2005 to only approx. 53% in 2012, and Tanzania mainland, where the trend shows an almost identical pattern. (see Figures 2, 3, 6 and 7, in Annex 6) 16. The URT government established a commission to investigate the reasons for the observed decline in CSEE pass rates, but the final report still remains confidential. It appears likely that factors external to the ZBEIP project may be primary causes for these negative trends. 54. The number of students passing the CSEE with Division III or better even (PDO Indicator #3) decreased over the last years from a value of 990 students in the year 2005 to only 757 students in the year 2012 therefore likewise clearly misses the set target values (see Figure 4 in Annex 6). 55. The failure to achieve the target values for PDOs 2 and 3 is the reason why the PDO achievement is rated Moderately Unsatisfactory. Before restructuring At Project closing Overall 56. As to World Bank - against initial - against formally guidelines, a weighted rating has to PDO indicator revised PDO targets indicator targets and be applied for outcome new indicator achievements in cases where a 1. Rating MU MU MU project was restructured. In the 2. Rating value 3 3 MU case of the project under consideration, there is no 3. Weight (% 77% 23% 100% disbursed) difference in ratings of the PDO 4. Weighted value 2.31 0.69 3 before and after the restructuring, hence the restructuring was 5. Final rating MU insignificant to the overall rating. 16 The results of the 2013 CSEE examinations for Tanzania overall show an overall trend of improvement, i.e. an increase of the overall pass rate of public school students by 14 %age points from 43.07% to 57.09%, equivalent to an increase by 32%. Disaggregated figures for Zanzibar were not available at the time of the completion of this Report. 11 57. Note that the final achievements of both individual components are rated Satisfactory in the Aide Memoire of the last Implementation Support mission of November 2013 and Moderately Satisfactory in this ICR (see Annex 2, Outputs by Components), since the envisaged outputs have been delivered without any major shortcomings. It can be expected that the impacts of improved learning environment created through project activities will be seen in student learning outcomes in the years to come. 3.3 Efficiency 58. The efficiency of project is assessed as substantial: The project broadly succeeded in implementing its program within the expected time frame, with an extension of the closing date of only four months to complete activities. A comparative (across countries) unit cost analysis undertaken for two major Project outputs, i.e., classroom construction and textbook provision, comes to the conclusion that the efficiency standards are maintained and in line with other similar projects in Africa. (See Annex 3). 59. Over the project implementation period, the government has made considerable effort in executing the Project efficiently, with the aim of improving value for resources spent. In particular (a) under Component 1, the Government reduced the scope of the original design which was too extensive by focusing on just the core school facilities 17, and eliminating or reducing add-ons such as hostels (which were deemed unnecessary in a small island environment) and staff houses; (b) met the targets for school construction/rehabilitation while containing construction costs; and (c) procured all textbooks for 1-4 grades instead of books for (a lower number of) secondary school students. 60. Efficiency gains in textbook procurement were realized when USAID offered in September 2009 to provide secondary school math textbooks for Forms I-IV and V-VI, which had initially expected to be covered by Project Component 2, resulting in resource savings. 61. At mid-term, options for savings in the amount of US$6.5million equivalent, or about 20.6% of the credit allocation, had been identified by reducing the scope of work and limiting resources to priority areas in construction without jeopardizing the PDO. The savings realized were used to cover cost-overruns elsewhere or needs that emerged during school construction, i.e., the construction of housing with cafeteria for student-teachers and completion of staff houses at BWMTTC /Pemba. 3.4 Justification of Overall Outcome Rating Rating: Moderately Unsatisfactory 62. Based on Modest efficacy with respect to the achievement of the PDO and related indicators, the overall outcome rating of the project is rated Moderately Unsatisfactory – even though the overall relevance and efficiency were both rated Substantial. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 63. Since, as described, the outputs of the project have been completed only in the latter part of the implementation period, and basically constitute improvements in learning 17 For example, classrooms, laboratories, administration blocks and school utilities. 12 environment, the impacts on enrollment and success of more disadvantaged population groups, i.e., poor and/or female students, cannot be established for the time being, and neither can any other social development impacts. (b) Institutional Change/Strengthening 64. The intensive collaboration of the Bank team with Ministry staff involved in project implementation has contributed to a significant improvement of project management capacities within the MoEVT, including monitoring & evaluation through the EMIS. While the EMIS was quite weak in the earlier phases of project implementation, it now produces reliable and comprehensive data, an achievement that helped, for example, for the preparation of the recently approved GPE Project. (c) Other Unintended Outcomes and Impacts (positive or negative) 65. So far, unintended outcomes and impacts have not been observed. 4. Assessment of Risk to Development Outcome Rating: Substantial 66. The risks identified in the PAD were appropriate, but the most relevant risk, the inexperience and weak implementation capacity of the MoEVT was not explicitly mentioned in the risk section of the main text in the PAD. The overall project financial management risk at appraisal was assessed as Substantial, and analogous, the procurement risk as High. An overall risk rating was not determined in the PAD. 67. Given that there is no follow-up operation, neither from the Bank nor from other donors except for some elements of the GPE project, and also that government resources are limited there are some questions about the medium- to long-term sustainability of the project’s achievements. These questions have also been raised in the Borrower’s ICR. The government has allocated the annual amount of TSZ1 million (equivalent to about US$612) to each school for maintenance purposes, including payment of utilities. However, it seems that maintenance of school facilities constructed is not always assured as desirable. 18 18 Observations during school visits suggested that some of the school compounds lack fences, making the ground vulnerable to intrusion (by livestock, passers-by, etc.) and pollution; not all schools have maintenance plans in place to take care of both clippings left by students all over the compounds, landscaping and small repairs. Further, there is an unreliable provision of water for school sanitation in the absence of boreholes which could be used during public water supply shortages. In several schools, newly bought science equipment and materials remain unused (some not even unpacked), due to the non-availability of lab technicians, ICT specialists and librarians. One head master, whose school was recently moved into a newly constructed building, observed that as a result of the installation of electric equipment, e.g., fans, lights, maintenance costs would be much higher than previously. He anticipated increased expenditure on stocking science labs once the materials provided by the project were used up. Although unable to provide exact figures for these newly incurred expenses, in the (likely) extent the government allocation was not enough he hoped to cover the shortfall through parental contributions. 13 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory 68. The Bank team prepared the Project in a relatively short time, within a period of less than one year, from the first Identification Mission in May 2006 and Concept Review in October 2006 to Approval in April 2007 19, and did full due diligence on all relevant Bank- internal review procedures and consultations: Preparation steps included a Quality Enhancement Review (QER, January 2007), and two review meeting with AFTQK in February 2007. The QER, amongst others, expressed concerns about the risk of implemen- tation overload particularly with respect to required procurement of all sorts of educational equipment and materials and add-on subcomponents, made suggestions to eliminate, merge and sequence activities, and also recommended hiring a local staff for general implemen- tation support right from the beginning. The AFTQK review as well expressed considerable concerns about the overall implementation risks given the inexperience of the implementing agency. The detailed project description in the PAD does not provide reason for the assumption that these concerns and/or recommendations have been fully taken on board. However, the design of the project was broadly endorsed during the review processes, including by the QER and AFTQK. No concerns by any of the reviewers were documented on the appropriateness of the results chain and indicators. 69. Despite these efforts, in hindsight QaE cannot be rated other than moderately unsatisfactory for the following reasons: (i) The lack of readiness for implementation caused early delays with massive impacts on the implementation period; (ii) the inclusion of too many activities into the design, specifically of Component 2, strained the capacity of the MoEVT which had been diagnosed as weak during preparation; (iii) the overoptimistic objectives and indicators related to learning outcomes distracted attention away from the solid and necessary work on creating a conducive learning environment, which was central to the Project, de facto. It also has to be noted that there are inconsistencies and ambiguities in the PAD, for example, as to the resources set aside for each component, and the de facto existence of a component 3 for project management, which is not mentioned in the PAD main text, but appears later on in Annex 5 on Project Costs. Another issue that came out only in the last Implementation Support mission of November 2013 is that two Intermediate Indicators that were mentioned in the PAD main text had (erroneously) not been included into the Results Framework (Annex 3), and hence, overlooked and never monitored. (b) Quality of Supervision Rating: Moderately Satisfactory 70. The Bank team in charge of implementation support for the Project, based in Dar-es- Salam, took significant efforts, particularly in the earlier stages of project implementation, to help government staff to get up to speed on implementation, procurement etc. Altogether 11 formal implementation support missions were undertaken (two per year, except for 2011, the year of the MTR, and in 2012, where only one implementation support mission was undertaken). The close proximity of the major part of the team allowed for frequent direct 19 See above, on political economy issues. 14 exchanges with the MoEVT team through an uncounted number of mutual technical visits 20, phone and email follow-up in addition to the formal implementation support missions, which was an important factor in realizing Project achievements. The Aide-Memoires provided sufficient detail of progress made and challenges. There was a high degree of stability on the Bank side, with only one TTL-change during the implementation period, and the change was for a member of the task-team who had already been involved into the preparation of the project and who then led the team until project completion. He was supported from mid-2009 by an international textbook specialist, from November 2010 by a local staff with senior level experience in housing construction, and from January 2013 by an expert on teacher development. Various other experts were brought in and provided targeted support on individual activities. Finally, the Bank’s procurement and financial management specialists were available on a regular and by-needs basis for hands-on-support to the implementation team. 71. However, while pro-active efforts were taken to tackle the initial implementation problems and deliver the planned outputs, the team did not respond to the challenges related to the design of the results framework. The QAG QaE Review recommended adjusting what was perceived as unrealistic ratings (see above, para 32). The ISR# 7 (of June 2010) followed suite, and announced a revision of indicator target values; the comments of the Sector Manager at the time questioned the value of the indicators related to the CSEE as well as the assumptions taken at the design phase, and the Country Director, at the time recommended restructuring and advancing the Mid-Term Review (MTR). The team did not pursue this route, hoping that the envisaged results on the level of PDO indicators could still be achieved with accelerated implementation progress. Both DO and IP ratings were kept Satisfactory, based on positive development of trends related to CSEE examinations, which turned to be encouraging, at the time. It is true that indicator targets were adjusted in ISR#8 (of March 2011), i.e., the baseline data which initially referred to 2005 were modified to refer to 2007 from then on - but these changes were not processed as a formal restructuring. The project ratings were finally downgraded (to MU for DO, and MS for IP) one year following the MTR, in June 2012 (ISR#10). The Project was not formally restructured until April 2013. At this point, the team missed a chance to revisit decisions made at the design stage, specifically on the results framework and some critical assumptions, which were discovered to be obsolete. 72. It is questionable, whether a restructuring of both PDO and PDO indicators would have made any difference to the ultimate achievements in terms of outputs. For the project team on the ground such a restructuring would have been perceived as a merely bureaucratic exercise, which would not have made any difference when it came to turning implementation performance around and assuring the delivery of number and quality of schools constructed or rehabilitated, teachers trained and learning materials distributed. The team must be commended though for turning project implementation around at an advanced stage of project implementation, and that is why Bank performance for Quality of Supervision is rated Moderately Satisfactory. 20 There were at least 10 technical visits of team members to Zanzibar, and 6 visits of project counterparts from Zanzibar to the Bank Office in Dar-es-Salam. 15 (c) Justification of Rating for Overall Bank Performance Rating: Moderately Unsatisfactory 73. Despite the need for considerable efforts from the team to build MoEVT implementation capacity and provide hands-on support, outputs were delivered in a satisfactory way (see Component ratings in Annex 2). However, since Bank’s performance in ensuring quality at entry being rated Moderately Unsatisfactory, as was the overall Outcome rating of the Project, the overall rating for Bank Performance is Moderately Unsatisfactory. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory 74. URT Government had limited involvement into the preparation of the project, and had not applied for Project Preparation Facility (PPF) funds to provide resources for activities that would have enhanced readiness for implementation during preparation. However, the RGZ was highly supportive of the project from the beginning, and showed high ownership. During the final Implementation Support mission, the MoEVT expressed his gratitude to the Bank for having supported this project which was on top of the priority list for Zanzibar and considered the most successful one implemented by the MoEVT so far. The MoEVT pro- actively prepared its own highly informative, comprehensive and thorough ICR, which was discussed with the mission team, and is annexed to this report (see summary in Annex 5). (b) Implementing Agency or Agencies Performance Rating: Satisfactory 75. Implementation was the responsibility of the MoEVT. While it is true that the Implementation Agency had to address considerable capacity challenges at project start, this was more than equalized by the pro-activity, ownership and enthusiasm of Ministry staff to make things work. The challenges for the Ministry had been aggravated by the resignation of the Procurement Specialist in April 2008, who was never replaced. The Ministry procure- ment team, including a consultant procurement specialist in mid-2009, supported through training and frequent hands-on guidance by the Bank Procurement Specialist, filled in. 76. At mid-term, in June 2011, civil works for school construction were well underway with overall progress of about 40% expected to be completed by January 2012. Due to poor contractor performance, the contract for three schools (Pajemtule, Tunguu and Dimani) had to be terminated in June 2012. The government successfully initiated and completed the procurement process and recruited three different contractors to complete the remaining work by project closing. 77. Critical policy changes affected the pace and scope of project implementation in component 2: In the second semester of 2009 it became clear that the MoEVT had decided to revise the full primary school curriculum (and not only the upper grades, as initially planned). This had implications for project activities and schedule, for example for teacher training for upper primary which was postponed until the later years of the project. In light of these changes, it was decided for the Project to rather provide lower grade textbooks, and have the upper grade textbooks delivered from other sources. 78. The textbook policy, fundamental for the preparation of textbooks linked to the new Standard I curriculum, was submitted for the Minister’s approval and issuance by August 1, 2011. The new curriculum came into effect in school year 2010/11. New text books were not ready until 2012, hence implementation of the new curriculum started without all necessary provisions in place, using existing textbook and teaching and learning materials. 16 (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 79. The Zanzibar MoEVT was very enthusiastically engaged in the implementation of the Project. But there were capacity gaps and a need to learn and adjust to World Bank procedures. The Project benefitted from full support of the Minister, and the implementation team was led by a pro-active and closely engaged Project Coordinator. Upcoming challenges were addressed and strong efforts taken to move project implementation forward. Last but not least it is noteworthy that the Borrower prepared a very detailed and rich own Implementation Completion Report which perfectly complements the Bank ICR. 6. Lessons Learned 80. The lessons learnt from this project are about due diligence, and about paying attention to political economy-related factors of influence that might not have been immediately obvious during project preparation and implementation: 81. Assure readiness for implementation upon approval, and where this is not possible, listen to early warning signs and follow advice given to mitigate insufficient readiness and/or capacity issues right at the start of project implementation. This includes taken into consideration advice given during preparation: A dedicated Implementation Specialist would have been able to follow-up and help solve the initial challenges related to the procurement and finalization of project designs, which would have helped the project stay within the timeline. A Construction Specialist on board right from the beginning of project implementation might have been able to provide more focused support. This is not a new lesson, but one that was reinforced during the implementation of this project. 82. Revisit assumptions made during appraisal on a regular basis, listen to feed-back and, in response, restructure rather early than late. There were feed-back loops, and red flags did in fact show indicating that results achievement was not on track - but they were put aside in favor of other priorities: The challenges the Project faced in matching the anticipated results chain could have been addressed at a time when a turnaround would still have been possible. The impact of a changes or additions to the results framework is negligible, when they come in a late stage such as the last year of implementation. 83. Pay attention in a pro-active manner to situations that are potentially politically sensitive. One of the more structural issues this project suffered from might be related to a comparative marginalization vis-à-vis interventions on the mainland: Compared to operations in education and other sectors in mainland Tanzania, this was a rather small project, on an island separated physically and politically from the mainland. Perhaps due to its relative isolation and the sensitive political economy between the Tanzania federal government and Zanzibar, the Project did not receive the assistance it needed from the mainland government, (at least not in the early stages of project implementation). 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies (b) Cofinanciers (c) Other partners and stakeholders 17 Annex 1. Project Costs and Financing (a) Project Cost by Component 21 Actual/Latest Appraisal Estimate Estimate %age of Components USD millions USD millions Appraisal (SDR millions) Total cost IDA Total cost w/contingen IDA Funding IDA funding Funding cies Component 1: Increase enrolment and improve quality in secondary 30.6 31.2 35.1 28.7 106.6% education through provision of (SDR 19.9) infrastructure Component 2: Improve quality of education through teacher training, 11.9 10.9 11.3 11.3 105.3% curriculum reform, and provision of (SDR 7.8) learning materials 0.8 Project Management 1.5 1.6 1.5 55.3% (SDR 0.5) Unallocated 0.5 0.5 0.5 0 Physical Contingencies 1.5 Price Contingencies 2.8 -0.1 Exchange rate loss (SDR -0.07) 43.3 103.1% Total Project Costs 48.5 48.5 42.0 (SDR 28.2) (b) Financing Appraisal Actual/Latest Type of Estimate %age of Source of Funds Estimate Cofinancing (USD Appraisal (USD millions) millions) Tax Borrower 6.5 7.8 120% contributions International Development Association 42.0 43.3 103.1% TOTAL 49.8 51.1 102.6% 21 Cost by component was estimated based on the SDR amounts by category indicated in the closing letter to the URT MoF of April 22, 2014. 18 Annex 2. Outputs by Component Component 1: Increased enrollment and improve quality in secondary education through provision of infrastructure ($35.1 million) (Moderately Satisfactory) 84. Overall implementation progress of this component which seeks to improve the quality of secondary education through provision of infrastructure was Moderately Satisfactory. All activities related to this component have been implemented as planned. 85. The Project built 19 School and rehabilitated 6, and with that, added more than 16,100 student spaces, 339 classrooms, 75 science laboratories, 28 computer laboratories, 27 libraries and 54 staff houses. The supply, delivery and installation of all earmarked goods (furniture, equipment) in respective schools have been completed. A total of 672 teachers have been allocated to the new schools. 86. Detail of completed schools, student enrollments and overall components output details are given in a Table 2 below. Table 1: Summary of Component 1 Implementation status PROJECT TARGETS STATUS AT END OF LAST ISM Construction of 10 upper secondary schools Construction of 10 schools have been completed and handed over. All completed schools are in use. Due to budget constraints student’s hostels were dropped, and only 3 instead of 10 staff houses were constructed in all earmarked schools. Construction of 9 lower secondary schools Construction of 9 schools have been completed and handed over. All completed schools are in use. Construction of facilities for BWMTTC Construction has been completed, handed over and put in use. All earmarked facilities were constructed as per original plan. Rehabilitation of 6 schools All schools rehabilitation completed handed over and in use. 87. During the Project implementation period the pupil-classroom ratio in secondary schools (IOI#1) has developed much more favorably than was foreseen at the time of appraisal: The actual ratio decreased, despite the considerable increases in enrolment, from a value of 83 in the year 2006 to 58 students per classrooms in the year 2012 (see graph on Pupil-Classroom Ration 2006-2012) which is considerably better than the targeted value of 92. This improvement has to be analyzed in light of the development of overall enrolment figures across time. The trend cannot be attributed to the Project, given the late completion of classrooms constructed or rehabilitated. 19 Table 2: Overall Component 1 Outputs Name of school Lower (l) Constructed No. of Students No of No. of No. of No of Higher (c) class- capacity science computer library staff (h)** Rehabilitate rooms labs labs houses Sec. School d (r) 1 B.W. Mkapa TTC c 12 480 3 1 1 10 2 Chaani l c 8 320 3 1 1 0 3 Chwaka Tumbe h c 12 480 3 1 1 3 4 Dimani l c 12 480 3 1 1 0 5 Dole l c 8 320 3 1 1 0 6 Fidel Castro h r 14 22 560 3 1 1 16 7 Forodhani* h r 10 667 3 1 1 0 8 Hamamni* h r 17 1360 3 2 2 0 9 Kiembe Samaki h c 26 1040 3 1 1 0 10 Kiwani Mauwani h c 12 480 3 1 1 3 11 Konde l c 12 480 3 1 1 0 12 Kwamtipura* h c 24 23 1920 3 1 1 0 13 Matemwe h c 12 480 3 1 1 3 14 Mpendae* l c 12 960 3 1 1 0 15 Mwanda h c 12 480 3 1 1 3 16 Mgwachani l c 8 320 3 1 1 0 17 Paje Mtule h c 12 480 3 1 1 3 18 Pandani h c 12 480 3 1 1 3 19 Shamiani* l c 12 480 3 1 1 0 20 Tumekuja* h r 12 480 2 1 1 0 21 Tunguu l c 8 320 3 1 1 0 22 Utaani (Chasasa) h r 22 24 880 2 2 1 2 23 Utaani (Chasasa) l c 12 480 3 1 1 2 (extension) 24 Uweleni* h r 14 721 2 1 1 0 25 Uzini h c 12 480 3 1 1 3 26 Wawi h c 12 480 3 1 1 3 Total 16 (h) 339 16,108 75 28 27 54 9 (l) ** Higher secondary schools always also include the lower secondary grades. * Schools with double shift (morning and afternoon session) 22 Initially planned: 8 classrooms 23 Initially planned: 24 classrooms 24 Initially planned: 16 classrooms 20 Component 2: Improve quality of education through teacher training, curriculum reform, and provision of learning materials ($12.1 million). (Moderately Satisfactory) 88. All activities described in Component 2 were completed. While the accomplishments under component 2 are equal in outputs to those in the PAD, almost every activity was revised during implementation to improve its efficacy, cost- effectiveness, or to better fit current needs. The main activities in this component were (i) the training of teachers, and (ii) the provision of textbooks; these are reflected in the Intermediate Indicators. 89. Looking at the IOIs #2-6, focusing on the percentage of teachers who are qualified to teach at the secondary level, notably in math, biology, chemistry, physics, the picture is positive overall given that considerable progress can be observed for all five indicators 1. Only one (IOI #2) out of the five indicators met the quite ambitious PAD target value of 65%, though: As shown in Figure 10 below, the share of teachers at secondary schools who are actually qualified to teach at this level, has increased faster than was expected in the PAD reaching a value of 79% in the year 2012 (compared to a target value of 65%). It has to be acknowledged, however, that the significant overall increase was not due to the increase in qualification of math and science teachers, which were in the center of the project: The subject-specific shares of qualified teachers each (approximately) reached the target value, but did not overachieve. Only for mathematics the 65% mark was missed by a relatively large margin (actual value in 2012 was 47%). 2 1 However, it should be emphasized that the data obtained from MoEVT is rather imperfect for IOs 3-6 since it does not allow grasping the extent of out-of-field teaching (i.e. it only provides the information whether a teacher teaching e.g. chemistry is qualified at all, but not whether he/she is actually qualified to teach chemistry). 2 This has, however, to some extent to be attributed to the fact that the baseline value of 28% was particularly low for this subject which made the target already from the beginning very ambitious. 21 90. Similarly, for the IOIs #7-10 which correspond to the textbook-pupil ratios in Forms III + IV3 for the core subjects Mathematics, Physics, English and Kiswahili considerable improvements can be observed since initiation of the ZBEIP project. In fact, the actual values even surpass the target ratio of one textbook per student in all four core subjects and it can hence be assumed that, at least on average over all schools, core subject textbooks are sufficiently available for secondary school students.4 (See details in part F of the Data Sheet above). 91. The PAD suggests two more IOIs in its main text (p. 4), i.e., the percentages of teachers successfully completing (and passing the examination) in the in-service language and mathematics/science courses, respectively, which were however not included in the Results Framework Annex 3 (pp. 20-22). For this reason, these two indicators were neglected, except for during the last Implementation Support mission in November 2013. It became apparent that the required data were apparently not centrally available for most of the years. Yet, MoEVT was able to provide examination results for the year 2012 for the science and mathematics trainings on the island of Unguja only (see Figure 11), which look relatively encouraging given that approx. 90.7% of the teachers passed the final exam (PAD target value was 90%). Textbook Procurement 92. The project funded the provision (development, purchase and distribution) of 213,395 Standard 1-4 textbooks of Mathematics, English, Social Science, Science and Kiswahili. 93. During the original project design, funds had been designated to provide textbooks for lower and higher secondary school students. In addition, some funds were set to support the purchase of textbooks for upper primary (Standards 5 and 6) to align with the new lower and upper primary curriculum, which introduced English as the medium of instruction in Standard 5 instead of the beginning of lower secondary. 3 Given that some of the textbooks are used for both, Forms III and IV, the mission team decided to calculate avg. ratios for these two classes together since it was impossible to fully disentangle the numbers used for Forms III and Form IV and numbers for Form IV only would hence potentially be biased. 4 The most likely reason for this rather surprising finding is low data quality on the number of textbooks given out from MoEVT’s central storage which could have led to an exaggeration of the indicator values. However, it can also not be ruled out that poor textbook management led to the situation where more textbooks than actually needed were given out to schools. 22 However, the Government subsequently realized that teaching Standard 4 leavers the old curricula would create a policy shock that would have adverse effects on the students. Instead, the new curriculum was phased in grade by grade, which meant Standards 5 and 6 would not require textbooks until the last year of the project, while Standards 1-4 would require textbooks immediately. Teacher Training 94. The project funded the training of a total of 3,780 primary and secondary school teachers in Math, Science and/or English (see table below.) Training Activities completed in 2013 Activity 2.A.1 2.A.2 2.A.3 Topic Math & Math & Science English Science Content Content Recipient Level Diploma Upper Primary Primary Secondary School Teachers Recipient Math and Math and English, Social Science, Specialized Science Science Math, and Science Subject Days of Training 20 40 20 Supervised Visits 2 3 2 Final Assessment Yes Yes Yes Participants Total Target 500 1,039 2,160 3,699 Complete by 2013 350 0 900 1,250 Complete in 2013 150 1,050 1,339 2,539 Total Complete 500 1,050 2,230 3,780 95. As of January 2013, a large quantity of teacher training had not been completed 5. They included English and communication for upper primary math, science, and geography teachers, content for upper primary math and science teachers, and content for secondary math and science teachers. This brought about two issues: the majority of the project and government funds were depleted on other activities, and, as most teacher training was described in the PAD as multiple years of center- based training, it was impossible to complete as originally described. While the 2013 restructuring freed some Component 1 funds be used for training, they were insufficient to fund all the center-based training described in the PAD. 5 The January 2013 Aide Memoire describes the causes that were inside and outside the Government’s and World Bank’s control. 23 96. Instead of the classroom-style center-based training described in the PAD6, the World Bank and Government teams worked together to develop an approach that ended up being more economical, timely, and potentially more effective at improving teacher practice. The original plan was constrained in that (i) altogether 20 days of center-based training (stretched over two years) was impossible to realize without severely disrupting the school year; (ii) the transport and food costs were insurmountable; (iii) research demonstrates that the most effective teacher training is directly applicable and incorporated into teacher daily practice, which is unlikely with compressed traditional classroom-style training. 97. Hence, trainings hours were adjusted to incorporate fewer center-based training days, and the prepared pace of center-based training was increased to cover more material. One-third of the training days originally designated as center-based were replaced with school-based training (SBT). Instead of teachers from approximately 50 schools traveling substantial distances to each training center for day-long training, and the government paying for their travel and meals, teachers would congregate at nearby schools – usually from six to eight schools together -- and conduct more interactive and discussion-based trainings that were facilitated by a fellow teacher, known as a sub-cluster leader. 98. To shift from the didactic lecture style of center-based training to peer learning sessions, the Teacher Education Department capitalized on their experience designing micro-teaching courses. Micro-teaching is a method where teachers are required to develop classroom activities and materials, lead a group of peer teachers through them, and discuss methods of improvement. Micro-teaching resembles the practice-based approaches research has found highly correlated with translating teacher learning into improved classroom practice. 99. The Teacher Education Department developed 8 group activity modules to be completed by teachers in during SBT and at home and largely incorporated micro- teaching. Each session was centered on a different topic: teaching new concepts, lesson planning, games, and student evaluation. Leaders built on center-based training on these concepts, and each teacher worked to develop an activity on the topic for a topic they were currently teaching. On the second day, teachers would lead their peers in the activity who would afterwards provide feedback. To ensure that sub-cluster leaders provide useful training, trainers would conduct supervisory visits to SBT and support sub-cluster leaders as necessary. 100. By reducing travel times and shifting to half-day SBT sessions after classes, teachers were able to receive a portion of their training without interrupting the academic schedule. In addition, shorter trips or walking to the SBT venues reduced or eliminated travel fees; half-day sessions meant reduced meal costs. 101. More importantly, by working in small peer groups and establishing local leaders in sub-cluster leaders, teachers met with after the project reported making use of the activities they developed during training in the classroom and a strengthened confidence in their abilities. 6 Three cohorts with 360 teachers each were each to participate at 2 years courses with 2 weeks of center-based training and one supervision visit per year, completed by a final examination at the end of the 2 years period. 24 102. In more detail, the following Sub-components were covered by this component: 2.A.1: In-service upgrading of course content for secondary science and mathematics teachers 103. This subcomponent sought to finance the development and delivery of in- service upgrading for 500 secondary teachers qualified to teach Forms 1 and 2 but not Forms 3 and 4. Upgrading includes 20 days of in-service training and 2 follow-up visits by qualified staff to provide feedback on teachers' classroom practice. As of July 2013, 500 teachers had received the training and follow-ups. 104. Many teachers interviewed during the last Implementation Support mission of November 2013 felt more confident and competent in their subjects. However, review of follow-up suggested that the ability to teach in English was still very limited. Given that secondary school teachers are expected to use English as the medium of instruction, almost all teachers will require substantially more training before they are qualified to use English as their primary means of instruction. In addition, teachers who received the training but had minimal content knowledge beforehand, and the training was insufficient to prepare them; they require additional training to meet the expectations set for them by the curriculum and student standards. Although the purpose of the upgrading was to provide for more qualified teachers in Form 3 and 4, some headmasters were not ready to allocate the trained teachers to higher levels due to high demand at all levels. 2.A.2 In-service upgrading of course content for upper primary science and mathematics teachers 105. To prepare for the imminent changes to the curriculum in primary schools, including the transition to English as the medium instruction in upper primary core courses (including math and science), the project included in-service upgrading for approximately 1,039 upper primary math and science teachers 7 . The course was intended include a refresher of the appropriate curriculum, information on the revised curriculum, and teaching methods. It was to be delivered in English to enhance familiarity with the terminology and English communication. Over 1,050 teachers received the training, practice, observations, and evaluation largely in the past 11 months. Although the project did not complete the activities over multiple years as described in the PAD, it was completed using a practice-based approach that evidence suggests is more effective at translating into changing teacher practice. In addition, it built substantial capacity for continuous teacher learning by building a substructure of teacher networks, local pedagogy leaders, and training designers as well as a culture of teacher improvement that did not exist before the project. 7 This value included head teachers and school inspectorates 25 2.A.3: In-service upgrading of upper primary English, science, math, and social science teachers in English and Communication skills 106. As core courses continue the transition to English as the medium of instruction, 2,000 teachers were planned to receive training on use of the English Language as a medium of instruction through a two-year training program. The program was to include two weeks of center-based training annually and one supervision visit each year. As of February 2012, the course design, training of trainers, and 360 teachers had completed the training. As of January 2013, 300 social science and 500 math teachers remained to receive the training. In the past 6 months, 400 mathematics teachers received 20 days of English language training in Zanzibar. 2.A.4: Capacity building for Head Teachers, Subject Advisers, and Inspectors 107. At the beginning of the project, half of all head teachers had received no training for their position. This activity was included to fund development a two-week training course for head teachers, a training of trainers, and delivery of the training at all training centers in Unguja and Pemba. The training was completed by February 2012. As of January 2013, a final report had been drafted. In May, members of the Government received and provided feedback to the consultant who wrote the report. The final version of the report was shared during the mission and the World Bank received a copy. 2.A.5: Revision of pre-service primary (certificate) curriculum 108. The purpose of this subcomponent was to align the pre-service primary teacher certificate curriculum with the new primary school curriculum for core subjects, and was designed to be implemented in the training college by August 2008. The activity was completed by July 2012. 2.A.6: Interim Upgrading of pre-service secondary teacher (diploma) curriculum in science and mathematics 109. While the activity outlined in 2.A.2 provide some training to current secondary math and science teachers, it did not address those entering the system. All activities under this component were completed by July 2013. Training similar to that outlined above would be incorporated into the pre-service training of new math science teachers and provided to all teachers attending the State University of Zanzibar (SUZA). To increase the impact of the trainings and capitalize on its provision, they invited 100 current teachers to the Unguja training and 50 in Pemba. By combining their efforts they made highly efficient use of limited resources and provided training to a total of 325 teachers. The World Bank mission congratulates the Government on this activity’s completion. 2.A.7: Management of entrants to teacher training 110. To better understand the characteristics, motivation, incentive receptiveness, capability, and needs of teacher training entrants, it was agreed previously to finance a study of the management of teacher training. The study should help provide 26 consistency and coherence among the project’s several interventions to improve teacher quality and ensure a broader understanding of teacher issues. It should include an analytical review of the adequacy of incentives, improvements in the working conditions for teachers, and enhancement of accountability measures. The World Bank provided feedback on the report draft in July and submitted a finalized report before the mission. 2. B: Provision of instructional materials in key subjects for all secondary schools and students 111. This component served to support the development of a policy to sustain provision of textbooks in secondary school, the purchase and distribution of science equipment, consumables to all schools teaching science, and the purchase and distribution of library books and other learning materials for each secondary school. It was planned initially that all materials would be purchased and distributed by 2007. 112. On textbooks, see more detailed discussion above. In January 2013, the initial distribution was completed to attain a ratio of one new book per student in each subject based on the previous year’s attendance data. Remaining books were kept in warehouse and are distributed upon request by head teachers. Because ongoing distribution cannot account for future needs, this practice will result in schools approaching but never quite having sufficient textbooks for every child. During the mission, group discussed how textbook inventory and distribution could be better planned. 113. As of July 2013, materials for disabled students had been purchased and 60% of materials had been distributed. It is unknown if the Human Resources Department distributed the remaining materials as agreed. 2. C: Revision of primary curriculum and provision of appropriate learning materials 114. This component was designed to support a consultancy and workshops to develop a new curriculum for upper primary standards, train 900 upper primary school teachers, and purchase and distribute textbooks in English in five subjects for each pupil in upper primary school as well as a package of instructional materials for each school. Instead of revising only the upper primary curriculum, the MoEVT decided it was more appropriate to revise the entire primary curriculum, which was completed by 2009, and performance standards have been created as the curriculum is phased in. When the PAD was developed, the Government intended to implement the new curriculum in all primary grades simultaneously. However, concerned that it would shock the system and poorly serve students to be trained in one curriculum for half of primary school and a different curriculum for the remaining portion, they began those that started Standard 1 in 2008 using the new curriculum. Currently, cohorts of students in Standards 1 through 5 are taught using the new curriculum, and Standards 6 and 7. 27 2. D: Support for science education for girls 115. To support improvement of female student performance in mathematics and science, science camps for girls were to be i) designed; ii) offered to 100 female secondary students of science each year; and iii) evaluated externally in the third year. To date the science camp has been conducted once for approximately 300 female students, the content for four years has been drafted, and the one camp was evaluated externally, and the final version of the report was shared with the mission. The 2011 camp was cancelled due to difficulties finalizing the content manual. It was cancelled in 2012 for funding concerns and difficulties finalizing the content manual. In January 2013, it was agreed that the content for a camp would be finalized and photocopied at the MoEVT instead of published to lower costs by February 28, and the camp would be conducted by April 15. The final camps were cancelled due to a lack of funds. Instead of funding future science camps, the materials will be used in after-school science clubs, which will be more economical and – as demonstrated by the implementation struggles faced during this project – more likely to be implemented. The Zanzibar Institute for Education has committed to training 45 teachers (five per training center) after the project’s close using government funds. The materials were printed by the Ministry and published the final report and the materials on the MoEVT website. 2. E: Provision of life skills, including HIV/AIDS, gender, and environmental awareness 116. This component was designed to help biology and geography teachers incorporate life skills and other topics into their teaching as well as support extracurricular secondary school workshops. By 2011, 1,600 teachers had received training on the manual's use, and by 2013 over 100 extracurricular health club leaders in Pemba received training. While it was agreed in July that an additional training would be conducted by the project’s closed, the Government did not prioritize funding. Three of the four schools visited during the last Implementation Support mission reported receiving copies of the manual, and they had indicated training the teachers on its use. Many teachers reported a positive impact among their students. Challenges noted include high teacher workloads, preventing extracurricular work, and some trained teachers transferring schools. The teacher management report cited that the life skills education had positive life outcomes on students, but the school health clubs suffered from lack of allocated time, resources, and support mechanisms necessary for them to sustain themselves in the long-term. 28 Annex 3. Economic and Financial Analysis 117. This annex includes an analysis of the economic rationale for investments in basic education in Zanzibar, and the cost effectiveness of ZBEIP focusing on efficiency using the unit-cost analysis method. It is worth noting that the project was the main intervention in the education sector in Zanzibar and first World Bank credit to the government of Zanzibar. The Project played a pivotal role in the changes observed in the education system during its implementation. 118. Economic theory states that education helps building human capital which is essential for development of society and is typically associated with higher earnings over an individual’s life span. A report by wage indicator survey 2013 on wages in Zanzibar found out that workers with tertiary education earn thrice as much as those with primary education8.Moreover, educated individuals tend to be better-nourished, healthier, more productive, and generally show an increased participation in public life. Educated women usually enjoy more autonomy and their level of education level is associated with lower fertility rates and lower infant/child mortality rates. In line with various theoretical models (e.g. the Augmented Solow Model), many empirical studies using cross-national data from industrialized and developing countries highlight the important contribution of education to economic growth 9 . Current education spending is a fundamental determinant of the future educational profile of the working age population, thus, impacting future labor market outcomes and other socio-economic outcomes, including better earnings, secured employment opportunities, and overall well-being. 119. In order to assess whether the investments were cost-effective and efficient, it is important to conduct an efficiency analysis, which in this case will be undertaken using the unit cost-analysis method. In the PAD economic and financial analysis provides the private and social rate of returns to education and argues that among the levels of the education systems, secondary education is the level most deserving of expansion from the economic perspective. Furthermore, the PAD also stipulates the social benefits of expansion in the secondary education. The nature of education projects is such that analyses conducted immediately upon a project’s completion are unlikely to be fully comprehensive as many of the benefits that accrue from investments in education will materialize in the medium to long term; furthermore, at this point it is not feasible to quantify and monetize the benefits hence prohibiting cost-benefit analysis. Unit - Cost Analysis 120. As mentioned earlier, data availability is a major constraint, making rigorous, quantitative analysis difficult. Although detailed information on the costs of this project are available, data from other comparable projects is generally unavailable. 8 Tijdens, K.G., Besamusca, J., Kahyarara, G. (2014). Becker, 1964; Smits and Hoşgör, 2006, Jejeebhoy, 1995; Martin and Juarez, 1995; Lutz and Goujon, 2001; Basu, 9 2002; Goujon and Lutz, 2004; Joshi and David, 2006; Samir, Barakat, Goujom, Skirbekk, Sanderson, and Lutz, 2010, Harbison and Myers,1964, Denison 1962, and Benavot 1989. 29 Since this is the first education project in Zanzibar we will make the comparison using information from other similar projects in nearby countries in Africa (namely Burundi and Eritrea) as opposed to comparisons within Zanzibar. The strategy used will be comparing the cost per unit of output with different projects. In this regard we will compare the unit cost of two outputs namely: 1. Construction of classrooms which comprises up to 83% of the total funds and 2. Development, production and distribution of textbooks under ZBEIP. Nevertheless, comparisons of unit cost across projects have to be taken with a grain of salt as projects’ activities are not entirely comparable. Moreover, input prices can vary widely across countries or time causing the production of the same output to have substantially different costs (e.g. labor and transportation costs or energy). This adds two complications 1. of bringing all projects’ cost figures to a comparable year (i.e. deflating the costs), (which in our case has been chosen as 2007 due to availability of the data for the projects and other country specific variables needed to make this transition) and 2. Comparing costs across countries. The variables used during these comparisons are World Bank's annual inflation rates (to bring the cost figures to 2007) and Purchasing Power Parity (PPP) conversion factor to market exchange rate ratio (for comparison across countries). Table 1: Cost per unit output in USD ZBEIP ESRP EESIP (Burundi) (Eritrea) Intervention Construction of Construction of Construction Classroom Classroom of Classroom Includes furniture? Yes Yes Yes For which year are 2014 2007 2009 costs mentioned? Cost per classroom 103,539.8 27,316.02 45,193 Comparable cost 34,827.33 27,316.02 32,456.91 per classroom in 2007 Intervention Provision of Provision of N/A textbooks textbooks Includes Yes Yes N/A distribution? For which year are 2014 2007 N/A costs mentioned? Cost per textbook 5.29 1.15 N/A Comparable cost 1.77 1.15 N/A per textbook in 2007 121. Component 1 of ZBEIP consisted of improving the quality of secondary education via means of construction of infrastructure. Around 20 schools were constructed with a provision of over 300 classrooms which have a capacity of over 30 16,000 students. Table 1 compares the unit cost of constructing a classroom with ESRP10. and EESIP11. We can notice that the cost per classroom under ZBEIP when converted to the 2007 levels and applying PPP conversion factor is similar to EESIP and a little more than ESRP. 122. Similarly, part of the component 2 consisted of developing, producing and distributing textbooks to students. Overall around 213,395 textbooks of Mathematics, English, Social Science, Science and Kiswahili were provided to the students costing around 1.13 million USD. As seen, the unit cost of a textbook for ZBEIP (extrapolated backwards to 2007, see above) although a little higher is similar to ESRP. This slight difference in unit cost can be explained by the fact that ESRP provided textbooks for both primary and lower secondary whereas ZBEIP has provided all the books for lower secondary hence making the cost a little higher. This is in line with the trend that as level of education increases, the cost of textbook pertaining to a particular education level also increases. 123. The slightly higher unit cost among both the components of ZBEIP, more so in the case of ZBEIP and ESRP can be explained by following points which are discussed before as well: (i) these activities are not fully comparable; (ii) input prices may vary across time; and (iii) even though PPP conversion factors have been applied to make the comparison comparable there is a huge difference by GDP per capita (Burundi = $251 and Zanzibar = $656) between Zanzibar and Burundi making the components under ZBEIP more "expensive". 124. This unit cost analysis is important in the case of ZBEIP since this is the first World Bank credit to the government of Zanzibar and hence there are no other avenues (comparing with other projects in Zanzibar) to compare the efficiency of the project. Looking at Table 1 we can safely conclude that the efficiency standards are maintained and are in line with other similar projects in Africa. 10 ICR (Report No: ICR2429): Education Sector Reconstruction Project for Republic of Burundi. 11 ICR (Report No: ICR2429): Eritrea Education Sector Investment Project. 31 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility Names Title Unit / Specialty Lending Aidan Gerard Mulkeen Senior Education Specialist MNSHE Anne Muuna Kisumo Program Assistant ECSPF Arun R. Joshi Lead Education Specialist AFTEE Cecile Ramsay Operations Adviser AFTOS Donald Paul Mneney Sr. Procurement Specialist AFTPE Edes Said Tairo Resource Management Analyst MNACA Edith Ruguru Mwenda Senior Counsel LEGAM Ivar Strand Operations Officer AFTH1 - HIS TTL Mercy Mataro Sabai Sr Financial Management Specialist AFTME Pascal Tegwa Senior Procurement Specialist AFTPE Rest Barnabas Lasway Senior Education Specialist AFTH1 Serigne Omar Fye Senior Environmental Specialist AFTN1 Ammon Y. Mbele Consultant AFTH1-HIS Bruce N. Jones Consultant AFTH1 Claire Louise Greer Consultant SEGOM Daniel Ncayiyana Consultant AFTH2-HIS David Charles Harding Consultant WBIGC Geoffrey D. N. Shoo Consultant AFTU1 Michael Mwangi Wairugu Consultant AFTH1-HIS Patricia Maughan-Colon Consultant MNSHD Patrick D. Murphy Consultant HDGPE Philip Khorassandjian Consultant AFTH1-HIS Silvester Okumu Kasuku Consultant AFTTR Supervision/ICR Agnes Nderakindo Mganga Team Assistant AFCE1 Aidan Gerard Mulkeen Senior Education Specialist AFTEE Andrew Paul Trembley ETC HDNED Anne Muuna Kisumo Program Assistant ECSPF Arun R. Joshi Lead Education Specialist AFTEE TTL Barbara Weber Sr. Operations Officer AFTEE Sr Financial Management Bella Lelouma Diallo AFTMW Specialist Donald Paul Mneney Senior Procurement Specialist AFTPE Edith Ruguru Mwenda Senior Counsel LEGAM Eva K. Ngegba Program Assistant AFTHE Gisbert Joseph Kinyero Procurement Specialist AFTPE Isis Gaddis ETC AFTP5 Jane A. N. Kibbassa Sr. Environmental Specialist AFTN3 Kristine Schwebach Social Development Specialist AFTCS Kristoffer Welsien Water & Sanitation Specialist TWIAF 32 Leah Mukuta Program Assistant AFCE1 Luis Benveniste Sector Manager EASHE Luis M. Schwarz Sr. Finance Officer CTRLA Sr Financial Management Mercy Mataro Sabai AFTME Specialist Financial Management Michael Okuny AFTME Specialist Nkahiga Mathus Kaboko Education Specialist AFTEE Nobuyuki Tanaka Education Economist AFTEE Pascal Tegwa Senior Procurement Specialist AFTPE Rekha Menon Sector Leader AFTHD Rest Barnabas Lasway Senior Education Specialist AFTH1 Sophie Nelly Rabuku Team Assistant AFCE2 Alfonso F. de Guzman Senior Education Specialist LCSHE Carlos A. Rojas Consultant AFTEE TTL Juma Kayonko Consultant AFTEE Kitila Alexander Kanyama Mkumbo Consultant AFCE1 Malte Reimers Consultant AFTEW Patrick D. Murphy Consultant HDGPE Rehema Mercy Mashayo Consultant AFCE1 Tobias Von Platen-Hallermund Consultant AFTN3 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands No. of staff weeks (including travel and consultant costs) Lending FY07 35.23 373,723.55 Total: 35.23 373,723.55 Supervision/ICR FY08 22.14 146,609.82 FY09 25.37 95,596.69 FY10 7.81 67,557.53 FY11 16.55 107,881.64 FY12 32.73 115,249.16 FY13 24.22 81,817.72 FY14 21.55 102,661.72 Total: 150.37 717,374.28 (c) Formal Missions Missions Pre-appraisal November 20-December 1, 2006 Appraisal January 29-February 14, 2007 Technical (SWaP workshop) June 4-15, 2007 Implementation Support #1 October 1-5, 2007 33 Implementation Support #2 February 12-16, 2008 Implementation Support #3 October 05-10, 2008 Implementation Support #4 March 2-6, 2009 Implementation Support #5 September 24-October 5, 2009 Implementation Support #6 May 17-26, 2010 Implementation Support #7 December 13-20, 2010 Implementation Support #8 (MTR) June 6-16, 2011 Implementation Support #9 February 6–15, 2012 Implementation Support #10 January 14 – February 15, 2013 Implementation Support #11 November 14-30, 2013 34 Annex 5: Results Across Implementation Period Actual Values 2008 – 2012 End-of-Project Objective Baseline Baseline PDO/IO Level Results Indicators Original Formally PAD 1 used in RP 2 2008 2009 2010 2011 2012 Target Revised Target PDO 1: Numbers of pupils, disaggregated by gender, T: 5,389 T: 7,250 T: 8,446 T: 8,725 T: 17,030 T: 11,877 T: 13,051 taking the Certificate of Secondary Education F: 2,682 F: 3,708 F: 4,301 F: 4,411 F: 9,372 F: 6,386 F: 7,400 24,455 13,962 Examination (CSEE) M: 2,707 M: 3,542 M: 4,145 M: 4,314 M: 7,658 M: 5,491 M: 5,651 PDO 2: Numbers of pupils, disaggregated by gender, T: 4,897 T: 6,136 T: 6,533 T: 7,066 T: 12,202 T: 9,020 T: 6,935 passing the CSEE F: 2,455 F: 3,186 F: 3,289 F: 3,584 F: 6,695 F: 5,002 F: 3,924 22,254 12,705 M: 2,442 M: 2,950 M: 3,244 M: 3,482 M: 5,507 M: 4,018 M: 3,011 PDO3: Number of pupils, disaggregated by gender, T: 990 T:1,402 T: 1,274 T: 1,408 T: 1,598 T: 939 T: 757 passing the CSEE in Div. III or better F: n/a F: 578 F: 507 F: 624 F: 700 F: 460 F: 373 4,647 2,653 M: n/a M: 824 M: 767 M: 784 M: 898 M: 479 M: 384 PDO 4: Lower secondary completion rate: Number of T: n/a T: 27% T: 31% T: 32% T: 61% T: 42% T: 46% pupils taking the CSEE / population aged 17 years F: n/a F: 28% F: 32% F: 32% F: 67% F: 45% F: 52% n/a 49% M: n/a M: 26% M: 30% M: 31% M: 54% M: 39% M: 40% IO 1: Pupil/Classroom ratio in secondary education 83 83 75 72 72 65 58 92 92 IO 2: Percentage of teachers teaching at secondary 59% 59% 63% 62% 68% 70% 79% 65% 65% level who are qualified to teach at the secondary level IO 3: Percentage of teachers of mathematics who are 28% 28% n/a 3 n/a3 n/a3 48% 47% 65% 65% qualified to teach secondary school mathematics IO 4: Percentage of secondary biology teachers 38% 38% n/a3 n/a3 n/a3 55% 59% 65% 65% qualified to teach secondary school biology IO 5: Percentage of secondary chemistry teachers 43% 43% n/a3 n/a3 n/a3 53% 62% 65% 65% qualified to teach secondary school chemistry IO 6: Percentage of secondary physics teachers 39% 39% n/a3 n/a3 n/a3 55% 57% 65% 65% qualified to teach secondary school physics IO 7: Textbook/pupil ratio - Mathematics (Forms III + n/a n/a n/a n/a n/a n/a 1.4 1.0 1.0 IV) IO 8: Textbook/pupil ratio - Physics (Forms III + IV) n/a n/a n/a n/a n/a n/a 1.1 1.0 1.0 IO 9: Textbook/pupil ratio - English (Forms III + IV) n/a n/a n/a n/a n/a n/a 1.5 1.0 1.0 IO 10: Textbook/pupil ratio - Kiswahili (Forms III +IV) n/a n/a n/a n/a n/a n/a 1.5 1.0 1.0 1 Original PAD baseline values correspond to the actual values for the year 2005 for the PDOs and the year 2006 for the IOs. 2 Restructuring paper’s baseline data refer to actual values for the year 2007 for PDOs and the year 2006 for the IOs. 3 According to MoEVT, the required information for this indicator is not available before the year 2011. 35 Annex 6: Graphs: Trends in Results Achievements over Time Figure 3: Time trend in shares of pupils passing CSEE Figure 4: Time trend in shares of pupils passing CSEE 100.00% with Division III or better 90.00% 25.00% 80.00% 70.00% 20.00% 60.00% Total 15.00% Total 50.00% Male 40.00% Male Female 10.00% 30.00% Female 20.00% 5.00% 10.00% 0.00% 0.00% 2005 2006 2007 2008 2009 2010 2011 2012 2005 2006 2007 2008 2009 2010 2011 2012 36 37 38 Annex 7. Zanzibar Ministry of Education and Vocational Training: Implementation Completion Report – shortened and summarized version I. PROJECT BASIC DADA AND KEY DATES Zanzibar Basic Education Improvement Project Name: Project (ZBEIP) Project ID: P102262 IDA Credit/Grant No.: 42930-TA Implementing Agency: Ministry of Education and Vocational Training Effectiveness Date: January 29, 2008 Closing Date: November 30, 2013 Credit Amount: XDR 28,200,000 US$ 42,000,000 Program Duration 5 years 4 Months Remaining Period to Closing 0 months Disbursed Amount: US$ 42 million PDO Rating: Unsatisfactory Implementations Progress Satisfactory Rating II. CONTEXT III. PROJECT OBJECTIVES AND RESULTS FRAMEWORK 3.3 Review of PAD PDO: A closer examination of the PDO indicators and subsequent review by the Government and Bank supervision missions revealed the following: (i) Unexpected Policy Changes: Some of the policy changes expected to be implemented during the project phase did not materialize or took place at different times periods than anticipated. (ii) Wrong Assumption: The appraisal mission made wrong assumptions: (i) the abolition of standard 7 was expected to take place in 2011 but will occur in 2016 instead; (ii) It was believed that passing standard 7 was a requirement to attend secondary school which is untrue. (iii) Wrong Time Frame: Mismatch of PAD Results framework (2005-2011) with PDO indicators (2006-2011) for Intermediate Outcome Indicators which did not match the project implementation period (2008-2013). (iv) Questionable Source of Statistical Data: The baseline data did not match official data published in the Annual Budget Speech. The source of the data in the PAD can no longer be identified. 3.4 PDO Revision: Given the inconsistencies above, the Government and the Bank revised PDO targets. Based on the revised PDO, it was noted that, overall, Intermediate Outcome Indicators show significant progress. (i) The overall share of secondary teachers qualified to teach at the secondary level increased from 59 percent in 2006 to 79 percent in 2012, surpassing the target value of 65 percent. 39 (ii) In science, the share of secondary teachers qualified to teach mathematics, biology, chemistry and physics has increased by about 20 percentage points each between 2006 and 2012, though the numbers still fall slightly below the target of 65 percent. The 65 percent benchmark has been almost reached in chemistry (where currently 62 percent of teachers are qualified), though the gap is larger in mathematics (where still only 47 percent of teachers are qualified, which is still a significant improvement compared to 28 percent in 2006). (iii) The student class ratio improved between 2006 and 2011 (from 83 to 73), and is more favorable than the target value of 92. Hence, while the PAD assumed student-class ratios would stabilize during the period of project implementation, they have in fact improved. Textbook/student ratios in Form 4 is (1/1) in Mathematics, English and Physics in 2013. Development Indicators Revised Base Original End Revised Objective Value 2007 of Project Achievements Objective at Project Completion 2013 Improve Completion Absolute # of pupils Examinees: Examinees” Examinees” of Lower secondary disaggregated by gender 7.477 24,455 14,268 education with (i) taking & (ii) passing Passes: 6,804 Passers: Passers: 12,984 successful the CSEE examination Div III: 1,421 22,254 Div III 2,711 performance among and (iii) passing with Div Div III 4,647 students III or better Intermediate outcomes Expand School Pupils/Classroom ratio 83 92 73 infrastructure to keep pace with perspective enrolment growth Provide in-service % of teachers successfully 0 Target: 90 ? English language completing (and passing training for primary the examination) in the in- school teachers service language courses. Provide in-service % of teachers successfully 0 Target: 90 ? math and science completing (and passing training for teachers the examination) in the in teaching these service math and science subjects courses Increase the % of secondary teachers (i) 59% Increase these (i) 79% availability of qualified to teach at the (ii) 28% ratios to at (ii) 48% teachers qualified to secondary level (diploma (iii) 38% least 65%) (iii) 58% teach the subjects they or higher; % of secondary (iv) 43% (iv) 63% are teaching teachers of (ii) Math, (iii) (v) 39% (v) 59% Biology; (iv) Chemistry; and (v) Physics, qualified to teach those subjects at secondary level, or who have successfully completed in service upgrading in this course Make text books in Pupil/text book ratio - 1/1 1/1 core subjects available to all secondary students 40 3.5 Consequently, it was concluded that: (i) the PAD PDO targets were unrealistic and unattainable; and (ii) none of the parameters in the PAD’s projection model is clearly linked or directly associated to the implementation of the Project. The full outcome of component one will not be seen until the students entering form one in 2014 take the CSEE in 2018. Similarly, it is anticipated that the full impact of inputs related to quality will not be achieved until 2020. IV. OUTPUTS AND RESULTS ACHIEVED (OUTCOMES): (a) Overview: (b) Status of Implementation by component: Component 1: Increasing Enrollment through Provision Of Infrastructure: 4.3 By November 2013, construction of 19 new schools and rehabilitation of 6 schools was complete. Furniture and laboratory equipment were procured, delivered, and installed. The overall output under this component is 342 classrooms with student capacity of 13,680, 75 science laboratories, 28 computer labs, 27 libraries and 52 staff houses. 4.4 Availability of adequate financial resources affected the implementation of activities under this component. As a result, the scope of civil works had to be reduced. All student hostels were removed. Teacher housing was reduced from 10 houses per school to 3. 4.6 Field visits revealed that the sustainability of the school facilities and their potential impact could be undermined by the following factors: (i) Absence of fences in all the newly constructed facilities exposing the school to vandalism and livestock grazing. (ii) Water shortages negatively impact staff and student attendance due to unsanitary conditions. (iii) The proper utilization of science laboratories, computer labs and libraries is undermined by the lack of lab technicians, ICT specialists and librarians. As a result, new resources remain unused. Component 2: Improve Quality Of Education Through Teacher Training, Curriculum Reform And Provision Of Learning Materials SUB-COMPONENT 2A: Teacher Training: The Department of Teacher Education was responsible for implementing activities under this sub-component. These are: 4.7 In-service Upgrading for Secondary Science and Mathematics: The purpose was to use in-service training to upgrade the skills of at least 500 non–degree science and mathematics teachers, currently teaching maths and science in Form 1-2. The course material was developed in a 5 day workshop in 2009. This was followed by the training of trainers in February 2010. 4.8 The first cohort of training for 350 teachers was conducted in 2010. The second cohort of training for 150 teachers was conducted in September 2013. Upon completion of the training, assessment in the form of written exams and course work was conducted, with mixed results: While some changes in teaching and use of resources were observed, the written exam results were discouraging. The average of each test group was 55-65%. A follow-up of a small number of trainers and teachers, revealed that: (i) while training materials were in English, the course was mainly 41 conducted in Kiswahili due to low English language proficiency among trainers and trainees; (ii) the targeted trainees were non-qualified teachers many of whom had never taught science and math. Consequently, it was concluded that: (a) the duration of the course was too short to upgrade the capacity of these teachers; (b) there is need for sustained long term effort to address the English language deficiency of both teachers and students in order to improve quality of education. 4.11 In-service upgrading for primary science and mathematics teachers: To prepare for the imminent changes to the curriculum in upper primary schools, an in- service upgrading course was designed for approximately 1,039 math and science teachers in primary school in Standards 4 and higher (along with subject specialists and head teachers). 4.12 By July 2013, a training of trainers was planned, designed, and conducted for 40 trainers. Teacher training began in February 2013. By July 1,466 science teachers had participated in 18 six-hour days of in-service training; 573 math teachers had participated in 10 six-hour days of training. 4.13 In January 2013, the initial estimate of 1,039 recipients was revised to 950 to more conservatively reflect the number of qualifying teachers. However, the PMT found additional teachers that would benefit from the program such as those teaching math or science instead of the subject they are trained in, resulting in the number of participants to match the original number. 4.14 According to the PAD, teachers were to participate in 40 days of content training. Given the constrained timeline, in January 2013 it was proposed that in- service training time could be replaced by peer learning and in-school discussion to be facilitated by qualified supervisors. A curriculum and activities have been developed and 100 supervisors were selected and trained to facilitate peer learning sessions. However, no peer learning sessions have been conducted due to lack of funds for transportation of facilitators and teachers to cluster sites. 4.17 In-service upgrading for primary teachers of English and communication: This sub-component aimed at training approximately 2,000 Primary English teachers in English and Communication, since in 2015 the new syllabus will take effect. In standard 5-6, 5 subjects (math, science, geography, ICT and English will be taught in English. The target group was teachers of math, science, English and humanities. 4.18 By November 2013, 2000 upper primary teachers had received upgrading training in using English and Communication. 4.19 Trained teachers have been deployed ad-hoc and correctly deploying them is a matter that has to be undertaken seriously by the Ministry. This requires collaboration between all stakeholders including the TTC, and follow-up has to be strengthened. 4.20 While the English proficiency of approximately 40% of the target teachers is considered good, the level of proficiency of the majority is very low. In light of this and considering the arguments presented in the previous section, achieving the desired outcome of this activity is unlikely 4.21 Capacity building for Head teachers, Inspectors & Subject Advisors: […] 4.22 A training manual was developed in collaboration with experienced staff from the inspectorate and representatives from head teachers and the Zanzibar higher education institutions. 500 copies of the manual were printed. Training of Trainers (TOT) was conducted in Unguja and Pemba, attended by 18 school inspectors from 42 Unguja and 14 in Pemba. 442 head teachers, of which 84 were from private schools, 27 subject advisors and 9 TC coordinators were also trained on leadership to supervise and support teachers. 4.23 Though follow-up was not part of the original plan in PAD, follow-up for 288 schools public and private schools was planned, of which 81% were conducted. 4.24 While the schools visited were responsive to the introduction of computers many were not responding to other the training deliverables. The following challenges were observed: inability to design a vision and mission for their schools; lack of emphasis on pedagogical leadership, prioritizing child-centered methodologies; and using proper teaching aids. 4.25 The provision of two buses for Pemba and Unguja and office equipment (computers, photocopiers, stationery) to the department was crucial to the training and the follow-up. This allows the department to reach, inspect and follow up with more schools more frequently, which benefits the learning child and the community at large. 4.26 Revision of Pre-Service Primary Teacher (certificate) curriculum: This activity aimed to revise the curriculum to ensure compatibility with the provisions of Education Policy of 2006. Implementation was scheduled for 2010/2011but was delayed due to recruitment issues. By July 2012 the new curriculum was complete and is currently being used. 4.27 Interim upgrading of pre-service secondary teachers (diploma) curriculum in science and mathematics: The sub-component was supposed to finance the development of an addition to the secondary teacher education diploma and to prepare teachers of science and mathematics to teach these subjects up to form 4. 4.28 While the activity outlined in 2.A.2 provides some training to current secondary math and science teachers, it did not address teachers entering the system. To address this gap, similar training was incorporated into the pre-service training and provided to all teachers attending SUZA. 175 secondary schools teachers received interim upgrading training 4.29 Management of Entrants to teacher training: To better understand the characteristics, motivation, incentive receptiveness, capability, and needs of teacher training entrants, the sub-component financed a study of the management of teacher training. 4.36 The study made numerous recommendations. Key issues that require urgent attention include: (i) provision of competent science teachers; (ii) provision of well-trained pre-primary teachers; (iii) addressing the shortage of teachers in Pemba; (iii) regular monitoring and evaluation of the interventions introduced in the system in order to ensure that the targeted impacts are achieved to ensure sustainability. 4.37 SUB- COMPONENT 2B: Provision of Instructional Materials in Key Subjects for All Secondary Schools. 4.38 All text books, laboratory equipment and consumables were procured and distributed to all schools, TTCs, TCs. By November 2013, a ratio of one new book 43 per student in each subject based on the previous year’s attendance data had been attained. 4.39 Library equipment apparatus and consumables were procured and distributed to TCs and secondary schools in Unguja and Pemba. This will enable teachers to teach science subjects efficiently and provide an opportunity to students in conducting practical exercises in relevant subjects. 4.40 The new text books and other learning material are expected to improve of teaching-learning process in schools. 4.41 SUB-COMPONENT 2C: Revision of Primary Curriculum and Provision of Appropriate Learning Material. 4.42 The shortening of the primary curriculum from seven years of schooling to six years has been under consideration by the Government of Zanzibar since 2006. To ensure the provision of quality primary education, the Zanzibar Institute of Education prepared a revised primary curriculum in 2010. The process of facing out the old primary curriculum was initiated in 2010-2011 academic year. 4.44 Provision was made under the project to procure books and distribute to each primary school in a ratio of one textbook per pupil in the upper primary (standards 5- 6). It should be noted, however, that: (i) the target pupils (currently in standard 2) will enter Standard 5 in the year 2014; one year after the project completion; (ii) Since the old primary education syllabus is being gradually phased out, it is imperative that the production of all new textbooks should be in conformity with the revised primary education curriculum; (iii) Primary 1 to 4 curriculums has no instructional package (teachers’ guide). This will deprive teachers of material essential for effective teaching in the learning process; (iv) Enhanced teacher ownership of the revised curriculum provision of textbooks and teacher guides is essential. 4.45 Considering the projected enrolment in 2011, it was estimated that 1,243,448 textbooks and 30,400 accompanying teachers’ guides would be required. The new curriculum for Primary Education was prepared and officially used since 2010 in all public schools. This is including the subject syllabi. The curriculum orientation training to teachers, head teachers and inspections was conducted as follows:- List of curriculum implementers involved in curriculum orientation training SN. TARTET GROUP NO. 1. Std 1 teachers 895 2. Std 2 teachers 900 3. Std 3 teachers 780 4. Std 4 teachers 450 4. Primary school inspectors 28 5. Primary school head – teachers 240 Total 3,293 4.46 SUB-COMPONENT 2D: Support for science education for girls. For many years girls’ performance in science and mathematics was poor compared to boys’. To 44 bridge the gap, a Science Camp program was initiated, which aimed to improve girls’ performance in science and mathematics at ordinary secondary and advance level. 4.47 Based on a collaborative planning process, the Science Camp program was designed to benefit 100 girls annually, by changing their attitude toward science. The camp was held only once in 2010. 450 students from 199 secondary schools participated. Most participants were from Form 1 with some from Forms 2 and 3. Science camp modules were printed. These modules will be used to train students through school-based science camps. 4.50 The evaluation report showed that organized camps produced meaningful results for participants. About 88 % of the campers passed their Form 2 national examinations and were selected to join Form 3 in different secondary schools. Further analysis indicated that campers’ performance in science subjects was slightly above average in their respective schools. However, it was not clear whether their performance is attributable to participation in the camp. 4.53 SUB COMPONENT 2E: Provision of life skills, including HIV/AIDS gender and environmental education. The aim of this sub-component was strengthening education for life skills in secondary schools and teachers colleges by 2013. The old life skills manual was reviewed, revised, updated and 6000 copies were printed and distributed to all secondary schools. Training on how to teach life skills through integration in core subjects was conducted. About 500 Form I teachers from Unguja and Pemba have been trained since 2009. In 2010 and 2011 600 form 2 teachers and 500 Form 3 teachers were trained. 4.54 During 2012/13 life skills peer education training was conducted for 450 student leaders in Unguja and Pemba (50 per each TC). The training strengthened peer education programs in school health clubs. Life skills training was also provided to 100 health club teachers to support school health clubs and develop their delivery of extracurricular instructions. V. PROJECT DESIGN AND READINESS FOR IMPLEMENTATION: 5.1 Thanks to previous experience in civil works projects, the Ministry already had a core team of engineers and procurement specialists. These teams, whose members were later seconded to the PMT, participated in the preparation and appraisal of the project. As a result, the appraisal team ensured that the project is commensurate with the Governments’ capacity to implement it by putting in place the necessary implementation arrangements. This included proper procurement and financial management as well as adequate supervision, monitoring and evaluation of project activities. 5.2 At appraisal stage World Bank architect insisted on using cost estimations from any recent construction. This was found to be unrealistic, given that at least two years had elapsed since the design, costing and tendering of such structures. At the tendering stage, therefore, it was noted that cost of civil work contracts had increased by 100% due to delays in the design of the facilities and increased cost of construction material and other necessary inputs. As a result, the PMT had to reduce the scope of the civil works by cutting all hostels in district rural schools and reducing staff housing from 10 to 2 in every school. The cost estimate of each school was, therefore, reduced from the initial estimate TZS7-10 billion to TZS3-4 billion per school. 45 5.3 Procurement Arrangements: By November 2013, procurement of all goods and services was completed. Procurement under the project was managed by the Procurement Management Unit of the Ministry. Since the beginning of the project, the following has been achieved: (i) in the procurement of goods, 43 contracts amounting to US$ 10.4 million have been procured; (ii) in the procurement of consultant services, 23 contracts with a total of US$ 2.7 million have been procured; (iii) in the procurement of works, 17 contracts for construction of Secondary Schools with a total of USD 31.8 million have been procured. […] The overall assessment with regards to procurement is Satisfactory. 5.6 Disbursement Arrangements: The table below presents the disbursement of the credit by year and category of expenditure. In the absence of a project preparation facility (PPF) all project preparatory activities including the establishment of the Project Management Unit had to be initiated or carried out soon after the credit became effective in January 2008. These activities included preparation of term of references, recruitment of consultants, preparation of bidding documents, tendering of bids, etc. While most of the procurement and civil works staff were in place, the financial management staff were recruited in mid-2008. As a result there were no meaningful disbursement during 2008-9. As shown in the table less than 0.5% or $156,000 of the credit was disbursed in the first two years of project implementation. 5.8 In general, the disbursement of funds by the Bank and PMT to contractors and consultants was found to be highly satisfactory. Initially the PMT noted that contractors were accumulating payment requests. Upon recognition of this anomaly, the PMT advised them to submit requests for payment on monthly basis. A number of contracts interviewed by the consultant expressed their full satisfaction on the timely reimbursement of funds which they found was more prompt compared to other donor funded projects. Disbursement by Category of Expenditure and Year Category of Expenditure Consultancy Operating Workshops Year Civil Works Goods Services Cost &Training Total % 21,467 2008 8,350 9,242 - 8,091 47,149 0.001 2009 - 5,122 39,951 11,753 51,536 108,362 0.003 2010 1,720,543 1,853,859 402,328 184,168 489,447 4,650,345 0.11 2011 11,899,853 728,745 545,437 130,716 398,626 13,703,376 0.33 2012 9,066,927 2,489,584 703,181 243,553 395,982 12,899,226 0.31 2013 5,380,152 3,855,890 1,073,154 100,588 328,809 10,738,594 0.26 Total 28,067,474 8,941,550 2,773,293 692,244 1,672,491 42,147,053 1.00 % 67 21 7 1 4 100 VI. IMPLEMENTATION 6.1 Bank Borrower performance: The Bank undertook bi-annual field mission with adequate skills mix. The mission always prepared a detailed aide memoire in close collaboration of the PMT and Senior Ministry officials. The Aide Memoires provided detailed assessment of progress to date and issues that would require immediate remedial measures. It specifically included a section with agreed 46 arrangement and completion schedule. This arrangement made it easier for the PMT to obtain Government approval and support on any pending issue. 6.2 The presence of the Task Team Leader (TTL) in Dar also made it easier for PMT to interact with and visit the Bank office in Dar in order to solve any immediate bottlenecks. As a result the performance of the Bank and the Government was generally found highly satisfactory. 6.3 This is attributed to: (a) government commitment to grant the education sector priority in the allocation of both development and recurrent budget (20%); (b) Senior Ministry Officials giving the project high visibility by organizing official opening ceremonies spearhead by the President of Tanzania or Zanzibar; (ii) dedication on the part of the project management team (PMT) to enhance implementation while ensuring efficiency and cost effectiveness; (iii) frequency of Bank supervision missions which allowed the possibility of addressing implementation bottle necks on time; and (iv) demonstrated responsiveness and flexibility by the Bank in addressing issues that emerged during implementation; etc. 6.4 Adherence to Schedule: The project was supposed to be implemented in 65 months from the date of Credit effectiveness. Based on the credit disbursement data in chapter V of the report less than 0.005% of the credit was disbursed in the first two years of project implementation. In the absence of Project Preparation Advance (PPA), the implementation of project activities had to be initiated from scratch. It appears that the appraisal team did not fully appreciate the challenging task of establishing an effective institutional arrangement or PMT to plan, manage and implement project activities as proposed in the PAD. 6.5 Consultants and Contractor Performance: The project recruited 5 firms and 6 individual consultants to design/supervise civil works and prepare various studies respectively. In addition it contracted 6 civil works and building contractors to contract schools. At completion, with the exception of few hitches which were immediately tackled by the Bank and the Government, overall, the performance of most contractors and consultants were found generally satisfactory. 6.6 Environment Safeguards: […] 6.7 To ensure adherence to environmental guidelines, Bank mission included an environmentalist and the PMT regularly monitored construction sites. As a result there were no serious adverse environmental impacts that can be associated to the implementation of the project. This includes involuntary resettlement of people. VII. CONCLUSIONS AND RECOMMENDATIONS 7.1 The Bank intervention in the education sector in Zanzibar was highly relevant and timely. It came at a time when the Government of Zanzibar introduced major reforms in the Education Sector. In spite of project design flaws and by the sheer commitment of the Government, close collaboration and team work among the PMT and Bank project management team, all project activities were completed on schedule. 7.2 Though it is too early to assess outcomes/impact since most of the project activities were just completed, the project is expected to increase access, considering the significant increase of facilities “hardware” which was the main focus of the project (70% of credit). With respect to improving quality “Software”, the project will also achieve its target outputs. It is however, doubtful if the limited resources (30% of 47 the credit) allocated to this component will achieve the results proposed in the PDO. This is primarily due to: (a) lack of appreciation at appraisal of the magnitude of the tasks ahead; (b) use of unreliable data which resulted overestimation of expected outcomes in the short run. 7.3 To enhance outcome in the medium term and impact in the long-term, the Government included in the Zanzibar Education Development Plan (ZEDP) Action Plan 2012/13 – 2015/16 measures that are aimed to address the issues and shortcoming highlighted in this report. These measures include: (i) Budget Allocation: Government plans to sustain the political commitment and priority granted to the education sector in the allocation of annual budget. In the past 5 years approximately 5% of GDP and 20% of annual recurrent budget were allocated to the Education Sector. (ii) Institutionalization of Activities: Project activities were built and integrated into the Ministry’s core work plan and program and effectively decentralized to respective departments. This implies that, though the project implementation is completed, each department will continue to carry on implementing these activities; (iii) Education Sector Donor Support Group: ZEDP Action Plan 2012/13 – 2015/16 identifies key strategies to address both quality and access issues. Development partners such as the African Development Bank (AfDB), Global Partnership for Education (GPE), Swedish International Development Agency (SIDA), UNESCO, UNICEF, World Bank, Government of Oman, etc. have made commitment to support the implementation of the Plan; 7.4 Finally, the following lesson could be drawn from the implementation of this project: (a) Adherence to Project Cycle; there are no short cuts in the design of a development projects. The Project Cycle should be followed methodically to ensure that quality of project design is not compromised; (b) Institutions Culture Matters. In spite of major design weakness such as cost under estimation, inflated PDO targets, absence of Project Preparation Advance (PPA) the project was implemented on schedule and has met all its target deliverables. This could be attributed to some of the following factors: • Importance of institutional memory: Following credit approval the Ministry was restructured and senior management staff were rotated. In order to sustain institutional memory and secure smooth implementation, the MoEVT opted to maintain the Project Coordinator and rest of PMT as appraised; • Delegation of authority, close collaboration, team work which are imbedded in the MoEVT institutional culture appear to have played a significant role in the satisfactory implementation of the project; • Building synergy: It appears that in spite of the high turnover of Bank project management staff particularly the Task Team Leader and the Procurement Specialist, their close working relationship with the PMT created a conducive environment for the implementation of project activities. 48 MAP 49