Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) Angola Learning for All Project (P122700) AFRICA | Angola | Education Global Practice | IBRD/IDA | Specific Investment Loan | FY 2014 | Seq No: 8 | ARCHIVED on 30-Jun-2017 | ISR27175 | Implementing Agencies: Ministry of Education, Government of Angola Key Dates Key Project Dates Bank Approval Date:26-Sep-2013 Effectiveness Date:16-Jun-2014 Planned Mid Term Review Date:28-Mar-2016 Actual Mid-Term Review Date:28-Mar-2016 Original Closing Date:30-Apr-2018 Revised Closing Date:28-Feb-2020 Project Development Objectives Project Development Objective (from Project Appraisal Document) The proposed Project Development Objectives are to improve teachers’ skills and knowledge as well as school management in project designated areas; and to develop a system for systematic student assessment. Has the Project Development Objective been changed since Board Approval of the Project Objective? No PHRPDODEL Components Name Component 1: Improving Teachers’ Skills and Knowledge and School Management in P:(Cost $51551820.00 M) Component 2: Establishing a System for Student Assessment:(Cost $7890000.00 M) Component 3. Project Management:(Cost $6558180.00 M) Contingencies Overall Ratings Name Previous Rating Current Rating Progress towards achievement of PDO  Moderately Unsatisfactory  Moderately Satisfactory Overall Implementation Progress (IP)  Moderately Unsatisfactory  Moderately Satisfactory Overall Risk Rating  Substantial  Substantial 6/30/2017 Page 1 of 13 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) Implementation Status and Key Decisions The Angola Learning for All Project became effective on June 16, 2014. During the first phase of implementation, the project faced a number of challenges and significant delays in implementation. After the Mid-Term Review (MTR) of the project in May 2016, a decision was made to restructure the project to enable the Ministry to successfully achieve the project development objectives. This restructuring included (i) an extension of 22 months; (ii) a revised implementation schedule; (iii) changes to component costs due to exchange rate differences and budget constraints in Angola; and (iv) a revised results framework to account for realities faced on the ground. This restructuring was approved in late January 2017. The Ministry of Education has been able to overcome initial bottlenecks and the Project is now finally on track to achieve its development objectives. Several recommendations from the MTR have been incorporated into project implementation: the Project Steering Committee comprised of Ministry leaders meets regularly to guide implementation; a Deputy Project Manager was recruited and oversees day-to-day operations; the Financial Manager has successfully trained accountants in all 18 Provinces and bank accounts have been opened in each Province to facilitate the flow of funds and support activities at the local level; and more than 250 local staff at the provincial level, municipal level, and teacher training institutions have been trained in how to manage the funds, conduct procurement activities, and implement activities related to the teacher training and the school grants programs. Some highlights of achievements under each component are described below. Component 1: Improving Teachers’ Skills/Knowledge and School Management • All 167 Zones of Pedagogical Influence (ZIPs) supported by the Project have been established in each municipality of the country. A manual to guide the ZIP structure was produced and training delivered. • 19 Facilitators have been successfully recruited and trained and are actively working in all 18 Provinces to establish and activate School Councils. To date, they have established School Councils in 70 of the total 167 ZIPs. • A manual and template to create school improvement plans and guide the school grants program has been drafted. The final version is expected in July 2017 and training sessions for School Councils, ZIP Coordinators, and School Directors are expected to begin in each Province no later than August 2017. • The institution to develop the in-service teacher training program was successfully recruited and the training program is underway. The two modules on Portuguese and Mathematics were finalized and delivered during several training sessions – a training of trainers for 90 teacher trainers was conducted in November 2016; a training for 728 teachers/ZIP Coordinators from each municipality conducted in January 2017; and then the training in Portuguese for all 15,000 teachers from all 167 ZIPs supported by the project was delivered in May 2017. (The module on Mathematics is expected to be delivered in August 2017.) • Equipment and materials to support the teacher training activities and provide additional resources at the ZIP level have been procured and are expected to be delivered no later than October 2017. Small libraries have already been disseminated and are being used at schools to support early literacy skills. Component 2: Establishing a System for Student Assessment • The Project includes an impact evaluation to determine the degree to which the interventions on teacher training and school management are able to improve education quality and learning outcomes. The design of the IE was finalized in November 2015. • Two survey instruments – EGRA and SDI – were successfully implemented in 2016 and baseline data collected for the Project. In early 2016, the testing instruments were adapted and teams of enumerators trained to administer the surveys. The surveys were administered throughout Angola in a sample of around 300 schools. Data was cleaned and analyzed the last few months of 2016 and a draft of the final report prepared in March 2017. • Initial discussions on the EGRA and SDI results have taken place within the Ministry and a dissemination plan to share more broadly is under development. • Terms of Reference have been drafted and an international expert has been identified to develop a 5-year assessment strategy. • The recruitment processes to hire two institutions – one to conduct a viability study regarding national examinations and another to develop a national learning assessment program in grades 4 and 6 – are underway. These activities have been delayed but are getting back on track. Component 3: Project Management • All key staff positions have now been filled including a Deputy Project Manager, a Financial Manager, 2 Accountants, a Procurement Specialist, 2 Procurement Assistants, and a Senior Monitoring and Evaluation Specialist. • Training for nearly 250 staff from Provincial Education Departments and Teacher Training Institutions has taken place and Bank accounts opened in all 18 Provinces. 6/30/2017 Page 2 of 13 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) • A firm has been recruited to further develop the Education Management Information System (SIGE) and include a specific module to organize data collected on various indicators tracked by the Project. • Independent Verification Agents have been recruited to monitor activities in the Provinces and audit the use of funds at the local level. 6/30/2017 Page 3 of 13 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) Risks Systematic Operations Risk-rating Tool Risk Category Rating at Approval Previous Rating Current Rating Political and Governance  --  Substantial  Substantial Macroeconomic  --  High  High Sector Strategies and Policies  --  Low  Low Technical Design of Project or Program  --  Substantial  Substantial Institutional Capacity for Implementation and Sustainability  --  Substantial  Substantial Fiduciary  --  Moderate  Moderate Environment and Social  --  Low  Low 6/30/2017 Page 4 of 13 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) Stakeholders  --  Moderate  Moderate Other  --  --  -- Overall  --  Substantial  Substantial Results Project Development Objective Indicators PHINDPDOTBL  Demonstrated improvement in trained teachers’ skills and knowledge as a result of the training (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 80.00 Date 01-Nov-2012 01-Dec-2016 20-Mar-2017 28-Feb-2020 PHINDPDOTBL  Percentage of active school councils (schools targeted by the project only) (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 41.90 100.00 Date 01-Nov-2012 01-Dec-2016 20-Mar-2017 28-Feb-2020 PHINDPDOTBL  Number of sample-based assessment exercises carried out (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 1.00 2.00 2.00 3.00 Date 01-Nov-2012 01-Dec-2016 20-Mar-2017 28-Feb-2020 Overall Comments Significant progress has been achieved on the PDO indicators. While the results for the teacher training still show as zero percent, this is because there haven't been any post-tests of the training yet to show progress. However, the training program is underway. The program is using a cascade approach - the institution hired to develop the training program conducted the first training of trainers (90) in November 2016. Then in January 2017, these trainers trained 728 teachers/ZIP Coordinators from each Municipality. Then in May 2017, these 728 trainers trained 15,000 teachers from the ZIPs supported by the Project. During these trainings, a pre-test was administered to the teachers. When the post-test is administered in August then we will have some results to determine if the training is having a positive impact on improving teachers' knowledge of Portuguese and Mathematics. 6/30/2017 Page 5 of 13 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) Following the Project restructuring, the Ministry worked closely with all of the 18 Facilitators in each Province to collect data and determine an accurate result for the number of active School Councils. Based on the work of the Facilitators thus far, and according to meeting minutes gathered, the total number so far is 70. Many of these represent the ZIPs closest to the capital and many of which already had schools with some form of community group established (parents' association, school assembly, etc.) in place. The greater challenge now will be reaching those furthest from the capital and those without as much community involvement already in place. As part of the Impact Evaluation, the Ministry implemented both EGRA and SDI to collect the baseline data for the Project. A draft report with the results of these two exercises has been prepared and was discussed during a workshop held on March 30, 2017. The Project is now revising the report and preparing a communications plan to disseminate the results. Intermediate Results Indicators PHINDIRITBL  Direct project beneficiaries (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 2,230.00 3,259.00 19,576.00 Date 01-Nov-2012 01-Dec-2016 20-Mar-2017 28-Feb-2020 Comments These beneficiaries include ZIP supported schools, teacher training institutions, Ministry of Education staff - central, Provincial Education Departments (including accountants, ZIP Provincial Coordinators, etc.), and Municipal Education Departments (ZIP Coordinators). Once the report is available from the teacher training that took place in May, this figure will be updated with the number of teachers participating. PHINDIRITBL  Female beneficiaries (Percentage, Custom Supplement) Baseline Actual (Previous) Actual (Current) End Target Value 50.00 30.00 46.00 50.00 6/30/2017 Page 6 of 13 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) PHINDIRITBL  ZIPs set up and active (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 167.00 167.00 167.00 Date 01-Nov-2012 01-Dec-2016 20-Mar-2017 28-Feb-2020 PHINDIRITBL  ZIP facilitators recruited and trained (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 30.00 30.00 30.00 Date 01-Nov-2012 01-Dec-2016 20-Mar-2017 28-Feb-2020 Comments There are 19 ZIP Facilitators hired directly by the Project. For those Provinces with a higher number of Municipalities, the Ministry identified 11 School Inspectors to serve as additional ZIP Facilitators. The first 19 Facilitators were trained in late 2015. In November 2016, a second round of training took place in Luanda which included all 30 ZIP Facilitators. PHINDIRITBL  Set of school report cards produced and disseminated (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 842.00 Date 01-Nov-2012 01-Dec-2016 20-Mar-2017 28-Feb-2020 Comments There are already two report cards produced by the Ministry. The Project is supporting the creation of a new one that will focus more on education quality. The Project has been developing a template to be used. There will be training in August with all school directors from the 842 schools supported by the Project. This training will include information on the new school report card to be developed. These new report cards together with the other two are then expected to be produced in the 2017/2018. 6/30/2017 Page 7 of 13 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) PHINDIRITBL  In-service teacher training modules aligned with learning goals developed (six modules) (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target Value No No No Yes Date 01-Nov-2012 01-Dec-2016 20-Mar-2017 28-Feb-2020 Comments The in-service teacher training program is being developed in 3 phases. The first two modules for Portuguese and Mathematics have been finalized. They were piloted during the trainings that took place in November 2016 and January 2017. Revisions were incorporated and the modules are now being produced to be ready for the training program with 15,000 teachers in May and August 2017. PHINDIRITBL  Teachers recruited or trained (Number, Corporate) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 15,000.00 Date 01-Nov-2012 01-Dec-2016 20-Mar-2017 28-Feb-2020 Comments The first round of training at the school level took place in May 2017. This was expected to include all 15,000 teachers. These same teachers will participate in the training program over the next 3 years. The next progress report is due July 1 during which figures should be provided on the total number in attendance. PHINDIRITBL  Teacher training schools equipped (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 18.00 Date 01-Nov-2012 01-Dec-2016 20-Mar-2017 28-Feb-2020 Comments The 18 teacher training schools to be supported by the Project have been identified. The list of equipment to be purchased has also been agreed upon. Procurement of this equipment is now underway. The equipment is expected in late 2017/early 2018. 6/30/2017 Page 8 of 13 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) PHINDIRITBL  ZIP center schools equipped (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 167.00 Date 01-Nov-2012 01-Dec-2016 20-Mar-2017 28-Feb-2020 Comments The 167 ZIPs to be supported by the Project have been identified. The list of equipment to be purchased has also been agreed upon. Procurement of this equipment is nearly finalized. The equipment is expected to be delivered in mid-July 2017. However, there are a few security concerns in some ZIPs that have not yet established a secure space where this equipment will be stored. These need to be resolved before the equipment can be delivered. PHINDIRITBL  Number of school kits produced and distributed (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 842.00 Date 01-Nov-2012 01-Dec-2016 20-Mar-2017 28-Feb-2020 PHINDIRITBL  Number of education projects approved (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 520.00 Date 01-Nov-2012 01-Dec-2016 20-Mar-2017 28-Feb-2020 Comments The "Projecto Educativo da Escola" (PEE) manual has been developed and is now under review. It is expected to be finalized in July. The training for School Directors on the guidelines will take place July 2017. 6/30/2017 Page 9 of 13 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) PHINDIRITBL  Number of grants disbursed (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 520.00 Date 01-Nov-2012 01-Dec-2016 20-Mar-2017 28-Feb-2020 Comments The school grants will only happen after schools have developed their "Projectos Educativos". The first grants aren't expected until late 2017 or early 2018. PHINDIRITBL  Assessment framework reviewed (Yes/No, Custom) Baseline Actual (Previous) Actual (Current) End Target Value N N N Y Date 01-Nov-2012 01-Dec-2016 20-Mar-2017 28-Feb-2020 PHINDIRITBL  Minimum standards defined as well as bank of items and standardized tests constructed (Yes/No, Custom) Baseline Actual (Previous) Actual (Current) End Target Value N N N Y Date 01-Nov-2012 01-Dec-2016 01-Dec-2016 28-Feb-2020 PHINDIRITBL  Impact evaluation phases completed (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target Follow-Up Data Value Design Data Collected Baseline Report Drafted Analysis, and IE report Date 01-Nov-2013 01-Dec-2016 20-Mar-2017 28-Feb-2020 Comments 6/30/2017 Page 10 of 13 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) The report with the baseline data for the IE has been drafted. It is undergoing a second round of revisions. The IE Specialist is now conducting field visits and monitoring implementation in the treatment and control groups. PHINDIRITBL  System for learning assessment at the primary level (Yes/No, Custom) Baseline Actual (Previous) Actual (Current) End Target Value Y Y Y Y Date 01-Nov-2012 01-Dec-2016 20-Mar-2017 28-Feb-2020 Comments The Ministry is developing a dissemination plan to share the outcomes of the EGRA and SDI surveys. This is to ensure the utility of the learning assessment system and use of data to improve education quality. PHINDIRITBL  Utility of the learning assessment system (Number, Custom Supplement) Baseline Actual (Previous) Actual (Current) End Target Value 1.00 1.00 1.00 3.00 PHINDIRITBL  Number of provincial, municipal and teacher training schools officers trained in management around the project (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 266.00 254.00 Date 01-Nov-2012 01-Dec-2016 20-Mar-2017 28-Feb-2020 Comments A number of trainings took place in November 2016 to launch activities are expected to take place at the Provincial and Municipal levels. 6/30/2017 Page 11 of 13 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) These included training for Provincial accountants (approx. two from each Province - 34 ) in FM and Procurement, training for teacher training school Supervisors (30), training in Projectos Educativos and school grants (54), training in Moodle for the trainers implementing the in-service teacher training activities (20), and another 128 staff Provincial and Municipal staff in basic project management. To reinforce capacity, additional rounds of training are envisioned in 2017 and 2018. PHINDIRITBL  Number of teacher kits produced and distributed (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 15,000.00 Date 01-Dec-2016 01-Dec-2016 20-Mar-2017 28-Feb-2020 Overall Comments Since the Project restructuring, the Project has established a strong monitoring and evaluation system to carefully assess project implementation and the status of various indicators. Now that the teacher training program is underway, progress on several indicators is advancing. Data on Financial Performance Disbursements (by loan) Project Loan/Credit/TF Status Currency Original Revised Cancelled Disbursed Undisbursed Disbursed P122700 IDA-52540 Effective XDR 48.80 48.80 0.00 13.01 35.79 27% Key Dates (by loan) Project Loan/Credit/TF Status Approval Date Signing Date Effectiveness Date Orig. Closing Date Rev. Closing Date P122700 IDA-52540 Effective 26-Sep-2013 16-Jan-2014 16-Jun-2014 30-Apr-2018 28-Feb-2020 Cumulative Disbursements 6/30/2017 Page 12 of 13 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) Restructuring History Level 2 Approved on 29-Jan-2017 Related Project(s) There are no related projects. 6/30/2017 Page 13 of 13 Public Disclosure Copy