Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004614 IMPLEMENTATION COMPLETION AND RESULTS REPORT TF019053 ON A SMALL GRANT IN THE AMOUNT OF USD1.7 MILLION TO THE Co-Operative Republic of Guyana FOR THE Guyana Early Childhood Education Project (P129555) March 20, 2019 Education Global Practice Latin America And Caribbean Region Regional Vice President: Axel van Trotsenburg Country Director: Tahseen Sayed Khan Senior Global Practice Director: Jaime Saavedra Chanduvi Deputy Practice Manager: Rita Kullberg Almeida Task Team Leader(s): Hongyu Yang ICR Main Contributor: Nicole Mammoser ABBREVIATIONS AND ACRONYMS CAS Country Assistance Strategy CEN Country Engagement Note CPD Continuous Professional Development ECD Early Childhood Development ECE Early Childhood Education ESP Education Strategic Plan GECEP Guyana Early Childhood Education Project GPE Global Partnership for Education MEIS Ministry of Education Information System MoE Ministry of Education NCERD National Center for Education Resource Development NDA (Ministry of Education’s) Nursery Diagnostic Assessment PDEC (Early Childhood Development) Program Delivery Evaluation Checklist TABLE OF CONTENTS DATA SHEET ....................................................................... ERROR! BOOKMARK NOT DEFINED. I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 4 II. OUTCOME ...................................................................................................................... 8 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 16 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 17 V. LESSONS LEARNED AND RECOMMENDATIONS .............................................................. 19 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 21 ANNEX 2. PROJECT COST BY COMPONENT ........................................................................... 28 ANNEX 3. RECIPIENT, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ...... 29 ANNEX 4. HIGHLIGHTS OF MASTER TRAINING MODULES 1, 2 AND 3 .................................... 34 ANNEX 5. MOE’S EARLY CHILDHOOD DEVELOPMENT PROGRAM DELIVERY EVALUATION CHECKLIST ........................................................................................................................... 36 The World Bank Guyana Early Childhood Education Project (P129555) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P129555 Guyana Early Childhood Education Project Country Financing Instrument Guyana Investment Project Financing Original EA Category Revised EA Category Not Required (C) Organizations Borrower Implementing Agency Co-Operative Republic of Guyana Ministry of Education Project Development Objective (PDO) Original PDO The objective of the Project is to improve emergent literacy and numeracy outcomes for children at the nursery level and primary grade one in Hinterland Regions and Targeted Remote Riverine Areas. FINANCING FINANCE_T BL Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) Donor Financing Total 0 0 0 Total Project Cost 1,700,000 1,698,273 1,698,273 Page 1 of 37 The World Bank Guyana Early Childhood Education Project (P129555) KEY DATES Approval Effectiveness Original Closing Actual Closing 27-May-2015 05-Jun-2015 30-Sep-2018 30-Sep-2018 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Substantial RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 19-Oct-2015 Satisfactory Satisfactory 0.00 02 05-Apr-2016 Satisfactory Satisfactory 0.79 03 06-Oct-2016 Satisfactory Satisfactory 0.92 04 09-Apr-2017 Highly Satisfactory Highly Satisfactory 1.20 05 18-Oct-2017 Highly Satisfactory Highly Satisfactory 1.70 06 29-Mar-2018 Highly Satisfactory Highly Satisfactory 1.70 07 18-Sep-2018 Highly Satisfactory Highly Satisfactory 1.70 ADM STAFF Role At Approval At ICR Regional Vice President: Jorge Familiar Calderon Axel van Trotsenburg Country Director: Sophie Sirtaine Tahseen Sayed Khan Senior Global Practice Director: Claudia Maria Costin Jaime Saavedra Chanduvi Page 2 of 37 The World Bank Guyana Early Childhood Education Project (P129555) Practice Manager: Reema Nayar Rita Kullberg Almeida Task Team Leader(s): Hongyu Yang Hongyu Yang ICR Contributing Author: R. Nicole Mammoser Page 3 of 37 The World Bank Guyana Early Childhood Education Project (P129555) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES Context 1. During the Project preparation period from mid-2014 to mid-2015, Guyana had essentially attained universal primary education, and the Government’s focus had shifted to strengthening the foundation for learning through investments in Early Childhood Education (ECE), which covers the period from the first year of nursery education (at age 3.5 years) to Grade 2 of primary education (at approximately age 8). 2. The education services provided to Guyana’s Hinterland Regions (Regions 1, 7, 8, and 9) and the remote riverine areas within its Coastal Regions (Regions 2, 3, 4, 5, 6 and 10) were below national standards.1 Shortfalls in the provision of education services were attributed to several factors: dispersed settlement patterns of inhabitants, difficult terrain where populations often lived, the high cost of administering projects in the country’s interior, and a lack of local skills and know-how required to support the implementation of development projects. 3. The quality and equity of ECE services were the most critical issues in need of attention in the Hinterland and remote riverine areas. Thirty-one percent of Guyana’s 1,700 nursery teachers remained untrained, 2 with a much higher proportion of untrained teachers (53%) in Hinterland areas. Even among trained teachers, instructional methodologies did not incorporate best practices. Research had shown that young children learn best through play-based, hands-on activities, and this was particularly true for mathematical concepts. Yet Guyanese teachers preferred structured, didactic instructional methods rather than allowing children to engage in independent or small group learning activities that are open-ended, play- based, and hands-on, and therefore more suitable for their intellectual development. 4. The Nursery and Grade 1 classrooms in the Hinterland Regions and remote riverine areas suffered from an inadequate supply of materials for teaching and learning, particularly those that children could use themselves (i.e., blocks, puzzles, and magnetic numbers and letters) for visual expression and development of pre-literacy and numeracy skills. Typically, Nursery and Grade 1 learning aids were created by teachers using locally sourced materials, but these were not aligned with the nursery curriculum. 5. In addition, the untrained teachers who were operating with inadequate teaching materials most often served children from households with very limited learning opportunities (i.e., parents with lower levels of adult literacy and little knowledge of how to support their children’s early learning at home). Unsurprisingly, this translated into lower results in emergent literacy and numeracy skills for Nursery students in the poorer Hinterland Regions. The Ministry of Education’s Nursery Diagnostic Assessment (NDA), 3 administered in 1 Poverty is highest in Guyana’s four Hinterland Regions, which contain approximately 10% of the overall population and are where most Amerindians live. Guyana’s six Coastal Regions include many remote riverine areas; the riverine schools targeted b y the Project were at minimum a 90-minute boat ride from the nearest regional township. Approximately 40% of the population in the Hinterland and riverine areas impacted by the Project lives below the poverty line. 2 A trained teacher has received a teaching certificate from the Cyril Potter College of Education (either through its main campus outside of Georgetown or one of its satellite learning centers), or the University of Guyana’s School of Education. 3 The NDA, which provides data on children’s preparedness for two years of Nursery-level education, was first administered in September of 2014 to a 10% sample of Nursery students. The September 2015 sample was expanded to include 30% of all Nursery Page 4 of 37 The World Bank Guyana Early Childhood Education Project (P129555) September 2015 to 7,100 first- and second-year Nursery students nationwide, showed that while 50.5% of students in the Coastal Regions attained at least an “approaching mastery� level of emerge nt literacy, their peers in the Hinterland scored 39.6%. For emergent numeracy, 53.5% of Nursery students in the Coastal Regions attained at least an “approaching mastery� level, while only 41.9% of students in the Hinterland did so.4 6. Given the context provided in the preceding paragraphs, the relevance of a project that would target improved ECE for Guyana’s most vulnerable children was well recognized. Ample international evidence5 has shown that gaps in knowledge and ability between disadvantaged children and their more advantaged peers open early, tend to persist throughout life and are difficult and costly for countries to close. Improvements to the quality of the early education offered to Nursery and Grade 1 students in the Hinterland and remote riverine pockets in the Coastal Regions would foster positive outcomes, including building valuable skills and boosting the earning potential of these students, and ultimately strengthening Guyana’s workforce and helping to grow its economy. 7. In December 2014, the Global Partnership for Education (GPE)6 approved a US$1.7 million grant for Guyana to improve ECE in Hinterland Regions and in remote riverine communities over a three-year period. The World Bank was designated the coordinating agency for the grant, and the Ministry of Education (MoE) was designated the implementing agency. The Project’s focus on improving the pedagogic skills of ECE teachers in Guyana’s disadvantaged regions and supporting the cognitive development of young children both at school and at home related directly to the GPE’s objectives of improving teacher effectiveness and early grade learning outcomes. 8. The Project also fully aligned with the Government’s 2014-2018 Education Strategy Plan (ESP),7 which prioritized the goals of increasing learning outcomes for all levels of education and decreasing differentials in learning outcomes between students in the Hinterland and Coastal Regions. Further, the Project’s emphasis on improved teacher training as a means to promote the learning achievement of Guyanese children was entirely relevant to the second pillar of the Bank’s Country Assistance Strategy (CAS) for the period FY 2009- 2012, improving education quality. The CAS was discussed by the Executive Directors on May 26, 2009 and remained in effect at the time of Project appraisal.8 students nationwide. This sample was deemed large enough to yield generalizable data for the country. 4 NDA results for Nursery students in the remote riverine areas spread throughout Guyana’s Coastal Regions are included in overall Coastland results. When separated out, these students have historically performed at the same (lower) levels as children in the Hinterland, and therefore were also targeted by the ECE Project. 5 Examples include: James J. Heckman, Early Childhood Education (2016); Isabel V. Sawhill, Jeffrey Tebbs, William T. Sickens, The Effects of Investing in Early Education on Economic Growth (2006); OECD, Investing in High Quality Early Childhood Education and Care (2012). 6 The GPE launched in 2002 to accelerate progress towards the UN Millennium Development Goal of universal primary education by 2015. Guyana previously received a US$32.9 million grant from GPE to implement the Education for All - Fast Track Initiative Project, which supported MoE efforts to attain universal primary school completion for boys and girls and was in effect from September 2004 through December 2012. 7 At Project close, the MoE was in the process of reviewing the 2014-2018 ESP with a variety of stakeholders (including officials from the Ministry of Finance, development partners, and representatives from civil society and the private sector) in order to develop the next iteration of the plan. 8 The CAS for the period 2009-2012 continued to guide the Bank’s assistance to Guyana while it developed its current strategic engagement framework there, laid out in the 2016-2018 Country Engagement Note (CEN). The CEN supports continued engagement in areas that benefit from broad-based support within Guyanese society, including improving the quality of education, enhancing resilience to natural disasters, and supporting private sector development initiatives. Page 5 of 37 The World Bank Guyana Early Childhood Education Project (P129555) Theory of Change (Results Chain) 9. The Project’s Theory of Change as conceived at appraisal is represented in the following results chain: Project Development Objectives (PDOs) 10. The objective of the Project was to improve emergent literacy and numeracy outcomes for children at the Nursery level and Primary Grade 1 in Hinterland Regions and targeted remote riverine areas. 11. The PDO was closely aligned with objectives detailed in the MoE’s Nursery Education Action Plan, which was prepared as a component of the 2014-2018 ESP. The PDO would be achieved by strengthening the capacity and effectiveness of Nursery and Grade 1 teachers, providing additional teaching and learning materials to Nursery and Grade 1 classrooms, and reinforcing primary caregiver engagement in children’s early learning. Page 6 of 37 The World Bank Guyana Early Childhood Education Project (P129555) Key Expected Outcomes and Outcome Indicators 12. Following are the two indicators that assess achievement of the PDO: PDO Indicator 1: Percentage of Nursery students attaining at least “approaching mastery� lev el of emergent literacy, defined as at least 50% correct responses in each of four competency areas: autobiographical awareness, color recognition, letter identification, and alphabet recitation. PDO Indicator 2: Percentage of Nursery students attaining at least “approaching mastery� level of emergent numeracy, defined as at least 50% correct responses in each of four competency areas: quantity differentiation, shape identification, numeral identification, and counting fluency. Both indicators would be measured using the results of the NDA administered to a 30% sample of Nursery students in the Hinterland Regions. Indicator baselines would be based on results from the NDA administered at the beginning of the 2015-2016 academic year, when the first cohort entered Year 1 of Nursery school (Cohort 1). Components Component 1: Capacity Building for Nursery and Year 1 Teachers (US$650,000) 13. This component would strengthen teachers’ content knowledge and pedagogy, particularly with regard to the implementation of new strategies for the development of emergent literacy and numeracy skills, through an in-service teacher training program. The program would include, inter alia, instructional strategies for emerging learners, training in cross-curricular planning and differentiated instruction, and orientation on administering the MoE’s NDA. The expected outcome was an improvement in the quality of instruction and in turn in learning at the Nursery and Grade 1 levels in the Hinterland Regions and riverine areas targeted by the Project. Component 2: Improved Supply of Teaching and Learning Materials (US$400,000) 14. This component would support the procurement and distribution of ECE Resource Kits to all Nursery and Grade 1 classes in the Hinterland Regions and riverine areas targeted by the Project, along with a Teacher’s Manual for effective classroom use of the kits. The expected outcome was that teachers would utilize the Resource Kits to promote more experiential and inquiry-based learning activities for the students, enabling them to learn through play. Component 3: Primary Caregiver Education (US$350,000) 15. This component would support Primary Caregiver Education that would foster and reinforce the development of emergent literacy and numeracy among Nursery and Grade 1 students and their siblings. This would be accomplished via the design and implementation of parent/caregiver training sessions that would be offered locally, as well as the execution of a mass media campaign focused on the important role a parent/caregiver plays in a child’s intellectual development, comprised of radio and television programming, print materials, and a website. The expected outcome was that parents/caregivers would be more proactively engaged in supporting and furthering their children’s learning at home. Page 7 of 37 The World Bank Guyana Early Childhood Education Project (P129555) Component 4: Project Implementation Support, Administration and M&E (US$300,000) 16. This component would support project implementation, administration and supervision, and ensure appropriate monitoring and evaluation of project activities, impacts and fiduciary matters. II. OUTCOME Assessment of Achievement of each Objective/Outcome 17. The objective of improving emergent literacy and numeracy outcomes for children at Nursery levels 1 and 2 was achieved, as evidenced by the results of PDO Indicator 1, Percentage of Nursery students attaining at least an “approaching mastery� level of emergent literacy, and PDO Indicator 2, Percentage of Nursery students attaining at least an “approaching mastery� level of emergent numeracy. 18. The baselines for PDO Indicators 1 and 2 (39.58% for emergent literacy and 41.91% for emergent numeracy) were set based on the September 2015 NDA results for the Hinterland Regions. PDO Indicator targets were set at 45% for both emergent literacy and numeracy. Results from the June 2017 NDA, administered when the first cohort was completing its second year of Nursery education, showed that nearly 76% of students attained at least an “approaching mastery� level of emergent literacy, while nearly 82% of students attained at least an “approaching mastery� level of emergent numeracy, far exceeding the targets of 45%. 19. The Project impacted 231 Nursery schools in the Hinterland Regions and 73 Nursery schools in remote riverine areas spread throughout the Coastal Regions. The following table summarizes NDA results for the 231 Hinterland schools, and for all 326 Nursery schools in the Coastal Regions. Page 8 of 37 The World Bank Guyana Early Childhood Education Project (P129555) Table 1. Percentage of Nursery Students Attaining at least “Approaching Mastery� Level of Emergent Literacy and Emergent Numeracy (Cohort 1) Hinterland Regions Coastal Regions National September 2015 (NDA for Nursery 1 administered to ~7,100 students) Literacy: 39.58% Literacy: 50.51% Literacy: -Students in Cohort 1 begin Nursery 1 year Numeracy: 41.91% Numeracy: 53.54% 48.48% -Teachers have not yet received training under the Project (new (Baselines for PDO Numeracy: curriculum is under development) Indicators 1 and 2) 51.42% -Classrooms have not received ECE Resource Kits -Parent/caregiver education activities have not yet begun -NDA results for Hinterland used to set baselines for PDO Indicators 1 and 2 September 2016: (NDA for Nursery 2 administered to ~8,000 students) -Students in Cohort 1 begin Nursery 2 year Literacy: 68.28% Literacy: 75.01% Literacy: -Teachers have been trained to deliver Module 1 of newly Numeracy: 77.03% Numeracy: 76.71% 73.78% designed curriculum Numeracy: -ECE Resource Kits arrive in classrooms/teachers have been 76.77% trained in their use -Parent/caregiver education activities begin (Parent Circle pilot underway, mass media campaign launched) -NDA results for Hinterland used to measure students’ learning during Nursery 1 June 2017: (NDA for Nursery 2 administered to ~7,700 students) -Students in Cohort 1 complete Nursery 2 year Literacy: 75.83% Literacy: 89.01% Literacy: -Teachers will be trained to deliver Module 2 of newly designed Numeracy: 81.82% Numeracy: 90.72% 85.10% curriculum prior to start of 2017-18 academic year (Targets of 45% for Numeracy: -ECE Resource Kits being used in classrooms PDO Indicators 88.07% -Parent/caregiver education activities underway (Parent Circle 1 and 2 exceeded) pilot and mass media campaign ongoing) -NDA results for Hinterland used to measure students’ learning during Nursery 2 Note: “Approaching mastery� means that students scored at least 50% correct responses in each of the competency areas tested. The NDA for Nursery 1 includes four competency areas for both emergent Literacy and Numeracy, while the NDA for Nursery 2 includes six competency areas for Literacy and five for Numeracy. The percentages in the table are an average of scores in all Literacy and Numeracy competency areas. 20. It is natural that students would score higher on each successive NDA, given that each time the assessment was administered, they had spent more time in a classroom learning environment. However, the extent of improvement in the percentage of Hinterland students attaining at least an “approaching mastery� level of emergent literacy (a 36-percentage point increase) and numeracy (a 40-percentage point increase) during the cohort’s Nursery years is significant. As a benchmark for increase the NDA results for students in the Hinterland can be compared to the Coastal Regions. The percentage point increases achieved by Hinterland Nursery students in emergent literacy and numeracy (noted in the preceding paragraph) are on par with those achieved by their peers in Coastal Regions, where students attaining at least an “approaching mastery� level of emergent literacy and numeracy rose by 38 and 37 percentage points respectively. This indicates that Hinterland students are experiencing learning gains at rates equivalent to Coastland students. Further, the 12- Page 9 of 37 The World Bank Guyana Early Childhood Education Project (P129555) percentage point gap that existed between the Hinterland and Coastal Regions in the percentage of students attaining at least an “approaching mastery� level of emergent numeracy narrowed by three percentage points (to a 9-percentage point gap) during Cohort 1’s two years in Nursery. In emergent literacy, the learning gap remained on par, from a 11 to a 13-point gap. Although the gap was not reduced, project PDO targets were met, and it remains an achievement that the Hinterland did not fall further behind. 21. The Project Team also tracked the NDA performance of the cohort of students that entered its first year of Nursery school in September 2016, the year following Cohort 1. This second cohort of Hinterland students (Cohort 2) saw even larger learning gains than the Cohort 1. These notable performance improvements may be due to Cohort 2’s exposure to the full gamut of Project interventions. Their teachers brought to bear the benefits of two rounds of training (Training Modules 1 and 2 in the summers of 2016 and 2017), and the ECE Resource Kits were utilized for experiential, play-based classroom learning during both of Cohort 2’s Nursery years. In addition, caregiver-focused educational initiatives were underway both years (Parent Circles and the national mass media campaign), boosting the support of children’s early learning at home. The percentage of Cohort 2 students attaining at least an “approaching mastery� level of emergent literacy increased by 51 percentage points during two years of Nursery school, while the percentage of students attaining at least an “approaching mastery� level of emergent numeracy increased by 50 percentage points, compared to learning gains of 43 and 39 percentage points respectively in the Coastal region. As such, the long- standing performance gap that existed between Hinterland and Coastland students narrowed significantly during Cohort 2’s two years in Nursery school. The 11- and 14-point gaps in the percentage of Hinterland and Coastland students attaining at least “approaching mastery� levels of literacy and numeracy were reduced to 2 and 3 percentage points respectively, a remarkable achievement and a testament to the efficacy of the project interventions. Table 2. Percentage of Nursery Students Attaining at least “Approaching Mastery� Level of Emergent Literacy and Emergent Numeracy (Cohort 2) Hinterland Regions Coastal Regions Learning Gap September 2016: (NDA for Nursery 1 administered to ~8,000 students) -Students in Cohort 2 begin Nursery 1 year Literacy: 36.63% Literacy: 47.13% Literacy: 10.50 -Teachers have been trained to deliver Module 1 of newly Numeracy: 37.53% Numeracy: 51.78% Numeracy: 14.25 designed curriculum -ECE Resource Kits arrive in classrooms/teachers have been trained in their use -Parent/caregiver education activities begin (Parent Circle pilot underway, mass media campaign launched) June 2018: (NDA for Nursery 2 administered to ~8,600 students) -Students in Cohort 2 complete Nursery 2 year Literacy: 87.83% Literacy: 89.75% Literacy: 1.92 -Teachers will be trained to deliver Module 2 of newly Numeracy: 87.70% Numeracy: 90.84% Numeracy: 3.24 designed curriculum prior to start of 2017-18 academic year -ECE Resource Kits being used in classrooms -Parent/caregiver education activities underway (Parent Circle pilot and mass media campaign ongoing) Page 10 of 37 The World Bank Guyana Early Childhood Education Project (P129555) 22. The Project’s two PDO Indicators were measured using NDA results for the Hinterland schools, which represented 76% of schools targeted by the Project.9 The decision to use Hinterland NDA results was made early in the course of Project implementation, when it became evident that the MoE’s Information System (MEIS), which was being developed to monitor progress toward 2014-2018 ESP goals, did not yet have the functionality required to disaggregate NDA results specific to the Project’s set of remote riverine schools from the larger result sets for each of the Coastal Regions. As such, NDA results for the 73 Project-targeted remote riverine schools are embedded in the overall results for the Coastal Regions. This does not detract from the above results; if anything, it contributes to an underrepresentation of efficacy, as the effects of the intervention may have contributed to the rise in the scores in the Coastal region as well. 23. Even in the absence of disaggregated NDA results for the 73 remote riverine schools throughout the Project life span, a decision was made to continue to work with the schools so that teachers and students in these disadvantaged areas would benefit from teacher training, classroom monitoring and new learning materials. Students in the remote riverine areas had historically performed academically at the same level as their Hinterland peers,10 and the Bank worked with the MoE to advance efforts to capture NDA results for the Coastal sub-set of riverine Nursery students throughout Project implementation. 24. At Project close, the MoE was able extract results data from the September 2016 NDA for 40 of the 73 Project-targeted Nursery schools in the remote riverine areas. Seventy-eight percent of students attained at least an “approaching mastery� level of emergent literacy, and 83% of students attained at least an “approaching mastery� level of emergent numeracy. Literacy results achieved by students in the 40 remote riverine schools surpassed those of students in the Hinterland and Coastal Regions by 10% and 3% respectively. Their numeracy results were also higher than those of their Hinterland and Coastal Region peers by 6%. These initial results are very encouraging, and likely to be sustained given that the Project’s impact on improved quality of teaching is expected to continue over time. The MoE continues to improve the functionality of the MEIS in order to fully capture the learning gains realized by the students in the remote riverine areas. 25. In addition, in the course of Project implementation, it was realized that the MoE’s Primary Diagnostic Assessment (PDA) tool, which was planned to be used to test students for learning readiness at the start of each year of primary schooling, would still be under development at Project close. As with the remote riverine schools, the Project Team determined that Grade 1 teachers and students would remain part of the Project, even though the assessment tool that would measure literacy and numeracy learning outcomes for the students would not be ready. The rational was that Grade 1 teachers and students should benefit from the same interventions as Nursery teachers and students in order to ensure a smooth transition from Nursery to Primary school. As with the remote riverine NDA data, the Bank worked with the MoE throughout Project implementation towards capturing Grade 1 level PDA data. 26. Limited results data from the Grade 1 PDA was available at Project close. The PDA is still a work in progress; to date it includes only a literacy assessment, and, although it is administered to all Grade 1 students, 9 Correspondingly, 66.4% of Project resources allocated to Components 1 and 2 were spent on Hinterland teachers and classrooms, while 26.9% of resources were spent on teachers and classrooms in remote riverine areas. 10 Similarity in learning outcomes between children in the Hinterland and those in remote riverine areas is attributed to the fact that both groups experience the same ECE-related issues, including limited availability of trained teachers, difficult geography that often leads to poor school supervision, and parents with limited resources with which to stimulate their children’s early learning. Page 11 of 37 The World Bank Guyana Early Childhood Education Project (P129555) the local capacity required to compile, tabulate and enter all resulting data is limited. In September 2016, PDA data for 53,582 Grade 1 students was collected, while in September 2017, about half the amount of data was collected, with results for 26,602 students obtained nationally. While the data included results for students in the Hinterland Regions and remote riverine areas both years, the 2016 and 2017 data sets do not reflect representative or comparable samples. As such, gains in emergent literacy and numeracy associated with the Project’s impact on Grade 1 students cannot be directly measured. However, based on the strong results of the final NDA administered to the Cohorts at the end of its second year of Nursery in June 2017/2018, just three months before the cohorts began Grade 1 in September of the same year, it is likely that the objective of improving literacy and numeracy outcomes for Grade 1 students was achieved. Following the successful rollout of the NDA, the MoE has shifted focus to completing the development of the PDA in order to ensure its use as an assessment tool that will provide comprehensive and conclusive data on learning outcomes for primary school students going forward. 27. Student gains in emergent literacy and numeracy in the Hinterland Regions (as captured by the PDO Indicators and by NDA results data for select riverine Nursery schools) support the assertion that teacher training and monitoring and the provision of high-quality learning materials under the Project have begun to positively impact student achievement. Following is a discussion of the Project activities and related Intermediate Results Indicators that support this achievement. 28. Capacity building for Nursery and Grade 1 teachers. Under Component 1 of the Project, two consultants developed a new curriculum for Nursey and Grade 1 education, along with a training program for a cadre of Master Trainers who would in turn train in-service Nursery and Grade 1 teachers. Curriculum development was informed by focus group sessions with education stakeholders to understand the training needs of in-service ECE teachers, by a series of teacher/classroom observation visits in select school districts within the Hinterland and remote riverine areas, and by the results of the NDA administered to Nursery students in September 2015. 29. During June of 2016 2017, and 2018, 41 Master Trainers11 attended week-long training workshops in Georgetown, covering pedagogical content and teaching methods included in Modules 1, 2, and 3 of the Training Plan. Following the training workshops, Master Trainers trained in-service Nursery and Grade 1 teachers in the field 12 prior to the start of the academic year in early September. As the academic year progressed, Master Trainers visited classrooms to observe a sample of 30% of the teachers trained under the Project. The Master Trainers used the MoE’s Early Childhood Development Program Delivery Evaluation Checklist (PDEC) to monitor and evaluate in-service teachers’ performance in implementing the new curriculum using updated instructional strategies. Master Trainers also provided teachers with feedback and coaching during monitoring visits. 30. At Project close, 526 Nursery and Grade 1 teachers from Hinterland and remote riverine areas had completed training in all topics included in Training Modules 1, 2 and 3, exceeding Intermediate Result Indicator 1’s target of 400 teachers. Trained teacher evaluations conducted during the 2016 -17 academic year resulted 11 The majority of Master Trainers were Head Teachers for Nursery 1 and 2, and Nursery Field Officers, who evaluate and coach in-service Nursery teachers throughout the regions. Ten Training Monitors also participated in the Master Trainer Training Workshops. Training Monitors ensured that Master Trainers were effectively delivering training to classroom teachers, and assisted Master Trainers with trained teacher evaluations during the academic year. 12 Master Trainers conducted teacher training at 16 Department of Education offices across the country. Page 12 of 37 The World Bank Guyana Early Childhood Education Project (P129555) in 54.5% of teachers attaining an overall rating of “3� or “4� (out of four) on the PDEC,13 nearing the 60% target for Intermediate Result Indicator 2. The Project Team chose to report results for the trained teacher evaluations carried out during the 2016-17 academic year, as that year a representative sample of 30% of trained teachers was assessed. During the 2017-18 academic year, a set of teachers was specifically selected for evaluation based on geographical remoteness, in order to ensure that these teachers benefitted from the opportunity to be monitored and mentored by Master Trainers.14 31. A list of the training topics included in Modules 1, 2 and 3 is included in Annex 4. The Program Delivery Evaluation Checklist is included in Annex 5. 32. Supply of ECE Resource Kits to classrooms. The Project’s second component supported the procurement and distribution of 750 Resource Kits to all Nursery and Grade 1 classrooms targeted by the Project, achieving the target for Intermediate Result Indictor 4. 15 The kits included a variety of materials (connectable pieces in different shapes, colors, and sizes, wood block sets, magnets, art and craft items, small books, flash cards, etc.) to promote independent and small group learning through play. A Teacher’s Manual to guide effective use of items contained in the kits was developed by the MoE and reviewed with teachers as part of their training under Component 1. 33. During trained teacher evaluations during 2016-17, 78.6% of teachers were observed to be utilizing the Resource Kit as part of their teaching plans, exceeding the 70% target for Intermediate Result Indicator 5. It is notable that 71.2% of the observed teachers scored either a “3� or a “4� on the PDEC indicator that measured the degree to which teachers engaged children in play-based activities using materials from the kit, indicating proficiency in their use of the kit. 34. With the support of the Ministry of Indigenous Peoples’ Affairs, the Project Team developed a novel distribution method that moved hundreds of kits from a central staging site in Georgetown,16 where the kits were packed and labeled, to delivery to local schools in the four Hinterland Regions in just nine days. The distribution chain involved trucks, planes, and ferries, and the efforts of MoE regional and district personnel and many community members throughout the regions in order to ensure that the kits arrived at their destined schools in time for teacher training prior to the start of the academic year. The distribution method was deemed so effective that the MoE requested that the Project Team formally document it for future use. 35. Primary caregiver education. The Project’s third component supported the development of emergent literacy and numeracy among Nursery and Grade 1 children through the piloting of a Parent Circle initiative. A local consultant developed a training program and accompanying Facilitator’s Manual elaborating a series of exercises designed to strengthen parenting awareness, knowledge, and skills in support of children’s emerging 13 Master Trainers used the checklist to grade teachers on a scale of “1� to “4� for qualitative indicators related to 13 observable classroom behaviors. A score of “4� reflects strong evidence of an indicator, while a score of “1� represents very weak evide nce. 14 The final Master Trainer Training Workshop, focused on Module 3 topics, took place in June 2018. The MoE will assume responsibility for trained teacher evaluations during the 2018-19 academic year (following Project close) as part of its Continuing Professional Development program. 15 The Project Team also procured and distributed 68 additional kits to Nursery and Primary schools in more accessible riverine areas, as well as to Department of Education offices in the regions, local Learning Resource Centers (see footnote 19) and community health centers, select Special Ed facilities, and to the Cyril Potter College of Education and University of Guyana. 16 Teaching students at the Cyril Potter College of Education in Georgetown helped to assemble, package and label the ECE Resource Kits prior to distribution to the regions. Page 13 of 37 The World Bank Guyana Early Childhood Education Project (P129555) literacy and numeracy. 36. Fourteen Parent Circle Facilitators 17 were trained in use of the manual to engage parents and caregivers in children’s early learning at home. The Parent Circle pilot was carried out during the 2016-17 school year via weekly sessions held in thirteen communities in Hinterland Regions 1 and 7. A total of 212 caregivers of children between ages three and six benefitted from the sessions.18 37. Following the 2016-17 pilot, nine new communities expressed interest in forming Parent Circles, and the program expanded beyond Regions 1 and 7 to incorporate groups in Regions 3 and 4 for the 2017-18 school year. While the Project supported an additional training workshop for new and returning facilitators, the Parent Circle initiative moved under the purview of the MoE during its second year, with Regional and District Education Officers monitoring the Parent Circles during 2017-18. 38. To reach a wider audience with the message of the importance of fostering literacy and numeracy in children, a creative agency was hired to develop the Read-Play-Love mass media campaign. The agency collaborated with the Nursery and Primary Assistant Chief Education Officers and ECE specialists from the National Center for Educational Resource Development (NCERD, the MoE’s in-service teacher training unit) to develop posters, brochures, “tip sheets�, and DVDs19. This collateral was then dispatched to public venues including schools, community health centers, Learning Resource Centers,20 and local markets for display and use.21 Radio and television advertisements were also developed, and ran on commercial radio and TV stations, as well as on the MoE’s Guyana Learning Channel, reaching a nationwide swath of parents and caregivers of young children. A website was also launched to further promote active parent and caregiver involvement in children’s literacy and numeracy development. The campaign was implemented during the 2016-17 and 2017- 18 school years.22 39. Examples of collateral developed for the Read-Play-Love campaign can be viewed at http://www.readplaylove.gy.23 17 Parent Circle Facilitators included retired educators, Community Development Officers, Community Health Workers, primary school teachers, and local business owners. 18 Parent Circle meetings were held in a variety of venues, including nursery and primary schools, community health centers, and local restaurants. Participants were given materials (games, books, and drawing and writing materials) to use at home to engage their children in play-based development of literacy and numeracy skills. 19 The DVDs contain infomercials featuring “ECE Champions" from a wide cross-section of Guyanese society (politicians, sports and music personalities, education leaders, etc.) who encourage parents and caregivers to embrace their role as their child’s first teacher. The DVDs were played during “Parent Days� at schools, “Well Baby� days at community health centers, and at the start of community meetings held at a variety of venues. 20 Learning Resource Centers are multi-purpose educational spaces in communities throughout Guyana. They host libraries and satellite learning hubs and provide a venue for community meetings. 21 A version of the distribution model that was developed to deliver the ECE Resource Kits was utilized to distribute print and DVD Read-Play-Love collateral to the regions. 22 Intermediate Result Indicator 6 was achieved with the timely implementation of the Read-Play-Love campaign at the start of the 2016-17 academic year. 23 The creative agency that designed the Read-Play-Love campaign won four industry awards for this work in 2017: three silver awards (for a public service video, a public service website, and an integrated promotional media campaign) and a bronze award (for a public service TV announcement). Page 14 of 37 The World Bank Guyana Early Childhood Education Project (P129555) Overall Outcome Rating Rating: Satisfactory 40. Improvements in emergent literacy and numeracy outcomes for students at the Nursery level, evidenced by the two PDO Indicators that exceeded targets, suggest an Overall Outcome Rating of Highly Satisfactory. However, given the limited ability to measure learning outcomes for a portion of Project beneficiaries (Nursery students in remote riverine areas and Grade 1 students), Overall Outcome is rated Satisfactory. 41. The Satisfactory rating is further supported based on the Project’s relevancy, efficacy and efficiency: • The Project was relevant as a short-term intervention that focused on much needed improvements in the ECE offered to Guyana’s most disadvantaged early learners. Such interventions have the potential to provide long-term benefits in the form of improved earnings for individuals and bolstered economic growth overall.24 • The Project was effective in achieving its PDO as measured by the two PDO Indicators, both of which exceeded targets. Further, the recently available NDA data for Nursery students in Project-targeted remote riverine schools supports the assertion that students in these areas are indeed experiencing improved learning outcomes on par with their Hinterland peers. • The Project was efficient in that a relatively small grant enabled significant improvements in the ECE landscape in a short time period. A training program provided Nursery and Grade 1 educators with new curriculum and teaching methodologies to foster their students’ literacy and numeracy skills. ECE Resource Kits were introduced in classrooms and used by teachers to promote the play-based learning that is most suitable for young children. And a successful Parent Circle pilot and wide-reaching media campaign were implemented, each of which encouraged parents and caregivers to support children in their earliest stages of learning (and were the first examples of parent/caregiver-focused interventions by the MoE). Other Outcomes and Impacts 42. Capacity building within the MoE. MoE personnel strengthened their knowledge and capacity in a variety of areas as they supported the Project. Master Trainers and Training Monitors developed a great deal of expertise through the Project training they received, and as they trained and evaluated in-service teachers. Consultants from NCERD helped design the Master Trainer training modules, authored the Teacher’s Manual for use of the ECE Resource Kit, and developed the training program and accompanying Facilitator’s Manual for the Parent Circle pilot. A host of MoE employees at the regional and district levels compiled, analyzed, and submitted NDA and trained teacher evaluation data. These people will continue to put their expertise to use within the MoE, contributing to efforts to strength teaching and learning in the ECE sector going forward. 43. Implementation of the MoE’s Nursery Education Action Plan. The Project was specifically designed 24As noted in Context section, examples include: James J. Heckman, Early Childhood Education (2016); Isabel V. Sawhill, Jeffrey Tebbs, William T. Sickens, The Effects of Investing in Early Education on Economic Growth (2006); OECD, Investing in High Quality Early Childhood Education and Care (2012). Page 15 of 37 The World Bank Guyana Early Childhood Education Project (P129555) to support the objectives of the Nursery Education Action Plan. Project activities directly contributed to improving the quality of literacy and numeracy teaching in disadvantaged areas, and to decreasing differences in learning attainment between the Hinterland and Coastal Regions. 44. Other impacts. Project activities had many influential effects, including: • Master Trainers and Training Monitors anecdotally reported very positive feedback from ECE teachers on the new curriculum and interactive teaching methodology developed under the Project, which have bolstered their confidence and their sense of purpose as educators. This enthusiasm will go a long way toward ensuring continued improvement of teaching and learning outcomes. • Many Grade 1 teachers re-organized their classrooms so that they were less formal, replacing rows of desks that faced a teacher’s desk and chalkboard with an arrangement that resembled a Nursery classroom set-up, with desks grouped together for collaborative learning, “interest corners� that students could migrate to, and dedicated wall space for displaying student work. • Teachers from lower Primary grades borrowed elements from the ECE Resource Kits in Nursery and Grade 1 classrooms to make their own teaching lessons more experiential. • Nursery teachers reported that they had witnessed improved classroom performance for individual children whose parents participated in the Parent Circle initiative. • The MoE now requires that Nursery schools administer the NDA a second time at the end of the Nursery 1 and 2 years (as was done for Cohort 1) in order to accurately track students’ progress during and upon transitioning out of the Nursery sector. • The MoE will utilize the logistics chain developed for the ECE Resource Kits for future dispatch initiatives to ensure that materials reach schools and classrooms in a timely manner. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME Key factors during Project preparation 45. Consultations were conducted with indigenous communities in the Hinterland Regions during appraisal, providing a forum for community members to become familiar with the proposed Project and to provide feedback on Project activities and their anticipated impacts. Community concerns, outlined in the July 2014 Amerindian Peoples Plan, informed development of the curriculum design, teacher training and parent and caregiver education elements of the Project. 46. The Project’s Results Framework was designed to utilize the Nursery Diagnostic Assessment and ECE Program Delivery Evaluation Checklist (two tools previously developed by the MoE), as well indicators that were already being tracked by the MoE to monitor implementation of the Nursery Education Action Plan. Both these factors eliminated the need to establish additional M&E channels, thus streamlining the M&E process for the Project Team. Page 16 of 37 The World Bank Guyana Early Childhood Education Project (P129555) 47. Component 4 under the Project25 ensured that sufficient attention was given to supporting Project implementation, given the MoE’s ongoing need to strengthen institutional capacity in areas of teacher training, educational outreach to parents and caregivers, and fiduciary, procurement and M&E aspects of project management. Key factors during Project implementation 48. Project coordination, implementation, financial management and procurement duties were managed by a small Project Team based in the MoE Planning Unit. The three team members (a Nursery Technical Coordinator who was responsible for overall Project management and two supporting Finance Officers) had extensive experience working on previous Bank projects. Technical expertise was provided as required by the MoE’s Nursery and Primary Assistant Chief Education Officers, a Procurement Specialist from the Planning Unit, and staff from NCERD. With the Bank team’s supervision and support, Project implementation ran very smoothly. 49. Collaborative planning and stakeholder buy-in were essential elements in ensuring that Project activities were well-designed and had the highest likelihood of yielding positive impacts. Curriculum development, as described previously, was informed by focus group sessions with education stakeholders and teacher/classroom observation visits. Community consultations in select Hinterland and riverine areas on parental involvement in early education informed the content and methodology included in the parent and caregiver education modules used during the Parent Circle pilot, and also helped shape the messaging used in the Read-Play-Love mass media campaign. 50. Feedback loops were essential to improving Project activities and outcomes during the three-year implementation period. Recommendations received from Master Trainers following their annual training workshops, their weeks training in-service teachers, and teacher monitoring visits during the course of the academic year, as well as observations from in-service teachers who had undergone training, informed and shaped ongoing curriculum development and the related teacher training modules.26 Anecdotal and survey feedback provided by parents and caregivers and Facilitators of the Parent Circle pilot related to perceptions of the effectiveness of group interactions and materials improved the second round of the initiative during the 2017-18 school year. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME Bank Performance, Compliance Issues and Monitoring and Evaluation 51. Bank performance is rated as Satisfactory. The Project’s quality at entry was substantiated by GPE’s Quality Assurance Review from late 2014, which described the Project as an effective instrument for supporting education sector priorities in Guyana and noted the Project’s high likelihood of success given its components, which reflected research on a multitude of ECE interventions abroad that had been tested and worked well. GPE’s review also assessed implementation readiness as high and likely to ensure aid effectiveness. 25Project Implementation Support, Administration and M&E (US$300,000). 26A Dropbox “virtual space� was set up to facilitate feedback sharing and interactive conversations between the Master Trainers and the consultants who developed the curriculum and related Training Plan. Page 17 of 37 The World Bank Guyana Early Childhood Education Project (P129555) 52. The Bank Team effectively guided the Project Team, ensuring that Project implementation progressed on schedule and that all financial management and reporting duties were carried out in a satisfactory manner. Further, the Amerindian Peoples Plan was well served, as Project activities were designed and then fine-tuned as required during the life of the Project to ensure that the concerns of affected indigenous communities were addressed. However, due to circumstances outside the Bank’s control, it was not possible to fully measure achievement of the PDO because of a lack of data on literacy and numeracy outcomes for some of the students targeted by the Project. While the Bank and Government opted to continue to support teachers and students in the remote riverine areas and Grade 1 classrooms (thus maintaining the PDO’s targeted beneficiary population), a restructuring was not deemed an efficient use of time and resources given the Project’s relatively small budget and condensed timeline. 53. M&E quality is rated as Substantial. As previously described, the Project’s Results Framework relied on two existing MoE instruments (the NDA and the ECD Program Delivery Evaluation Checklist) that were developed to monitor and evaluate implementation of the 2014-2018 ESP. The Results Framework also utilized indicators that the MoE had already designed to track performance of the Nursery Education Action Plan, supplementing these with several Project-specific intermediate indicators. These efficiencies minimized additional data collection requirements and costs, and avoided duplication of monitoring activities, thus resulting in a streamlined M&E process. 54. Due to the lack of consistent measurement of emergent literacy and numeracy outcomes for the remote riverine schools and the Grade 1 classrooms encompassed by the PDO, M&E is considered to have moderate shortcomings. Baselines and targets for both groups of students could not be set because, as described previously, NDA results for the riverine schools could not be disaggregated from the greater group of Coastal Region schools, and delays in the development of the MoE’s PDA assessment tool resulted in incomplete data on Grade 1 student learning. 55. Both PDO Indicators exceeded their targets substantially. The modest targets chosen by the Project Team (5% and 3% increases in the percentage of children attaining an “approaching mastery� level of emergent literacy and numeracy respectively) reflect the intrinsic uncertainty around target setting for an initial effort to measure learning outcomes at the Nursery level using a newly designed assessment tool (the NDA). The choice not to revise indicator targets upward during the course of Project implementation reflects the decision, described previously, that a restructuring would not be an efficient use of available time and resources. 56. During the life of the Project, the MoE made strides in its maiden effort to collect and analyze reliable school-level data using the NDA and PDA assessment tools and the MEIS system. Effective use of such data evidence will improve education sector planning and policy making over time. Risk to Development Outcome and Sustainability of Project achievements 57. The MoE remains committed to strengthening the emergent literacy and numeracy skills of children in the Hinterland and remote riverine areas of Guyana and intends to build upon the progress that was made under the Project. 58. Use of the Nursery Diagnostic Assessment tool to assess readiness for learning will continue, and training in the administration of the NDA will be provided to new teachers going forward. To ensure that Page 18 of 37 The World Bank Guyana Early Childhood Education Project (P129555) teachers continue to utilize improved pedagogical practices in their classrooms, NCERD will incorporate the three teacher training modules developed under the Project into standard training for new Nursery and Grade 1 teachers. In-service teachers who were not included in the Project will also be trained in the modules as part of the Continuing Professional Development (CPD) program. 59. With respect to the ECE Resource Kits, the regional Departments of Education have allocated funds to be used to replenish the kits as needed. The MoE has committed to purchasing kits for Nursery and Grade 1 classrooms nationwide that did not receive them under the Project.27 Teacher training in the effective use of the Resource Kit as a teaching tool is now a component in CPD training. 60. As described previously, the MoE’s School Support Unit assumed oversight of nine Parent Circle groups during 2017-18, and at Project close was reviewing whether to dedicate resources to sustaining and/or expanding the Parent Circle initiative. The MoE has maintained the Read-Play-Love website since the second year of the mass media campaign (2017-18) and will continue to do so following Project close. NCERD retains the source materials for all Read-Play-Love campaign assets for possible repurposing as part of future public outreach. V. LESSONS LEARNED AND RECOMMENDATIONS 61. Training and follow-up mentoring over a longer period can sustain teaching improvements. The “cascade� model of teacher training, where Master Trainers train, evaluate and then support teachers in the field over several years, is effective. The Project’s strategy of providing training and monitoring over three consecutive years helped to ensure that new teaching practices would stick and allowed the Master Trainers and teachers to grow together as a community of learners. Results show that Teachers’ performance improved with the frequency of follow up visits, and is a crucial component of training. 62. Feedback and flexibility to adapt throughout implementation is critical to optimize training plans. Communication was regular between the two consultants who headed curriculum development and training and the Master Trainers as they delivered training in every region and followed up with teacher monitoring visits. Input on what was working well and what teachers needed more of was welcomed by the consultants, who modified training ideas, priorities and activities accordingly. For example, when Master Trainers conveyed concerns about many teachers’ lack of mathematical literacy (witnessed during monitoring visits during the 2016-17 school year), the consultants re-worked sections of Training Module 2 for an increased focus on teaching mathematics in time for the Summer 2017 Master Trainers Training Workshop. This approach can result in greater engagement from teachers, and in turn, better learning outcomes. 63. Involvement of local consultants can be critical to Project success. The Project benefitted greatly from the support of a local ECE expert, who was a 40-year veteran of the Education sector and had advanced from a classroom teacher to the MoE’s Chief Education Officer during the course of her career. She brought invaluable knowledge about the cultural and educational environment in Guyana to her role as partner to the external consultant who developed a new Nursery and Grade 1 curriculum and related teacher training program. The local consultant visited all of the regions during the curriculum development, Master Trainer and teacher training phases, and her stature helped to secure local school and community support for the Project’s 27 The Resource Kits will likely need to be replaced approximately every three years. Page 19 of 37 The World Bank Guyana Early Childhood Education Project (P129555) efforts to train teachers in new pedagogical methodologies. She also served as a sounding block for the Master Trainers, who could candidly share with her what they were witnessing in classrooms, as well as their own ideas for teacher training. To complement a flexible approach, local knowledge can be crucial to success. 64. Logistical and motivational challenges can impact participation in interventions such as Parent Circles, and must be addressed to ensure their effectiveness. Two retired Regional Education Officers served as Monitors during the Parent Circle pilot in 2016-17. They conducted a participant survey that validated the relevancy of the pilot: 87% of parents/caregivers agreed that Parent Circle meetings had helped them support their children’s developing literacy skills, and 85% agreed that the meetings helped them support their children’s numeracy skills. Eighty-six percent of participants responded that they would recommend the Parent Circle program to other caregivers. The survey also captured the main challenges that impeded regular attendance at meetings: competing demands for parents’ time due to household and childcare duties and employment obligations, and transportation issues (excessive time spent walking to meetings, lack of funds to cover the coast of boat transport to meetings). These challenges could be addressed in several ways. A stipend could be paid to parents who want to participate in Parent Circles, and meetings could take place on a monthly basis in the form of half-day workshops, rather than weekly sessions. Alternatively, the Parent Circle program could be modified to provide parents and caregivers with a single orientation session that would cover strategies for stimulating a child’s emerging literacy and numeracy skills and equip parents with a set of take- home materials (games, books, flash cards, puzzles, etc.) that could be used to engage children, along with a simple usage manual. A full assessment of participants needs not just in terms of content, but also in terms of delivery/logistics, can anticipate participation issues, and provide options for course correction along the way. 65. Use of new and local assessment instruments can help develop capacity, but reliance on them carry risks of ability to measure Project outcomes. The NDA and PDA were relatively new instruments to the Government at time of project preparation. Through support of the Project, the MoE gained improved capacity of instrument administration, data collection, and analysis, and their full integration into MoE operations is a huge success. However, reliance on these tools, which were not fully functional at time of approval, came at the expense of ability to measure some parts of the PDO. Projects should not avoid use of new and local assessment tools, but may consider incorporating other measures to assess the PDOs, as well as include specific and additional support to the development and use of the instruments themselves. . Page 20 of 37 The World Bank Guyana Early Childhood Education Project (P129555) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: The objective of the Project is to improve emergent literacy and numeracy outcomes for children at the nursery level and primary grade one in Hinterland Regions and Targeted Remote Riverine Areas. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion PDO Indicator 1: Percentage of Percentage 39.58 45.00 75.83 nursery students attaining at least “Approaching Mastery� 15-Oct-2015 30-Sep-2018 30-Sep-2018 level of emergent literacy Comments (achievements against targets): Achieved and exceeded the target. “Approaching mastery� is defined as at least 50% correct responses in each required competency area. The percentages from each of the following four competency areas were averaged to produce a final score: autobiographical awareness, color recognition, letter identification, and alphabet recitation. Baseline percentage based on the results of the September 2015 Nursery Diagnostic Assessment, which provides data on children’s preparedness for entry to Nurs ery school. The NDA was administered to a 30% sample of children nationwide who were beginning Year 1 of Nursery (as the Project Cohort was). The NDA was administered to the same cohort of children twice more: 1) at the beginning school year 2016-17 to measure Nursery Year 1 results, and 2) at the end of school year 2016-17 to measure Nursery Year 2 results. Page 21 of 37 The World Bank Guyana Early Childhood Education Project (P129555) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion PDO 2: Percentage of nursery Percentage 41.91 45.00 81.82 students attaining at least “Approaching Mastery� level of 15-Oct-2015 30-Sep-2018 30-Sep-2018 emergent numeracy Comments (achievements against targets): Achieved and exceeded the target. “Approaching mastery� is defined as at least 50% correct responses in each required competency area. The percentages from each of the following four competency areas were averaged to produce a final score: quantity differentiation, shape identification, numeral identification, and counting fluency. Baseline percentage based on the results of the September 2015 Nursery Diagnostic Assessment, which provides data on children’s preparedness for entry to Nurs ery school. The NDA was administered to a 30% sample of children nationwide who were beginning Year 1 of Nursery (as the Project Cohort was). The NDA was administered to the same cohort of children twice more: 1) at the beginning school year 2016-17 to measure Nursery Year 1 results, and 2) at the end of school year 2016-17 to measure Nursery Year 2 results. A.2 Intermediate Results Indicators Component: Component 1: Capacity Building for Nursery and Grade 1 Teachers. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion IRI 1: Number of Nursery and Number 0.00 400.00 526.00 grade 1 teachers completing the Project-financed training 30-Sep-2014 30-Sep-2014 30-Sep-2018 program Comments (achievements against targets): Target achieved. Page 22 of 37 The World Bank Guyana Early Childhood Education Project (P129555) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion IRI 2: Percentage of teachers Percentage 0.00 60.00 54.50 attaining a rating of “3� or “4� (out of 4) on the Early 30-Sep-2014 30-Sep-2018 30-Sep-2018 Childhood Development Program Delivery Evaluation Comments (achievements against targets): Target substantially achieved; 91% of target achieved. Indicator was measured by Master Trainers and Training Monitors using the MoE’s Program Delivery Evaluation Checklist (PDEC) to evaluate how well teachers were impleme nting the instructional strategies taught during their training. Classroom visits were made to 30% of trained teachers during the 2016-17 school year. Master Trainers used the PDEC to grade teachers on a scale of “1� to “4� for qualitative indicators related to 13 observable classroom behaviors. A score of “4� reflects strong evidence of an indicator, while a score of “1� represents very weak evidence. The Proje ct Team chose to report results for the trained teacher evaluations carried out during the 2016-17 academic year, when a representative sample of 30% of trained teachers was assessed. During the 2017-18 academic year, a set of teachers was specifically selected for evaluation based on geographical remoteness, in order to ensure that these teachers benefitted from the opportunity to be monitored and mentored by Master Trainers. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion IRI 5: Percentage of teachers Percentage 0.00 70.00 78.60 observed to be utilizing the Resource Kits during post- 30-Sep-2014 30-Sep-2018 30-Sep-2018 training classroom observations. Page 23 of 37 The World Bank Guyana Early Childhood Education Project (P129555) Comments (achievements against targets): Target exceeded; 112% of target achieved. Indicator was measured by Master Trainers and Training Monitors using the MoE’s Program Delivery Evaluation Checklist (PDEC) to evaluate the degree to which teachers were using the ECE Resource Kits in their classrooms. Classroom visits were made to 30% of trained teachers during 2016-17 school year. Teachers needed to score a minimum of “2� (on a scoring scare of “1� to “4�) on the PDEC’s observable classroom behavior indicator for Resource Kit use. Of the 78.6% of teachers observed to be using the Resource Kits during classroom observations, 71.2% received a score of “3� or “4�, indicati ng effective use of the kit as a teaching tool. The Project Team chose to report results for the trained teacher evaluations carried out during the 2016-17 academic year, when a representative sample of 30% of trained teachers was assessed. During the 2017-18 academic year, a set of teachers was specifically selected for evaluation based on geographical remoteness, in order to ensure that these teachers benefitted from the opportunity to be monitored and mentored by Master Trainers. Component: Component 2: Improved Supply of Teaching and Learning Materials. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion IRI 4: Number of Resource Kits Number 0.00 750.00 750.00 distributed to nursery and grade 1 classrooms 30-Sep-2014 30-Sep-2018 30-Sep-2018 Comments (achievements against targets): Target 100% achieved. Component: Component 3:Primary Caregiver Education. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion IRI 6: A mass media campaign Text Mass media campaign Mass media campaign Mass media campaign is implemented, promoting is not implemented is implemented. implemented Page 24 of 37 The World Bank Guyana Early Childhood Education Project (P129555) active primary caregiver 30-Sep-2014 30-Sep-2018 30-Sep-2018 involvement in nursery school aged children’s literacy and numeracy development. Comments (achievements against targets): Target 100% achieved. The Read-Play-Love mass media campaign launched in September 2016. Component: Unlinked indicators. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Direct project beneficiaries Number 0.00 5000.00 8713.00 30-Sep-2014 30-Sep-2018 30-Sep-2018 Female beneficiaries Percentage 0.00 0.00 50.00 51.00 30-Sep-2014 30-Sep-2018 30-Sep-2018 Comments (achievements against targets): Achieved and exceeded the target. Target also surpassed for percentage of female beneficiaries; 102% of target achieved. Beneficiaries include children attending two years of Nursery education and Grade 1 of Primary under the Project, as well as Nursery and Grade 1 teachers trained under the Project. 212 primary caregivers living in Regions 1 and 7 are also counted as Project beneficiaries, as they benefitted from voluntary participation in the Parent Circle pilot (Component 3) during the 2016-17 school year. Page 25 of 37 The World Bank Guyana Early Childhood Education Project (P129555) B. ORGANIZATION OF THE ASSESSMENT OF THE PDO Objective/Outcome 1: Improve emergent literacy and numeracy outcomes for children at the Nursery level and Primary Grade 1 in Hinterland Regions and targeted remote riverine areas. 1. PDO Indicator 1: Percentage of Nursery students attaining at least “approaching mastery� level of emergent literacy, defined as at least 50% correct responses in each of four competency areas: autobiographical awareness, color recognition, letter identification, and alphabet recitation. Outcome Indicators 2. PDO Indicator 2: Percentage of Nursery students attaining at least “approaching mastery� level of emergent numeracy, defined as at least 50% correct responses in each of four competency areas: quantity differentiation, shape identification, numeral identification, and counting fluency. 1. IRI 1: Number of Nursery and Grade 1 teachers completing the Project-financed training program. 2. IRI 2: Percentage of teachers attaining a rating of “3� or “4� (out of 4) on the Early Childhood Development Program Delivery Evaluation Checklist. 3. IRI 4: Number of Resource Kits distributed to Nursery and Grade 1 Intermediate Results Indicators classrooms. 4. IRI 5: Percentage of teachers observed to be utilizing the Resource Kits during post-training classroom observations. 5. IRI 6: A mass media campaign is implemented, promoting active primary caregiver involvement in nursery school-aged children’s literacy and numeracy development. Key Outputs by Component 1. Component 1 / Capacity building for Nursery and Grade 1 teachers: (linked to the achievement of the Objective/Outcome 1) Page 26 of 37 The World Bank Guyana Early Childhood Education Project (P129555) • Development of a new curriculum for Nursery and Grade 1 education and a corresponding in-service teacher training program. • Implementation of the in-service teacher training program over three years (2016, 2017, and 2018). 526 Nursery and Grade 1 teachers were trained to deliver a new curriculum using new instructional strategies for emerging learners in the Hinterland Regions and remote riverine areas of the Coastal Regions. • Classroom monitoring visits carried out during three academic years (2016-17, 2017-18, and 2018-19) to observe a 30% sample of trained teachers in order to evaluate their implementation of new instructional strategies. 2. Component 2 / Improved supply of teaching and learning materials: • Procurement and distribution of 750 ECE Resource Kits to all Nursery and Grade 1 classes targeted by the Project, along with a Teacher’s Manual for effective classroom use of the kits. 3. Component 3 / Primary caregiver education: • Design and implementation of a Parent Circle Pilot, carried out in 13 Hinterland communities during 2016-17, to strengthen parenting awareness, knowledge, and skills in support of children’s emerging literacy and numeracy. A related Facilitators Manual was developed for leaders of the Parent Circles. • Design and execution of the Read-Play-Love mass media campaign, which emphasized the important role a parent/caregiver plays in a child’s early intellectual development. (Collateral included radio and television ads, print materials, DVDs, and a website.) Page 27 of 37 The World Bank Guyana Early Childhood Education Project (P129555) . ANNEX 2. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Closing Percentage of Approval Components (US$M) (US$M) (US$M) Component 1: Capacity Building for Nursery and US$0.65 US$0.65 100% Year 1 Teachers Component 2: Improved Supply of Teaching and US$0.40 US$0.40 100% Learning Materials Component 3: Primary Caregiver Education US$0.35 US$0.35 100% Component 4: Project Implementation Support, US$0.30 US$0.30 100% Administration and M&E Total US$1.7 US$1.7 100% Page 28 of 37 The World Bank Guyana Early Childhood Education Project (P129555) ANNEX 3. RECIPIENT, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Borrower’s Report Guyana Early Childhood Education Project October 2018 Introduction The objective of the Guyana Early Childhood Education Project (GECEP) was to improve emergent literacy and numeracy outcomes for children at the Nursery level and Primary Grade 1 in Hinterland Regions and targeted remote riverine areas. This would be achieved by strengthening the capacity and effectiveness of Nursery and Grade 1 teachers through an in-service teacher training program (Component 1 of the Project), providing additional teaching and learning materials to Nursery and Grade 1 classes (Component 2), and reinforcing primary caregiver engagement in children’s early learning through a parent education pilot program and a mass media campaign (Component 3). Upon its close, the Project celebrates the good beginnings that have resulted from its efforts. GECEP achieved positive change in early childhood educational performance through better trained teachers who have made strides in pedagogy and content knowledge, new classroom materials that support developmentally appropriate learning activities, and an emphasis on parental involvement as a means to improve the likelihood of positive educational outcomes for children. Assessment of Project Outcomes Component 1: Capacity Building of Nursery and Primary Grade 1 Teachers The following activities led to success under Component 1: • Building content knowledge and pedagogical expertise at the regional level through the development of Master Trainers who learned from international leaders and local experts in the ECE, Literacy, Numeracy, and Parent Education arenas. • Conducting intensive, interactive, learn-by-doing practitioner development sessions at the sub- regional level to ensure that the learning experience is embedded within teachers’ local context. • Vigilant monitoring and on-the-spot coaching of teachers in their own teaching and learning environments. As a direct result of these efforts, there are now 41 Master Trainers and 526 trained teachers who have been monitored and coached. Component 2: Improved Supply of Teaching and Learning Materials The following activities led to success under Component 2: • ECE Resource Kits were distributed to all Nursery and Primary Grade 1 spaces in the Hinterland, remote riverine areas, Departments of Education, pre-service teacher training institutions, and Learning Resource Centers. Kits were also distributed to special education facilities in Education Districts 4, 6, 7 and in Georgetown, and to the Pediatric Unit of Georgetown Public Hospital. • Teacher training on how to effectively utilize the contents of the Resource Kit to enhance teaching and learning, including incorporating natural materials prevalent in local environments. Page 29 of 37 The World Bank Guyana Early Childhood Education Project (P129555) • Close monitoring of teachers’ use of Resource Kits in the classroom. As a direct result of these efforts, there are 750 ECE Resource Toolkits in use across a variety of teaching/learning environments. Component 3: Primary Caregiver Education The following activities led to success under Component 3: • A one-year pilot of a primary caregiver capacity building intervention was conducted and closely monitored. • A national mass media campaign (Read-Play-Love) was launched and carried out, reminding citizens that nurturing care of the young is the nation’s responsibility. • The PTA meeting structure was altered to ensure that time was devoted to parenting and other ECE matters, rather than solely to school events and administration. As a direct result of these efforts, 14 Parent Circle Facilitators were trained (with 12 additional Facilitators trained for a second round of the Parent Circle initiative in 2017-18) and over 200 caregivers benefitted from the Parent Circle intervention. In addition, over 38,000 pieces of print media and 500 ECE infomercial DVDs were distributed nationwide, and seven radio jingles were produced and aired. Component 4: Project Implementation Support, Administration and M&E As all matters related to the Project’s finances and administration have been found by auditors to be sound and well-executed, success under Component 4 is highlighted through the Project’s two PDO Indicators. The growth manifested in children from the Hinterland in three short years is to be commended and seen as a powerful reminder about the importance of educational quality and equity. Two cohorts of children were followed under the Project: Cohort 1 (the cohort tracked for World Bank results framework reporting), which entered the Nursery Program in September 2015 and exited in June 2017, and a second cohort (Cohort 2), which entered the Nursery Program the following year (September 2016), and exited in June 2018. Cohort 1 benefitted from one year of Project intervention during the 2016-17 school year, while Cohort 2 benefitted from two years of Project intervention. As demonstrated in the table below, by the end of Cohort 1’s Nursery experience, children from the Hinterland showed a larger percentage change in performance as compared to their Coastland counterparts. For Cohort 2, this movement was repeated on a larger scale. A comparison of the two cohorts shows that a higher percentage of Hinterland children from Cohort 2 were approaching mastery in both emergent literacy and numeracy than those from Cohort 1. For children from the Coastland, exit performance remained relatively similar to what was observed in the Cohort 1. Further, the exit scores of Hinterland children from Cohort 2 gained on the scores achieved by children from the Coastland and show improvement over the exit performance of Hinterland children vis-à-vis Coastland children in Cohort 1. These results provide strong evidence of a narrowing of the Hinterland/Coastland performance gap in emergent literacy and numeracy during the Project’s lifetime. Page 30 of 37 The World Bank Guyana Early Childhood Education Project (P129555) Evaluation of the World Bank during Preparation, Implementation and Supervision of the GECEP Project The Bank prepared a Project that furthered the main objectives (improved literacy and numeracy outcomes for Guyana’s youngest learners) of the MoE’s Nursery Education Action Plan. During the course of implementation, the Bank team effectively supervised the Project Team in all efforts related to project management and monitoring and evaluation of indicator targets, as well as procurement and financial aspects of the Project. The Bank team’s support was optimal, and directly influenced the Project’s strong overall performance (smooth implementation over three years, achievement of indicator targets). Lessons Learned Each component of the GECEP presented its own confluence of experiences that led to many “teachable moments� for the Project Team and the MoE leadership. Component 1: Capacity Building of Nursery and Primary Grade 1 Teachers • Modelling and experiential-based learning, as teaching strategies, were crucial to the success of the training program. • Empowering education districts with their own cadre of ECE specialists increases the likelihood of monitoring and feedback that is specific to improving the teaching process for educators at their level. • Frequent and ongoing monitoring and feedback to ECE educators by their trainers is critical to the sustained success of a training program. • Review of the trained teacher monitoring instrument at least once per training cycle validates the reliability and validity of evaluation criteria. Page 31 of 37 The World Bank Guyana Early Childhood Education Project (P129555) • Empowering Master Trainers to tailor training delivery and content (within reason vis-à-vis the modules presented during Master Trainer Workshops) to the specific needs of their regions improved beneficiary buy-in and increased satisfaction with the capacity building experience. • Advance planning of training workshops allowed enough lead time for course correction in instances where plans were unexpectedly derailed (e.g., by poor weather). • From a fiduciary perspective, having clearly codified regulations around the advances given to execute training and monitoring-related duties can assist in making this process more streamlined. Component 2: Improved Supply of Teaching and Learning Materials • Beginning procurement processes early saves time and money, as the Project Team was able to properly research a range of options for the ECE Resource Kits. • Distribution planning benefitted from the support of the Ministry of Indigenous Peoples’ Affairs. The Ministry ‘s logistical, technical, and human resource assistance went beyond what the Project Team would have received had it relied solely on MOE resources and personnel. • Utilizing experts from multiple subject disciplines in the conceptualization of the toolkit items was crucial to ensuring that the items would stand up to the rigors of cross-curricular planning and subject integration for enriched instruction. • Maintenance of a log of private sector entities that assisted with Resource Kit distribution will be useful for similar future endeavors. Component 3: Primary Caregiver Education • The following should be taken into account by the MoE as it considers expanding the Parent Circle program: o To inform expansion planning and budgeting, an assessment of factors including geography, participant and facilitator travel requirements, session duration, and personnel and materials needs must be carried out. o Employment of full-time Monitors should be considered in order to provide effective support to those participating in Parent Circles. o The regional Departments of Education should support an expanded Parent Circle program by assigning personnel to make routine supervisory visits to the Circles. o The Parent Teacher Association should be made aware of the program’s objectives, activities, and outcomes. The organization’s support of the program at the local level is critical for its success and sustainability. Component 4: Project Implementation Support, Administration and M&E • Sound data collection and analysis methods are critical for decision making. • Training personnel at the regional and sub-regional levels to input school-specific NDA results increased the efficiency and integrity of the data entry process. Sustainability of Project Achievements The MoE and regional Departments of Education have undertaken the following actions that demonstrate ownership and intention to sustain the positive strides made under GECEP: Page 32 of 37 The World Bank Guyana Early Childhood Education Project (P129555) • Many of the Master Trainers have been engaged by the MoE to lead the annual Continuing Professional Development courses provided to Nursery teachers in the regions, and to work as Nursery Field Officers. • Regional Education Development Officers have made budgetary allocations for the replenishment of toolkit items as required, while the MoE has procured ECE Resource Kits for all Nursery schools in the Coastal Regions. • The MoE has committed to producing additional Read-Play-Love print materials and DVDs for distribution to all Nursery schools in the Coastal Regions, and the Guyana Learning Channel will continue to air the infomercials developed under the campaign. Page 33 of 37 The World Bank Guyana Early Childhood Education Project (P129555) ANNEX 4. Highlights of Master Training Modules 1, 2 and 3 Highlights of Training Module 1, 2016: • Trainers reviewed the learning goals and practiced the steps of a “language experience approach� to teaching literacy. • Trainers explored phonemic awareness activities that can be taught over the course of a school day. • Trainers investigated “read aloud� strategies that support comprehension (making connections between text and self, asking questions before, during, and after reading, discussing images and new vocabulary, etc.). • Trainers explored connections and differences between pedagogy and andragogy (studying learning from adult and child perspectives, understanding environments that support learning success) • Trainers explored learning principles (readiness, practice, effect, primacy, recency, etc.). • Trainers explored oral language teaching strategies (storytelling, questioning, puppetry, etc.) by reviewing video and professional articles, and practicing teaching activities. • Trainers studied Nursery Diagnostic Assessment data and planned instructional strategies based on evaluation of NDA results. • Trainers explored the developmental stages of learning to read by studying video and reading professional articles. • Trainers explored the role of educators in supporting learning through play via music, story, interest center, tabletop and playground activities. • Trainers participated in a workshop on the components of the Literacy Hour. • Trainers each prepared a workshop plan for training in-service teachers in the field, including detailed teaching activities and a budget, and presented and enacted these plans with peers. Highlights of Training Module 2, 2017: • Trainers reviewed videos and professional articles on mathematical learning in children’s development. • Trainers investigated effective modeling of mathematical language. • Trainers investigated Barrier Games to understand how the effective use of vocabulary (related to number size, quantity, order, etc.) deepens children’s mathematical understanding. • Trainers learned and played a range of mathematical games to reinforce concepts including subitizing, number sequencing and linking quantity with symbols and words. • Trainers learned to use Hundred Boards to teach children about the patterns of a base ten number system (one more, one less, ten more, ten less, skip counting, etc.). • Trainers investigated graphing and surveying in early childhood (sorting, representing through people, bar and pictorial graphs, etc.) • Trainers reviewed a range of fiction and non-fiction picture books addressing mathematical concepts. • Trainers explored multiple ways to represent mathematical thinking through pictures, words, numbers, charts, graphs, and performance as a means of understanding that children take different routes to the same answer. Page 34 of 37 The World Bank Guyana Early Childhood Education Project (P129555) • Trainers practiced creating open-ended problem-solving questions from traditional math questions. • Trainers explored exercises that could be used to empower parents to support mathematical learning at home. • Trainers explored the mathematical learning potential of different classroom interest centers (shop, sand, water, music, books, etc.). • Trainers each prepared a workshop plan for training in-service teachers in the field, including detailed teaching activities and a budget, and presented and enacted these plans with peers. Highlights of Training Module 3, 2018: • Trainers participated in exercises to understand how reading comprehension takes place, i.e., the ways in which readers bring prior experiences to reading and engage in “meaning making� while reading. • Trainers undertook a series of open-ended writing tasks (“quick writes�, poetry prompts) to understand how to effectively support early writers. • Trainers examined all the ways in which children compose (through play, building, drawing, songs, etc.) to understand how to support multiple modes of composing. • Trainers studied children’s writing samples to practice generating strengths-based feedback and develop “next steps� lesson plans. • Trainers reviewed professional articles on reading and writing with young children to explore topics of teacher questioning, learning strategies and lesson planning based on learner assessment. • Trainers solved traditional and open-ended math problems to explore what it means to “teach for understanding�. • Trainers investigated how to move from praising and correcting learners to providing them with specific feedback related to what learners know, say and do. • Trainers reviewed professional articles on gender responsiveness to reflect on how learning environments can expand gender limits. • Trainers shared video and narrative documentation of the successes, adaptations and struggles they faced as they trained teachers in the regions during the two prior academic years. • Trainers investigated the writing and illustrating strategies of children’s authors to inform their selection of reading materials that will best support young readers and writers. • Trainers prepared and presented poems, songs and chants as a way to support local language acquisition and reinforcement. • Trainers each prepared a workshop plan for training in-service teachers in the field, including detailed teaching activities and a budget, and presented and enacted these plans with peers. Page 35 of 37 The World Bank Guyana Early Childhood Education Project (P129555) ANNEX 5. MoE’s Early Childhood Development Program Delivery Evaluation Checklist Observable Behavior Quality Indicators Rating for Overall Rating/ each Indicator Comments (scale of “1� to “4�) 1. Classroom encourages -Classroom setting is child-centered with interaction between teacher space for one-to-one, small group and large and children and among group activities children, and independent -Space layout and equipment support work learning opportunities (e.g.: all interest corners are readily accessible to children) -Classroom furniture is easily movable -Teacher’s desk is easily placed 2. Classroom setting -Industrious hum among students promotes learning -Children are busily engaged in their respective activities -Interest corners reflect children’s work -Books, reference materials are placed at eye level -Teaching aids are appropriate, attractive, and 3-D -Ready access to different types of learning resources 3. Teacher shows evidence -Completed weekly plan on hand of advanced planning -Evidence of planned cross-curricular integration -Materials are at hand during instructional time 4. Teacher has well-written -Current lesson plan format is being used lesson plan that reflects use -Goals for the lesson are clearly outlined of the integrated approach with links to the subject’s curriculum -Assessment strategies are outlined -Integration of literacy benchmarks and/or math attainment goals are evident regardless of topic 5. Objectives relate to -Activities children are engaged in match materials and activities lesson goals -Materials chosen directly to support the learning goal -Materials are readily available/accessible to children -Activities and materials promote self- directed learning -The available choice of activities directly supports the learning goal 6. Displays sound knowledge Can generate purposeful “teachable of the reading components moments/lessons� related to: -phonics instruction -oration and reading fluency -vocabulary development -reading comprehension Page 36 of 37 The World Bank Guyana Early Childhood Education Project (P129555) 7. Displays mastery of -Engages children in phoneme manipulation phonemic awareness (i.e., activities (singing, rhymes) to reinforce rhyme, alliteration, syllables, awareness of the letter sound compound words, breaking -Can repeat the letter sound clearly words apart) and -Helps children create word lists to practice phonological awareness the letter sound (letter/sound relationships -Supports children’s phonological spelling as they write 8. Engages children in read -Morning is divided into blocks of time for aloud, shared, guided and each reading type independent reading -Children are actively engaged and participating appropriately for each reading type -Uses workbooks provided from the RRS/ARS 9. Displays sound knowledge Can generate purposeful “teachable of the numeracy program moments/lessons� related to: -The 5-day Plan for numeracy - “Big ideas� related to numbers and operations, measurement and estimation, geometry, data management -Numeracy attainment goals 10. Engages children in play- -Creates/uses student-directed activities based activities to support using manipulatives to practice concepts at the goals for learning different stations -Uses materials from the ECE Resource Kit to support play-based learning -Models specific vocabulary when engaging with children 11.Displays sound knowledge -Uses cross-curricular planning/integrated of multi-grade teaching teaching to reinforce how to teach concepts using activities/ content from different subject areas -Children of different ability levels can experience the concept -Tailors activities to meet the needs of learners -Evidence of mixed ability and cross-grade grouping 12. Uses differentiated -Uses more than one approach for explaining instruction concepts -Alters teaching environment to group’s needs -Alters expected output/product to group’s needs -Open-ended questions used to demonstrate students’ understanding -Encourages children’s use of “home language� to communicate 13. Manages time for each -Complete whole group instruction and gives stage of the lesson children sufficient time to practice -Children are not left idle due to early task completion -Clear cycle of teaching, consolidation, assessment and reflection Note: A rating of “1� represents very weak evidence of the indicator, and “4� represent s strong evidence. Page 37 of 37