OBSERVER MANUAL OBSERVER MANUAL Second Edition (2021) © 2021 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW, Washington, DC 20433 Telephone: 202-473-1000; Internet: www.worldbank.org Some rights reserved. This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the information included in this work. Nothing herein shall constitute or be considered to be a limitation upon or waiver of the privileges and immunities of The World Bank, all of which are specifically reserved. Rights and Permissions This work is available under the Creative Commons Attribution 4.0 IGO license (CC BY 4.0 IGO) https://creativecommons.org/licenses/by/4.0/. Under the Creative Commons Attribution license, you are free to copy, distribute, transmit and adapt this work, including for commercial purposes, under the following conditions: Attribution – Please cite the work as follows: Molina, Ezequiel, Adelle Pushparatnam, Carolina Melo, Tracy Wilichowsk, Ana Teresa del Toro Mijares, Elaine Ding, Jenny Beth Aloys, Emma Carter, and Nidhi Singal (2021). Teach Primary (Second Edition). Washington, DC: The World Bank. License: Creative Commons Attribution CC BY 4.0 IGO. Translations – If you create a translation of this work, please add the following disclaimer along with the attribution: This translation was not created by The World Bank and should not be considered an official World Bank translation. The World Bank shall not be liable for any content or error in this translation. Adaptations – If you create an adaptation of this work, please add the following disclaimer along with the attribution: This is an adaptation of an original work by The World Bank. Views and opinions expressed in the adaptation are the sole responsibility of the author or authors of the adaptation and are not endorsed by The World Bank. Third-party content: The World Bank does not necessarily own each component of the content contained within the work. The World Bank therefore does not warrant that the use of any third party-owned individual component or part contained in the work will not infringe on the rights of those third parties. The risk of claims resulting from such infringement rests solely with you. If you wish to reuse a component of the work, it is your responsibility to determine whether permission is needed for that reuse and to obtain permission from the copyright owner. Examples of components can include, but are not limited to, tables, figures, or images. All queries on rights and licenses should be addressed to Teach, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; e-mail: teach@worldbank.org. Cover and interior design: Danielle Willis, Washington, DC, USA. TABLE OF CONTENTS FOREWORD ....................................................................................................... 2 ACKNOWLEDGMENTS ...................................................................................... 4 INTRODUCTION ................................................................................................. 5 PROCEDURES FOR CODING ........................................................................ 11 OBSERVER MANUAL ...................................................................................... 19 TIME ON TASK ............................................................................................. 22 TIME ON LEARNING .................................................................................................... 23 QUALITY OF TEACHING PRACTICES........................................................ 24 CLASSROOM CULTURE ............................................................................. 25 SUPPORTIVE LEARNING ENVIRONMENT .................................................................. 26 POSITIVE BEHAVIORAL EXPECTATIONS .................................................................. 28 INSTRUCTION .............................................................................................. 29 LESSON FACILITATION ............................................................................................... 30 CHECKS FOR UNDERSTANDING................................................................................ 31 FEEDBACK ................................................................................................................... 32 CRITICAL THINKING.................................................................................................... 33 SOCIOEMOTIONAL SKILLS ....................................................................... 35 AUTONOMY .................................................................................................................. 36 PERSEVERANCE ......................................................................................................... 37 SOCIAL & COLLABORATIVE SKILLS ........................................................................... 38 CHECKLIST: OTHER ASPECTS OF EDUCATIONAL QUALITY .................... 39 FREQUENTLY ASKED QUESTIONS............................................................... 44 1 FOREWORD School enrollment has increased substantially over the last 25 years in low-and middle-income countries. Schooling, however, does not guarantee learning. A large share of children complete primary school lacking even basic reading, writing, and arithmetic skills1—a state of affairs that has been2 dubbed the “global learning crisis.” Before the pandemic, the Learning Poverty rate in low- and middle-income countries was 53 percent—meaning that over half of all 10-year-old children couldn’t read and understand a simple text. COVID-19 has only deepened the learning crisis, and the impacts on human capital of this generation could be long-lasting. Estimates show that Learning Poverty is projected to rise to 70 percent due to the pandemic, and we are now facing a “crisis within a crisis.” The challenges brought on by COVID-19, in addition to the global learning crisis underway before the pandemic, require strengthening teachers’ capacities to teach well and meet the evolving challenges educational systems face today. As we look ahead to what must be done to recover learning losses, supporting teachers and high-quality teaching is more critical than ever so that students and schools can recover as quickly and effectively as possible. To this end, the World Bank has launched the Global Platform for Successful Teachers, to help countries enhance their teacher policies by following five key principles: 1) Making teaching attractive; 2) Improving pre-service education; 3) Improving selection, allocation and monitoring of teachers; 4) Providing high-quality professional development and school leadership; and 5) Helping teachers use technology wisely. Ensuring that every teacher around the world has access to high-quality professional development opportunities to improve their instructional practice is a key component of effective teacher policies; evidence shows, however, that many teachers around the world today do not have access to these opportunities.3 A first step to this goal is having reliable and valid data on current teaching practices, so that these insights can inform and shape the content and focus of teacher professional development programs and policies. Data on current teaching practices are particularly important because research has shown that a key driver of educational quality is the quality of the teacher-student interactions in the classroom. Also known as process quality, this refers to the way that teachers interact with their students in the classroom—including how they deliver instruction, how they instill socioemotional skills in their students, and how they create a classroom culture conducive to learning.4 Data on the quality of teacher-student interactions in the classroom are needed to improve teaching quality, to drive the policy dialogue on the importance of better supporting teachers and so that professional development can be appropriately tailored to teachers’ needs. Today, however, most education systems in low- and middle-income countries do not regularly monitor teaching practices or the quality of interactions between teachers and students in the classroom. Even when education systems attempt to capture teaching practices, most tools used in low- and middle-income countries fall short.5 In response to these challenges, the World Bank developed Teach Primary, an open access classroom observation tool. The Teach Primary classroom observation tool was developed to help countries measure teaching practices, inform policy dialogue, and ensure professional development programs are driven by and responsive to teachers’ needs. The tool forms part of the Teach suite of tools, which also now includes Teach ECE and Teach Secondary. Teach Primary measures the quality teacher-student interactions, focusing on techniques and behaviors known to nurture children’s cognitive and socioemotional skills. Teach Primary holistically measures what happens in the classroom. It does so by considering both time spent on learning as well as the quality of teaching practices. Teach Primary was developed to be adaptable to different contexts and includes the use of local video footage to train observers, which ensures that the tool is contextualized and anchored in the local setting. Teach Primary is open access and includes a set of complementary resources to support in every step of the tool’s implementation, including initial conversations with relevant 2 stakeholders, training observers, using the tool to collect data in the field, cleaning and analyzing the data, and producing and sharing results. Before the tool was launched in 2019, Teach Primary underwent a rigorous development and validation process over a two-year timeframe. A Technical Advisory Panel6 provided extensive feedback and inputs on the tool’s design. The tool was also piloted in over 1,000 classrooms across Mozambique, Pakistan, the Philippines, and Uruguay, and tested with global video footage from 11 low- and middle- income countries. Since its launch, Teach Primary has been used to support the way countries track and improve teaching practices, adapted to each country’s context and needs. As of December 2021, we estimate that Teach has been or is being implemented in over 42,500 schools worldwide, involving almost 180,000 teachers, more than 3.6 million students, and 25 organizations. In Mozambique, for example, Teach was used to capture teaching quality across the country, and this data has informed the development of educational interventions focused on supporting teachers to improve student learning. In Guyana, Teach is being leveraged as an M&E tool to assess the effectiveness of a new curriculum at the primary level, by capturing teaching practices before and after rolling out the curriculum. And in Punjab, Pakistan, an adapted version of the tool is being used by pedagogical leaders to conduct up to 30,000 classroom observations per week and provide tailored and personalized feedback to teachers to help them improve their teaching This cycle of observation and feedback has yielded a 20 percent rise in average teaching scores as tracked by the classroom observations across a period of two years.7 The use of the tool to date, and its impact in promoting better teacher policies that support improved educational quality, show the importance of having validated, reliable, and accessible tools to track teaching quality. As we look ahead to the work that must be done to accelerate learning recovery post-COVID-19, it is more important than ever that we support quality learning for all, especially the students that face the most disadvantages. In line with the World Bank’s commitment to accelerate global action on disability-inclusive development and to ensure that all education projects and programs financed be disability-inclusive by 2025, the tool underwent an important revision process in 2020–2021 to strengthen the way it was capturing inclusive teaching practices. As part of this process, a Technical Advisory Board of experts in inclusive education provided feedback on the tool, and the revised tool was validated using a library of global videos.8 These changes are reflected in the Second Edition of the Teach Primary tool presented in this Observer Manual, together with user feedback from the first two years of using the tool in the field. Teach Primary brings us closer to ensuring that every student has a skilled, supported, and motivated teacher, a crucial precondition to achieve learning for all. We hope that this Second Edition of the Teach Primary tool, used in conjunction with initiatives to improve teacher professional development such as Coach, will continue to support education systems in tracking teaching practices in primary education classrooms and help countries better support teachers, ultimately contributing to efforts to address the global learning crisis and promote educational quality for all children —in the learning recovery period and beyond. Omar Arias Practice Manager, Global Knowledge and Innovation Team 3 ACKNOWLEDGMENTS The Teach Team is led by Ezequiel Molina and Adelle Pushparatnam. The first edition of Teach Primary was developed by a core team that also included Carolina Melo Hurtado and Tracy Wilichowski. The second edition of Teach Primary was developed by a team that also included Jenny Beth Aloys, Emma Carter, Ana Teresa del Toro Mijares, Elaine Ding and Nidhi Singal. Diego Luna-Bazaldua, Carolina Moreira Vásquez, Gabrielle Arenge, Gill Althia Francis and Maria Tsapalis were members of the extended team. The Observer Manual and Tool were designed by Danielle Willis. Amy Gautam was the chief copy editor. Patrick Biribonwa, Restituto Jr. Mijares Cardenas and Cassia Miranda provided administrative support to the project. The team received guidance on the first edition of Teach Primary from a Technical Advisory Board composed of Lindsay Brown, Pam Grossman, Heather Hill, Andrew Ho, Sara Rimm-Kaufman, Andrew Ragatz, Erica Woolway, and Nick Yoder. The team is also grateful for the contributions of the Teach Classroom Observation Working Group that collaborated on the first edition of the Teach Primary tool and its complementary resources, composed of Salman Asim, Tara Beteille, Marguerite Clarke, Michael Crawford, David Evans, Deon Filmer, Francisco Haimovich, Samira Halabi, Amer Hassan, Peter Holland, Dingyong Hou, Nathalie Lahire, Toby Linden, Javier Luque, Juan Manuel Moreno, Shawn Powers, Halsey Rogers, Shwetlena Sabarwal, Shabnam Sinha, Lars Sondergaard, Simon Thacker, Waly Wane, and Noah Yarrow. The team received guidance on the second edition of Teach Primary from an Inclusion Advisory Panel composed of Jo Westbrook, Rabea Malik, and Joshua Josa. The team is also grateful to Hanna Alasuutari, Jennae Bulat, Brent Elder, Amer Hasan, Anne Hayes, Huma Kidwai, Kimberly Ann Korotkov, Rebecca Rhodes, Deepti Samant Raja, Ruchi Singh, Elena Soukakou, and Charlotte Vuyiswa McClain-Nhlapo, who provided useful comments and feedback on the revisions to the second edition of the Teach Primary tool. The team also benefited from the support of Hafsa Alvi, Tamara Arnold, Estefanía Avendaño, Jennifer Bulley, Yanina Gallo, Julia Hahn, Aakriti Kalra, Julia Ladics, Anahita Matin, Abdal Mufti, Octavio Medina Pedreira, Mahjabeen Raza, Hina Saleem, Marie Evane Tamagnan, and Sergio Venegas Marin in the development of the tool. A number of colleagues provided insightful comments, feedback, and inputs throughout the tool’s development, including Gonzalo Dibot, Guadalupe Goyeneche, Michael Handel, Amer Hassan, Ines Kudo, Victoria Levin, Alonso Sanchez, Virginia Tort Gómez, Paula Prendeville, Elina Rostan, Kirill Vasiliev, and Noah Yarrow. In addition, the team is grateful to the teams that applied a preliminary version of Teach in their various contexts. This included Franco Russo, Binh Thanh Vu, and Javier Luque in the Philippines; Koen Martijn Geven, Tazeen Fasih, and Ali Ansari in Pakistan; Francisco Haimovich Paz and Helena Rovner in Uruguay; Marina Bassi in Mozambique; and Sara Rimm-Kaufman at the University of Virginia. Overall guidance for the development and preparation of Teach Primary was provided by Omar Arias, Practice Manager for the Global Knowledge and Innovation Team. The team acknowledges support from the Global Leads of the Curriculum, Instruction and Assessment Thematic Group, the Teachers Career and Professional Development Thematic Group and the Inclusive Education Thematic Group for their guidance and advice throughout the process. The team is especially grateful to Jaime Saavedra, Senior Director of the Education Global Practice, for his leadership, direction, and relentless support. The team is grateful for generous support provided by the Systems Approach for Better Education (SABER) Trust Fund, funded in large part by the United Kingdom’s Department for International Development (DFID) and Australia’s Department of Foreign Affairs and Trade (DFAT), that supported the development of Teach Primary. The team is also grateful to the Inclusive Education Initiative at the World Bank, which supported the review and revision of the Teach Primary tool in 2020-2021. The team apologizes to anyone inadvertently omitted from this list and expresses its gratitude to all who contributed to Teach Primary, including those whose names may not appear here. Lastly and most importantly, the team members would like to thank all the teachers who have welcomed us into their classroom as part of this project. HAVE QUESTIONS? Contact us at teach@worldbank.org. 4 INTRODUCTION 5 What is Teach? Teach is an open access classroom observation tool that provides a window into one of the less explored and most important aspects of student learning: what goes on in the classroom. The tool has been designed to help countries track and improve teaching quality. This Observer Manual focuses on the Teach Primary tool (for grades 1–6). Teach is also available for the early childhood education context through Teach ECE. Teach Secondary and Teach Remote are currently under development. How does Teach Primary (Second Edition) differ from the First Edition? Teach Primary was first released in 2019. In 2020–21, Teach Primary underwent a review process to strengthen the way the tool captures inclusive teaching practices. This Second Edition includes adjustments made through the revision process. Importantly this version of the tool includes the addition of a new behavior (Behavior 1.4b) focused on capturing disability bias, as well as adjustments to the Low, Medium, and High codes for other behaviors. This revised version also includes revised examples across the tool’s areas that better reflect inclusive teaching practices. Finally, this version includes the addition of a checklist to be used together with the classroom observation tool. The checklist focuses on assessing aspects of the learning environment related to educational quality and inclusion, including the accessibility of the physical environment and some aspects of the classroom setup and materials available, which are important elements to guarantee an inclusive and high-quality educational experience for all students. Why is it important to measure teaching practices? A growing body of research indicates teaching is the most important school-based determinant of student learning, and the difference between the impact of a weak teacher or a great teacher on student test scores is equivalent to one to two years of schooling. Moreover, evidence suggests several consecutive years of effective teaching can offset learning shortfalls and help students reach their full potential (Bau and Das 2017; Buhl-Wiggers et al. 2017; Hanushek and Rivkin 2010; Nye, Konstantopoulos, and Hedges 2004; Snilstveit et al. 2016). Evidence shows, however, that many teachers today do not receive the support they need to be effective in the classroom (Popova et al. 2018). Teachers require ongoing feedback, practice, and support to improve their teaching, and it is essential that all teachers around the world have access to continuous and high- quality learning opportunities to improve their practice. The first step toward offering better support to teachers so they can improve their teaching is to measure current teaching practices. Teach Primary was developed with this goal in mind. How can Teach Primary be used? Teach Primary can be used for different purposes depending on the country context and project goals. Teach Primary can be used as a system diagnostic, allowing governments to get a clear snapshot of the current state of teaching practices and teaching quality in classrooms. In this capacity, Teach Primary can be leveraged as a monitoring and evaluation (M&E) tool to assess the results of a specific education policy or program that targets teacher practices, such as the deployment of a new curriculum or a new instructional model. Teach Primary can also be used as part of a teacher professional development system to identify individual teachers’ strengths and weaknesses and to provide targeted support to teachers. Teach Primary was not developed to be used in high-stakes teacher evaluation or decision-making processes. 6 What does Teach Primary measure? Teach Primary differs from other classroom observation tools in that it captures: • The time teachers spend on learning and the extent to which students are on task, • The quality of teaching practices that help develop students’ socioemotional and cognitive skills, and • Other aspects of the learning environment such as the accessibility of the physical environment, including the classroom set-up and materials available. As part of the Time on Task component, three “snapshots” of 1–10 seconds are used to record both the teacher’s actions and the number of students who are on task throughout the observation. The Quality of Teaching Practices component is organized into three primary areas: Classroom Culture, Instruction, and Socioemotional Skills9 (see graphic on page 7). These areas have 9 corresponding elements that point to 28 behaviors. The behaviors are characterized as low, medium, or high, based on the evidence collected during the observation. These behavior scores are translated into a 5-point scale that quantifies teaching practices as captured in a series of two, 15-minute lesson observations. CLASSROOM CULTURE: The teacher creates a culture that is conducive to learning. The focus here 1 is not on the teacher correcting students’ negative behaviors but rather the extent to w hich the teacher creates: (i) a supportive learning environment by treating all students respectfully, consistently using positive language, responding to students’ needs, and both challenging stereotypes and not exhibiting bias in the classroom; and (ii) positive behavioral expectations by setting clear behavioral expectations, acknowledging positive student behavior, and effectively redirecting misbehavior. INSTRUCTION: The teacher instructs in a way that deepens student understanding and encourages 2 critical thinking and analysis. The focus here is not on content-specific methods of instruction, but rather the extent to which the teacher: (i) facilitates the lesson by explicitly articulating lesson objectives that are aligned to the learning activity, explaining content using multiple forms of representation, and connecting the learning activity to other content knowledge or students’ daily lives, and by modeling the learning activity through enacting or thinking aloud; (ii) does not simply move from one topic to the next but checks for understanding by using questions, prompts, or other strategies to determine students’ level of understanding, by monitoring students during group and independent work, and by adjusting his/her teaching to the level of students; (iii) gives feedback by providing specific comments or prompts to help clarify students’ misunderstandings or identify their successes; and (iv) encourages students to think critically by asking open-ended questions and providing students with thinking tasks that require them to actively analyze content. Students exhibit critical thinking ability by asking open-ended questions or performing thinking tasks. SOCIOEMOTIONAL SKILLS: The teacher fosters socioemotional skills that encourage students to 3 succeed both inside and outside the classroom. To develop students’ social and emotional skills, the teacher: (i) instills autonomy by providing students with opportunities to make choices and take on meaningful roles in the classroom. Students exhibit their autonomy by volunteering to participate in classroom activities; (ii) promotes perseverance by acknowledging students’ efforts, rather than focusing solely on their intelligence or natural abilities, by having a positive attitude toward students’ challenges by framing failure and frustrations as part of the learning process, and by encouraging students to set short-and long-term goals; and (iii) fosters social and collaborative skills by encouraging collaboration through peer interaction and by promoting interpersonal skills, such as perspective taking, empathizing, emotion regulation, and social problem solving. Students exhibit social and collaborative skills by collaborating with one another through peer interaction. Finally, Teach Primary is accompanied by a checklist to assess other aspects of the learning environment related to educational quality and inclusion, including the accessibility of the physical environment and some aspects of the classroom setup and materials available, which can be used together with the classroom observation components. 7 8 TEACH PRIMARY FRAMEWORK How was Teach Primary developed? To finalize a working version of the tool, the Teach Primary development team rigorously researched, revised, and piloted different iterations of the tool over a 2-year timeframe, and released a first edition of the tool in 2019. In 2020-2021, the Teach Primary tool underwent a review and revision process to strengthen the way it captures inclusive teaching practices. The following describes the step-by-step process that the tool has undergone for its development: The initial development team—which comprised 1 education measurement expert, 1 instructional 1 expert,1 psychologist, and 1 teacher—assessed 5 classroom observation tools widely used in the United States to create an inventory of teaching practices that are commonly evaluated. 10 The team then built upon this list to include behaviors from international classroom observation tools used in low- and middle-income countries.11 Based on this preliminary analysis, the team created an inventory of 3 areas and 43 elements.12 2 The development team hosted a working group of 22 education experts and practitioners to help further reduce and prioritize elements for the Teach Primary framework. Participants were asked to indicate whether any elements were missing from the inventory, to rank the elements and areas by relevance, and to identify elements they characterized as unobservable. This process resulted in a downsized framework of 25 elements. 3 The development team reviewed the theoretical and empirical evidence from low- and middle- income countries to further eliminate elements from the framework. This process resulted in a downsized framework of 14 elements. 4 These 14 elements comprised the first working version of the tool, which aimed to capture both quality and frequency of teaching practices as measured by each element.13 This preliminary tool was piloted in person in Pakistan and Uruguay and through classroom video footage in Afghanistan, China, Pakistan, the Philippines, Tanzania, Uruguay, and Vietnam. From these pilots, it became apparent that observers struggled to code reliably when they had to simultaneously capture the frequency and quality of teaching practices for each element. In response, the development team revised the structure of the tool to address this challenge. This process resulted in a tool comprised of 10 elements. 5 The development team convened a technical advisory panel, including Lindsay Brown, Pam Grossman, Heather Hill, Andrew Ragatz, Sara Rimm-Kaufman, Erica Woolway, and Nick Yoder, to provide written feedback on the tool. These comments were compiled and addressed as part of a 1-day technical workshop. During the workshop, the experts advised the team on which issues to prioritize and how to incorporate the comments to further improve the tool. 6 This updated version of the tool was applied in four settings, where observers were given a certification exam that ensured they could reliably code using Teach Primary. In Mozambique, 74 percent of the observers passed the certification exam; in Pakistan and the Philippines, 96 percent passed; and in Uruguay, 100 percent passed. The observers also provided comments on the tool and training that was considered during the revision process. The development team worked closely with Andrew Ho14 to analyze the psychometric properties of 7 the tool. Using the data from Punjab, Pakistan the team found those teachers who exhibit better teaching practices, as measured by Teach Primary, are associated with an additional 0.068– 0.124 SD increase in student test scores. This is after controlling for a host of variables, including class size, teacher content knowledge, and other student and teacher characteristics. Based on this analysis and feedback from the trainers and observers, the development team revised each element’s structure and complementary examples to improve the tool’s consistency and clarity. As part of this process, the Time on Learning element was modified to capture teachers’ time on instruction and students’ time on task through a series of snapshots. This process resulted in a tool that comprised 1 low inference and 9 high-inference elements. The final stage involved 9 testing these revisions using video footage of 11 low- and middle-income countries from the Teach Primary video library. The tool was released to the public in 2019. 8 Since its release to the public and as of December 2021, Teach Primary has been applied in thirty countries. Teach Primary has been implemented by the World Bank and by external organizations, including J-PAL, IDinsight, IRC, Save the Children, and Education World Trust, as well as individual schools. Through these different implementations, the Teach Primary team has gained insights about the tool’s use in the field and has adjusted and made revisions to the tool and its complementary materials in order to better support implementations in the field. The Teach Primary team has also conducted additional analyses and studies to verify the psychometric properties of the tool (Luna-Bazaldua, Molina, and Pushparatnam, 2021). In 2020 and 2021, Teach Primary underwent a revision process to strengthen the way the tool 9 measured inclusive teaching practices. Inclusive teaching practices are defined as those that create increased opportunities for all students to access learning. The revised version of the tool (Second Edition) reflects some important adjustments from the original version. A group of experts in inclusive education provided extensive feedback on the proposed revisions to the tool, and these comments were incorporated into a revised version of the tool. 10 The Second Edition Teach Primary tool was piloted with a set of 10 videos from a global video library. In addition, the tool was also piloted in Rwanda. These codes were compared to assess that the revised tool was better capturing inclusive teaching practices. The revised Second Edition of the Teach Primary Observer Manual and Tool was published in 2021. 10 PROCEDURES FOR CODING 11 Protocol Before, during, and after an observation, observers should be cognizant and respectful of the school environment by following this protocol: BEFORE DURING AFTER SUPPLIES: SETUP: CONCLUSION: Ensure you have the manual, Sit toward the back of the Thank the teacher for being able to observation packet, a classroom to view the entire conduct the observation. pencil/pen, consent forms,15 and classroom; ensure your presence When the second observation a watch/phone. does not block students’ view of concludes, remain silently in the ARRIVAL: the lesson. classroom, finish coding, and Introduce yourself to the If visiting a classroom with remain in the classroom without principal and arrive at the another observer, seat yourself creating distractions. designated classroom at least separately and refrain from DISCRETION: 10 minutes before class begins. talking with him/ her at any point Avoid discussing any of the scores during the lesson. Introduce yourself to the with the teacher. If a teacher asks teacher, explain the purpose of Ensure your cellphone is muted how s/he performed, politely remind the visit, and remind the and abstain from texting, phone him/her this is not a performance teacher of the confidential calls, Facebook/Twitter, taking evaluation. For example: nature of the observation: pictures, or any other distracting activities. “The goal of the observation was to “Good morning, Mr./Ms. learn about teaching practices; the [teacher’s surname], I work with OBSERVATION: notes from this observation will be [affiliate organization]. Your Begin the observation when used as part of a larger study on school has been randomly class is scheduled to begin; if the teaching practices in [district/city selected to partake in a survey teacher is delayed, wait until s/he name]. I very much enjoyed that includes classroom arrives and make note of the watching your lesson and observations. The purpose of time on the observation sheet. appreciate you allowing me into the survey is to learn about your classroom.” teaching practices in In the case of multigrade [district/city name]. As such, I’m classrooms, treat the observation Refrain from discussing the here to simply learn from you— as one grade and document it on classroom scores with anyone. You these observations will not be the observation sheet. can provide your supervisor’s used for evaluative purposes, number if the teacher insists. NO INTERACTION: and your identity will remain Refrain from discussing what entirely confidential. Please Avoid engaging with or distracting the students or occurred during the lesson in a proceed with the lesson as you joking or disrespectful way. normally would.” teacher and do not participate in classroom activities, even if This may affect your credibility as DISSENT: explicitly asked. an observer. If a teacher does not want to be Do not check students’ observed, kindly remind him/her CHECKLIST (If applicable): textbooks, worksheets, Complete the “Fill in after the that the observation is not an notebooks, or other classwork. evaluation, his/her identity will be observation” section. kept anonymous, and no Avoid positive or negative With the assistance of the teacher, information will be shared with nonverbal expressions and ask necessary questions such as school authorities. Kindly note, a convey a neutral attitude to avoid “how many students have a pencil” teacher cannot be forced to be inadvertently distracting the and count. If you finish coding the observed; if the teacher teacher. second observation before the continues to decline consent, Redirect the teacher and class is over you may begin other exit the classroom and students to the lesson if they ask aspects of the checklist provided document what happened on the questions or focus their attention you are silent and not a distraction observation sheet. on your presence. (standing up, moving around the CHECKLIST (If applicable): classroom, etc.). Complete the “Fill in before the classroom observation” section. Inform the teacher that you will have a checklist to fill out when class ends. Ask him/her to notify the students they must stay after class and follow your instructions. 12 Length of the observation Observations should be divided into two 15-minute segments.16 The first observation segment begins at the scheduled class time; however, if the teacher or students are not present during the scheduled class time or the lesson is delayed, the observation begins when the teacher enters the classroom. After each 15-minute observation, observers should spend 10–15 minutes scoring the observation, depending on the length of the class. For example, in a 45-minute class, the first observation segment begins at the scheduled class time and is 15 minutes in length. The observer then stops (even though class is still going) and spends the next 15 minutes scoring segment 1. The observer then spends the remaining 15 minutes of class observing segment 2. After the class has concluded, the observer spends another 15 minutes scoring segment 2. Observers should always record the length of each observation segment on the scoring sheet. If the lesson ends before the predetermined length of the observation, observers should still code the segment. It is important to accurately record the information on segment length, delayed starts, and early finishes, as this will be used in data analysis. Note-taking Once the observation begins, the observer uses the note-taking form to document what the teacher says by noting specific behaviors, questions, instructions, and actions. These notes are essential to code objectively and reliably since they provide evidence for the chosen scores. When note-taking, it is important to be as descriptive as possible. Observers will use their notes and compare them with the descriptions in the manual to determine the behavior quality ranges and assign an overall combined score for each element. As soon as observers finish an observation, every score should be justified with evidence from the observation. When note-taking, it is important to look for specific student and teacher behaviors that are clearly included in the tool. All observers should create a note-taking system that works for them; below are some helpful note-taking techniques.17 TECHNIQUE WHAT IS OBSERVED WHAT IS WRITTEN SCRIPTING: After a lesson on forming past tense sentences, the T: Who can take an action quotes by teachers teacher asks students to relate the current lesson to a verb from yesterday and (T) or students (S) previous one on action verbs by forming a sentence create a past tense using both strategies. She asks, “Who can take an sentence? action verb from yesterday and create a past tense S: Amna jumped over the sentence?” A student raises her hand and responds, puddle. “Amna jumped over the puddle.” TALLIES: Throughout the lesson, the teacher says “very good” 8 “Very good” shortcuts for times in response to student participation and answers. frequently used words or phrases SHORTHAND: The teacher reviews a student’s paragraph and FB- T: opening paragraph specific symbols or provides feedback by saying, “Great job on the first is compelling b/c of letters to represent paragraph. The way you open with a personal story is personal story behaviors very compelling.” ANECDOTES: At the start of an activity, the teacher asks if everyone 6 Ss no book, T cont. summaries of what has a textbook. Six students raise their hands to teaching at board, 3 Ss was seen or heard indicate they do not. The teacher continues teaching at playing (disruptive). the board. Meanwhile, three students are playing with a ball of paper and distracting others. 13 The Teach Primary tool is composed of the Observer Manual and the Observation Sheet; observers should actively use and read the manual to determine scores. Measuring Time on Task For the Time on Learning element, observers will take three “snapshots,” or 1–10 second scans of the classroom, and use only information gathered during the snapshot to code the behaviors. For the first behavior, observers will record whether the teacher is providing a learning activity for most students by indicating “no” if s/he is not providing a learning activity and “yes” if s/he is. If the teacher is providing a learning activity, scan the classroom from left to right to determine whether students are on task. If 0 or 1 student is off task, score the second behavior high (H). If 2 to 5 students are off task, score it medium (M). If 6 or more students are off task, score it a low (L). If the teacher does not provide a learning activity for most students, record a “not applicable” (N/A) for the second behavior and continue coding the other elements of the tool. See Page 23 for more details on the snapshot method and how to code this element. Measuring Quality of Teaching Practices (i) Assigning quality ranges to each behavior To assign the most objective score, the manual describes each behavior in three quality ranges: low, medium, and high. These are detailed descriptions and include examples that help observers decide which quality score best applies to each element. After the first observation segment concludes, the observer assigns a “low, medium, or high” rating to each behavior. For this, it is necessary to read the notes and compare them with the descriptions provided in the manual. It is very important for observers to adhere to the manual as closely as possible, whether or not they agree with it. This symbol signifies that the given behavior has a corresponding FAQ; observers should thoroughly familiarize themselves with the FAQs prior to carrying out observations and should refer to the FAQs while coding to help clarify any confusion. It is very important that observers give one score for every behavior. If observers want to change an answer, they must clearly remove the invalid score by fully erasing or striking through it. Some behaviors may not be observed. For those behaviors, the manual provides the option to write “N/A.” Observers can only score “N/A” if presented the option on the scoring sheet (0.2, 1.3, 4.2). If a behavior is scored “N/A,” this behavior should not influence the overall score for the corresponding element. The following example shows what this would look like in practice: 14 (ii) Assigning scores to each element After assigning quality ranges to the behaviors, the element scores should be decided according to the overall quality of each element. Scores range from 1 to 5, with 1 being the lowest score and 5 the highest. It is necessary to carefully read the descriptions for the different behavior levels and assign an element score that best describes the observed scenario in the classroom. While the final score should follow the calculated scores from the behaviors, observers should always go back and reread the element description and its corresponding behaviors to determine if the score fits the overall description of the element. For example, observers may score an element a 4 even if it contains high, medium, and low behavior scores if what was observed exceeds the overall medium description but does not constitute a high description. The final score need not be a mathematical calculation and the score should reflect the evidence presented in the entire segment. 4 H L M (iii) Assigning scores for behavior 1.4 After assigning a “low, medium, or high” quality rating for the sub-behaviors 1.4a and 1.4b separately, an overall quality rating can then be decided for behavior 1.4. In determining this overall quality rating, the following guidelines for different sub-behavior rating combinations should be adhered to: If both 1.4a and 1.4b are assigned the same quality rating, then this rating would constitute the overall quality rating for the behavior. For example, if 1.4a and 1.4b are both assigned “high” then the overall quality rating for behavior 1.4 would remain “high.” L L L M M M H H H If “low” is assigned to either 1.4a or 1.4b then the overall rating for the behavior would remain “low”, irrespective of the combination. For example, if 1.4b was rated “low” then this rating would take precedence in deciding the overall score, even if the rating for 1.4a was “medium” or “high.” L L M L L H If one sub-behavior is rated “high” and the other “medium,” “high” would take precedence. For example, if 1.4a was rated “high” and 1.4b was rated “medium” then the overall behavior score for 1.4 would be “high.” M H H Common challenges in classroom observations Before coding with a classroom observation tool, it is crucial to understand the importance of inter-rater reliability, which describes the degree to which observers agree on the scores associated with a specific observation. For example, an observation is reliable if two observers use the tool to observe the same teacher and arrive at the same (or nearly the same) scores. Observers should be aware of several challenges when conducting classroom observations that have the potential to adversely affect objectivity and reliability when using the tool: 15 Personal Experiences In some cases, past experiences and personal opinions influence how observers score the rubric. This is particularly problematic for people who have pr eexisting notions of what constitutes “good teaching.” Moreover, their exposure to different teaching styles has the potential to influence their reliability. For example, some observers may think, “When I went to school, this is how we learned” or “My daughter’s teacher does this.” Despite this prior knowledge, it is important to remember codes must be based solely on the manual, regardless of opinion or experience. Additional Information In some cases, observers adjust their scores based on additional or preexisting information they have on the teacher, school, or students. Sometimes, they also assume certain behaviors by incorrectly inferring the teacher’s intentions. For example, “I am going to give the teacher a 5 for positive environment, because even though she was impatient with the student, I know it is because she worked a double-shift today.” This additional information should not influence the scoring of the observation as codes should solely reflect what happens in the classroom during the allocated observation time. Comparison Often, observers conduct several observations during a short timeframe and compare teaching styles and ability across observations—this ultimately hinders the reliability of the tool. For example, an observer may rate a teacher lower on a behavior because in an earlier observation, s/he saw the same teacher, or a different teacher, use a better strategy to communicate the same information. It is necessary to observe each segment independently and avoid comparison to other situations or teachers to maintain reliability. Separation of Elements In some cases, separating the content of the elements can feel forced since everything that happens in the classroom is interconnected; that is, observers may strongly feel an action falls under more than one element. One observed action can serve as evidence for more than one Teach Primary behavior or element, but the scoring of each must be done independently. For example, a teacher may provide feedback during the lesson, so students reflect on their mistakes. This feedback may encourage students to think critically; however, this does not mean the teacher automatically scores high on the critical thinking element, since other behaviors in the critical thinking element may be absent. In this case, observers should keep both elements separate and score them independently. Weighing of Specific Events or First Impressions In some cases, observers may witness a situation that surprises them or triggers a negative or positive impression. This incident may influence how they assess the entire observation. To maintain reliability, it is important to consider the event in the broader context of the observation and not let first impressions or salient events disproportionately influence the overall score. Therefore, observers should write detailed notes of the observation to determine how much weight to give a specific event. Additionally, each segment should be considered in-and-of itself, and observers should focus on what occurs in the current segment. For instance, even if the teacher intends to do an activity later in the class, it is important for observers to only score what actually happens in that segment, rather than boosting the score of one of the behaviors based on an intention that never occurred. This is particularly applicable for distinguishing what occurs in segment 1 versus segment 2 (e.g., what is observed in segment 1 may not be considered for scoring in segment 2, and vice versa). Central Tendency In some cases, observers assign medium-level scores more often than they should. This reluctance to assign high or low scores occurs (i) when observers are not confident in their ability to identify the appropriate level, or believe that high or low scores are very rare and are largely unattainable; or (ii) due to the fear (for themselves or the teacher) of assigning more extreme scores. It is important observers score the behaviors exactly as defined in the manual without being influenced by how the scores may be used or how they reflect on the observer or the teacher. 16 Observer certification and Reliability Exam A training participant must pass the Teach Primary Reliability Exam before becoming a certified reliable Teach Primary observer. Observer certification provides quality control and increases reliability of the Teach Primary tool across observers. It ensures all certified observers can use the tool to accurately and consistently score classroom observations in accordance with the Teach Primary scale. The Teach Primary Reliability Exam consists of watching and scoring three, 15-minute video segments and scoring them according to the Teach Primary rubric. Participants have 15 minutes to code each segment and cannot stop, rewind, or rewatch the videos during the exam. To pass the exam, participants must be reliable on 8 of the 10 elements for each segment. For example, if an observer scores 100 percent on the first segment, 100 percent on the second segment, and 70 percent on the third segment, s/he would not pass the exam. For the Time on Learning element, participants are considered reliable if they are in exact agreement with the master score for 2 out of the 3 snapshots. For all the other elements, participants are considered reliable if they score within 1 point of the master score. Participants who do not pass on the first attempt will be given feedback and allowed one additional opportunity to pass the exam. The second exam will consist of three different videos. Participants who do not pass the second attempt will not be certified as Teach Primary observers. Teach Primary certification is valid for one year. 17 Endnotes 1 World Bank (2018). 2 UNESCO (2013). 3 Popova et al. (2018). 4 For more on the importance of process quality, please see Curby, Brock, & Hamre, 2013; Hatfield, Hestenes, Kintner-Duffy, & O’Brien, 2013; Kane, Taylor, Tyler, & Wooten, 2011; and Muijs et al., 2014. 5 Ladics et al. (2018). 6 The team received guidance in the development of the first edition of the tool from a technical advisory panel composed of Lindsay Brown, Pam Grossman, Heather Hill, Andrew Ho, Sara Rimm-Kaufman, Andrew Ragatz, Erica Woolway, and Nick Yoder. For more on the development of the first edition of the tool, please see “Development and Validation”. 7 Computed using administrative data provided by the Punjab Programme Monitoring and Implementation Unit (PMIU) through the Punjab Integrated Education Dashboard and the AEO Classroom Observation Tool Dashboard. For more information on the use of the modified Teach tool in Punjab, Pakistan, as well as in other contexts, please consult the Teach in Action: Three cases of Teach implementation to date brochure available on the Teach Primary website. 8 The team received guidance in the development of the second edition of the tool from a Technical Advisory Board composed of Jo Westbrook (Senior Lecturer in Education, University of Sussex), Rabea Malik (CEO and Research Fellow, IDEAS Pakistan) and Joshua Josa (Quality, Equity and Sustainability Team Lead, USAID). For more on the development of the second edition of the tool, please see “Development and Validation”. 9 It should be noted that it is impossible to draw a clear line between teaching practices linked to academic versus socioemotional learning. Many teaching practices included in common professional teaching frameworks do impact students’ socioemotional development, but are usually thought of in terms of academic rather than socioemotional learning. Explicitly linking teaching practices with socioemotional outcomes in measures used for assessment will serve to increase the salience of students’ socioemotional skills to teachers, as well as to other stakeholders and policy makers, thus ensuring a focus on both academic and socioemotional learning in the classroom. 10 The Teach framework built upon the inventory created by Gill and others (2016), who conducted a content analysis of the differences in dimensions of instructional practice of five commonly used classroom observation tools comparing the behaviors they measure with the extent to which they predict student learning. The tools included CLASS, FFT, PLATO, Mathematical Quality of Instruction, and UTeach Observational Protocol. The content, predictive power, and potential bias of these instruments were also analyzed as part of this preliminary framework (Gill, Brian, Megan Shoji, Thomas Coen, and Kate Place. 2016. “The Content, Predictive Power, and Potential Bias in Five Widely Used Teacher Observation Instruments.” National Center for Education Evaluation and Regional Assistance, Washington, DC. 11 These included OPERA, SCOPE, SDI, Stallings, and TIPPS. 12 Elements refer to groups of multiple, similar behaviors that aim to capture teaching practices related to positive learning outcomes. 13 For example, the tool aimed to capture not just the quality with which a teacher checked for understanding (adjusting the lesson, prompting students to determine their level of understanding, etc.), but the frequency with which the teacher checked for understanding in each lesson. 14 Andrew Ho is Professor of Education at the Harvard Graduate School of Education. He is a psychometrician whose research aims to improve the design, use, and interpretation of test scores in education policy and practice. 15 Protocol to enter the classroom may vary from context to context; however, it is important to have the necessary approvals in place before arriving at the school. 16 These times may differ slightly from context to context. 17 Adapted from Archer, Jeff, et al. 2016. “Better Feedback for Better Teaching: A Practical Guide to Improving Classroom Observations.” San Francisco, CA: Jossey-Bass. 18 OBSERVER MANUAL 19 OBSERVATION SHEET SCHOOL ID: TEACHER ID: CODER ID: GRADE: SUBJECT: SEGMENT 1 CLASS SIZE: girls _____ boys _____ SCHEDULED TIME: __ __ : __ __ to __ __ : __ __ ACTUAL TIME: __ __ : __ __ to __ __ : __ __ SEGMENT LENGTH: _____ min TIME ON TASK 0. TIME ON LEARNING 1st Snapshot (4m) 2nd Snapshot (9m) 3rd Snapshot (14m) 0.1 Teacher provides learning activity to most students Y N Y N Y N 0.2 Students are on task N/A L M H N/A L M H N/A L M H QUALITY OF TEACHING PRACTICES Areas / Elements / Behaviors Scoring Final Scores A. CLASSROOM CULTURE 1. SUPPORTIVE LEARNING ENVIRONMENT 1 2 3 4 5 1.1 The teacher treats all students respectfully L M H 1.2 The teacher uses positive language with students L M H 1.3 The teacher responds to students’ needs N/A L M H The teacher does not exhibit bias and a. Gender L M H Determine 1.4 Sub-scores ▷ ▷ L M H challenges stereotypes in the classroom b. Disability L M H score 2. POSITIVE BEHAVIORIAL EXPECTATIONS 1 2 3 4 5 2.1 The teacher sets clear behavioral expectations for classroom activities L M H 2.2 The teacher acknowledges positive student behavior L M H 2.3 The teacher redirects misbehavior and focuses on the expected behavior, rather than the undesired behavior L M H B. INSTRUCTION 3. LESSON FACILITATION 1 2 3 4 5 3.1 The teacher explicitly articulates the objectives of the lesson and relates classroom activities to the objectives L M H 3.2 The teacher explains content using multiple forms of representation L M H 3.3 The teacher makes connections in the lesson that relate to other content knowledge or students’ daily lives L M H 3.4 The teacher models by enacting or thinking aloud L M H 4. CHECKS FOR UNDERSTANDING 1 2 3 4 5 4.1 The teacher uses questions, prompts or other strategies to determine students’ level of understanding L M H 4.2 The teacher monitors most students during independent/group work N/A L M H 4.3 The teacher adjusts teaching to the level of students L M H 5. FEEDBACK 1 2 3 4 5 5.1 The teacher provides specific comments or prompts that help clarify students’ misunderstandings L M H 5.2 The teacher provides specific comments or prompts that help identify students’ successes L M H 6. CRITICAL THINKING 1 2 3 4 5 6.1 The teacher asks open-ended questions L M H 6.2 The teacher provides thinking tasks L M H 6.3 The students ask open-ended questions or perform thinking tasks L M H C. SOCIOEMOTIONAL SKILLS 7. AUTONOMY 1 2 3 4 5 7.1 The teacher provides students with choices L M H 7.2 The teacher provides students with opportunities to take on roles in the classroom L M H 7.3 The students volunteer to participate in the classroom L M H 8. PERSEVERANCE 1 2 3 4 5 8.1 The teacher acknowledges students’ efforts L M H 8.2 The teacher has a positive attitude towards students’ challenges L M H 8.3 The teacher encourages goal setting L M H 9. SOCIAL & COLLABORATIVE SKILLS 1 2 3 4 5 9.1 The teacher promotes students’ collaboration through peer interaction L M H 9.2 The teacher promotes students’ interpersonal skills L M H 9.3 Students collaborate with one another through peer interaction L M H 1 20 OBSERVATION NOTES Teacher ID: SEGMENT 1 0.1 0.2 _________ 1.1 1.2 1.3 1.4a 1.4b _________ 2.1 2.2 2.3 _________ 3.1 3.2 3.3 3.4 _________ 4.1 4.2 4.3 _________ 5.1 5.2 _________ 6.1 6.2 6.3 _________ 7.1 7.2 7.3 _________ 8.1 8.2 8.3 _________ 9.1 9.2 9.3 2 21 OBSERVER MANUAL TIME ON TASK 22 0 The teacher maximizes time on learning. The teacher maximizes time on learning by ensuring most students are on task and provided with a TIME ON learning activity most of the time. This can be observed in the classroom through the following LEARNING behaviors: YES NO 0.1 LEARNING ACTIVITIES: NONLEARNING ACTIVITIES: This includes any activity that is related to class content, This includes any activity that is not related to class content, The teacher is teaching independent of its quality. including activities related to classroom management such as or provides a learning For example, learning activities can include a teacher lecturing, taking attendance or disciplining students or any other activity activity for most small group/teamwork, or students working on a worksheet or that leaves students waiting. reading independently. Note that if the teacher leaves the For example, when the teacher is silently writing on the board students classroom, but has provided students with a learning activity, this without asking students to copy. Other examples of nonlearning would still count as a learning activity. activities include: when a teacher takes attendance, s/he may read the students’ names individually; when there are misbehaviors, s/he may stop the lesson to redirect student misbehavior; when there are outside disruptions, s/he may stop teaching to see what is going on; when checking homework, s/he may check each student’s homework individually, while the other students wait with nothing to do. In addition, basic classroom processes may be prolonged, such as transitioning to a new activity, getting materials ready for a lesson, or completing administrative tasks. LOW MEDIUM HIGH 0.2 6 or more students are off task 2–5 students are off task All students are on task (one student may be off task) 1 Students are on task Students off task: This includes students who are not participating in the learning activity provided by the teacher either because they are quiet but distracted, or because they are disrupting the class. For example, in the first category, students may be staring out the window, resting their head on the desk, looking down to the floor or at the observer, or sleeping. In the second category, they may be passing notes, whispering, talking to another student during an activity that does not require talking, moving around the class, shouting, or in any other way disrupting the class. 1 Behavior 0.2 is scored as N/A if the teacher is not teaching or providing a learning activity, that is, if 0.1 is scored No. 23 OBSERVER MANUAL QUALITY OF TEACHING PRACTICES 24 CLASSROOM CULTURE SUPPORTIVE LEARNING ENVIRONMENT POSITIVE BEHAVIORAL EXPECTATIONS 25 A.1 CLASSROOM CULTURE The teacher creates a supportive learning environment. The teacher creates a classroom environment where students can feel emotionally safe and SUPPORTIVE supported. Moreover, all students feel welcome, as the teacher treats all students respectfully. LEARNING This can be observed in the classroom through the following behaviors: ENVIRONMENT Score 1 2 3 4 5 Behavior LOW MEDIUM HIGH Quality In this classroom, the teacher In this classroom, the teacher is In this classroom, the teacher is is ineffective at creating a somewhat effective at creating a effective at creating a supportive Range supportive learning supportive learning learning environment. environment. environment. 1.1 The teacher does not treat all The teacher treats all students The teacher treats all students students respectfully. somewhat respectfully. respectfully. The teacher treats all For example: The teacher may yell at For example, the teacher does not treat For example: The teacher uses students’ students respectfully some students, scold them, students disrespectfully (e.g., s/he does not names, says “please” and “thank you,” or shame/ridicule them, or use physical yell at or ridicule students), but the teacher shows some other culturally relevant sign of punishment to discipline them. does not show outward signs of respect respect. toward students either (e.g., call students by their names, say “please” or “thank you,” or other culturally relevant signs of respect). 1.2 The teacher does not use positive The teacher uses some positive The teacher consistently uses positive language in his/her communication language in his/her communication with language in his/her communication with The teacher uses with students. students. students. positive language with For example: The teacher may say “well For example: The teacher consistently uses students2 done” or “good”, although this happens encouraging phrases such as “Great job!” infrequently. when students show their work to him/her, or “You can do this!”, or “You are such a talented group of students.” 1.3 The teacher is not aware The teacher responds to students’ The teacher promptly responds to of students’ needs OR does not needs but may not address the students’ needs in a way that The teacher responds to address the problem at hand. problem at hand. specifically addresses the problem at students’ needs3 For example: A student may not have the For example: A student may be upset hand. required supplies for the lesson, and the because s/he does not have a pencil, and For example: If a student does not have a teacher does not notice or sees it and the teacher asks another student to share pencil, the teacher allows the student to ignores it. Alternatively, a student may be his/her pencil, but s/he refuses. The teacher borrow one from his/her spare pencil box. upset because of a bad grade or a carries on with the lesson without solving personal problem, and the teacher ignores the problem. the student or is dismissive of the issue (e.g., the teacher tells the student to “get over it” or “pull yourself together”). 2 Only verbal communication is counted as positive language; nonverbal displays of positive language would not count toward this behavior 3 This behavior is scored as N/A if there are no observable emotional, material, or physical needs. 26 A.1 continued CLASSROOM CULTURE The teacher creates a supportive learning environment. The teacher creates a classroom environment where students can feel emotionally safe and SUPPORTIVE supported. Moreover, all students feel welcome, as the teacher treats all students respectfully. LEARNING This can be observed in the classroom through the following behaviors: ENVIRONMENT Score 1 2 3 4 5 Behavior LOW MEDIUM HIGH Quality In this classroom, the teacher In this classroom, the teacher is In this classroom, the teacher is is ineffective at creating a somewhat effective at creating a effective at creating a supportive Range supportive learning supportive learning learning environment. environment. environment. 1.4 The teacher exhibits bias or The teacher does not exhibit bias but The teacher does not exhibit bias AND reinforces stereotypes in the does not challenge stereotypes either. challenges stereotypes in the The teacher does not classroom. classroom. exhibit bias and challenges stereotypes in the classroom4 1.4a The teacher could show this by The teacher provides students of all The teacher provides students of all providing students with unequal genders with equal opportunities to genders with equal opportunities to Gender opportunities to participate in participate in the classroom and has participate in the classroom, has similar classroom activities or by expressing similar expectations for all students. expectations for all students, AND unequal expectations for students’ challenges gender stereotypes in the For example: The teacher calls equally on all behaviors or capabilities. classroom. genders to answer difficult questions and For example: A teacher seats girls exclusively praises both boys and girls after correctly For example: The teacher calls equally on all at the back of the classroom or only calls on answering questions. genders to answer difficult questions and boys to answer difficult questions. praises both boys and girls after correctly The teacher asks boys and girls to clean the answering questions. The teacher asks boys and Alternatively, the teacher calls equally blackboard and distribute learning materials girls to clean the blackboard and distribute on students of all genders to answer (e.g., textbooks) to the class. learning materials (e.g., textbooks) to the class. difficult questions, but only assigns girls In addition, the teacher uses examples and to clean the blackboard or hand out explanations that portray female rather than learning materials (e.g., textbooks) to male scientists, doctors, and astronauts and/or the class. encourages discussions with students about gender stereotypes and/or gender equality. Other examples of gender bias are teachers scolding boys but not girls after incorrectly The teacher may also actively encourage equal answering a question or misbehaving. They participation through comments such as: "Let's may also give praise to girls but not boys after hear more from the girls” or 'Now we have heard correctly answering a question. from a girl, let's hear from a boy." 1.4b The teacher may provide students with The teacher provides students of all The teacher provides students of all unequal opportunities to participate ability levels with equal opportunities to ability levels with equal opportunities to Disability in learning activities, use stigmatizing participate in the classroom and has participate in the classroom, has similar terms, or express low expectations for similar expectations for all students. expectations for all students, AND student’s behaviors or capabilities. challenges disability stereotypes in the For example: The teacher enables students with classroom. For example: The teacher seats students with disabilities to work with other class members disabilities separately from other students. during group work, provides opportunities for For example: The teacher has students with students with disabilities to ask questions, and disabilities work with others during group work The teacher may use stigmatizing terms about participate in whole class learning activities. AND uses examples and explanations that people with disabilities, in general, or express portray people with disabilities in important bias towards students with disabilities in the Alternatively, the teacher praises positions. classroom through low expectations for their students with disabilities in the same behavior or capabilities. manner as other students in the classroom. 4 The chances to participate should be considered proportionally to the ratio of different genders in the classroom. 27 A.2 CLASSROOM CULTURE The teacher promotes positive behavior in the classroom. The teacher promotes positive behavior by acknowledging students’ behavior that meets or POSITIVE exceeds expectations. Moreover, the teacher sets clear behavioral expectations for different parts BEHAVIORAL of the lesson. This can be observed in the classroom through the following behaviors: EXPECTATIONS Score 1 2 3 4 5 Behavior LOW MEDIUM HIGH Quality In this classroom, the In this classroom, the teacher In this classroom, the teacher is Range teacher is ineffective at promoting positive behavior. is somewhat effective at promoting positive behavior. effective at promoting positive behavior. 2.1 The teacher does not set behavioral The teacher sets unclear or The teacher sets clear behavioral expectations for classroom tasks superficial behavioral expectations expectations throughout the lesson for The teacher sets clear and/or activities. for classroom tasks and/or activities. classroom tasks and/or activities. behavioral expectations For example: The teacher says, “Work on For example: When introducing a group For example: Upon introducing a group activity for classroom activities your reading comprehension skills,” activity, the teacher says, “Please sit in to the class, the teacher explicitly states the without providing instruction on what the your preassigned groups and behave,” expected behavior for students in the group. expected behavior is for the activity. without clarifying what such behavior This may include, “Use a quiet indoor voice” or would entail. “Take turns speaking.” If students are working independently, the teacher gives directions on what to do when they complete the activity. The teacher says, “Please quietly get up, bring your worksheet to me, and read while you wait for your classmates to finish.” Alternatively, the teacher is not observed setting clear behavioral expectations, but students are well-behaved5 throughout the lesson. 2.2 The teacher does not acknowledge The teacher acknowledges some The teacher acknowledges students’ student behavior that meets or students’ behavior, but is not positive behavior that meets or exceeds The teacher exceeds expectations. specific about their expected expectations. acknowledges positive behavior. For example: A teacher says to the class, “I just student behavior For example: If a group is following noticed that members of Group A are taking behavioral expectations, the teacher says, turns to speak and are proactively working on “This group is working well together” or the next assignment.” “This group is doing a good job,” without clarifying why or how. 2.3 Redirection of misbehavior is Redirection of misbehavior is When a problem arises, redirection of ineffective and focuses on effective but focuses on misbehavior effectively addresses the The teacher redirects misbehaviors, rather than the misbehaviors rather than the problem at hand and focuses on the misbehavior and expected behavior. expected behavior. Alternatively, expected behavior. redirection of misbehavior is focuses on the expected For example: If s/he notices a distracted somewhat effective and focuses on For example: If students are talking loudly and student, the teacher stops lecturing and being disruptive during a lesson, the teacher behavior, rather than the calls out the name of the student, asking the expected behavior. says, “Remember to use quiet voices,” and the undesired behavior5 her, “Why are you not paying attention in For example: Upon noticing that three students quiet down. class?” Alternatively, the teacher students are not working on the assigned continues to ignore the student who is Alternatively, the teacher is not observed problems, the teacher says, “You three distracted, but the distracted student redirecting students’ behavior, but the need to stop talking now, you are making begins to tease and argue with the peer too much noise.” This statement focuses students are well-behaved throughout sitting next to her. This shifts the focus of on the disruptive students’ negative the lesson. the entire class away from the lesson and behavior, rather than on what is expected onto those two students. of them. Consequently, the disruptive students quiet down. In another scenario, the teacher redirects the students by asking them to “Focus on the task at hand.” Even though the teacher focuses on the positive behavior expected from the students, for the most part, they continue to talk. 5 A misbehavior occurs when a student causes a disruption in the classroom that either interferes with the flow of the lesson, distracts other students, or upsets the teacher. 28 INSTRUCTION LESSON FACILITATION CHECKS FOR UNDERSTANDING FEEDBACK CRITICAL THINKING 29 B.3 INSTRUCTION The teacher facilitates the lesson to promote comprehension. The teacher facilitates the lesson to promote comprehension by explicitly articulating the objectives, explains the content using multiple forms of representation, and connecting the lesson with other LESSON content knowledge or students’ experiences. This can be observed in the classroom through the FACILITATION following behaviors: Score 1 2 3 4 5 Behavior LOW MEDIUM HIGH Quality In this classroom, the teacher is In this classroom, the teacher is In this classroom, the teacher is ineffective in facilitating the somewhat effective in facilitating effective in facilitating the lesson Range lesson to promote the lesson to promote to promote comprehension. comprehension. comprehension. The teacher does not state or write the The teacher either explicitly states and/or The teacher explicitly states and/or writes a 3.1 lesson objective(s), nor can one be writes a broad lesson objective OR the specific lesson objective (e.g., a learning The teacher explicitly inferred from the lesson activities. objective is not explicitly stated and/or goal) and the lesson activities align to the For example: The teacher asks students to written but can be inferred from the stated objective. articulates the lesson activities. take turns reading a text about planting and For example: Near the beginning of class the teacher objectives of the harvesting crops. S/he then spends the rest For example: The teacher says, “Today we’re states, “Today we’re going to learn to multiply lesson and relates of the lesson discussing farming and the going to learn about multiplication,” without fractions.” Each lesson activity is clearly related to the specific processes involved. The teacher further specification while the activity is clearly objective of multiplying fractions. classroom activities does not state a lesson objective, and it is learning to multiply fractions. Alternatively, the difficult to infer a lesson objective from the lesson activities may clearly work toward how to the objectives activities as the objective could be to divide whole numbers, but this is not developing oral reading fluency, developing explicitly articulated by the teacher. vocabulary, or learning about agriculture. The teacher explains content using The teacher explains content using two The teacher explains content using three or 3.2 one form of representation OR content forms of representation. more forms of representation. The teacher explains is simply not explained. For example: : The teacher states, “A fraction For example: The teacher states, “A fraction is a content using For example: The teacher states, “A fraction is a combination of a numerator and combination of a numerator and denominator,” and is a combination of a numerator and denominator,” and writes an example of a writes the example of ¼ on the board. Later in the multiple forms of denominator,” without providing any written fraction on the board. In a language arts lesson, the teacher uses a visual aide as part of or other visual representation of a fraction lesson, the teacher states that a verb is an his/her explanation of content by folding a piece of representation during the lesson segment. Alternatively, the action word and writes a sentence which paper into quarters and coloring in one square. In a teacher may not provide any explanation of contains a verb which is underlined on the language arts lesson, the teacher states that a verb content, uses too many technical terms board. is an action word and writes a sentence which without explaining what s/he means, and/or contains a verb which is underlined on the board. may explain ideas without a logical order or The teacher then mimes a series of actions and connection. Moreover, the teacher may say, asks students to identify these examples of verbs. “A fraction is a combination of a numerator and denominator,” without defining those terms. Alternatively, the teacher may not provide any explanation of content. The teacher does not connect what is The teacher may attempt to connect the The teacher meaningfully connects the lesson to 3.3 being taught to other content lesson to other content knowledge or other content knowledge or students’ daily lives. knowledge or students’ daily lives. students’ daily lives, but the The teacher makes The teacher may use examples that may connections are superficial, confusing, For example: When teaching a class on fractions, the teacher relates the content to students’ experiences by connections in the be related to other content or students’ or unclear. asking, “Who has had to slice a cake? How did you make lives, but the teacher does not attempt to lesson that relate to connect it to the learning activity. For example: When introducing a lesson on sure there were enough slices for everyone? Learning fractions, the teacher says, “When we cut a about fractions can help us divide a cake between people.” other content For example: During a lesson on fractions cake, we use fractions,” and goes on to The teacher also connects the lesson to a prior lesson by knowledge or the teacher uses a picture of a cake and explain fractions. The connection to students’ saying, “Remember yesterday when we learned about lives is superficial and nonspecific. halves? We learned that when we cut a cake in half, two students’ daily lives divides it into fourths but does not make a connection to students’ experience with Alternatively, the teacher says, “Remember people can share it equally. Today we will learn how to slicing cake. Alternatively, the teacher says, yesterday we learned the rules for adding divide the cake into fourths, so four people can share it. “Remember, yesterday we learned about whole numbers? Now we are going to use When we were forming halves, we made sure we had two whole numbers? Today, we are going to those rules and apply them to adding halves of identical size. The same is true when we are learn how to add fractions.” fractions.” However, when explaining how to forming fourths: we have to keep slices the same size.” add fractions, the teacher does not link the The connection between the current lesson and other rules back to the rules for whole numbers. content knowledge and/or students’ daily lives is clear. The teacher does not model. The teacher partially models the The teacher completely models the learning 3.4 learning activity. activity by enacting all parts of the procedure OR For example: The teacher spends the entire The teacher models class reading a passage and asking by enacting the procedure AND thinking aloud. For example: In an English class where the students questions about the text but does objective of the activity is to write a by enacting or For example: The teacher demonstrates different ways not present any procedure. In a math class, paragraph, the teacher only demonstrates to solve a math problem (enactment of a procedure) thinking aloud6 the teacher simply gives students problems how to write a topic sentence. In a math class, and while doing so, s/he says what s/he is thinking at to complete on their own and does not the teacher shows (enacts) how to draw a bar each step of the equation (think aloud). Or if students demonstrate a procedure to solve the graph but does not clarify how s/he extracted are calculating the perimeter of their desk, the teacher problems. the data from the text to create the bar graph. demonstrates each step in the process (full enactment of a procedure) through the use of pictures, and/or physical objects, or other materials that might be locally available. While doing so, the teacher says what s/he is thinking at each step of the process. 6 Modeling can take place at any time in the lesson (including at the end). If the learning activity is procedural in nature, modeling will include an enactment of the procedure for students to observe; however, if the activity focuses on developing a thinking skill, a complete model will include a think aloud. An action is considered modeling so long as the teacher demonstrates/enacts procedures or thinking processes related to the learning activity. 30 B.4 INSTRUCTION The teacher checks for understanding for most students. The teacher checks for understanding to ensure most students comprehend the lesson content. CHECKS FOR Moreover, the teacher adjusts the pace of the lesson to provide students with additional learning UNDERSTANDING opportunities. This can be observed in the classroom through the following behaviors: Score 1 2 3 4 5 Behavior LOW MEDIUM HIGH Quality In this classroom, the teacher In this classroom, the teacher is In this classroom, the teacher Range does not check for any student’s understanding. effective at checking only a few students’ understanding. is effective at checking for most students’ understanding. 4.1 The teacher either does not ask The teacher uses questions, prompts, or The teacher uses questions, questions/prompt students at all OR other strategies that are effective at prompts, or other strategies that are The teacher uses when s/he does, the class responds determining only a few students’ level effective at determining most questions, prompts, or in synchrony, which is accepted of understanding. students’ level of understanding. without further checking for other strategies to For example: The teacher asks, “What is For example: The teacher says, “Please understanding. 7+8?” Only a few students respond by raising put your thumb up if you agree or down if determine students’ For example: When explaining a concept, their hand, a group from which the teacher you disagree with this statement: level of understanding the teacher asks, “Have you all calls upon 1 or 2 students to provide an Equilateral triangles have equal angles.” understood?” The students in the class answer. Alternatively, the teacher asks the The teacher also asks students to respond in unison, “Yes, we have.” question but does not ask students to raise demonstrate their knowledge by having all Another example is that the teacher their hands in response and simply allows students share their answers, e.g., by inquires, “This is correct, right?” after students to willingly volunteer their answers. asking each student to read out the completing a problem set. The class or an sentence s/he wrote using past tense individual student replies, “Yes, this is verbs. correct.” 4.2 The teacher does not monitor The teacher monitors some students The teacher systematically monitors students when they are working when they are working independently or in most students by circulating the The teacher monitors independently or in groups. groups to check their understanding. classroom and approaching individual most students during students or groups to check their For example: The teacher sits at his/her For example: The teacher observes some desk or remains standing in front of the student work for accuracy, clarifies concepts, understanding. independent/group class when students are working. or asks questions. For example: When students are working, work7 the teacher walks around the classroom, making sure to approach students or groups in a systematic way. The teacher observes most students’ work, clarifies concepts, and asks questions. 4.3 The teacher does not adjust teaching The teacher slightly adjusts teaching, The teacher substantially adjusts for students.8 but this adjustment is brief and teaching for students by providing The teacher adjusts superficial. students with more opportunities to For example: The teacher may notice that teaching to the level learn. The teacher may further present many students are getting the wrong For example: As students complete an answer but does not re-explain the alphabet worksheet, the teacher notices they information in a different way to help of students are not dotting their ‘i’s. In response, s/he students better understand the concept concept or provide additional opportunities to learn. briefly reminds the class to dot their ‘i’s. being taught. The teacher may also provide more challenging tasks for Or, when solving the multiplication problem 7 those who already have an advanced x 3, a student confuses the process with understanding. addition and answers '10' on their worksheet. In response, s/he reminds the student that For example: As students complete an they are doing multiplication and not addition alphabet worksheet, the teacher notices equations. they are not dotting their ‘i’s. In response, s/he briefly stops the activity and reviews the differences between capital and lower case ‘i’s before continuing with the alphabet activity. In realizing that a student is not understanding the process of multiplication, the teacher may draw a picture or use concrete objects to help the student better understand the operation. Alternatively, if the teacher notices that a student has already completed the worksheet, s/he may give that student another activity to complete while waiting for the rest of the class. 7 This behavior is scored as N/A if there is no observable group or independent work. 8 Even if there is no perceived need for adjusting, if the teacher does not adjust teaching, this behavior is scored as low. 31 B.5 INSTRUCTION The teacher provides feedback to deepen student understanding. The teacher provides specific comments or prompts9 to help identify misunderstandings, understand successes, and guide thought processes to promote learning. This can be observed in the FEEDBACK classroom through the following behaviors: Score 1 2 3 4 5 Behavior LOW MEDIUM HIGH Quality In this classroom, the teacher In this classroom, the teacher is In this classroom, the teacher Range is ineffective at providing feedback to deepen students’ somewhat effective at providing feedback to deepen students’ is highly effective at providing feedback to deepen students’ understanding. understanding. understanding. 5.1 The teacher either does not provide The teacher provides students The teacher provides students with students with comments/prompts with general or superficial specific comments/prompts that The teacher provides about their misunderstandings OR comments/prompts about their contain substantive information specific comments the comments provided are simple, misunderstandings. that helps clarify students’ evaluative statements (e.g., misunderstandings. or prompts that help “That is incorrect”). For example: In a math class, the teacher says, “You forgot to include the negative For example, the teacher says, “Do you clarify students’ For example: When a student answers a sign,” without providing further information remember what happens when we misunderstandings teacher’s question incorrectly, the teacher or prompts. multiply a positive and a negative responds by saying, “That is not the number? Let’s look at your notes. Now, correct answer,” and moves on. let’s look at your answer. What do you need to change to find the correct answer?” 5.2 The teacher either does not provide The teacher provides students with The teacher provides students with students with comments/prompts general or superficial comments/ specific comments/prompts that The teacher provides about their successes OR the prompts about their successes. contain substantive information that specific comments or comments provided are simple, helps identify students’ successes. For example: If students are writing stories evaluative statements (e.g., “That is as part of an assignment, the teacher says, prompts that help correct”). “Good job on the third paragraph,” without For example: If students are writing stories, the teacher says, “You do a good identify students’ For example: When a student answers a specifying what that particular student did job getting the reader interested in this successes teacher’s question correctly, the teacher that made it good. paragraph when you write ‘no one knew responds by saying, “That is correct,” and what would happen.’ This sentence makes moves on. me want to read more.” Alternatively, the teacher highlights one student’s work and say to the class, “Look at the work of this classmate, see how s/he used the number line to solve this subtraction problem?” and then proceed to explain how the student solved it. 9 Prompts are pieces of information, such as guiding hints or questions, which are given by the teacher and encourage students to think through misunderstandings or identify successes. 32 B.6 INSTRUCTION The teacher builds students’ critical thinking skills. The teacher builds students’ critical thinking skills by encouraging them to actively analyze content. CRITICAL This can be observed in the classroom through the following behaviors: THINKING Score 1 2 3 4 5 Behavior LOW MEDIUM HIGH Quality In this classroom, the teacher In this classroom, the teacher is In this classroom, the teacher Range is ineffective at developing critical thinking skills. somewhat effective at developing critical thinking skills. is effective at developing critical thinking skills. 6.1 The teacher does not ask open- The teacher asks students at least The teacher asks students 3 or more ended questions OR asks only one two open-ended questions but does not open-ended questions AND at least The teacher asks open-ended question. The teacher build on student responses, OR the 1 of them builds upon student open-ended questions may ask closed-ended questions that teacher asks 2 open-ended questions responses by asking students to justify have a predetermined answer. and 1 of them is a follow-up to a their reasoning, further explain, or that require reasoning, For example: The teacher asks, “Who is student response. clarify their ideas. explanation, or generalization the main character in this story?” or For example: The teacher asks, “Why was the For example: The teacher asks, “How do or have more than one correct “Which is greater, -2 or -6?” character unhappy? What makes you think you think the main characters in the story answer that?” OR “Why is -2 greater than -6?” And would prepare for the competition?” After then asks, “How do you use the number line a student responds, the teacher then to determine if -8 or -4 is greater?” follows up by asking, “What facts or ideas make you think that?” Then s/he asks another student, “What do you think happens next?” In a math class, the teacher asks, “How do you know -2 is greater than -6?” After the student responds, the teacher follows up by asking, “What would happen if the numbers were positive?” Later in the lesson, the teacher asks, “How do you use the number line to determine if –8 or –4 is greater?” 6.2 The teacher does not provide The teacher provides superficial The teacher provides substantial thinking tasks. Classrooms with no thinking tasks. Superficial thinking tasks thinking tasks. Substantial thinking The teacher provides thinking tasks include those where are tasks such as matching sets of items, tasks are tasks such as making thinking tasks students simply listen to the teacher or identifying concepts or key pieces of predictions, identifying patterns, perform rote tasks. information, and comparing and explaining thinking, making that require students to actively For examples, refer to the thinking task contrasting characteristics. They also connections, and interpreting analyze content, as opposed to table on the next page. include applying learned information or information. They also include applying simply receiving information or techniques to tasks similar to those the learned information or techniques to building fluency (i.e., rote teacher has already demonstrated. new tasks the teacher has not demonstrated. learning) For examples, refer to the thinking task table on the next page. For examples, refer to the thinking task table on the next page. 6.3 Students do not ask open-ended Students do not ask open-ended Students ask open-ended questions. questions, nor do they perform questions; however, they do perform For example, after working on subtraction The students ask open- thinking tasks. superficial thinking tasks. problems, a student asks, “Why does 6 – ended questions or For examples, refer to the thinking task For examples, refer to the thinking task table 9 equal a negative number?” perform thinking tasks table on the next page. on the next page. Alternatively, they perform substantial thinking tasks. For examples, refer to the thinking task table on the next page. 33 Thinking Task Table These examples are intended to help observers decipher what constitutes a thinking task and to differentiate between the quality levels. It is important to note that these examples are not comprehensive. In addition, context and students’ learning levels should be weighed considerably when scoring 6.2 and 6.3. Language LOW MEDIUM HIGH Classes 1. Students repetitively read Students match photos to a letter. For example, The teacher has several short words written on the the alphabet. different letters are written on the board. The teacher board. S/he reads “cat” while pointing at the letters and Learning calls students up one at a time and gives them an asks students what would happen if they changed the to read image of a piece of fruit. S/he says, "What piece of first letter to “p” or “s.” Then s/he asks them to choose fruit do you have? Think, what is the first letter of the a word and see what happens if they change the first name of your fruit and put your picture on the board letter. under the appropriate letter." 2. Students take turns After reading a story, the teacher writes a series of After reading a story, the teacher says, "Now I want Reading reading a story or simply questions on the board that students need to answer you to predict what might happen next in the story. listen to the teacher read a independently. These questions ask students to Write down what you think would happen next and comprehension story. identify key aspects of the story, such as the then share with your neighbor when you're done." protagonist, the setting, and the sequence of events. 3. Students repetitively write The teacher asks students to write sentences where Students are asked to analyze three different Learning writing example sentences. the focus is on a specific sentence structure using a sentences by listing the similarities and differences list of specific verbs or nouns. between the sentence structures and to explain why using one sentence structure is better than another. Math LOW MEDIUM HIGH Classes 1. The teacher has students Students compare numbers based on size and The teacher puts sequences of numbers on the memorize numbers 1–100. organize them by descending or ascending order. board and has students find the patterns. For Learning about For example, the teacher writes 8, 29, 72, 63, and 7 example, the teacher writes the following three numbers on the board. S/he tells students to write the number sequences on the board: 3, 13, 17, 23; 6, 15, numbers in ascending order. Alternatively, the 24, 30, 36; and 4, 12, 28, 32, 40. S/he then tells teacher tells students, "Look at this set of numbers: students to identify what each group has in common. 2, 5, 10, 19, 24. Write down in two columns which are even numbers and which are odd numbers." 2. Students listen to the The teacher explains the process of subtraction. She The teacher explains the process of subtraction. The teacher explain the concept then asks students to complete several subtraction teacher then writes a “menu” on the board including Learning about and then copy the examples problems (e.g., “What's 10-5?") and to write their prices. The teacher has students imagine they have subtraction from the board. answers in their notebooks. $20 and asks them to figure out how much change they would receive from buying different items. 3. Students listen to the In a lesson on bar graphs, the teacher uses a chart In a lesson on bar graphs, the teacher draws a bar teacher explain the concept of numbers and draws a bar graph showing the graph showing the class's favorite foods. S/he then Learning about and then copy the examples class's favorite foods. S/he then asks students, asks students to work in pairs to interpret the bar graphs from the board. "Which bar is tallest? Which bar is the shortest?" information to identify and rank foods from the most preferred to the least preferred. S/he then tells them to calculate how many more students prefer the most preferred compared to the least preferred food. 4. Students are told to repeat In a lesson on fractions, students are given pieces of The teacher tells students to fold a piece of paper the definition of a fraction to paper cut into various shapes and are instructed to into sixths. Then s/he says, "Color in 3/6ths of your Learning about their neighbor. fold the paper into various shapes that represent paper. Write down the fraction of the colored part of fractions fractions. The teacher shows them how to fold into the paper and see how many other fractions you can various fractions and then, having them work in pairs, write that represent this area. What patterns do you says, "One of you will fold your paper into 1/5, the identify among the fractions?” other will fold your paper into 1/3. Then, whoever has the larger fraction should stand up." 5. The teacher calculates the After explaining how to find the area of a rectangle, After learning how to find the area of a rectangle, Finding the area area of three different the teacher draws a rectangle on the board, gives students are asked to compute the area of the rectangles on the board and measurements, and has students use formulas they classroom, which is in the shape of a rectangle. of a rectangle has students copy the know to determine the area. information in their notebooks. 6. The teacher writes a The teacher writes a word problem on the board and The teacher writes a word problem on the board and word problem on the board shows students how to solve it. The teacher then shows students how to solve it. The teacher then Solving word and shows students how to gives students a set of word problems to solve. gives students a set of word problems to solve. The problems solve it. teacher calls on students to explain how they solved the different problems. 34 SOCIOEMOTIONAL SKILLS AUTONOMY PERSEVERANCE SOCIAL AND COLLABORATIVE SKILLS 35 C.7 SOCIOEMOTIONAL SKILLS The teacher allows students to make choices and encourages students to participate in the classroom. AUTONOMY The teacher provides students with opportunities to make choices and take on meaningful roles in the classroom. Students make use of these opportunities by volunteering to take on roles and expressing their ideas and opinions throughout the lesson. This can be observed in the classroom through the following behaviors: Score 1 2 3 4 5 Behavior LOW MEDIUM HIGH Quality In this classroom, the teacher In this classroom, the teacher is In this classroom, the teacher Range is ineffective at developing students’ autonomy. somewhat effective at developing students’ autonomy. is effective at developing students’ autonomy. 7.1 The teacher does not explicitly The teacher explicitly provides students The teacher explicitly provides provide students with choices. The with at least one superficial choice that students with at least one The teacher provides teacher decides how learning activities is not related to the learning objective. substantive choice that is related to students with choices should be completed, without providing the learning objective. For example: The teacher allows students to different options for how students can choose between different colored pencils to For example: The teacher allows students approach the task. complete an assignment, decide where to sit to choose between writing an essay or For example: Students are asked to in the classroom when completing a task, doing a presentation about their favorite complete a set of math problems following choose the order in which to complete the sport. In a science class, the teacher lets a prescribed set of steps. Alternatively, the activities, or vote on which student students choose an animal to investigate. teacher tells students to write sentences presentation was the best. In a math class, a teacher allows students without providing intentional choices. to choose how they will solve a multiplication problem (e.g., by using concrete materials, drawing pictures or using a number line). 7.2 The teacher does not provide The teacher provides students with The teacher provides students with students with opportunities to take opportunities to take on limited roles in opportunities to take on meaningful The teacher on roles in the classroom. the classroom. roles in the classroom, in which they provides students For example: The lesson is primarily For example: Students take attendance, are responsible for parts of a learning activity. with opportunities lecture-based and highly structured; assign tasks, pass out materials, or write on subsequently, students’ participation the board. Limited roles also include For example: The teacher gives a student to take on roles in is limited to copying down information. housekeeping tasks such as fetching water, the opportunity to solve an equation on the classroom In this lesson, students never get wiping the board, or cleaning the classroom. the board and explain to the class how the chance to come to the board or read a s/he tackled the main challenges of the text. problem. 7.3 Students do not volunteer to Only a few students volunteer to Most students volunteer to participate in the classroom. participate by expressing their ideas participate by expressing their ideas The students volunteer and taking on roles. and taking on roles. to participate in the For example: When the teacher asks a For example: When the teacher asks a classroom question, only a few students put their hand question, many students put their hand up up to answer; later when the teacher asks to share their answers. The students could another question, the same few students put also volunteer without the teacher asking their hand up. (e.g., a student offers to share a related experience when the teacher is explaining a concept). 36 C.8 SOCIOEMOTIONAL SKILLS The teacher promotes students’ efforts, has a positive attitude toward challenges, and encourages goal setting. PERSEVERANCE The teacher promotes students’ efforts toward the goal of mastering new skills or concepts, instead of focusing solely on results, intelligence, or natural abilities. In addition, the teacher has a positive attitude toward challenges, framing failure and frustrations as useful parts of the learning process. The teacher also encourages students to set short- and/or long-term goals. This can be observed in the classroom through the following behaviors: Score 1 2 3 4 5 Behavior LOW MEDIUM HIGH Quality In this classroom, the teacher In this classroom, the teacher In this classroom, the teacher Range is ineffective at developing students’ perseverance. is somewhat effective at developing students’ is effective at developing students’ perseverance. perseverance. 8.1 The teacher does not acknowledge In this classroom, the teacher sometimes In this classroom, the teacher student efforts. Although the teacher acknowledges student efforts, but most frequently acknowledges students’ The teacher may praise students for “being smart” praise is focused on outcomes or efforts toward mastering new skills acknowledges or “intelligent,” the teacher does not student intelligence. or concepts and identifies these focus on students’ efforts or work. efforts explicitly. students’ efforts rather For example: When a student does well on a For example: The teacher says, test, the teacher says, “I know you put so For example: When students solve a than focusing only “Very good! You’re the smartest student in much hard work into this!,” but most times, the difficult problem they had been struggling on results, intelligence, the class” or “Well done! You’re so smart!” teacher praises students by saying they are with, the teacher praises and highlights “smart” or “intelligent.” the efforts they made to solve the or natural abilities problem. The teacher says, “You have progressed so much on our multiplication problem sets! I’m glad you asked me for help. If you keep practicing and using the strategies we learned in class, you’ll master them all very soon!” 8.2 The teacher has a negative attitude The teacher has a neutral attitude The teacher has a positive attitude toward students’ challenges. toward students’ challenges. Although toward students’ challenges, and The teacher has a For example: The teacher explicitly scolds the teacher does not penalize a student for helps students understand that failure positive attitude students for making mistakes or becomes making mistakes or struggling with a new and frustration are normal parts of the concept, the teacher does not make it learning process. towards students’ impatient with a student for taking time to understand a new concept. clear that failure and frustration are normal For example: When a student is struggling challenges10 parts of the learning process either. with a problem set, the teacher says, For example: When a student is struggling to “Remember, it’s okay to feel frustrated solve a math problem on the board, the when we’re trying to do something new! teacher simply gives the student the answer in Let’s think about how we can go about a neutral manner (i.e., not in an angry or this.” The teacher also encourages impatient manner). students to think through different resources they could turn to for help (e.g., asking a friend for advice, looking for answers in the textbook, using concrete materials or pictures when solving math problems). 8.3 The teacher does not encourage The teacher encourages students to set The teacher encourages students to students to set short- or long-term either short- OR long-term goals.11 set short- AND long-term goals.11 The teacher encourages goals.11 The teacher may reference both long- For example: For short-term goal setting, the goal setting teacher says, “How many pages of the book and short-term goals at the same time, will you read each day this week?” For long- particularly when encouraging students term goal setting, the teacher says, “I want to set a short-term goal that would help you to write down how much progress you’ve them achieve a long-term goal. made on the goals we set at the beginning of For example: The teacher says, “Let’s the school year.” think about the goals we set for ourselves Alternatively, the teacher may talk at the beginning of the school year. What about the importance of setting goals in is one thing you will do this week that will a general way. get you closer to that goal?” Alternatively, the teacher talks about the short- and For example: The teacher says, “It’s important long-term goals separately (as in the to think about what you want to be when you examples for “Medium”). grow up.” In addition, the teacher highlights how characters in a story set a short- or long- term goal for themselves and how they worked toward it. 10 These challenges may include making mistakes, scoring low on a test, or feeling frustrated when trying to understand a concept. 11 Short-term goals are goals that students aim to achieve within a month or less, and long-term goals are goals that span a longer timeframe (e.g., over the school year, when they grow up). 37 C.9 SOCIOEMOTIONAL SKILLS The teacher fosters a collaborative classroom environment. The teacher encourages students’ collaboration with one another and promotes students’ SOCIAL & interpersonal skills. Students respond to the teacher’s efforts by collaborating with one another in the classroom, creating an environment free from physical or emotional hostility. This can be observed COLLABORATIVE in the classroom through the following behaviors: SKILLS Score 1 2 3 4 5 Behavior LOW MEDIUM HIGH Quality In this classroom, the teacher In this classroom, the In this classroom, the teacher Range is ineffective at developing students’ collaborative skills. teacher is somewhat effective at developing students’ is effective at developing students’ collaborative skills. collaborative skills. 9.1 The teacher does not promote The teacher promotes superficial The teacher promotes substantial collaboration among students. student collaboration through sharing student collaboration by asking them The teacher promotes opinions, materials, or ideas. to work together to produce a product, For example: The teacher does not students’ collaboration provide any opportunities to work in For example: The teacher asks students to solve a problem, complete a groups or pairs. read their neighbor’s work or share crayons worksheet, or present a new idea. through peer interaction with each other. For example: The teacher asks students to form pairs or groups to complete a task that requires collaboration, such as creating a diagram of the water cycle or coming up with skits to illustrate a set of vocabulary words. The teacher asks students to read their neighbor's work and then provide feedback on how their neighbor can improve their work. 9.2 The teacher does not promote The teacher promotes students’ The teacher promotes students’ students’ interpersonal skills. interpersonal skills in a brief or interpersonal skills by encouraging The teacher promotes superficial manner. perspective taking, empathizing, students’ interpersonal For example: The teacher tells students to emotion regulation, or social problem “Help each other” during a group exercise, solving. skills, asks a student to “Say you’re sorry” to a For example: The teacher asks a student, such as perspective taking, classmate, or encourages students to take “How do you think that made him/her turns during an activity. However, the teacher (classmate or character in a book) feel?” empathizing, emotion does not explain why these behaviors are regulation, and social problem important. After reading a story about a character solving12 who is blind, the teacher asks students to imagine what it would be like if they couldn't see. 9.3 Students do not collaborate OR when Students collaborate superficially; Students collaborate with one students interact with one another, there may also be minimal instances another by working together to Students collaborate they display negative behaviors. where students display negative produce a product, solve a problem, with one another For example: When asked to pick partners behaviors (e.g., teasing, pushing, complete a worksheet, or present a bullying); however, these behaviors are new idea. There are no displays of through peer interaction for an activity, students purposefully isolated and minor or playful (i.e., no negative behavior. exclude one or more of their peers. student is upset) and are not a core For example: Students work in groups to characteristic of the classroom. complete a task that requires For example: Students share materials among collaboration, such as creating a diagram themselves in a group, but they complete the of the water cycle or coming up with skits activity independently and do not collaborate to illustrate a set of vocabulary words. with one another on problem sets. Alternatively, students help each other solve a math problem. 12 Perspective taking: The ability to consider a situation from a different point of view. Empathizing: The ability to recognize and share another’s emotions. Emotion regulation: The ability to effectively manage and respond to an emotional experience. Social problem solving: The process that an individual goes through to solve an interpersonal problem. This may involve applying aspects of perspective taking, empathizing, or emotion regulation to a social situation. 38 CHECKLIST: OTHER ASPECTS OF EDUCATIONAL QUALITY 39 Overview The Teach Primary classroom observation tool is accompanied by a checklist that assesses other aspects of educational quality and inclusion. While the use of this checklist together with the classroom observation tool is suggested, its use is not mandatory. The aim of this checklist is to 1) identify the number of students with disabilities in the classroom; 2) capture elements related to the accessibility of the physical environment; and 3) capture other elements related to educational inclusion, such as the availability of learning and teaching materials for all students. This data can be leveraged together with results from the classroom observation tool to provide a more comprehensive vision of the quality and inclusion of the education provided to students. The checklist includes a set of questions that enumerators should complete before and after conducting the observations. If the checklist will be used, enumerators should share this with teachers upon their arrival at the classroom. Then, enumerators should complete the first section (Fill in before the classroom observation) at the start of their visit and complete the second section (Fill in after the classroom observation) after class has concluded. Some of the questions in the second section will require input from the teacher; these questions have been highlighted via asterisks (*) within the checklist. A subset of the questions within the checklist focus on capturing the approximate number of students with a disability within the classroom. These questions have been adapted from the Washington Group Short Set on Functioning (WG-SS). We recommend using the original set of items (Washington Group/UNICEF Child Functioning Module-Ages 5–17 years) and its related protocols to collect more precise data on the number of students with disability in the classroom. 40 Checklist: Other Aspects of Educational Quality The following checklist is a proposed addition to be used together with the Teach Primary Classroom Observation Tool; while its use together with the classroom observation component is suggested, it is not mandatory. The aim of the checklist is to assess additional elements related to educational quality, including but not limited to the accessibility of the physical environment. Items indicated with an asterisk are to be asked to the teacher. *items to ask teacher Fill in before the classroom observation School ID Teacher ID Teacher name* Coder ID Date D D M M Y Y Y Y Scheduled class time* Actual class time Time the lesson started Female Male Total class enrollment* Female Male Total students in class Grade/level of class* Subject Female Male Number of adults assigned to work in Total number of teachers (not including number of this classroom* assistants) Total number of assistants Number of assistants providing specialized support to one or a select group of students Other (please specify role): Fill in after the classroom observation Time the lesson finished Female Male How many students have access to the A textbook for the class (e.g., language or mathematics) following resources?* A pencil or pen An exercise book Adapted teaching and learning materials such as Braille or large-font textbooks 41 Female Male Compared with students of the same A lot of difficulty seeing, even if wearing glasses? age, how many students enrolled in A lot of difficulty hearing, even if using a hearing aid(s)? the class have the A lot of difficulty walking or climbing steps? following difficulties?13* A lot of difficulty remembering or concentrating? A lot of difficulty with self-care, such as washing all over or dressing? A lot of difficulty communicating (in his/her usual language), for example understanding or being understood? A lot of difficulty managing their behavior (e.g., hitting students repeatedly, disrespecting the teacher)? What is the official language of instruction?* What proportion of All the students speak this language at home. enrolled students speak the same More than half of the students speak this language at home. language at home as Less than half of the students speak this language at home. the official language of instruction?* None of the students speak this language at home. (check one) What language(s) did the teacher teach in?* How many students Female Male have Individualized Education Plans (IEPs) or receive specialist support?* Female Male Number of adults present in this Total number of teachers (not including number of classroom* assistants) Total number of assistants Number of assistants providing specialized support to one or a select group of students Other (please specify role): 13 These items represent adaptations of the Washington Group Short Set questions. Questions have been modified to facilitate application within the context of the Teach Primary observation protocol and are intended to be posed collectively about all students in the classroom, rather than individually. More detailed data concerning students with disabilities may be obtained through application of the Washington Group/UNICEF Child Functioning Module-Ages 5-17 years. 42 Did you have to end Yes If yes, indicate the following: an observation before Segment: End Time: Reason: the segment was No finished for any reason? Were the students left Yes for _____ minutes unsupervised? No Were any severe Yes Please describe what was observed: negative verbal/ physical interactions No observed? Yes No Are the following resources available in A blackboard and/or whiteboard for the class the classroom? Chalk or a marker available for writing on the board during the lesson Any other teaching and learning materials apart from textbooks (e.g., laboratory equipment/ manipulatives/ Information Communication Technology resources) Can the following be A weatherproof roof observed in the A working electricity connection classroom? Windows Is there sufficient light and contrast for reading what is written on the board from the back of the room? Enumerator Note: read chalk writing on the board from the back of the classroom Is students’ work displayed in the classroom? Other than students’ work, are other posters or charts displayed in the classroom? Students who are not sitting at desks If yes, how many? Can the teacher reach all students’ workspaces/desks in the classroom? A main entrance which wide enough for a person in a wheelchair to enter Can the following be Steps leading up to the classroom observed outside of A proper ramp in good condition usable by a person in a the classroom? wheelchair to access the classroom 43 FREQUENTLY ASKED QUESTIONS 44 Time on Task (0.1a) When the class is in transition, how do I know when the transition has ended? Transitions occur in most classes. As indicated in the manual, consider what most of the students are doing and if the teacher is providing opportunities to learn. A transition officially ends when most students are provided with the next learning activity. For example, if the teacher says, “Take out your workbooks and complete the exercise on page 3,” but students have not yet taken out their workbooks at the time of the snapshot, this is still considered a learning activity as the teacher has provided a learning activity for most students. However, the students may be off task. Remember that actions such as clapping for classmates are not considered a transition as students are encouraging their classmate in their learning. (0.1b) How do I code the snapshot if a learning activity happens concurrently with administrative activities? Even though the teacher is doing administrative tasks (which are considered non-learning activities), it counts as a learning activity if most students are provided with a learning activity. For example, while taking attendance, a teacher may ask students to identify phonemes and put their names on the wall under the first letter of their name. (0.2) Are students off task if they leave the room during the snapshot? They are counted as off task. If they leave the room before the snapshot, observers should not count them as off task. Quality of Teaching Practices (1.1) Must a teacher use students’ names to treat students respectfully? In some cultures, the use of names may not be a common sign of respect. If the teacher does not use names but exhibits other signs of respectful behavior (e.g., the teacher uses terms of endearment to refer to students, uses a respectful form of a word, or uses other respectful language), this may still be scored a high. (1.2a) Is nonverbal communication counted as positive language? Although praise for students may come in many forms, behavior 1.2 only seeks evidence of “positive language.” As such, nonverbal communication, such as clapping or smiling, does not impact the overall score. However, if the teacher makes a statement such as “Let’s give a round of applause,” this is counted toward positive language—not because of the applause, but because the teacher verbally communicates positive language. (1.2b) What is considered “consistent” positive language? Specifically, where should I draw the line betwe en a medium and a high score? Both the consistency and the quality of the comments should be taken into consideration. For example, if a teacher simply says, “You are such a talented group of students” and “Awesome!” in a 15 -minute segment, it is weighted more heavily than the teacher saying “Good” four times. However, if the teacher says “Very good” 7 times, this would constitute a high rating. The following basic thresholds may be used as a loose guide to determine scoring: 0 instances of positive language constitutes a low score, 1–4 instances is a medium score, and at least 5 instances is a high score. (1.3a) If a student needs to go to the bathroom, is that considered a need? Yes, although the examples in the manual have to do with providing materials or emotional support, please remember that these are simply examples and are not comprehensive. Any observable emotional, material, or physical needs are captured here. If a student needs to go the bathroom, that could affect how s/he pays attention during the class, and it is important for the teacher to address. It is important to note, what is not captured here is a student’s need to understand academic content as this is captured when the teacher adjusts the lesson (behavior 4.3). 45 (1.3b) During a partner activity, the teacher rearranges partners to include a student without a partner. Does this count as responding to a student need? Yes, although rearranging students in the classroom is not automatically considered responding to student needs, if a student does not have a partner or group for an activity and the teacher rearranges students to include the student, then this is considered to be addressing a student need. For this to count, there must be an identifiable student need—e.g., the student either has to visibly not have a partner, or the teacher might ask, “Who doesn’t have a partner?,” and the student responds that s/he does not have a partner. (1.3c) Does asking a student if s/he has a specific need automatically count as responding to a student n eed? No, a teacher simply asking if a student has a need does not necessarily count as responding to a student need. For example, if the teacher asks students if they are hungry or tired in an attempt to engage them, this does not automatically count as responding to a student need. However, this is scored a medium if a student indicates the perceived need does indeed exist by indicating s/he is tired or hungry, or if it is clear that a student is tired or hungry. If the teacher addresses the problem by giving that student something to eat, this is scored a high. (1.4a) If a teacher uses explicit language that encourages equal opportunities in the classroom, but this is not reflected in other behaviors exhibited by the teacher, can this still be scored as high? No. If a teacher says, “Now we have heard from a boy, let’s hear from a girl” or “We have not heard from any girls, is there a girl who can answer the question?” but continues with the lesson by only giving opportunities for boys to participate in learning then this would be scored as low. When explicit language that encourages equal opportunities is used within the classroom it is important to observe whether the teacher’s actions also reflect this and if there is any clear indication of gender bias or stereotyping occurring this behavior should take precedence in deciding the overall rating. For example, if the teacher is using language that indicates the promotion of equal opportunities such as, “I would like both boys and girls to answer,” and proceeds with alternating between asking a boy then a girl when there is a clear gender imbalance in the classroom (e.g., 3 boys and 28 girls), then this would be scored as low as boys are clearly being given unequal opportunities to participate in the lesson. (1.4b) What happens if a student makes a remark that expresses gender bias or stereotyping? If a student is seen as making a comment in the classroom that expresses gender bias or stereotyping such as, “Girls can’t do math!” or “Cleaning is not a boy’s job!” and this is not addressed by the teacher, this would be scored as low. If the teacher responds by acknowledging the inappropriateness of the comment but not by challenging it, then this would be scored as medium. Alternatively, if the teacher acknowledges the inappropriateness of the comment and challenges the stereotype by saying, “That is not true, there are many boys who help their parents with cleaning at home,” then this would be scored as high. (1.4c) Can a teacher’s use of resources be counted as an example of challenging gender or disability stereotypes? If a teacher uses resources or examples during classroom activities that challenge gender or disability stereotypes (e.g., text or image that shows a man cooking or cleaning for children), this can be counted as an example of challenging gender or disability stereotypes and would be coded as high. (2.1) How are behavioral expectations different from directions or instructions for an activity? Behavioral expectations focus on the expected behavior during an activity, whereas instructions for an activity focus on the steps required to complete an activity. For instance, the teacher may provide instructions for an activity by saying, “Read the first paragraph and then answer the questions on page 12”—this tells students what they need to do to carry out the activity. On the other hand, the teacher may state behavioral expectations by saying, “If you have any questions, quietly raise your hand”—this sets clear behavioral expectations for students to follow during the activity. 46 (2.3) A student was sleeping in class, but I know he was up all night working. The teacher seems sympathetic toward him and is letting him sleep. Does this affect the score? There are two issues here. First, observers need to be very careful to not let any outside information influence their coding. No matter what the reason, only code and score what is observed during the coding segment. The second issue is the definition of misbehavior. Two factors may be considered when deciding if the student is misbehaving: if the student is causing a disruption in the classroom (distracting students who are trying to pay attention to the lesson), AND if the teacher is bothered by this disruption. If neither the teacher nor the other students are bothered by the student sleeping and it is not disruptive to the flow of the lesson, the behavior 2.3 score could still be a high, depending on the other evidence in the classroom. (3.1) Students are just reading and discussing a story for class. The teacher says, “Today we’re going to talk about [title of story].” Does this count as stating the lesson objective? A lesson objective should directly align with what the class is learning about and doing in their activities. For example, if the teacher says, "Today our topic is multiplication" and the class undertakes an activity specifically focused upon double digit multiplication, this would be scored as medium as the topic is not fully representative of the lesson objective. However, if students are simply learning the basic process of multiplication and undertaking an activity where they solve equations such as 2x2=4, then this would be scored as high as it fully reflects the stated objective. In this case, although the teacher clearly defines the activity for the class, there should be a lesson objective from reading the story (to learn new vocabulary, different parts of speech, etc.). Thus, this statement alone does not count as an explicit definition of the lesson objective. (3.2a) What do we mean by a ‘form of representation’? Forms of representation refer to the way that teachers represent and explain lesson content. Examples of the six forms of representation commonly used by teachers in the primary classroom include the use of: • Spoken language - e.g., The teacher verbally explains content to students. This includes when students listen to the teacher read a text, or when the teacher plays spoken language heard via radio, video or other technologies for the students. • Music - e.g., The teacher uses singing, chanting, and/or other musical forms when explaining content to students. Students may or may not sing/chant along. This includes when the students listen to music and/or sounds heard via radio, video or other technologies. • Text - e.g., The teacher uses letters, words, numbers and/or symbols on the board when explaining content to students. This includes when the teacher has students look at printed text on the board, posters, worksheets, textbooks and/or on a projected screen. • Visual Aides - e.g., The teacher uses pictures, posters, images within books and/or other graphics when explaining content. This includes other visual forms such as drawing on the board, sign language, and images found in video (or other technologies). • Concrete Objects - e.g., The teacher directly refers to and/or manipulates physical items such as objects and/or other materials when explaining content to students. This may include the use of Braille or other tactile based languages. • Movement - e.g., The teacher uses dance, exercise and/or other bodily movements when explaining content to students. Remember that each of the above categories can only be counted once. For example, if a teacher uses visual aids twice within the lesson, for example, by showing students an image of a fish on a card and then later showing students pictures of sea creatures from a storybook, this would still only count as one form of representation. 47 (3.2b) Can one example count as more than one form of representation? Yes. A teacher can use one object to explain and represent lesson content in multiple different forms. For example, a teacher may read out loud from a textbook (spoken language) while students read along in their own textbook (text). A teacher may also hold the textbook up at the front of the classroom and point to a diagram (visual aide) when explaining lesson content (spoken language). If a teacher asks students to read or complete an activity from a textbook without using the textbook to explain lesson content, this is not considered a form of representation. Textbooks count as a form of representation only when teachers use textbook content to explain lesson content. (3.2c) Do all forms of representation need to be displayed or initiated by the teacher? No. A teacher may ask a student to come to the board to do an activity (e.g., draw a triangle with a right angle) and refer to this example in their teaching. In these cases, the example being referred to would also count as a form of representation. (3.2d) Do all forms of representation need to be seen by the whole class? If a teacher explains content to a student during an independent activity or group work-and this can be seen/heard by the observer-then forms of representation used in these instances would also count. For example, if a teacher refers to a picture produced by a student while giving feedback to an individual student, then this can be considered as an example of a visual aid and can be counted as an additional form of representation, provided no other examples of visual aides have been used within the lesson segment. (3.3a) What exactly counts as students’ daily lives and how is it determined to be “meaningful?” The teacher needs to explicitly state how the content is related to students’ lives, rather than observers inferring what they think is related to students’ lives. If the teacher only mentions objects students may encounter in their daily lives, such as “Let’s count the flowers,” this is not considered a meaningful connection. However, if the teacher makes an explicit statement that connects to students’ li ves, such as “Here is a flower like the one that we have in the garden,” that would be an attempt to make a connection. In the above example, barring other evidence, the behavior is scored a medium because it is not explicitly connected to the lesson objective. However, if after making the explicit connection to their own garden, s/he connects the example to the lesson objective by saying, “So if we have 2 gardens with 6 flowers each, how many flowers are there total?,” this constitutes a high score because the teacher explicitly relates the example to both students’ daily lives and the lesson objective. (3.3b) What counts as making connections to other content knowledge? Does recalling what was learned in a previous lesson count as a connection? It may—particularly if the teacher attempts to explicitly connect the lesson to the past content knowledge. For example, if the teacher says, “Remember when we learned the alphabet? Today we will use the alphabet to form syllables,” this is scored a medium because although the teacher explicitly connects new content to past content, s/he only does so superficially. However, if the teacher further explains how to use the alphabet to form syllables, this is scored a high because the teacher not only recalls what was learned in a past lesson and references how it connects to new content but builds upon past content to contextualize new material. If the teacher simply recalls what was learned in a previous lesson without making an explicit connection to the current lesson, this is scored a low. For example, the teacher may say, “Remember how we learned about fractions yesterday? Today we’re going to learn about decimal places.” 48 (3.4a) I’m having trouble with modeling. How do I know when I see it? What should I sp ecifically look for in modeling? Modeling a procedure or skill will mirror an activity that students are asked to do in that lesson or in the near future. Teachers can model by enacting the procedure (showing how to perform a task) or thinking aloud. Cognitive modeling, or a “think aloud,” refers to when a teacher explicitly discusses a thought process or strategy to students by thinking through the challenge aloud (e.g., how to extract important information from a word problem, how to determine theme in a text). When the teacher enacts a procedure, s/he shows all, or some, of the steps in a process for a complete or partial model. Showing the end product could look different across disciplines; however, it essentially gives students an example for which to strive. (3.4b) Does modeling always have to happen before the activity? Although the traditional idea of modeling is when the teacher enacts or thinks aloud a task and then students complete the same activity, modeling does not always have to take place before the activity. Modeling can occur whenever the teacher enacts a procedure or thinks aloud regardless of whether it is at the beginning or end of the activity. For this to occur, it is important that the enacted task or presented think aloud is the same as the task students are expected to perform or have performed. Modeling can occur at the end of class if the teacher walks students through the thinking process as s/he solves a problem. However, simply revealing the answer to a learning activity or a math problem is not considered modeling. (3.4c) What is the difference between an instructional explanation and modeling? To model for students, the teacher needs to perform the task or parts of the task s/he is asking students to do. This is different than giving them directions or explaining an activity as it involves teacher demonstration. The teacher may also demonstrate his/her thinking process as part of the modeling. If the task is to learn the meaning of new words in a text and the teacher simply provides students with a definition of a word, this may contribute to student understanding, but it does not necessarily constitute modeling. An example of modeling is if the teacher were to demonstrate how s/he uses context clues to find the meaning of a word. For example, the teacher may say, “When I don’t know the meaning of a word (in this case, “abrupt”), I reread the sentence, and think about the context, here I read……., therefore I know this means something like sudden or unexpected.” In a math classroom, the teacher may be working with students to estimate lengths in standard units. S/he may explain the length of a centimeter and provide examples of common objects that are a centimeter long—this is part of his/her instructional explanation (3.2). To model, the teacher may show students how to estimate. For example, s/he may show the width of his/her finger is approximately 1 cm and that s/he can use this knowledge to try to estimate the length of a pencil by thinking about (or measuring) how many of his/her finger widths fit along the length of the pencil. (3.4d) I’m still having troubling identifying modeling. Any other tips? To determine whether the teacher has modeled, ask yourself: 1. What is the learning activity? What are students being asked to do or learn? Did the teacher show students what this process or skill looks like? a. Is the thing students are being asked to do a process or a thinking skill? b. If students are asked to do a thinking skill, the teacher has to do a think aloud to be scored a high. If the task is procedural, the teacher should show students all steps in the process. 2. Students then complete a similar activity in that lesson or in the near future. 49 (3.4e) If the teacher models a procedure —for division, for example —but then students are requested to do a different division activity, is it considered modeling? If students do some of the procedure, it could be partial modeling. However, if what students do is unrelated to the procedure shown by the teacher, it does not count as modeling. So, while the activity does not need to be identical, some or all of the procedures modeled need to be included in the activity to be counted as evidence toward modeling. (3.4f) Can students and teachers co-construct a model, or should it be entirely teacher-led? Although we often think of teachers presenting a model for the benefit of the student, some cases arise where modeling is not completely led by the teacher and students may be a part of the process. For example, students and the teacher co-construct knowledge by enacting a procedure together to get to the final product. (4.1a) Can an activity be a way to check for understanding? It is important to stick to the manual by remembering that the teacher needs to ask questions to check for understanding. However, the questions asked by the teacher can be written or verbal, which would be inclusive of an activity. For instance, the teacher may pass out a written quiz to students and check their answers to determine their level of understanding. It is important to note that just giving a quiz is not a check for understanding; the teacher must check students’ answers during the segment for it to count as a check for understanding. Additionally, checking homework (or work that was assigned prior to the observed segment) is not counted toward checking for understanding unless it is clear that the content of the work is related to the current lesson. (4.1b) How do I know what constitutes an “effective” check for understanding? Specifically, what is the difference between a medium and a high score? This behavior is designed to capture the extent to which the teacher makes an effort to check if students understand the content. In an effective check for understanding, the teacher gives individual students the opportunity to show what they know. For example, a highly effective way to check for understanding is by having students come to the board to complete a math problem. This is classified as such because the teacher is able to see the extent to which each individual student understands and is able to complete the task; however, this system does not allow for the teacher to gain information about MOST students’ understanding. What differentiates between a medium and a high score is whether the teacher gains information on MOST students’ understanding over the course of the lesson. For example, a highly effective way a teacher could determine most students’ understanding is by asking them to agree or disagree with statements by showing a thumbs up or down. This behavior does not capture if the teacher does something with that information (this is captured in behavior 4.3). (4.2a) During independent/group work the teacher walks around but does not approach or talk to students at all. Does this count as monitoring? Yes. The teacher can verify students’ understanding without providing comments; at times it is difficult to tell whether the teacher is looking at student work as s/he walks around the classroom. Thus, if the teacher simply walks around the classroom during independent or group work, this is scored a medium. Visual cues should also be taken into account: e.g., the teacher points to students’ work, leans in, or says something observers may not be able to hear. If the teacher is observed monitoring most students in this way, it may be scored a high. 50 (4.2b) The teacher asks students to write the school name and date in their notebooks. They spend a significant amount of time doing this. Does this count as independent work? Yes, writing in their notebooks is a learning task for students who do it independently. Other examples of independent work are: copying down examples from the board when the teacher asks them to and independently completing tasks given by the teacher (e.g., write down a proper noun, draw a picture, complete math equations, etc.). If students are reading something in unison (e.g., the alphabet) and the teacher circulates the classroom and approaches individual students and corrects them, this would count as a whole-group activity. Thus, it does not count as independent/group work. The teacher’s comments are captured under feedback (5.1) and/or adjusting (4.3). (4.3) Most of the adjustment examples are about explanation of content. Are there other ways a teacher could adjust? Although the teacher may effectively adjust by further explaining content, adjusting teaching means giving more opportunities to learn, so the teacher may also do this in other ways. For example, the teacher may give more time to finish a task or provide students who finish early with additional or more advanced tasks. Sometimes an overlap between feedback and adjusting teaching may occur, since the teacher can comment on students’ work and adjust the lesson; however, not all feedback should be counted as adjusting. If a teacher asks a student to present information in a different way to facilitate other students’ understanding (e.g., by writing larger text or speaking louder so other students can hear), this would also count as an adjustment to teaching. The teacher may also adjust by making preparations before the activity in order to accommodate the different needs or learning levels of students. This could include initially providing some students with an easier or more complex task based on their level of understanding. A teacher may also provide an adapted activity for a student who has a particular learning need, for example, through the use of braille for a student who is vision impaired or sign language for a student who has hearing difficulties. (4.3b) When can changing the language of instruction be counted as an adjustment to teaching? Another example of adjusting is changing the language of instruction to facilitate understanding of content. This can occur in response to a student misunderstanding or as part of the teacher’s explanation of content when it is evident that students are having difficulty understanding a concept or skill (e.g., a teacher may ask a question and no student answers, prompting them to change the language of instruction to facilitate understanding). In many multilingual contexts, it is not uncommon for teachers to move fluidly between languages and in these settings the observer needs to be aware to only consider instances where the teacher makes a deliberate attempt to adjust their teaching to facilitate understanding of content. Changing language of instruction as an example of adjustment to teaching can also only be scored as medium unless another example of adjustment is observed that is substantial. This is because observers may not understand the language of instruction that teachers change to and therefore cannot make a judgment on whether it is a slight or substantial adjustment to teaching. (5.1/5.2) There is only one instance where the teacher provides specific comments. Is this enough for scoring a high? Yes, but it depends on the quality of the teacher feedback. If the teacher gives one comment and provides substantive information about what a student did well on or helps clarify misunderstandings, this could be scored as a high. For example, while giving feedback to a student, the teacher may say, “With what number should ascending order start? Biggest or smallest? The smallest. But you started with the biggest. Ascending order starts with smallest, so it has to be like this.” However, if the comment is somewhat vague or in the form of a hint, this would likely be considered a medium. For example, while students are completing independent work the teacher may circulate and tell a student, “Don’t write it there, start writing it from here” or “Leave room for your words to breathe.” These comments are not specific. 51 (6.1) The teacher asks many open-ended questions but does not give students a chance to respond or answers on behalf of students. How should I score this? This is a good example of what may distinguish a high from a medium. If a teacher asks many open- ended questions but does not give students a chance to respond or answers on behalf of students, then the teacher cannot build upon student responses. Thus, this is scored a medium. To score a high, the teacher must ask 3 or more open-ended questions AND at least 1 of these questions should build upon student responses. (6.2/6.3) How do I score this behavior if students are completing a worksheet? How do I know if the worksheet includes a thinking task or not? If it is impossible to determine what is on the worksheet, this would not count toward a thinking task. Remember, you can only score what you see or hear. If you receive some indication of what is on the worksheet (e.g., through the teacher’s instructions or students’ questions), score the task according to the quality ranges outlined in the manual. (6.3) Does answering thinking questions count as performing a thinking task? Answering an open-ended question counts as performing a thinking task if students perform a thinking task with their answer. For example, after reading a story, the teacher could ask, “How do you think the main character felt after losing the competition?” If a student responds, “I think he felt sad because he practiced very hard, and really wanted to win the competition,” this would count as performing a substantial thinking task as the student is explaining his/her thinking. (Refer to the Thinking Task Table for more examples of thinking tasks). However, if the teacher asks an open-ended question and students answer by simply repeating knowledge they have learned, it is not considered a thinking task. For example, the teacher may ask, “What happened after the main character lost the competition?” If a student says, “He cried,” this d oes not count as a thinking task because the student is simply recalling information. Note, that even if you score medium or high for 6.3 due to a student's answer to an open-ended question, this should not count as evidence for 6.2, unless the teacher provides an additional thinking task. (7.1a) Can an open-ended question/task count as providing students with choices? If the teacher asks an open-ended question, this would likely not count as a choice. A thinking task could be counted toward the teacher providing students with choices if the teacher’s instructions explicitly imply s/he intends for students to make a choice. For example, the teacher could say, “Select one of these topics for your essay” or “You can decide which method to use to solve the problem.” (7.1b) How do I code this behavior if there is no clear learning objective? If there is no stated learning objective or if the objective cannot be inferred from the learning activities, this behavior cannot be scored a high. It is scored a medium if a choice is explicitly presented and low if no choice is presented. (7.3) What contributes as evidence toward volunteering? What is captured under this behavior is whether students are volunteering information or simply doing as required in a certain situation. Reciting information in call-and-response fashion or responding in unison to the teacher’s questions in a rehearsed or expected fashion—e.g., all students answering “Yes” when the teacher asks, “Do you understand?”—does not count as volunteering to participate in the classroom. 52 Although the example in the manual is “students raise their hand,” students are also volunteer ing information when they answer questions without being called upon. Therefore, even if they do not raise their hand, if most students volunteer answers in response to the teacher’s questions, this is still scored a high. For example, the teacher may ask, “Who knows the answer?” If most students call out their responses (with or without raising their hand) (e.g., “Me!,” “The answer is 5!,” etc.), then this is scored a high; if only a few students answer, then it is scored a medium. Note, most students need to volunteer throughout the class to be scored a high. (8.1a) What if students do not seem to be making any effort in the class? How do I score this behavior? If the teacher does not acknowledge any effort, even if s/he does not provide any tasks or questions that seem to challenge students or they do not seem to be making any effort, this should still be scored a low. Teachers can always find things students are doing or have done (recent homework, for example) that can earn them acknowledgment for their efforts, even if it seemed to be easy for them. (8.1b) What is the difference between acknowledging students’ effort (8.1) and using positive language (1.2)? Acknowledging students’ effort includes comments that focus specifically on the work and effort of the student. While acknowledging students’ effort may also count as positive language, a comment that constitutes positive language does not necessarily cons titute acknowledging students’ effort. For example, “You have made so much progress on your writing! I can tell you have been practicing!” is a comment that counts toward positive language AND acknowledging students’ effort. “Good job!! You are such a fast writer!” is an example of positive language, but does NOT count toward acknowledging students’ effort. (8.2a) If no mistake is observed, how can I tell the teacher’s attitude toward challenges? As the 3 choices are low, medium, and high, the teacher’s attitude will always fit into 1 of those 3 categories. Any question could be a challenge to students, so watching the teacher throughout the segment should provide enough information to code this behavior. If the teacher has a neutral attitude, does not get angry/impatient, or does not scold or penalize students for making mistakes, then it is scored a medium. (8.2b) The teacher did not scold a student but did seem annoyed. How should I score this? The example of a negative attitude toward challenges includes “scolding,” but it is important to consider other forms of negativity, such as annoyance and impatience. It is important to take cultural differences into consideration (like for 1.1). (8.2c) In scoring positive attitude toward students’ challenges, should I consider the “best” incident or the average over the course of the segment? For this behavior, observers should consider the average attitude of the teacher over the course of the segment. For example, the teacher might show a positive attitude toward students’ challenges when a student makes a mistake and the teacher says, “It’s ok, we’re learning.” However, if besides that isolated incident the teacher consistently and explicitly scolds or becomes impatient with students, this is scored a low or a medium (depending on the balance of incidents over the segment). However, if no clear indications of a negative attitude arise, then one instance of a positive attitude is enough to make the score for this behavior a high. (9.2) How could a teacher promote perspective taking, empathizing, emotion regulation, and social problem solving? An example of perspective taking is: A boy gets upset because his classmates excluded him from a game. The teacher encourages perspective taking by explaining to the boy that his classmates might not have known that he wanted to join in the game, and then encouraging him to ask them if he could participate. 53 An example of empathizing is: When a student says the incorrect answer and her classmates laugh, the teacher promotes empathy by saying, “Don’t laugh, remember that we all get answers wrong sometimes and we are all here to learn.” An example of emotion regulation is: When a student is nervous presenting to the class the teacher promotes emotion regulation by saying, “It’s ok to feel nervous when we stand up in front of the class but remember each time you try it gives you more confidence. So, let’s try by taking a deep breath and remember I am here to help you,” thereby providing strategies and support for the student to deal w ith his or her emotions. An example of social problem solving is: There is a problem between two students. The teacher encourages social problem solving by acknowledging the issue, recognizing students’ emotions, and suggesting they brainstorm a solution together. The teacher may also intentionally model interpersonal skills; for example, the teacher may demonstrate how to stand up to a bully. What if I still have a question? Read, read, read the manual and these FAQs. If your question remains unanswered, ask your trainer or email teach@worldbank.org. It is much better to address your question than to make an assumption and incorrectly code an observation segment. 54 “[Teach] is the single most important thing the World Bank has done in the last thirty years.” Eric Hanushek Paul and Jean Hanna Senior Fellow, Hoover Institution, Stanford University “Based on sound theory and robust evidence, Teach provides practical guidance for teachers on observing classroom behavior. In addition, it presents examples of supporting an engaged learning environment that supports all learners. The revision of Teach Primary is hugely welcomed as an essential tool that brings new knowledge on how to measure inclusive teaching practices that will be important for observing learners with disabilities in the classroom and making Teach truly a valuable tool for all learners.” Charlotte Vuyiswa McClain-Nhlapo Lead Social Development Specialist, World Bank Group “Up to this point, the lack of open source, flexible, easy-to-learn observational measures that can be used systematically in classrooms has stood as a major stumbling block in international efforts to improve education. Observing not just what teachers teach, but how teachers teach—it is a critical step for improvement. Teach not only fills this gap but stands apart from typical observational measures in that it can be used systematically but has some flexibility built into to adjust and adapt to cultural variation. The Teach development process has been meticulous, building on a strong theoretical base and decades of empirical research.” Sara Rimm-Kaufman Professor of Education, Center for Advanced Study of Teaching and Learning, Curry School of Education, University of Virginia “Teach represents a major innovation in our efforts to improve education for all. It helps us address crucial questions such as: How can teachers create an engaging and supportive learning environment? How should teachers teach so they can help students develop strong content foundations and critical thinking skills? How can teachers nurture independent, resilient, and socially competent learners? Teach will be catalytic for enhancing learning all around the world!” Oon Seng Tan Director, Centre for Research in Child Development, National Institute of Education, Singapore “The revision of Teach Primary is hugely welcomed as an essential tool that brings new knowledge on how to measure inclusive teaching practices that will be important for observing learners with disabilities in the classroom and making Teach truly a valuable tool for all learners .” Charlotte Vuyiswa McClain-Nhlapo Lead Social Development Specialist, World Bank “Teach provides excellent guidance for observing and rating global classroom instruction. The instrument is impressive not only for its comprehensiveness but also its specificity—naming key classroom practices and describing concrete examples of how those practices occur at different levels of quality. A useful and accessible tool for heads of school, administrators, and even teachers themselves.” Heather Hill Jerome T. Murphy Professor in Education, Harvard Graduate School of Education; Creator of the Mathematical Quality of Instruction (MQI) instrument “Teach provides a practical tool for educators around the world who are serious about improving the quality of classroom practice. Designed specifically for a global audience, Teach builds on a strong research base and has been tested in multiple countries. Although observation protocols have been used primarily to evaluate teaching, their greatest promise lies in the possibility of creating a common instructional vision and providing specific feedback to teachers on how to improve their instruction. Teach will no doubt provide such learning opportunities for teachers and leaders worldwide.” Pam Grossman Dean and George and Diane Weiss Professor, Graduate School of Education, University of Pennsylvania; Creator of the Protocol for English Language Arts Teaching Observation (PLATO) instrument “Teach is a classroom observation tool that has clearly been designed with the realities of the Global South in mind. The clear explanations, well-crafted examples, and FAQs ease interpretation and ensure commonality of understanding between observers. The simplicity of the tool makes it particularly suited for the purpose of monitoring classrooms and also for capturing insights for further improvement in teacher practices. Teach is also the first-ever classroom observation tool that capture teachers’ efforts to foster soft skills.” Sara Ruto Director, People’s Action for Learning (PAL) Network Contact us at teach@worldbank.org and visit us at www.worldbank.org/education/teach