Project Management Description This document provides a guide for implementors to follow when producing master codes for video segments. A master code is an extension of the tool, as it defines how to interpret the tool in a given context. Master codes include justification for why a behavior or element, as observed in classroom footage, warrants a particular Teach score. reliability as they set the standard for how observers should score a teaching practice, based on the tool’s definition. When learn about the tool, they rely on master codes to understand what practices constitute “high� or “low� based on the in the manual. After the implementors have been trained on the tool and deemed reliable, they prepare the master codes for the local vide during the observer training. The master coding process can take 2-3 weeks and requires a bit of project management coord quality assurance. To begin the master coding process, implementors individually code each video they are assigned to. They discuss their master codes. Each team member is responsible for justifying their draft codes and collectively deciding what t should be. To facilitate this process, each implementor group is divided into teams of 2-3 and assigned a different role: - The quality controller (QC) oversees the quality of master codes; this includes ensuring the codes are consistent, of high qua written. The quality controller oversees a team of 1-2 other implementors to ensure each code includes an in-depth explana specific examples from the videos. - The writer (W) writes out the master code justifications. S/he is responsible for taking detailed notes during the coding working closely with the quality controller to ensure the codes are of high quality and accurately reflect what’s happening in classroom. - The coder (C) is a third optional member of the team. S/he submits their codes to the quality controller and writer. S/he is responsible for coordinating everyone’s timetables to schedule the meeting, keeping time during the meeting, and ensuring master code is submitted to the Teach trainer. Once the codes have been decided upon as a team, the writer proceeds to draft the master codes for clearance by the qualit The coder is responsible for ensuring the final version is submitted to the Teach trainer. Implementors will be responsible for coding a series of 15-minute videos with their teammates. The video assignments, team designated coding dates and final due dates will be determined by the Teach Trainer and will be shared with Implementors w have roles in supporting the management of the master coding (as outlined above). The tabs in this Excel provide templates and managing these responsibilities. Tabs - The Video Library tab provides an example of how the Teach Trainer will organize the video segments, the school and teach subject, grade, and video title. It is also where the Teach trainer will track which videos each implementor is responsible for they'll be coding with, what their role is (QC, W, C), the date the implementors need to have their codes ready (per video), that the final agreed upon code for the video segment is due to the Teach Trainer. - The Ex. Master Code tab provides an example of how master codes should be documented in Excel and justified in Microso segments. Master codes include a detailed score. Master codes are key for e tool’s definition. When observers low� based on the definitions outlined er codes for the local videos to be used oject management coordination and they are assigned to. They then meet to llectively deciding what the final codes ned a different role: are consistent, of high quality, and well- udes an in-depth explanation with tes during the coding sessions and what’s happening in the roller and writer. S/he is also meeting, and ensuring the final or clearance by the quality controller. e video assignments, teammates, ared with Implementors who will each s Excel provide templates for organizing ents, the school and teacher names, the mentor is responsible for coding, who odes ready (per video), and the date el and justified in Microsoft Word. Video No. Video Title Coder 1 - Writer Coder 2 1 Country_01_Shapes KL JB 2 Country_O2_Colors KP CM 3 Country_03_Numbers JB CA 4 Country_04_Reading CA KP 5 Country_05_Music CM AK 6 Country_06_MorningCircle AK KL 7 Country_07_FreePlay CM CA 8 Country_08_Art KP JB 9 Country_09_ActivityCenters JB KP 10 Country_10_StoryTime CA JB 11 Country_11_Science CM KL 12 Country_12_Counting AK CA 13 Country_13_Syllables JB KP 14 Country_14_Sorting KP AK 15 Country_15_Matching AK KL Coder 3 Discussion Date Discussion Time CA Wed, June 10 08:30-10:00 AM EST AK Wed, June 10 11:00-12:30 PM EST KP Thurs, June 11 09:00-10:30 AM EST AK Thurs, June 11 10:30-12:00 PM EST JB Friday, June 12 10:00-11:00 AM EST CA Friday, June 12 11:00-12:00 PM EST KP Monday, June 15 09:30-10:30 AM EST KL Monday, June 15 10:30-11:30 AM EST CM Monday, June 15 11:00-12:00 PM EST CM Tuesday, June 16 09:00-10:00 AM EST KP Tuesday, June 16 10:00-11:00 AM EST KL Tuesday, June 16 11:00-12:00 PM EST CM Wed, June 17 08:30-09:30 AM EST JB Wed, June 17 10:00-11:00 AM EST CA Wed, June 17 11:00-12:00 PM EST Coder Total Videos to Write Total Videos to Code KL 1 6 JB 3 8 CA 2 7 KP 3 8 CM 3 7 AK 3 7 Writer Uploads Mastercode for Panel Online Meeting Link and Instructions Review by Friday, June 12 Monday, June 15 https://worldbankgroup.webex.com/meet/CountryMasterCoding Wednesday, June 17 Thursday, June 18 Friday, June 19 NOTES *Quality Checker Panel Reviews Master Code Draft, Discusses and Writer Finalizes Codes (On or Finalizes any Discrepancies by before) Wednesday, June 17 Friday, June 19 Monday, June 22 Wed, June 24 Quality Checker Reviews all Master Codes & Finalizes Monday June 22-Tues June 23 Thurs June 25- Friday, June 26 District School Name Age Grade Teacher 3-4 years Janet Escuela Escola Primaria Completa de Marracuene 3-4 years Maurice Jones Sigandela 3-4 years Tracy Willis 4-5 years Margaux Waters Escola Primaria Complea Filipe Marracuene 4-5 years Sergio Gonzalez Samuel Magaia 4-5 years Conor Waite Mixed Jeanette Peters Escola Primaria Complea Filipe Marracuene Mixed Angela Lee Samuel Magaia Mixed Alonso Price Cidade de Escola Primaria Completa Unidade 6 years Ashley Randell Maputo 13 6 years Victoria Donovan Cidade de Escola Primaria Completa Unidade 6 years Nancy Waters Maputo 13 3 years Jose Herrera Escola Primaria Complea Filipe Marracuene 3 years Carmen Samuel Magaia 3 years Segment Focus Video Title Shapes Country_01_Shapes Colors Country_O2_Colors Numbers Country_03_Numbers Reading Country_04_Reading Music Country_05_Music Morning Circle/Weather Country_06_MorningCircle Free Play Country_07_FreePlay Art and Coloring Country_08_Art Blocks and Sorting Country_09_ActivityCenters Reading a story Country_10_StoryTime Water and Sand Experiment Country_11_Science Counting Fingers and Toes Country_12_Counting Letters and Sight Words Country_13_Syllables Sorting sticks and rocks Country_14_Sorting Practicing reading sight words Country_15_Sentences SAMPLE MASTER CODE Country Video Code Snapshots e1 b1 b2 b3 b4 Pakistan 3 1A_G3_Poems_PAK 4 H M N/A M Sample Master Code Justification Let’s Recite Poems (PAK_07_BabyElephant) Time on Learning 1st Snapshot: Y-M (4-5 minute) The teacher provides a learning activity for most children. Although the class is transitioning to the next activity teachers is providing the children a learning activity by modeling a behavior as she gestures for the children whil says [03:58], “Look at me.� Most children are on-task, looking at the teacher as she begins to explain the next activity. 2nd Snapshot: Y-M (9-10 minute) The teacher provides a learning activity for most children. She is teaching them the song “Twinkle Most children are on-task, watching and listening to the teacher as she demonstrates the song and repeating th 3rd Snapshot: Y-H (13-14 minute) The teacher provides a learning activity for most children. She is pointing to various body parts and asking the ch name them. Most children are on-task, watching the teacher as she points to different body parts and responding with the co name for the part. The teacher is effective at maximizing time on learning by ensuring most children are provided with a learning a during most of the class and engaged in those activities. Supportive Learning Environment – 4 Behavioral Marker Scores: H, H, H, M The teacher consistently treats all children respectfully. She calls children by their names throughout the segment. Additionally, she never yells, scolds or ridicules children. The teacher consistently treats all children respectfully. She calls children by their names throughout the segment. Additionally, she never yells, scolds or ridicules children. The teacher frequently uses positive language with children. The teacher says [01:19; 11:27], “Very good� at lea as well as [3:29] “Great�, and again says twice [06:12; 12:33] “All of you are good. So, don't shout.� The teacher [14:36], “Ok great, all of you have head.� Note, both the teacher and the children clap in different moments, but teacher does not use positive language in conjunction with clapping this is not considered as evidence towards t behavior. The teacher promptly responds to the children’s needs in a way that specifically addresses all the issues at hand example, the teacher allows a child to go the bathroom [10:15] and while later she says, [10:25] you may go" and again [13:51], "No wait, let them come back first, who are already outside" addressed the issue at hand, even though the children need to wait for their peers to return first. The teacher treats children of all genders with equal regard. There is no evidence of gender bias since and girls to participate equally. However, there is no evidence the teacher challenges gender stereotypes. The teacher is mostly effective at creating a supportive classroom environment where children supported. The teacher makes all children feel welcome by treating them with respect. The teacher frequently uses positive language with the children and while she provides children of all genders equal opportu participate in classroom activities, she does not challenge gender stereotypes. The teacher promptly responds to children’s needs. Positive Behavioral Expectations – 3 Behavioral Marker Scores: M, M, M The teacher sets clear and specific behavioral expectations throughout the lesson though her behavioral expecta are short. For example, she says [00:50], “Sit down students. Your turn will come later. Be seated, [02:38] and [10:20] “Wait for her then you may go.� The teacher acknowledges student behavior that meets or exceeds expectations but is not specific about their e behavior. When the teacher says [12:38] “Who is a good child (asks class)?� “All of you are good children, right? acknowledges superficially children’s behavior. Although the teacher says, “In this class we have to sit quietly a make noise, okay?�, this comment is a reminder of behavior expectations and does not refer specifically to child are demonstrating the behavior in the classroom. The teacher focuses on a mixture of the misbehavior and the expected behavior to address misbehavior example, she says [06:17] “Don’t talk with each other� which focuses on the misbehavior as well as [03:44] “Be s Be straight� which focuses on the expected behavior. Additionally, as she continually redirects misbehavior thro the class her redirections are only somewhat effective. The teacher is not very effective at promoting positive behavior in the classroom. The teacher sets clear behavio expectations throughout the lesson though they are short, and she does not string expectations together. She d acknowledge children’s behavior that meets or exceeds those expectations. When redirecting on a mixture of the misbehavior and the expected behavior and her redirections are only somewhat effective. Facilitation of Learning – 4 Behavioral Marker Scores: M, H, M, M The teacher explicitly states various activities such as [05:48], “Let’s sing a poem�, [04:05] “Now we will play however, she does not state the learning objective of the activities. The teacher’s explanation of content is clear as content includes directions for activities (FAQ 3.2a). The teacher explains the different activities, teaches them the poem, and uses graphic examples with the different pictures o animals at the beginning of the class. The teacher attempts to make a connection that relates to children’s daily lives, but the connection is superficia practicing saying the different parts of the body in English the teacher says [14:38], “Ok great all of you have hea are all parts of body.� practicing saying the different parts of the body in English the teacher says [14:38], “Ok great all of you have hea are all parts of body.� The teacher partially models for the children the process of how they should show one another the picture and animal it is. She does not show a complete model as she doesn’t give the response, which the children are requi do. The teacher is mostly effective at facilitating learning to promote comprehension. Although the lesson objective explicitly stated, the activity the children are doing is explicitly stated. The teacher provides content and does makes a superficial connection between the activities and children’s daily lives. The teacher partially models an activity during the class. Checks for Understanding – 4 Behavioral Marker Scores: M, N/A, H The teacher is somewhat effective at determining a few children’s level of understanding. The few students understanding through the game at the beginning where children identify animals in the pictures and the few children who recite poems. Although near the end of the segment she is the different parts of the body, she is the one pointing to the different parts of the body and the children answe synchrony. While all the children touch their head when she asks [14:28], “What is this?� they are tasked with na the body part not acting it out and they answer in synchrony. There is no observable group or independent work. The teacher substantially adjust teaching for children. When the teacher begins reciting the poem [08:45] “Twinkle twinkle little star…� it appears the children are supposed to be reciting the poem with her same time. However, as the children are not doing very well, she starts again [08:48] and eventually shifts to tea them in a call and response fashion [08:58] to help them learn the poem. Further evidence supports that this wa first time the children were learning “Twinkle twinkle� as earlier in the class the teachers ask the children you remember. Sing Loudly!� and one child starts to sing [07:13] “Twinkle� but seems to have forgotten. earlier in the lesson the secondary teacher slightly adjusts for children by expanding their language when after identifying the bird at a parrot the teacher asks [03:38] “Is it beautiful?� The teacher is mostly effective at checking for understanding. She checks the understanding of does not provide any independent or group activity but does substantially adjust the lesson additional learning opportunities. Feedback – 3 Behavioral Marker Scores: M, M The teacher provides a few superficial comments that help clarify children’s misconceptions. performing the sit-stand game the teacher provides some superficial feedback when she says [05:26], “Be quick active!� While the child is reciting the poem, the teacher prompts the child [06:57] “Do like this� while the child reciting the poem, reminding her to make the motions as well as recite. Later, when the child is reciting “Twinkle twinkle� the teacher again prompts the child, helping remind her of what phrase comes next [10:45]. The teacher provides superficial comments about children’s work that help them identify their success. teacher provides additional prompts [03:07] “Is he right…. Is it crow? Think it again! I think it's crow. Is it parrot? agree� when the child correctly identifies the bird as a parrot these prompts do not provide children with any information about the success, only ensuring that all the children agree. However, at the end of the discourse th teacher says [03:31] “Ok great! They know it is a parrot� which is a superficial comment that helps children iden success. The teacher somewhat effective at providing feedback about children’s work. The teacher provides superficial c that help children to identify what they did well and what they misunderstand. that help children to identify what they did well and what they misunderstand. Critical Thinking – 2 Behavioral Marker Scores: L, M, M There is no evidence the teacher asks children any open-ended questions. All questions have a pre answer such as [03:07], “Is he right?�, [03:13] “Is it crow?�, [12:41] “What is this?�, and [13:17] The teacher provides superficial thinking tasks for the children. While some activities simply build fluency, such a stand game or memorizing poems, other activities require children to provide some analysis of information. For at the beginning of the class the children must analyze pictures to identify animals [02:43] and at the end of the the children must identify the body parts to which the teacher is pointing [12:41]. There is no evidence that children ask open-ended questions; although they do perform the superficial tasks provided by the teacher. The teacher is not very effective at encouraging children to think critically by helping them identify and synthesi relevant information. The teacher does not ask open-ended questions throughout the segment superficial thinking tasks. Children do not ask open-ended questions, though they do perform Autonomy – 3 Behavioral Marker Scores: L, H, H The teacher does not provide children with choices as she decides how learning activities should be completed w providing different options to children. The teacher provides children with opportunities to take on roles in the classroom. Some of the opportunities ar superficial, such as when children come to the front of the classroom to simply recite a poem [06:42]. beginning of the class, a few children are provided with the opportunity to be responsible for parts of a learning as they ask their classmates questions about the pictures [01:26] “What is the name of this animal?� Most children volunteer by raising their hands and shouting [00:19] "Me.. Me.. Me!" While there are points late class where only a few children volunteer throughout the class it appears that most children volunteer most of t The teacher is somewhat effective at promoting autonomy. She does not provide children with provide an opportunity to take on a meaningful role during the lesson. Most children appear to in the activities. Perseverance – 2 Behavioral Marker Scores: L, M, L The teacher does not acknowledge student effort. She does not acknowledge any effort that the children have exerted. While she uses positive language such as “Very good� she does not specifically acknowledge the effort children have exerted. The teacher has a neutral attitude towards student challenges. Although children are not penalized for their mis the teacher does not help them see the value of the challenge either. There is no evidence the teacher encourages children to engage in short- or long-term planning. The teacher is not very effective at fostering perseverance. She exhibits a neutral attitude towards challenges, fa help children plan, and does not acknowledge children’s efforts. e2 b5 b6 b7 la5 su5 la10 su10 la15 su15 3 H L M Y M Y M Y H s transitioning to the next activity the she gestures for the children while she next activity. m the song “Twinkle Twinkle Little Star.� nstrates the song and repeating the lines. arious body parts and asking the children to y parts and responding with the correct are provided with a learning activity heir names throughout the heir names throughout the [01:19; 11:27], “Very good� at least twice don't shout.� The teacher also says clap in different moments, but if the considered as evidence towards this addresses all the issues at hand. For [10:25] "Wait for her first, then ready outside" she has specifically eers to return first. nce of gender bias since she allows boys llenges gender stereotypes. children feel emotionally safe and h respect. The teacher of all genders equal opportunity to teacher promptly responds to though her behavioral expectations me later. Be seated, [02:38] “Be seated� but is not specific about their expected All of you are good children, right?� and n this class we have to sit quietly and not does not refer specifically to children who to address misbehavior in the class. For misbehavior as well as [03:44] “Be seated. inually redirects misbehavior throughout om. The teacher sets clear behavioral tring expectations together. She does not redirecting misbehavior, she focuses only somewhat effective. [04:05] “Now we will play sit-stand�; activities (FAQ 3.2a). The teacher clearly mples with the different pictures of the s, but the connection is superficial. While 4:38], “Ok great all of you have head. These 4:38], “Ok great all of you have head. These how one another the picture and ask what onse, which the children are required to ion. Although the lesson objective is not cher provides clear explanation of hildren’s daily lives. The derstanding. The teacher is able to check a n identify animals in the segment she is asking children to name f the body and the children answer in is this?� they are tasked with naming ns reciting the poem [08:45] ng the poem with her as they recite at the 08:48] and eventually shifts to teaching her evidence supports that this was not the e teachers ask the children [07:06] “Don’t t seems to have forgotten. In additionally, anding their language when understanding of a few children. She lesson to provide children with isconceptions. While the children are k when she says [05:26], “Be quick and 6:57] “Do like this� while the child is when the child is reciting what phrase comes next when she is stuck em identify their success. Although the again! I think it's crow. Is it parrot? Ok I o not provide children with any ever, at the end of the discourse the comment that helps children identify their The teacher provides superficial comments questions have a pre-determined and [13:17] “How many?� tivities simply build fluency, such as the sit- some analysis of information. For example, [02:43] and at the end of the lesson the superficial thinking helping them identify and synthesize hout the segment though she does provide perform superficial thinking tasks. ng activities should be completed without Some of the opportunities are y recite a poem [06:42]. However, at the responsible for parts of a learning activity name of this animal?� While there are points later in the most children volunteer most of the time. children with choices, though she does appear to volunteer to participate any effort that the children have pecifically acknowledge the effort that the are not penalized for their mistakes, planning. tral attitude towards challenges, fails to e3 b8 b9 b10 b11 e4 b12 b13 b14 e5 3 M M L L 4 M N/A H 3 b15 b16 e6 b17 b18 b19 e7 b20 b21 b22 H L 2 L M M 4 H M M e8 b23 b24 b25 e9 b26 b27 b28 2 L M L 2 L M L