Acknowledgements This guidance note was prepared by Diego Luna Bazaldua, Victoria Levin and Julia Liberman of the Learning Assessment Platform (LeAP) in the Global Engagement and Knowledge Unit, Education Global Practice. The guidance note was produced in November 2020 and may be updated in the future. The team worked under the overall guidance of Omar Arias (Practice Manager). Valuable inputs were received from peer reviewers: Marguerite Clarke, Alison Marie Grimsland, Sachiko Kataoka, Alonso Sanchez, and Tigran Shmis as well as Toby Linden, Laura Gregory, Anna Boni, Diego Angel-Urdinola, Ciro Avitabile, Catalina Castillo Castro, Ruth Karimi Charo, Ning Fu, Ildo Lautharte, An Thi My Tran, and other members of the Education Global Practice who participated in discussions on the paper. This work is sponsored by the Russia Education Aid for Development (READ) Trust Fund. Guidance note on using learning assessment in the process of school reopening NOVEMBER 21, 2020 ______________________ Diego Luna Bazaldua Victoria Levin Julia Liberman Abstract As countries consider how to reopen schools safely in the context of COVID-19, one key question is how to assess students' learning to support learning recovery. The expected magnitude of learning losses, particularly among students with the highest needs, makes it essential for key stakeholders in the education process — policymakers, teachers, school principals, students, and their parents — to determine where students are in their learning trajectory relative to what had been expected prior to the pandemic, so they can adjust instruction and allocate resources accordingly. To collect this information, stakeholders can rely on student learning assessment, which is an essential feedback mechanism in the education system. This note provides key steps that countries with different availability of resources should consider in developing their plans for learning assessment activities to support learning recovery in the context of school reopening. Throughout this note, assessment of student learning is defined as gathering and evaluating information on what students know, understand, and can do to make informed decisions about the next steps in the educational process. In addition, some considerations and country examples for the implementation of high-stakes examinations are discussed. This note concludes with examples of learning assessment activities that countries around the world are planning or implementing during the COVID-19 pandemic. Likewise, this note highlights important lessons that can support resilience to future emergencies and crises. iii Table of Contents Introduction ........................................................................................................................................... 1 I. Key questions and learning assessments to address them ................................................................. 3 II. Key considerations for learning assessment activities ..................................................................... 4 III. Diagnosing returning students' learning levels to guide instruction ............................................... 7 IV. Continuously assessing students to measure and improve learning ............................................. 15 V. Monitoring system-wide learning to inform decision making, including resource allocation ....... 17 VI. Adjusting high-stakes examinations in the context of school reopening ...................................... 22 VII. Alternatives for countries with no reopening plans and for those that have already reopened ... 22 References ........................................................................................................................................... 24 Annex A: Purposes and uses of learning assessment activities .......................................................... 27 Annex B: An in-depth look at diagnostic assessment and its uses...................................................... 30 Annex C: Country examples of learning assessment activities during COVID-19 ............................ 33 Annex D. Timeline of learning assessment activities for countries with moderate or sufficient resources ............................................................................................................................................. 44 iv Introduction As a measure to minimize the spread of the coronavirus (COVID-19) among the population, most countries around the world made the decision to partially or fully close schools in early 2020. This situation resulted in an unprecedented number of students not being able to attend school in person, which at the peak of school closure in late March 2020, affected around 1.6 billion students worldwide.1 School closures have had severe consequences on students' learning opportunities as well as their socioemotional and cognitive development. About half a year later, in October 2020, over 740 million students reside in countries implementing full school closure policies, and over 130 million students reside in countries where schools have not returned to full in-person instruction. As countries consider how to reopen schools safely in the context of COVID-19, one key question is how to assess learning to support learning recovery. During school closures, some students continued to learn through various modalities, such as online learning platforms, television and radio, and paper packets, while others stopped learning altogether. When schools do reopen for in-person instruction, students will return with very different levels of knowledge and skills, with disadvantaged students most likely to exhibit the greatest learning losses. Here, learning loss is defined as the stagnation or decrease in student achievement relative to expected performance attributed to the disruption of in-person schooling either due to seasonal factors (e.g., summer breaks) or emergencies (e.g., the COVID-19 pandemic) (Cooper, Nye, Charlton, Lindsay, & Greathouse, 1996). Based on existing literature on the effects of school closures on learning, World Bank simulations predict that a 5-month-long school closure can result in an average learning loss of 0.6 quality-adjusted years of schooling (Azevedo, Hasan, Goldemberg, Iqbal, & Geven, 2020).2 But there is likely to be high variation around this average learning loss, particularly depending on the extent of learning that took place while schools were closed. As documented before COVID-19, school closures for regular school breaks and previous emergencies and social crises resulted in greater learning losses for more disadvantaged families who have had fewer opportunities to engage in learning at home due to the impact of the digital divide, lack of learning resources at home, or constraints on parental involvement to support the remote learning process (Alexander, Pitcock, & Boulay, 2016). Empirical results from Belgium confirm that the lockdown and school closures during the COVID-19 pandemic produced learning losses of 0.19 standard 1 http://pubdocs.worldbank.org/en/479101585591761971/COVID19-Education-Sector-Brief-March-27.pdf 2 Quality adjustment is defined using the operationalization of the Learning-adjusted years of schooling (LAYS) measure. LAYS is defined as a standardized metric that adjusts the standard years-of-schooling in a country using a measure of learning productivity—how much students learn for each year they are in school, based on students’ achievement in a learning assessment. https://openknowledge.worldbank.org/bitstream/handle/10986/30464/WPS8591.pdf?sequence=1&isAllowed=y 1 deviations in mathematics and 0.29 standard deviations in Dutch language compared to cohorts from previous years (Maldonado, & De Witte, 2020). This note provides key steps that countries should consider in developing their plans to effectively use learning assessment as part of school reopening and learning continuity activities. It is important to situate any learning assessment plans as an integral part of a broader set of policies and systemic initiatives that Ministries of Education and schools are implementing in the context of the school reopening process (Rogers & Sabarwal, 2020; UNESCO, UNICERF, World Bank, WFP & UNHCR, 2020). These school reopening plans would also include policies to protect health and safety, initiatives to prevent learning losses, programs to promote safe and inclusive schools, supportive measures for teachers and school management, provisions for school equipment for learning, and protection of education financing. During the return to school for in-person instruction, children’s health and well-being must be ensured before any learning assessment is implemented in the classroom. To demonstrate their real abilities and competencies during an assessment, children must feel safe in school, (re)adjust to the in-person learning environment and social interactions with peers and teachers, and be emotionally ready to continue with their studies (Kutsyuruba, Klinger, & Hussain, 2015). It is thus expected for teachers to use the first few days of in- person instruction to ensure a nurturing rapport with their students, given the critical role of student well-being and positive school environment in students’ ability to learn. The guidance discussed in this note is aligned with test development activities and standards in the field of learning assessment (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 2014; Downing, 2006; Educational Testing Service, 2014; Greaney & Kellaghan, 2008; Greaney, & Kellaghan, 2012; Kellaghan, Greaney, & Murray, 2009). The expected magnitude and heterogeneity of learning losses makes it essential for key stakeholders in the education process — policymakers, teachers, school principals, students, and their parents — to determine where students are in their learning trajectory relative to what had been expected prior to the pandemic, so they can adjust instruction and allocate resources accordingly. To collect this information, stakeholders will have to plan, design, and administer student learning assessment. This note is organized by key topics relevant for policymakers, teachers, principals, and other education stakeholders interested in monitoring students' learning, allocating resources to support students whose learning has been most affected by the school closures, and ensuring that teachers provide instruction targeted at the right level for each student. At the same time, each of the topics discussed here can support greater resilience of education systems to future emergencies that may interrupt the learning process in the classroom. First, the note lists key questions that policymakers and other stakeholders may have about student learning once schools reopen and the types of learning assessments that can provide the evidence to answer those questions. Second, the note describes the main considerations for selecting learning assessment activities, taking 2 into account constraints on resources (including time until reopening) and the alignment of reopening to the academic calendar. The note then introduces key steps that countries should consider: 1) diagnostic classroom assessment — to be carried out when schools reopen in order to understand where returning students are relative to their expected learning trajectory; 2) formative and summative classroom assessment activities — to be carried out throughout the remaining school year to monitor students' learning progress and the extent of learning recovery, 3) large-scale assessment activities — to monitor COVID- 19-related learning losses and recovery at the system level, whenever resources allow, 4) considerations linked to high-stakes examinations during the pandemic, and 5) learning assessment alternatives for countries where schools continue to be closed indefinitely, as well as for countries that have already restarted in-person instruction. In addition, four annexes are included at the end of the document. Annex A presents a classification of learning assessment types that are used in most national assessment systems. Annex B includes an in-depth review on diagnostic classroom assessment tools. Annex C illustrates how different countries around the world are either planning or already implementing assessment activities during the pandemic and as part of school reopening. Annex D includes a timeline for key assessment activities before and after schools reopen. I. Key questions and learning assessments to address them Assessment of student learning consists of gathering and evaluating information on what students know, understand, and can do in order to make an informed decision about the next steps in the educational process (Clarke, 2012). Student learning assessment can help policymakers and teachers answer the following key questions on student learning throughout the school year: Immediately after schools reopen • What do students know and are able to do when they return to school, and how can teachers align instruction to ensure that all students are learning at the right level? • How can policymakers direct resources to schools, teachers, and students who need them the most? Throughout the school year • Are students making adequate progress in their learning since reopening? • Are students catching up with their pre-COVID-19 learning trajectory and improving once that level has been reached? 3 Different types of learning assessment can address these questions. In general, information on learning tends to come from three types of assessment activities (see Annexes A and B for more details). Box 1 focuses on the two assessment types, which are most relevant in the context of school reopening — classroom assessment (including diagnostic, formative, and summative assessment) and large-scale assessment. The third type of assessment, high-stakes examinations, can also be relevant in certain contexts, particularly for returning students in end-of-cycle grades that will take exams for purposes of certification of school or grade completion, or for admission to subsequent level or the education process or higher education. II. Key considerations for learning assessment activities The key steps presented in this note take into account several critical factors that would guide the selection of appropriate learning assessment activities. These factors include the availability of resources, including financial resources and the technical capacity of the Ministry of Education and assessment agency, and the available time before reopening that can be used for planning and preparing for assessment activities. In addition, the alignment between the reopening schedule and the academic calendar (including any pre-scheduled assessment activities) is also taken into account. Consideration of these factors and recognition of the constrained environment in which key education stakeholders are likely to be operating prior to and during the reopening of schools drives the focus of the note on the priority decisions that stakeholders need to make as part of school reopening and increases the adaptability of the plan to the needs and contexts of different countries. Under non-COVID-19 circumstances, it takes approximately 12 to 18 months to develop, administer, and analyze results of a national large-scale assessment (McCallin, 2006; Roid, 2006)3; under the COVID-19 pandemic, some activities, such as item piloting, may not be feasible to implement while schools are closed, but sufficient time and adequate quality control mechanisms should be in place as part of the item and test development and content review processes. In addition, it is important for Ministries of Education to note that the pandemic and its impacts on the whole education system may produce inaccurate assessment results; this can be mitigated by employing multiple assessment strategies for determining gaps in student knowledge and skills so that teachers, schools, districts, and the Ministry can respond appropriately. Therefore, the planning process should take into account potential contingencies that may delay the assessment process and consider how this may affect the validity of assessment results. 3 Australia’s National Assessment Program — Literacy and Numeracy (NAPLAN) is produced in a timeline of 18 months, approximately. See https://www.nap.edu.au/about/test-development for more information about their national assessment development process. 4 Box 1: Definitions of classroom and large-scale assessment Classroom assessment and system-level large-scale assessment are particularly relevant as schools reopen because they provide key information to teachers and policymakers about students' knowledge and skills. Classroom assessment provides real-time information to support teaching and learning in individual classrooms. Based on their time of administration and intended use, classroom assessments are classified as: • Diagnostic classroom assessment usually takes place prior to any instruction to help teachers determine the extent to which students' knowledge and skills are aligned with curriculum-based expectations. • Formative classroom assessment is administered as part of daily teaching practice in the form of checks for understanding, quizzes, group classroom activities, and homework. It provides ongoing feedback to teachers and students to monitor students' progress towards curriculum learning goals. • Summative classroom assessment tends to happen at the end of a curriculum unit or at the end of the school year to establish whether students achieved the learning goals specified in the curriculum. Information obtained from formative and summative classroom assessment is best used to support teachers to adjust their instruction to the students' level and to provide constructive feedback to students. Classroom assessment provides information about individual student's performance in a specific subject area, and it can be used to provide feedback and guide personalized instruction. Moreover, classroom assessment activities are relatively inexpensive and logistically simple to administer by trained teachers as part of daily instruction. Large-scale assessment4 (including at the national or sub-national level) monitors learning trends at the system level. Large-scale assessment provides information to policymakers and practitioners on the overall performance levels within an education system and the factors that contribute to that performance across the student population and for key subgroups of students. Large-scale assessments are typically based on assessing a sample of students on a few core subjects at regular intervals. Given the standardization involved in these assessments, they are best suited to support informed system-wide decision-making regarding resource allocation and implementation of initiatives to support schools and students. 4 International (also known as cross-national) large-scale assessments are not reviewed in detail in this note since the timing of these studies depends on international agreements and is not aligned to the school reopening process of any specific country. Overall, most international large-scale assessments are implemented every three to five years. Due to the COVID-19 pandemic, the next Programme for International Student Assessment (PISA) study was postponed from 2021 to 2022 after consultations with participating countries. The next Progress in International Reading Literacy Study (PIRLS) is still planned for 2021. 5 Availability of resources The availability of resources is a key factor in determining the menu of options for assessing learning once schools reopen. Critical resources for implementing learning assessments include time until reopening, staff, and financial resources of various institutions (Ministry of Education, Ministry of ICT, assessment agencies) at multiple levels (central, district, school, classroom). Countries with sufficient resources could ideally plan for systematic implementation of both classroom assessment to target instruction and large-scale assessments to assess and monitor learning and support resource allocation at the system level as soon as schools reopen. However, when resources (including time until reopening) are limited, policymakers may only be able to rely on classroom assessment to monitor learning of individual students and at the school level. The use of classroom assessment may present limitations to monitoring system-level learning; thus, countries must understand that there are accuracy tradeoffs when using the results of these classroom assessments to make system-wide decisions to support schools and students. Where the administration of a large-scale assessment is not feasible, decisions about resource allocation to support learning recovery could be based on the results of previous rounds of large-scale assessments or existing administrative data. Alignment of assessment activities with school reopening and school calendar Learning assessment activities will also need to be aligned to the timing of both school reopening and the overall school calendar. Some countries, like Scotland, are planning to reopen or have reopened schools for in-person instruction coinciding with the beginning of their school year. Other countries and local governments (e.g., some states in Brazil or India; see Annex C) are waiting for the attainment of certain health-related metrics before opening their doors, so their reopening might not coincide with the expected academic year. Furthermore, other countries, like South Korea and Israel, have experienced intermittent school closures when an increase in coronavirus cases was observed after schools reopened. Learning assessment activities will first follow the reopening schedule, with diagnostic classroom assessment (and, if resources allow, large-scale assessment) activities carried out as schools reopen (see Figure 1). Aligning these assessments with the reopening of schools will generate the information teachers need to provide personalized instruction and support learning recovery. Following the reopening, and throughout the remaining school year, ongoing formative classroom assessment conducted by teachers as part of daily instruction would support the learning process and provide real-time evidence on whether students are progressing in their learning. Countries like South Korea have been implementing continuous formative assessment in the classroom and online since schools reopened (see Annex C). At the end of the school year, schools may implement summative classroom assessment to determine if students acquired the knowledge and skills required for the next school grade. 6 As for large-scale assessment, the timing of its administration will depend largely on the availability of resources. For some countries like Vietnam, the previously developed plans and logistics for administering a large-scale assessment could coincide with eventual school reopening. These assessments would provide optimal evidence of COVID-19 learning losses and enable resource allocation decisions to support schools and students lagging behind. In other cases, particularly when resources and logistics for the assessment implementation are constrained, large-scale assessment can be conducted later during the school year in order to monitor system-wide learning trends in core curriculum subjects. In most countries, high-stakes examinations tend to occur close to the end of the school year. Countries around the world have made different decisions regarding the administration format, content adjustments, rescheduling, or cancellation of their high- stakes examinations during the pandemic. Because examinations are typically developed to make certification or admission decisions, policymakers and other stakeholders should not use these assessments as an indicator of learning loss. Furthermore, examinations may or may not be aligned in content with the national curriculum and may not be administered to all students in a particular grade, limiting the type of inferences that can be made regarding learning loss during the pandemic. Learning assessment is an essential ingredient in a comprehensive strategy for school reopening. In addition to assembling the necessary resources and aligning the timing of the assessment activities to the school reopening schedule and the academic calendar, it is important to consider learning assessment activities as part of a comprehensive and coherent instructional strategy using flexible and potentially multi-modal in-person and remote instruction with robust contingency plans, as the pandemic may produce intermittent school closures after the initial opening (UNESCO et al., 2020). In addition, sufficient attention to hygiene and safety measures during the administration of in-person learning assessments would minimize risks for students, teachers, and assessment administrators. This is particularly relevant for students in early grades who commonly require one-on-one assessment administration, which increases the duration of close in- person interactions between teachers and students. III. Diagnosing returning students' learning levels to guide instruction To identify individual students' learning status as they return to the classroom, teachers can conduct diagnostic classroom assessment using either assessment tools developed by the Ministry of Education or already-developed classroom assessment tools that may be publicly available. Depending on the time available before reopening, teachers’ assessment competencies, and the available resources, teachers may also develop their own diagnostic assessments. Given that students will be returning to classrooms with very different levels of knowledge and skills, they will need to be 7 assessed on the extent to which they have mastered the subject-level knowledge and skills they are expected to have based on the pre-pandemic learning trajectory. The classroom assessment activities will enable teachers to meet students at their individual level in terms of their learning trajectory and provide personalized instruction. When resources (including time until reopening) are significantly limited, diagnostic classroom assessment on core subjects, such as language and mathematics, can be prioritized to ensure that students acquire the foundational knowledge and skills upon which other skills and competencies would be scaffolded. 5 In contexts where more resources are available, diagnostic classroom assessment can be expanded to all subjects, particularly given the relatively minor logistical requirements. Table 1 summarizes key considerations for implementing diagnostic classroom assessments in contexts with different level of resources, such as financial and technical capacity and planning time until reopening. For illustration purposes, the information in Table 1 shows four ordered resource scenarios as part of a continuum of resource availability for assessment activities. Figure 1: Alignment of learning assessment activities with school reopening and school calendar 5 These core subjects are relevant learning building blocks for the rest of the curriculum. Understanding literacy and numeracy skills of students as they return to school will help teachers adjust their instructional practices to support students’ understanding of the broader curriculum. 8 Even in fragile or conflict settings or settings with extremely limited resources, it is important to communicate to all schools and teachers the importance of conducting diagnostic classroom assessment when schools reopen. Such guidance can be as simple as directing teachers to gauge students’ ability at the beginning of the school year based on the teachers' expectations of what students should know, to more developed and structured guidance to leverage existing tools, such as EGRA or EGMA, to assess knowledge in foundational literacy and math subjects in early grades. As discussed in Annex C, countries like Chile and Kenya have started to work on developing and using diagnostic classroom assessment tools and training materials to support teachers and school principals in monitoring students' learning when schools reopen. The steps below describe the actions for different stakeholders to prepare, administer, and use diagnostic classroom assessments to determine students' learning status and facilitate appropriate instruction. These plans are presented for two resource scenarios described in Table 1 — "extremely limited" and "moderate." Diagnostic classroom assessment in contexts with extremely limited resources For countries with extremely limited resources and/or less than a month until reopening, the focus would be on determining returning students' competencies in core subjects: 1) Ministries of Education distribute to schools existing classroom assessment tools and existing supplementary training materials to administer and score these tools. As part of their activities and products to support teachers, some Ministries of Education may have available classroom assessment tools from previous years. For instance, New Zealand and Finland have available classroom assessment tools to support teachers in their classroom practice. Ministries of Education could use their existing tools for diagnostic classroom assessment when schools reopen; however, given the expected learning losses due to school closures, teachers should receive classroom assessment tools that capture the content of previous grade's instruction. 9 Table 2: Implementation of diagnostic classroom assessment under four resource scenarios Availability of resourcesa for implementation of diagnostic classroom assessment Extremely Scarce Moderate Sufficient Limited Assessment Use existing classroom Use existing classroom Develop new Develop new diagnostic strategy assessment tools for assessment tools for diagnostic classroom classroom assessment core subjects; core subjects. Add assessment tools tools aligned to the otherwise use existing items and tasks to cover aligned to the curriculum for all school tools particularly to key elements in the curriculum. Use grades and subjects. assess foundational curriculum. some content from knowledge and skills. already developed tools, if needed. Subjects Core subjects or core Core subjects. Prioritize core All school subjects. foundational skills subjects and one (e.g., literacy and additional relevant numeracy). subject per grade. If possible, expand to all school subjects. Grades Prioritize early school Prioritize early school As many school All school grades. grades. grades but cover higher grades as possible. grades, if possible. Assessment Archived information Archived information National curriculum. The national curriculum Inputs of existing classroom of existing classroom Additional sources and learning standards. assessment tools. assessment tools. Check of information, such Additional information is Otherwise, publicly for test content as teacher guides, found in teacher guides, available frameworks alignment to the lesson plans, and lesson plans, and (e.g., EGRA or national curriculum. textbooks, consulted, textbooks. EGMA) for as needed. foundational literacy and numeracy skills. Timeline Start as soon as Start as soon as Start at least three Start at least three months possible. Consider possible. Consider time months before before schools reopen. time for logistics. for logistics. schools reopen. Teacher Distribute existing If resources allow, Develop Develop supplementary training supplementary develop supplementary supplementary written guidelines for guidelines to support written guidelines for written guidelines assessment teachers in assessment core subjects’ for assessment administration, scoring, administration, assessment administration, and results interpretation. scoring, and results administration, scoring, scoring, and results In addition, produce interpretation and use. and results interpretation and training tutorials, and interpretation and use. use. instruction and Otherwise, distribute assessment materials existing guidelines. available in printed and digital formats. Note: a Resources include the availability of sufficient financial and human resources necessary for the planning and implementation of classroom assessment activities and time left until school reopening to implement the preparatory steps. 10 Distributing such classroom assessment tools and supplementary training materials to administer and score these tools is particularly critical for early grades, and if there are no such classroom assessment tools already developed based on the national curriculum for these grades, publicly available assessment tools such as Early Grade Reading Assessment (EGRA; RTI International, 2015) and Early Grade Mathematics Assessment (EGMA; Platas, Ketterlin-Gellar, Brombacher, & Sitabkhan, 2014) could be used, with the caveat that EGRA and EGMA may or may not be aligned to the curriculum. Additional guidance and supplementary training materials would help teachers to understand how to score the assessments and to interpret results for guiding personalized instruction. Training on the use of assessment results can also help teachers evaluate students’ understanding and better utilize assessment data to communicate with parents on how students’ learning can be supported at home. It is important to conduct the distribution of assessment materials to schools as soon as possible to allow principals and teachers to familiarize themselves with the assessment materials and plan for their administration once schools reopen. 2) Once schools reopen, teachers administer the existing classroom assessment tools. Before diagnostic assessments are administered, the first several weeks after schools reopen should be focused on re-establishing the classroom culture, ensuring student well- being, and reviewing critical material from foundational subjects covered during the previous school year. After these critical activities, diagnostic assessment administration needs to occur during the first few weeks after schools reopen. Before administration, students and parents should be informed about the purpose of this assessment (i.e., to help teachers and principals understand students' current learning status and implement additional supporting interventions) and that results will not be used for high-stakes decisions. In contexts where teachers cannot access classroom assessment tools, teachers could consider implementing oral questioning and checking for understanding as a diagnostic approach to determine what students know and can do. 3) Teachers score the assessments and interpret the results to guide personalized instruction, as well as communicate the results to school principals, who allocate support and additional interventions to students with greatest need. After administering the assessments, teachers would score them based on provided scoring guides, obtaining a total score, and scores on performance on different assessment sections and specific tasks. Teachers can use the information obtained from these assessments to adjust their instruction according to their students' proficiency level. These results can also be used by teachers as an initial point of reference to monitor learning progress throughout the rest of the school year. Teachers would also communicate individual student scores to school principals, who may use this diagnostic information to provide additional support to teachers and students. Examples of school-level supports include tutoring in core subjects of the curriculum, planning differentiated instruction according to students' needs, and providing additional 11 instruction or complementary learning materials for students who show the greatest need. Any reports of the results to students and parents should be sensitive to the potential for discouragement and the increased risk of dropout; thus, they should focus on the available support that will be provided in the school to aid learning recovery and suggestions of topics to review at home to help students accelerate their learning. These communications can be done one-on-one during parent-teacher meetings rather than sent directly to students' homes. Diagnostic classroom assessment in contexts with moderate resources For countries with moderate resources, the time until school reopening can be used to develop new diagnostic classroom assessments based on the national curriculum and on the timing of school closures (see Annex D for the timeline of activities to align development and distribution of diagnostic classroom assessment tools with school reopening). As noted above, developing such assessment tools for determining the learning status of students' foundational competencies is of the highest priority, particularly for students in early grades. In contexts with greater resources (including more time until reopening), diagnostic assessments can be expanded to more grades and can include more subjects. 1) Ministries of Education identify the curriculum content to be assessed for each assessed subject and school grade based on the previous school grade's curriculum. This classroom assessment development process should start at least three months before schools reopen to permit policymakers and other stakeholders to develop high-quality diagnostic classroom assessment tools (such as brief tests or quizzes) for teachers. If time until reopening is more limited, the Ministry of Education may opt to use existing assessment tools for some subjects, while developing new ones for others. Assessment of core subjects (e.g., literacy and mathematics) can be prioritized and most relevant additional school subjects (e.g., science, history or social studies) assessed as resources allow. The contents to be assessed may be identified by reviewing the curriculum of the last school grade attended by the student, particularly around and since the date of school closure. Availability of resources would determine how many school grades would be covered by the assessment, ensuring that students in the early grades and students in transition grades are assessed as a priority. In addition, teacher guides, lesson plans and textbooks aligned with the national curriculum can be a good starting point to identify learning contents and examples of classroom activities to use to assess students' knowledge and skills. It is important to align the selection of the classroom assessment content with the anticipated learning and skills students would have been expected to exhibit had they concluded the previous school year without school closures. Ministries of Education develop and distribute to schools detailed plans of the classroom assessment tasks and activities to assess the selected curriculum content. 12 This process should start no later than two months before schools reopen. During the classroom assessment development process, Ministries of Education can gather feedback from subject matter teachers and curriculum specialists regarding the assessment specifications, test content, and classroom activities to implement as part of the assessment process. Subject matter experts would work with the Ministry of Education in the development and content review of each diagnostic classroom assessment tool to be provided to teachers. In normal circumstances, these assessment items and assessment tools would be piloted in a sample of students enrolled in the target school grade. Given that this may not be feasible due to the pandemic, it is recommended that the Ministries of Education devote a good amount of time and effort working with assessment experts and subject matter experts, who can review and judge that each item and assessment tool meets content quality standards. Development of detailed guidelines for these assessments will enable the production of rigorous diagnostics, resulting in greater value of this exercise for teachers, students, and other stakeholders. In contexts with limited resources or time constraints, Ministries of Education can focus on creating classroom assessment tools, supporting materials and exercises only for core subjects of the curriculum. For subjects or school grades not covered by these new diagnostic assessments, Ministries of Education may support teachers by listing expected learning progression and learning standards for returning students, with the intention of this summarized information to be used by teachers to produce their own diagnostic classroom assessment tools. After these classroom assessment materials are finalized, Ministries of Education can distribute them to schools and, ultimately, to teachers in printed and/or digital format. For instance, Kenya's National Examinations Council has made all classroom assessment materials available in electronic form in its online portal (see Annex C). 2) Teachers are trained on the administration, scoring, interpretation and use of diagnostic classroom assessment tools at least one month before schools reopen. Training could be done by specialists from the Ministry of Education, assessment experts, or practitioners from teachers' colleges and universities. Teacher training could occur in person with proper health and safety precautions or remotely (for example, over the phone, through distribution of printed materials, videoconferences, e-courses and other online resources) taking into account staff availability and capacity to scale up training in a cost- effective manner. In this process, it is essential for Ministries of Education to set clear expectations for teachers' responsibilities in assessing student learning as schools reopen and throughout the school year (in the case of formative and summative assessments), as well as to provide instructions on where and how to record student classroom assessment results. Approaches that emphasize effective classroom assessment practices that are known to support learning — such as effective constructive feedback mechanisms, instruction adjusted to students' ability level, additional tutoring strategies, and peer-to- peer support — can be shared with teachers as well. To further support teachers in the administration and effective use of these diagnostic classroom assessments, Ministries of Education may consider developing additional guides 13 and online training materials before implementing the teacher training interventions. These materials may include forms or checklists to record each student's starting point and measure progress towards the achievement of curriculum-based learning goals in each subject, information which can then be used to communicate learning advancement to the student, parents, school, and broader community. For instance, Chile's national assessment agency has put training tutorials and additional assessment materials to facilitate the administration, scoring, interpretation and use of the diagnostic classroom assessment tools they have developed for teachers and school principals online (see Annex C for more details). Figure 2 illustrates an adaptation of a teacher’s checklist to be used to prepare the socioemotional diagnostic tool's administration remotely to students in grades 1 to 3 while they are still at home. Figure 2. Teacher Checklist for the remote use of Socioemotional Diagnostic tool. Course: Remote Socioemotional Diagnostic Assessment ✓ Download and review the document Interactive Socioemotional Diagnosis Area administered at Home. Download and review the booklet for teachers on Socioemotional Assessment at Home. Define how the assessment activity will be carried out: in remote class or through caregivers. Inform caregivers and students that the assessment will be carried out and the modality of administration. Prepare digital materials. Coordinate schedule for the assessment with caregivers. In remote class Carry out the assessment with the students, taking notes of the assessment process. Prepare digital materials. Send digital assessment and training materials to caregivers. Through caregivers Indicate to caregivers the timeline and submission process of the drawings and comments. Analyze information received. Note: Adapted from Agencia de Calidad de la Educación (2020) 3) Once schools reopen, teachers administer comprehensive diagnostic assessments of students' knowledge and skills in relevant subjects of the curriculum using the diagnostic classroom assessment tools made available by the Ministry of Education. See description in the previous section Diagnostic classroom assessment in contexts with extremely limited resources. 4) Teachers score and interpret the assessment results for each student in the classroom, and use this information to support personalized instruction, provide 14 constructive feedback, and promote learning recovery. See description in the previous section Diagnostic classroom assessment in contexts with extremely limited resources. IV. Continuously assessing students to measure and improve learning In the weeks and months after schools reopen, students' progress towards specific learning goals can be continuously measured by teachers via formative and summative classroom assessment, which will facilitate adjustment of instruction. In- person classroom assessment practices can help teachers target their instruction in response to students' progress along their learning trajectory. While diagnostic classroom assessment can provide information to identify the learning level of returning students, continuous classroom assessment in the subsequent weeks and months of the school year will provide real-time evidence that students are learning and catching up with their pre-pandemic learning trajectories. The process for supporting teachers to ensure effective implementation and use of formative and summative classroom assessment throughout the school year is, in some ways, similar to that of providing them with support on diagnostic classroom assessment. When resources (including time) are extremely limited, support can focus on core subjects, using existing guidelines and training materials and relying, if needed, on publicly available tools that can be printed and administered in the form of paper-based assessments and oral questioning to check for students’ understanding. In contexts with more resources, the curriculum for core subjects can be reviewed to identify key content for formative and summative assessment; based on that content, formative and summative assessment guidelines and teacher training materials can be developed and provided to teachers. With moderate and sufficient resources, such classroom assessment materials and teacher training can be expanded to include more subjects. The timing of these activities would depend on the capacity of the Ministry of Education and time until reopening. Although it would be ideal for teachers to receive training on formative and summative classroom assessment practices prior to reopening, these activities can also be carried out later on. As noted above, all teachers should be instructed to carry out formative and summative classroom assessment, and teachers teaching core subjects in the early grades should be afforded priority for support and training in implementing these assessment activities. In contexts with sufficient resources, teachers and schools may opt for a combination of computer-based and paper-based assessments. The below key steps apply to countries with moderate or sufficient resources, which have time to prepare and carry out plans and activities on formative and summative classroom assessment practices prior to school reopening (see Annex D for the timeline of activities to align provision of teachers with effective classroom assessment practices with school reopening). 15 1) Ministries of Education identify curriculum content for inclusion in formative and summative assessment guidelines, prioritizing core subjects. This process begins with a thorough content review of the assessed subject, identifying content around key points in the curriculum aligned to the learning progression students are expected to follow during the school year. At a minimum, the identified curriculum content to be included in these assessment guidelines should reflect the learning that students are expected to acquire throughout the school year and key learning goals to be achieved by the end of the school year. Conducting the content review at least three months before schools reopen would allow the Ministry of Education to plan subsequent steps linked to logistics of implementation and teacher training. 2) Ministries of Education develop formative and summative classroom assessment guidelines and design teacher training modules on effective classroom assessment practices. This process should happen at least two months before schools reopen. The design of the training can focus on aspects within classroom practice that can enhance student learning: constructive feedback mechanisms, pedagogical approaches to adjust teachers' instruction and regulate students' learning, and effective use of learning materials (e.g., textbooks, lesson plans, or teacher guides) to accelerate learning. 3) Ministries of Education provide formative and summative classroom assessment guidelines to teachers with various supporting materials and training opportunities. Given the expected learning losses during school closures, assessment guidelines, and curriculum materials from the previous grade can also be included to facilitate teachers' support to students with significant needs. Distributing materials to teachers at least one month before schools reopen would allow them sufficient time to become familiar with the guidelines and plan their classroom implementation. In addition to receiving classroom assessment guidelines, conducting in-person or remote training for teachers could increase the effective use of these materials in the classroom. 4) After school reopening and as teachers and students advance in the coverage of the curriculum, teachers incorporate formative assessment activities to monitor students' ongoing progress towards the achievement of the curriculum-based learning goals. Assessments activities may include both activities in the classroom and at home, and may employ educational technologies. When administered and used effectively, formative assessment can help teachers provide timely feedback to students on their learning progress, to adjust their instruction to teach at the right level, and to allocate additional instructional resources for students struggling with certain learning content. 5) Once teachers and students reach certain curriculum-based learning goals, such as concluding a specific learning unit or reaching a school-year milestone, teachers engage in summative classroom assessment that covers learning content in a 16 cumulative manner. It is expected that students will show an optimal performance in summative assessments once they have acquired the knowledge and skills covered at school and were supported with additional work at home. In the reopening school year, the results of high-quality summative assessment can also inform students, parents, teachers, and principals about students' progress and on the advancement of learning recovery. V. Monitoring system-wide learning to inform decision making, including resource allocation Along with providing information on the learning status and progress of individual students, learning assessment can allow policymakers to monitor system-wide learning trends and make evidence-based decisions to support learning, including via efficient allocation of resources. In the context of school reopening, policymakers need to understand the magnitude of learning losses experienced during school closures at the system level and to identify where learning needs are greatest so that additional resources (financial, human, instructional) can be allocated based on those needs. Large-scale assessment can address such information needs and motivate education policies aimed at improving student learning, as this type of assessment is typically designed for the purpose of monitoring system-wide learning trends and to support evidence-based decision-making. Large-scale assessment could be implemented at the national or subnational level depending on the structure and degree of centralization of the education system. Findings from large-scale assessments can impact education policy through a clear definition of standards for student and education system performance in the months following reopening, curricular reforms that consider the learning loss, the global reallocation or targeted provision of resources, modification of classroom practices, modifications and extensions to the school calendar and school time schedule, or enhancement of teacher training initiatives. Nevertheless, compared to classroom assessment, large-scale assessments require significantly more resources and logistics to implement, limiting their feasibility in resource-constrained contexts. Given the multiple constraints arising from the COVID-19 pandemic, Ministries of Education may find it difficult to plan to administer a large-scale assessment to align with school reopening. Large-scale assessments need significant investment of financial resources, time and technical capacity, both in terms of their preparation and administration. Moreover, the preparatory steps require at least four to six months of time, which may not align with the time left until reopening. If the administration of large-scale assessments is not feasible as part of school reopening, system-level decisions about resource allocation to support learning recovery could be based on the results of previous rounds of these assessments or existing administrative data (for example, disaggregated information about learning when schools were closed in each administrative school district within a country could be a proxy of the impact of the pandemic on learning loss). Annex 17 C discusses the plans and decisions made by the Brazilian state of São Paulo regarding the postponement of its state-level large-scale assessment once schools reopen. Still, in some countries, it may be possible to complete the preparatory steps and administer a large-scale assessment as schools reopen, particularly if plans for such an assessment may have started prior to COVID-19. As discussed in Annex C, countries like Vietnam have implemented prescheduled large-scale assessment studies after schools reopened to monitor learning loss at the system level. In other contexts, the detailed plan below can be used as a guide to plan for a large-scale assessment when sufficient resources can be allocated to this activity (see Annex D for the timeline of activities to align administration of the large-scale assessment with school reopening). Countries planning to administer a large-scale assessment once schools reopen can include contextual questionnaires to capture information about factors that impact student achievement. During the COVID-19 pandemic, these questionnaires could incorporate some questions about students’ health and well-being, coping mechanisms during the pandemic, and the use of remote learning resources while students were at home. For countries that have the resources to implement a large-scale assessment in the context of school reopening, the following seven steps can ensure effective implementation and use of assessment results. Compared to previous years, countries may require adjustments in the implementation of their next large-scale assessment study to reduce logistical complexities to allow for implementation during a pandemic. For instance, after schools reopened in May and June of 2020, South Korea postponed the administration of its annual large-scale assessment, and instead the country's national assessment agency is planning to collect data from a smaller sample of students compared to previous years (see Annex C). 1) Ministry of Education defines the assessment scope and content including determining the level of assessment administration (national/subnational), subjects and grades to be assessed, whether the assessment would be sample- or census-based, and timing of the assessment administration. As the decisions on the level of administration (i.e., national or subnational, including district, city, or province levels), subjects and school grades included in the study impact on resource and logistics requirements for the assessment implementation, making these decisions at least four to six months before schools reopen would allow policymakers and other stakeholders sufficient time to prepare and coordinate during the planning and implementation of the large-scale assessment. While optimal for monitoring learning, the vast majority of countries do not have sufficient resources to implement annual large-scale assessments in all school grades and subjects, even under normal circumstances. In the context of school reopening, policymakers and the national steering committee that oversees large-scale assessment development may opt to simplify the system-level assessment process by 18 constraining it to core subjects, such as language and mathematics,6 and administering the assessment in the same grades as previous large-scale assessments conducted in the country. This would allow the Ministry of Education to reuse an assessment from previous years and to temporally compare achievement and learning losses with respect to the baseline results of pre-pandemic student cohorts, with the caveat that aspects such as the timing of the assessment administration or the hybrid teaching model during the pandemic may have an impact on students' performance.7 Although census-based assessments (those with all the students in the target population taking the assessment) allow for a greater level of disaggregation (and have potential for use as a diagnostic assessment), sample-based assessments (with a representative share of targeted schools and students being assessed) would be more feasible to administer during the pandemic and would still provide the necessary information on system-level learning trends and learning losses to enable resource allocation and other policy decisions. With respect to the timing for the large-scale assessment implementation, countries like Vietnam have been able to implement a national large-scale assessment in the initial weeks after schools reopened for the new school year (see Annex C). However, other countries such as South Korea are opting to postpone administration of large-scale assessments to a few weeks or months after schools reopen to prioritize other initiatives focused on promoting the return of students to the school and supporting teachers, school principals, and other stakeholders in this process. 2) Assessment agency in charge of large-scale assessment development and implementation produces the assessment tools and specifications following the guidelines and decisions made by the Ministry of Education. The test blueprint and content development of large-scale assessment requires the coordinated work and communication between staff within the national assessment agency and subject matter experts, which may involve additional time during the pandemic; therefore, national assessment agencies should plan the assessment content development in advance and try to conclude this work at least two to three months before schools reopen. During the large- scale assessment development process, the assessment agency typically receives feedback from subject matter teachers and curriculum specialists regarding the assessment specifications and content. These specialists may also participate in the review of the assessment content to corroborate its quality, pertinence, and alignment with the national 6 These core subjects are highly related to achievement in the other subject areas and are good predictors of both future achievement and risk of dropout. In addition, where resources allow, national assessments may include other subjects such as science, geography, or history. 7 In the context of large-scale assessments, temporal comparability of test scores allows countries and other stakeholders to monitor learning progress over time for students enrolled in the same school grade and using assessments developed under the same assessment framework. In this case, temporal comparability could allow countries to compare students’ learning loss before and after schools closed. Temporal comparability is commonly met following one of two methodological approaches: (1) tests that share a proportion of common items, or (2) studies where a sample of students are administered the two tests. 19 curriculum. Under non-pandemic circumstances, these items and assessment tools should be piloted in a sample of students enrolled in the target school grade, but this step may not be possible while schools are closed. Due to this limitation, it is recommended that the Ministries of Education devote sufficient time and effort working with assessment experts and subject matter experts who can review and judge that each item and assessment tool meets content quality standards. 3) Assessment agency plans the logistics to implement the assessment. This process includes communication and notification to schools ahead of the assessment administration, provision and printing of testing and administration materials, training of test administrators with emphasis on COVID-19-related health and safety measures to follow during the test administration, and transportation of test administrators and materials to and from schools, among others. For large-scale assessments to be conducted in the first few weeks after reopening, this step should be finalized at least three weeks before schools reopen to allow coordination between the Ministry of Education, the assessment agency, and participating schools. 4) Dissemination plan is prepared in advance, given the urgency to provide necessary information based on assessment results to policymakers and other key stakeholders. It is best to finalize the dissemination plan for assessment results at least three weeks before schools reopen in order to coordinate with key decisionmakers about the kind of information that will be communicated, when, and how. In the context of school reopening, the dissemination strategy for assessment results would prioritize timely reporting to key decisionmakers, particularly policymakers responsible for resource allocation. Such reporting would inform policymakers about the learning status of returning students with respect to the national learning standards. The reporting may also include information on key factors associated with students' achievement and potential learning losses that occurred during school closures. This information can support decisionmakers to allocate resources to schools and students, implement adjustments to the curriculum, or propose extensions of time students are in school.8 5) With all logistics in place, the assessment agency can proceed to implement the system-level large-scale assessment in the participating schools, either in the first few weeks after schools reopen or at a later point in the school year, as it has been done in countries like Vietnam that closed and reopened schools in the middle of the school year (see more detail on country examples in Annex C). The most important factor in the timing of such an assessment during the pandemic is ensuring that it occurs only when safety measures are in place to minimize any potential health risks for students, teachers, school 8 In addition, dissemination plans may include policy briefs describing main findings and recommendations, as well as customized reports for different audiences in a non-technical language. 20 staff, and test administrators. With sufficient resources and planning time, assessment implementation could occur in the first few weeks after schools reopen, but some countries may postpone the large-scale assessment implementation for a few months in order to make sure all resources are in place and logistics requirements are met. As per regular practice of large-scale assessments, test administrators track school and student participation and have checklists and forms to ensure that protocols are followed during the test administration. Where possible, electronic devices such as tablets can be used to record assessment results and thus speed up data management processes. 6) Assessment agency implements data management and conducts data analysis. Timely and rigorous data management and data quality verification procedures can facilitate accurate interpretation of assessment results and ensure that the results properly reflect student performance. This initial data management process can also help the assessment agency to make sense of the student achievement data and plan ways to present key findings to stakeholders. Analysis of large-scale assessment results typically includes descriptive analyses that summarize students' overall performance and the average achievement levels of relevant subgroups, such as achievement by gender, school type, or country region. In this global pandemic, large-scale assessment results can help stakeholders, particularly policymakers responsible for resource allocation, to understand the system-wide magnitude of learning losses and identify resources and interventions to stimulate learning recovery. Information from background questionnaires combined with results from large-scale assessments can also help stakeholders understand individual and community-level factors linked with students' learning loss, which can be used to target scarce resources for the improvement of learning outcomes during and after the pandemic. For large-scale assessments conducted in the first few weeks after schools reopen, this initial report of results to policymakers can happen in the second month after reopening to allow decisionmakers to take immediate action regarding resource allocation and implementation of initiatives to support schools and students. 7) Dissemination of large-scale assessment results in the form of reports, briefs, and guidelines can help teachers, school principals, and other stakeholders support students in their transition back to school. To make the results useful and relevant to teachers and students, dissemination can occur no later than three months after the assessment was administered in schools. In contexts where census-based assessments were carried out, assessment agencies can develop mechanisms to quickly communicate assessment results to teachers to help them understand the performance of each of their students. At the same time, results should be communicated to the broader education system to garner political support on the need-based allocation of resources to schools, teachers, and students. 21 VI. Adjusting high-stakes examinations in the context of school reopening Students in end-of-cycle grades have particular assessment-related concerns due to school closures and the timing of school reopening. While the above discussion focuses on classroom assessment and large-scale assessment as these are relevant for students in all grades and at all education levels, students in end-of-cycle grades may be concerned about the process of certification of their knowledge and skills to enable graduation and selection into the next education level, which usually involves the administration of high- stakes examinations (Clarke, 2012). Over the last months, several countries around the world have continued to administer high-stakes examinations with certain modifications. Some countries, like Ecuador, have adapted their in-person paper-based exam used for secondary school graduation and university admission into an online exam only for university admission, which could be taken either at home on personal digital devices or in testing centers with COVID-19 hygiene and safety measures (see Annex C for more details). Many countries (e.g., China, South Korea, and Mexico) postponed the administration of their high-stakes university entrance exams, allowing policymakers to gain some time to monitor the pandemic and make decisions regarding the exam implementation. In some instances, content covered on the exam has been adjusted: for example, Hong Kong temporarily removed the English and Chinese oral language component from its university entrance exam. Box 2 discusses how these high-takes decisions might be made in the context of school reopening. VII. Alternatives for countries with no reopening plans and for those that have already reopened As the COVID-19 pandemic continues, some countries have decided to keep their schools closed to contain the spread of COVID-19 with no definite timeline for the school reopening process, while other countries have already reopened schools for in-person instruction, including through hybrid models that combine in-person instruction with remote learning approaches. In some of these countries, the reopening process has been stalled or reversed due to outbreaks or rising COVID-19 cases in the community. 22 Box 2: The process for students' certification and selection in the context of school reopening In many countries around the world, high-stakes examinations are typically implemented near the end of the school year to make high-stakes decisions (such as school/grade certification and selection to the next school level). While it is generally not appropriate to use high-stakes examinations to assess learning loss at the classroom and system levels, some countries may choose to make school reopening decisions based on the timing of these examinations in the national school calendar. For example, South Korea started the school reopening process with senior high school students to allow them to prepare for the national university entrance examination (see Annex C). For the critical end-of-cycle school grades, Ministries of Education may evaluate the prerequisites for implementing high-stakes exams in the context of COVID-19 with appropriate health and safety measures. If technology allows, remote approaches to high-stakes exam administration could be considered in countries where there is uncertainty about the return of students to physical classrooms on a full-time basis. At the same time, the implementation of any technology-based solution may be complemented by measures to prevent any unintended negative impacts on educational equity, particularly for students who do not have access to the assessment technology solutions at home or in their local community. As schools reopen and students begin to prepare for the examinations, it is essential for Ministries of Education and National Examination Agencies to provide students, parents, and teachers with timely information about any changes in the examination process. For example, if it is decided that examinations will be administered in a different format (e.g. online or in a different location), or if the content is modified as compared to what students have expected to be covered by the exam, timely communication of this information to students and teachers alongside provision of adequate resources to prepare for this change can ensure that all students, regardless of location, income level, or disability status can safely take the exams, preserving the actual and perceived fairness of this process. Likewise, clear stipulation by Ministries of Education of additional hygiene and sanitation regulations and proper monitoring of adherence to these regulations by assessment agencies, proctors, and students that participate in the examination administration can safeguard the health and safety of everyone involved. With the uncertainty of when full-time schooling in the classroom will resume and the expectation of continued disruption to in-person instruction, all students, including those with limited connectivity and lack of access to devices for remote learning, need to have the resources to prepare for and to participate in high-stakes examinations. For instance, in many countries, governments, the private sector and international organizations have joined efforts to donate or lend learning devices (e.g., computers or tablets) and provide free or low-cost internet access to students and families during the pandemic in order to support remote learning. Finally, it is important for high-stakes examinations, whether in person or online, to adhere to the principles of universal design to allow all students to have equal opportunity to demonstrate what they know and can do. In fact, testing organizations and government assessment agencies in many countries devote considerable resources to ensure that tests can be adapted for students with disabilities. The at-home online versions of high-stakes exams also need to ensure that all students can be provided with necessary accommodations. 23 Countries with no plans for reopening schools may consider complementing remote learning with remote formative assessment strategies to ensure that learning continues and to guide students’ engagement with remote learning initiatives. Remote formative assessment may be implemented using one or more administration modalities depending on the penetration rate of different technologies in the country, including printed materials, landline phones, feature mobile phones, internet, smartphones, tablets, and computers. 9 In low-resource contexts, the use of basic technologies (including printed materials, landlines, and feature mobile phones) may be most appropriate for remote assessment; in these circumstances, remote assessment has proven to be a useful way to engage students in learning activities and to determine students’ learning level.10 Countries with higher penetration rates of more advanced technologies may opt for combinations of basic assessment technologies and online-based assessments delivered through smartphones, tablets, or computers11. Countries that have already reopened schools for in-person instruction can administer learning assessment as part of the learning recovery process, even if Ministries of Education had not prepared comprehensive assessment activities prior to reopening. As solid understanding of what students know and can do provides the necessary information for tailoring instruction to students’ level, learning assessment is vital to support learning recovery. In these circumstances, countries can monitor student learning with formative and summative classroom assessment complemented, when feasible, with large-scale assessment studies. In the longer term, regular learning assessment activities can promote a resilient national assessment system that can respond to future shocks. References Agencia de Calidad de la Educación. (2020). Diagnóstico Integral de Aprendizajes. Agencia de Calidad de la Educación. website: 9 See: https://blogs.worldbank.org/education/are-students-still-learning-during-covid-19-formative-assessment- can-provide-answer 10 Angrist, N., Bergman, P., Evans, D. K., Hares, S., Jukes, M. C., & Letsomo, T. (2020). Practical lessons for phone- based assessments of learning. BMJ Global Health, 5(7), e003030. URL: https://gh.bmj.com/content/bmjgh/5/7/e003030.full.pdf 11 Besides, countries could consider the inclusion of short learning assessment modules as part of household surveys in order to measure student learning at the system level if schools do not reopen in the short term. The use of household surveys (conducted either in person or by phone) as an approach to assess students may be efficient while schools are closed, as it allows policymakers to examine access to and use of remote learning resources as well as providing a rich set of household-level data. 24 https://diagnosticointegral.agenciaeducacion.cl/documentos/Manual_de_Uso_Docente s_Directivos.pdf Alexander, K., Pitcock, S., & Boulay, M. C. (2016). The Summer Slide: What We Know and Can Do About Summer Learning Loss. New York, NY: Teachers College Press American Educational Research Association, American Psychological Association, National Council on Measurement in Education (2014). Standards for educational and psychological testing. Washington, DC: AERA Angrist, N., Bergman, P., Evans, D. K., Hares, S., Jukes, M. C., & Letsomo, T. (2020). Practical lessons for phone-based assessments of learning. BMJ Global Health, 5(7), e003030. URL: https://gh.bmj.com/content/bmjgh/5/7/e003030.full.pdf Azevedo, J.P., Hasan, A., Goldemberg, D., Iqbal, S.A., & Geven, K. (2020). Simulating the potential impacts of COVID-19 school closures on schooling and learning outcomes: a set of global estimates. Washington, D.C.: World Bank. Website: http://pubdocs.worldbank.org/en/798061592482682799/covid-and-education-June17- r6.pdf Clarke, M. (2012). What matters most for student assessment systems: a framework paper. Systems Approach for Better Education Results (SABER) student assessment working paper no. 1. Washington, DC: World Bank. Cooper, H., Nye, B., Charlton, K., Lindsay, J., & Greathouse, S. (1996). The effects of summer vacation on achievement test scores: A narrative and meta-analytic review. Review of educational research, 66(3), 227-268. Downing, S.M. (2006). Twelve steps for effective test development. In S.M. Downing and T.M. Haladyna (Eds.). Handbook of test development (pp. 3-25). Mahwah, NJ: Lawrence Erlbaum Associates, Publishers Educational Testing Service. (2014). 2014 ETS standards for quality and fairness. Princeton, NJ: Educational Testing Service Greaney, V., & Kellaghan, T. (2008). Assessing National Achievement Levels in Education. Washington, DC: World Bank Greaney, V. & Kellaghan, T. (2012). National Assessments of Educational Achievement Volume 3: Implementing a National Assessment of Educational Achievement. Washington, DC: The World Bank Kellaghan, T., Greaney, V., & Murray, S. (2009). National Assessments of Educational Achievement Volume 5: Using the Results of a National Assessment of Educational Achievement. Washington, D.C.: The World Bank. Kutsyuruba, B., Klinger, D. A., & Hussain, A. (2015). Relationships among school climate, school safety, and student achievement and wellâ€?being: a review of the literature. Review of Education, 3(2), 103-135. 25 Liberman, J., Levin, V., & Luna-Bazaldua, D. (2020). Are students still learning during COVID-19? Formative assessment can provide the answer. Website: https://blogs.worldbank.org/education/are-students-still-learning-during-covid-19- formative-assessment-can-provide-answer Maldonado, J. E. & De Witte, K. (2020). The effect of School Closures on Standardized Student Test Scores. FEB Research Report Department of Economics. McCallin, R. C. (2006). Test Administration. In S.M. Downing and T.M. Haladyna (Eds.). Handbook of test development (pp. 625-652). Mahwah, NJ: Lawrence Erlbaum Associates, Publishers Platas, L. M., Ketterlin-Gellar, L., Brombacher, A., & Sitabkhan, Y. (2014). Early grade mathematics assessment (EGMA) toolkit. Research Triangle Park, NC: RTI International. Website: https://ierc- publicfiles.s3.amazonaws.com/public/resources/EGMA%20Toolkit_March2014.pdf Rogers, F. H., & Sabarwal, S. (2020). The COVID-19 Pandemic: Shocks to Education and Policy Responses. Washington, DC: World Bank. Website: https://openknowledge.worldbank.org/bitstream/handle/10986/33696/148198.pdf?seq uence=4&isAllowed=y Roid, G. H. (2006). Designing Ability Tests. In S.M. Downing and T.M. Haladyna (Eds.). Handbook of test development (pp. 527-542). Mahwah, NJ: Lawrence Erlbaum Associates, Publishers RTI International. (2015). Early Grade Reading Assessment (EGRA) Toolkit, Second Edition. Washington, DC: United States Agency for International Development. Website: https://ierc- publicfiles.s3.amazonaws.com/public/resources/EGRA%20Toolkit%20V2%202016.p df UNESCO, UNICERF, World Bank, WFP & UNHCR. (2020). Framework for reopening schools. URL: http://documents1.worldbank.org/curated/en/113901588910605861/pdf/Framework- for-Reopening-Schools.pdf 26 Annex A: Purposes and uses of learning assessment activities Classroom assessment is implemented to provide timely and constructive feedback to students about their knowledge and skills, as well as the specific areas where they need to improve to progress in the attainment of learning goals. At the same time, classroom assessment can inform teachers when and how they should adapt their classroom instruction to their students' needs. Depending on the timing when they occur and their frequency, classroom assessment is classified as: • Diagnostic assessment. This assessment usually happens prior to any instruction to help teachers determine what students know and can do as they transition from a lower to an upper school grade or, in the current context, once schools reopen. The content of this diagnostic assessment is based on what students should know according to the curriculum. • Formative assessment. This assessment occurs throughout the school year to provide ongoing feedback to teachers and students, and to monitor their progress towards curriculum learning goals. • Summative assessment. This assessment tends to happen at the end of a curriculum unit or at the end of the school year to establish whether students achieved the learning goals specified in the curriculum. Whereas classroom assessment is usually implemented by teachers or schools on a small scale, national large-scale assessments are developed by assessment units within national ministries of education or by independent agencies working in coordination with the national Ministry. National large-scale assessments have the role of monitoring learning trends in the national education system in core subjects (such as language and mathematics) proxy for student achievement in other areas of the curriculum, whereas classroom assessment is usually implemented for all subjects of the curriculum. While classroom assessment tends to occur on an ongoing basis, most national large-scale assessments may happen on an annual (or less frequent) basis due to the logistics behind their planning, implementation, analysis, and results dissemination. National large-scale assessments can be either census-based when all students meeting specific inclusion criteria are assessed (such as all students enrolled in a specific school grade), or sample-based when only a representative group of students of the broader target population participate in the assessment; sample- and census-based assessments help to monitor the achievement of learning goals at the system level, but only census-based assessment can be used for the accountability of each school taking part of the assessment process. Additional contextual information may be gathered during a national assessment study to understand how student, family, or school factors are linked to student achievement. 27 Although national large-scale assessments of student learning and classroom assessment represent separate top-down and bottom-up approaches, respectively, to understand students' achievement, policymakers and other stakeholders should plan to use them in combination once schools reopen, particularly because these two approaches provide information at different levels of granularity and allow different stakeholders to take account at the classroom or system levels. It is important to emphasize that these assessment tools are not meant to be used to directly evaluate teachers or schools, although the information from large-scale assessments may help policymakers and Ministries of Education to identify schools that need additional support. High-stakes examinations are assessments that have consequences for students, such as certification of their knowledge and skills or admission into the next grade or school level. Similar to many national large-scale assessments, examinations are designed and implemented by either units within national ministries of education or by independent examination agencies. High-stakes examinations are not developed to determine learning loss, but some countries have made decisions on the plans for school reopening based on the timeline for administering these assessments. For example, South Korea prioritized the return of senior high school students over students in lower school grades, given the importance of these exams in the society. In addition, the use of these exams and high- stakes decisions for students will probably be part of the national discussions in many countries once schools reopen. 28 Table A1. Assessment types and their key differences Large-scale, system-level assessment Classroom National International Examinations Purpose To provide To provide feedback on To provide feedback To select or certify immediate the overall health of the on the comparative students as they move feedback to system at particular performance of the from one level of the inform classroom grade/age level(s), and education system at education system to the instruction to monitor trends in particular grade/age next (or into the learning level(s) workforce) Frequency Daily For individual subjects For individual subjects Annually and more offered on a regular offered on a regular often where the system basis (such as every 3-5 basis (such as every allows for repeats years) 3-5 years) Who is All students Sample or census of A sample of students All eligible students tested? students at a particular at a particular grade or grade or age level(s) age level(s) Format Varies from Usually multiple choice Usually multiple Usually essay and observation to and short answer choice and short multiple choice questioning to answer paper-and-pencil tests to student performances Coverage of All subject areas Generally confined to a Generally confined to Covers main subject curriculum few subjects one or two subjects areas Additional Yes, as part of Frequently Yes Seldom information the teaching collected from process students? Scoring Usually informal Varies from simple to Usually involves Varies from simple to and simple more statistically statistically more statistically sophisticated techniques sophisticated sophisticated techniques techniques 29 Annex B: An in-depth look at diagnostic assessment and its uses Diagnostic assessment is a key tool in the learning and instruction process that can provide information on what students know, and help teachers plan instruction accordingly. Diagnostic assessment usually happens at the beginning of the school year. In the context of the COVID-19 pandemic, diagnostic assessment is intended to be implemented after schools reopen and after students’ safety, hygiene, and well-being is ensured. Some key uses of diagnostic assessments implemented in the classroom are: • To inform teachers about each student’s strengths and areas for improvement in the assessed subject. • To generate inputs for a learning path tailored to each student’s learning level. • To identify students’ misconceptions and to provide constructive feedback. • To identify a baseline measure to track student progress over time. • To serve as a reference point for programs and interventions focused on promoting learning recovery. • To inform teachers’ lesson plans and learning activities in the classroom. In normal (non-pandemic) circumstances, teachers could use a variety of approaches to determine what students know and can do when they start the school year. The most basic and direct form of assessment is checking for understanding by verbally asking questions during lessons. While rich in terms of the information produced, such approach may be time-consuming and might not allow teachers to capture the actual learning level of all students in a systematic manner. In order to have an organized and time-efficient approach to understand the level of knowledge with which students are coming back to the classroom, teachers may complement any oral assessment with the implementation of low- stakes quizzes, multiple-choice questions, or open-ended short-answer questions - these approaches allow teachers to score students’ performance systematically and, where possible, determine students’ learning level with respect to some criterion. A teacher can design these diagnostic quizzes and open-ended questions according to criteria of what is relevant to know about students’ learning levels. In some countries the Ministry of Education, such as Indonesia or New Zealand (see example below), provides these assessment tools to teachers with additional guidelines and materials for scoring and interpreting results. At the same time, teachers can develop additional questions or their own diagnostic assessment tools to gather more information about their students’ learning levels when they return to the classroom. A key quality that diagnostic assessments should have is that they should be easy to score and interpret by the teacher. In New Zealand, for example, the Global Strategy Stage (GloSS) assessment is used to inform whether students are behind or at the appropriate 30 level of knowledge as expected. The GloSS assessment consists of 22 tasks that increase in complexity. Assessment tools can be accessed in digital or printed forms and include student tasks, administration manuals for teachers with instructions and protocols about how to conduct the assessment session, and sheets to score students’ answers and note observations about students’ performance on the tasks. All assessment materials are available in English and MÄ?ori (New Zealand Ministry of Education, 2020). The figure below shows one of the tasks included in GloSS, an addition task. This task is focused on evaluating the approaches that students use to answer a part-whole addition problem. The task includes a description of what the teacher should say when he poses the question to the student, what potential behaviors and strategies are expected from the student, and the stage at which students are depending on the strategy used to respond to the task. Student’s responses to each task are written by the teacher in the corresponding cell of the recording sheets (see figure below). The task score is based not only on whether the student's answer is correct, but it also incorporates a qualitative element on the specific problem-solving strategy used to answer the task. A total assessment score is calculated by adding up the scores of each task. This total score informs the overall mathematics knowledge stage a student is currently at. The teacher can use this information to adjust instruction accordingly and provide additional supports to students who perform below the expected knowledge stage. 31 While GloSS and many other diagnostic assessment tools were developed before the COVID-19 pandemic, they may represent useful resources for policymakers and other stakeholders that would like to determine students’ knowledge and skills when schools reopen in the same country they were developed (New Zealand Ministry of Education, 2020)12. If policymakers want to use already developed diagnostic assessment tools from another country, it is recommended to perform an assessment content review by subject matter experts, review of its technical properties by learning assessment experts, and an adaptation of the tool to the new context before assessing students. 12 New Zealand Ministry of Education (2020). Assessment Resources: GloSS forms. URL: https://nzmaths.co.nz/gloss-forms 32 Annex C: Country examples of learning assessment activities during COVID-19 Chile 13 Implementing diagnostic classroom assessment during COVID-19 In Chile, nationwide school closures were implemented from March 18, 2020, affecting about 3.6 million children. By late September 2020, the great majority of schools remain closed while the Ministry of Education develops protocols for school reopening.14 Schools can submit their intent to reopen to the Ministry of Education for approval; as part of their submission, schools must provide a reopening plan and verify that they have hygiene and safety measures in place for reopening. The Chilean Ministry of Education has granted permission to reopen to schools across the country that have complied with the regulations and safety standards to prevent the spread of the virus. Agencia de Calidad de la Educación, the national assessment agency responsible for the development and administration of the national large-scale assessments in Chile, has produced a new set of student assessment tools available for school leaders and teachers during the pandemic. The assessments and supporting materials (e.g., video tutorials, assessment administration protocols, guidelines for test score interpretation, among others) produced by this agency are meant to be used for formative purposes to guide the remote teaching during the pandemic or to support classroom teaching in schools that have resumed face-to-face instruction. This set of diagnostic tools include a socioemotional questionnaire, as well as reading and mathematics assessments. The socioemotional questionnaires have been developed for students in grades 1 to 12, it measures socioemotional well-being and socioemotional skills. For students in early grades (i.e., grades 1 to 3), the remote socioemotional 13 ¿Qué es el Diagnóstico Integral de Aprendizajes? (n.d.) Agencia de Calidad de la Educación. URL: https://diagnosticointegral.agenciaeducacion.cl/ Diagnostico Integral de Aprendizajes (2020, September 21). Agencia de Calidad de la Educacion. URL: https://diagnosticointegral.agenciaeducacion.cl/documentos/Manual_de_Uso_Docentes_Directivos.pdf Establecimientos podrán aplicar el Diagnostico Integral de Aprendizajes en Modalidad 100& Virtual . (2020, September 1). Agencia de Calidad de la Educación. URL: https://www.agenciaeducacion.cl/noticias/establecimientos-podran-aplicar-el-diagnostico-integral-de-aprendizajes- en-modalidad-100-virtual/ 14 Chile. COVID-19 Situation Report No. 4 (2020, September). UNICEF. URL: https://www.unicef.org/media/83701/file/Chile-COVID-19-SitRep-September-2020.pdf%20.pdf 33 questionnaire can be administered by caregivers at home, whereas students in grades 4 and above can answer a self-administered version of the questionnaire. The reading assessments are for students in grades 2 to 10. The reading assessments developed for each school grade are aligned to the learning goals described in the national reading curriculum. The reading skills assessed by these tools include locating information, interpreting and making connections, and reflection. The mathematics assessments are for students in grades 3 to 10. Similar to the reading assessments, the mathematics assessments are aligned to the national mathematics curriculum. The mathematics topics covered and content specifications vary by school grade: assessments in grades 3 to 7 measure numbers and arithmetic, patterns and algebra, geometry, and measurement and data. Assessments in grades 8 to 10 include numbers, algebra and functions, geometry, and probability and statistics. School use of these tools is voluntary. In order to use these tools, school principals must register their school online, assign one or more teachers in their school as the focal points responsible for the assessment administration, and plan the assessment administration following the guidelines and support materials produced by the assessment agency. All assessments are available in printed and electronic formats for schools that register in the agency online platform. Likewise, the assessments and questionnaires can be administered in a paper-and-pencil format or in a digital format. For students that take the paper-and- pencil assessment version, results have to be uploaded manually to the assessment agency platform. The platform automatically generates results reports with information to guide the interpretation of the results. Assessment results are meant to be used to support instruction and learning during the pandemic, with no external consequences for students and schools. 34 Ecuador 15 Administering high-stakes examinations during COVID-19 Before the COVID-19 pandemic, Ecuador implemented its national high-stakes examination Ser Bachiller for both certification of competencies for completion of secondary education and admission into higher education. Ser Bachiller was composed of 120 items measuring four domains: mathematics, language and literature, natural sciences, and social sciences. Ecuador has a staggered school calendar for students in different regions of the country: students in the Costa region start the school year in April, whereas their peers in the Sierra region begin in September of the same year. Therefore, Ser Bachiller was administered in person in a paper-based format twice a year, depending on the country region where students were located. Ecuador's government decided to close all schools nationwide in March of 2020 as part of the emergency response to the pandemic. Since then, schools have remained closed for in- person instruction in the whole country, affecting directly to more than 5 million Ecuadorian students. By the time school closures were announced, students from the Costa region had already taken their corresponding examination in January of 2020; however, there was uncertainty about how to continue with the high-stakes exams for students in the Sierra region. As a solution and part of the emergency response to the pandemic, the Ministry of Education decided to substitute Ser Bachiller with a capstone project scored by teachers to certify the completion of secondary education. This change in the certification policy left the Ministry of Higher Education, Science, Technology, and Innovation without a critical standardized source of information to select students into higher education. After considering different alternatives, the Ministry of Higher Education decided to implement a new briefer version of Ser Bachiller with 40 items less than the previous version of the exam. The national assessment agency of Ecuador supported the Ministry in the development of this new examination. Moreover, the latest version of Ser Bachiller is taken online rather than in-person. For students without internet connectivity or digital devices at home, the Ministry of Higher Education arranged 15 Ser Bachiller (n. d.) Instituto Nacional de Evaluación Educativa. URL: http://evaluaciones.evaluacion.gob.ec/BI/ser-bachiller/ Examen de Accesso a la Educacion Superior (n. d.). Secretaria de Educación Superior, Ciencia, Tecnología e Innovación. URL: http://admision.senescyt.gob.ec/ 164.246 personas rindieron el Examen de Acceso a la Educación Superior (2020, October 1). Secretaria de Educación Superior, Ciencia, Tecnología e Innovación. URL: https://www.educacionsuperior.gob.ec/se-realizo-la-reprogramacion-del-examen-de-acceso-a-la-educacion-superior- para-aquellas-personas-que-estaban-programadas-y-no-pudieron-rendirlo-en-la-programacion-ordinaria/ Prepárate para rendir el Examen de Acceso a la Educación Superior. Conoce los temarios que debes repasar previo a tu Evaluación (n. d.). Secretaria de Educación Superior, Ciencia, Tecnología e Innovación. URL: https://www.educacionsuperior.gob.ec/wp-content/uploads/2020/07/Temario-preparacion-de-EAES.pdf 35 computer labs in schools and universities to allow students to take the exam with COVID- 19 safety precautions. The Ministry of Higher Education made some online resources available to support students' transition from the paper-based exam to the online exam. For instance, a mock exam was made publicly available to familiarize students with the exam's reduced content, new delivery format and the online platform. Students also had access to guidelines and protocols that explained the allowed and prohibited actions before and during the online exam administration. In addition to students' grades and affirmative action policies for university admission in Ecuador, this new examination has allowed the Ministry of Higher Education to make informed admission decisions during the pandemic. Simultaneously, this change in policies has allowed the government to have one new examination tool for higher education admissions untied to the certification purposes of the previous version of Ser Bachiller. 36 Kenya 16 Administering national large-scale assessment during COVID-19 Kenya closed schools in March 2020, which affected over 17 million students nationwide. While the government had initially decided to keep schools closed until January 2021, in late September, it decided to reopen schools for students in grades 4, 8 and 12 in mid- October 2020. While schools were closed, the Kenya National Examinations Council worked on planning a census-based national large-scale assessment to be administered immediately after schools reopen. The first assessment round was conducted in grades 4 and 8 one week after schools reopened for those specific school grades. Similar assessments are planned for the other school grades when students return to school in January 2021. The national assessment results will be used to provide reliable and valid information to policymakers about students' strengths and weaknesses, so they can implement interventions towards improving learning outcomes at the system level. The national assessment will cover all primary school education from grades 1 to 8. In grades 1 to 3, students will be assessed in foundational literacy (English and Kiswahili) and foundational numeracy. In grades 4 to 6, students will take assessments for the subjects of mathematics, science, English, and Kiswahili. Students in grades 7 and 8 will be assessed in mathematics, English, Kiswahili, social studies, and religious education. The paper-based assessment will be administered at the school level. The assessments and additional supporting tools and materials will be uploaded to Kenya National Examinations Council's assessment portal. Schools will be supported to print and administer the assessment tools. Teachers will undertake the scoring of the assessments and upload the scores on the assessment portal. 17 Once all the scores have been uploaded on the assessment portal, a team of subject specialists, researchers, and data analysts will verify, analyze, interpret the data, and compile a national report on learning. Schools will access their reports online, while the national report will be disseminated through national forums. It is envisaged that the learning assessments will provide feedback to inform policy on 16 Oduor, A. (2020, September 19). Mass ‘tests’ await pupils once schools resume. The Standard. URL: https://www.standardmedia.co.ke/education/article/2001386891/mass-tests-for-primary-schools Dahir, A. L. (2020, October 7) After suspending the academic year, Kenya begins a phased reopening of its schools . The New York Times. URL: https://www.nytimes.com/live/2020/10/07/world/covid-coronavirus#after-suspending- the-academic-year-kenya-begins-a-phased-reopening-of-its-schools 17 In normal circumstances, two or more trained enumerators score students’ answers to open -ended questions following inter-rater reliability protocols to minimize any bias in the scoring process. In order to ensure consistency in the scoring process in the current context, it is highly recommended that teachers receive in-depth training on the administration, scoring and use of these assessments followed by scoring exercises. 37 strategic measures to be put in place to mitigate further the impact of COVID-19 on learning outcomes. Kenya National Examinations Council has emphasized that these assessments will help teachers to understand students' learning status. Kenya National Examinations Council has stated that the results will not be used to rank students or determine their transition to the next grade. 38 São Paulo, Brazil 18 Administering diagnostic, formative and summative classroom assessments during COVID-19 Like other countries in the Latin American region, Brazil decided to close schools in March 2020 in the face of the pandemic; since then, the vast majority of Brazilian schools have remained closed. There have been discussions and plans to allow teachers and students to return to the classrooms in November 2020; meanwhile, different States in Brazil have been conducting school reopening pilot initiatives. Brazil has a robust national assessment system that includes sample- and census-based national assessments conducted nationwide, high-stakes examinations, and supports for teachers to conduct classroom assessments.19 At the same time, the Brazilian education system is decentralized, which allows state and local governments to implement additional educational policies and initiatives, such as state-level learning assessments, to increase quality and address local challenges in their education systems. São Paulo's state Ministry of Education recognizes that learning loss probably occurred during these months. Therefore, as part of the school reopening plans, this state-level Ministry of Education has set guidelines regarding learning assessment initiatives that will take place when students return to the classrooms. Specifically, the Ministry of Education recommends to teachers implementing diagnostic, formative, and summative face-to-face classroom assessments to determine what students have learned and where the learning gaps are. As part of the general classroom assessment strategies and guidelines, São Paulo's Ministry of Education recommends to school staff and teachers that they prioritize the assessment of foundational skills, including reading, writing, mathematical reasoning, communication, and problem-solving skills. The Ministry of Education does not provide explicit recommendations on what type of assessment to develop and administer in their classroom, as long as it is aligned in its content with the Brazilian National Common Core curriculum. Moreover, in order to avoid stress during the first days after returning to school, the Ministry of Education recommends delaying the implementation of any external assessments until the school environment is adapted to the new post-closure context. São Paulo's Ministry of Education also provides tailored assessment guidelines for specific school grades. For instance, it emphasizes the importance of assessment of literacy skills among students in early grades (i.e., grades 1 and 2), recognizing that the development of 18 Consulta sobre minuta de Resolução SARESP 2020 (2020, October 7). Conselho Estadual de Educação. URL: https://desaocarlos.educacao.sp.gov.br/category/diario-oficial/ 19 Guimaraes de Castro, M. H. (2012). Developing the enabling context for student assessment in Brazil. Systems Approach for Better Education Results (SABER) student assessment working paper . Washington, D.C.: World Bank Group. 39 these skills can prevent more significant future learning deficits. Teachers are encouraged to review the national literacy policy produced by the national Ministry of Education, which includes specific guidelines for the instruction of literacy and supporting materials for teachers. In terms of diagnostic classroom assessment, the Ministry of Education suggests assessing the knowledge and skills that students developed while schools were closed. Diagnostic assessments of each student should allow teachers to determine their learning progress in relation to the learning objectives and skills that they were expected to develop while instruction was being delivered remotely. Simultaneously, teachers should inquire about what learning activities students carried out and what difficulties they encountered while they were engaging in remote learning. Regarding the formative assessment guidelines, teachers are encouraged to implement different activities to determine what students know and can do: administration of quizzes and classroom tests, group projects, homework, and portfolios. Results from formative and diagnostic assessments should guide the implementation of school initiatives to promote learning recovery. On the subject of summative assessment, teachers and schools are encouraged to assess curriculum contents actually taught in the classroom during the school year 2020 rather than curriculum-based guidelines, given the disruption. Teachers should consider the exceptional circumstances of this year during the administration of summative assessments, recognizing the efforts of students and school staff to ensure the learning process during the pandemic and preventing the increase in grade repetition and school dropout. Finally, the state Ministry of Education has decided to suspend the implementation of the state-level large-scale assessment Sistema de Avaliação do Rendimento Escolar do Estado de São Paulo (SARESP; Assessment System of School Performance of the State of São Paulo) this year due to the prolonged interruption of in-person school activities for most of school year 2020. This annual assessment has been implemented every year to monitor learning progress at the state level and create educational indicators to provide feedback into the system. 40 South Korea 20 Administering classroom assessment and examinations during COVID-19 Due to the pandemic, South Korea delayed the beginning of its school year, which was supposed to start in March, and then partially reopened schools in May and June of this year, with students required to attend in person once or twice per week. The school reopening process started with senior high school students in May to prepare them for the national university entrance examination. The government postponed the date for the in- person examination administration from late November of this year to early December to provide students with more time to prepare for this assessment. After senior high school students, students in lower grades gradually started to go back to the classrooms with additional safety and hygiene measures in place in the months of May and June. For instance, class times and lunch hours are staggered, and extracurricular activities are not allowed. When students are not physically present in the classroom, they continue their learning at home through online remote classes and additional homework. Because of the digital divide in the country, the government and private sector have allocated funds to provide laptops to disadvantaged students. Under this hybrid learning model implemented in South Korea, student assessments take place both in the classroom and online. Teachers administer paper-based assessments in the classroom when students are allowed to attend schools in-person. Online performance assessments based on students' class participation, online presentations or online discussions, are implemented when classes are done in an online format. Similar to the national university entrance examination, the South Korea's national large- scale assessment has been postponed. The national assessment agency is planning an in- person administration for their next national study but with a smaller sample size compared to previous years. 20 Responding to COVID-19: Online Classes in Korea. A Challenger Toward the Future of Education (2020, June). Ministry of Education. Republic of Korea URL: https://apa.sdg4education2030.org/sites/apa.sdg4education2030.org/files/2020-06/Responding%20to%20COVID- 19.pdf Press release to announce the scoring results of the 2021 University Academic Proficiency Test September mock evaluation (2020, October 1) Korean Institute for Curriculum and Evaluation. URL: http://www.kice.re.kr/boardCnts/view.do?boardID=10023&boardSeq=5051245&lev=0&m=050101&s=kice Strother, J. (2020, May 19). South Korean high school seniors are eager to return to the classroom. The World. URL: https://www.pri.org/stories/2020-05-19/south-korean-high-school-seniors-are-eager-return-classroom 41 Vietnam 21 Administering national large-scale assessment, classroom assessment, and examinations during COVID-19 Vietnam was one of the first countries to respond successfully to the COVID-19 pandemic by rapidly implementing health and safety measures to prevent the spread of the virus. Vietnam's education system responded early to COVID-19 risks soon after the first outbreak in Wuhan, China, by closing all schools for almost three months starting from January 27th, 2020. Vietnam reopened primary and pre-schools in early April 2020 and high schools at the end of April 2020. During the closures for the general education level, the government implemented interventions to support continued learning at home. Specifically, core subjects in select grades were being broadcast on the national television channels: mainly English, math, Vietnamese for Grades 9 and 12. Some (mainly private) schools organized regular e- learning on Zoom and other communication platforms. The Ministry of Education and Training (MOET) issued guidelines on distance learning and accreditation of distance learning on March 25, 2020. As schools reopened, the MOET also streamlined the curriculum and reduced Term II length and prioritized core subjects. Classroom assessment activities, including mid- and final-term assessments, have been conducted either online or in-person in schools. The school year duration was extended until July 15, 2020 and the national high school graduation and university entrance examination was rescheduled from late June to early August of 2020. Vietnam's national large-scale assessment administered in grades 5, 9, and 12, was carried out as one of the assessment programs supported by the Renovation of General Education Project funded by the World Bank. The national assessment had been planned ahead since 2017 and the first round of NLSA was conducted in May 2019, after two years of test development. The second round of the NLSA was being conducted in July 2020. In the context of three months of school closures and social distancing, the pre- and post-closure timeline of the NLSA will bring about insights into the impact of the pandemic on education outcomes. The subjects covered in this national assessment are mathematics, and Vietnamese language and literature tests, conducted at Grade 5; mathematics, literature, physics, chemistry, and biology at Grade 9; and mathematics, literature, history, geography, chemistry, physics, and biology at Grade 12. In addition to the standardized tests, the national assessment included questionnaires to students, teachers, parents and 21 Vietnamese education has adapted to a "new normal" in the fight against COVID-19. (2020, July 15). Ministry of Education and Training. URL: http://rgep.moet.gov.vn/tin-tuc/giao-duc-viet-nam-thich-nghi-voi-binh-thuong-moi- trong-cuoc-chien-chong-covid-19-5009.html The education sector actively implements activities to prepare for the application of new curriculum and textbooks . (2020, July 15). Ministry of Education and Training. URL: http://rgep.moet.gov.vn/tin-tuc/nganh-giao-duc-tich-cuc- trien-khai-cac-hoat-dong-chuan-bi-cho-viec-ap-dung-chuong-trinh-sach-giao-khoa-moi-5010.html 42 school principals to identify additional factors linked to student performance and the potential learning loss. The NLSA data analysis and results are expected to be available in September of 2020. 43 Annex D. Timeline of learning assessment activities for countries with moderate or sufficient resources Assessment Time before reopening (in weeks) Time after reopening (in weeks) type End of curriculum 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 10 11 unit or school year Teachers are MoE develops trained on the Teachers and distribute to administration, administer Diagnostic MoE identifies schools detailed scoring, and score classroom curriculum content for plans of the interpretation comprehensive assessment assessment classroom and use of these diagnostic assessment tasks diagnostic assessments Focus on ensuring students' readjustment to school and activities classroom of students assessment tools and socioemotional well-being MoE develops MoE provides REOPENING formative and formative and Teachers summative summative implement classroom classroom summative Formative & MoE identifies assessment assessment classroom curriculum content for summative guidelines and guidelines to Teachers incorporate formative assessment assessment inclusion in formative classroom design teacher teachers with activities to monitor students' ongoing progress that cover and summative assessment training modules various learning assessment guidelines on effective supporting content in a classroom materials and cumulative assessment training manner practices opportunities Note: Logistics for Some assessment countries MoE identifies curriculum content for assessment, Assessment agency develops test blueprint. Assessment Assessment data Assessment Large-scale administration may school grades assessed, sample vs. census Subject matter experts develop items and administration management and results assessment and findings implement coverage review them in content. in schools data analysis dissemination dissemination LSAs later plan are defined in the school year Diagonally shaded areas show overlapping activities among diagnostic, formative, and summative classroom assessments, and large-scale assessments. 44 45 worldbank.org/leap 46