Saudi Arabiaʼs Digital and Distance Education Experiences from the COVID-19 Pandemic and Opportunities for Educational Improvement Anna Boni and Laura Gregory Editors Saudi Arabia's Digital and Distance Education Experiences from the COVID-19 Pandemic and Opportunities for Educational Improvement © Ministry of Education of Saudi Arabia This work is a product of the staff of The World Bank. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work and does not assume responsibility for any errors, omissions, or discrepancies in the information, or liability with respect to the use of or failure to use the information, methods, processes, or conclusions set forth. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Nothing herein shall constitute or be construed or considered to be a limitation upon or waiver of the privileges and immunities of The World Bank, all of which are specifically reserved. Rights and Permissions The material in this work is subject to copyright. All queries on rights and licenses, including subsidiary rights, should be addressed to Ministry of Education, Riyadh, Saudi Arabia Cover photo: Ministry of Education, Riyadh, Saudi Arabia Cover design: Cheryl Flood Designs Saudi Arabia's Digital and Distance Education | iii Contents Foreword v Acknowledgments vi Abbreviations vii Executive Summary viii 1. The Study 1 1.1 Objectives and Scope 1 1.2 Background and Context 2 1.3 Methodology 6 2. Enabling Digital and Distance Education 11 2.1 Planning and Communicating Mitigation Measures 11 2.2 Preparing Schools, Teachers, Students, and Parents 12 2.3 Providing for all Students 13 2.4 Monitoring Attendance and Engagement 13 3. New and Adapted Tools and Materials for Education Continuity 16 3.1 The Madrasati Platform 16 3.2 Recorded Lessons on iEN Channels and YouTube 17 3.3 Other Tools and Enrichment Materials 17 3.4 Processes for Developing and Sharing New Teaching and Learning Materials 22 4. Teaching and Learning Practices During the Pandemic 24 4.1 Planning for Learning 24 4.2 Virtual Classroom Practices 25 4.3 Assessing Student Progress and Adapting Teaching 26 4.4 Devices and Internet Accessibility 29 5. Perceived Effects of the Pandemic on Student Learning and Teacher Skills 31 5.1 Assessing Student Learning 31 5.2 Perceptions of Effectiveness 32 5.3 Teacher Skills 33 iv | Saudi Arabia's Digital and Distance Education 6. Engagement and Well-being 38 6.1 School Community 38 6.2 Parental Engagement 39 6.3 Student Engagement 40 6.4 Student Well-being 42 7. Implications for Next Steps 44 7.1 Safe Reopening of Schools 44 7.2 Options for Blended Learning Models 45 8. Conclusions and Recommendations 52 8.1 Driving Purposeful Educational Change 53 8.2 Designing and Acting at Scale, for All 55 8.3 Empowering Teachers 56 8.4 Engaging the Ecosystem 58 8.5 Incorporating Data and Evidence for Ongoing Improvement 60 8.6 Additional Considerations 61 References 64 Appendixes 66 Appendix A. Duration of School Closures by Country 67 Appendix B. Survey Weighting Methodology 68 Appendix C. School Principal Questionnaire Responses 70 Appendix D. Teacher Questionnaire Responses 74 Appendix E. Student Questionnaire Responses 83 Appendix F. Parent Questionnaire Responses 85 Appendix G. Supervisor Questionnaire Responses 89 Appendix H. Virtual Classroom Observation Tool 97 Saudi Arabia's Digital and Distance Education | v Foreword Under Vision 2030, Saudi Arabia has embarked on a path to build a diverse and sustainable economy grounded on human capital—the knowledge, skills, and health investments that help people realize their potential as productive members of society. The recently published “Human Capability Development Program” is testament to the efforts being made in Saudi Arabia to accelerate human capital formation. The World Bank is pleased to be a partner of the Government of Saudi Arabia in these efforts. The COVID-19 pandemic brought unprecedented challenges to the development of human capital in Saudi Arabia, as it did in countries across the world. Saudi Arabia embarked on a unique and fast-paced journey to tackle this challenge, taking innovative actions to mitigate the potentially devastating effects in education through the wide deployment of digital and distance education. The story of Saudi Arabia’s journey to rapid and widespread digital and distance education is one that many countries are interested to learn from. The study between Saudi Arabia’s Ministry of Education and the World Bank, conducted in real time as the pandemic was unfolding, provides a comprehensive analysis of experiences, including the voices of those most affected—the students, teachers, principals, supervisors, and parents. The study reveals innovation at all levels of the education system, particularly among teachers. Most notable is the way in which education technologies have enhanced teaching practices and enriched students’ learning. Also, the opening of virtual classrooms to observation, including by allowing parents to be part of the learning process with their children, is a positive outcome of the experience. This report reflects the commendable efforts of the Ministry of Education to monitor and evaluate its performance during this most challenging period for education service delivery. It highlights the successes of the approach, which can be used to support the development of practice in other contexts, and shares the lessons learned and opportunities to promote innovative learning and teaching for the future. The findings and lessons that have emerged from this study stand to strengthen education and resultant human capital for many years to come. Keiko Miwa Issam Abousleiman Regional Director Regional Director Human Development GCC Countries World Bank World Bank vi | Saudi Arabia's Digital and Distance Education Acknowledgments This report compiles the results of a study requested by H.E. Prof. Hamad Bin Mohammed Al- Asheikh, Minister of Education, and undertaken by the World Bank’s Education Global Practice under the supervision and coordination of the Center for Research on Educational Policy and the General Administration of E-learning and Distance Education of the Kingdom of Saudi Arabia’s Ministry of Education (MOE). The World Bank team was led by Laura Gregory and Anna Boni, and included Mahmoud Abduh A. Elsayed, Larisa Marquez, Hanada Taha Thomure, Sarah Wazzi, Rabab Saleh, Aidan Clerkin, Gerry Shiel, Heather Staker, Thomas Arnett, Clare McGrorry, Alistair Brown, Louise Ruskin, Peter Glendenning, David Millar, and Mohamed Ragheb. The MOE supervision and coordination team was led by Prof. Abdulrahman Mirza, General Director of the Center for Research on Educational Policy, and Dr. Auhood Alfaries, General Supervisor of the General Administration of E-learning and Distance Education, and included Dr. Fawaz Alrouqi, Mrs.Taghreed Al-Rehaili, Dr. Adel Fahad Alrasheedi, and Dr. Ghadah Alkhadim of the Center for Research on Educational Policy, and Dr. Manal Alothman, Dr. Abdulah Al-Mohaya, Dr. Talal Abozandah, and Dr. Nora Mohammed Almuhanna of the General Administration of E-learning and Distance Education. The World Bank team would like to thank the following for their support and contributions to the study and the associated “Blended Learning Design Workshop”: Dr. Muhammed Bin Saud Al- Meqbel (Deputy Minister for General Education), Dr. Abdurrahman Bin Omar Al-Barrak (Deputy Minister for Planning and Development), Prof. Saleh Ibrabim Al-Qassomi (Deputy Minister for International Cooperation), Prof. Ahmed Abdulrahman Al-Juhaimi (National Center for Professional and Educational Development), Dr. Abeer Sulaiman Al-Humaineedy (Digital Transformation and Information Security), Dr. Sami Abdulaziz Al-Shuwairkh (Curriculum Development Center), Hashem Alqahtany (Tatweer Educational Technologies), Dr. Hanaa Abdulrahim Yamani and Dr. Tahani Aldosemani (General Administration of E-Learning and Distance Education), Fahad Alqahtani (Tatweer Company for Educational Services), Dr. Mohammed Al-Dhelaan (General Supervisor of Information Technology), Dr. Hamoud Al-Khemeis (Ministry Agency for International Cooperation), Mr. Yaser Hashem Alqahtany (TETCO), and Ms. Nour Alkouja (Microsoft). Special thanks are extended to the focus group participants teachers, supervisors, school principals, parents, and students for their time, openness, and insights, as well as to the numerous survey respondents, plus the teachers who opened their virtual classrooms for observation, along with their supervisors who received training on the custom-designed observation tool and conducted the observations and follow-up interviews. Without these dedicated educators, parents, and students, this in-depth study would not have been possible. Within the World Bank, the team benefited from the guidance and support of Issam Abousleiman (Country Director), Keiko Miwa (Regional Director for Human Development), Andreas Blom (Regional Education Manager), Yisgedullish Amde (Program Manager), Sameh El-Saharty (Program Leader for Human Development in the GCC). Cristobal Cobo was the education technology advisor to the team and peer reviewer of the report, along with Zaki B. Khoury. Additional contributions to the study and associated Blended Learning Design Workshop were provided by Mariem Sghaier, Emma Etori, Maria Elena Paz Gutzalenko, Khlood Al Hagar, May Bend, and Luma Akkad. Cheryl Flood designed the cover. Saudi Arabia's Digital and Distance Education | vii Abbreviations EdTech education technology HCI human capital index ICT information and communications technology K — 12 kindergarten to grade 12 LAYS learning-adjusted years of schooling MOE Ministry of Education MOH Ministry of Health MOS measure of size (total enrollment in school) MS Microsoft NCEPD National Center for Educational Professional Development NALO National Assessment of Learning Outcomes NGO non-governmental organization OER open educational resources PISA Programme for International Student Assessment PPS probability proportional to size PIRLS Progress in International Reading Literacy Study SD standard deviation TALIS Teaching and Learning International Study TIMSS Trends in International Mathematics and Science Study TOTMOS total enrollment of all schools viii | Executive Summary Executive Summary Faced with the crisis of the COVID-19 pandemic in March 2020, Saudi Arabia embarked on a journey to adapt the way in which schooling operated, enabling a continued education for children across the country. This was a unique journey, and one that will have lasting impacts on education in Saudi Arabia. The World Bank studied this journey in detail over the 2020–21 school year, as the pandemic was underway. This report compiles the results of this study and provides a comprehensive review of the experiences of digital and distance education in Saudi Arabia, along with an analysis of opportunities for future educational improvement. The study aimed to answer three main questions. Firstly, how well did Saudi Arabia provide for, and achieve, continued education of K–12 students during the COVID-19 pandemic? Secondly, what were the strengths of Saudi Arabia’s response to the COVID-19 pandemic in K–12 education? And finally, what are the opportunities for educational improvement following the digital and distance education experience? To answer these questions, a variety of data gathering instruments were designed and implemented by the World Bank team. The data sources included focus groups; surveys of nationally-representative groups of school principals, teachers, students, parents, and supervisors; virtual classroom observations; and interviews with key personnel. Analyses of these data informed the conclusions and recommendations presented in this report, which were reviewed by global experts in the fields of education and education technologies. How well did Saudi Arabia provide for, and achieve, continued education of K–12 students during the COVID-19 pandemic? This study found that Saudi Arabia has clearly succeeded in providing for continued K–12 education during the COVID-19 pandemic, particularly from the start of the 2020–21 school year. This included multiple ways of engaging students, and a host of new tools, providing teachers with a unique opportunity to experiment and innovate to meet their students’ needs. In developing the digital and distance education, attention was rightly paid to both the technology aspects and the educational aspects, with the teacher and student interaction prioritized, along with the necessary communication aspects that were needed to bring about rapid and widespread change. Almost all students (98 percent) were able to access the Madrasati platform—the main vehicle for virtual schooling—during the 2020–21 school year. This is an impressive achievement given the scale and speed of the rollout and the fact that alternative arrangements were also made available such as in-person school visits, recorded lessons, and printed materials. In terms of regular attendance and engagement throughout the year, log-in data for the Madrasati platform shows that at least 75–85 percent of students were logged in to the Madrasati platform on average in any given school week, and others were also logged in directly to Microsoft (MS) Teams (not through the Madrasati platform), plus some students who needed face-to-face support attended school in-person occasionally or on a regular basis. A redistribution of assessment weightings—with less weight placed on end- of-semester examinations and more on smaller activities such as homework and assignments—may have helped to encourage student engagement throughout the year. Therefore, attendance and engagement appears to have been very strong during the digital and distance education. Schools were responsible for ensuring that all students had access to education and this was monitored at the regional and district level. To determine the success of these efforts and to quantify the Saudi Arabia's Digital and Distance Education | ix situation of student attendance and engagement with digital and distance education (or alternative arrangements) across the full school year at the national level, it would be important to put in place a system to regularly gather and report on relevant data beyond the Madrasati log-in numbers. Such monitoring could help to identify changes over time and differences (such as geographic or gender) that might suggest the need for targeted interventions and additional support. In addition, it has not been possible to determine changes in student learning—whether the digital and distance education led to increases or decreases—due to the absence of an externally marked, standardized assessment of student achievement before and during the pandemic. Most teachers and school principals indicated high confidence in the effectiveness of the digital and distance education offered in terms of students’ academic achievement and skills development, with some variation indicating that the experience may have been better for some than for others. For example, 68 percent of teachers felt that academic achievement was better this year than previous years, while 64 percent of principals of girls’ schools and 57 percent of principals of boys’ schools felt the same. This stands in contrast to the learning losses seen around the world among other countries that pivoted to digital and distance education for periods of time during the COVID-19 pandemic. However, the different style of learning was not always easy for students. For example, most students (76 percent) found that following classroom instructions or understanding lessons could sometimes or regularly be hard, and two-thirds (67 percent) of students missed seeing their friends and teachers a lot. More than half of students (52 percent) felt that they would have learned more if they could have returned to in-person schooling (more so for elementary students at 58 percent). Overall, there is a strong desire to return to in-person learning, as stated by 85 percent of supervisors, 75 percent of school principals, 72 percent of parents, and 63 percent of teachers. In addition, more than three- quarters of students reported that they are looking forward to returning to school to see their friends (92 percent), do better in their schoolwork (85 percent), meet with their teachers, get help with their schoolwork, and participate in school activities. There was high satisfaction with the resources and tools provided for digital and distance education with most stakeholders wanting to see a continued use of some of the tools and products. For example, 75 percent of parents were satisfied or very satisfied with the quality of distance education over the last school year, and 98 percent of teachers said that they would find it useful to continue to use the Madrasati platform. There is also high demand for more digital content and further improvements to the quality of the digital content. What were the strengths of Saudi Arabia’s response to the COVID-19 pandemic in K—12 education? Many strengths within Saudi Arabia’s response to the COVID-19 pandemic in K–12 education have been identified in this study. These will be important to build on and to share with other countries. Some of the key areas of strength include the following. 1. Rapid provision of distance education at scale for education continuity. Digital and distance education was rolled out across the country at a large scale and at an impressively fast rate, with recorded lessons available through satellite television and online for over 6 million K–12 learners, made possible by previous investments in digital and distance education. 2. Prioritized student-teacher connections. The student experience was placed at the center of the digital and distance education provision through the Madrasati platform. Flexibility in the system allowed for varying situations, such as students needing to attend school to meet with teachers and obtain printed materials. x | Executive Summary 3. Availability of a wide range of tools and resources. An extensive range of teaching and learning tools, curriculum materials, and enrichment resources were made available for students and teachers. 4. Regular monitoring of user data and feedback loops to improve tools and services. Monitoring of Madrasati usage and user interaction with the platform was regular and resulted in proactive actions and decisions. In addition, feedback from users was regularly sought and acted upon. This resulted in high rates of user satisfaction. 5. Low-tech alternatives for students without internet access. Students without access to the Madrasati platform were able to utilize dedicated satellite TV channels for recorded lessons and regularly visit schools to receive printed materials and support from teachers. 6. Provision of special distance education. Satellite TV channels were put in place for students with disabilities and special educational needs, along with school visits as needed. 7. High satisfaction with professional training and support. There has been high and consistent satisfaction across all stakeholder groups with the training and support provided to access the tools and resources provided through the Madrasati platform. 8. Good use of teacher communities of practice and coaching. The use of in-school coaching with assigned school digital learning focal points and the creation of professional communities of practice have enabled teachers to share good practices. 9. Frequent and clear communication with stakeholders. Communication has been strong, with clear and timely guidance that allowed all stakeholders to understand their roles. 10. Enhanced parental engagement. Distance learning arrangements established through the Madrasati platform have significantly strengthened parental and family engagement in their children’s education. This enhanced connection between schools and families, if continued, will be a powerful driver for educational improvement and stands to be one of the greatest outcomes of the digital and distance education experience in Saudi Arabia. What are the opportunities for educational improvement following the digital and distance education experience? The study identified several strategies, practices, and procedures arising from the digital and distance education that could be further developed to bring about lasting improvements in children’s schooling experiences and learning outcomes. These recommendations extend beyond education technologies (EdTech) and distance education because improvements in other areas of education are critical for the success of digital and distance education as well as in-person learning in schools. The recommendations are summarized in table 0.1 and detailed in chapter 8. Some of the key recommendations include the following. 1. More explicitly target policies and resources on disadvantaged and struggling students. With a prolonged period of distance education (over one school year), some students will be behind where they should be, and there is a significant risk of widening disparities given the proportion of students who did not regularly log in to Madrasati or who were unable to engage fully for various reasons. Schools should identify and support these students through the regular in- person class schedules and, in some cases, with additional targeted supports such as tutoring. Additional resources to schools with large proportions of disadvantaged and struggling students may be required. Saudi Arabia's Digital and Distance Education | xi 2. Plan for long-term purposeful blended learning models that best suit existing practices and conditions, such as the “Flipped Classroom” and “Flex and Enriched Virtual.” When considering blended learning models for the future, be sure to have a clear purpose and vision for the change and only make changes where they are needed or will be beneficial. Two blended learning strategies most suitable to Saudi Arabia’s current context are the “Flipped Classroom” and “Flex and Enriched Virtual.” This study and materials from the accompanying “Blended Learning Design Workshop” provide definitions and examples of these strategies. 3. In the short-term, ensure that a plan for providing devices and connectivity to all students who do not have them is feasible and enacted as a matter of urgency. For the long-term, set policies for teacher and student access to devices and connectivity. If blended models such as the Flipped Classroom are expected in the next or future school year, the MOE should make sure that all students have access to the required devices and internet connections at home. A plan to make this happen should be developed (beyond support from charitable organizations) or, if already developed, shared with stakeholders to ensure that it is feasible and that all parties are on board. One of the most cited areas for development raised by stakeholder groups in the nationally- representative surveys conducted as part of this study was to improve the infrastructure internet connection and access to devices to improve accessibility and reduce inequality. Around 41 percent of students reported having a lot of trouble connecting, and 45 percent reported having a little trouble connecting. A long-term plan will also be needed to ensure all students and teachers have access at home and in school to devices and connectivity. 4. Target professional learning to reduce variability in teacher performance. While many teachers have increased their professional skills to support student achievement, there is a need to reduce variability in teacher performance to effectively plan and deliver high-quality learning experiences for all students. This means that professional learning should be targeted to where it is needed, especially for schools and teachers that have not yet reached good performance levels in digital and distance education. 5. Support and require teachers to recognize and meet the socioemotional needs of their students. Distance learning challenges relating to students’ physical and mental well-being were raised by stakeholders in this study, including feelings of isolation, boredom, laziness, lack of physical activity, eye strain from too much time on a device, and concern over the lack of social interaction with their peers. Going forward, it will be important to support the knowledge and understanding of teachers to adapt their lessons and expectations to effectively meet the socioemotional needs of students, particularly at the time of transition from distance to in-person learning. Careful transition planning will be required to ensure that students are well supported as they return to in-person classes, for example by establishing a well-being framework and formalized support for schools providing evidence-based intervention to guide practice. 6. Ensure continuity in national assessments to provide crucial information on overall levels of student learning. A stable system of externally marked, standardized national student assessments will allow valid monitoring over time. The National Assessment of Learning Outcomes (NALO) has been implemented for several years covering different subjects but has not yet been implemented over the COVID-19 pandemic phase to allow a comparison against previous years. Because of this, the effect of the pandemic on student learning could not be assessed directly. This should be a priority for the next school year, implemented at an appropriate time (after students have settled into the new school year), ensuring that the instrument used can be matched to a recent pre-COVID-19 baseline. As the NALO now moves to a census basis, it will be important to allow a significant period of stability in the national assessment (avoiding frequent design and xii | Executive Summary methodology changes) to reap the benefits of monitoring changes in learning levels over time. 7. Improve data and monitoring of student attendance and engagement with distance education and in-person schooling. More accurate monitoring of student attendance in virtual settings would be needed in the future. Currently, complete data is not available at a central level to enable a judgment on the degree to which all students received an adequate education across the full 2020–21 school year. Systems of central monitoring and reporting on student attendance in class—both in-person and at a distance—are needed to ensure that students are not left behind and that there is a better understanding of the extent of attendance and engagement across the country. Through the survey written responses, teachers, parents, school principals, and supervisors also raised the issue of monitoring attendance and engagement. Any new arrangements for gathering and monitoring student attendance should connect with existing data systems to streamline the process and not present an additional burden to teachers. In addition to the above key recommendations, the study highlights three points worthy of consideration at this point in Saudi Arabia’s education journey, beyond the digital and distance education initiatives. Firstly, with a redesign of curricula and teacher assessment currently underway, it will be important to ensure a move away from predetermined lesson delivery to aligning instruction to students’ current learning levels, skills, and goals. This concept of skill building, starting from where each child is at, needs to permeate throughout the education system and be well understood by all of those involved in educating children and young people. Secondly, the expected time for curricular reforms to be effectively embedded into the teaching practices and learning experiences of all children and young people should be realistic and set accordingly. International examples suggest that around 5 to 10 years for these types of reforms are necessary. Finally, there is a rare opportunity now to build on the strengths created by the need for distance education in response to the COVID-19 pandemic, the most important of which is likely to have been the successful communication processes and parental engagement strategies that were implemented. The opening up of the classroom to parents through virtual classes, and the increase in communication between schools and families, bodes well for long-term benefits, particularly if this is sustained as students and teachers return to in-person schooling. In summary, Saudi Arabia’s journey toward a virtual school model, rolled out at scale from the start of the 2020—21 school year to provide continued education during the COVID-19 pandemic, has been remarkable. Innovation has taken place at all levels, particularly among teachers. There is now a greater awareness of how education technologies can support and enhance teachers’ work and students’ learning experiences. Beyond that, there are now new methods and styles of communication and better awareness of what is happening in classrooms across the country. The resultant innovations and disruption to business-as-usual in Saudi Arabia’s schools will affect change in children’s learning experiences well beyond the pandemic. Saudi Arabia's Digital and Distance Education | xiii Table 0.1 Summar y Matrix of Recommendations 5. Incorporating Data 1. Driving Purposeful 2. Designing and Acting at 3. Empowering Teachers 4. Engaging the Ecosystem and Evidence for Educational Change Scale, for All Ongoing Improvement Recommendations specific to digital and distance education and COVID-19 pandemic response • Continue to use and adapt • In the short-term, ensure that a • Identify and promote innovative • Employ strategies to gain • Improve data on student the Madrasati platform plan for providing devices and and evidence-based effective the trust of stakeholders who attendance and engagement and develop further digital connectivity to all students who teaching and learning practices may be anxious about a safe with distance education and content do not have them is feasible in digital and distance education return to in-person schooling, in-person schooling and enacted as a matter of for example by encouraging urgency schools to incorporate student voices in return to school plans • Plan for long-term purposeful • Provide additional targeted • Assess and address learning blended learning models resources to students not losses on return to in-person that best suit existing regularly attending virtual schooling practices and conditions, classes such as the “Flipped Classroom” and “Flex and Enriched Virtual” Recommendations related to broader areas of education reform including digital and distance education • More explicitly target • Set policies for long-term • Reduce variability in teacher • Support and require teachers • Ensure continuity in policies and resources access of teachers and performance through targeted to recognize and meet the externally assessed, on disadvantaged and students to devices and professional learning, and socioemotional needs of their standardized national struggling students connectivity for both distance increase consistency of practice students assessments to provide and in-person schooling through clear guidelines for crucial information on digital and distance education, overall levels of student for example on student learning engagement and use of cameras • Promote policies and • Evaluate the impact of teachers’ • Revisit student assessment communications that learning to better understand tools and practices, plus emphasize high expectations which teacher professional training of educational for all students across all key development investments have professionals, to better track skills been most impactful and should students’ development of continue critical cross-disciplinary and 21st century skills Recommendations specific to other areas of education reform • Strengthen the quality of • Review and strengthen the • Engage school communities • Reexamine student teaching as a priority to roles of school principals and in setting high behavioral assessment policies and improve learning outcomes supervisors to lead education standards and creating consider developing improvement positive school cultures and a national assessment climates framework to coordinate and communicate intent Note: These recommendations are described in more detail in chapter 8. Saudi Arabia's Digital and Distance Education | 1 1. The Study In March 2020, the COVID-19 pandemic forced Saudi Arabia to cease regular in-person schooling and require students to stay at home. This marked the start of an innovative and unique journey to change the way in which schooling operated so that children across the country could continue their education. This journey is one that will have lasting impacts on education in Saudi Arabia. The World Bank studied this journey in detail over the 2020– 21 school year. The results of this study are provided in this report, which comprehensively reviews the experiences of digital and distance education in Saudi Arabia and analyzes the opportunities now available to make long-lasting educational improvement. 1.1 OBJECTIVES AND SCOPE The objectives of the study were to review Saudi Arabia’s digital and distance education experiences in response to the COVID-19 pandemic, to evaluate its impact, and to identify opportunities to harness the most effective new practices to “build back better” from the crisis. The scope of the study extended from kindergarten to grade 12 (K–12) with a focus on elementary, intermediate, and secondary education. In-depth data gathering took place in grades 3, 6, 9, and 12. Research questions The study aimed to answer three key research questions: 1. How well did Saudi Arabia provide for, and achieve, continued education of K–12 students during the COVID-19 pandemic? 2. What were the strengths of Saudi Arabia’s response to the COVID-19 pandemic in K–12 education? 3. What are the opportunities for educational improvement following the digital and distance education experience? Study themes To answer these research questions, the World Bank team reviewed the recent global literature and the work underway by the World Bank and other international organizations on education during the COVID-19 pandemic, as well as the experiences of other countries across the world as they responded to the pandemic. This helped the team to identify a common set of themes that could be used to prepare a set of custom-built data collection tools and to frame the analysis. The six themes identified included: 1. Enabling digital and distance education 2. New and adapted tools and materials for education continuity 3. Teaching and learning practices during the pandemic 2 | 1. The Study 4. Estimated effects of the pandemic on student learning 5. Engagement and well-being 6. Implications for next steps. The effectiveness of the digital and distance education provision, in terms of student learning outcomes, could not be directly analyzed because of the lack of a comparable standardized national assessment during the study period. Significant efforts were made to develop banks of test items, mapped to expected learning outcomes, and have them readily available and convenient for teachers to use in assessing their students. However, a formalized moderation system is not yet in place to ensure consistency of teacher judgment in summative assessments of student learning within and across schools, districts, and regions. 1 Therefore, the results of teacher-assigned grades and marks could not be used to reliably determine overall changes in student learning outcomes before and during the COVID-19 pandemic. Instead, the perceptions of effectiveness of the digital and distance education were gathered from a wide range of stakeholders and examined within this study. 1.2 BACKGROUND AND CONTEXT Globally, when the COVID-19 crisis first emerged, little was known about the disease, how it was spread, or how to treat it. Governments in 167 countries had closed all schools by the end of March 2020 as a precautionary measure, leading to 1.5 billion children out of the classroom. The shocks to education extended beyond school closures—which were expected to have an impact on learning as well as the health and safety of children and young people —to an economic crisis and long-run associated costs (World Bank 2020a). To mitigate these impacts, plans and policies were needed for (1) coping with the crisis, including protecting health and safety and preventing learning losses; (2) managing continuity in the complicated reopening phase including prioritizing within the curriculum and helping students to catch up; and (3) improvement and acceleration , including harnessing the best practices to make schools more resilient, equitable, and effective (World Bank 2020a). The COVID-19 pandemic led to an urgent need for alternative models of education service delivery across the world, particularly as the time needed to tackle the virus (and its recurrent waves) extended beyond initial expectations. The pandemic also highlighted the need to build resiliency in education systems to withstand any future requirement for large-scale school closures, something that only a few countries had experienced in the past. In Saudi Arabia, schools were closed for in-person learning on March 9, 2020. Just one day later, distance education in the form of recorded lessons was made available through TV and YouTube channels for all school grades. This swift response, an immediate nationwide implementation of distance education at large scale was made possible because of Saudi Arabia’s previous investments in e-learning, significant accumulated experience in education technologies, quick decision-making, and well-coordinated effort across multiple entities. After the summer, schools remained closed for regular in-person learning and synchronous online teaching took place for the full school year. These efforts and innovations were documented by the MOE in its report released in the Spring Semester of 2020: “The Saudi MOE: Leading Efforts to Combat Coronavirus Pandemic (COVID-19)” (MOE 2020). 1 For further details regarding moderation, see footnote no. 4 and OECD 2020. Saudi Arabia's Digital and Distance Education | 3 One year after the crisis began, by March 2021, almost all countries had reopened their schools for in-person learning, either in full or in part, with just 32 countries, including Saudi Arabia, continuing with fully distance education. 2 In fact, Saudi Arabia has had among the longest periods of implementation of nationwide distance education across the world, totaling 41 weeks (appendix A). However, Saudi Arabia stands apart from the other countries with long-term nationwide school closures (shown in appendix A) in that synchronous virtual classes operated for the full 2020–21 school year with mandatory attendance, while asynchronous alternatives were made available for students without connectivity or devices alongside regular in-person school contact. This makes the education response and experience of Saudi Arabia during the COVID-19 pandemic unique. Global estimates of the effect of COVID-19-related school closures and economic contractions suggest that children could lose, on average, 0.6 years of effective schooling, and experience a reduction of US$872 in yearly earnings, approximately equivalent to US$16,000 over a student’s work life (at present value). The share of lower secondary-aged students who are below minimum proficiency levels may increase globally from 40 percent to 50 percent under the “intermediate” scenario of these estimates (Azevedo et al. 2020). Results from individual countries indicate widespread and concerning “learning losses” since the start of the COVID-19 pandemic. For example, in the United States, students started the fall semester of 2020 being, on average, three months behind where they should have been in mathematics and a month and a half behind historical averages for reading (Dorn et al. 2020). In England, after a second lockdown in early 2021, primary school children were 3.5 months behind non-pandemic years in mathematics, and 2.2 months behind in reading; progress made in the in-person Autumn term was lost after the second lockdown (Education Policy Institute). In Belgium, grade 6 students lost 0.19 standard deviations (SD) in mathematics, and 0.29 SD in Dutch after 9 weeks of school closures (Donelly and Patrinos 2021). While data is still coming in on learning losses around the world, the overall picture appears to be one of concern over learning loss, increased inequalities, and a need for targeted catch-up and remediation. While a return to school (where it was safe to do so) and improved distance education will have helped, significant catching up may still be required given these losses. As countries look to invest more in education technologies (EdTech), spurred on by the necessities of distance education and the promises of efficiencies and new learning opportunities, the World Bank recently published its guidance, advocating for attention to five key principles when education systems invest in EdTech (World Bank 2020b). The five key principles are outlined in box 1.1. Some of the interconnected and varied topics that the principles touch upon are illustrated in figure 1.1. 2 response# UNESCO Global Monitoring of School Closures database: https://en.unesco.org/covid19/education‌ ‌‌durationschoolclosures (accessed on June 5, 2021). 4 | 1. The Study Box 1.1 Five Key Principles for EdTech Investments 1. Ask Why: EdTech policies and projects need to be developed with a clear purpose, strategy, and vision of the desired educational change. EdTech considerations should focus on “education” and not just on the “technology.” “Education at its core is a human-centered, socially intensive endeavor that connects a community of learners. Technology should support and enable those connections” (World Bank 2020b, p. 12). 1. Design and Act at Scale, for All: The design of EdTech initiatives should be flexible and user-centered, with an emphasis on equity and inclusion, in order to realize scale and sustainability for all . EdTech has exacerbated inequalities in many places, and this need not be the case. Understanding users’ needs and contexts will lead to more inclusive investments. 1. Empower Teachers: Technology should enhance teacher engagement with students through improved access to content, data, and networks, helping teachers better support student learning. Global evidence indicates that the effective use of EdTech results in the role of teachers becoming more central, and not peripheral. 1. Engage the Ecosystem: Education systems should take a whole-of-government and multi-stakeholder approach to engage a broad set of actors to support student learning. This includes, alongside governmental agencies, students, teachers, school leaders, parents/caregivers, non-governmental organizations (NGOs), academia, and the private sector. 1. Be Data-Driven: Evidence-based decision making within cultures of learning and experimentation, enabled by EdTech, leads to more impactful, responsible, and equitable uses of data. There will be an abundance of data, and therefore a need for capacity to utilize data and evidence to inform decisions that improve teaching, learning, and the management of the education system. World Bank 2020b. Saudi Arabia's Digital and Distance Education | 5 Figure 1.1 Fi1.1 Figure Five ve Key PrinciKey Principles ples an and d Interconnect Interconnected ed Topics for EdTech InvestTopics ments for EdTech Investments Source: World Bank 2020b. Source: World Bank 2020b. The 2021–22 school year closed one month early (ending on April 22, 2021 for elementary schools and April 30, 2021 for intermediate and secondary schools). After that, virtual summer The 2021–22 school year closed one month early (ending on April 22, 2021 for elementary school operated in each education directorate for elementary school students (between 3pm schools and 8pm)and April and 30, 2021 intermediate for intermediate school students (between and9amsecondary and 1pm) schools). After that, for approximately virtual 4 weeks via the school summer Madrasati platform. operated inStudents with connectivity each education and devices directorate could opt into for elementary this program, school students which focused in foundational skills and core subject areas such as mathematics, Arabic, (between 3pm and 8pm) and intermediate school students (between 9am and 1pm) for science, and Qur’anic studies. This was an opportunity for remedial education to take place, compensating 4 approximately weeks for via the Madrasati disadvantage platform. and falling behind that Students with would some students connectivity and devices have experienced, could opt in line into with a this program, growing body ofwhich focused evidence in foundational on the skills and need for remediation and core subject of the benefits areas such tutoring asto catch up help studentsArabic, mathematics, (World Bank science, 2021). At studies. and Qur’anic the start This of the summer was school session, an opportunity around for remedial 47,000 students and 1,500 teachers were involved. While this represents only 1.2 percent of education to take students and place, of 0.4 percent compensating and middle and for disadvantage teachers in elementary falling schools, behindschool that some the summer is a students would mechanism have that, experienced, in the future, couldin beline usedwith amore in a growing body targeted way evidence of to on the need support remedial needs for by selecting remediation students the the and most of benefits in tutoring need, and to incentivizing help students their participation. catch up (World Bank 2021). At the start The MOEof the has summer announced school session, return around to school for the students plans47,000 and start of the 1,500 teachers 2021–22 were school year, which will involved. see this While fully vaccinated represents onlyintermediate and 1.2 percent of secondary students and school students of 0.4 percent returning teacherstoin regular in-person schooling on August 29, 2021. Kindergarten and elementary school students elementary and middle and unvaccinated schools, the intermediate andsummer school secondary is a students school mechanism will that, in the continue future, with could distance used in auntil beeducation more thetargeted way population to supportrate vaccination remedial reachesneeds by selecting 70 percent the students or October most 30, 2021 need, and is in (whichever reached first). incentivizing There their are some situations (such as in overcrowded schools) where participation. it will be difficult to incorporate social distancing, and so the MOE has issued guidance on hybrid The MOE models whereby in-person has announced contact return to schooltime in schools plans for theisstart reduced and of the a mix of synchronous 2021–22 school year, and asynchronous learning takes place using the digital and distance education tools and which will see fully vaccinated intermediate and secondary school students returning to mechanisms employed over the 2020–21 school year. More broadly, blended learning models, regular in-person schooling on August 29, 2021. Kindergarten and elementary school students and unvaccinated intermediate and secondary school students will continue with distance education until the population vaccination rate reaches 70 percent or October 30, 6 | 1. The Study where there is a combination of in-person and distance learning, will be further explored by the MOE to see how they can be helpful in the education process beyond the pandemic, and for particular situations such as children studying in very remote locations. 1.3 METHODOLOGY Data was gathered specifically for the study using the following custom-designed tools. Focus groups Virtual focus group discussions took place between February and April 2021 with groups of stakeholders, as shown in table 1.1. The main objective of the focus group discussions was to augment data collected for the study from the surveys and other sources by providing a deeper understanding of each group's experiences and perceptions. The focus group discussions were planned and moderated following a script that aligned with the study’s themes. The advantages of using this model are that focused questions direct thinking and reflections, moderators can elicit comprehensive responses, participants are guaranteed to respond, and it provides opportunities for peer-to-peer discussions that lead to deeper understanding for the researchers. While efforts were made to include a range of participants by region, type of school (general or Qur’anic, and boys’ or girls’ schools), ages and grades, and subject specializations, the virtual engagement of participants often limited the representative nature of the sample. The results of the focus group discussions were used in the design phase for the survey questionnaires. In addition, the results from the focus groups are used throughout this report in the discussion of various aspects of Saudi Arabia’s experience of K–12 digital and distance education, with quotes from the focus group discussions providing insights from stakeholders’ own voices. Table 1.1 Number of Focus Group Participants Focus group Number of participants Elementary teachers 7 Intermediate and secondary teachers 9 Supervisors 11 Parents 8 Boys (intermediate and secondary) 7 Girls (intermediate and secondary) 6 School principals 3 TOTAL 51 Surveys The following stakeholders were surveyed to gather information on their views, perceptions, and experiences in relation to the study’s objective and themes: 1. School principals 2. Teachers 3. Students Saudi Arabia's Digital and Distance Education | 7 4. Parents 5. Supervisors Questionnaires were designed for each of the groups of stakeholders covering the study theme areas. The questionnaires were first developed in English and then translated into Arabic, and were reviewed and tested by MOE officials, supervisors, and teachers. The questionnaires were designed in Microsoft (MS) Forms, using a mixture of question types with response selections, plus an opportunity for written comments. A nationally-representative sample of school principals, teachers, students, parents, and supervisors was selected and the questionnaires were sent via the regional education authorities to schools in April 2021. Specific attention was given to a robust method for obtaining nationally-representative results, as opposed to only hearing from those motivated to respond, while also having a process that was manageable to implement and minimized administrative requirements of regional educational administrations, district education offices, and schools. To focus the data gathering, four grade levels were targeted: grades 3 and 6 in elementary school, grade 9 in intermediate school, and grade 12 in secondary school. A two-stage, systematic, cluster sampling strategy was used to select schools, with selection probability proportional to size (PPS). In this case, the measure of size (MOS) was the number of students in a school at the target grades. Separate samples for elementary, intermediate, and secondary schools were selected. The sampling frame included all public schools. Systematic sampling involved sorting the schools in the sampling frame by boys’/girls’ school, school type (such as general or Qur’anic schools), geographical region (13 administrative regions), and total school size (number of students). This implicit stratification means that the sample reflects the population in terms of these variables. The target number of schools was based on an analysis of similar studies, such as OECD’s Programme for International Student Assessment (PISA) and the balance between taking a larger number of schools to minimize clustering (similarities within schools) and a smaller number to minimize non-response bias, which would have been detrimental to the validity of the results. This led to the decision to select 200 schools at each of the elementary, intermediate, and secondary levels (totaling 600 schools). After the selection of the 600 schools, it became apparent that only 4 of the 47 regional education administrations were not included. Given that this might be construed as an oversight (as opposed to an artefact of the sampling methodology), one school from each of those regional education administrations was selected in addition to the original sample. These additional schools were selected by sorting the regional education administrations and total school enrollment, and then selecting the second last school as the additional school: one elementary, two intermediate, and one secondary. This led to an intended sample size of 604 schools (table 1.2). The school principals of the selected schools were given the link to the school principals’ questionnaire. Teachers were selected within the sampled schools according to instructions provided by the MOE and communicated through the regional education administrations. To have a manageable sample, the following core subjects were chosen: Arabic, mathematics, science, and English. Elementary school principals were given instructions on how to select one grade 3 class (which are general classes and not specific to the core subject areas) and one grade 6 class for each of the core subject areas. Intermediate school principals were given instructions on how to select one grade 9 class for each of the core subject areas, and secondary school principals were given instructions on how to select one grade 12 class for each of the core subject areas. 8 | 1. The Study The teachers of the selected classes were given the link to the teachers’ questionnaire. All students in one of the classes for each grade in each school were given the link to the students’ questionnaire, and the parents of the selected students were asked to submit one response to the parents’ questionnaire. The supervisors of Arabic, mathematics, science, and English from the district education offices in which the nationally-representative sample of schools were located were asked to complete the supervisors’ questionnaire. The intended sample size for the surveys is shown in table 1.2, along with the achieved sample size and the response rates. An explicit target for response rates was not set; in fact, response rate benchmarks are only one part of broader assessments of survey quality (Sturgis, Smith, and Hughes 2006). However, compared to other similar electronic surveys, the achieved response rates are similar or higher. 3 Table 1.2 Surveys: Intended Sample Size and Response Rates Principals Teachers Students Parents Supervisors Intended sample 604 2,617 22,218 22,218 1,334 Responses received 368 1,832 9,352 5,575 858 Response rate (%) 61 70 42 25 64 Note: For students and parents, the intended sample was approximated because the data on class size from the school principal survey was deemed to be unreliable. An estimate of 27.6 students per class was applied, which is the average of the class sizes reported in Saudi Arabia by teachers of nationally-representative classes of grades 4 and 8 reading, mathematics, and science from the Trends in International Mathematics and Science Study (TIMSS) and the Progress in International Reading Literacy Study (PIRLS). The MOE monitored survey responses after the questionnaire links were sent, and followed up with regional education administrations, who in turn followed up with district education offices and schools in which responses had not yet been received. This process helped to increase the reliability of the survey results. After the survey period was closed, the data was cleaned and weights were added to allow for the calculation of nationally-representative responses to each question. The weights account for selection probability and non-response. Details on the weighting methodology are provided in appendix B. The cleaned dataset with weights was used to create tables of results for each question by level of education and by boys’ or girls’ schools, as shown in appendixes C to G. These results have been used throughout this report to provide evidence on the various aspects discussed around Saudi Arabia’s K–12 digital and distance education. Respondents to the questionnaires had an opportunity to write a comment upon completion of the questions. A thematic analysis was conducted on the written responses. Analysts scanned the responses from each of the stakeholder groups, removed unsubstantial responses (such as “no comment”), reviewed the response for each group in more detail, noting emerging themes, then identified common themes, which formed the basis of the full analysis. The selected common themes and sub-themes are shown in table 1.3. The results of this analysis were used throughout the study analysis, with quotations enabling key themes to be portrayed in respondents’ voices. 3 See, for example, UK Higher Education Academy 2016. Saudi Arabia's Digital and Distance Education | 9 Table 1.3 Questionnaire Open-ended Question: Thematic Analysis Themes and Sub-themes Theme Sub-theme 1. Advantages and benefits of distance learning Skill gains Comfort of learning at home Ease and efficiency considerations Parental and family engagement 2. Disadvantages and challenges of distance Internet connectivity and devices learning Measurement and evaluation of student performance Students’ physical and mental well-being Student motivation and engagement Distance learning schedule and class duration Teachers’ skills and performance 3. Recommendations for digital and distance Improving infrastructure learning Improving student assessment Improving the Madrasati platform Reviewing distance learning schedule and class duration Planning for professional development Maintaining/reinforcing communication with parents 4. Future preferences around the type of Distance learning education Face-to-face learning Blended learning Virtual classroom observations The observation of a sample of virtual classroom practice was undertaken within the study to obtain qualitative evidence about the digital and distance education approaches in Saudi Arabia during the COVID-19 pandemic. As one of the pillars of triangulation used in the study, classroom observation is unique in that it provides direct insight into the teaching practices that are actually being delivered in the classroom, as opposed to what teachers may say, or believe, they or their students are experiencing. A sample of 64 virtual classes were requested for observation, spread across grade levels (lower and upper elementary school, intermediate school, and secondary school), subjects (Arabic, mathematics, science, and English), urban and rural locations, and boys’ and girls’ schools. However, due to technical difficulties, one virtual classroom observation could not be completed, resulting in a total of 63 virtual classroom observations undertaken. A customized virtual classroom observation schedule was developed to assess the quality of teaching and learning practices within the digital and distance education context. The teachers’ regular supervisors were trained by the World Bank in using this tool to observe the following aspects of the virtual lesson: 1. The use of digital teaching and learning resources 2. Planning the learning 3. Establishing a positive climate for learning 10 | 1. The Study 4. Teaching and learning 5. Feedback and assessment Following the observation of the lesson, the supervisor had a 30-minute general debrief and discussion about the lesson, about the teacher’s views on digital and distance education, and about aspects of communication and student engagement, assessment, and parental engagement. Guiding questions were given to the supervisors and the teachers’ responses were recorded and provided to the World Bank and MOE analysis teams. In total, 63 virtual classroom observations took place around March 2021. The findings from these observations and the post-observation teacher interviews are discussed throughout this report. Interviews The study team conducted a range of virtual interviews via videoconferencing and through emailed proformas. Key officials responsible for various aspects of digital and distance education, aligned with the study’s themes, were consulted. This included officials from the MOE’s Departments of General Education, Planning and Development, Digital Transformation and Information Security, General Administration of E-Learning and Distance Education, Information Technology, and the Curriculum Development Center, as well as the Tatweer Company for Educational Services. The interviews provided insights into the themes of the study, enabling the analysis outlined in this report. Saudi Arabia's Digital and Distance Education | 11 2. Enabling Digital and Distance Education The digital and distance education provided across Saudi Arabia during the COVID-19 pandemic required meticulous planning and careful attention to the roles and needs of all stakeholders. This chapter examines the planning, communication, preparation, provision, and monitoring of digital and distance education, particularly in advance of (and during) the 2020–21 school year. 2.1 PLANNING AND COMMUNICATING MITIGATION MEASURES Like other countries across the world, Saudi Arabia was faced with the crisis of having to pivot away from in-person schooling in early 2020 due to the COVID-19 pandemic. Just one day after regular in- person classes were ceased in March 9 of the same year, distance education in the form of recorded lessons was made available to more than 6 million K–12 children across the country. This immediate implementation of distance education at large scale was made possible because of Saudi Arabia’s previous investments in e-learning, significant accumulated experience in educational technologies, quick decision-making, and well-coordinated efforts across multiple entities. The immediate asynchronous distance education modalities made available to school children in the late part of the 2019–20 school year were sufficient to enable most children to continue their studies at that time. However, it became clear that more would be needed before the start of the 2020–21 school year to ensure effective continuity of learning as the crisis continued to require the ceasing of regular in-person schooling. In the years before the pandemic, Saudi Arabia invested in digital learning, starting with the establishment in 2005 of the National e-Learning Center, followed by the Tatweer Company for Education Services (T4edu) in 2012, and Tatweer Educational Technologies (TETCO) in 2016 to support the MOE’s goals. Several initiatives were underway by these organizations under the MOE’s supervision. These initiatives aimed to promote digital and technology-enabled teaching and learning in schools and raise teachers’ and students’ digital skills. One of the key initiatives underway before the pandemic was the Education Information Network (iEN) Portal, which houses electronic copies of course materials, recorded lessons, enrichment activities, question banks, and professional development resources for teachers. The rich experiences and expertise built up across these organizations—for example, in procuring, implementing, and evaluating the trialed learning management systems and developing recorded model lessons—laid the groundwork for the ramping up of a unique digital and distance education model in response to the crisis. As the pandemic first hit, these established resources were made available and scheduled for asynchronous learning opportunities for children during the crisis situation. More lessons were recorded to complement those already available. The recorded lessons were aired 24-hours a day on dedicated iEN TV channels (one for each grade) and iEN YouTube channels. Over the summer of 2020, the MOE decided to further enhance the online learning and teaching experience and require virtual live connections between students and teachers for all lessons 12 | 2. Enabling Emergency Digital and Distance Education (synchronous learning), wherever possible. This was a bold decision made for the benefit of students’ learning and required infrastructure capable of hosting over 5 million students online at the same time. Building on existing initiatives, and a new partnership with Microsoft (MS), the Madrasati (“My School”) platform—a national platform owned by the MOE and integrated with MS tools and virtual labs—was created in time for the start of the 2020–21 school year. One of the key success factors in the rapid transition to online learning, in addition to having built up expertise from prior investments in digital learning, was strong leadership and quick decision- making. An action plan that addressed key dimensions such as leadership, infrastructure, content, and pedagogy was developed. This plan commenced with engaging large, experienced service providers such as Microsoft. Attention was paid to aspects of change management in rolling out the virtual school program, particularly at the start of the 2020–21 school year. A “Back to School” information platform was established to raise e-learning awareness and enhance students’ digital citizenship skills. This was a single, central place for parents, students, and teachers to access all the information, user guides, video training packages, expectations, and regulations necessary for a successful e-learning experience. Infographics were used extensively through social media to encourage engagement with distance education. 2.2 PREPARING SCHOOLS, TEACHERS, STUDENTS, AND PARENTS Preparing 400,000 teachers to use the new digital tools and significantly alter their teaching methods in such a short period of time was an enormous but essential task. Teacher training was made widely available; a teacher in each school was designated as the e-learning focal point to support their colleagues, and networking support across the teaching profession appears to have been widespread. Supervisors have played a key role in the professional development of teachers through communicating and sharing good practice across subject areas and schools during the pandemic. The impact of many of the teacher professional development programs provided by the MOE has been evaluated locally, but impact data has not yet been collated at the national level. The MOE has collected data relating to the number of training programs as well as the total number of hours of teacher professional development provided during the pandemic. Going forward, it will be important for the MOE to collect and collate national impact data. This will allow the MOE to understand the overall level of effectiveness of the resources it has invested in, plus highlight which aspects of the program have worked well and which need amendment. The provision of technical support in multiple ways was essential to help students, parents, and teachers connect, particularly in the first few weeks of distance education. This included a dedicated call center, integrated online live chat, support staff based in district offices, and guidance to schools on how to help with log-in enquiries. More than half (54 percent) of teachers felt that they had received the right level of communication and advice to help them prepare for distance education, while 33 percent felt that they had received too much information. Teachers found advice from the internet, school principal and other teachers, followed by the supervisor, the most helpful. In response to the urgent need for online learning approaches, the MOE planned a new online curriculum that aimed to provide an overarching digital educational content for its K–12 general and special education schools. The design of the curriculum went through a series of quality-controlled stages, including the identification of curriculum goals, development of curricular structures, and Saudi Arabia's Digital and Distance Education | 13 selection of curriculum content. Thereafter, individual courses and lessons were designed by educational specialists who created training materials and teaching manuals and had responsibility for filming, recording, editing, and determining the most engaging and effective delivery methods for each lesson. In developing the new online curriculum, the MOE made changes to existing time allocations for individual subjects. For example, the amount of curricular time allocated to mathematics and science was increased, whereas time for the Arabic language, arts, and physical education was reduced. The number of weekly classes, overall, was reduced. This reduction in the scope of the curriculum and learning time was a temporary response to the crisis. An extended school year (with three semesters) was announcements after the end of the 2020—21 school year. This will help to ensure that the curricular breadth and depth is re-established and further enhanced moving forward, particularly for the important foundational skills such as literacy and numeracy. 2.3 PROVIDING FOR ALL STUDENTS At the start of the pandemic, access to devices and Wi-Fi connections was presumed to be relatively high among students across the country, but not universal. Therefore, the system of distance education for the 2020—21 school year needed to have flexibility to cater to students’ different circumstances and ensure students’ equitable access to education. Initially, around 70—80 percent of students were expected to log onto virtual school through the Madrasati platform. However, within a short period of time, 98 percent of all students had logged onto the Madrasati platform at some point, which is a remarkable achievement. Those students unable to access Madrasati due to the lack of a device or connectivity were expected to regularly attend school to receive materials and check-in with their teachers in an asynchronous mode. 2.4 MONITORING ATTENDANCE AND ENGAGEMENT The MOE has closely monitored the levels of engagement on the Madrasati system on a weekly basis. Figure 2.1 shows that students and teachers were able to access the Madrasati platform within a matter of weeks (with a dip following the break between semesters). After concerted communication efforts in the first two weeks of the school year, more than 90 percent of students had logged onto Madrasati. By the tenth week of school, 98 percent of the student population had logged onto Madrasati (figure 2.1). This indicates that the technical support and follow-up mechanisms were effective. In total, 98 percent of students across the country were able to access the Madrasati platform at some point during the year. Teacher engagement was felt to be high, with 89 percent of school principals reporting through the survey that they had seen a reduction in teacher absence compared to previous years. On average in a week, student engagement with the Madrasati platform, through logging in, was around 75–85 percent (figure 2.2). In addition, some students directly accessed virtual classes through MS Teams (not through the Madrasati platform), and those students needing face-to-face support attended school in-person occasionally or on a regular basis. Future work to analyze data on students’ interactions with EdTech, both synchronously and asynchronously, will help to gain a better understanding of student attendance and engagement, including the degree to which they maintained contact and engagement throughout the virtual classes. Students dropping out of the virtual classes was mentioned by some teachers and supervisors in the comment section of the 14 | 2. Enabling Emergency Digital and Distance Education questionnaires, although this question was not specifically asked in the questionnaire so the extent of this view is not known. However, 67 percent of teachers found that it was difficult to engage students throughout the lesson’s required time, and 59 percent found poor internet connections to be a main problem to student engagement, highlighting the need to move beyond measures of logging in to ascertain true levels of student attendance and engagement. Data is not yet centrally available on the degree to which students studying on an asynchronous mode regularly accessed education through school visits. Data is also not yet available on the degree to which students accessed recorded lessons. From the nationally representative surveys, 19 percent of students reported having met in-person with their teacher; however, only 6 percent of teachers reported meeting students regularly in school, so the degree to which these students received in-person support from their teachers may have been less than expected. It will be important going forward with any further digital and distance education provision that those children not attending virtual classes through the Madrasati platform are monitored in terms of their attendance and engagement, with schools and districts following up on individual students and the center monitoring performance on this across the country. In terms of obstacles to student engagement, one-third of teachers said that the lack of availability of digital devices for students was a main problem regarding student engagement in class (32 percent), with almost a further half saying it was an occasional problem (47 percent). Students themselves reiterated this concern: 41 percent of students said that poor internet connection can make it hard to study at home. Figure 2.1 Madrasati Cumulative Log-in, 2020–21 School Year Figure Percent 2.1 who Madrasati have Cumulative ever logged into Log-in, Madrasati during 2020–21 the semester (%) School Year Percent who have ever logged into Madrasati during the semester (%) Semester 1 weeks: Semester 2 weeks: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 1 2 3 4 5 6 7 8 9 10 11 12 S tudents 58 81 91 93 95 95 96 97 97 98 98 98 98 98 98 98 67 88 91 94 95 96 97 97 97 97 97 98 T eachers 87 96 97 97 98 98 98 99 99 99 99 99 99 99 99 99 95 99 100 100 100 100 100 100 100 100 100 100 S chool leadership 96 98 100 100 100 100 100 100 100 100 100 100 100 100 100 100 97 99 100 100 100 100 100 100 100 100 100 100 S upervisors 56 76 86 87 88 92 94 96 96 97 97 97 97 98 98 98 55 86 93 95 96 97 97 97 98 98 98 98 P arents 24 32 35 36 37 37 32 38 38 38 39 39 39 39 39 40 9 11 12 12 13 13 14 14 15 16 16 16 Source: MOE General Source: MOE GeneralDirectorate Directorate for for E-Learning E-Learning and and Distance Distance Education. Education. Saudi Arabia's Digital and Distance Education | 15 Figure 2.2 Madrasati Average Weekly Log-in, 2020–21 School Year Figure 2.1 Madrasati Average Weekly Log-in, 2020–21 School Year Average percent who logged into Madrasati each week (based on daily log-in data) (%) Average percent who logged into Madrasati each week (based on daily log-in data) (%) Semester 1 weeks: Semester 2 weeks: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 1 2 3 4 5 6 7 8 9 10 11 12 S tudents 36 74 77 74 74 76 79 81 83 85 85 85 85 85 83 81 64 78 79 81 80 82 81 78 78 77 77 75 T eachers 68 92 94 94 94 94 95 95 95 96 96 96 96 95 95 93 94 98 98 98 98 98 98 97 98 98 98 98 S chool leadership 91 98 98 98 97 97 97 97 96 96 95 95 94 93 93 94 96 96 95 95 95 95 94 93 94 94 92 93 S upervisors 37 63 60 52 73 73 77 79 80 82 82 82 81 79 75 62 53 75 82 86 86 87 87 86 86 84 82 74 P arents 14 16 12 11 10 10 10 11 10 10 10 10 10 10 10 11 8 10 10 10 10 10 10 10 10 10 10 10 MOE General Source: MOE Source: General Directorate Directorate for forE-Learning and E-Learning Distance and Education. Distance Education. 30 16 | 3. New and Adapted Tools and Materials for Education Continuity 3. New and Adapted Tools and Materials for Education Continuity The digital and distance education provided during the COVID-19 pandemic included a wide variety of tools and materials that were created or adapted for education continuity. These included the Madrasati platform, the iEN educational channels (YouTube and TV), the National Education iEN Portal, and the Virtual Kindergarten. These are reviewed in turn in this chapter. 3.1 THE MADRASATI PLATFORM The Madrasati platform was launched in August 2020 to ensure the continuity of distance education and to provide unified and comprehensive educational services under one platform. The project’s completion and the transfer of more than 5 million students and over 400,000 teachers, plus students’ parents, and educational staff onto the platform was achieved in a remarkable time span. The Madrasati platform is more than a learning management system. It is a unified e-learning platform containing links to tools and services for use by students, teachers, school principals, and parents. Madrasati is designed as a virtual school model that provides students with an online learning environment simulating the in-person school model through diverse instructional processes and activities. It is also designed with the aim to enable and empower teachers and school principals with the useful tools and basic teaching and learning services. These include Microsoft (MS) Teams, Office 365, iEN Portal, and interactive tools as depicted in figure 3.1. The Madrasati platform has enabled over 154 million virtual classes and provided over 16 million diverse educational resources, including videos, educational games, augmented reality, 3D objects, interactive experiences, educational stories, and books. These support synchronous and asynchronous learning. The Madrasati platform also provides tools for instructional planning and design, as well as for measurement and evaluation such as e-tests (with almost 700 million test samples and 41 million homework tasks created by teachers for students). Support to teachers is provided on the platform through question banks that contain over 93,000 validated questions for most subjects. Teachers can use these to create homework or tests to measure educational goals. Teachers, supervisors, and school principals can also follow up on performance through a system of reports and indicators. Various channels for communication and interactions among parties are also offered, such as discussion forums, chat rooms, e-mail, and teacher rooms. To keep the Madrasati platform relevant and responsive, frequent feedback from stakeholders was gathered from users and common issues were addressed. Software updates were also regularly implemented. The platform is considered as a developing (not static) tool based on agile and governed processes. The innovative and flexible Madrasati model has achieved considerable success and played Saudi Arabia's Digital and Distance Education | 17 a prominent role in the continuity of distance education in Saudi Arabia. Satisfaction with the Madrasati platform is indicated by, for example, 98 percent of teachers stating that they would find the Madrasati platform useful after the pandemic. 3.2 RECORDED LESSONS ON IEN CHANNELS AND YOUTUBE A significant number of educational channels were also allocated to allow students to continue with their education outside of their physical learning environment. Twenty-three iEN satellite channels, broadcasting lessons for all academic levels, continuous education, and special education, were provided for free through video broadcasts on TV (available to those without internet access), and 24-hours on YouTube. The 23 iEN channels undergo a continuous, extensive review process to assure the quality of recorded lessons, which are then published on iEN YouTube channels in addition to being used in the Madrasati platform as part of either synchronous or asynchronous lesson planning. The iEN recorded lessons have been used by students in Saudi Arabia as well as students in other countries. Over 230 million views were reached within a year, placing Saudi Arabia in the top position for the most-watched educational channels among Arab countries: a total of 24 million hours watched, 186,000 satellite broadcasting hours, and more than 25,000 filming hours. 3.3 OTHER TOOLS AND ENRICHMENT MATERIALS Several other digital tools and enrichment materials were made available, as follows. The Education Information Network (iEN) Portal The National Education Portal iEN was developed in 2015, in partnership with Tatweer Company for Educational Services, to provide renewable educational services and enriching solutions based on international quality standards. The iEN portal is a content management system aiming to provide high-quality digital content and materials to students, parents, teachers, supervisors, and school principals, and is open to all students in Saudi Arabia as well as in overseas Saudi schools. The portal had already been leveraged during in-person education prior to the COVID-19 pandemic, enhancing teachers’ capacity to use technology in teaching and learning processes. The portal offers diverse educational resources such as digital books, e-tests and self-assessment tools, recorded lessons, lesson plans (over 450,000 e-lesson plans created with teachers’ participation), instructional design guidelines, and a variety of educational games, videos, and content resources based on virtual and augmented reality and three dimensional (3D) technology. The iEN National Education Portal is also linked to the Madrasati platform, allowing digital content to be uploaded directly onto the platform. The iEN National portal is updated and developed, and its content enriched, in a continuous way. The Back-to-School website The Back-to-School website provides multiple educational and instructional materials for K—12 and can be accessed through the Madrasati platform. Over 300 multimedia resources and educational materials, such as user guides, infographics, training manuals, and interactive guidance videos are provided to guide users on how to use the Madrasati platform and apply distance education strategies effectively. 18 | 3. New and Adapted Tools and Materials for Education Continuity Figure 3.1 Madrasati Interactive Tools Figure 3.1 Madrasati Interactive Tools 33 Figure 3.1 Madrasati Interactive Tools (cont.) Saudi Arabia's Digital and Distance Education | 19 Figure 3.1 Madrasati Interactive Tools (cont.) 34 20 | 3. New and Adapted Tools and Materials for Education Continuity Figure 3.1 Madrasati Interactive Tools (cont.) Figure 3.1 Madrasati Interactive Tools (cont.) 35 Saudi Arabia's Digital and Distance Education | 21 Providing for children with special needs The MOE has taken specific measures to support children with special needs through digital and distance education. Three open educational channels (on iEN and YouTube) were allocated to cover all curricula tailored to students with special needs, with thousands of recorded lessons supported with sign language for students with hearing difficulties. Other alternatives and supportive measures were offered to children with special needs through the Madrasati platform and MS Teams, in addition to the option for some children to attend school for individualized one-on-one sessions and support. The iEN educational channels, in addition to the Back-to-School portal, included specific guidelines to parents to encourage their involvement in their children’s learning. Special education teachers and supervisors were also trained on using the Madrasati platform, and comprehensive reference guides supporting distance education processes were developed for special education programs, centers, and institutes. The first electronic guide for kindergarten children with special needs was also issued. This guide aims to clarify the mechanism for providing early intervention services to children with special needs who are younger than six years of age, including content such as clarifying measures and procedures for early intervention programs, and specifying mechanisms for raising the efficiency and effectiveness of services provided to children with special needs. Parents of children with special needs can also access this guide and other various educational materials through the Virtual Kindergarten application. The Virtual Kindergarten The Virtual Kindergarten application was inaugurated on November 20, 2019, coinciding with World Children’s Day. The application aims to provide an interesting and entertaining virtual educational system for children between the ages of three and six years. The application provides children with the opportunity to simulate the reality of kindergarten, with educational videos, stories, games, interactive activities, as well as instructions and assessment tools for guardians to help them support their children’s learning from home. As such, families unable to bring their children to nearby schools are able to benefit from this virtual interaction. During the first semester of 2020–21, the Virtual Kindergarten application witnessed over 3.5 million views. By the spring of 2021, more than 300,000 children and 283,000 parents had been registered on the application. Teacher accounts have also been added to the application, allowing teachers to view digital content; use it for lesson planning and evaluation of students’ levels and progress; share teaching and educational content with children during synchronous learning; or direct them to specific content during asynchronous learning. During the pandemic, the Administration of Early Childhood Education has also prepared a guide for early childhood education, updating admission and evaluation criteria, in addition to developing foundational practices for reading skills from kindergarten to the third grade. This study did not include an analysis of the effectiveness of the Virtual Kindergarten. 22 | 3. New and Adapted Tools and Materials for Education Continuity 3.4 PROCESSES FOR DEVELOPING AND SHARING NEW TEACHING AND LEARNING MATERIALS A key feature of Saudi Arabia’s journey with digital and distance education has been around the processes of developing and sharing the new materials and methods. Accessing new digital tools and materials The ease of accessibility and use of the Madrasati platform, as well as other digital tools, including iEN channels, MS 365 tools and the Virtual Kindergarten has been commended by stakeholders, including teachers and supervisors. The MOE has ensured to address early difficulties in accessing digital content and lessons on the Madrasati platform through providing recorded lessons on TV and YouTube, as well as providing digital devices to students in need through different initiatives, including the charitable Takaful. Guidance and professional development opportunities The Madrasati platform, through its Back-to-School portal, provided various guidance materials and training videos for all stakeholder groups on how to effectively use the platform and enhance the distance learning experience. For example, videos explaining teachers’ roles in distance education, as well as instructional guides and infographics on how to prepare for and deliver virtual lessons, and mechanisms for student evaluation, were provided. In addition, various professional development activities and opportunities were also available for teachers, supervisors, parents, and principals to enhance their distance education experience. Almost all parents found the digital tools available for children’s education to be useful or very useful (94 percent) and found accessing the information that they needed to support their children’s learning to be easy or very easy. Teachers have particularly benefited from the training and support that they received from the MOE and their colleagues to prepare them to deliver their virtual lessons. Almost all teachers (93 percent) found that accessing the information to implement distance education was easy. Training sessions on using the Madrasati platform and its tools such as how to use MS 365 applications, how to incorporate new pedagogical strategies in distance education, how to design interactive content, or how to use digital assessment and evaluation methods, are a few examples of professional development opportunities provided. In addition, live webinars presented by teachers, under the supervision of the MOE, were also provided to demonstrate successful experiences and practices and encourage teachers to implement them in their distance education lessons. Electronic supervision Electronic supervision has also helped in sharing new teaching and learning materials as well as good practices in distance teaching. Almost all school principals reported that using digital tools during the period of distance education helped them to monitor virtual classes, give feedback to their teachers (93 percent), and review academic achievement indicators for their school (91 percent). Supervisors have also played an important role in coordinating virtual training programs, workshops, and other professional development opportunities for teachers based on their digital education needs. As indicated during focus group discussions, supervisors sent links to online training sessions Saudi Arabia's Digital and Distance Education | 23 directly to teachers’ emails, in addition to scheduling virtual meetings for teachers to share teaching and learning materials and exchange experiences to learn from one another. Supervisors also invited teachers to attend model lessons to showcase and share good practices in using new teaching and learning materials and tools. Professional learning communities Distance education has increased opportunities for self-development and the exchange of successful experiences among teachers. Teachers are now active members of both formal and informal professional learning communities, supporting each other to try new ways of working and helping them to identify creative solutions to address the challenges of distance learning. Many individual teachers and supervisors have produced video clips and shared files that explain to other teachers how to use digital tools. Some have shared sample lessons with peers, for example, on YouTube. The pandemic provided an opportunity for everyone to educate themselves. They are developing themselves in the technology of education; teachers are researching and learning more than before. We had 50 to 70% [previous] knowledge in technology, while now we have a higher technical knowledge rate, up to 90%. Female supervisor Joint online learning platforms are used effectively to capture promising policies and practices, as well as success stories. Such platforms could strengthen further regional cooperation and exchange, as well as support work toward joint solutions for education continuity such as developing training programs, putting in place reliable learning assessments, and identifying ways to maximize education investments in times of crisis. Peer mentoring provides valuable opportunities for teachers to support each other through the shift to remote and fully online teaching and learning. However, not all teachers in Saudi Arabia receive sufficient peer mentoring resources and some have never had the experience. Overall, from the surveys it is clear that there is high demand for more digital content and for improvements in the quality of digital content available. Almost all school principals (97 percent) reported that they would like more digital content to be created. In addition, almost all school principals (98 percent) and 94 percent of teachers stated that they would like to see improvements in the quality of digital content. 24 | 3. New and Adapted Tools and Materials for Education Continuity 4. Teaching and Learning Practices During the Pandemic Beyond infrastructure and digital resources, the quality of teaching and learning being delivered in the virtual classrooms is a key determinant of the effectiveness of digital and distance education. Teachers’ planning and facilitation of learning remains just as crucial in virtual learning environments as it is in more traditional school environments. 4.1 PLANNING FOR LEARNING A lesson plan is a teacher’s guide to what students will learn during the lesson, how they will organize the learning experiences, what resources will be used to support the learning, and importantly, how they will evaluate the impact of the lesson. When used well, it is a key tool to effective teaching practice and improved learners’ experiences. The Madrasati platform includes tools for teachers to develop lesson plans, which are quality assured by supervisors. In addition, the platform includes a “Learning Paths” feature in which teachers set individual learning goals for students and monitor those. During the virtual classroom observations, supervisors across all levels and selected subjects evaluated the quality of lesson planning as high in over 90 percent of the lessons they observed. Supervisors reported that, in many lessons, teachers reviewed and made good reference to earlier work completed by their students and linked this effectively to the observed lesson. This is critical for making sure that the material presented to students is in line with what they know and can do. In a few lessons, teachers took time to review the learning objectives of the previous lesson, which helps students to remember previous lessons and to place current lessons within that context. Most supervisors described a high level of organization and structure in the lessons that they observed. Several supervisors described lessons that were planned and structured, for example, using organizers, such as Revision, Introduction, Explanation/Presentation, and Assessment. In many lessons, teachers had set out clear objectives or goals for the lesson and worked to a lesson plan that contained the key learning activities of the lesson. In a few lessons, supervisors noted good practice in involving students themselves in agreeing and setting learning objectives for observed lessons. For example, in one lesson, the teacher had assigned a student to be a “young teacher” for the duration of the lesson, leading and encouraging discussion and other learning activities. This approach had encouraged fellow students to participate more actively in lessons. In several lessons, it was clear that they were being planned and enriched to take good advantage of the wide range of digital video materials from the iEN platform and other digital resources, such as YouTube. In many cases, lesson planning created opportunities for related homework and other extracurricular activities. Saudi Arabia's Digital and Distance Education | 25 In fewer than 10 percent of lessons observed, supervisors raised concerns about the quality of lesson planning. For example, in these lessons, teachers did not sufficiently link their lessons to students’ earlier learning nor provide clear goals or objectives for students. In these cases, teachers were inclined to simply copy the standardized online lesson plans without personalizing or reviewing the planning to take account of their teaching context or the needs of their students. Lesson planning, in best practice, reflects the realities of the classroom, is focused on students’ learning, is customized to suit the needs of particular cohorts of learners, and is periodically updated. This reflects the best of practice described by supervisors in the virtual classroom observations. High- quality lesson planning should also build in an opportunity for self-reflection at the end of each lesson, where teachers evaluate their practice, and share ideas with peers, including about their achievements and challenges. 4.2 VIRTUAL CLASSROOM PRACTICES Regarding virtual classroom practices, elementary school teachers highlighted the enhanced interaction that is possible through digital tools. They pointed to the transparency and ease of use of the different tools, and the ability to use several tools and platforms simultaneously during class― making increased interaction with students possible. An example provided was the virtual class on MS Teams, where interactive educational games can be used (e.g., in an Arabic language class) and responses to questions can be sent on social media groups (e.g., WhatsApp). Digital learning has also helped teachers use time more effectively; for instance, if a student has a specific question, instead of using class time or having to come back in the next class with more information, teachers can now find the content quickly with the use of digital tools. Distance education provides me with easy access to content, if the student has a specific question or wants to watch a specific clip, I can go online and find a video in one minute via YouTube… Male teacher Technology also enabled more practical opportunities to be seen or experienced by students, especially in classes that did not have facilities or opportunities at school for students to practice. I did not have a laboratory but now I use applications and everyone uses devices Female teacher In several lessons, supervisors noted that teachers had adapted their teaching methods successfully to take good account of the constraints and opportunities of digital and distance learning. In many lessons, effective use was made of a wide range of interactive video clips, digital textbooks, and other virtual tools. For example, in some lessons, teachers made good use of digital tools, such as Nearpod and Padlet, to display high-quality visual images of plants and animals and to demonstrate chemical processes and reactions. Several teaching techniques used traditionally in the classroom continued to be deployed in the distance education lessons observed by supervisors. In a few observed lessons, children continued to 26 | 4. Teaching and Learning Practices During the Pandemic learn through play activities, educational games, by recitation, and through song. In science lessons, teachers were able to demonstrate experiments for students, encouraging the development of skills of observation and deduction. In almost all the observed lessons, teachers demonstrated a wide variety of teaching techniques and approaches. For example, from one lesson: The teacher used various teaching strategies, such as problem-solving, dialogue and discussion, as well as technology such as the interactive whiteboard, Microsoft Teams program, and video clips from the iEN portal. Supervisor The wide variety of technology, programs, and applications available allows information to be presented to students in interesting ways and teachers have noticed that it makes learning more enjoyable for students. Teachers find it helpful to use applications to attract students’ attention and involve them in class or group discussions, which allows teachers to be aware of the level of students’ understanding of the lesson content. Feedback from teachers suggests that they derived professional enjoyment from providing virtual lessons. They find the process of teaching more enjoyable and are enthusiastic about the use of the platform, the variety of available applications and other digital techniques. Some comments indicate that teaching this way lessens teachers’ burden requiring less time and effort from them. Overall, teachers felt that technology created a positive context for dialogue and discussion, which puts students at the heart of the educational process. In terms of ensuring quality teaching, distance education has facilitated supervisors’ capacity to review and evaluate teachers across a wide range of criteria using digital tools. The supervisor now has constant knowledge of what our teachers are doing. We have access to details of reports and files of their achievements, statistics, and figures. We can know the number of simultaneous and asynchronous lessons provided, and the type of questions. All the details can be obtained now with the use of e-learning. Supervisor 4.3 ASSESSING STUDENT PROGRESS AND ADAPTING TEACHING During this period of rapid evolution in education systems across the globe, there is a clear understanding of the importance of evaluating the impact on learners and learning outcomes in moving from an in-person experience of learning to a distance and digital model. This has led to an extensive range of assessment methodologies being developed in Saudi Arabia. The Madrasati platform includes a wide range of assessment tools to support formative and summative assessment practices in the virtual classrooms of elementary, intermediate, and secondary schools. This includes over 1 million test items to support the development of subject-based assessments. Online training for teachers has been developed and made available on the Madrasati platform and through MS Teams to support teachers in preparing class-based tests to evaluate digital learning. Saudi Arabia's Digital and Distance Education | 27 Teachers are adapting to these new online assessment tools to perform live oral tests, and the use of students’ video and audio recording of performance tasks. In the focus groups, teachers described the range of tools they can use on the Madrasati platform to assess students’ performance and provide feedback on their learning. Some teachers use the e-homework and test tools on Madrasati to send students e-test forms, where answers are generated in the platform. The Madrasati platform also has a broad range of enrichment materials enabling teachers to design a range of extension activities, which can provide individual students with a direct link to a series of bespoke extension materials. Teachers are now regularly sending out worksheets through e-mail, WhatsApp, or Telegram channels to parents. Parents have been encouraged to support their children with home experiments, and to use the learning games on the platforms. While most parents welcome the greater engagement in their children’s education, feedback from the surveys suggests that, for a few, this additional commitment is felt to impinge of their working and family life. The Madrasati platform contains various tools needed to give feedback. I ask students to do a specific application and then I correct the work and provide them with feedback and Madrasati provides us with [means of] communication. Intermediate teacher Most school principals view teachers’ skills in online assessment and test creation during the COVID-19 pandemic as either advanced (58 percent) or satisfactory (40 percent). Feedback from school principals through the surveys and focus groups suggest that teachers in the intermediate sector require further development in the creation of valid assessment tools. This is an important message, indicating that it is now time to move from universal training to a more targeted approach. Building on the collaborative culture established at the start of the pandemic, it would be helpful to establish communities of practice to support teachers as they implement the new learning in assessment or parental engagement. Teachers report that they systematically administer pre- and post-tests to their students to estimate gaps in learning. These results are then collated by the school principal (by subject and grade). Remedial and follow-up programs are then developed for individual students based on this information. School principals stated that, as parents are now more aware of their children’s performance and learning needs as a result of their direct engagement with their children’s online learning, their cooperation in supporting the learning needs of their children has increased. In most of the virtual classroom observation lessons, supervisors reported that teachers used self- marking or peer-to-peer marking of tests, quizzes, and assignments. In several lessons, teachers asked students to use self-marking activities using digital tools, including Quizizz, Padlets, and WordWall. Students, particularly in the upper grades, are also accessing the Question Bank, which contains hundreds of thousands of structured questions from the Curriculum Development Center and the Department of Educational Supervision, to support learning and evaluate performance. While most stakeholders presented a positive overview of the assessment culture across schools, this is not the view of all. Several important and common issues were identified by some survey respondents, which could impact on learning outcomes for students and the subsequent remediation programs. 28 | 4. Teaching and Learning Practices During the Pandemic The important disadvantage [is] the difficulty of evaluating the student, a true evaluation that reflects their level Supervisor It would be important that, on return to in-person education, teachers are equipped to meet the challenge of rapidly assessing students’ knowledge to identify learning gaps. This knowledge should be used to plan for learning by creating appropriate learning pathways for all students, but particularly those students in transition or preparing for high-stakes examinations. While the use of tests and quizzes help to assess knowledge, to capture deep understanding and the application of skills, “in-class” activities such as questioning and student feedback are important additions to the teacher’s assessment portfolio. In most of the lessons from the virtual classroom observations, supervisors reported that teachers made effective use of questioning and other strategies to check the level of students’ understanding. In many lessons, teachers asked questions throughout the lesson and this provided opportunities to improve the quality of students’ understanding. In one lesson: ... the discussion revealed some students’ lack of understanding of the skill which led the teacher to explain it again Supervisor (virtual classroom observer) Helpfully, in a few lessons, teachers took care to ask progressively more challenging questions using Bloom’s Taxonomy (Kastburg 2003) to ensure that the learning of higher-order skills was successfully taking place. Teachers mentioned that feedback and communication with students is a constant feature of their online classes. Teachers give their students feedback directly in the chat box and individual feedback is given by email (Outlook) or directly to students’ mobile phones. Other teachers formed WhatsApp groups for each class where students could send their questions and they could answer with explanations and feedback. I provide positive feedback directly during the lesson. If the performance [level] is good, or if they get bad grades, we can even schedule lessons to enhance capabilities and even repeat the lesson if there is a need as well, in coordination with parents at an appropriate time, in order to enhance students’ knowledge and help them understand. There are WhatsApp groups for students, every class as a group, and we are always available to answer and help them Teacher Teachers asserted that distance education has helped their students acquire a range of new skills, namely: a sense of initiative; self (independent) learning; critical thinking; logical analysis; problem- solving; communication and presentation skills; cooperation and cooperative learning (e.g., students overcame fears, being timid, play more online in groups, etc.); technical skills and digital skills (e.g., Saudi Arabia's Digital and Distance Education | 29 digital design); research skills; and 21st century skills. It is important to capture the development and progression of these new skills as part of a student’s profile. Going forward consideration should be given to extending the range of assessment tools to capture these important skills for learning and life. 4.4 DEVICES AND INTERNET ACCESSIBILIT Y The interaction between teaching and learning practices and the ability to access the technology infrastructure on a regular and reliable manner is paramount to effective distance and digital education. Where the teaching and learning practices are strong, and the technology infrastructure is accessible to all, then new and exciting digital relationships are being developed between students, teachers, parents, and the wider community. These networks are strengthening teaching and promoting more effective communication and stronger parental engagement in the learning processes. To increase access and reduce potential inequality, all Saudi children and young people need regular access to a device for educational use. This includes televisions, computers, laptops, tablets, and smartphones as well as connectivity to the internet. Distance education is a wonderful and effective experience, but the biggest obstacle to it is the communications infrastructure. Principal From the analysis of the surveys, stakeholders including teachers, parents, and students report that even with the effective provision established by the MOE, digital and distance learning has a number of challenges related to the unavailability, instability, and interruption of the internet. Internet connectivity is a particular challenge in remote villages and developing areas across Saudi Arabia. While 80 percent of students found that learning at home was very easy or okay (suitable), over 86 percent reported that poor internet connectivity impacted on their learning a little or a lot. In addition, the costs associated with buying internet memberships and purchasing devices (especially for families with several children) were further constraints raised by stakeholders. It is comfortable psychologically, mentally and physically, and it is not expensive, and there is no psychological pressure and tension from some of the situations that occur in schools Male student The accessibility of the Madrasati platform from any device has made it easier for students to access learning at anytime from anywhere. On average across all school levels, 87 percent of students report that they have their own device for home learning (74 percent among elementary school students). Most students (88 percent) use a smartphone to access their online lessons, with students in the early grades more often than those in the higher grades using a tablet. Most intermediate and secondary school students are very familiar with the functionality of their smartphone making the transition from in-person learning to digital and distance learning far smoother and enabling the focus to be on the learning rather than the interface with the Madrasati system. In the first few weeks of the move from in-person to distance and digital learning, there were a few challenges related to the lack of 30 | 4. Teaching and Learning Practices During the Pandemic availability of devices for all students and particularly in families where there were several children. Some families who also have less financial freedom may only have one device for their children. The MOE recognized that not all students had access to a device to support their learning and worked to increase access to digital learning for all, and in coordination with administrations and schools, it has provided devices to families in need, through the “Takaful” program, as well as other initiatives. In addition, the MOE separated the elementary school’s schedule from that of the intermediate and secondary’s schedule, which enabled families to share devices more easily. There is a difficulty in learning at home because of the lack of devices for large families like ours. Most of the time we had to take out my brother in the first class to send my sister’s assignments. Female student A number of male students reported that, while they had initially used their smartphones to access distance education at home, they had found this challenging and reverted to a laptop or desktop computer to support their digital and distance learning. Both male and female students felt that the use of headphones had been helpful when they were working at home; it reduced environmental noise levels and helped them to focus. In the first semester I used the mobile phone and it was very difficult, so in the second semester I used the computer Male student One of the most cited recommendations from stakeholder groups was to improve the infrastructure (internet connection and devices). All respondent groups included comments on the importance of providing better broadband coverage to ensure the provision, reliability, and stability of the internet connection if distance education continues, especially in remote and developing areas. Providing a lower cost subscription or covering the costs of the internet (e.g., providing charging cards), and providing devices to students, as well as devices for teachers, were also considered important. Saudi Arabia's Digital and Distance Education | 31 5. Perceived Effects of the Pandemic on Student Learning and Teacher Skills The cumulative effect of ceasing regular in-person schooling in Saudi Arabia since March 2020 on student learning outcomes can only be assessed through understanding how effective the digital and distance education offering has been in continuing learning and mitigating any negative impacts. Understanding this is important as it will provide an idea of the degree to which the pandemic has affected student learning outcomes and what level of additional support and compensatory measures are needed to mitigate any negative impact. One measure of the level of effectiveness of the digital and distance learning offerings would be to compare data on student learning outcomes during the pandemic compared to before the pandemic. However, these data are not available, therefore this study has used perceptions of effectiveness gathered through stakeholder reflections as a proxy measure. In the questionnaires and focus groups, all stakeholders were asked about their views on the effectiveness of the digital and distance education during the 2020–21 school year in terms of academic achievement and skills such as creativity and problem solving and this chapter examines the findings. Perceptions related to teachers’ skills are highlighted since improvements in those could have a long-lasting effect on future student learning. 5.1 ASSESSING STUDENT LEARNING Assessment of student learning needs to take place on several levels. Most importantly, teachers need daily information on what their students can and cannot do in order to adjust the pace and content of their lessons. This is known as formative assessment. Students and parents also need regular summative assessment, which allows them to gauge how well they are progressing. At the district, regional, and central level, comparable information is needed over time and across geographic and other variables to determine how well the system is performing and where additional focus and resources are needed. This is often done through the use of national and international assessments of student achievement. As part of the Madrasati platform, several interactive tools were provided to support teachers in implementing formative and summative assessment. For example, the E-Exams tool allowed teachers to assign electronic examinations to students. Teachers could build these examinations using their own devised questions or preprepared questions from the Questions Bank. Student progress could also be tracked using the Learning Paths tool, based on learning goals set by the teacher. Teachers, supervisors, and school principals were asked in the survey designed for this study about the levels of student achievement for this year’s cohort compared to previous years for the same grades. These perceived effects of the pandemic on student learning are summarized in the next section. 32 | 5. Perceived Effects of the Pandemic on Student Learning and Teacher Skills Beginning in grade 3, students sit examinations at the end of each of the two school terms. These have been developed by individual schools and marked by teachers, so a wide range of content and marking criteria are likely to have been implemented. The predominantly school-based assessment system did not include a comprehensive approach toward moderation; that is, a mechanism for attempting to ensure that assessment judgments are consistent and comparable across different teachers and schools.4 Therefore, the results of these examinations could not reliably determine changes over time in student learning achievement. In the 2020—21 school year, there were additional differences in the examination administration, adding to the lack of comparability across years. The 2020—21 examinations were developed centrally, including multiple choice questions that were marked automatically, and written response questions that were marked by students’ own teachers. An analysis of the examination marks over four years (from 2018 to 2021) showed a jump from around 73—76 percent (on average across grades and subjects in public and private schools) in 2018 to 2020 to 90 percent in 2021. It is not known whether this is due to the change in examination design and implementation, or to a change in teacher allocated marking (possibly overly compensating for students’ more difficult circumstances), or a real increase in learning achievement. A robust assessment of changes in overall national levels of student learning achievement could be made through the use of standardized assessments such as the National Assessment of Learning Outcomes (NALO), which has assessed grades 3 and 6 or 4 and 8 in mathematics, science, and Arabic in Saudi Arabia over the past 5 years, or the Saudi Early Grade Reading Assessment (SEGRA) implemented for grades 1—3 in 2019. However, a NALO or SEGRA round has not yet been implemented since the COVID-19 pandemic. There is an urgent need to implement a comparable standardized and externally marked assessment to gain robust information on changes in student learning achievement levels since the need for distance education. Given the lack of formally moderated school-based assessment results or a nationally administered and externally marked standardized assessment comparable to previous years, the study has relied on the perceptions of effectiveness of the digital and distance education provided during the COVID-19 pandemic as provided by teachers, supervisors, school principals, and parents through the study’s surveys. 5.2 PERCEPTIONS OF EFFECTIVENESS The study showed generally high confidence in the effectiveness of the digital and distance education offered in terms of students’ academic achievement and skills development, with some variation indicating that the experience may have been better for some than for others. Academic achievement When asked about students’ academic progress in the 2020–21 school year compared to other cohorts in previous years, 68 percent of teachers felt that academic achievement was better this year. Among school principals, it was a little lower at 61 percent. More principals of girls’ schools reported this compared to boys’ schools (64 percent compared to 57 percent, respectively). 4 In many countries, school-based assessments rely on a system of examination moderation under which schools develop internal standards to establish the appropriateness of the marks awarded by different teachers. Countries also ensure consistency across schools using external moderation, often conducted by ministries of education or independent agencies. The external moderation can take different forms. Some education systems rely on school visits to conduct moderation. Other systems require schools to send students’ work to an examining authority. A third option includes moderation of samples of students’ work. After external practitioners mark students’ work, moderators compare the marks with those awarded by teachers and decide whether to accept the school-based mark (if the difference is small), adjust school marks, or replace school marks with those awarded by the moderator. Saudi Arabia's Digital and Distance Education | 33 Supervisors were less sure about learning gains during the digital and distance education: 46 percent thought there had been an increase in academic achievement, while 31 percent believed there had been a decline (with the remainder feeling it was about the same). Among supervisors, 59 percent felt there was an increase in academic achievement compared to previous years in girls’ schools while only 33 percent felt that about boys’ schools. At the other end, a decline in academic achievement was stated by 41 percent of supervisors of boys’ schools compared to 20 percent for girls’ schools. In terms of the time needed to catch up upon return to in-person schooling, 73 percent of school principals thought that less than one semester would be needed for students to catch up with their learning. This was less for elementary school principals (65 percent) reflecting a greater need for support for the younger learners. Teachers were more likely to believe that less than one semester was needed for catch up—81 percent. Students, however, were less confident about the effectiveness of their learning experiences during digital and distance education. Around a half of students (52 percent) felt that they would have learned more if they were in regular in-person school, while 40 percent felt that they learned around the same. Students in elementary school, and girls in intermediate and secondary school, were more likely than boys in intermediate and secondary education to say that they would have learned more in in-person education. Skills development Students, of all ages and stages, experienced a range of difficulties as they “switched” from in-person to online learning. As well as the important access to devices and internet connectivity, they needed to quickly change mindset and learn new skills. For example, staying focused during the delivery of synchronous lessons, self-regulation, independent learning skills, and digital skills and knowledge. One student, in his survey response, summarized the challenges he managed to overcome as: Breaking the barrier of fear of participation and error Male student Social skills were particularly hard to foster during digital and distance education—72 percent of school principals felt that distance education led to a lack of opportunities for students to learn social skills. Supervisors were more likely to note this, at 87 percent. Teachers were mixed on their views about distance education helping students develop other skills. For example, 57 percent thought it helped a lot with critical thinking while 38 percent felt it did not help at all. Among school principals, 73 percent thought that their students had increased skills such as creativity and problem solving compared to previous years, while 68 percent of teachers felt the same. 5.3 TEACHER SKILLS The pandemic has challenged education systems to ensure learning continuity, substantially increasing the demands placed on teachers. Education systems, more than ever, require effective teachers that facilitate and support learning instead of delivering content; that use a 34 | 5. Perceived Effects of the Pandemic on Student Learning and Teacher Skills combination of in-person and digital methods to deliver lessons; that foster creative thinking, communication, and collaboration; and that instill a love of learning, how to persevere, and have self-control. Jaime Saavedra, Global Director of Education, World Bank (Saavedra 2021) Approaches to developing teachers’ digital skills There has been a global rush of education systems toward digital solutions in order to maintain continuity of student learning and progress. However, approaches to the development of teachers’ skills to meet the changed learning environment have been more mixed. Teachers’ skills in utilizing the tools and strategies required to make distance learning both engaging and successful vary across the world. Saudi Arabia’s teachers were comparatively well-prepared to take forward digital approaches. According to the 2018 results from the OECD’s Teaching and Learning International Study (TALIS), 73 percent of teachers in Saudi Arabia reported that the use of information and communications technology (ICT) for teaching was included in their formal education or training, and 76 percent of teachers reported that ICT skills for teaching were included in their existing professional development activities (Mann et al. 2020). These figures are higher than the average of the OECD countries. Educators in Saudi Arabia believe that a high proportion of the teachers could already demonstrate the skills necessary for distance education before the COVID-19 pandemic. The survey carried out in connection with this study established that 59 percent of supervisors and 81 percent of school principals rated a high proportion of teaching staff as having advanced or satisfactory skills for distance education prior to the COVID-19 pandemic. Among teachers, 91 percent indicated that they believed they already had advanced or satisfactory skills for distance education before the COVID-19 pandemic. Intermediary and secondary school teachers interviewed as part of the virtual classroom observations pointed out that there was a high level of preparation for e-learning in the country before the pandemic and so some felt that, from a technical standpoint, little had changed. These teachers had already practiced e-learning solutions and were implementing them in their classes before the pandemic; for example, the Google suite for homework and the flipped classroom model. Teachers had been trained on some of the tools through the MOE’s partnership with Microsoft Corporation, which began in 2011, when several training classes on virtual education were delivered, as well as conferences on e-learning and distance learning, including inputs from experts from around the world. Being a high school teacher, the application was developed long before the start of the pandemic, it started with the ministry’s experience within the Future Gate program, which is a program to implement distance education. Teachers were trained in the use of this technology and we had virtual classes, assignments and discussions on the e-wall. Therefore, the application of e-learning was in advance before the start of the pandemic and we are familiar with it, not only my school, but many schools in all regions. And on the first day when we were forced to stay at home, the education process continued without any interruption, and it was very Saudi Arabia's Digital and Distance Education | 35 easy because we were prepared. Teacher The initial transition to distance education was supported by an immediate teacher professional development training plan. The MOE and National Center for Educational Professional Development (NCEPD) offered online training programs to all school leaders and teachers to aid digital transformation. These programs were selected from training directorates throughout Saudi Arabia and focused on utilizing distance learning applications and basic online teaching practices. A further training plan was put in place during the summer of 2020. This included programs designed to support teachers’ digital literacy and skills, and to raise educators’ awareness of effective distance education strategies and pedagogy. When Madrasati was launched at the beginning of the 2020 academic year, all users were provided with training to assist them to engage with it effectively. Subsequently, educators were provided with various professional development opportunities. Examples included training sessions on cyber security, digital assessment methods and tools, and student engagement techniques.  The helpful tools and applications available on the Madrasati platform are helping teachers to succeed in meeting many students’ learning styles and needs. Teacher Aspects of teachers’ professional skills The main intended outcomes of the teacher professional development programs were clear from the outset. These were to support the transformation to distance education by enabling educators’ digital skills, support teachers’ online teaching performance, and ultimately decrease the impact of the ceasing of regular in-person schooling on learning outcomes. Survey responses indicate that school principals and supervisors believe that almost all teachers in Saudi Arabia now have advanced or satisfactory skills for digital education. This is an increase from over 75 percent before the COVID-19 pandemic closed. Thirty-seven (37) percent of school principals thought teachers in their school had advanced skills for distance learning before the pandemic, while 68 percent felt they had advanced skills by the end of the school year. Almost all teachers (95 percent) believe that they would now be well or very well prepared if distance education returns. Teachers indicated that they had increased their professional skills a lot in a number of key areas. Supervisors believed that teachers’ professional skills had increased a lot, though the percentage of supervisors was smaller in each case (table 5.2). 36 | 5. Perceived Effects of the Pandemic on Student Learning and Teacher Skills Table 5.2 Reported Increase in Teachers’ Skills (%) Teachers who believe that their Supervisors who believe that professional skills increased teachers’ professional skills a lot increased a lot Planning effective lessons 84 68 Delivering effective lessons 84 74 Classroom behavior management 79 63 Managing time 84 68 Engaging and motivating my students 76 55 Assessing my students’ progress 75 58 Supervisors observing virtual lessons in urban and rural settings across Saudi Arabia were able to identify clearly a number of important professional skills being demonstrated by teachers. Around 94 percent of the teachers whose lessons were observed demonstrated a very good or good level of knowledge and skills of digital tools and learning materials. In almost all of the observed lessons, supervisors reported very positively about the quality of pedagogical approaches used by teachers. In several lessons, supervisors noted that teachers had adapted their teaching methods successfully to take good account of the constraints and opportunities of distance learning. In many lessons, effective use was made of a wide range of interactive video clips, digital textbooks, and other virtual tools. For example, in a few lessons, teachers made good use of digital tools, such as Nearpod and Padlet, to display high-quality visual images of plants and animals and to demonstrate chemical processes and reactions. Delivering effective lessons Teachers used their cameras in less than a quarter of all lessons observed by supervisors. In almost all lessons observed, teachers used a wide range of techniques to demonstrate their "presence" as leaders of learning. In many lessons, teachers engaged well with their students at the beginning of lessons, using friendly greetings and showing an interest in their students as individuals. Many supervisors reported how successful teachers had been in creating a safe and happy learning environment where students were enabled to participate and contribute. In most lessons, teachers made effective use of their voices, using a range of suitable and relevant tones. This provided students with variety in their lesson, allowed the teacher to emphasize key points and to acknowledge and praise the efforts of their students. Classroom behavior management In almost all observed lessons, supervisors reported high levels of appropriate student behavior compatible with learning effectively. They reported, too, that almost all teachers acknowledged the positive behavior of their students. In several lessons, teachers referred to an “e-learning charter”, which had been developed at an earlier stage of the school year. This highlighted student responsibilities; for example, lesson entry and exit arrangements, raising hands, and not interrupting classmates. Teachers made good use of the charters, ensuring that students’ behavior and etiquette were conducive to effective digital learning. In almost all lessons, teachers used a range of methods to acknowledge and promote positive student behavior. Saudi Arabia's Digital and Distance Education | 37 Engaging and motivating students Teachers used an effective range of approaches to praise and motivate their students. Appropriate words of praise and thanks were regularly used and small tokens and emojis were frequently used to acknowledge students’ success. Supervisors also observed the use of the flipped classroom where the teacher assigned a student to be a “young teacher” for the duration of the lesson. This allowed the student to lead the lesson and was successful in encouraging discussion and facilitating other learning activities. This approach—if used regularly with a wide range of students—could help to build stronger relationships within classrooms and encourage greater numbers of students to actively participate in lessons. 38 | 5. Perceived Effects of the Pandemic on Student Learning and Teacher Skills 6. Engagement and Well-being Well-being is a crucial aspect of school life and of the whole school community. Schools, and particularly teachers, play an important role in engaging with their students to encourage and motivate them in their learning and to help them to develop a wide range of positive attitudes and values. As part of this, schools have a key role to play in supporting students to make healthy lifestyle choices and to understand the effects of their choices on their health and well-being, particularly following the challenges of the pandemic. This chapter examines aspects of student, parent, and school community engagement with digital and distance education and student well-being. 6.1 SCHOOL COMMUNIT Y All stakeholders—principals, supervisors, teachers, parents, and students themselves—contribute to the quality of learners’ experiences and to the development of their attitudes and values. The strength of the relationships between all these stakeholder partners within the school community helps to determine the levels of students’ engagement, their well-being, and the quality of their achievements. There is a direct and two-way causal link between well-being and academic achievement. Well- being is a prerequisite for achievement. Equally, achievement is important for contributing to that sense of well-being. It will be important that ensuring student well-being is a key aspect of any plans for a return to in-person schooling. Students may have experienced family loss, anxiety, and social isolation from peers during the pandemic. Careful transition planning will be required to ensure that students are well supported as they return to in-person classes. Several countries have established well-being frameworks and formalized support for schools providing evidence-based intervention to guide practice.5 Despite the challenges of the pandemic, including the necessary removal of regular in-person schooling, several, sometimes unexpected, outcomes related to relationships between stakeholders were reported in this study through stakeholder surveys and focus groups. For example, most parents (79 percent), principals (86 percent), and teachers (82 percent) believed that the period of distance education had led to closer relationships between teachers and students and across the whole school community. Similarly, almost all principals (97 percent) and teachers (95 percent) and most parents (87 percent) reported that the period of distance education had provided better opportunities for parents to engage in their child’s learning. Clear and frequent communication between the MOE, schools, and families, along with tools for easy communication, may have contributed to this positive result. 5 See, for example, the Australian Student Wellbeing Framework: https://www.dese.gov.au/student-resilience-and-wellbeing/australian- student-wellbeing-framework; New South Wales Student Wellbeing: https://education.nsw.‌gov.au/ ‌‌student-wellbeing; the Scottish Government https://www.gov.scot/publications/ ‌‌developing-positive-whole-school-ethos-culture-relationships-learning-behaviour/; and UK Wellbeing Framework for Schools: https://www.‌mentallyhealthyschools.org.uk/resources/wellbeing-framework-for-schools/. Saudi Arabia's Digital and Distance Education | 39 6.2 PARENTAL ENGAGEMENT As well as having access to high-quality and well-connected technology, other prerequisites for effective digital and distance learning include a supportive home environment that is conducive to high-quality learning. In Saudi Arabia, distance and digital education has helped to strengthen active parental and family participation in their children’s learning. Distance learning has provided many advantages to the teacher, the student and the family, and it has consolidated the concepts of development and digital transformation, and the culture of family cooperation and involvement in the educational process Supervisor Parental engagement in their children’s learning is accepted as a key factor for effective student achievement in more traditional in-person school settings. The impact of parental involvement has also been shown to be a significant factor in the success of digital and distance learners (Woofter 2019). Students valued the support that they received from their parents in their education, particularly at the beginning of the distance learning process when the arrangements for distance and digital education were new. In many cases, parents facilitated the interaction with the Madrasati system at the outset, helping their children to organize themselves to engage effectively with online learning. At first, I was confused and afraid, and we knew the news from my father; but over a quick period, things got organized; and my father explained to me the program clearly, provided me with the device, and gave me the necessary information Student Encouragingly, evidence through stakeholder surveys and focus groups provides a strong picture of parental buy-in and support for the approach to digital and distance education. Parents reported that communication across the education system was strong and timely. Over 93 percent of parents reported that they received helpful communication about the move to distance learning from the MOE, and 76 percent of parents had been in touch with their child’s school. Parents now use a wide range of approaches to communicate with teachers regarding their children’s learning. One of the most common means of communication between parents and teachers was WhatsApp, which was used by over 82 percent of parents. Commendably, most parents had become actively engaged in their children’s learning with 74 percent attending virtual lessons, and 90 percent following up on their children’s homework, class activities, and assessment results. Parents have welcomed the establishment of a parents’ account on the Madrasati platform, which enables them to track their children’s progress, and to see their grades and their school attendance. Parental involvement in distance education has played an important role in encouraging high levels of student engagement. During the period of distance education, parents have paid greater attention to their children’s attendance at virtual classes, and 40 | 6. Engagement and Well-being to their interactions with teachers and peers. This positive outcome has enabled the level of active parental engagement in their child’s learning to increase, and the study data would suggest that there has been a significant improvement from the TALIS 2018 data. Parents now communicate and cooperate more closely with teachers, which means that additional, individualized support can be provided at home in areas where teachers have identified specific weaknesses in individual students’ learning. For example: The mother knows the level of her son or daughter in a clear way, because she attends lessons with them, not like before during face-to- face education [where] she comes to school, the teacher tells her your daughter is good without going into the details, and that’s it; now the mother knows where her the son or the daughter has a problem—e.g., a certain letter that she did not master, or reading a specific thing—so the mother cooperates with the teacher to help her daughter. Teacher Overall, parental comments about digital and distance learning and their enhanced role in helping to facilitate their children’s learning were very positive. Not unexpectedly, there were a few concerns raised regarding the increased demands on parents, particularly where there were several children learning at home in one family, or where there were issues regarding access to the internet or devices. Teachers and principals were also very positive about how the period of distance learning had resulted in improved engagement with parents. Almost all teachers (95 percent) believe that distance education had provided better opportunities for parents to engage in their child’s learning and 66 percent of principals found that communicating with parents and updating them on their children’s performance was easier this year than in previous years. Some teachers expressed a note of concern over the extent to which some parents provided too much support for their child’s learning. For example, 64 percent of teachers expressed concerns that the extent to which parents themselves completed their child’s assigned work was an occasional or main problem. As the MOE considers its plans for a return to in-person schooling, and explores the introduction of blended learning it will be important to build on the strong levels of parental participation demonstrated during COVID-19 and ensure that parents are fully included in future arrangements for the education of their children. It will be important, too, to ensure that professional development opportunities for school principals and teachers on effective practice in parental engagement, based on existing good practice, are provided in preparation for a return to in-person schooling. 6.3 STUDENT ENGAGEMENT A key feature of Saudi Arabia’s approach to digital and distance learning was the establishment of clear, timetabled school routines. This included a structured start to the school day and required students’ attendance in virtual classes with their regular teachers. Students followed a modified timetable, the duration of which was appropriately reduced in hours to minimize undue screen time. The timing of lessons for elementary school students was moved to later in the day to allow working parents and older siblings to support younger family members. Saudi Arabia's Digital and Distance Education | 41 During the period of distance education, principals were expected to attend in-person school each school day, and teachers were expected to attend one day a week, at which time students could meet with their teacher as needed. In each school, a support team was put in place to help students and their parents if they had any trouble with accessing the online platform and the digital learning resources. However, in responding to the questionnaire, only 6 percent of teachers reported that they met regularly (every week) with students in the school building to give them additional help, and 63 percent reported never meeting students in the school building (the remaining met with students occasionally). In contrast, though, 19 percent of students reported that they had had at least one in-person meeting with their teacher(s) in school. Enhancing the arrangements for monitoring the occurrence of in-person meetings between teachers and students would be beneficial for the MOE to ensure that its policies are adhered to with a high level of consistency. In surveys and focus groups, students expressed their views on learning at home: 88 percent of students reported that they enjoyed learning from home a lot (54 percent) or a little (34 percent), and only 12 percent did not enjoy it at all. More than half of students (57 percent) stated that when they learn from home they could focus more, and they had more options to learn new things at any time based on their own schedule. Students also stated that their parents could help them a lot (30 percent) or a little (42 percent). However, 75 percent reported that following classroom instructions or understanding lessons could be hard (a lot or a little). During distance learning, students also missed opportunities for social interactions. Around two-thirds (67 percent) of students reported that they miss seeing their friends and teachers a lot. Poor internet connections presented some students with difficulties, with 41 percent reporting having a lot of trouble connecting and 45 percent having this problem a little. Most of the students managed to find a quiet space to study: 48 percent of students had no problem finding a quiet area to study and a further 34 percent said that they found it a little hard to find a quiet area. In 80 percent of lesson observations, cameras were off in both male and female teachers’ classrooms. A few teachers provided a rationale for having their cameras off, including privacy considerations (among girls’ schools, for instance), the need and type of activity performed in class, teachers’ preferences, and the quality of internet connection. As distance education approaches may become part of the routine education offer to Saudi Arabian children, it may be helpful to explore this area further and capture the range of issues influencing the preferred option of having cameras off during lessons. Of the 20 percent of teachers who had their cameras on fully or partially during the observed lesson, almost all of those were teachers in the elementary sector. This may indicate that teachers are aware of the importance of active engagement and strong communication in the early years. In the virtual classroom observation sample, the subject being taught had no apparent link with whether cameras were or were not used. However, there is some evidence of an association between specific supervisors (and/or districts) and the use or non-use of cameras. In line with the trend in teachers’ practice, approximately 43 percent of students had their cameras off during the observed lessons. In some cases, teachers reported that they did not ask students to put on their cameras to protect the privacy of parents supporting their children. In several cases students put their cameras on to share work rather than show their face. During the pandemic, practice regarding the use of the camera has been in most cases developed by teachers to suit their own individual circumstances, and the needs of their students and families. Going forward, it would be important to develop clear practice guidelines linked to effective learning and teaching approaches for teachers and supervisors so there is high-level consistency of practice 42 | 6. Engagement and Well-being across the country that builds on existing best practices. Regardless of whether teachers selected to use their cameras, supervisors positively evaluated teachers’ abilities to foster a positive climate within a distance learning context. These approaches included motivational engagement with students, encouragement, fun, participation, student focused learning, skillful use of technology and building positive teacher-student relationships. In most lessons, teachers made effective use of their voices, using a range of suitable and relevant tones. This provided students with a measure of variety in their lesson, allowed the teacher to emphasize key points and to acknowledge and praise the efforts of their students. In many lessons, teachers engaged well with their students at the beginning of lessons, using friendly greetings and emojis, showing an interest in their students as individuals. Many supervisors reported how successfully teachers had used friendly words and warm tones to create a safe and happy learning environment in which students were enabled to participate and contribute. 6.4 STUDENT WELL-BEING In the surveys of parents, 83 percent believed that the impact of Saudi Arabia’s approaches to distance education resulting from the pandemic had either a positive effect (33 percent) or neutral effect (50 percent) on their child’s mental and/or physical health. Around one in six (17 percent) believed that distance learning had a negative impact on their child’s mental and/or physical health. The importance of students’ well-being was raised, too, in each of the focus group meetings of principals, supervisors, teachers, parents and students. Principals and supervisors believed that any initial concerns or challenges to students’ physical and mental health during the pandemic were now being more successfully addressed. A few students in focus groups had described how they were initially fearful and uncertain about arrangements for distance learning. They believed, however, that things had quickly improved with the guidance and support of their parents, teachers, and supervisors. Based on their regular observations of lessons, supervisors in the focus group believed that the mental health of both teachers and students appeared to be positive and that anxiety levels seemed to be low. One supervisor (and some teachers) believed that distance learning was making a positive impact on students who may be shy or be otherwise reluctant to contribute during in-person lessons. On returning to in-person learning, it will be important for schools and teachers to consider how best they can create a similarly supportive and welcoming learning environment in which all students can flourish in the classroom as well as at home. The MOE has issued practice advice related to mental health, which emphasizes the role of student guidance in supporting students online. The role of student guidance includes communicating directly with parents, helping parents to tackle issues such as stress and anxiety, and helping to reduce any feelings of social isolation. The MOE guidance emphasizes the psychological and social well-being of all students. It aims to ensure that all support and aid services are provided to students with disabilities. The guidance provides information and expectations, including about: 1. Enabling programs and group meetings online. 2. Providing students with training workshops through electronic platforms. 3. Following up on students with psychological and behavioral problems. Saudi Arabia's Digital and Distance Education | 43 4. Arranging the transfer of students with psychological and behavioral problems to the Extension Services Unit for further support. These services provide support for more specialized psychosocial, behavioral, and educational issues. Students have a positive attitude toward going back to in-school learning: 92 percent look forward to seeing their friends again; 85 percent to doing better in their schoolwork; 83 percent to meeting their teachers; 76 percent to getting more help with their schoolwork; and 75 percent to participating in school activities. Nevertheless, more than 80 percent of students reported that when they go back to school, they will miss being with their family at home (84 percent) and learning at their own pace (82 percent). Many (57 percent) reported that they will miss the help they get from their parents. In stakeholder surveys, around 80 percent of teachers and supervisors reported that their lessons at home took place in a calmer environment (than is their experience of in-school lessons). Both teachers and parents noted that distance education helped students to concentrate more effectively and offered students fewer distractions to their learning at home compared to school. The very positive behavior of students in a distance learning environment was also commented upon very favorably by supervisors after their observations of learning. However, these results might best be considered in the context of the regular classroom setting in Saudi Arabia, where evidence from TIMSS 2019 suggests that there is scope for improvement in the management of classroom behavior. Table 6.1 provides some evidence that there is scope for improvement in the behavior of students within the traditional classroom setting in Saudi Arabia. Table 6.1 Percentage of Grade 8 Students Reporting that their Teacher Has to Keep Telling them to Follow the Classroom Rules ( % )  Saudi Arabia International average Every or almost every lesson 40.5 25.4 About half the lessons 18.5 16.2 Some lessons 20.7 30.9 Never 20.4 27.4 Source: IEA TIMSS 2019. It will be important that the improvements in behavior, noted in Saudi Arabia, resulting from distance learning approaches are maintained and improved further once students return to in-school learning. In surveys and focus groups, students have shown a commendable capacity to analyze and reflect upon the social and emotional challenges that they experienced in having to adapt to their new learning environments. It will be important to actively engage students in plans for a return to in-person schooling, and to listen to their concerns and support needs, particularly in light of any behavioral issues identified as a feature of in-person schooling by a number of students, and identified in other studies, including TIMSS and PIRLS. 44 | 6. Engagement and Well-being 7. Implications for Next Steps The experience of emergency system-wide distance education provision for over one school year in Saudi Arabia has led to the development of new systems, tools, content, and practices that stand to change K–12 education in profound ways in the future. This bodes well for any future emergency distance education requirements. However, the challenge now lies in safely reopening schools for essential in-person learning, particularly since Saudi Arabia’s children and young people have been away from in-person education for a significant amount of time, harnessing the most effective practices established to date, and reimagining education so that learning takes place for all children and young people, in accessible, affordable, and relevant ways. 7.1 SAFE REOPENING OF SCHOOLS Across the globe, the closure of regular in-person schooling in response to the COVID-19 pandemic has created an unprecedented risk to the education, protection, and well-being of children and young people. To support a safe planned return to in-person learning, UNICEF, UNESCO, the World Bank, and the World Food Programme published a joint “Framework for Reopening Schools”, which provides high-level guidance to support countries in planning for a safe re-opening of schools, along with a supplement “From Reopening to Recovery.” The guidance highlights four areas of priority including: safe operations, identifying and reducing learning loss, meeting the needs of all children, including the most marginalized and those with special educational needs, and addressing the well- being of students and educators. Any decisions to reopen schools must ensure safety is a priority, in line with each country’s overall response to COVID-19. While the “Framework for Reopening Schools” provides sound high-level guidance, it is also important at a national level to consider how plans to reopen schools will impact on school leaders, teachers, students, and their families. The MOE has appropriately identified key issues related to health and safety matters, and mental health and well-being, which need to be included in any strategic plan for the reopening of schools. The study found that communication around the commencement of the emergency response to distance education, from the MOE to all stakeholders, was well managed. This included clear guidance for supervisors, school principals, and teachers on their roles and responsibilities. It will be important that the range of communication channels that were established at the onset of the COVID-19 pandemic are utilized to ensure that all stakeholders are aware of the decision-making process and rationale around the criteria for reopening of schools and the associated arrangements for high quality learning. As schools plan to reopen, it would be particularly important to build on strengthening the relationship between parents and teachers established during digital and distance education. The development of a clear stakeholder communication and engagement strategy would be an important tool in the preparation for post-pandemic learning in Saudi Arabia. While it is clear from the stakeholder surveys that there is a strong desire to return to in-person learning—as stated by 85 percent of supervisors, 75 percent of school principals, 72 percent of Saudi Arabia's Digital and Distance Education | 45 parents, and 63 percent of teachers—it will be important to gain the trust and confidence of those stakeholders who are anxious or reluctant to return to in-person education. 7.2 OPTIONS FOR BLENDED LEARNING MODELS In the longer term, beyond the need for alternative modes of education due to emergency situations such as the COVID-19 pandemic, new methods of delivering education that make the best use of new technologies and practices can be considered. The innovation that took place over the past school year indicates that the education system is ready and able to adapt, with the right conditions and practices in place. Most school principals and teachers (70 percent and 72 percent, respectively) would like to see a hybrid arrangement take place in the future (some days in school and some working from home). More than half of intermediate and secondary school principals believe that a blended education approach would be very effective (53 percent of intermediate school principals and 52 percent of secondary school principals). However, only a third (33 percent) of elementary school principals felt the same. This suggests that distance education is seen as being somewhat less suitable for the younger students at that level, or that additional support would be required to ensure effective implementation of blended learning in elementary schools. Blended learning models Blended learning is a formal education program in which a student learns at least in part through online learning with some element of student control over time, place, path, and/or pace and at least in part at a supervised brick-and-mortar location away from home.6 The modalities along each student’s learning path within a course or subject are connected to provide an integrated learning experience. When adopting blended learning, it is important to first identify the goals that blended learning will help to accomplish. Educational goals, not technology, should guide blended learning design decisions. Possible goals that blended learning can support summarized in box 7.1. 6 https://www.blendedlearning.org/basics/. 46 | 7. Implications for Next Steps Box 7.1 Goals that Blended Learning Can Support 1. To support students with inconsistent attendance 2. To adapt schooling to students’ individual needs 3. To allow students to customize their education 4. To increase parent engagement in students’ learning 5. To improve communication between stakeholders 6. To encourage peer-to-peer learning 7. To make classrooms less crowded 8. To make classrooms more manageable for teachers 9. To ensure excellent instruction across classrooms 10. To give teachers more time for tutoring students individually or in small groups; communicating and building relationships with students and their families; giving students feedback on their work; or designing projects and activities that deepen students’ understanding 11. Other goals as determined by teachers, principals, supervisors, or education authorities. Once goals have been identified, there are a variety of blended learning models that can be used as templates for designing blended learning experiences.7 Among the various blended learning models, below are three that may be highly relevant to Saudi Arabia’s context. 1. Flipped Classroom: This is a model that flips the traditional relationship between class time and homework.8 Students learn at home via online coursework and lectures, and teachers use class time for teacher-guided practice or projects. This model enables teachers to use class time for more than delivering traditional lectures. 2. Enriched Virtual: This is an alternative to full-time online school that allows students to complete the majority of coursework online at home or outside of school, but attend school for required in- person learning sessions with a teacher.9 Unlike the Flipped Classroom, Enriched Virtual programs usually do not require daily school attendance; some programs may only require twice-weekly attendance, for example. 3. Flex: This is a model that lets students move on fluid schedules among learning activities according to their needs.10 Online learning is the backbone of student learning in a Flex model. Teachers provide support and instruction on a flexible, as-needed basis while students work through course curriculum and content. This model can give students a high degree of control over their learning. Each model has advantages and disadvantages when it comes to accomplishing educational goals, as shown in table 7.1. 7 https://www.blendedlearning.org/models/. 8 https://www.blendedlearning.org/models/#flip. 9 https://www.blendedlearning.org/models/#enrich. 10 https://www.blendedlearning.org/models/#flex. Saudi Arabia's Digital and Distance Education | 47 Table 7.1 Relative Advantages of Three Blended Learning Models Flipped Enriched Flex Classroom Virtual Support students with inconsistent attendance Adapt schooling to students’ individual needs Allow students to customize their education Increase parent engagement in students’ learning Improve communication between stakeholders Encourage peer-to-peer learning Make classrooms less crowded Make classrooms more manageable for teachers Ensure excellent instruction across classrooms Give teachers more time for other educational activities Requirements of blended learning models Blended learning models also vary in how much they require a shift in conventional educational practices. The greater the shifts, the more planning and effort are required for ensuring successful adoption (table 7.2). Table 7.2 Shifts Required for Three Blended Learning Models Flipped Enriched Flex Classroom Virtual Ensuring students have devices and internet connectivity needed to access online learning at home and/or school X X X Creating or curating online learning resources that adequately cover the scope of basic content for each course or subject X X X Creating structures to ensure students complete online learning assignments independently and engage with online learning X X X materials Developing schedules and structures for in-person learning that ensure students receive the support they need X X Changing from whole-class pacing and conventional grading to individualized pacing and mastery-based grading X Shifting teachers' mindsets from providing direct instruction to supporting self-directed learning X Shifting students’ mindsets from compliance and passive learning to self-directed active learning X Creating structures to ensure students do not fall behind with self- directed learning and individualized pacing X 48 | 7. Implications for Next Steps Effective implementations of the blended learning models tend to demonstrate the “winning moves” summarized in box 7.2. Box 7.2 “Winning Moves” from Successful Implementations of Blended Learning Models 1. Flipped Classroom Online lesson • Clear and concise • Accurate • Extra quizzes In-person activities • Connected and aligned to the online lesson • Application of the concepts from the online lesson • Peer-to-peer coaching • Planned by a team 2. Enriched Virtual and Flex Group discussion (synchronous; online or in-person) • Friendship building • Open-ended questions • Productive social norms Independent work (asynchronous and online [Enriched Virtual] or in-person [Flex]) • Easy to navigate • Built like a game with points, rules, and goals • Interesting and engaging 1-on-1 Check-ins (in-person, email, phone, video, platform) • Frequent • Quality feedback from the teacher • Positive teacher-student relationship Collaborative Work • Exploring an open-ended question • Productive social norms Other activities (peer coaching, virtual and in-person events) • Selected to meet specific student needs All digital learning requires baseline infrastructure to be feasible. If this is not yet available, the infrastructure to support innovation would need to be created. The necessary baseline infrastructure required is: 1. Access to devices and internet in settings where students will learn. Saudi Arabia's Digital and Distance Education | 49 2. Access to quality online learning resources. Transforming an education system is a difficult undertaking. Even when stakeholders recognize a need or desire to do things differently, it makes no sense to completely tear down or abandon an existing system and replace it with a new system that is unproven and unrefined. Incumbent systems persist because they continue to provide value. However, the challenge when innovating within an incumbent system is that incumbent systems only foster innovations that enhance and improve upon the ways in which the system currently operates. Thus, a “dual transformation” strategy for innovation is recommended. Strategy 1 focuses on innovations that can be fostered within an existing system and produce short-term benefits. Strategy 2 focuses on innovations that need to be nurtured with a degree of independence from existing systems and have more long-term potential for broad transformation. These two potential strategies are detailed in box 7.3. Innovation principles There are four important principles that need to be considered when embarking on innovative solutions such as blended learning. These are: 1. Create a culture of innovation. Educators who work on the front lines need to be involved in designing blended learning models because they are most familiar with the circumstances that a model must address. Encourage educators to be creative and flexible and to approach innovation with a growth mindset. 2. Design innovations to align with local context. Blended learning is not something that can be copied from the United States, for example, and transplanted into Saudi Arabia. Many of the details for successful implementation are not easily specified. Additionally, the models need to be adapted to align with Saudi Arabia’s unique context and goals. Examples of some of the differences in context, which may impact the successful implementation of blended learning models, are described in table 7.3. Table 7.3 An Example of Differences in Contexts for Education Innovations such as Blended Learning Saudi Arabia United States Single public education system administered by 13,000 different public education systems MOE administered by local districts Larger families, on average, with stay-at-home Smaller families, single-parent households, and parents and support from extended families working parents more common Schools typically combine genders, allowing Gender-separated schools greater system efficiencies Most teachers hold a Bachelor degree and a A greater proportion of teachers hold Masters small proportion hold a Masters degree degrees or higher Madrasati platform Many small competing platforms 50 | 7. Implications for Next Steps Box 7.3 A Proposed Dual Transformation Strategy for Blended Learning in Saudi Arabia Strategy 1. Flipped Classroom As existing schools continue with distance learning or start to reopen, the adoption of the Flipped Classroom model could be encouraged. The Flipped Classroom is a blended learning model more easily adopted by existing schools because it maintains many of the features of conven- tional schooling (e.g., age-based student groupings, single teacher classrooms, single-paced instruction). • Train teachers on the benefits and techniques of the Flipped Classroom model. • Provide online learning content that teachers can use to flip their classrooms (e.g., video lessons on Madrasati, lesson materials that teachers can use to easily create online lessons, etc.). • Provide technologies that teachers can use to create their own flipped lessons and monitor students’ engagement and progress with independent online learning (e.g., similar to edpuzzle11 or Pear Deck.12 Strategy 2. Flex and Enriched Virtual Flex and Enriched Virtual models can be difficult to adopt into existing schools because they diverge significantly from the ways in which schools have learned to operate. Rather than trying to push these models into existing schools, create new programs that leverage these models to serve students who are not served well by existing schools (e.g. Enriched Virtual programs for students who live in remote locations or have inconsistent attendance; Flex programs for students who struggle with the uniform instruction or pacing of conventional schools). • Recruit creative and entrepreneurial educators to develop these programs. Provide them with opportunities to study and visit these types of programs. • Allow families to opt into these programs based on their particular needs and inter- ests. (Do not force families or teachers who are satisfied with existing schools into these programs.) • Expand access to these programs as they improve and become more attractive as alternatives to conventional schooling. 3. Organize teams appropriate to the level of innovation. Innovations will fail if the teams responsible for developing those innovations are not organized in a manner appropriate to the level of innovation. Table 7.4 illustrates examples of team structures appropriate for various levels of innovation 4. Use discovery-driven innovation. There are many factors affecting the success of an innovation that cannot be anticipated at the beginning of an innovation design and adoption effort. Teams could use a discovery-driven approach to innovation, as described by McGrath and Macmillan (2009) because many important aspects of an effective blended learning design are hard to anticipate. 11 https://edpuzzle.com/. 12 https://www.peardeck.com/. Saudi Arabia's Digital and Distance Education | 51 Table 7.4 Examples of Team Structures based on Level of Innovation Level of innovation Team Structure New resources (e.g., adopting a Lightweight team — Managers shuttle back and forth among new curriculum) departments to ensure that everyone’s work fits together. Team members represent the interests of their departments. New processes (e.g., changing Heavyweight team — Members of the team leave behind teaching practices, teacher roles) departmental interests and instead work collectively to meet the project’s goal. A manager with significant clout needs to lead the team to settle differences in perspective among the team. New priorities (e.g., engaging Autonomous unit — Members with different areas of expertise disengaged students with Flex or separate from an existing organization to create a new Enriched Virtual programs) organization aligned with new priorities. 52 | 7. Implications for Next Steps 8. Conclusions and Recommendations Saudi Arabia’s journey toward a virtual school model, rolled out at scale from the start of the 2020— 21 school year in response to the requirement to cease in-person schooling due to the COVID-19 pandemic, has been remarkable. Innovation has taken place at all levels, particularly among teachers. There is now a greater awareness of how education technologies can support and enhance teachers’ work and students’ learning experiences. Beyond that, there are now new methods and styles of communication and better awareness of what is happening in classrooms across the country. The resultant innovations and disruption to business-as-usual in Saudi Arabia’s schools will affect change in children’s learning experiences well beyond the pandemic. From an in-depth study of this journey, it is clear that Saudi Arabia succeeded in providing continued K–12 education during the COVID-19 pandemic, particularly from the start of the 2020–21 school year. This included multiple ways of engaging students and a host of new tools, providing teachers with a unique opportunity to experiment and innovate to meet their students’ needs. In developing the digital and distance education, attention was rightly paid to both the technology aspects and the educational aspects, with the teacher and student interaction prioritized, along with the necessary communication aspects that were needed to bring about rapid and widespread change. In terms of achieving continued K–12 education during the prolonged period of distance education, while it has not been possible to determine the proportion of students regularly engaged with virtual schooling throughout the 2020–21 school year, 98 percent of students were able to access the Madrasati platform at some point and at least 75—85 percent engaged with it on a regular basis (with more logging into virtual classes directly through MS Teams and others attending in-person school). This is an impressive achievement given the scale and speed of the rollout and the fact that alternative arrangements were made available such as in-person school visits, recorded lessons, and printed materials. Identifying and supporting those students who could not or did not regularly engage online or who found distance education difficult will be vitally important, with remedial education such as tutoring programs provided as a priority in the upcoming school year. Many strengths within Saudi Arabia’s response to the COVID-19 pandemic in K–12 education have been identified in this study and are summarized in this chapter. These will be important to build on and to share with other countries. In addition, this study identified several strategies, practices, and procedures arising from the emergency digital and distance education that could be further developed to bring about lasting improvements in children’s schooling experiences and learning outcomes. These are also summarized in this chapter along with recommendations for next steps, organized around the World Bank’s five key principles for education investments in EdTech (box 1.1). These strengths and areas for development cover strategic and operational levels, recognizing that change is required at both of these levels and, most importantly, in every classroom across the country. The recommendations extend beyond the EdTech and the distance education because improvements in other areas of education are critical for the success of digital and distance education as well as in-person learning in schools. To help organize the recommendations, a matrix is provided at the end of the chapter (table 8.1). Saudi Arabia's Digital and Distance Education | 53 8.1 DRIVING PURPOSEFUL EDUCATIONAL CHANGE The first principle in the World Bank’s five key principles for EdTech investments is to “Ask Why.” It is not uncommon around the world for EdTech investments to be less effective than expected or difficult to scale up. In many cases, those country investments would have been more effective if there had been an explicit and clear purpose, strategy, and vision for the desired educational change. In these cases, the “technology” has tended to be more of a focus than the “education.” Or, the human and social aspects of education were not placed at the core of the initiatives so that the technology was focused on supporting and enabling those connections. In this case, with the pandemic, there was a clear need for technological solutions to allow millions of children across the country to continue their education during the period of distance education. In Saudi Arabia, the use of digital technologies to mitigate the effect of the need to cease regular in-person schooling due to the COVID-19 pandemic was clearly focused around the students’ learning experiences. Strengths The study found the following key strengths in the planning for purposeful educational change in Saudi Arabia’s digital and distance education. 1. Rapid provision of distance education at scale. Education provision in the form of recorded lessons was made available through satellite television and YouTube channels for more than 6 million K–12 learners within one day of the announcement to cease regular in-person schooling. The speed with which this happened was commendable and made possible by previous investments in prerecorded lessons along with fast decision-making. 2. Student-teacher connections prioritized. In developing their distance and digital education system, Saudi Arabia put students at the center of the initiative. Regular opportunities for virtual live connections between students and teachers (synchronous learning) is a key element of the educational experience for students and their families, strengthening teacher-student and teacher- family engagement. The decision (over the summer of 2020) to mandate daily virtual classroom connections was a bold one, made for the benefit of students’ learning, requiring infrastructure capable of hosting over 5 million students online at the same time. 3. Wide range of tools and resources made available. The Madrasati platform was further enhanced with an extensive range of teaching and learning tools, plus curriculum and enrichment resources, including open educational resources (OER) to support the learning needs of children and young people across K–12. 4. Cycle of EdTech feedback and improvement led to high satisfaction. Constant and close monitoring of students’ and teachers’ experiences with the Madrasati platform throughout the crisis, coupled with authentic responses to the issues identified, led to high rates of satisfaction. In addition, very clear and timely communication with all stakeholders through a variety of channels helped to make the Madrasati roll-out a success. For example, 75 percent of parents were satisfied or very satisfied with the quality of distance education over the last school year, 89 percent of teachers were able to deliver all or nearly all of the lessons that were expected from them, 94 percent of teachers found that accessing the information to implement distance learning was easy, and 97 percent of school principals said that their teachers had found it easy to adapt to the new arrangements. 5. Apparent increase in student skills. Encouragingly, there is a widespread perception that students have developed a range of new educational and global skills during their distance and digital 54 | 8. Conclusions and Recommendations experience, including independent learning, self-reliance, a sense of responsibility, research skills, problem-solving, and digital skills. Among teachers, 68 percent felt that their students’ progress in skills such as creativity and problem solving was better this year compared to previous years. Areas for development and recommendations To further develop clarity in the purpose, strategy, and vision for educational change that utilizes digital technologies and can pivot to distance education when needed, the following would be needed. 1. More explicitly target policies and resources on disadvantaged and struggling students. The effective preparation, communication, monitoring, and responding exhibited during the emergency response should continue into the next phase of school reopening to help all children benefit from the best possible education. Saudi Arabia has had among the longest episodes of nationwide distance education in the world, and the lack of in-person education over this time is likely to have a long-term effect on students, particularly those who were not able to engage well in distance education. Countries around the world (including high-performing countries with good digital infrastructure and distance education experience) have started to document learning losses from emergency distance education provision, particularly in subjects such as mathematics, among the younger age cohorts, and for students lacking the devices, connections, or supports that are needed to fully engage in the distance education that was offered. Now that the Madrasati platform is well embedded in the education system, it will be important to turn more policy attention to those students who need additional support to meet minimum learning standards and reach their full potential. Raising the learning outcomes of underachieving students (those in the bottom levels of performance) will increase overall average learning levels and national human capital formation. 2. Plan for long-term purposeful blended learning models that best suit existing practices and conditions, such as the “Flipped Classroom” and “Flex and Enriched Virtual.” In considering blended models of education for the longer term, there should be a clear statement of the educational purpose and vision for the changes, applying them only where they are needed or would be beneficial compared to existing practices. A proposed dual transformation strategy for blended learning that would be most suitable to Saudi Arabia’s situation is detailed in box 7.3. Strategy 1—the Flipped Classroom can more easily be adopted because it utilizes many of the features of conventional schooling. This model would be used useful for the following goals: a. To ensure all classes receive excellent instructional content (through online coursework and lectures). b. To increase the use of time in class with teacher and peers for active rather than passive learning activities such as teacher-guided practice or projects. c. To enable teachers to use some of the time saved on traditional instruction to gain a better understanding of their students’ learning needs and provide additional support on specific content areas or with individuals or small groups of students who are struggling. Strategy 2—Flex and Enriched Virtual models are further from traditional schooling and would be best applied as new programs and not in existing schools. They can be beneficial for students who live in remote locations or who have inconsistent attendance or who struggle with uniform instruction or pacing of conventional schools (see section 7.1 for further details). Saudi Arabia's Digital and Distance Education | 55 3. Continue to use and adapt the Madrasati platform and develop further digital content. In the short-term, particularly the 2021—22 school year, the Madrasati platform should continue to be used and further developed as (a) an effective and efficient learning organizational platform, (b) a source of content and tools to support delivery of the curriculum, and (c) a communication method. Users clearly want to see the continued use of the platform past the pandemic. As seen in the nationally-representative surveys for this study, many users are also requesting for improvements to be made to the content and to the linking of Madrasati to other required platforms such as NOOR to streamline education information and services. 4. Strengthen the quality of teaching as a priority to improve learning outcomes. Now that “business- as-usual” has been disrupted in Saudi Arabia’s schools, it will be important to seize the opportunity to make lasting changes that improve student learning outcomes. This will mean focusing on the quality of teaching. Teachers have been asked to make many changes in their day-to-day work during the pandemic. Those changes that have improved student learning should continue. Further support to teachers, as they become familiar with the upcoming curriculum reform, should focus on improving their skills in providing a high-quality education for each student. 8.2 DESIGNING AND ACTING AT SCALE, FOR ALL There is global recognition that digital learning initiatives can unintentionally promote inequality of access and engagement (World Bank 2020b). Therefore, the second key principle—“Design and Act at Scale, for All”—emphasizes the importance of flexibility, ensuring that student learning, and not solely the lure of technology, remains at the heart of all developments. In moving from the educational planning due to the COVID-19 pandemic to envisioning a state-of-the-art digital education system of the future, Saudi Arabia must ensure that the system fully meets the needs of all children and young people across at all stages of its development. Strengths 1. Low-tech alternatives for students without internet access. During the distance education, efforts were made to provide for students without access to Madrasati through dedicated iEN satellite TV channels and mandated regular visits to their school to receive printed materials and support from their teacher, with schools submitting weekly reports to ensure that this follow-up was happening. 2. Provision of special distance education. Satellite TV channels were put in place for students with disabilities and special educational needs, along with school visits as needed. While a detailed review of this type of education was beyond the scope of the study, the fact that multiple modes of education provision were offered can be seen as a strength. Areas for development and recommendations There is a significant risk of widening disparities with continued distance education or blended models given that some students were not able to log-in consistently or were unable to engage fully for various reasons. Urgent attention is therefore needed on the following aspects. 1. Provide additional targeted resources to students not regularly attending virtual classes. Schools should identify and support these students through the regular in-person class schedules and, in some cases, with additional targeted tutoring. Where schools have large proportions of students in this situation, additional resources should be given to those schools to help them cater to these disadvantaged students. For any further digital and distance education provision, those children not attending virtual classes through the Madrasati platform should be monitored in terms of their 56 | 8. Conclusions and Recommendations attendance and engagement, with this data made available at the district, region, and central levels. 2. Set policies for long-term access of teachers and students to devices and connectivity for both distance and in-person schooling. Efforts were made through charitable organizations to source devices and cards for internet access for students to support digital learning. However, data were not available to assess the level of need for these and the degree to which the need was met. A long-term plan will be needed to ensure all students and teachers have access at home and in school to devices and connectivity. Any future digital and distance education or blended learning models will require this, along with technical support within schools. These policies should include attention to online safety. Careful selection and prioritizing of the selected infrastructure and procedures will be needed to ensure all schools have the basic requirements for digital education both within and outside of school buildings. 3. In the short-term, ensure that a plan for providing devices and connectivity to all students who do not have them is feasible and enacted as a matter of urgency. If blended models such as the Flipped Classroom are expected in the next or future school years, all students should have access to the required devices and internet connections at home. A plan to make this happen should be developed or, if already developed, shared with stakeholders to ensure that it is feasible, and all parties are on board. One of the most cited areas for development raised by stakeholder groups in the nationally-representative surveys was to improve the infrastructure—internet connection and access to devices—to improve accessibility and reduce inequality. All respondent groups included comments on the importance of providing better broadband coverage to ensure the provision, reliability, and stability of the internet connection, especially in remote and developing areas. 4. Promote policies and communications that emphasize high expectations for all students across all key skills. In moving to the “post-pandemic” stage, if there is a further requirement to cease regular in-person schooling or limited attendance is required for health reasons, the MOE should ensure that all students, whether learning at home or in-school, continue to have the same opportunities to build on their new skills, including technical and social skills. This means setting policies and communications to schools and teachers that emphasize the need to set high expectations for each and every student across all key skills. 8.3 EMPOWERING TEACHERS In preparation for a national move to digital and distance education in 2020, Saudi Arabia invested heavily in the professional development of teachers, extending their knowledge and understanding of the Madrasati platform and digital learning, and supporting new teaching and learning practices as they moved into distance education delivery. The third principle—“Empower Teachers”—looks at empowering teachers to better support student learning. Building on best practices to strengthen teaching and learning in all classrooms will be an important driver of sustained change in returning to in-person learning. Strengths Supervisors reported that, overall, digital learning has increased student motivation and engagement, and encouraged many teachers to use a more innovative range of resources creating higher-quality learning experiences for students. Overall, it is clear that there is high demand from supervisors and teachers for more digital content and for improvements in the quality of the available digital content. Saudi Arabia's Digital and Distance Education | 57 1. Significant investments in teacher professional development. The investment in professional development for educators and leaders across all stages to support digital and distance education has been strong. Saudi Arabia worked at speed to roll out extensive professional learning at scale to empower and equip staff with the knowledge, skills, and understanding to interact with, and utilize, the many tools and resources of the Madrasti platform. 2. High satisfaction with professional training and support. There has been high and consistent satisfaction across all stakeholder groups with the training and support provided to access the tools and resources provided through the Madrasati platform. 3. Good use of teacher communities of practice and coaching. The use of in-school coaching with assigned school digital learning focal points, and the creation of professional communities of practice, have enabled teachers to share good practices. This has increased teachers’ confidence to use new digital educational tools in their virtual classes. 4. Apparent increase in teacher skills. School principals noted an increase in teachers’ skills related to distance education, with just 37 percent rating their teachers’ skills as “advanced” before the pandemic, rising to 68 percent during the school year. More than 80 percent of teachers felt that their skills had increased “a lot” in planning and delivering effective lessons and managing their time. Areas for development and recommendations Saudi Arabia has put in place an extensive range of professional learning opportunities to upskill teachers with the knowledge, skills, and practice to deliver high-quality digital and distance learning during the pandemic. The following are areas for further development. 1. Reduce variability in teacher performance through targeted professional learning, and increase consistency of practice through clear guidelines for digital and distance education, for example on student engagement and use of cameras. While many teachers have increased their professional skills to support student achievement, there is a need to reduce variability in teacher performance to effectively plan and deliver high-quality learning experiences for all students. For example, supervisors noted that lesson planning was not sufficient for some of the lessons they observed as part of this study. This means that professional learning should be targeted to where it is needed, especially for schools and teachers that have not yet reached good performance levels in digital and distance education. In addition, clear practice guidelines for digital and distance education, linked to effective learning and teaching approaches, should be shared with teachers and supervisors to achieve a high level of consistency of practice across the country, for example on issues of student engagement and camera use, where considerable variation of practice was found. 2. Review and strengthen roles of school principals and supervisors to lead education improvement. Supervisors and school principals developed a range of leadership skills throughout the pandemic to build capacity and quality assure teaching and learning in a digital and distance learning context. Post-pandemic, it will be important to ensure that all teachers are provided with a similar level of support and guidance from supervisors and school principals. More broadly, it will be important to review and strengthen the support and quality assurance roles of the school principals and supervisors as this will be a key driver for improving the effectiveness of the education system. 3. Identify and promote innovative and evidence-based effective teaching and learning practices 58 | 8. Conclusions and Recommendations in digital and distance education. It would be important to disseminate examples of innovative practice developed by teachers across schools and districts, such as the “young teacher strategy” to promote and encourage active student engagement and collaboration, building student leadership for learning. 4. Evaluate the impact of teachers’ learning to better understand which teacher professional development investments have been most impactful and should continue. To determine the impact of the extensive range of professional learning activities developed by the MOE and partners, it will be important for the MOE to evaluate the quality of teachers’ and educational leaders’ professional learning activities to determine which aspects have made the biggest difference to the quality of learning and teaching in “classrooms” across the country. Going forward, it will be important for the MOE to collect and collate national impact data. This will allow the MOE to understand the overall level of effectiveness of the resources it has invested in, plus highlight which aspects of the program have worked well and which need amendment. 8.4 ENGAGING THE ECOSYSTEM The adage that “it takes a village to educate a child” was never truer than in planning for post- COVID education. The fourth principle, “Engage the Ecosystem” outlines the importance of engaging the wider education community in providing a high-quality education experience for Saudi Arabia’s children and young people. The MOE has effectively engaged a wide range of actors in the development and delivery of their digital education system including wider governmental agencies, students, teachers, school principals, supervisors, curriculum specialist, EdTech and ICT specialists, parents, academia, and the private sector. Strengths Saudi Arabia has successfully brought together voices from across the education, ICT and EdTech communities in developing a comprehensive response to the COVID-19 pandemic building on the previous investments in e-learning, significant accumulated experience in educational technologies, quick decision-making, and well-coordinated efforts across multiple entities. Specific strengths include the following. 1. Frequent and clear communication with stakeholders. Communication has been strong, with clear and timely guidance that allowed all stakeholders to understand their roles. 2. Leading private sector providers engaged early with clarity on expected outcomes. Based on previous EdTech experiences, Saudi Arabia was able to move quickly with an understanding of how to procure, develop, and locally enhance the digital tools. Working with experienced and world-leading companies was a key success feature of the journey. 3. Enhanced parental engagement. Distance education arrangements established through the Madrasati platform have significantly strengthened parental and family engagement in their children’s education. Teachers communicated most often with parents about students’ attendance and achievement, their welfare, and the technical problems of access and connectivity. This enhanced connection between schools and families, if continued, will be a powerful driver for educational improvement and stands to be one of the greatest outcomes of the digital and distance education experience in Saudi Arabia. Saudi Arabia's Digital and Distance Education | 59 Areas for development and recommendations The engagement model used for the development and implementation of the Madrasati platform should similarly be used to address upcoming needs as schools recommence in-person learning. The socioemotional needs of children and young people will need to be addressed in addition to the academic learning. The following practices should be enacted across the country, not just in pockets of good practice only. 1. Support and require teachers to recognize and meet the socioemotional needs of their students. Distance learning challenges relating to students’ physical and mental well-being were raised by stakeholders, including feelings of isolation, boredom, laziness, lack of physical activity, eye strain from too much time on a device, and concern over the lack of social interaction with their peers. Going forward, it will be important to support the knowledge and understanding of teachers to adapt their lessons and expectations to effectively meeting the socioemotional needs of students, particularly at the time of transition from distance to in-person learning. In addition, the MOE should ensure that the well-being of students is promoted, for example through stronger links to parents and the deployment of school counsellors focused on student welfare, to maintain the positive advantages created through the period of distance and digital education. Careful transition planning will be required to ensure that students are well supported as they return to in-person classes. Several countries have established well-being frameworks and formalized support for schools providing evidence-based intervention to guide practice.13 2. Engage school communities in setting high behavioral standards and creating positive school cultures and climates. It will be important to explore further the feedback from supervisors, school principals, teachers, parents and students regarding overcrowding, in-school bullying, and poor behavior that impedes learning. All schools should provide a supportive and nurturing environment for learning, and have in place clear behavioral expectations of all adults and children, addressing any behaviors that are felt to impede students’ learning. Raising the skills of teachers and school principals in effective classroom management techniques would be one way to ensure appropriate practices across all schools. 3. Employ strategies to gain the trust of stakeholders who may be anxious about a safe return to in-person schooling, for example by encouraging schools to incorporate student voices in return to school plans. As schools and families prepare for a return to in-person schooling, there will be some who are anxious due to the continued threat of the virus and because it marks a significant change to the past 18 months, where many students and teachers felt very comfortable and safe. Therefore, it will also be important to actively engage students and their families in plans for a return to in-person schooling, and to listen to their concerns and support their needs. This is particularly true in light of behavioral issues identified as a feature of in-person schooling by a number of students, and identified in other studies, including TIMSS and PIRLS. 13 See, for example, the Australian Student Wellbeing Framework: https://www.dese.gov.au/student-resilience-and-wellbeing/austra lian-student-wellbeing-framework; New South Wales Student Wellbeing: https://education.nsw.‌gov.au/ ‌‌student-wellbeing; the Scottish Government https://www.gov.scot/publications/ ‌‌developing-positive-whole-school-ethos-culture-relationships-learning-behaviour/; and UK Wellbeing Framework for Schools: https://www.‌mentallyhealthyschools.org.uk/resources/wellbeing-framework-for-schools/. 60 | 8. Conclusions and Recommendations 8.5 INCORPORATING DATA AND EVIDENCE FOR ONGOING IMPROVEMENT The fifth principle, “Be Data-Driven”, highlights the importance of not only collecting data but analyzing it to form data-driven decisions that will improve the quality of learning and teaching, and importantly the learning outcomes for children. Strengths There is evidence of a move toward the frequent collection and use of relevant data for decision- making, as follows. 1. Regular monitoring of user data and feedback loops to improve tools and services. Monitoring of Madrasati usage and user interaction with the platform was regular and resulted in proactive actions and decisions. In addition, feedback from users was regularly sought and acted upon. 2. New tools and items to support student assessment. The Madrasati platform includes an extensive range of assessment tools to support formative and summative assessment practices. This includes almost 94,000 test items to support the development of subject-based assessments. With the inclusion of these tools and items, clear signals were given to schools and teachers of the importance of assessing students’ progress and communicating that to students and their parents. Areas for development and recommendations A number of areas within this study could not be examined in the necessary detail due to a lack of data or sufficiently robust data. Some stakeholder groups also highlighted areas of improvement that are needed in this area. The following would be considered priority areas for development. 1. Ensure continuity in national assessments to provide crucial information on overall levels of student learning. A stable system of externally marked, standardized national student assessments will allow valid monitoring over time. The National Assessment of Learning Outcomes (NALO) has been implemented for several years covering different subjects, but has not yet been implemented to allow a comparison over the COVID-19 pandemic phase against previous years. This meant that the effect of the pandemic on student learning could not be assessed directly. This should be a priority for the next school year, implemented at an appropriate time (after students have settled into the new school year), ensuring that the instrument used can be matched to a recent pre-COVID-19 baseline. As the NALO now moves to a census basis, it will be important to allow a significant period of stability in the national assessment (avoiding frequent design and methodology changes) to reap the benefits of monitoring changes in learning levels over time. 2. Reexamine student assessment policies and consider developing a national assessment framework to coordinate and communicate intent. Distance education challenges around measurement and evaluation of student achievement were frequently cited as areas for development by stakeholders in the nationally-representative surveys for this study. Respondents mentioned that it is difficult to know the accurate level of student performance and learning, while others questioned the credibility and fairness of assessments. Similar issues have been raised during regular in-person schooling (for example, in OECD 2020). Valid, consistent, and reliable monitoring of student learning by teachers, districts, and the center is necessary to identify those children needing additional support and to understand the overall levels of learning and progression. A national assessment framework (outlining the full suite of assessment instruments) would be one way to ensure coordination of the various student assessments—showing adequate coverage across Saudi Arabia's Digital and Distance Education | 61 grades and purposes while limiting assessments that have no or limited educational value—and to communicate the intended purpose and rationale of each assessment to all stakeholders. 3. Improve data on student attendance and engagement with distance education and in-person schooling. More accurate monitoring of student attendance in virtual settings would be needed in the future. Schools had a clear mandate to follow-up on those students who did not have access to the digital learning. However, data is not centrally available to enable a judgment on the degree to which all students received an adequate education during the 2020–21 school year, whether online or in-person. Systems of central monitoring of student attendance in class—both in-person and at a distance—would benefit from review to ensure that students are not left behind and that there is a better understanding of the extent of attendance and engagement across the country. 4. Assess and address learning losses on return to in-person schooling. There is a need to identify and address learning loss across the student population, creating evidence-based interventions to ensure each student’s achievement is appropriately on track. It would be important that, on return to in-person education, teachers are equipped to meet the challenge of rapidly assessing students’ knowledge to identify learning gaps. This knowledge should be used to plan for learning by creating appropriate learning pathways for all students, but particularly those students in transition or preparing for high-stakes examinations. 5. Revisit student assessment tools and practices, plus training of educational professionals, to better track students’ development of critical cross-disciplinary and 21st century skills. There is a need to strengthen and modernize formative and class-based assessment, including effective in-class questioning techniques, to realistically track and monitor students’ progress using data trends, and enable high-quality feedback to be provided directly to students to bring about improvement. This will require targeted professional development for teachers, supervisors, and school principals. It would be particularly important to capture the development and progression of the new global skills, namely a sense of initiative; self (independent) learning; critical thinking; logical analysis; problem-solving; and communication and presentation skills as part of a student’s profile. Going forward, consideration should be given to extending the range of assessment tools to capture these important skills for learning and life. 8.6 ADDITIONAL CONSIDERATIONS In addition to the areas for development and recommendations captured under the above framework, as the MOE implements its new plans for education reforms, which stretch well beyond recovery from the COVID-19-pandemic distance education, there are a number of points that should be considered based on global research, experiences, and lessons learned. 1. Curriculum and teacher assessment redesign to move away from predetermined lesson delivery to aligning instruction to students’ current learning levels, skills, and goals. In the new educational reforms, attention should be given to building skills based on where students are at with their learning and achievement, rather than delivering a set of lessons for students to absorb. This concept of skill building, starting from where children are, needs to permeate throughout the education system and be well understood by all of those involved in educating children and young people. It means re-examining how the curriculum is designed and communicated to teachers, and how teachers are assessed in supporting their students to gain these skills, ensuring that the curricular breadth and depth is re-established and further enhanced moving forward, particularly for the important foundational skills such as literacy and numeracy. Attention needs 62 | 8. Conclusions and Recommendations to be given to how expected practices and standards are communicated to teachers and parents, and the types of teaching and learning materials and activities that are promoted. 2. Time expectations for curricular reforms. Embedding meaningful curricular reform into the daily teaching practices and learning experiences of all children and young people takes time when done well. In some cases, from international examples, around 5 to 10 years are necessary. Expectations related to the time needed to successfully implement the new curricular reforms should be set accordingly, particularly in the context of wide ranging and rapid change in teaching practices during the COVID-19 pandemic. 3. Building on the successful communications and parental engagement strategies. Finally, there is a rare opportunity now to build on the strengths created by the educational response to the COVID-19 pandemic, particularly in relation to the clear communication process between schools and families as the return to in-person learning is rolled out, which appears to have increased substantially, with parents now far more knowledgeable about school practices. This may be one of the biggest successes of the past year—opening up the classroom—and one that will have long-term benefits. It will be important, too, to ensure that professional development opportunities for school principals and teachers on effective practice in parental engagement, based on existing good practice, are provided in preparation for a return to in-person schooling. Saudi Arabia's Digital and Distance Education | 63 Table 8.1 Summary Matrix of Recommendations 5. Incorporating Data and 1. Driving Purposeful 2. Designing and Acting at 3. Empowering Teachers 4. Engaging the Ecosystem Evidence for Ongoing Educational Change Scale, for All Improvement Recommendations specific to digital and distance education and COVID-19 pandemic response • Continue to use and adapt • In the short-term, ensure that a • Identify and promote innovative • Employ strategies to gain the • Improve data on student the Madrasati platform and plan for providing devices and and evidence-based effective trust of stakeholders who may be attendance and engagement with develop further digital content connectivity to all students who teaching and learning practices in anxious about a safe return to in- distance education and in-person do not have them is feasible and digital and distance education person schooling, for example by schooling enacted as a matter of urgency encouraging schools to incorporate student voices in return to school plans • Plan for long-term purposeful • Provide additional targeted • Assess and address learning losses blended learning models that resources to students not regularly on return to in-person schooling best suit existing practices and attending virtual classes conditions, such as the “Flipped Classroom” and “Flex and Enriched Virtual.” Recommendations related to broader areas of education reform including digital and distance education • More explicitly target • Set policies for long-term access of • Reduce variability in teacher • Support and require teachers • Ensure continuity in externally policies and resources on teachers and students to devices performance through targeted to recognize and meet the assessed, standardized national disadvantaged and struggling and connectivity for both distance professional learning, and increase socioemotional needs of their assessments to provide crucial students and in-person schooling consistency of practice through students information on overall levels of clear guidelines for digital and student learning distance education, for example on student engagement and use of cameras • Promote policies and • Evaluate the impact of teachers’ • Revisit student assessment tools communications that emphasize learning to better understand and practices, plus training of high expectations for all students which teacher professional educational professionals, to better across all key skills development investments have track students’ development of been most impactful and should critical cross-disciplinary and 21st continue century skills Recommendations specific to other areas of education reform • Strengthen the quality of • Review and strengthen the roles of • Engage school communities in • Reexamine student assessment teaching as a priority to school principals and supervisors setting high behavioral standards policies and consider developing a improve learning outcomes to lead education improvement and creating positive school cultures national assessment framework to and climates coordinate and communicate intent 64 | References References Azevedo, Joao Pedro, Amer Hasan, Diana Goldemberg, Syedah Aroob Iqbal, and Koen Geven. 2020. “Simulating the Potential Impacts of COVID-19 School Closures on Schooling and Learning Outcomes : A Set of Global Estimates.” Policy Research Working Paper No. 9284. Washington, DC: World Bank. https://openknowledge.‌ ‌‌ handle/‌10986/33945 License: CC worldbank.‌org/ BY 3.0 IGO. Donnelly, Robin, and Harry Anthony Patrinos. 2021. “Learning Loss During COVID-19: An Early Systematic Review.” Covid Economics 77, 30 April 2021: 145-153. https://cepr.‌org/content/ covid-economics-vetted-and-real-time-papers-0. Dorn, Emma, Bryan Hancock, Jimmy Sarakatsannis, and Ellen Viruleg. 2020. COVID-19 and Learning Loss Disparities Grow and Students Need Help. McKinsey and Company. https://www.mckinsey. com/industries/public-and-social-sector/our-insights/covid-19-and-learning-loss-disparities-grow- and-students-need-help. Education Policy Institute. 2021. Understanding Progress in the 2020/21 Academic Year: Initial Findings from the Spring Term. London: Department for Education. https://assets.publishing.service.gov.uk/ government/uploads/system/uploads/attachment_data/file/991230/Understanding_Progress_ in_the_2020_21_Academic_Year_Initial_Report_3_.pdf Fink, D. L. 2005. Integrated Course Design. Manhattan, KS: The IDEA Center. Retrieved from http:// www.ideaedu.org/Portals/0/Uploads/Documents/IDEA%20Papers/IDEA%20Papers/Idea_ Paper_42.pdf. Kastberg, Signe. 2003. “Using Bloom’s Taxonomy as a Framework for Classroom Assessment” The Mathematics Teacher 95 (6). McGrath, Rita Gunther and Ian C. Macmillan. 2009. Discovery Driven Growth: A Breakthrough Process to Reduce Risk and Seize Opportunity. Harvard Business School Press. Mann, Anthony, Markus Schwabe, Pablo Fraser, Gabor Fülöp, and Grace Adoley Ansah. 2020. How the COVID-19 Pandemic is Changing Education: A Perspective from Saudi Arabia. Paris: OECD Publishing. Ministry of Education. 2020. The Saudi MOE: Leading Efforts to Combat Coronavirus Pandemic (COVID 19). https://iite.unesco.org/wp-content/uploads/2020/10/The-Saudi-MOE-Leading-Efforts-to- Combat-Coronavirus-Pandemic-COVID-19.pdf. OECD (Organisation for Economic Co-operation and Development). 2020,  Education in Saudi Arabia, Reviews of National Policies for Education. Paris: OECD Publishing. https://doi. org/10.1787/76df15a2-en. Saudi Arabia's Digital and Distance Education | 65 Saavedra, Jaime. 2021. “Realizing the Promise of Effective Teachers for Every Child—A Global Platform for Successful Teacher.” https://blogs.worldbank.org/education/‌realizing-promise-effective-teachers- every-child-global-platform-successful-teachers. Sturgis, Patrick, Patten Smith, and Graham Hughues. 2006. A Study of Suitable Methods for Raising Response Rates in School Surveys. Research Report No. 721. UK Department for Education and Skills. UK Higher Education Academy. 2016. HEA Surveys 2016: Raising Response Rates. https:// www.‌heacademy.ac.uk/system/files/downloads/guides-raising-response-rates.pdf UNESCO, UNICEF, World Bank, World Food Programme, and UNHCR. 2020. Framework for Reopening Schools. ‌‌pdf/ https://documents1.worldbank.org/curated/en/‌1139015889‌1060‌58‌61‌/ Framework-for-Reopening-Schools.pdf UNESCO, UNICEF, World Bank, World Food Programme, and UNHCR. 2021. Framework for Reopening Schools Supplement: From Reopening to Recovery—Key Resources. World Bank. 2021. Policy Actions for School Reopening and Learning Recovery. https://www.worldbank. org/en/news/factsheet/2021/04/30/notes-on-school-reopening-‌and-‌learning-recovery. World Bank. 2020a. The COVID-19 Pandemic: Shocks to Education and Policy Responses. Washington, DC: World Bank: https://openknowledge.worldbank.org/handle/10986/‌33696. World Bank. 2020b. Reimagining Human Connections: Technology and Innovation in Education at the World Bank. Washington, DC: World Bank. Woofter, S. 2019. Book Review: Building Equity: Policies and Practices to Empower All Learners. American Journal of Qualitative Research, 3(1), 136- 139. https://doi.org/‌10.29333/‌ajqr/5815. 66 | References Appendixes Saudi Arabia's Digital and Distance Education | 67 APPENDIX A. DURATION OF SCHOOL CLOSURES BY COUNTRY A pp e n dix A . D u r a t i o n o f S c h o o l C lo su re s b y C o u n try Number of weeks of full or partial school closings due to the COVID-19 pandemic (ordered by highest to lowest number of full closure Number of weeks of full or partial school closings due to the COVID-19 pandemic (ordered by highest to lowest number of full closure w weeks) Full P artial Full P artial Full P artial Full P artial closure closure closure closure closure closure closure closure Bangladesh 47 47 P oland 21 35 S omalia 15 15 Finland 8 23 P anama 46 46 E thiopia 21 31 E ritrea 14 49 S eychelles 8 19 E l Salvador 45 45 Turks and Caicos Island 21 21 Chile 14 44 Cameroon 8 18 Mexico 44 44 Bhutan 20 49 Qatar 14 41 Zambia 8 18 Myanmar 43 48 Indonesia 20 48 Kyrgyzstan 14 38 Belgium 8 15 Kuwait 42 42 North Macedonia 20 43 J amaica 14 37 Aruba 8 13 Iraq 41 50 Gambia 20 28 Germany 14 28 Croatia 8 10 S audi Arabia 41 41 Cabo Verde 20 20 Namibia 14 25 P alau 8 8 Bolivia (P lurinational State of) 39 47 Gibraltar 20 20 Y emen 14 19 Botswana 7 20 Costa R ica 39 46 S uriname 20 20 Cayman Islands 14 17 Cote d'Ivoire 7 13 Brazil 38 44 Bahamas 19 41 Maldives 14 16 Curaçao 7 9 Venezuela 38 38 Trinidad and Tobago 19 40 Canada 13 40 Mauritius 7 8 Uganda 37 50 S lovenia 19 34 Bosnia and Herzegovina 13 38 Djibouti 7 7 Honduras 37 46 Barbados 19 31 Latvia 13 36 Monaco 6 13 E cuador 34 44 E quatorial Guinea 19 22 Italy 13 35 France 6 10 Guatemala 33 45 Montserrat 19 22 Thailand 13 25 P apua New Guinea 6 6 J ordan 33 42 Algeria 19 21 Central African republic 13 23 S witzerland 6 6 P hilippines 33 34 Czechia 18 38 Grenada 12 45 Norway 5 19 P araguay 32 40 Hungary 18 33 S lovakia 12 37 S valbard 5 19 Dominican R epublic 32 32 Haiti 18 31 Netherlands 12 22 Viet Nam 5 14 Bahrain 29 44 S aint Vincent and the Grenadines 18 28 S enegal 12 22 Liechtenstein 5 13 S aint Lucia 29 40 S int Marteen 18 28 P ortugal 12 21 Tuvalu 5 12 Lebanon 29 37 Malawi 18 26 Mali 12 17 Uruguay 4 27 Comoros 29 35 Nigeria 18 24 Fiji 12 13 Anguilla 4 16 E swatini 28 49 Guinea-Bissau 18 23 R epublic of Korea 11 41 Benin 4 15 S outh Sudan 28 49 Ukraine 18 19 S yrian Arab R epublic 11 29 S ingapore 4 11 Angola 28 46 P alestine 17 49 Cyprus 11 25 Faroe Islands 4 7 S ri Lanka 28 43 Bulgaria 17 32 Micronesia (Federated States of) 11 24 S olomon Islands 4 7 Kenya 28 37 Greece 17 32 S ão Tomé and P ríncipe 11 19 Cook Islands 4 4 Democratic P eople's R epublic of Korea 28 36 Cambodia 17 29 E stonia 11 16 S amoa 4 4 R wanda 27 47 Morocco 17 21 United R epublic of Tanzania 11 15 Madagascar 3 16 Guyana 27 45 Malta 17 19 S ierra Leone 11 14 J apan 3 11 Nepal 26 53 Dominica 17 17 Togo 11 14 New Zealand 3 9 Turkey 26 38 Iran (Islamic R epublic of) 16 45 Uzbekistan 11 12 Greenland 3 4 India 25 51 Montenegro 16 36 S aint Kitts and Nevis 11 11 Vanuatu 3 4 Mozambique 25 45 Israel 16 33 Timor-Leste 11 11 Kiribati 3 3 Belize 25 38 Georgia 16 32 Ghana 10 40 Tonga 3 3 Malaysia 25 35 United Kingdom 16 27 Bermuda 10 29 Marshall Islands 1 2 S erbia 25 34 Gabon 16 21 Lithuania 10 29 Niue 1 1 Zimbabwe 25 34 E gypt 16 19 Congo 10 26 United States of America 0 47 Antigua and Barbuda 24 41 Niger 16 16 S pain 10 15 Australia 0 27 Democratic R epublic of the Congo 24 33 R epublic of Moldova 16 16 Lesotho 9 41 S weden 0 23 Colombia 23 43 P eru 15 42 British Virgin Islands 9 40 Nicaragua 0 15 Azerbaijan 23 37 United Arab E mirates 15 42 Kazakhstan 9 36 R ussian Federation 0 13 P akistan 23 33 Libya 15 40 China 9 27 Iceland 0 6 Afghanistan 23 31 Liberia 15 37 Lao P DR 9 19 Tokelau 0 4 Chad 23 28 S an Marino 15 33 Brunei Darussalam 9 17 Belarus 0 0 Mauritania 23 23 S outh Africa 15 29 Luxembourg 9 15 Burundi 0 0 Argentina 22 46 Tunisia 15 28 Burkina Faso 9 14 Nauru 0 0 Mongolia 22 34 Austria 15 27 Armenia 9 12 Tajikistan 0 0 R omania 22 32 Cuba 15 24 Oman 8 26 Turkmenistan 0 0 Ireland 22 26 S udan 15 22 Albania 8 24 Guinea 22 22 Andorra 15 16 Denmark 8 23 Source: UNESCO global monitoring map of school closures (accessed on June 5, 2021): https://en.unesco.org/covid19/educationresponse. Data as of March 29, 2021. Note: Full school closures refer to situations where all schools were closed at the nationwide level due to COVID-19. Partial school closures refer to school closures in some regions or for some grades, or with reduced in-person instruction. 92 68 | Appendixes APPENDIX B. SURVEY WEIGHTING METHODOLOGY Weights were calculated for the questionnaires given to school principals, teachers, students, parents, and supervisors to account for selection probability and non-response. Details of the methodology for calculating the weights are provided below. Principal Questionnaire Weights Schools were selected using systematic sampling with probability proportional to size (PPS), with the measure of size (MOS) being the total school enrollment. The probability of a school being selected was: MOS x 200 TOTMOS Where MOS is the total enrollment in the school, 200 is the number of schools selected, and TOTMOS is the total enrollment of all schools. The school base weight is the inverse of the probability of selection, or TOTMOS MOS x 200 A correction for non-response was added at the level of each of the 13 administrative regions. The correction was: Expected Number of Schools Number of Schools in the Achieved Sample Finally, the weights were adjusted so that the weighted number equaled the size of the achieved sample: Sample n in cell x Number in the Achieved Sample Number of Schools in the Achieved Sample Teacher, Student, and Parent Questionnaire Weights To calculate teacher, student, and parent questionnaire weights, the school principal questionnaire included questions on the number of classes for the selected subjects (Arabic, mathematics, science, and English), and the number of students in the class selected for the student and parent questionnaires. However, the quality of the data received for these questions were poor indicating a widespread misunderstanding of the questions. For this reason, weights for the teacher, student, and parent questionnaires were based on ensuring that the sample proportions matched the population proportions by splitting the sample and population into cells based on the 13 administrative regions, whether it was a boys’ or girls’ school, and type of school (regular, Qur’anic, etc.). The weight per cell was: Percentage in Population Percentage in the Achieved Sample Supervisor Questionnaire Weights Supervisor weights (as with the teacher, student and parent questionnaire weights) were based on Saudi Arabia's Digital and Distance Education | 69 match sample and population proportions by splitting the sample and population into cells. For the supervisor questionnaire weights the cells were based on the 13 administrative regions and subject (Arabic, mathematics, science, and English). 70 | Appendixes Appendix CC. APPENDIX . Sc h o o l P rin c ip SCHOOL a l Q u e s tio n n PRINCIPAL a ire R e s po n s e s QUESTIONNAIRE RESPONSES Percentage Percentage ofof school school principals principals E lementary Intermediate S ec ondary All levels Male F emale Overall Male F emale Overall Male F emale Overall Male F emale Overall 1a. I am a: Male principal 54 - 34 92 6 57 79 9 42 76 6 45 Female principal 46 100 66 8 94 44 22 92 58 24 94 55 1b1. T he s c hool level that I s upervis e is : E lementary 100 100 100 - - - - - - 33 19 30 Intermediate - - - 100 100 100 - - - 39 39 37 S econdary - - - - - - 100 100 100 28 43 33 1b2. S tudent' genders in the s c hools I s upervis e are: Male 100 - 62 100 - 59 100 - 47 100 - 56 Female - 100 25 - 100 42 - 100 52 - 100 40 Male and female - - 13 - - - - - 1 - - 4 2a. I would rate the s kills of the teac hers in my s c hool for distanc e educ ation before the C OVID-19 pandemic as : Advanced 23 44 27 53 41 48 27 43 35 36 42 37 S atisfactory 58 46 58 28 45 35 38 40 40 41 43 44 Needing improvement 18 11 15 20 15 18 35 17 25 24 15 19 2b. I would rate the s kills of the teac hers in my s c hool for distanc e educ ation now as : Advanced 47 76 56 72 72 72 63 82 73 61 77 68 S atisfactory 42 19 32 25 28 27 35 18 26 34 22 28 Needing improvement 11 6 12 3 - 2 2 - 1 5 1 5 2c . I have us ed the following to help me s upport my work during distanc e educ ation: Advice from MOE A lot 76 58 67 66 67 66 61 69 65 68 66 66 About right/satisfactory 17 35 26 29 30 29 30 22 26 25 27 27 A little 7 8 6 5 2 4 9 10 9 7 6 6 Not at all 0 - 0 0 2 1 - - - 0 1 0 Advice from education departments/administrations A lot 75 71 69 59 61 60 61 64 63 65 64 64 About right/satisfactory 19 20 25 30 35 32 30 29 29 26 30 29 A little 6 6 5 11 4 8 8 7 8 9 6 7 Not at all - 3 1 0 0 0 1 - 0 0 1 0 Advice from education offices A lot 82 59 70 60 58 59 62 43 53 68 52 60 About right/satisfactory 11 32 20 31 39 34 32 26 28 24 32 28 A little 4 9 5 7 3 5 5 30 18 5 16 9 Not at all 4 - 6 3 - 2 2 1 2 3 1 3 Advice from professional  networks learning  A lot 68 54 63 49 55 52 48 39 44 55 48 52 About right/satisfactory 12 39 22 26 41 32 39 37 38 25 39 31 A little 19 5 14 18 3 12 9 17 13 16 9 13 Not at all 1 3 2 7 1 4 4 8 6 4 4 4 Online professional development courses and materials   A lot 51 60 52 47 69 56 51 64 57 49 65 55 About right/satisfactory 35 36 38 26 28 26 45 24 35 34 28 33 A little 14 4 10 28 3 18 4 12 8 17 7 12 Not at all 1 - 0 - - - - - - 0 - 0 Guidance from B ack to S chool portal A lot 29 38 29 36 36 36 38 28 32 34 33 33 About right/satisfactory 27 38 34 23 50 35 48 30 39 32 39 36 A little 39 18 32 39 7 26 8 34 21 31 20 26 Not at all 4 6 4 2 8 4 6 8 7 4 8 5 Madrasati  technical support A lot 43 50 44 37 53 44 61 34 47 46 44 45 About right/satisfactory 44 29 38 31 40 35 22 45 33 33 40 35 A little 11 21 16 25 3 16 17 14 15 18 11 16 Not at all 3 - 2 7 4 6 1 8 4 4 5 4 2d. Overall, I have found leading my s c hool during the C OVID-19 pandemic distanc e educ ation: Very easy 20 27 20 42 34 39 22 27 25 29 30 29 E asy 73 59 72 52 55 54 73 60 66 65 58 63 Difficult 6 43 8 4 11 7 5 13 9 5 12 8 Very difficult 1 2 1 2 - 1 - - - 1 0 1 2e. Overall, teac hers in my s c hool have found adapting to the new arrangements : Very easy 20 26 21 37 24 32 31 43 38 30 32 31 E asy 75 72 72 62 74 67 69 55 61 68 65 66 Difficult 5 2 7 0 2 1 1 3 2 2 2 3 Very difficult - - - - - - - - - - - - Saudi Arabia's Digital and Distance Education | 71 ofschool Percentageof Percentage school principals principals (cont.) (cont.) E lementary Intermediate S ec ondary All levels Male F emale Overall Male F emale Overall Male F emale Overall Male F emale Overall 2f. T his year, I have been going into the s c hool building: Daily 92 94 92 89 87 88 92 98 95 91 93 92 4-5 times a week 8 6 8 11 5 8 7 2 5 9 4 7 2-3 times a week - - - - 4 2 - - - - 2 1 One day a week - - - 0 4 2 1 - 1 0 2 1 3a. During distanc e educ ation, I would rate the following as pec ts of my role as : R eviewing  the academic achievement indicators for my school:  Very easy 33 33 32 40 27 35 24 41 32 33 34 33 E asy 58 65 58 52 64 57 70 49 59 59 58 58 Difficult 9 1 10 8 9 9 6 4 5 8 5 8 Very difficult - 2 0 - - - - 7 3 - 3 1 Monitoring virtual classes and giving feedback to teachers in my school: Very easy 37 54 44 46 41 44 43 44 43 42 45 44 E asy 57 45 48 50 56 53 51 43 48 53 48 49 Difficult 6 2 9 1 3 2 6 13 9 4 7 6 Very difficult - - - 3 - 2 - - - 1 - 1 Communicating with parents and updating them on their children’s performance: Very easy 24 20 27 32 25 29 21 44 33 26 32 30 E asy 38 46 37 47 53 50 67 26 45 49 40 44 Difficult 35 35 35 21 18 20 10 24 17 23 23 24 Very difficult 3 - 2 - 4 2 3 7 5 2 5 3 P reparing  teachers’ attendance schedules  and making sure all classes have a teacher present: Very easy 70 52 45 54 51 52 49 64 57 48 57 52 E asy 56 46 52 38 49 43 48 27 37 47 39 44 Difficult 5 2 3 8 - 4 3 9 6 5 4 5 Very difficult - - - 1 - 0 1 - 0 0 - 0 4a. Overall, my teac hers ’ s kills in online ass ess ment and test building are: Advanced 41 59 43 56 77 65 58 66 63 51 69 58 S atisfactory 52 34 51 43 23 35 40 34 37 45 30 40 Needing improvement 7 8 6 2 - 1 2 - 1 4 2 3 4b. When a teac her is abs ent, the c lass is us ually: Given another teacher to deliver their online class 50 58 49 51 50 50 31 34 32 45 45 44 Given independent (extracurricular) work to do 34 26 33 24 28 26 43 26 34 33 27 31 Given the time off 16 16 19 25 22 24 26 40 34 22 29 26 4c . F ollowing up on teac hers ’ written plans for ass igned c lass es is : Very easy 25 38 32 44 46 45 33 61 47 35 51 42 E asy 68 59 59 51 54 52 65 24 45 60 42 52 Difficult 7 2 9 5 - 3 1 15 8 5 7 7 Very difficult - - - - - - - - - - - - 5a. C ompared to previous years , the ac ademic progress (e.g., c ontent knowledge and s kills in s ubjec ts s uc h as s c ienc e, Arabic language, etc .) of students in my s c hool this year has : Increased 48 87 61 65 74 69 57 44 51 57 64 61 Not changed 10 9 9 21 16 19 32 39 35 20 24 21 Decreased 48 4 30 14 10 13 12 17 14 23 12 18 5b. I know this bec aus e: I have clear data to show this 18 20 18 32 18 26 18 33 26 23 25 23 It is my professional judgement50 29 41 42 26 36 27 19 23 41 24 33 B oth 32 51 42 26 56 38 56 48 51 36 52 44 5c . C ompared to previous years , my students ’ s kill development (e.g., c reativity, problem s olving, etc .) this year has : Increased 52 74 59 76 77 76 84 81 83 70 78 73 Not changed 32 23 30 18 19 19 12 12 12 8 17 20 Decreased 15 3 11 6 5 5 4 7 5 21 5 7 5d. I know this bec aus e: I have clear data to show this 12 19 14 27 10 20 13 23 18 18 17 17 It is my professional judgement 60 45 51 41 29 36 46 20 33 49 28 40 B oth 28 36 36 32 61 44 41 57 49 33 55 43 5e. In my s c hool, I would rate distanc e educ ation for: across the school  Meeting the needs of students of different ability levels     Very effective 13 30 18 37 28 34 18 35 27 24 31 27 E ffective 35 46 35 48 40 45 67 33 50 49 38 43 P artly effective 50 22 45 14 32 22 13 26 20 26 28 28 Not effective 3 2 3 0 - 0 1 7 4 1 3 2 Allowing me to  keep an overview of  progress   students’  across the school   Very effective 8 27 16 30 27 29 24 42 34 21 33 26 E ffective 69 65 64 46 42 45 63 36 48 59 44 52 P artly effective 22 6 20 23 31 26 9 13 11 19 18 19 Not effective 1 2 1 1 - 1 4 10 7 2 5 3 72 | Appendixes Percentage Percentage of of school school principals principals (cont.) (cont.) E lementary Intermediate S ec ondary All levels Male F emale Overall Male F emale Overall Male F emale Overall Male F emale Overall P romoting students’ ability to become independent learners   Very effective 6 29 16 46 33 40 21 52 37 26 40 32 E ffective 41 54 42 39 38 39 49 32 41 43 38 41 P artly effective 49 15 39 14 24 18 28 10 18 30 16 24 Not effective 3 2 3 1 5 3 1 7 4 2 5 3 Delivering the curriculum  Very effective 15 45 26 51 34 44 23 29 26 31 34 33 E ffective 46 44 43 45 48 46 58 51 55 49 49 48 P artly effective 39 11 31 4 18 10 19 19 19 20 17 19 Not effective - - - 0 - 0 - 1 1 0 1 0 6a. C ompared to previous years , I have found distanc e educ ation makes : Communicating with parents and updating them on their children’s performance:  E asier 49 64 51 80 69 76 67 67 67 66 67 66 No change 33 8 27 15 22 13 27 17 22 24 13 20 More difficult 17 28 22 5 9 12 6 16 11 9 20 15 Allowing my teachers to regularly provide students with feedback about the quality of their work E asier 48 63 58 80 88 83 81 84 83 70 82 76 No change 43 27 34 6 9 7 16 12 14 21 14 17 More difficult 9 10 82 14 3 10 3 4 3 9 5 7 E ngaging students   in their learning E asier 55 79 62 79 92 84 84 86 85 72 87 78 No change 33 16 29 17 4 12 13 10 11 21 9 17 More difficult 12 6 9 4 4 4 4 4 4 7 5 6 7a. T he impac t of distanc e educ ation on the mental and phys ic al health of students in my s c hool has been: P ositive 42 58 43 70 67 69 58 54 56 57 60 57 Neutral 40 28 32 13 19 16 25 24 24 25 23 23 Negative 18 14 26 17 14 16 17 22 20 17 17 20 7b1. T he impac t of distanc e educ ation on the mental and phys ic al health of teac hers in my s c hool has been: P ositive 70 75 66 81 75 79 73 64 69 75 70 72 Neutral 21 19 19 16 10 14 17 24 20 18 18 18 Negative 9 5 15 3 14 8 9 13 11 7 12 11 7b2. T he impac t of distanc e educ ation on the mental and phys ic al health of the princ ipal my s c hool has been: P ositive 35 77 45 74 57 67 66 70 68 58 66 61 Neutral 49 18 35 18 26 21 31 9 20 32 17 25 Negative 17 6 20 9 17 12 3 21 12 10 17 14 7c . C ompared to previous years , levels of teac her abs enc e in my s c hool during distanc e educ ation have been: Higher 4 17 7 8 9 8 7 9 8 6 11 8 About the same 5 4 4 3 2 3 7 - 3 4 2 3 Lower 91 80 89 90 89 90 87 91 89 89 88 89 7d. Onc e s c hools open again, I think students will need this amount of time to c atc h up with their learning: Less than one semester 63 76 65 83 72 78 75 73 75 74 73 73 One semester 30 23 29 6 19 11 20 14 16 18 17 18 T wo semesters 5 2 4 11 10 11 5 5 5 7 6 7 12+ months 2 - 1 - - - - 8 4 1 4 2 7e. If s c hools have a hybrid or blended approac h (c ombining fac e-to-fac e teac hing and distanc e educ ation) in the future, I think it will be: Very effective 37 41 33 52 54 53 57 49 52 48 50 47 E ffective 52 25 45 31 28 30 30 30 31 38 29 35 P artly effective 8 15 11 18 14 16 12 11 11 13 13 13 Not effective 3 19 11 0 4 2 2 10 6 2 9 6 7f. I find that the benefits of distanc e educ ation are: It leads to a closer relationship between teacher and students in class Yes 87 77 76 90 94 92 90 89 90 89 89 86 No 13 23 24 10 6 9 10 11 10 11 11 14 It connects the school community as a whole (students/parents/teachers)  together  Yes 91 81 90 91 98 94 91 91 91 91 92 92 No 9 19 10 9 2 6 10 9 9 9 8 8 It results in a  more efficient  leading to better  learning environment  student  learning  Yes 81 94 84 85 83 84 87 87 87 84 87 85 No 19 7 16 15 18 16 13 13 13 16 14 15 It helps students to be more engaged in the lesson Yes 88 85 82 83 83 83 89 82 85 86 83 84 No 13 15 18 17 17 17 11 18 15 14 17 17 Saudi Arabia's Digital and Distance Education | 73 Percentageof Percentage ofschool schoolprincipals principals(cont.) (cont.) E lementary Intermediate S ec ondary All levels Male F emale Overall Male F emale Overall Male F emale Overall Male F emale Overall It provides better  opportunities  for personalized learning Yes 95 97 91 100 91 96 94 97 96 97 94 94 No 5 4 9 0 10 4 6 3 5 4 6 6 It provides better opportunities for parents to engage in their child’s learning  Yes 99 98 99 98 95 97 96 98 97 98 97 97 No 1 2 1 2 5 3 4 3 3 2 3 3 It reduces   my  workload overall  Yes 95 81 84 96 90 94 90 89 89 94 88 90 No 5 19 16 4 10 6 10 11 11 6 12 10 It provides more choices   for delivering lessons, making teaching more interesting Yes 92 94 93 89 87 88 86 85 85 89 87 89 No 8 6 7 11 13 12 14 15 15 11 13 11 7g. I think that the c onstraints to distanc e educ ation are: A  lack of opportunities for students to learn social skills Yes 93 68 86 65 64 65 74 61 68 77 64 72 No 8 32 14 35 36 35 26 39 32 23 36 28 It is difficult to engage students   for the full lesson Yes 87 52 78 67 67 67 65 58 61 73 60 68 No 13 49 22 33 33 33 35 43 39 27 40 32 It restricts  the  ability to determine how well the students are following the lesson  teachers’  Yes 86 71 79 49 67 56 52 57 55 62 63 63 No 14 29 21 51 33 44 48 43 45 38 37 37 It creates more work for  me  Yes 84 57 77 55 77 65 58 45 49 65 60 64 No 16 43 23 45 23 36 42 55 51 35 40 36 It requires too much screen time Yes 96 98 97 99 94 97 96 95 96 97 95 97 No 4 2 3 1 6 3 4 5 5 3 5 4 T he scheduled times for online classes affect my family life (prevents me from spending time with my family)    Yes 66 53 59 59 42 42 14 32 23 42 61 41 No 35 47 41 41 58 59 86 68 77 58 40 60 It creates more opportunities for (cyber)  bullying    Yes 40 37 45 61 40 39 24 43 34 35 41 39 No 60 63 55 39 60 61 76 57 66 65 59 61 T here is too much parental interruption  Yes 45 80 58 58 43 52 30 42 36 46 49 48 No 55 20 42 52 57 48 70 58 64 54 51 52 7h.What would you like to s ee happen in the future? A return to in-person school as soon as possible Yes 82 85 85 75 70 73 70 68 69 76 72 75 No 18 15 16 25 30 27 30 32 31 24 28 25 T eachers and students continuing to use  Madrasati    and digital content in the classroom and for homework when schools reopen for in-person education Yes 87 76 82 90 91 90 94 81 88 90 84 87 No 13 24 19 10 9 10 6 18 12 10 16 13 S ome days of the week in school and some days   learning from home  Yes 74 72 67 80 65 73 90 54 70 81 61 70 No 26 28 34 20 35 27 10 46 30 20 39 30 T he option for students to  learn  full-time from home Yes 40 44 36 64 57 61 51 47 50 53 51 50 No 60 56 64 36 43 39 49 53 51 48 50 50 More digital content created for teachers and students to use Yes 97 99 98 99 100 99 96 92 94 98 96 97 No 3 1 2 1 0 1 4 8 6 2 4 3 Improvements in the quality of the digital content  Yes 97 100 98 98 99 99 94 99 97 97 99 98 No 3 - 2 2 1 1 6 1 3 3 1 2 74 | Appendixes . Te Appendix DD. APPENDIX a ch e r Q u e s TEACHER tionnaire Responses RESPONSES QUESTIONNAIRE Percentage Percentage ofof teachers teachers E lementary Intermediate S ec ondary All levels Male F emale Overall Male F emale Overall Male F emale Overall Male F emale Overall 1a. I am a: Male teacher 100 - 52 100 - 50 100 - 49 100 - 50 Female teacher - 100 48 - 100 50 - 100 51 - 100 50 1a. I teac h grade: 3 elementary 40 46 43 - - - - - - 11 13 6 elementary 60 54 57 - - - - - - 18 15 3 intermediate - - - 100 100 100 - - - 32 32 3 secondary - - - - - - 100 100 100 39 42 1b. I teac h: Arabic 44 37 41 32 27 30 17 23 20 30 28 29 Math 28 29 28 24 24 24 19 18 18 23 23 23 S cience 18 21 20 24 27 25 49 46 47 32 33 33 E nglish 10 13 11 20 22 21 16 13 15 15 16 16 1c . I teac h: Male students 98 3 52 97 - 49 98 - 47 68 1 49 Female students 1 90 44 3 100 51 2 100 52 2 97 50 F+ M students <1 6 3 - - - < 1 <1 < 1 <1 2 1 1d. I have been teac hing for: Less than 1 year - < 1 <1 - - - - <1 < 1 - <1 <1 1 to 5 years 9 5 7 18 8 13 12 13 13 13 9 11 6 to 10 years 13 22 17 25 30 27 15 33 24 18 29 23 More than 10 years 78 74 76 57 62 60 72 54 63 69 62 66 2a. I would rate my s kills for distanc e educ ation before the C OVID-19 pandemic as : Advanced 36 53 44 32 39 35 32 42 37 33 44 39 S atisfactory 52 42 47 58 55 56 54 49 51 55 49 52 Needing improvement 12 4 9 11 6 8 14 9 11 12 7 10 2b. T his year, my s kills have inc reas ed in: P lanning effective lessons A lot 76 94 85 79 89 84 75 90 83 77 91 84 A little 3 3 3 1 - 1 4 1 2 3 1 2 Not at all 21 4 13 20 11 15 21 9 15 21 8 14 Delivering effective lessons A lot 77 92 85 81 89 85 76 88 82 78 90 84 A little 3 2 3 3 1 2 3 1 2 3 1 2 Not at all 20 6 13 16 9 13 21 11 15 19 9 14 Classroom behavioral management A lot 82 86 84 71 84 78 75 79 77 76 82 79 A little 2 3 3 5 2 3 8 3 5 6 3 4 Not at all 16 11 13 24 14 19 17 19 18 19 15 17 Managing my time A lot 83 85 84 80 85 83 79 89 84 81 87 84 A little 3 2 3 4 3 4 5 2 3 4 2 3 Not at all 14 13 14 16 11 14 14 9 12 15 11 13 E ngaging and motivating my students A lot 78 88 83 68 82 75 65 21 73 70 83 76 A little 3 4 3 7 3 5 10 4 7 7 4 5 Not at all 19 8 14 25 15 21 26 15 20 24 13 18 Assessing my students' progress A lot 68 86 76 68 80 74 64 85 75 66 84 75 A little 5 4 5 7 2 4 9 4 6 7 3 5 Not at all 27 10 19 25 18 22 28 11 19 27 13 20 2c . I would des c ribe the level of c ommunic ation and advic e I rec eived to help me prepare for distanc e educ ation as : T oo much 33 44 38 26 38 32 25 36 31 28 39 33 About right 54 48 51 64 53 58 53 55 54 57 52 54 T oo little 11 7 9 9 8 8 18 7 12 13 7 10 None 2 2 2 1 2 2 13 2 3 3 2 2 2d. T he following helped me during distanc e educ ation: Advice from MOE A lot 48 49 48 67 48 42 37 48 43 40 48 44 About right 26 36 31 37 34 35 29 35 32 30 35 32 A little 18 13 16 21 14 17 21 14 17 20 14 17 Not at all 8 3 5 6 4 5 13 3 8 9 3 6 Advice from E ducation  departments A lot 40 42 1 33 41 37 33 35 34 35 39 37 About right 27 40 33 36 37 37 27 42 34 30 40 35 A little 25 13 19 20 16 18 23 16 19 22 15 19 Not at all 9 6 7 11 6 8 18 7 12 13 6 10 Saudi Arabia's Digital and Distance Education | 75 Percentage Percentage of of teachers teachers (cont.) (cont.) E lementary Intermediate S ec ondary All levels Male F emale Overall Male F emale Overall Male F emale Overall Male F emale Overall Advice from E ducation  offices A lot 42 44 43 34 44 39 37 34 36 37 40 39 About right 31 37 34 38 36 37 26 45 36 31 40 36 A little 20 13 17 17 16 16 23 14 18 20 14 17 Not at all 8 6 7 11 4 8 14 7 10 11 6 9 Madrasati  technical support (phone/ticketing/call center/chat bot/ FAQ) A lot 36 39 38 28 35 32 34 35 34 33 36 34 About right 27 37 32 34 34 34 26 23 29 28 34 31 A little 22 16 19 22 18 20 21 18 20 22 18 20 Not at all 15 8 12 16 13 14 19 15 17 17 12 15 Advice from other teachers A lot 54 58 56 57 59 58 59 62 61 57 60 58 About right 22 29 25 26 27 26 24 29 26 24 28 26 A little 27 10 14 13 12 12 13 8 10 15 10 12 Not at all 7 3 5 4 3 3 4 2 3 5 2 4 my  Advice from  school principal A lot 73 65 69 71 60 66 69 60 64 71 61 66 About right 18 27 22 19 32 26 17 30 24 18 30 24 A little 7 6 7 7 7 7 12 8 10 9 7 8 Not at all 2 2 2 2 1 2 1 2 2 2 2 2 Advice from  my  supervisor A lot 46 54 50 51 52 51 47 48 47 48 51 49 About right 33 31 32 32 30 31 30 35 32 32 32 32 A little 15 10 13 10 14 12 14 12 13 13 12 13 Not at all 5 5 5 7 4 6 10 5 7 8 5 6 P rofessional development courses A lot 38 58 48 37 49 43 35 47 41 36 51 44 About right 32 29 31 36 39 37 29 40 34 32 37 34 A little 23 11 17 19 10 15 29 13 21 24 12 18 Not at all 7 2 4 8 2 5 8 <1 4 8 1 4 E ducational  R esources found  the Internet  through searching  A lot 61 75 68 63 75 69 60 73 67 62 74 68 About right 20 17 19 23 20 22 23 23 23 22 20 21 A little 17 8 12 10 5 8 13 3 8 14 5 9 Not at all 1 1 1 3 - 2 3 <1 2 3 <1 2 Guidance from  B ack to S chool  portal A lot 30 39 34 27 31 29 26 29 28 28 32 30 About right 26 35 30 31 37 34 26 41 34 28 38 33 A little 23 16 20 23 20 21 28 18 23 25 18 21 Not at all 21 10 16 19 12 16 19 12 16 20 12 16 Advice from professional learning communities   A lot 35 42 39 32 45 38 33 41 37 33 43 38 About right 28 37 32 33 36 34 27 37 32 29 37 33 A little 26 13 20 21 13 17 25 16 21 24 14 19 Not at all 10 8 9 14 6 10 15 5 10 13 6 10 2e. Overall, I find that acc ess ing the information and s upport I need to implement distanc e educ ation is : Very easy 42 41 41 44 43 44 40 36 38 42 40 41 E asy 52 54 53 48 52 50 51 61 56 50 56 53 Difficult 4 4 4 6 5 6 8 3 6 6 4 5 Very difficult 2 1 2 2 < 1 1 1 <1 1 2 <1 1 3a. When I teac h my less ons , I us e: Madrasati  Always 80 91 86 83 93 88 88 92 90 84 92 88 Often 10 3 7 10 5 8 8 6 7 10 5 7 S ometimes 9 4 6 4 2 3 3 2 2 5 2 4 Never 1 1 1 2 - 2 1 - 1 1 <1 1 Microsoft T eams Always 76 74 80 80 87 84 86 85 86 81 85 83 Often 11 7 9 10 7 9 7 5 6 9 6 8 S ometimes 10 8 9 8 6 7 5 9 7 8 8 8 Never 2 1 2 1 1 1 1 1 1 2 1 1 Other Microsoft Office 365 products (e.g., OneNote, Forms, P owerP oint, Word)   Always 39 65 52 44 70 57 38 73 56 40 70 55 Often 31 21 26 22 17 20 30 17 23 28 18 23 S ometimes 26 11 19 31 12 21 24 10 17 27 11 19 Never 3 3 3 3 1 2 9 <1 4 5 1 3 76 | Appendixes Percentage Percentage of of teachers teachers (cont.) (cont.) E lementary Intermediate S ec ondary All levels Male F emale Overall Male F emale Overall Male F emale Overall Male F emale Overall National E ducation P ortal  IE N  Always 37 40 33 30 36 33 22 30 26 26 34 30 Often 23 21 22 19 29 24 24 22 23 22 24 23 S ometimes 39 35 37 41 32 36 39 42 41 40 37 38 Never 11 5 8 11 3 7 15 6 10 13 5 9 IE N TV channels   Always 12 22 17 11 16 13 10 12 11 11 16 13 Often 11 19 15 13 17 15 12 16 14 12 17 14 S ometimes 44 38 41 44 47 46 42 48 45 43 45 44 Never 33 21 27 32 20 26 36 24 30 34 22 28 channels   Y ouTube  Always 19 39 29 22 37 29 23 26 24 21 33 27 Often 20 18 19 23 23 23 20 24 22 21 22 22 S ometimes 46 33 40 39 36 38 44 43 43 43 38 40 Never 15 10 13 16 4 10 13 7 10 14 7 11 Hardcopy textbooks   Always 38 45 41 34 40 37 30 38 34 34 41 37 Often 28 19 23 22 20 21 32 21 26 27 20 24 S ometimes 25 28 26 32 31 32 31 32 32 30 30 30 Never 10 8 9 12 8 10 6 10 8 9 9 9 (e.g.,  E ducational sites and apps   edu-games, platforms, quizzes, etc.)   Always 33 49 41 14 47 30 22 41 32 23 45 34 Often 20 30 25 29 31 30 27 30 28 25 30 28 S ometimes 35 20 28 43 20 32 41 26 33 40 23 31 Never 12 1 7 14 2 8 10 3 6 12 2 7 E -books   Always 52 62 57 52 54 53 43 50 46 48 55 52 Often 18 14 16 21 20 20 23 27 25 21 21 21 S ometimes 22 21 22 20 22 21 24 19 21 22 20 21 Never 8 3 5 7 4 5 10 5 8 8 4 6 3b. Outs ide virtual c lass es , I as k my students to us e (ass ignments /tas ks /homework): Madrasati  E very day 43 65 54 35 64 50 32 56 44 36 61 49 Weekly 43 24 34 45 25 35 49 34 41 46 28 37 Occasionally 11 10 11 18 10 14 19 8 13 16 9 13 Never 2 1 1 2 <1 1 1 1 1 2 1 1 products   Microsoft Office 365  (e.g., OneNote, Forms, P owerP oint, Word)   E very day 24 33 29 24 34 29 20 29 25 22 32 27 Weekly 27 32 30 28 29 29 28 33 31 28 31 30 Occasionally 39 31 35 43 34 39 41 35 38 41 34 37 Never 10 4 7 5 3 4 12 3 7 9 3 6 IE N National E ducation P ortal  E very day 22 26 24 23 30 26 17 23 20 21 26 23 Weekly 11 22 16 20 14 17 20 12 16 17 16 16 Occasionally 54 40 47 35 46 41 49 50 49 46 46 46 Never 13 11 12 22 10 16 15 15 15 17 12 14 IE N TV channels   E very day 10 20 15 15 20 17 13 12 13 13 17 15 Weekly 13 15 14 9 11 10 12 46 11 12 11 11 Occasionally 51 42 47 44 48 46 45 9 46 46 45 46 Never 26 23 24 32 22 27 30 33 31 29 27 28 Y ouTube channels   E very day 15 26 21 24 27 26 20 17 19 20 23 21 Weekly 12 16 14 17 16 16 17 12 14 15 14 15 Occasionally 57 45 51 38 47 42 48 56 52 47 50 49 Never 16 13 15 21 11 16 16 15 15 17 13 15 /electronic   Hardcopy  textbooks   E very day 49 52 50 44 57 51 38 49 44 43 53 48 Weekly 16 19 17 16 9 12 19 13 14 17 13 15 Occasionally 29 26 28 30 30 30 35 31 33 32 30 31 Never 6 3 4 11 4 7 8 6 7 8 5 6 E ducational games   E very day 11 32 21 9 27 18 6 20 13 8 25 17 Weekly 19 24 21 13 23 18 9 23 16 13 23 18 Occasionally 54 38 46 50 42 46 49 47 48 50 43 47 Never 17 6 12 29 8 18 36 10 23 28 8 18 Saudi Arabia's Digital and Distance Education | 77 Percentage Percentage ofof teachers teachers (cont.) (cont.) E lementary Intermediate S ec ondary All levels Male F emale Overall Male F emale Overall Male F emale Overall Male F emale Overall 3c . When c onduc ting less ons , I us e: A  smartphone Always 37 31 34 24 21 23 26 19 23 29 23 26 Often 11 15 13 15 14 14 13 13 13 13 14 13 S ometimes 33 31 32 33 38 36 33 36 35 33 36 34 Never 19 23 21 28 27 27 28 32 30 25 28 27 A tablet Always 18 20 19 9 18 14 12 15 13 13 17 15 Often 6 10 8 9 9 9 8 19 8 8 9 8 S ometimes 22 20 21 22 24 23 22 8 21 22 21 22 Never 54 51 52 60 48 54 58 58 58 57 53 55 A laptop Always 55 69 61 61 80 70 58 82 70 58 78 68 Often 10 13 12 13 9 11 14 5 9 13 8 11 S ometimes 27 14 21 18 9 14 17 9 13 20 10 15 Never 8 4 6 8 2 5 10 5 7 9 4 6 A P C (desktop)  Always 17 18 18 12 7 10 13 15 14 14 13 14 Often 3 6 4 7 5 6 8 14 6 6 5 6 S ometimes 15 16 16 16 17 17 16 4 15 16 16 16 Never 65 60 62 64 71 67 63 67 65 64 66 65 4a. I c onduc t virtual c lass es : Always 84 94 89 83 94 88 87 94 91 85 94 89 Often 7 1 4 9 3 6 7 2 5 7 3 5 S ometimes 9 5 7 8 3 5 6 4 5 8 3 6 Never - - - - - - - - - - - - 4b. T he teac hing and learning materials I have us ed for my less ons this year have been different to the c ontent I us ed in previous years : I  58 am using a mixture of previous 58 learning teaching and 58materials 59 and new  61 resources   60 54 61 58 57 60 58 19 No, I am still mostly using teaching and learning materials 15 17 that 22 I previously 16 used19 25 15 20 23 15 19 Yes, my lessons     incorporate23 completely 27 25 different content than 19 previous23 years   21 20 25 23 21 25 23 4c . I s et ass ignments and homework for my students to c arry out between my live virtual c lass es : Always 50 66 58 43 65 54 50 62 56 47 64 56 Often 32 21 26 30 25 27 26 21 23 29 22 25 S ometimes 16 10 13 22 8 15 21 12 16 20 10 15 Never 2 3 3 6 2 4 3 5 4 4 4 4 4d. I c reate plans for students who are struggling and falling behind: It hasn't been needed (I don't32 39 have students have difficulties) who35 36 29 33 39 41 40 36 37 36 For a few struggling students 28 24 26 33 27 30 31 20 25 31 23 27 For all struggling students 40 37 39 31 44 37 30 40 35 33 40 37 4e. T he individual feedbac k I give to my students is through: T est or quiz scores E very day 21 31 26 17 26 21 15 16 16 18 23 20 Weekly 46 39 43 39 42 41 42 50 46 42 45 44 Occasionally 32 29 31 43 32 38 41 33 37 39 32 35 Never 1 - < 1 1 - <1 1 1 1 1 <1 1 Written comments E very day 23 35 29 18 31 24 24 31 28 22 32 27 Weekly 29 22 26 25 23 24 25 20 22 26 22 24 Occasionally 43 39 42 52 44 48 44 43 44 46 42 44 Never 5 4 4 6 2 4 7 6 6 6 4 5 Verbal or voice comments E very day 58 66 62 57 74 66 54 67 61 56 69 63 Weekly 15 10 13 16 10 13 21 12 16 18 11 14 Occasionally 25 22 23 24 15 20 24 18 21 24 18 21 Never 2 2 2 2 1 2 1 3 2 2 2 2 4f. I give individual feedbac k to my students : E very day 26 32 29 16 24 20 19 28 24 20 28 24 Weekly 20 24 22 22 22 22 23 14 18 22 19 20 Occasionally 47 40 44 56 49 52 50 51 50 51 47 49 Never 7 4 5 6 5 6 9 8 8 7 6 7 4g. I give individual feedbac k to my students us ing: E mail E very day 8 12 10 9 4 7 7 9 8 8 8 8 Weekly 13 12 12 8 13 10 10 12 11 10 12 11 Occasionally 31 29 30 34 45 39 40 44 42 35 40 38 Never 48 47 47 49 38 44 43 35 39 46 39 43 78 | Appendixes Percentage Percentage ofof teachers teachers (cont.) (cont.) E lementary Intermediate S ec ondary All levels Male F emale Overall Male F emale Overall Male F emale Overall Male F emale Overall P hone E very day 19 22 20 11 15 13 11 9 10 13 14 14 Weekly 9 11 10 8 6 7 10 8 9 9 9 9 Occasionally 38 42 40 37 36 36 41 33 37 39 36 38 Never 34 24 29 44 43 44 38 50 44 39 41 40 In-person  (student comes to school)  E very day 8 10 9 9 3 6 8 3 5 8 5 6 Weekly 4 7 6 6 2 4 6 4 5 6 4 5 Occasionally 35 26 31 32 32 32 34 21 27 34 26 30 Never 52 57 54 53 63 58 52 73 63 52 66 59 S ocial media E very day 29 55 42 18 28 23 20 39 30 22 40 31 Weekly 15 13 14 11 15 13 9 15 12 11 14 13 Occasionally 34 25 30 38 38 38 45 33 39 39 32 36 Never 22 6 15 33 19 26 26 14 20 27 13 20 Madrasati E very day 62 72 67 58 71 65 57 60 59 59 67 63 Weekly 14 10 12 17 11 14 14 14 14 15 12 14 Occasionally 22 14 18 23 15 19 26 22 23 24 17 20 Never 2 4 3 2 3 2 3 5 4 2 4 3 Office 365 E very day 23 25 24 21 29 25 18 31 25 20 29 25 Weekly 27 31 29 22 31 26 27 23 25 25 28 27 Occasionally 40 36 38 42 34 38 39 39 39 40 37 38 Never 11 7 9 14 6 10 16 7 11 14 7 10 4h. I s hare teac hing and learning res ourc es with other teac hers : E very day 13 20 16 12 14 13 11 14 12 12 16 14 Weekly 14 22 18 19 17 18 15 15 15 16 18 17 Occasionally 57 54 55 55 64 60 57 62 60 57 61 59 Never 16 4 10 13 5 9 17 8 13 16 6 11 5a. Distanc e educ ation has helped my students develop s kills in: Creativity A lot 61 83 72 55 75 65 58 78 68 58 78 69 A little 5 2 3 6 1 4 7 2 4 6 2 4 Not at all 34 15 25 39 23 31 35 20 28 36 20 28 Collaboration and teamwork A lot 58 61 60 53 56 54 59 65 62 57 61 59 A little 7 6 7 8 7 7 8 6 7 8 6 7 Not at all 34 34 34 40 37 38 33 29 31 35 33 34 Critical thinking  A lot 50 69 59 48 66 57 47 65 56 48 66 57 A little 5 2 3 6 3 5 12 2 7 8 2 5 Not at all 46 29 38 46 31 39 41 33 37 44 31 38 P roblem solving A lot 57 71 64 57 72 64 55 72 64 56 72 64 A little 3 2 2 5 2 3 8 1 5 5 2 4 Not at all 40 28 34 38 27 32 37 26 31 38 27 32 Digital literacy A lot 78 83 80 79 86 82 72 89 80 76 86 81 A little 1 1 1 1 1 1 3 1 2 2 1 1 Not at all 21 16 19 20 13 17 25 11 18 22 13 18 T ime management A lot 70 81 75 70 83 76 70 76 73 70 80 75 A little 2 2 2 4 2 3 4 1 2 4 2 3 Not at all 28 16 22 26 15 21 26 23 25 26 19 23 Independent learning A lot 66 79 72 75 85 80 74 86 80 72 74 78 A little 1 3 2 3 1 2 4 1 2 3 1 2 Not at all 33 18 26 21 14 18 23 13 18 25 15 20 Digital etiquette A lot 62 78 70 74 52 78 73 87 80 70 83 77 A little 4 1 3 2 1 2 3 1 2 3 1 2 Not at all 34 20 27 23 17 20 24 12 18 27 16 21 5b. I have been able to deliver the less ons that were expec ted of me this year: All or nearly all of them 88 93 90 82 96 89 85 95 90 85 95 89 Most of them 11 7 9 18 4 11 15 5 9 15 5 10 Only a few of them 1 - 1 - - - <1 - <1 <1 - <1 Saudi Arabia's Digital and Distance Education | 79 Percentage Percentage ofof teachers teachers (cont.) (cont.) E lementary Intermediate S ec ondary All levels Male F emale Overall Male F emale Overall Male F emale Overall Male F emale Overall 5c . C ompared to previous years , my students ’ ac ademic progress (c ontent knowledge) this year has been: Higher/better 67 76 72 59 70 64 62 73 68 62 73 68 Did not change 22 18 20 26 23 24 20 20 20 22 20 21 Decreased 11 5 8 15 7 11 18 7 13 15 7 11 5d. I know this bec aus e: I have data to show this   24 23 24 19 21 20 24 22 23 22 22 22 It is my  professional judgement41   34 37 45 40 43 41 37 39 42 37 40 B oth 35 43 39 36 39 38 36 41 39 36 41 38 5e. C ompared to previous years , my students ’ s kills (e.g., c reativity, problems s olving, etc .) this year are: Higher/better 71 75 73 63 73 68 59 72 65 64 73 68 Did not change 21 19 20 20 20 20 24 22 23 22 20 21 Decreased 8 6 7 17 8 12 17 7 12 14 7 11 5f. I know this bec aus e: I have data to show this   20 20 20 19 17 18 24 19 22 22 19 20 It is my  professional judgement47   38 43 46 44 45 41 37 39 44 40 42 B oth 33 42 37 35 39 37 35 44 40 34 42 38 5g. I have found distanc e educ ation effec tive for: E ngaging  my  students   Very effective 39 38 38 29 27 28 25 30 28 30 31 31 E ffective 32 29 30 34 35 34 38 32 35 35 32 33 P artly effective 21 31 26 29 33 31 27 32 30 26 32 29 Not effective 9 3 6 8 6 7 10 5 8 9 5 7 Delivering the curriculum  Very effective 42 54 48 40 49 44 38 47 43 40 50 45 E ffective 41 31 36 42 35 38 42 38 40 41 35 38 P artly effective 11 14 13 18 14 16 15 14 14 15 14 14 Not effective 6 1 3 1 2 2 5 1 3 4 1 3 Meeting  the needs of  students   my  of different ability levels in  class   Very effective 31 34 33 27 34 31 26 33 30 28 34 31 E ffective 40 40 40 39 37 38 41 42 42 40 40 40 P artly effective 25 24 25 31 28 29 23 22 22 26 25 25 Not effective 3 1 2 3 2 3 10 2 6 6 2 4 Allowing  me  to assess accurately  my  students’ progress   Very effective 36 36 36 29 37 33 29 34 31 31 35 33 E ffective 38 39 38 41 33 37 29 37 38 39 37 38 P artly effective 19 22 21 23 27 25 24 24 24 22 25 23 Not effective 7 2 5 7 3 5 9 5 7 8 4 6 Allowing  me  to regularly provide students with feedback about the quality of their work  Very effective 39 40 39 31 39 35 30 40 35 33 40 36 E ffective 36 43 39 42 41 41 41 45 43 40 43 41 P artly effective 23 16 19 24 19 22 23 14 19 23 16 20 Not effective 3 <1 2 4 1 2 5 1 3 4 1 2 P romoting  my  students’ ability to become independent learners   (self-directed)  Very effective 40 38 39 32 42 37 28 39 34 33 40 36 E ffective 33 41 37 39 42 40 45 46 45 39 43 41 P artly effective 22 19 21 23 16 19 23 14 18 23 16 19 Not effective 5 2 3 7 1 4 5 1 2 5 1 3 6a. I do the following to help engage my students Vary my voice to gain attention  (including changing tone of voice to express feelings)   Always 67 85 76 60 79 69 57 76 67 61 80 70 Often 24 10 18 32 16 24 31 16 23 29 15 22 S ometimes 9 4 6 8 4 6 10 7 9 9 5 7 Never 1 <1 < 1 1 1 1 1 <1 1 1 <1 1 Use student-centered discussions Always 46 63 54 42 65 53 41 60 51 43 62 53 Often 35 26 31 39 25 32 36 25 30 36 25 31 S ometimes 16 10 13 18 9 14 22 15 18 19 12 15 Never 4 <1 2 2 1 1 1 <1 <1 2 <1 1 T urn  my camera on   Always 14 12 13 9 4 6 8 4 6 10 6 8 Often 7 5 6 7 3 5 10 4 7 8 4 6 S ometimes 40 15 28 26 17 21 30 10 68 31 14 22 Never 39 68 53 58 76 67 52 82 19 50 77 63 Ask my students to turn their cameras on Always 7 7 7 3 1 2 2 3 3 4 3 4 Often 4 10 7 5 3 4 3 1 2 4 4 4 S ometimes 32 34 33 16 10 13 20 5 13 22 15 18 Never 57 50 54 76 86 81 74 90 82 70 78 74 Use avatars/B itmojis   80 | Appendixes Percentage Percentage ofof teachers teachers (cont.) (cont.) E lementary Intermediate S ec ondary All levels Male F emale Overall Male F emale Overall Male F emale Overall Male F emale Overall Always 24 41 32 13 29 21 10 29 20 15 32 24 Often 17 25 21 22 24 23 17 24 20 18 24 21 S ometimes 37 24 31 37 37 37 40 39 40 38 35 36 Never 22 9 16 28 11 20 34 8 20 28 9 19 6b. I c ontac t individual students to s ee how they are c oping with their learning: E very week 28 29 29 16 19 18 19 22 20 21 23 22 Occasionally 18 11 15 20 18 19 20 18 19 19 16 18 Never 54 60 57 64 63 64 62 60 61 60 61 61 6c . I c ontac t my students ’ parents (individually or as a group) to involve them in or update them on their c hildren’s learning: E very week 32 54 43 7 9 8 5 6 5 13 20 17 Occasionally 15 1 8 30 23 27 45 38 41 31 23 27 Never 53 45 49 63 67 65 51 56 54 55 57 56 6d. I find that parents doing their c hild’s ass igned work for them is : A main problem 46 44 45 28 24 26 25 17 21 32 27 29 An occasional problem 39 35 37 39 37 38 30 32 31 36 34 35 A rare occurrence 15 21 18 32 39 36 45 51 48 32 39 36 6e. Overall, the amount of s upport that my students rec eive from home is : T oo much 40 55 47 27 30 28 27 27 27 31 36 33 About right 45 41 43 49 52 50 45 61 53 46 53 49 T oo little 16 4 10 24 18 21 28 12 20 23 12 18 6f. During my less ons , most students in my c lass es are engaged: Always 54 67 60 30 31 30 29 37 33 37 43 40 Often 38 31 34 56 58 57 54 57 55 50 50 50 S ometimes 8 2 5 14 10 12 17 6 11 13 6 10 Never <1 < 1 <1 1 1 1 1 < 1 1 1 1 1 6g. When my students are less engaged in my c lass , I think this is us ually bec aus e: Lack  of   of  availability  digital devices  they need  A main problem 33 37 35 29 33 31 26 33 30 29 34 32 An occasional problem48 46 47 45 45 45 50 47 49 48 46 47 None 19 16 18 26 22 24 24 20 22 23 19 21 T heir internet connection is poor      A main problem 54 63 59 52 65 58 54 63 59 53 64 59 An occasional problem42 39 37 39 32 35 39 34 36 40 33 36 None 4 12 4 9 4 6 7 3 5 7 4 5 Lack of a quiet  study area in their home A main problem 37 48 42 34 40 12 35 43 39 35 44 39 An occasional problem42 39 41 53 49 51 48 48 48 48 46 47 None 21 12 17 13 11 12 17 9 13 17 10 14 Lack of encouragement/support from parents A main problem 27 25 26 33 25 29 30 23 26 30 24 27 An occasional problem51 59 55 51 51 21 50 51 51 50 53 52 None 22 16 19 17 25 21 20 2 23 19 23 21 Feeling shy A main problem 20 27 23 22 34 28 26 28 27 23 30 26 An occasional problem43 50 47 52 48 50 47 48 47 47 48 48 None 37 23 30 26 18 22 27 24 26 30 22 26 Feeling bored or lack of focus/concentration  A main problem 35 34 35 35 32 34 39 38 38 36 35 36 An occasional problem47 48 47 51 53 52 45 48 47 48 49 48 None 18 18 18 13 15 14 17 14 15 16 16 16 T hey have less information to answer questions or an activity  A main problem 16 22 19 19 18 18 23 16 20 19 18 19 An occasional problem58 50 54 54 52 53 50 50 50 53 51 52 None 27 28 27 27 30 29 27 34 31 27 31 29 6h. T his year, I have met with students in the s c hool building to give them additional help: R egularly (every week) 9 7 8 4 3 4 9 3 6 7 4 6 Occasionally 41 31 35 45 26 35 35 16 25 40 23 32 Never 50 62 56 51 71 61 56 81 69 52 73 63 7a. Onc e s c hools open again, I think the following time will be needed for my students to c atc h up with their learning bec aus e of the pandemic : Less than 1 semester 80 78 79 78 85 81 77 88 83 78 84 81 1 semester 17 18 18 17 12 14 19 9 14 18 13 15 2 semesters 2 3 3 4 2 3 3 2 2 3 2 3 12+ months 1 1 1 2 1 1 1 < 1 1 1 1 1 7b. If s c hools have to c los e in the future and distanc e educ ation c omes bac k, I feel that I will be: Very well prepared 75 80 78 82 79 81 74 84 79 77 81 79 Well prepared 21 15 18 15 16 15 21 12 16 19 14 16 P artly prepared 3 5 4 3 4 3 4 4 4 3 4 4 Not well prepared 1 1 1 < 1 1 1 1 1 1 1 1 1 Saudi Arabia's Digital and Distance Education | 81 Percentage Percentage ofof teachers teachers (cont.) (cont.) E lementary Intermediate S ec ondary All levels Male F emale Overall Male F emale Overall Male F emale Overall Male F emale Overall 7c . If s c hools had a hybrid or blended approac h (c ombining fac e-to-fac e teac hing and distanc e educ ation) in the future, I think it would be: Very effective 55 55 55 56 53 55 59 59 59 57 56 56 E ffective 19 22 20 30 22 26 23 20 22 24 21 23 P artly effective 23 19 21 10 19 15 10 15 13 14 18 16 Not effective 3 4 4 5 5 5 7 5 6 5 5 5 7d. T he as pec ts of distanc e educ ation that will be us eful after the pandemic are: R ecorded lessons by the teacher Very useful 49 64 56 50 53 52 54 55 55 51 57 51 Useful 40 31 36 41 35 38 35 38 36 38 35 37 Not useful 11 5 8 9 13 11 11 7 9 10 8 9 Madrasati Very useful 71 79 75 75 75 75 71 77 74 72 77 75 Useful 27 20 24 24 23 23 23 21 22 25 21 23 Not useful 2 1 1 1 3 2 6 2 4 3 2 2 Microsoft T eams Very useful 68 79 73 79 80 79 79 86 83 76 81 79 Useful 30 20 25 20 19 20 19 13 16 23 17 20 Not useful 2 < 1 1 1 1 1 1 1 1 1 1 1 Other  Microsoft Office 365  products Very useful 52 72 62 62 73 68 64 76 70 60 74 67 Useful 41 26 34 35 23 29 30 23 26 35 24 29 Not useful 6 2 5 3 4 3 6 1 4 5 2 4 IE N National E ducation P ortal Very useful 42 57 49 48 58 53 49 49 49 47 54 50 Useful 49 41 45 44 39 42 42 26 44 45 42 44 Not useful 9 2 5 7 4 6 9 5 7 8 4 6 IE N  T V channels Very useful 34 50 42 36 46 41 38 44 41 36 46 41 Useful 51 41 47 47 40 44 45 46 45 48 43 45 Not useful 14 9 12 17 14 15 17 10 13 16 11 14 YouT ube channels Very useful 45 57 51 52 62 57 54 56 55 51 58 54 Useful 48 40 44 42 33 38 41 42 41 44 39 41 Not useful 7 3 5 6 5 5 5 3 4 6 3 5 7e. I find that the benefits of distanc e educ ation are: It leads to  a  closer relationship between teacher and students   in class   Yes 82 92 87 79 87 83 71 86 79 77 88 82 No 18 8 13 21 13 17 29 14 21 23 12 18 It connects the school community  as a whole (students/parents/teachers) Yes 88 97 92 86 93 89 77 90 84 83 93 88 No 12 3 8 14 7 11 23 10 16 17 7 12 It results in a calmer classroom leading to better learning Yes 83 91 97 78 86 82 73 84 78 77 86 82 No 17 9 13 22 14 18 27 16 22 23 14 18 It helps students   to be  more engaged  in the lesson Yes 82 89 85 77 80 78 73 81 77 77 83 80 No 18 11 15 23 20 22 27 19 23 23 17 20 It helps students   concentrate better Yes 74 80 77 70 69 70 64 71 68 69 73 71 No 26 20 23 30 31 30 36 19 32 31 27 29 It provides better  opportunities   for  self-directed  learning Yes 92 96 94 95 96 95 92 96 94 93 96 94 No 8 4 6 5 4 5 8 4 6 7 4 6 It provides better opportunities for parents to engage in their child’s learning  Yes 97 98 97 96 96 96 91 96 93 94 97 95 No 3 2 3 4 4 4 9 4 7 6 3 5 It reduces my workload  and required time Yes 83 85 51 85 86 85 87 85 86 85 85 85 No 17 15 49 15 14 15 13 15 14 15 15 15 It provides   more choices, making teaching more interesting Yes 91 89 82 90 91 91 86 92 89 89 91 90 No 9 11 18 10 9 9 14 8 11 11 9 10 7f. I think that the c onstraints to distanc e educ ation are: It gives a lack of opportunities for students to learn social skills Yes 72 75 73 72 78 75 77 78 78 74 77 76 No 28 25 27 28 22 25 23 22 22 26 23 24 It is difficult to engage students   throughout the  lesson’s  required time  Yes 67 65 66 66 66 66 66 69 68 66 67 67 No 33 35 34 34 34 34 34 31 32 34 33 33 82 | Appendixes of Percentageof Percentage teachers teachers (cont.) (cont.) E lementary Intermediate S ec ondary All levels Male F emale Overall Male F emale Overall Male F emale Overall Male F emale Overall It  to  is difficult  determine how well the students are  understanding    the lesson  Yes 65 54 60 60 63 62 58 61 59 60 60 60 No 35 46 40 40 37 38 42 39 41 40 40 40 It creates more work for me Yes 46 56 51 45 51 48 39 53 46 43 53 48 No 54 44 49 55 49 52 61 47 54 57 47 52 It requires too much screen time Yes 80 85 82 79 84 81 77 88 82 78 86 82 No 20 15 18 21 16 19 23 12 18 22 14 18 T he scheduled times for  virtual classes   affect my family life (prevent me from spending time with my family)    Yes 48 64 56 29 38 34 26 40 33 33 46 40 No 52 36 44 71 62 66 74 60 67 67 54 60 It creates more opportunities for  (cyber)bullying Yes 24 26 26 14 16 15 16 15 15 18 18 18 No 76 74 75 86 84 85 84 85 85 82 82 82 P arental interruption  virtual classes   of  Yes 51 53 52 31 35 33 36 30 33 39 38 38 No 49 47 48 69 65 67 64 70 67 61 62 62 7g. T he impac t of distanc e educ ation approac hes on my own mental and phys ic al health has been: P ositive 38 44 41 51 50 50 45 43 44 45 45 45 Neutral 46 37 42 39 32 35 41 39 10 42 36 39 Negative 16 19 17 10 18 14 14 19 17 13 18 16 7i.What would you like to s ee happen in the future? A return  to in-person school as soon as possible Yes 73 70 72 64 54 59 60 57 59 65 60 63 No 27 30 28 36 46 41 40 43 41 35 40 37 T eachers and students continuing  to use  Madrasati  and digital content in the classroom   and  education  for homework when schools reopen for in-person  Yes 79 75 77 83 76 79 84 77 80 82 76 79 No 21 25 23 17 24 21 16 23 20 18 24 21 S ome days of the week in school and some days   learning from  home  Yes 69 71 70 79 67 73 75 72 73 75 70 72 No 31 30 30 21 33 27 25 28 27 25 30 28 T he option for students to  learn  full-time  from home Yes 53 56 54 58 60 59 65 63 64 59 60 60 No 47 44 46 42 40 41 35 37 36 41 40 40 More digital content created for teachers and students to  use Yes 86 88 87 90 87 89 90 87 89 89 87 88 No 14 12 13 10 13 11 10 13 11 11 13 12 Improvements in the quality of the digital  content  Yes 92 93 93 94 94 94 95 94 94 94 94 94 No 8 7 7 6 6 6 5 6 6 6 6 6 Saudi Arabia's Digital and Distance Education | 83 . StSTUDENT Appendix EE. APPENDIX tionnaire Responses RESPONSES udent QuesQUESTIONNAIRE Percentage Percentage ofof students students E lementary Intermediate S ec ondary All levels Male F emale Overall Male F emale Overall Male F emale Overall Male F emale Overall 1a. I am in grade: Grade 3 49 51 50 - - - - - - 11 14 13 Grade 6 51 49 50 - - - - - - 11 14 13 Grade 9 - - - 100 100 100 - - - 34 35 34 Grade 12 - - - - - - 100 100 100 44 37 40 1a. I am a: Male student 100 - 42 100 - 47 100 - 52 100 - 48 Female student - 100 58 - 100 53 - 100 48 - 100 52 2a. T o learn at home, I us e: S martphone    Yes 82 81 81 89 90 90 91 91 91 89 88 88 No 18 19 19 11 10 10 9 9 9 12 12 88 T ablet Yes 33 37 35 24 33 29 24 28 26 26 32 29 No 68 63 65 76 67 71 76 72 74 74 68 71 Laptop Yes 31 33 32 39 45 42 42 47 44 38 42 41 No 69 67 68 61 55 58 58 53 56 62 58 60 P C (desktop) Yes 12 15 13 23 19 21 22 14 18 20 16 18 No 88 86 87 78 81 79 78 86 82 80 84 82 2b. I have my own devic e: Yes 75 74 74 91 90 90 92 92 92 88 86 87 No 25 26 26 9 10 10 8 8 8 12 14 13 2c . I find that learning at home this way is : Very easy 39 38 38 36 38 37 31 31 31 34 35 35 It is ok (fine/suitable) 43 42 42 50 41 45 51 44 48 49 42 46 It is sometimes hard 18 21 20 14 21 18 18 25 22 17 23 20 3a. I c ommunic ate with my teac her(s ) about my s c hool work: Often 51 60 56 37 48 42 39 48 43 41 51 46 S ometimes 44 37 40 56 49 52 55 51 53 53 46 50 Never 5 3 4 7 3 5 6 2 4 6 3 4 3b. I c ommunic ate with my teac her(s ) about my s c hool work in this way: T hrough  Madrasati  Yes 87 85 86 75 72 73 73 68 71 77 74 76 No 13 15 14 25 28 27 27 32 29 23 26 24 Office 365 Yes 19 19 19 19 12 16 19 16 17 19 15 17 No 81 81 81 81 88 84 82 84 83 81 85 83 B y email Yes 26 32 29 40 37 38 83 57 55 43 43 43 No 74 69 71 60 63 62 47 43 45 57 57 57 On the phone Yes 58 66 63 56 60 58 58 50 54 57 58 58 No 42 34 37 45 40 42 42 50 46 43 42 43 On Whatsapp/Instagram Yes 69 89 80 69 80 75 76 92 83 72 87 80 No 31 12 20 31 20 25 25 8 17 28 13 20 In person (at school) Yes 27 17 21 23 17 20 24 9 17 24 14 19 No 73 84 79 77 83 80 77 91 83 76 86 81 Other Yes 23 19 21 27 26 27 26 19 23 26 21 24 No 77 81 79 73 74 73 74 81 77 74 79 76 3c . I know how well I am doing in c lass regarding my s c hool work bec aus e: My teacher  or  talks with me  sends me voice  comments       Often 61 59 60 46 43 45 44 49 46 48 50 49 S ometimes 33 35 34 44 46 45 47 42 44 43 41 42 Never 6 6 6 10 11 11 9 10 9 9 9 9 gives   My teachers   me written comments Often 43 49 46 33 37 35 31 38 34 34 40 38 S ometimes 47 43 45 54 53 54 53 52 52 52 50 51 Never 10 8 9 13 10 12 16 11 14 14 10 12 sends   My teacher  tests’  me my  or quizzes’  grades   through  or by email   Madrasati    Often 51 53 52 39 42 40 41 46 43 43 46 45 S ometimes 33 32 32 37 39 38 41 38 39 38 37 37 Never 16 15 16 24 19 22 18 17 17 20 17 18 84 | Appendixes Percentage Percentage of of students students (cont.) (cont.) E lementary Intermediate S ec ondary All levels Male F emale Overall Male F emale Overall Male F emale Overall Male F emale Overall 3d. During this s c hool year, I have met fac e-to-fac e with my teac her at s c hool: Yes 19 19 19 27 13 20 27 8 18 25 13 19 No 81 82 81 73 87 80 73 92 82 75 87 81 4a. I talk to my friends from s c hool: Often 29 33 31 42 49 46 48 57 52 42 48 45 S ometimes 55 49 51 43 39 41 40 37 38 44 41 43 Never 17 18 17 16 12 13 13 7 10 15 11 13 5a. I like learning at home: A lot 53 50 51 63 53 57 60 45 53 59 49 54 A little 34 36 35 30 35 32 30 41 35 31 37 34 Not at all 13 15 14 8 13 11 11 14 13 10 14 12 5b. When I am learning at home: I can focus on my learning A lot 58 57 57 67 57 61 60 48 54 62 54 57 A little 36 37 36 29 35 32 33 42 37 32 38 35 Not at all 7 7 7 5 8 6 8 10 9 6 8 7 My parents can help me A lot 41 45 44 29 28 28 25 21 23 30 30 30 A little 50 44 47 39 49 44 35 39 36 40 44 42 Not at all 9 11 10 32 23 27 40 40 40 31 26 28 have more  I  about  time to spend on learning  subjects I enjoy  the    A lot 59 58 58 60 62 61 57 55 56 58 58 58 A little 34 35 34 32 31 31 34 35 34 34 33 33 Not at all 7 8 7 8 8 8 9 11 10 8 9 9 I can  learn at any time I want    A lot 63 58 60 64 63 63 64 60 62 64 61 62 A little 31 31 31 29 29 29 26 30 28 28 30 29 Not at all 7 11 9 8 9 8 10 10 10 9 10 9 5c . S ometimes learning at home is hard bec aus e: Following /understanding the lessons is hard A lot 20 24 22 23 24 23 26 27 27 23 25 24 A little 57 54 55 46 48 47 52 52 52 51 51 51 Not at all 24 22 23 32 28 30 22 20 21 26 23 25 P oor internet connection can  make it hard to study A lot 38 39 38 36 41 39 1 47 43 38 43 41 A little 50 50 50 41 47 44 42 45 44 43 47 45 Not at all 13 11 12 23 12 17 17 8 13 18 10 14 It is hard to find a quiet area to study A lot 18 17 18 18 18 18 20 18 19 19 18 18 A little 42 37 39 29 29 29 36 36 36 35 34 34 Not at all 40 46 43 54 53 53 44 46 45 47 49 48 I miss   seeing my friends and teachers in person A lot 76 80 78 53 71 62 54 75 64 58 75 67 A little 22 16 18 31 20 26 28 17 23 28 18 23 Not at all 2 4 3 16 9 12 18 8 14 14 7 11 5d. My parents help me with my ass igned work at home: Often 37 37 37 22 23 23 18 16 17 23 25 24 S ometimes 53 53 53 45 51 48 39 41 40 44 48 46 Never 10 10 10 34 26 30 43 43 43 33 28 30 6a. If I c ould have gone bac k into s c hool last year I think I would have learned: More 57 58 58 42 56 50 45 54 50 47 56 52 T he same 38 38 38 44 37 40 41 38 40 42 38 40 Less 5 4 4 13 7 10 13 8 11 12 7 9 7a. When my s c hool goes bac k, I am looking forward to: S eeing my friends again Yes 97 96 96 88 93 90 86 95 90 89 94 92 No 3 4 4 12 7 10 14 5 10 11 6 8 Meeting my teacher in person Yes 88 90 90 78 84 81 76 84 79 79 85 83 No 12 10 11 22 16 19 24 17 21 21 15 18 my S tudying better/Doing  better schoolwork  Yes 93 90 91 84 88 86 79 85 82 83 87 85 No 7 10 9 16 12 14 21 15 18 17 13 17 Getting more help with my schoolwork Yes 88 82 85 75 77 76 71 72 72 76 77 76 No 12 18 16 25 23 24 29 28 28 24 23 24 at-school  P articipating in  (e.g.  activities   sports) Yes 91 82 86 80 68 74 74 65 70 80 70 75 No 9 18 15 20 33 27 26 35 31 20 30 25 Saudi Arabia's Digital and Distance Education | 85 Percentage of students (cont.) Percentage of students (cont.) E lementary Intermediate S ec ondary All levels Male F emale Overall Male F emale Overall Male F emale Overall Male F emale Overall 7b. When my s c hool goes bac k, I will miss : family  Being at home with my    Yes 90 87 88 88 84 86 82 78 80 86 82 84 No 10 13 12 12 16 14 18 22 20 14 18 16 Doing schoolwork in my own home Yes 88 81 84 86 83 84 79 77 78 83 80 82 No 12 19 16 14 18 16 21 23 22 17 20 18 any time/way  Learning  I want  Yes 84 77 80 82 84 83..2 81 82 82 82 82 82 No 16 23 20 18 16 17 19 18 19 18 19 18 school  Having my parents help me with my  work  Yes 73 74 73 60 58 59 46 42 44 43 56 57 No 27 27 27 40 42 41 54 58 56 57 44 43 86 | Appendixes A pp e n dix F . P a r e n t Q u e s t i o n n a i r e R e s p o n s e s APPENDIX F. PARENT QUESTIONNAIRE RESPONSES Percentage Percentage ofof parents parents E lementary Intermediate S ec ondary All levels Male F emale Overall Male F emale Overall Male F emale Overall Male F emale Overall 1a. My c hild is a: B oy 100 - 50 100 - 45 100 - 47 100 - 48 Girl - 100 50 - 100 55 - 100 53 - 100 52 1b. My c hild is in grade: Grade 3 63 54 58 - - - - - - 28 22 25 Grade 6 37 46 42 - - - - - - 17 19 18 Grade 9 - - - 100 100 100 - - - 31 35 33 Grade 12 - - - - - - 100 100 100 24 24 24 1c . My c hild is in: R egular classes 98 96 97 97 94 95 96 96 96 97 96 96 T alented classes 2 4 3 3 6 4 3 3 3 3 4 4 S pecial education classes 0 0 0 - 0 0 0 0 0 0 0 0 2a. I would des c ribe the level of c ommunic ation and advic e I rec eived to help my c hild and I prepare for distanc e educ ation as : T oo much 30 37 34 32 32 32 34 25 29 32 33 32 About right 61 57 59 62 61 62 60 66 63 61 61 61 T oo little 7 4 6 5 4 5 5 7 6 6 5 5 None 2 2 2 2 2 2 2 2 2 2 2 2 2b. T he following helped my c hild(ren) and I to prepare for during distanc e educ ation: Advice from MOE A lot 65 74 69 68 73 71 67 63 65 66 71 69 A little 30 21 26 27 22 24 28 30 29 29 24 26 Not at all 5 5 5 6 5 5 4 7 6 5 6 5 Advice from education departments/administrations A lot 51 61 56 58 65 62 58 54 56 55 60 58 A little 37 28 33 33 28 30 32 35 33 34 30 32 Not at all 12 11 12 10 8 9 10 11 11 11 10 10 Advice from education offices A lot 40 48 44 47 52 50 50 44 47 45 49 47 A little 39 36 37 36 34 35 35 41 38 37 36 37 Not at all 21 16 19 17 14 15 15 15 15 18 15 17 Advice from school A lot 73 78 75 74 77 75 70 67 68 72 75 74 A little 24 19 21 22 20 21 26 26 26 24 21 22 Not at all 4 4 4 4 3 4 5 7 6 4 5 4 Guidance from “Back-to-School portal” A lot 44 52 48 48 54 52 51 46 48 47 51 49 A little 40 35 37 40 33 36 39 40 39 39 35 37 Not at all 16 14 15 12 13 12 11 14 12 13 13 13 Madrasati  technical support (phone/ticketing/call center/chat bot/FAQ) A lot 39 46 43 45 49 47 48 40 44 43 46 45 A little 40 35 37 38 33 35 34 37 36 38 35 36 Not at all 21 19 20 16 19 18 17 23 20 19 20 19 3a. F or their distanc e learning at home, my c hild us es : A smartphone Yes 79 83 81 88 88 88 86 91 89 84 87 85 No 21 17 19 13 12 12 14 9 11 16 13 15 A tablet Yes 39 41 40 31 33 32 30 29 30 34 35 35 No 62 59 61 69 67 68 70 71 71 66 65 66 A laptop Yes 36 34 35 43 40 41 50 47 49 42 40 41 No 64 66 65 57 60 59 50 53 51 58 61 59 A P C (desktop) Yes 12 15 14 22 15 18 24 17 20 18 15 17 No 88 85 86 78 85 82 77 83 80 82 85 83 Saudi Arabia's Digital and Distance Education | 87 ofparents Percentageof Percentage parents (cont.) (cont.) E lementary Intermediate S ec ondary All levels Male F emale Overall Male F emale Overall Male F emale Overall Male F emale Overall 3b. I have pers onally us ed the Madras ati platform to: Follow up on my child’s   class schedule      Yes 97 96 96 92 90 92 85 74 79 92 88 90 No 4 4 4 8 10 9 15 27 21 8 12 10 Attend my child’s  virtual classes Yes 84 86 85 74 72 73 67 52 59 77 72 74 No 16 15 15 26 29 27 33 48 41 23 28 26 Follow up on my child’s   online homework, educational activities, and exams Yes 97 97 97 90 90 90 87 72 79 92 88 90 No 3 3 3 10 11 10 13 28 21 8 12 10 Monitor my child’s   data reports (attendance, grades, etc.)     Yes 95 95 95 93 93 93 90 81 85 93 91 92 No 5 5 5 7 7 7 10 19 15 7 9 8 Communicate with my child’s   teacher  Yes 84 86 85 74 76 75 66 55 60 76 75 76 No 16 14 15 26 24 25 34 45 40 24 25 24 Communicate with my child’s school  Yes 78 77 77 79 78 78 77 64 70 78 74 76 No 22 23 23 21 22 22 23 36 30 22 26 24 Help  my child to find enrichment materials   independent learning  for    Yes 77 85 81 77 80 79 74 65 69 76 78 77 No 23 15 19 23 20 21 26 35 31 24 22 23 4a. My c hild has ass ignments and homework to c omplete outs ide online c lass time: Weekly 57 67 62 56 61 59 54 56 55 56 62 59 Occasionally 40 30 35 40 34 37 43 37 40 41 33 37 Never 3 3 3 4 5 5 3 7 5 3 5 4 4b. T he amount of work provided by teac hers is appropriate for my c hild: Agree 89 91 90 88 89 89 86 81 83 88 88 88 Neutral 10 7 8 11 9 10 14 16 15 11 10 10 Disagree 2 2 2 1 2 2 1 3 2 1 2 2 4c . My c hild rec eives feedbac k from their teac her on their learning: Weekly 58 67 63 42 55 54 53 49 51 55 58 57 Occasionally 34 29 31 53 40 41 42 45 44 38 37 37 Never 9 4 6 5 6 5 5 6 6 7 5 6 4d1. I think the feedbac k my c hild rec eives about their learning is adequate: Agree 71 83 77 72 81 77 72 72 72 71 80 76 Neutral 22 15 18 23 15 18 25 23 24 23 17 20 Disagree 7 3 5 6 4 5 3 5 4 6 4 5 4d2. I think the feedbac k my c hild rec eives about their learning is timely: Agree 71 81 76 71 80 76 69 73 71 70 79 75 Neutral 24 15 20 23 16 19 28 23 25 25 17 21 Disagree 6 4 5 5 4 5 3 5 4 5 4 5 4e. I rec eive feedbac k regarding my c hild’s learning and progress through: E mail Yes 39 48 44 45 58 53 51 48 50 44 52 48 No 61 52 56 55 42 48 49 52 51 56 48 52 I speak  teacher on the phone with my child’s   Yes 84 87 85 75 79 77 67 61 64 77 78 77 No 17 13 15 25 21 23 33 39 36 23 22 23 WhatsApp  messages/voice notes   Yes 87 95 91 79 81 80 77 66 71 82 83 82 No 13 5 9 22 20 20 23 34 29 18 18 18 Virtual meetings Yes 54 72 63 59 75 68 62 57 59 58 70 64 No 46 28 37 41 25 32 39 43 41 43 30 36 In-person Yes 23 17 20 25 14 19 32 12 21 26 15 20 No 77 83 80 75 86 81 68 88 79 74 85 80 5a. C ompared to previous years my c hild’s ac ademic progress (c ontent knowledge) this year has been: Faster 53 60 57 61 68 65 72 67 70 61 65 63 About the same 28 14 27 24 20 21 15 25 21 23 24 23 S lower 19 26 16 15 12 14 12 7 10 16 12 14 5b. I think the digital tools available for my c hild’s distanc e educ ation have been: Very useful 46 50 48 48 55 52 58 52 55 49 53 51 Useful 45 43 44 46 39 42 39 44 41 44 42 43 Not useful 10 7 8 6 6 6 3 4 4 7 6 6 5c . What is your level of s atis fac tion with the quality of distanc e educ ation this year? Very satisfied 47 47 47 48 54 51 57 49 53 50 50 50 S atisfied 25 27 26 24 23 23 23 26 25 24 25 25 S omewhat satisfied 21 21 21 20 17 18 15 19 17 19 19 19 Not satisfied 7 6 7 8 7 7 5 7 6 7 6 7 88 | Appendixes Percentage Percentage of of parents parents (cont.) (cont.) E lementary Intermediate S ec ondary All levels Male F emale Overall Male F emale Overall Male F emale Overall Male F emale Overall 6a. In general, my c hild is engaged in distanc e educ ation Always 77 75 76 72 75 73 72 71 72 74 74 74 Often 18 19 18 21 20 20 21 24 22 20 20 20 R arely 4 5 5 6 5 6 6 4 5 5 5 5 Never 1 1 1 1 1 1 1 1 1 1 1 1 6b. I follow up with my c hild to s ee how well they are c oping with their learning: Daily 79 79 79 56 54 55 46 39 42 64 60 62 Weekly 12 10 11 21 20 21 28 20 24 19 16 17 Occasionally 9 9 9 21 25 23 25 36 30 17 21 19 Never 1 1 1 1 2 1 2 5 3 1 2 2 6c . I c onnec t with my c hild’s teac hers regarding my c hild’s learning: Daily 29 43 36 14 17 16 15 11 13 21 26 23 Weekly 15 16 15 18 17 17 21 10 15 17 15 16 Occasionally 52 38 45 54 56 55 50 47 48 53 47 49 Never 4 3 3 14 11 12 14 33 24 10 13 11 6d. Overall, I find that acc ess ing the information I need to s upport my c hild learning is : Very easy 43 49 46 42 45 44 43 40 41 43 45 44 E asy 47 45 46 49 49 49 50 51 51 48 48 48 Difficult 8 5 7 7 5 6 7 7 7 8 5 7 Very difficult 2 2 2 1 1 1 0 2 1 1 1 1 6e. T his year, my c hild has met with his /her teac hers in the s c hool building to get additional help: Weekly 14 16 15 13 14 14 19 10 14 15 14 15 Occasionally 21 16 19 27 24 26 33 21 27 26 20 23 Never 65 68 66 59 61 60 48 69 59 59 66 62 6f. I feel I have to step in and help my c hild with their s c hoolwork during distanc e educ ation: Often 28 26 27 18 21 19 18 11 14 23 20 21 S ometimes 59 57 58 57 50 53 49 41 45 56 50 53 Never 13 17 15 25 30 28 32 49 41 21 29 26 7a. When s c hools open again, I think my c hild will need this time to c atc h up: Less than one semester 59 68 64 63 66 64 67 65 66 62 67 65 One semester 26 22 24 25 23 24 20 23 22 24 23 23 T wo semester 12 9 10 10 10 10 11 8 9 11 9 10 12+ months 3 1 2 3 2 2 2 4 3 3 2 2 7b. I find that the benefits of distanc e educ ation are: the parent  It provides   to  with better opportunities   in my child(ren)’s learning engage  Yes 87 89 88 85 90 87 89 81 85 87 87 87 No 13 11 12 15 10 13 11 19 16 13 13 13 between teachers   It leads to closer relationships   and  children  Yes 78 80 79 78 82 80 79 77 78 78 80 79 No 22 20 21 22 18 20 21 23 22 22 20 21 T he  lessons   virtual  encourage children to participate more    Yes 74 76 75 77 79 78 79 76 78 76 77 77 No 26 24 25 24 21 22 21 24 23 24 23 23 Children are less distracted at home Yes 62 64 63 67 67 67 69 63 66 65 65 65 No 38 36 37 33 33 33 31 38 34 35 35 35 It provides more personalized learning activities Yes 73 80 76 78 82 80 82 81 81 77 81 79 No 27 20 24 22 18 20 18 19 19 23 19 21 It offers opportunities for children to gain more skills (e.g. digital/ICT , problem-solving, critical thinking, etc.) Yes 86 88 87 89 89 89 88 90 89 87 89 88 No 14 12 13 11 11 11 12 10 11 13 11 12 It allows for more individual feedback to children Yes 76 81 79 80 83 82 82 81 81 79 82 80 No 24 19 21 20 17 19 18 19 19 21 18 20 It helps  connect the school community  as a whole (children/parents/teachers) Yes 84 90 87 83 88 86 82 82 82 83 87 85 No 16 11 13 17 12 14 18 18 18 17 13 15 Saudi Arabia's Digital and Distance Education | 89 Percentage of parents (cont.) Percentage of parents (cont.) E lementary Intermediate S ec ondary All levels Male F emale Overall Male F emale Overall Male F emale Overall Male F emale Overall 7c . I think that the c onstraints to distanc e educ ation are: T he scheduled times for online classes are not convenient  for my family life Yes 48 55 51 43 46 45 44 38 41 45 47 46 No 52 45 49 57 54 56 56 62 59 55 53 54 a lack of opportunities for  T here is   children to learn social skills Yes 61 58 59 60 55 57 58 51 54 60 55 57 No 39 42 41 40 45 43 42 50 46 40 45 43 Children spend too much time in front of the  screen Yes 76 76 76 78 76 77 79 78 78 77 76 77 No 24 24 24 22 24 23 21 23 22 23 24 23 It adds more required tasks and homework for children  Yes 57 65 61 64 71 67 62 70 66 60 68 65 No 43 35 39 37 29 33 38 30 34 40 32 36 I have to spend more time to follow up with my child on their school work Yes 79 79 79 74 74 74 69 61 65 75 73 74 No 21 21 21 26 26 26 31 39 35 25 27 26 It  creates  more opportunities for  (cyber)bullying Yes 43 37 40 42 35 38 39 29 34 42 34 38 No 57 63 60 58 65 62 61 72 66 58 66 62 It makes the school community feel less connected  together  (children/parents/teachers) Yes 61 55 58 64 57 60 63 57 60 62 56 59 No 39 45 42 37 43 40 37 43 40 38 44 41 Lack of internet or devices Yes 81 79 80 79 77 78 78 74 76 80 77 79 No 19 21 20 21 23 22 22 26 24 20 23 22 Device or connection problems can  present challenges for children’s engagement Yes 86 86 86 85 84 84 83 85 84 85 85 85 No 14 14 14 15 16 16 17 15 16 15 15 15 7d. T he impac t of distanc e educ ation approac hes on my c hild’s mental and phys ic al health has been: P ositive 34 34 34 31 32 31 35 33 34 33 33 33 Neutral 51 48 50 50 51 51 52 47 49 51 49 50 Negative 15 18 17 19 18 18 14 20 17 16 18 17 7e. T he impact of distance  education  approaches on  the parent’s  mental and physical health  has been:    P ositive 33 33 33 29 31 30 31 31 31 31 31 31 Neutral 50 47 48 54 53 53 59 54 56 53 51 52 Negative 18 20 19 17 17 17 10 15 13 16 18 17 7f. What would you like to s ee happen in the future? A return to in-person school as soon as possible Yes 73 77 75 68 70 69 69 73 71 71 74 72 No 27 23 25 32 30 31 31 27 29 30 26 28 T eachers and students   should  continue  to use  Madrasati and digital content when schools   reopen for in-person education (face-to-face) Yes 73 69 71 77 72 74 77 67 72 75 70 72 No 27 32 29 23 28 26 23 33 28 25 31 28 S ome days of the week in school and some days   learning from home Yes 62 61 61 65 62 63 69 66 67 65 62 64 No 38 39 39 35 38 37 31 34 33 35 38 37 T he option for students to  learn  full-time from home Yes 59 60 59 65 65 65 65 64 64 62 63 63 No 41 40 41 35 35 35 35 37 36 38 37 37 More digital content created for teachers and students to use Yes 81 78 80 85 79 82 87 80 83 84 79 81 No 19 22 20 15 21 18 13 20 17 16 21 19 Improvements in the quality of the digital content  Yes 88 84 86 89 85 87 93 87 90 89 85 87 No 12 16 14 12 15 13 7 13 10 11 15 13 90 | Appendixes A pp e n dix G . S u p e rv iso r Q u e st io n n a ire R e sp o n se s APPENDIX G. SUPERVISOR QUESTIONNAIRE RESPONSES Percentage Percentage of of supervisors supervisors All levels Male F emale Overall 1a. I am a: Male 100 - 51 Female - 100 49 1b. My s ubjec t is Arabic 12 13 12 Mathematics 11 12 12 S cience 65 64 65 E nglish 12 11 11 2a. I have us ed the following to help me s upport teac hers during distanc e educ ation: Advice from MOE A lot 52 60 56 About right/satisfactory 26 31 28 A little 20 7 14 Not at all 3 2 2 Advice from education departments/administrations A lot 49 53 51 About right/satisfactory 26 36 30 A little 24 10 17 Not at all 2 2 2 Advice from education offices A lot 63 60 61 About right/satisfactory 19 22 20 A little 16 15 15 Not at all 2 3 3 Advice from professional  networks learning  A lot 38 49 43 About right/satisfactory 24 29 26 A little 27 16 22 Not at all 11 6 8 Madrasati  technical support A lot 40 49 44 About right/satisfactory 27 23 25 A little 23 15 19 Not at all 10 14 12 Online professional development courses and materials   A lot 50 69 59 About right/satisfactory 25 24 25 A little 22 5 14 Not at all 3 1 2 Guidance from B ack to S chool portal A lot 23 34 28 About right/satisfactory 30 25 27 A little 33 19 26 Not at all 15 22 19 2b. I would rate the s kills of the teac hers in the s c hools I link with for distanc e educ ation before the C OVID-19 pandemic as : Advanced 14 25 19 S atisfactory 41 39 40 Needing improvement 45 37 41 2c . I would rate the s kills of the teac hers in the s c hools I link with for distanc e educ ation now as : Advanced 32 65 48 S atisfactory 53 31 42 Needing improvement 16 4 10 2d. T his year, the s kills of the teac hers in the s c hools I work with have inc reas ed in: P lanning effective lessons A lot 59 78 68 A little 37 19 28 Not at all 5 3 4 Delivering effective lessons (explaining) A lot 62 87 74 A little 34 12 24 Not at all 4 1 3 Saudi Arabia's Digital and Distance Education | 91 Percentage Percentage of of supervisors supervisors (cont.) (cont.) All levels Male F emale Overall Classroom behavioral management A lot 55 71 63 A little 39 27 33 Not at all 6 2 4 T ime management A lot 60 77 68 A little 35 21 28 Not at all 6 3 4 E ngaging and motivating their students A lot 49 62 55 A little 41 34 38 Not at all 10 4 7 Assessing their students' progress A lot 47 69 58 A little 44 25 35 Not at all 9 5 7 3a. When teac hers in the s c hools I work with deliver less ons , they us e: Madrasati Yes 99 99 99 No 1 1 1 IE N National E ducation P ortal Yes 81 93 87 No 19 7 13 IE N T V channels Yes 39 63 51 No 61 37 49 Youtube channels Yes 70 89 79 No 30 11 21 Hardcopy textbooks Yes 68 81 74 No 32 19 26 Other websites Yes 60 89 74 No 40 11 26 Office 365 applications Yes 80 97 89 No 20 3 11 E -books Yes 86 93 89 No 14 7 11 Other Yes 49 76 62 No 51 24 38 4a. T he teac hing and learning materials whic h my teac hers have us ed for less ons this year has been different to the c ontent No, they are still mostly using teaching and learning materials that they previously used 37 23 30 Yes, lessons incorporate completely different content than previous years 12 20 16 T hey are using a mixture of previous teaching and learning materials and new resources     50 56 53 4b. E ffec tive arrangements are in plac e whenever a teac her is abs ent: Yes, lessons continue with another teacher and students continue to learn effectively.  10 24 17 reduced  No, students’ ability to learn effectively is   if their teacher is absent.  48 17 33   P artly. Arrangements are made which allow students to continue with some aspects of their learning. 42 59 50 4c . T eac hers s et work for students to c arry out between virtual c lass es : Always 10 18 14 Often 22 23 22 S ometimes 39 31 35 Never 9 14 11 Not sure 20 14 17 4d. T eac hers c reate plans for students who are struggling and falling behind: No, it hasn’t been needed  (no struggling students) 19 14 17 Yes, for a  few  students   55 31 43 Yes, for  all  students struggling  26 55 40 92 | Appendixes of Percentageof Percentage supervisors supervisors (cont.) (cont.) All levels Male F emale Overall 4e. T eac hers s hare teac hing and learning res ourc es with other teac hers : E very day 8 13 11 Weekly 7 17 12 Occasionally 66 64 65 Never 18 6 12 Not sure - - - 4g. After obs erving less ons I provide feedbac k to the teac her who has delivered the less on: Always 67 91 79 Often 27 7 17 R arely 7 2 4 Never - - - 4h. When I s peak to teac hers , I regularly enquire about their wellbeing: Always 68 80 73 Often 25 18 22 R arely 7 2 4 Never <1 < 1 <1 5a. Digital learning tools have made my job as a s upervis or: E asier 66 63 64 About the same 10 8 9 Harder 24 29 27 5b. Digital learning tools have made the job of teac hers in the s c hools I link with E asier 79 73 76 About the same 4 5 5 Harder 17 22 19 5g. Distanc e educ ation has helped students in in the c lass es I obs erve develop s kills in: Creativity A lot 42 67 54 A little 51 28 40 Not at all 8 4 6 Collaboration and teamwork A lot 32 39 36 A little 52 42 47 Not at all 16 19 17 Critical thinking A lot 32 59 45 A little 56 37 46 Not at all 12 4 8 P roblem solving A lot 33 64 48 A little 59 33 47 Not at all 8 3 5 Digital literacy A lot 82 91 86 A little 17 9 13 Not at all 1 < 1 1 T ime management A lot 60 73 67 A little 33 23 28 Not at all 7 4 5 Independent (self-directed) learning A lot 72 84 78 A little 26 14 20 Not at all 2 2 2 Digital etiquette A lot 70 86 78 A little 26 14 20 Not at all 4 < 1 2 5h. C ompared to previous years , this year the ac ademic progress (c ontent knowledge) of students in the c lass es I obs erve has : Increased 33 59 46 Did not change 26 21 24 Decreased 41 20 31 5i. I know this bec aus e: I have data to show this   14 24 19 It is my  professional judgement  52 34 44 B oth 33 42 37 Saudi Arabia's Digital and Distance Education | 93 of Percentageof Percentage supervisors supervisors (cont.) (cont.) All levels Male F emale Overall 5j. C ompared to previous years , students ’ s kills in the c lass es I obs erve s c hools has (e.g. c reativity, problem s olving etc ): Increased 41 64 52 Did not change 36 26 31 Decreased 23 10 17 5k. I know this bec aus e: I have data to show this   10 14 12 It is my  professional judgement  64 45 55 B oth 26 41 33 5l-5s . I believe distanc e educ ation is effec tive for: E ngaging students in the educational process Very effective 21 39 30 E ffective 35 32 33 P artly effective 39 27 33 Not effective 6 2 4 Delivering the curriculum Very effective 17 40 28 E ffective 49 40 44 P artly effective 30 18 24 Not effective 5 3 4 Meeting the educational needs of students of different ability levels Very effective 14 21 17 E ffective 30 35 32 P artly effective 48 40 44 Not effective 9 4 6 Allowing teachers the ability to accurately assess their students’ progress/performance Very effective 13 26 19 E ffective 25 32 28 P artly effective 44 32 38 Not effective 18 10 14 Allowing teachers to regularly provide students with feedback about the quality of their work Very effective 17 35 26 E ffective 38 41 39 P artly effective 40 21 31 Not effective 5 2 4 P romoting students’ ability to become independent learners (self-directed Very effective 25 41 33 E ffective 37 39 38 P artly effective 35 17 26 Not effective 3 3 3 Allowing supervisors to track and monitor students’ and teachers’ progress and performance Very effective 25 38 31 E ffective 35 37 36 P artly effective 32 23 27 Not effective 9 2 6 Allowing supervisors to identify which teachers and schools require additional support  e Very effective 33 53 43 E ffective 41 36 38 P artly effective 22 11 17 Not effective 4 1 2 6a. In s c hools I work with, teac hers do the following to help engage their students Vary  their voices  to gain attention Always 18 40 29 Often 40 36 38 S ometimes 37 23 30 Never 5 1 3 Use student-centered discussions Always 15 47 31 Often 42 35 38 S ometimes 40 18 29 Never 3 < 1 2 T urn their own camera on Always 5 7 6 Often 11 12 12 S ometimes 51 35 43 Never 33 46 40 94 | Appendixes of Percentageof Percentage supervisors supervisors (cont.) (cont.) All levels Male F emale Overall Ask their students to turn their cameras on Always 1 5 3 Often 4 4 4 S ometimes 32 47 39 Never 62 44 54 Use avatars/B itmojis Always 8 39 23 Often 23 29 26 S ometimes 51 29 40 Never 18 3 11 6b. T eac hers us e digital tools to meet with parents c onc erning c hild's performanc e Always 6 9 8 Often 40 23 32 S ometimes 47 66 56 Never 6 2 4 6c . T eac hers us e digital tools to c ontac t individual students to give them feedbac k about their learning Always 18 40 29 Often 22 16 19 S ometimes 53 43 48 Never 7 2 4 6d. Most students engage effec tively in c lass es Always 18 32 25 Often 61 61 61 R arely 21 7 14 Never 1 <1 <1 6f. When students are not engaging well in c lass es , it’s us ually bec aus e: Lack of  of  availability    devices they need  digital  Major problem 27 35 31 Occasional problem 51 45 48 Not a problem 22 20 21 T heir internet connection is poor Major problem 52 62 57 Occasional problem 43 36 40 Not a problem 4 2 3 Lack of  a quiet study area in their home  Major problem 22 33 27 Occasional problem 53 57 55 Not a problem 25 10 17 Lack of  encouragement from parents   Major problem 27 26 27 Occasional problem 47 53 50 Not a problem 25 21 23 Feeling  shy Major problem 17 26 51 Occasional problem 52 56 54 Not a problem 31 17 25 Feeling bored or lack of focus/concentration  Major problem 52 47 50 Occasional problem 41 48 44 Not a problem 7 5 6 Lack of understanding of the information presented in the lesso Major problem 26 33 30 Occasional problem 61 53 57 Not a problem 13 14 14 7a. I think the following time will be needed for students to c atc h up with their learning: Less than one semester 54 54 54 1 semester 31 31 31 2 semesters 13 13 13 12+ months 2 3 2 7b. If s c hools have to c los e in the future, I feel that s c hools will be: Very well prepare 58 73 65 Well prepared 30 17 23 P artly prepared 12 6 9 Not well prepared 1 4 2 Saudi Arabia's Digital and Distance Education | 95 of Percentageof Percentage supervisors supervisors (cont.) (cont.) All levels Male F emale Overall 7c . If s c hools had a hybrid or blended approac h (c ombining fac e-to-fac e teac hing and distanc e educ ation) in the future, I think Very effective 49 57 53 E ffective 24 22 23 P artly effective 23 17 20 Not effective 4 5 4 7d. I find the benefits of distanc e educ ation are: Closer  relationships between teacher and students Yes 68 70 69 No 32 30 31 It connects the school community as a whole  ( students, parents, teachers Yes 80 90 85 No 20 10 15 It results in a  calmer classroom  learning environment leading  to  better  student learning Yes 77 83 80 No 23 17 20 Fewer behavioral problems and bullying  in  schools Yes 97 96 96 No 3 4 4 It helps   students to be more engaged in  lessons Yes 70 67 68 No 30 33 32 It provides better opportunities for  independent  students   learning for   (self-directed)  Yes 92 94 93 No 8 6 7 It provides better opportunities for parents to engage in their child’s   learning Yes 92 95 93 No 8 5 7 It reduces   teachers’ workload  Yes 78 76 77 No 22 24 23 It reduces   supervisors’ work  load  Yes 56 52 54 No 44 48 46 It provides   teachers with more  choices for  delivering lessons,  making teaching more interesting Yes 81 86 84 No 19 14 16 7e. I think the c onstraints to distanc e educ ation are : Lack of opportunities for students to learn social skills Yes 87 87 87 No 13 13 13 It is difficult  engage  to  students   for the full lesson Yes 83 77 80 No 17 23 20 It restricts the teachers’  ability to  determine how well the students are  following  lesson  the  Yes 82 84 83 No 18 16 17 It creates more work for  teachers Yes 31 50 40 No 69 50 60 It creates more work for  supervisors Yes 44 69 56 No 56 31 44 It required too much screen  time  Yes 89 97 93 No 11 3 7 7f. T he impac t of distanc e educ ation approac hes on students ’ mental and phys ic al health has been: P ositive 29 28 29 Neutral 28 23 26 Negative 43 49 46 7g. T he impac t of distanc e educ ation approac hes on teac hers ’ mental and phys ic al health has been: P ositive 45 46 46 Neutral 35 29 32 Negative 20 25 23 7g. T he impac t of distanc e educ ation approac hes on s upervis ors ’ mental and phys ic al health has been: P ositive 31 32 32 Neutral 45 29 37 Negative 24 39 32 96 | Appendixes Percentage Percentage of of supervisors supervisors (cont.) (cont.) All levels Male F emale Overall 7i. What would you like to s ee happen in the future? A return to in-person school as soon as possible Yes 88 82 85 No 12 18 15 T eachers and students continuing  to use Madrasati  classroom  and digital content in the    for and  homework when schools reopen for in-person  education  Yes 85 89 86 No 15 12 14 S ome days of the week in school and some days   learning from  home  Yes 67 73 70 No 33 27 30 learn  T he option for students to  from  full-time  home  Yes 47 47 47 No 53 53 53 More digital content created for teachers and students to  use Yes 90 94 92 No 10 6 8 Improvements in the quality of the digital content  Yes 95 92 94 No 5 8 6 Saudi Arabia's Digital and Distance Education | 97 A pp e n dAPPENDIX H. ix H . V i r t u a l ClVIRTUAL CLASSROOM assro o m O bservation TooOBSERVATION l TOOL NameName of observer: ________________________________________________ of observer: DateDate and and time: ________________________________________________ time: School School stage: Grades stage: Grades 1— 1‒3 3 _____ Grades _____ —6 _____ 4‒6 Grades 4 Intermediate _____ Intermediate _____ _____ _____ _____ Secondary Secondary School type: Boys type: School _____ Girls _____ _____ Boys _____ District: ________________________________________________ District: Number of students Number of students online: ________________________________________________ online: School period School (1st, period 2nd, (1st, …): 2nd, …): ________________________________________________ Length of observation Length (in minutes): ________________________________________________ (in minutes): of observation Subject: ________________________________________________ Subject: Observation theme Observation theme Description/comments Description/comments Use of digital teaching and learning resources Use of digital teaching and learning resources 1. Is the teacher’s camera on or off? q On q Off q On for part of the lesson q It was not possible to tell during the 4. observation q Other: _________________________________________________________________ 1. Is the teacher’s camera on or off? On Off On for part of the lesson It was not possible to tell during the observation 2. Are students’ cameras on or off? q On for the whole lesson q On for some of the lesson Other: __________________________________________________________________ 5. q On when requested by the teacher q Off for the whole lesson q It was not possible to tell during the lesson observation 2. Are students’ cameras on or off? On for the whole lesson On for some of the lesson 3. Are parents / caregivers attending the q Yes, some parents are present q It is not possible to tell q No parents are present lesson? On when requested by the teacher Off for the whole lesson It was not possible to tell during the lesson observation 123 98 | Appendixes Observation theme Description/comments 4. What digital tools and learning materials q Madrasati: identify which tool(s) and program(s): are being used for this lesson? Madrasati platform (to access the lesson), Madrasati homework, Madrasati exam etc. ___________________________________________________________________________ (check all that apply) q IEN portal q IEN TV channels q YouTube q Microsoft 365 applications: identify which program(s): MS Teams, OneNote, OneDrive, Forms, Sway, etc_____________________________ q Online educational games: _________________________________________________ q Textbooks (online) q Textbooks (hard copy) q Other: e.g. Classpoint _____________________________________________________ Planning the learning 5. How well has the lesson been planned for distance learning—is there a clear structure to the lesson? For example, is there evidence of the teacher setting the scene, reviewing previous learning, setting learning intentions and success criteria Establishing a positive climate for learning 6. If using their camera, is there a high level of teacher “presence” in the lesson; for example, maintaining appropriate eye contact, being animated, showing enthusiasm, making learning interesting and fun? Describe the evidence for your answer Saudi Arabia's Digital and Distance Education | 99 Observation theme Description/comments 7. If the teacher is NOT using her/his camera what technique(s) do they use to demonstrate a high level of “presence”, encouraging engagement and interaction? For example, using a friendly greeting / morning chat/ humor /emoji, etc. 8. Does the teacher speak using a clear voice, with strong projection and proper enunciation? 9. How successful is the teacher in ensuring that students’ behavior is compatible with their effective learning and does she/he acknowledge positive student behavior? Teaching and learning 10. How well does the teacher demonstrate q Very good knowledge and understanding their knowledge and understanding of q Good knowledge and understanding their subject? q Adequate knowledge and understanding q Poor knowledge and understanding 11. How knowledgeable and familiar is q Very good knowledge and skills the teacher with the digital tools and q Good knowledge and skills learning materials used in this lesson? q Adequate knowledge and skills q Poor knowledge and skills 100 | Appendixes Observation theme Description/comments 12. How knowledgeable and familiar are q Very good knowledge and skills the students with the digital tools and q Good knowledge and skills learning materials used in this lesson? q Adequate knowledge and skills q Poor knowledge and skills 13. To what extent are the teacher’s pedagogical approaches effective for this subject and grade? Does the teacher demonstrate a variety of teaching techniques and aids (visual, listening, doing etc.) to support the learning needs of all students? 14. Do students get the opportunity to collaborate through working in pairs or small groups online? For example, using breakout groups? Are students encouraged to share/show the teacher their work on screen using their camera? 15. How successfully does the teacher encourage students to develop strong thinking and learning skills, including: • Promoting the use of higher order thinking skills in her/his lesson? Saudi Arabia's Digital and Distance Education | 101 Observation theme Description/comments • Encouraging debate and discussion during the lesson? • Ensuring students have the opportunity to engage in meaningful thinking tasks that require them to actively analyze content? • Supporting student autonomy by providing students with opportunities to make choices and take on meaningful roles in the lesson? Feedback and assessment 16. How effectively does the teacher use questions, prompts, or other strategies to determine students’ level of understanding in the lesson? 17. In the lesson does the teacher use self- marking or peer-to-peer activities with her/his students, such as tests, quizzes, and assignments? 18. How effectively does the teacher provide feedback and praise to the class or individual students to help them in their learning? Other: Any other comments?