Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: How to Turn the Tide Noah Yarrow, Eema Masood and Rythia Afkar August 2020 This paper is a product of the staff of the World Bank. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. Rights and Permissions: © 2020 International Bank for Reconstruction and Development/The World Bank The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission maybe a violation of applicable law. The International Bank for Reconstruction and Development/ The World Bank encourages dissemination of its work and will normally grant permission to reproduce portions of this work promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA, telephone 978-750-8400, fax 978-750- 4470, http://www.copyright.com/. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA, fax 202-522-2422, e-mail pubrights@worldbank. org. Attribution: Please cite the work as follows: Yarrow, Noah; Masood, Eema; Afkar, Rythia. 2020. Estimated Impacts of COVID-19 on Learning and Earning in Indonesia: How to Turn the Tide. World Bank, Jakarta. © World Bank. Contact Information: The authors can be contacted at nyarrow@worldbank. org, emasood@worldbank.org, rafkar@worldbank.org Cover Photo: Achmad/World Bank Photo Credits: Achmad/World Bank, Akhmad Dody/World Bank, Ed Wray/World Bank, Ronaldy Irfak/Shutterstock Estimated Impacts of COVID-19 on Learning and Earning in Indonesia How to Turn the Tide By Noah Yarrow, Eema Masood and Rythia Afkar; thanks to Joao Pedro Wagner de Azevedo, Amer Hasan, Massimiliano Cali, Dilip Parajuli, Toby Linden, Neil Butcher, and Frederico Sander for review and suggestions. Abstract: We use the World Bank’s recently developed Country Tool for Simulating COVID-19 Impacts on Learning and Schooling Outcomes and data from the forthcoming Indonesia Education Service Delivery Indicator Survey to simulate and contextualize the potential impact of COVID-19 school closures on learning outcomes, proficiency levels, enrollments and expected earnings for Indonesian students in primary and secondary school.1 We estimate that Indonesian children have already lost 11 points on the PISA reading scale and US$249 in future annual individual earnings due to the four-month closure period from March 24 to the end of July 2020. We provide estimates for six- and eight-month closure scenarios, showing that these losses are expected to increase in the coming months as schools gradually re-open (and possibly re-close). To turn the tide of these human capital losses, districts, provinces and the central ministries should prepare for both improved face-to-face instruction, as well as improved quality of distance education, in order to recapture lost learning and improve overall system quality and resilience to possible future shocks. 1 1 Joao Pedro Azevedo, Koen Geven, Diana Goldemberg, Amer Hasan, Syedah Aroob Iqbal (2020) “Country tool for simulating COVID-19 impacts on learning and schooling outcomes”. World Bank, Washington DC (version 5). Contents 1 Introduction 3 2 Estimating the Impact of School Closures on Learning Performance 5 2.1 The Scenario Modeling Approach 6 2.2 Impacts on Learning and Proficiency Levels 7 2.3 Impacts of Different Scenarios on Learning Levels and Earning Potential 10 2.4 Impact of Income Shocks on Enrolments 12 3 The Potential of Education Technology 14 3.1 Reviewing Current Distance Education Practices 17 4 Considerations for Reopening Schools 19 4.1 Adequacy of School Facilities 20 4.2 Teacher Preparedness 20 5 Recommendations 22 6 Conclusion 26 Annex 28 Annex 1: LAYS Parameters 28 Annex 2: School Year Calculations 29 List of Figures Figure 1: Projected trends in PISA scores: Current (4 months), Optimistic (6 months) and 7 Pessimistic Scenarios (8 months) Figure 2: Change in PISA scores by poorest and richest quintiles: Current Scenario 8 Figure 3: Simulated effects of learning loss on PISA score distribution in Indonesia 9 Figure 4: Simulated effects of learning loss on share of students below minimum proficiency 9 Figure 5: Effect on learning adjusted years of schooling 10 Figure 6: Impact on average annual earning per student 11 Figure 7: Grade 4 student competency in literacy (SDI forthcoming) 11 Figure 8: Grade 4 student competency in math (SDI forthcoming) 12 Figure 9: Availability of household items by urban/rural areas (SDI forthcoming) 15 Figure 10: Availability of household items by poorest and richest quintiles (SDI forthcoming) 16 Figure 11: Examples of math-related questions on the teacher assessment 21 Figure 12: Projections of the pessimistic scenario: Modifying parameters of supply, access, and 25 effectiveness List of Tables Table 1: Estimated school days lost through school closures 29 Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 2 How to Turn the Tide 01 Introduction Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 3 How to Turn the Tide 01 Introduction The COVID-19 pandemic has led to school closures in Indonesia, similar to other countries, forcing more than 68 million young Indonesians out of their classrooms.2 Distance education mechanisms such as online learning and educational television are in place to mitigate the effects of school closures, but with significant variation in access and uptake across students and families.3 Economic contraction is expected to negatively affect school participation even as face-to-face instruction re-starts in Indonesia. School closures are expected to negatively impact academic learning, since effective distance learning requires new skills for both teachers and students. Exclusion and inequality are expected to increase, particularly for already marginalized and vulnerable groups. Drastic action is needed to support learning for all students as part of the continuing recovery and eventual reopening process. To mitigate the effects of the pandemic on education, it is essential to investigate its effects on learning outcomes. This paper presents simulations of the impacts of school closures and economic contraction in Indonesia using the framework and methodology developed by the World Bank (Azevedo et al.),4 adjusting the assumptions and parameters for Indonesia. We also draw on data from a Service Delivery Indicator (SDI) survey conducted by the World Bank, with the Ministry of Religious Affairs (MoRA) and the Ministry of Education and Culture (MoEC). The SDI survey is a national survey of education service quality, which was conducted in February and March 2019 with support from the Government of Australia’s Department of Foreign Affairs and Trade (DFAT). Among other issues of interest, it assesses the availability of key infrastructure and inputs, such as teacher presence and textbooks, as well as the outcomes of education investments, such as student learning levels, teacher classroom practices, and proxies of effort and management. The nationally representative sample included 87 MoEC schools, 253 MoRA madrasahs, and 10 schools of other religions under MoRA. The breadth of information collected in the SDI survey allows us to observe the potential impacts of the shock of the COVID-19 school closures in Indonesia on both the demand and supply side. It is also useful in helping to assess the preparedness of schools to reopen, in addition to other critera. This analysis can help policymakers estimate learning losses from the pandemic and develop effective recovery strategies for the reopening of schools. 2 MoEC Circular Letter No. 4/2020 on Implementation of Education activities in the COVID-19 Emergency Period which closed all schools and suspended exams on March 24, 2020 3 Indonesia High-frequency Monitoring of COVID-19 Impacts Education (World Bank forthcoming); INOVASI April 2020 study; https://blogs.worldbank.org/eastasiapacific/indonesias-education-technology-during-covid-19-and-beyond 4 Azevedo, Joao Pedro Wagner De; Hasan, Amer; Goldemberg, Diana; Iqbal, Syedah Aroob; Geven, Koen Martijn. 2020. Simulating the Potential Impacts of COVID-19 School Closures on Schooling and Learning Outcomes: A Set of Global Estimates (English). Policy Research working paper; No. WPS 9284; COVID-19 (Coronavirus). Washington, DC: World Bank Group. http://hdl.handle. net/10986/33945 Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 4 How to Turn the Tide 02 Estimating the Impact of School Closures on Learning Performance Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 5 How to Turn the Tide 02 Estimating the Impact of School Closures on Learning Performance 2.1 The Scenario Modeling Approach The World Bank has developed a country tool to simulate the potential impact of COVID-19 school closures on learning outcomes, years of schooling, and expected earnings, using data from the Human Capital Index of the World Bank, harmonized household survey data from the Global Monitoring Database, and the World Economic Outlook of the International Monetary Fund.5 The analysis presented in this report uses the framework and methodology developed by Azevedo et al.6 but adjusts the assumptions and parameters for Indonesia. Our analysis considers the effects of different periods of school closure (four, six, and eight months) and different levels of mitigation effectiveness to generate three scenarios; ‘current’, ‘optimistic’, and ‘pessimistic’. As of the date of writing this report, Indonesian schools have been closed for more than four months, so the ‘current’ scenario assesses the impact of school closures over this four-month period (March 24 through to the end of July 2020). The ‘optimistic’ scenario assumes all schools will open in September for a total of six months of closure, while the ‘pessimistic’ scenario estimates the impact if schools are closed for eight months. The Government’s plan to re-open schools7 prescribes a district-by-district approach based on infection rates and appropriate facilities for social distancing, with higher levels of education (secondary) opening before lower levels (primary). As of July 13, 2020, higher grades of schools in green zones established by the governmental reopening plan have begun to return to school, representing fewer than 10 percent of total students.8 Our scenarios do not factor in this degree of granularity, instead assuming, for ease of calculation, that all levels of schools in all districts open simultaneously. For the Current Scenario, we assume that schools have been closed for about 30 percent of the academic year, and that the overall effectiveness of alternative modalities to mitigate the effect of school closures averages 33 percent (i.e., that distance education methods achieve, on average, only 33 percent of the learning effect of in-classroom instruction).9 In calculating the latter estimate, we assume that supply of distance education by the Government covers 96 percent of the population, access to internet or television by students is around 86 percent (Susenas 2018)10 and the effectiveness of alternative modalities (predominantly online learning and educational television, but including some small group instruction by teachers, especially in 5 Joao Pedro Azevedo, Koen Geven, Diana Goldemberg, Amer Hasan, Syedah Aroob Iqbal (2020) “Country Tool for Simulating COVID-19 Impacts on Learning and Schooling Outcomes”. World Bank, Washington DC (version 5). 6 Azevedo, Joao Pedro Wagner De; Hasan, Amer; Goldemberg, Diana; Iqbal, Syedah Aroob; Geven, Koen Martijn. 2020. Simulating the Potential Impacts of COVID-19 School Closures on Schooling and Learning Outcomes: A Set of Global Estimates (English). Policy Research working paper; no. WPS 9284; COVID-19 (Coronavirus). Washington, DC: World Bank Group. http://hdl.handle. net/10986/33945 7 https://www.kemdikbud.go.id/main/blog/2020/06/buku-saku-panduan-pembelajaran-di-masa-pandemi-covid19 8 Author’s calculation based on available data at time of publication 9 Overall mitigation effectiveness of alternate modalities (m) = (G*A*E), where G = Government Supply of Distance Learning, A = Access to Distance Learning, E = Effectiveness of Distance Learning. In the case of Indonesia, assuming G = 96%, A= 86%, E = 40% => m = 33%. 10 The share of villages without television and internet signals is 4.2 percent (PODES/Village survey, 2018). Based on this information, we assume that G = 96 percent. According to Susenas, 41 percent of the population in Indonesia used or accessed the internet. Furthermore, 85% of the population watched TV in the past three months. However, this indicator is different from that of TV ownership because people could possibly watch television at a neighbor’s house or another location. Susenas recorded information on whether a household owns a television with a minimum 30-inch screen. The share of households with a television with a 30-inch (or larger) screen is only 14 percent. This small percentage may be because this was the minimum size of television referenced in the question (i.e. households with a television screen smaller than 30 inch are categorized into the ‘no TV’ group). To get a closer estimate of the share of households that own a television, we combine the television variable (watching television in the past three months) with household access to electricity. We assumed that households with access to electricity that have watched television in the past three months likely own a television. Taking into consideration that 96 percent of households have access to electricity according to Susenas 2018, we estimate that 86 percent of households have access to a television and/or the internet (A=86 percent). Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 6 How to Turn the Tide 02 Estimating the Impact of School Closures on Learning Performance remote areas) is estimated at 40 percent, meaning that students accessing alternative modalities are likely to learn slightly less than half of what they would in their classrooms.11 Multiplying the supply (96 percent) by access (86 percent) and by estimated effectiveness (40 percent) yields an overall effectiveness of alternative modalities of 33 percent compared with what children would have learned if schools had remained open. 2.2 Impacts on Learning and Proficiency Levels The Programme for International Student Assessment (PISA) scores are a measure of student learning for reading, math and science. Our model estimates that secondary students in Indonesia will lose on average about 11 points in reading under the Current Scenario of four months of school closures (Figure 1). Based on PISA 2018 data, this translates into a reduction of reading scores of 4 percent of their baseline scores for the poorest quintile and about 2 percent for the richest (Figure 2). Before COVID-19, the difference between the poorest quintile and the richest quintile was 57 PISA reading points (equivalent to 1.4 years of schooling) and after four months of school closures it is expected to be 64 PISA reading points (1.6 years of schooling). Figure 1: Projected trends in PISA scores: Current (4 months), Optimistic (6 months) and Pessimistic Scenarios (8 months) 380 370 37 3 1 360 PISA Scores in Reading 360 355 350 350 340 330 320 2018 Post-COVID Current Scenario Optimistic Scenario Pessimistic Scenario 11 Older students from high-income households may learn the same amount as they would have in the classroom if they are able to benefit from online tutors, online learning platforms that are not part of the government’s free program, and other learning and instructional aids. Younger and lower-income students in rural areas may not be able to benefit fully from the online learning and educational television options if they lack access to a computer or mobile devices, run out of data, lack electricity, are required to do household work and other tasks, and/or if their parents are unable to fully support their learning needs. In this case, they may learn only 5 percent of what they would in-class. Thus, the 40 percent figure is an estimate of the average efficacy of distance learning during school closures under the ‘current’ and ‘optimistic’ scenarios. Please see Azevedo et al. 2020 for a detailed discussion. Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 7 How to Turn the Tide 02 Estimating the Impact of School Closures on Learning Performance Figure 2: Change in PISA scores by poorest and richest quintiles: Current Scenario 335 Score at the end of school closure - Current Scenario 398 349 Baseline scores 406 40 points in PISA scale ≈ 1 year of schooling 0 200 400 600 ESCS: Economic, Social, Cultural Status PISA Scores in Reading Poorest 20% Richest 20% According to PISA 2018, about 70 percent of Indonesian students scored below the minimum proficiency level for reading.12 The average score for the highest income quintile was 406 in reading, which is still below minimum proficiency (the average for the lowest income quintile was 349).13 These relatively low levels of absolute achievement mean that school closures result in a smaller decrease in the proportion of students meeting minimum reading proficiency than it would in countries where a higher percentage of children are already at or above that level. Assuming that all students are equally affected by the learning loss, the share of students who do not meet minimum reading proficiency rises from 70 to 75 percent under the Current Scenario. However, if we assume that students with the lowest learning levels will fall behind, while higher performing students might actually move ahead as they will have better access to distance learning and support, the share of students who do not meet minimum reading proficiency could drop from 70 to 66 percent (as illustrated in Figure 3 by the simulated flatter line). Since Indonesia’s learning levels are so low when schools are open, under the Current Scenario school closures do not impact this indicator in a major way; the Human Capital Index (HCI) shows that 12.3 years of schooling in Indonesia only equates to 7.9 years of learning.14 12 Level 2 of Minimum Proficiency in PISA reading scores is at 407 out of 600. 13 Results for Countries and Economies, PISA 2018 Data, OECD 2020 and WB calculations. 14 https://www.worldbank.org/en/data/interactive/2018/10/18/human-capital-index-and-components-2018 Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 8 How to Turn the Tide 02 Estimating the Impact of School Closures on Learning Performance Figure 3: Simulated effects of learning loss on PISA score distribution in Indonesia Scenarios of score distribution for Indonesia Simulated (Uniform Impact) Simulated (Skewed) Simulated (Flatter) PISA 2018 Minimum Pro ciency Level 0.006 0.004 Density 0.002 0 0 200 400 600 800 PISA reading score Note: the three simulated distributions should be interpreted as: i. All are affected equally (Uniform Impact) ii. Those at the bottom fall behind (Skewed) iii. Those at the bottom fall behind and those at the top move ahead (Flatter) Figure 4: Simulated effects of learning loss on share of students below minimum proficiency Flatter 66% Skewed 74% Uniform Impact 75% PISA 2018 70% 60% 62% 64% 66% 68% 70% 72% 74% 76% Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 9 How to Turn the Tide 02 Estimating the Impact of School Closures on Learning Performance 2.3 Impacts of Different Scenarios on Learning Levels and Earning Potential Learning adjusted years of schooling (LAYS) accounts for the difference between the number of years a child attends school and the actual years of learning the child has completed according to harmonized test scores. In Indonesia, according to the HCI, a child born today can expect to complete 12.3 years of schooling but only learn the equivalent of 7.9 years of schooling.15 This means Indonesia’s pre-Covid LAYS is 7.9. In the absence of additional government interventions, school closures precipitated by the COVID-19 pandemic could result in an additional loss of between 0.4 and 0.7 of a year of LAYS for students in Indonesia: under the Current Scenario, it is estimated that the LAYS will reduce from 7.9 years to 7.5 years; under the Optimistic Scenario it will reduce to 7.4 years; and in the Pessimistic Scenario, it will decrease to 7.2 years. Figure 5: Effect on learning adjusted years of schooling Learning - Adjusted Years of Schooling (LAYS) Pessimistic 7.2 Optimistic 7.4 Current 7.5 Baseline 7.9 6.6 6.8 7.0 7.2 7.4 7.6 7.8 8.0 This drop in LAYS can be used to project how much the current cohort of students could lose in annual and lifetime earnings. Earnings are linked to learning levels, since education makes workers more productive by giving them the skills that allow them to increase their output.16 Students are estimated to lose US$249, US$367, or US$484 annually under Current, Optimistic, and Pessimistic Scenarios, respectively. Using 2017 US dollar purchasing power parity (PPP) in Indonesia, this would lead to a present value loss in lifetime earnings for all students of about US$151 billion under the Current Scenario, equivalent to 13.5 percent of 2019 GDP. Under the Pessimistic Scenario, the total losses are estimated at US$293 billion, equivalent to 26.2 percent of 2019 GDP.17 Figure 6 shows the drop in average annual earnings per student in the country under each of the scenarios. 15 https://www.worldbank.org/en/data/interactive/2018/10/18/human-capital-index-and-components-2018 16 World Bank. 2018. World Development Report 2018: Learning to Realize Education’s Promise. Washington, DC: World Bank. 17 We use the 2019 GDP figure of US$1,116.9 billion, following http://pubdocs.worldbank.org/en/963481563917599502/mpo-idn. pdf Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 10 How to Turn the Tide 02 Estimating the Impact of School Closures on Learning Performance Figure 6: Impact on average annual earning per student Average annual earning per student (2017 PPP $)* Pessimistic $5,299 Optimistic $5,416 Current $5,534 Baseline $5,783 $5,000 $5,100 $5,200 $5,300 $5,400 $5,500 $5,600 $5,700 $5,800 $5,900 According to the SDI Survey (forthcoming) data presented in Figures 7 and 8, learning outcomes of Grade 4 students pre-COVID-19 were between 1.5 and 2.5 grades below their expected level per the Indonesia national curriculum (Figures 7 and 8). Figure 7: Grade 4 student competency in literacy (SDI forthcoming) Distribution of students by Learning-Adjusted Years of Schooling in Literacy All schools Grade 0: Student is unable to reach Grade 1 level criteria Grade 1: 100% of correct answers in identifying words and identifying pictures. Grade 2: Student is able to read aloud all words from a sentence correctly and 100% of correct answers in Grade 1 level questions. Grade 3: Student is able to read aloud 80% words from a passage correctly in 45 seconds and 100% of correct answers in Grade 1 and Grade 2 level questions. Grade 4: Student is able to correctly answer at least 2/3 of reading comprehension questions and 100% of correct answers in Grade 1, Grade 2 and Grade 3 level questions. 31% 28% 28% 27% 27% 25% 13% 9% 7% 5% Grade 0 Grade 1 Grade 2 Grade 3 Grade 4 Sample size: 350 schools, Number of Grade 4 students tested = 3,365 MoEC schools (blue/green bars) and All MoRA schools (grey bars) Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 11 How to Turn the Tide 02 Estimating the Impact of School Closures on Learning Performance Figure 8: Grade 4 student competency in math (SDI forthcoming) Distribution of students by Learning-Adjusted Years of Schooling in Math All schools Grade 0: Student is unable to reach Grade 1 level criteria. Grade 1: 100% of correct answers in identifying numbers and correct answer of single-digit addition or single-digit subtraction Grade 2: Correct answer in ordering from smallest to largest and correct answer of double-digit addition or double- digit subtraction and 100% of correct answers in Grade 1 level questions. Grade 3: At least 5/6 of correct answers in triple-digit addition, single-digit, double-digit and triple-digit multiplication, single-digit and double-digit division and 100% of correct answers in Grade 1 and Grade 2 level questions. Grade 4: At least 2/3 of correct answers in ability to divide double-digits and identify the smallest result, solve a simple math problem, and complete a sequence, 100% of correct answers in Grade 1 and Grade 2 level questions and at least 5/6 of correct answers in Grade 3 level questions. 49% 45% 40% 36% 8% 5% 5% 5% 3% 3% Grade 0 Grade 1 Grade 2 Grade 3 Grade 4 Sample size: 350 schools, Number of Grade 4 students tested = 3,373 MoEC schools (blue/green bars) and All MoRA schools (grey bars) Assuming the same learning loss for Grade 4 students as found in PISA scores among secondary students, we can apply the projections of learning loss found in secondary students to the fourth grade students. This would mean that Grade 4 students are likely to lose an average of 0.4 of a point in both language and mathematics. Thus, the (already disturbingly low) LAYS for Grade 4 school students drops from 2.6 in literacy to 2.2, and from 1.5 in math to 1.1. 2.4 Impact of Income Shocks on Enrolments The SDI Indonesia Survey 2020 (forthcoming) reports that MoRA schools receive around US$50 per student from families in tuition and other charges annually, while MoEC schools receive on average US$275, even though legally tuition fees are not supposed to be charged for public schools.18 In the SDI survey conducted prior to the COVID-19 pandemic, 42 percent of sampled MoRA parents whose child had dropped out of school noted the primary cause of dropout as “supplementing household income by working", while similar responses were provided by parents of children in MoEC schools. Globally, dropout is more common among older children, as the opportunity cost is higher for older children who can earn money working or conduct 18 Act, Indonesia Education. “Act of the Republic of Indonesia No. 20/2003 on the National Education System” (2003). Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 12 How to Turn the Tide 02 Estimating the Impact of School Closures on Learning Performance household tasks and look after younger children while parents work rather than attending school. Thus, income shocks to the economy can be expected to precipitate higher dropout rates and, in turn, reduce school income. According to Indonesia Economic Prospects (World Bank, July 2020), the projected income shock in Indonesia caused by the COVID-19 pandemic in 2020 is estimated to be negative 1.1 percent.19 Economic hardship is linked to school dropout since there is a direct cost to attending school, as well as an opportunity cost, though government social assistance programs may reduce these costs. In 2018, 6 percent of all primary school age children in Indonesia were not enrolled in primary or secondary school (UIS 2018). There is no evidence of consistent gender discrimination in enrolment rates in Indonesia.20 If the income shock is negative 1.1 percent, our model under the Optimistic Scenario of four months of school closure estimates that the rate of out-of-school children (OOSC) is expected to increase by 0.13 of a percentage point for primary students, equivalent to 48,175 additional children dropping out of school at the primary level. At secondary level, the increase in OOSC is estimated at 0.15 of a percentage point, equivalent to 43,031 additional children dropping out at that level. The model estimates that, as households lose income, the dropout rate will increase as the opportunity cost of attending school increases. However, there may be parallel decreases in opportunity costs caused, for example, by reduced earning opportunities during the crisis or a reduced need to discontinue physical enrolment due to remote learning, among other factors. Our model only focuses on the income impact for school dropouts. The income shock to the economy is likely to negatively impact the demand for education by households. Parents might take older children out of school to ease the strain on family income, as was the case in previous economic shocks to Indonesia’s economy.21 Increased learning inequality is expected as students from poorer socioeconomic backgrounds and low attachment to learning may drop out of school. Demand for education may also shift away from schools with higher fees because of households’ reduced willingness or ability to spend on education. Some schools may close because of the financial strain. While this paper focuses on primary and secondary education, reductions in enrolment may be greatest at the tertiary level, where institutions, particularly private ones, are likely to come under severe financial strain.22 However, even at primary and secondary levels, potentially reduced budgets for education caused by dwindling public resources could result in public schools being strained for resources. 19 www.worldbank.org/iep 20 http://documents.worldbank.org/curated/en/603641592335430591/Inclusion-in-Indonesias-Education-Sector-A-Subnational- Review-of-Gender-Gaps-and-Children-with-Disabilities 21 Drawing extensively from World Bank. 1998. Education in Indonesia: From Crisis to Recovery. Available online at http://documents. worldbank.org/curated/en/558971468752104023/pdf/multi-page.pdf. 22 https://blogs.worldbank.org/education/covid-19-east-asia-how-regions-higher-education-systems-are-addressing-crisis-adapt Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 13 How to Turn the Tide 03 The Potential of Education Technology Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 14 How to Turn the Tide 03 The Potential of Education Technology The COVID-19 pandemic has forced a global discussion about the use of distance learning methods to provide a reliable replacement for face-to-face learning. As over 530,000 schools in Indonesia were closed due to the COVID-19 outbreak, the importance of effective educational technology platforms was highlighted as a concrete response to mitigating the loss of academic learning.23 On the supply side, the larger Indonesian providers have mainly been limited to providing tertiary education (such as Harukaedu), or as a supplement rather than substitute to traditional schooling (such as Ruangguru, an interactive e-learning platform for K-12 students in Indonesia). However, the rise in demand for online learning options both by students and teachers is accelerating due to the school closures and expanding use of digital technology in education service delivery.24 SDI Survey (forthcoming) data (Figures 9 and 10) provide an indication of access to these technologies at home. Figure 9: Availability of household items by urban/rural areas (SDI forthcoming) Availability of household items (SDI forthcoming) Television 97% 90% Computer 15% 9% Internet connectivity 5% 5% Mobile phone/Smartphone 95% 94% TV & Computer 15% 8% TV & Cellphone 93% 85% TV & Internet 5% 5% Computer & Cellphone 15% 9% Computer & Internet 3% 2% Cellphone & Internet 5% 5% TV & Computer & Cellphone 15% 8% TV & Computer & Internet 3% 2% TV & Cellphone & Internet 5% 5% Computer & Cellphone & Internet 3% 2% TV & Computer & Cellphone & Internet 3% 2% Urban Rural 23 https://blogs.worldbank.org/eastasiapacific/indonesias-education-technology-during-covid-19-and-beyond 24 Bhardwaj, R., and Yarrow, N. with Calì, M. 2020. EdTech in Indonesia: Ready for Take-off? Jakarta: World Bank Group. Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 15 How to Turn the Tide 03 The Potential of Education Technology Figure 10: Availability of household items by poorest and richest quintiles (SDI forthcoming) Availability of household items (SDI forthcoming) Television 86% 96% Computer 2% 29% Internet connectivity 1% 16% Mobile phone/Smartphone 95% 98% TV & Computer 2% 29% TV & Cellphone 82% 95% TV & Internet 1% 16% Computer & Cellphone 2% 29% Computer & Internet 0% 8% Cellphone & Internet 1% 15% TV & Computer & Cellphone 2% 29% TV & Computer & Internet 0% 8% TV & Cellphone & Internet 1% 15% Computer & Cellphone & Internet 0% 8% TV & Computer & Cellphone & Internet 0% 8% Quintile 1 Quintile 5 These data suggest that many children cannot access online learning at home, either because some areas lack adequate internet connectivity or because not everyone can afford to pay for it. While 92 percent of sample households owned a television and 95 percent of households had a mobile phone or smartphone, only 10 percent had a computer. Internet connectivity, essential to access online learning, was reported in only 5 percent of sample households. Rural area households have slightly less access to “computer and internet” compared with urban area schools, but just as likely to have a cellphone and internet access (Figure 9), while lower income households are less likely to have access to computers, internet connectivity, and other devices necessary to access online and digital learning (Figure 10). Providing equal access to necessary technology is key in curbing inequities that will otherwise escalate with the increasing adoption of online learning. In terms of school facilities, the SDI Survey (forthcoming) found at least one computer in 93 percent of MoRA schools, and 99 percent of MoEC schools, though a single computer would be inadequate for e-learning and Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 16 How to Turn the Tide 03 The Potential of Education Technology most likely to be used for administrative purposes. However, half of all schools surveyed lacked a working internet connection. With the provision of broadband internet and sufficient computers and mobile devices, it would be possible for teachers to be trained remotely in providing distance instruction to students, and for students to be familiarized with digital learning as schools re-open. This will be important to improve resilience to possible future shocks, whether re-closing in response to successive COVID-19 waves or in response to future climate-change or natural disasters. The existence and provision of connectivity and devices is, however, only part of the solution. Adaptation of course materials to be delivered in digital form by teachers may require extensive teacher professional development. While the re-opening of schools in Indonesia will start in districts and provinces that meet the guidelines, this is expected to include very small numbers of schools in the initial months.25 26 Temporary mitigation through distance education mechanisms may have to make way for more permanent systems that incorporate educational technology into the curriculum, allowing for more reliable replacement of existing learning structures. On the supply side, the lack of effective distance learning mechanisms for most children is likely to result in lower quality education service delivery, unless urgent action is taken now to improve internet and device access for all students and train teachers to deliver improved online learning. Teachers, parents, students, schools, and local governments, as well as MoRA and MoEC, are working to support student learning while protecting health and safety. Emergency distance learning programs have been deployed by internet and TV, as well as in some areas printed resource packets and small-group socially distanced education. 3.1 Reviewing Current Distance Education Practices Given that fewer than 10 percent of schools are expected to open initially under MoEC’s guidelines27 for school reopening by the end of July 2020, many students should expect to continue some type of distance education in the coming months. Surveys recently conducted by MoEC and MoRA found many positive practices but also areas of concern.28 According to these MoEC and MoRA surveys, most teachers are implementing distance education, with only 2.4 percent of teachers and principals reporting that they did not implement the ‘learning from home’ policy, primarily due to lack of supporting devices and connectivity. Mostly, these were in isolated areas. Money also appears to be moving to where it is needed, as 69 percent of principals reported changes in school budget allocations, mostly now allocated to support teachers (51 percent) and prepare learning materials (27 percent). However, capacity to support distance education is lacking. For example, 67 percent of teachers reported difficulties in operating digital devices (including to use online learning platforms). Confirming the SDI Survey (forthcoming) data, 30 percent of teachers reported that students may have limited access to supporting devices (internet access, digital devices, and so on). Limited internet data access has become teachers’ main 25 https://www.thejakartapost.com/news/2020/05/16/jakarta-administration-plans-to-reopen-schools-on-july-13.html 26 https://www.thejakartapost.com/academia/2020/06/03/stick-to-remote-learning.html 27 https://www.kemdikbud.go.id/main/blog/2020/06/buku-saku-panduan-pembelajaran-di-masa-pandemi-covid19 28 All data from MoEC April 2020. MoEC’s survey interviewed randomly selected 1,067 teachers and 988 school principals in 34 provinces covering 3T and non-3T areas, basic and secondary education levels, as well as public and private schools. Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 17 How to Turn the Tide 03 The Potential of Education Technology problem (reported by 20.4 percent of teachers), also leading to difficulties in observing students’ learning (reported by 20.2 percent of teachers). When asked about their perspective on the challenges faced by students in learning from home, most teachers reported limited access to supporting devices as the main issue (reported by 40.5 percent of teachers), followed by difficulties in operating digital platforms (24.5 percent) and difficulties concentrating on learning (23.3 percent) Remote and rural disadvantages were also reported. Online learning is used by 14 percent of teachers in 3T areas (remote, rural or border areas), compared with 20 percent of teachers in non-3T areas. This is partially countered by the higher rate of offline learning (primarily textbooks), which is used by 21 percent of teachers in 3T areas, while only 14 percent of teachers do offline learning in non-3T areas. Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 18 How to Turn the Tide 04 Considerations for Reopening Schools Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 19 How to Turn the Tide 04 Considerations for Reopening Schools 4.1 Adequacy of School Facilities Recently collected World Bank data indicate that 18 percent of surveyed households had a member of the family stop working or reduce working hours to provide child-care as a result of school closures. For these households, this burden most often fell to the mother (66 percent of the time), but also to the father (42 percent) or other family members.29 There is pressure from parents in Indonesia and around the world to open schools. Official World Health Organization (WHO) guidelines to mitigate the spread of the coronavirus note that it is important to wash hands regularly with soap.30 Given that unclean hands and contaminated surfaces can lead to transmission of the virus, cleanliness and hygiene are essential in schools. With proper cleaning and disinfecting facilities provided to students, teachers and other staff, transmission through contaminated surfaces could be mitigated.31 According to SDI Survey (forthcoming) data, only 50 percent of MoRA schools and 56 percent of MoEC schools surveyed contained handwashing facilities with both soap and water. However, 65 percent of urban schools have such facilities compared with 43 percent in rural areas. About 79 percent of the toilets in MoRA schools and 77 percent in MoEC schools were observed to be clean. However, rural schools were 15 percent less likely to have clean toilets than urban schools. In terms of classroom hygiene, only 14 percent of MoRA schools and 18 percent of MoEC schools qualified as ‘extremely clean’. Currently, the pupil-classroom ratio in Indonesian schools is about 22 students per classroom in MoRA schools and 27 per classroom in MoEC schools. Similarly, the ratio at public schools is 27 students per classroom and 22 at private schools, while at urban schools it is 28 students compared with 20 in rural schools. WHO suggests that seating arrangements should ideally ensure that all students are seated at least one meter away from each other.32 If needed, students should be split into separate sections or into different shifts. Prior to the pandemic, 5 percent of MoRA schools and 14 percent of MoEC schools were implementing double-shift instruction, so there is some experience nationally with this approach. However, the numbers are relatively small so this can likely only be used as a temporary measure, along with other measures such as conducting classes outside. SDI data collected in 2019 indicate that there is limited preparedness of school facilities to manage reopening safely in terms of WASH (water, sanitation and hygiene) facilities. Other essential facilities such as libraries in classrooms and schools with engaging, levelled books are also lacking, making a return to learning challenging. 4.2 Teacher Preparedness As our analysis on LAYS data has shown, significant learning deficits will have accumulated during school closures, regardless of which scenario comes to pass. To help students recover from time out of school, 29 Indonesia High-Frequency Monitoring of COVID-19 Impacts, World Bank (forthcoming); numbers add to greater than one hundred since, in one family, one member may stop working completely while another reduces their hours, two family members may reduce their hours, etc. 30 Advice for public. (n.d.). Retrieved June 27, 2020, from https://www.who.int/emergencies/diseases/novel-coronavirus-2019/ advice-for-public 31 http://pubdocs.worldbank.org/en/625501588259700561/Framework-for-Reopening-Schools-APRIL27.pdf 32 https://www.who.int/publications/i/item/considerations-for-school-related-public-health-measures-in-the-context-of-covid-19 Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 20 How to Turn the Tide 04 Considerations for Reopening Schools teachers will need to use pedagogy effectively to help them learn more material, faster. Data from the SDI survey on teacher pedagogy skills and subject knowledge indicate that this will be a significant challenge. The SDI Survey assessed teachers in Indonesia on their pedagogical skills. The assessment asked teachers to prepare a lesson plan based on a short story about road accidents and, given questions about what the aim of the lesson were, what learning outcomes they wanted the students to achieve from the lesson. Teachers in the survey performed poorly on the pedagogical assessment, with an average score of 22 percent in MoRA schools and 25 percent in MoEC schools. This means that their potential to use effective pedagogies to make up for lost student learning is low. In terms of content mastery, Grade 4 teachers in Indonesia scored around 60 percent on the literacy assessment and 65 percent on the mathematics assessment. These assessments tested their skills on primary level content knowledge (for example, simple addition, subtraction to higher- level problem-solving skills such as algebra and fractions in math, and grammar and composition skills in literacy), which are considered essential to teach Grade 4 students effectively. Figure 11: Examples of math-related questions on the teacher assessment Write the answer in the box provided (a) 144 = 12 (a) (a) (b) 12.15 - 11.83 = 0.32 (b) (b) (c) 3 7 21 (c) (c) ÷ = 4 8 32 When interviewed on their socio-emotional skills, teachers in Indonesia displayed a growth mindset more than a fixed mindset, which is encouraging in relation to their ability to learn and cope with adversity and challenges. For example, 90 percent of all teachers agreed with the following statement: ‘I have overcome setbacks to conquer an important challenge’. However, teachers were only seen to be mildly open-minded as measured by ‘Big 5 characteristics’, a measure that assesses intellectual curiosity, aesthetic sensibility, and creative imagination (‘I am someone who is inventive, finds clever ways to do things’). This could be an essential socio-emotional skill when adapting to new technologies, but Indonesian teachers only scored on the mid-level (3.52/5) on this item. The above data suggest a strong need to support teachers with ongoing capacity building before and after schools reopen. Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 21 How to Turn the Tide 05 Recommendations Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 22 How to Turn the Tide 05 Recommendations The above analysis shows that Indonesia, similar to most other countries around the world, was poorly prepared to manage the effects of the COVID-19 pandemic and that this is already having large, negative impacts on student learning outcomes. Some children have already returned to school and would benefit from rapid remediation of learning loss. Most schools are still closed, and these students would benefit from improved distance education to avoid additional losses to learning and future earning potential; these investments can improve long-term education system resilience and quality. Indonesia requires strategies to support both improved face-to-face instruction, as well as improved quality of distance learning, in order to recover and increase its human capital. We offer the following recommendations: To support face-to-face learning: 1. Ensure that reopening of schools is preceded or accompanied by re-enrolment campaigns to minimize student dropout. Groups that may be at higher risk of dropout should receive targeted support and communications, for example local hotlines for parents and community members to report needs for individual enrolment support. 2. Work with provincial and district governments to design an intensive school and madrasah infrastructure campaign to ensure working sinks, toilets, and appropriate classroom spaces to accommodate students with social distancing in place, as well as well-stocked classroom libraries to support a love of reading and learning. 3. Place strong emphasis on learning recovery once students are back in school to prevent permanent impacts on the opportunities of children and youth. This will require multiple measures targeted at reversing learning losses, including: a. Formative assessment in the classroom to identify learning loss; b. More focused pedagogies and curriculum that enable targeted teaching at students’ specific learning levels, supported by practical training and evaluations for principals and teachers; and c. Blended use of teaching and technology to ensure that all students receive the support they need to become effective learners. To support distance learning: 4. Expand access to online learning and distance education by expanding internet and radio connectivity, as well as access to devices, starting in areas with the greatest need. As schools could become a source of internet access in remote communities, MoEC and MoRA can, together with local governments, prioritize 100 percent internet access for all schools to support teachers to boost student learning. 5. Work with teachers and school leaders, through provincial and district offices, to organize check-ins by phone or in person, while respecting social distancing protocols to keep children engaged in learning and troubleshoot online and television access issues. Local governments can focus their efforts on Program Indonesia Pintar (PIP)33 recipients to ensure that these lower-income students receive additional support. 33 PIP (Program Indonesia Pintar) aims to lower the cost of access to education by providing cash transfer directly to poor students. Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 23 How to Turn the Tide 05 Recommendations 6. Examine, evaluate and improve the quality of distance learning materials available through public and private sources. Teachers and students currently have many options to choose from for online and other distance-learning modalities, but little or no information about which of these resources are most likely to be effective learning resources for their students. Improving the quality of distance learning materials is likely more beneficial, on average, than expanding access to distance learning (see discussion below). Discussion In terms of cost-benefit analysis, or where resources can be most effectively focused, we tried to answer one of the many questions facing decision-makers: is it more effective to spend money and political capital on increasing access to distance learning, or the quality of distance learning? We start with our baseline assumptions that about 96 percent of the Indonesian population is covered by internet and/or TV, the two main modes of learning support during school closure. This is a measure of supply, based on the PODES/Village survey (2018). However, only 86 percent of the population can access this supply, having the means (internet and/or TV; see footnote 7 for a detailed discussion). We then assume that distance learning is 40 percent as effective as face-to-face instruction (see footnote 8 for detailed discussion). The access and effectiveness estimates are imperfect, but we have a high degree of confidence that overall supply and access is high, while effectiveness is low, regardless of the specific parameters. Based on these assumptions, which is the most efficient investment? The estimated impact on PISA reading scores under the Pessimistic Scenario of eight months of school closure (an additional four months beyond July) is shown in Figure 12. This assumes the original high levels of supply and access with 40 percent effectiveness, leading to a 21-point drop in reading scores. Pessimistic Scenario Variation 1 shows our estimate of what would happen if resources were invested to achieve 100 percent supply and access, while maintaining 40 percent effectiveness; a 19-point drop in reading scores, equivalent to reducing the impact of school closure on reading scores by 9.5 percent. Pessimistic Scenario Variation 2 shows our original estimate of supply and access unchanged, but a 50 percent increase in the effectiveness of distance learning. This would lead to a 16-point drop, equivalent to reducing the impact of school closure on reading scores by 24 percent. While the magnitude of the change in the underlying variables is important, as are other essential questions such as cost, this thought experiment indicates that the largest gains in learning are likely from improved quality of distance instruction. Our closing recommendation is therefore to prioritize the quality of distance learning programs and materials to ensure that students and teachers are receiving the highest quality support in these difficult times, while also working to expand access to distance learning for those students who do not yet have it. Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 24 How to Turn the Tide 05 Recommendations Figure 12: Projections of the pessimistic scenario: Modifying parameters of supply, access, and effectiveness34 380 371 370 PISA Scores in Reading 360 355 350 352 350 340 330 320 2018 Post-COVID Pessimistic Scenario - Original Pessimistic Scenario 1 Pessimistic Scenario 2 (S=96%, A=86%, E=40%) (S=100%, A=100%, E=40%) (S=96%, A=86%, E=60%) 06 34 S= Government Supply of Distance Learning, A = Access to alternative modalities, E = Effectiveness of alternative modalities, m = Overall mitigation effectiveness of alternative modalities, m=(G*A*E) Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 25 How to Turn the Tide Conclusion Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 26 How to Turn the Tide 06 Conclusion There is an opportunity to learn from the successes and failures of the various approaches during the COVID-19 pandemic to mobilize more effective and equitable approaches to close learning gaps for all children. This can help build a stronger and more resilient system, with teachers at the heart of a learning process that is fun and engaging for students, and in which school directors are collaborative instructional leaders. It is essential to ‘build back better’, since Indonesia’s pre-COVID-19 education system failed to achieve high levels of learning for all students, and was susceptible to disruption by external shocks. Improvements in both face-to-face and distance learning are important, since both will likely continue in tandem for some time. Investments in distance learning are an opportunity to improve support to rural and remote teachers and schools, improve data collection and sharing, and increase accessibility of high-quality learning materials for students, while also opening up new options for parental engagement and support. They can also make the system resilient to climate, natural and manmade disasters. Improving face-to-face learning to recover from the effects of COVID-19 and build a stronger system in the longer term will require: (i) early warning systems to prevent dropout; (ii) pedagogy and curriculum for teaching at the right level and building foundational skills; and (iii) ramped-up support for parents, teachers, and students, including socio-emotional support. The World Bank presents a detailed set of recommendations for educational change as part of its flagship publication, The Promise of Education in Indonesia (forthcoming). The Flagship Overview is available at https:// documents.worldbank.org/en/publication/documents-reports/documentdetail/968281574095251918/ overview.) Achieving these objectives in the decentralized system for MoEC, and the centralized but largely non- public system of MoRA, requires coordination with other ministries and a broad array of stakeholders. It is a complex challenge that multiple actors are already working on. The soon-to be-released Peta Jalan Pendidikan (Education Roadmap) presents a transforming vision of education services, where the best candidates are selected to be teachers, they are supported to help all students learn, school leaders are drawn from the best teachers, and school infrastructure, governance and digital tools support a student population that is excited about learning. MoRA is working along related lines, with an ambitious program to improve student assessment, teacher training, management of school resources and data systems for informed decision-making. Implementing these recommendations will require substantial resources, so it is essential that education budgets are protected at a time when families will be less able to support education at home and the demands on public schools might increase. Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 27 How to Turn the Tide Annex Annex Annex 1: LAYS Parameters Below is a breakdown of parameters used to present likely financial changes in learning adjusted years of schooling (LAYS) caused by school closures in financial terms (using US dollars). Years for currently enrolled cohorts to enter the labor market 10 Years of working life 45 Adult survival rate from HCI 2018 database 0.83 Discount Rate 3% Returns to education 11%35 Labor force participation 67 35 35 Comparable estimates of returns to schooling around the world (English). Policy Research working paper; no. WPS 7020. Washington, DC: World Bank Group. http://documents.worldbank.org/curated/en/830831468147839247/Comparable-estimates- of-returns-to-schooling-around-the-world Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 28 How to Turn the Tide Annex Annex 2: School Year Calculations Table 1: Estimated school days lost through school closures Date and month Standard school calendar COVID-19 adjusted Estimated school calendar days lost March 1 to 31 School in session 5 April 1 to 22 School in session 16 April 23 to 24 Ramadhan break 0 April 27 to 30 School in session 4 May 1 to 16 School in session School closed on or around 11 May 18 to 31 Eid break March 24 0 June 1 to 26 School in session 19 June 29 to July 11 End of academic year holiday 0 July 13 Start of academic year 2020/21 - July 13 to 31 School in session 14 Total school days lost from March 24 to July 31: From 69 to 81 days Total expected number of school days in AY 2019/20: 238 The number of school days was calculated using SDI data collected on the academic year 2017/18. The mean value of the sample of 349 schools and madrassahs was 238, with the lowest reported number of school days was 174, and the highest 270. Using the mean value and the expected number of school days missed based on a five-day school week, children will have lost at least 69 days of instruction. For schools running on a six-day schedule, students will have lost an additional 12 days, for a total of 81. The means the number of days lost is at 69 and as many as 81, so the portion of the year lost is about between 29 percent and 34 percent. Generally, the number of school days is decided by the District/Provincial Education Office, therefore it is different across districts. For example, in Demak primary students in schools with five instructional days per week have in total 214 school days (165 days for learning, 49 days for orientation days, exams, other activities), while students in schools with six instructional days/week have 262 school days (208 effective for learning, 54 days for other activities). In Jambi, it is between 200 and 245 days in a year.36 The DKI Jakarta calendar has 222 school days, a figure that includes other activities such as exams, orientation, etc. 36 Decision Letter of Head of District Education Office Demak No: 420/3544/2018, Decision Letter of Head of Province Education Office Jambi No: 232/DISDIK-1.1/V/2019, Decision Letter of Head of Province Education Office DKI Jakarta No: 509/2019. Estimates of COVID-19 Impacts on Learning and Earning in Indonesia: 29 How to Turn the Tide