LEARNING POVERTY IN THE TIME OF COVID-19: A CRISIS WITHIN A CRISIS DECEMBER 2020 This brief summarizes the results of simulations estimating the potential impacts of the COVID-19 pandemic in learning poverty. Of 720 million primary school age children, 382 million are learning poor, either out of school or below the minimum proficiency level in reading. COVID-19 could boost that number by an additional 72 million to 454 million. In a post-COVID-19 scenario of no remediation and low mitigation effectiveness for the effects of school closures, simulations show learning poverty increasing from 53 percent of primary- school-age children to 63 percent. Temporary school closures in more than 180 countries that every child should be in school and be able to read have kept nearly 1.6 billion students out of school, further and understand an age-appropriate text by age 10.2 complicating global efforts to reduce learning poverty. This formulation reflects the aspiration of Sustainable Although most countries have made heroic efforts to Development Goal 4 that all children must not only be put remote and remedial learning strategies in place, in school, they must also be learning. learning losses are accumulating rapidly. Countries and regions have responded in various ways, but they have Of 720 million primary school age children, 382 million are found it difficult to reach even half the students. Students learning poor, either out of school or below the minimum currently in school stand to lose $10 trillion in labor proficiency level in reading. COVID-19 could boost that earnings over their working lives.1 That is one-tenth of number by an additional 72 million to 454 million. In a global GDP, or half the United States annual economic post-COVID-19 scenario of no remediation and low output, or twice the global annual public expenditure on mitigation effectiveness for the effects of school closures, primary and secondary education. simulations show learning poverty increasing from 53 percent of primary-school-age children to 63 percent. A little over a year ago, the World Bank and UNESCO Institute for Statistics (UIS) launched a new multidimen- Most of this increase seems to occur in lower-middle-in- sional indicator, learning poverty, based on the concept come and upper-middle-income countries, especially in East Asia and the Pacific, Latin America, and South Figure 1: Learning deprivation and poverty Asia. Countries that had the highest learning poverty (horizontal axis only) before COVID-19 (predominantly in Sub-Saharan Africa and in the low-income country group) might have the smallest absolute and relative increases in learning poverty, reflecting how great the learning crisis was in those countries before the pandemic. Measures of learning poverty and learning deprivation sensitive to changes below the minimum proficiency level, such as gap and severity measures, show differences in learning loss regional rankings. Sub-Saharan Africa stands to lose the most. Countries with higher inequality among the learning poor, as captured by the proposed learning poverty severity measure, could need far greater adapt- ability to respond to broader differences in student needs. 1 of competencies, leaving a potential space for ambiguity. This ambiguity is solved by using the SDGs definition Learning poverty: of minimum proficiency level. A measure of deprivations of The headcount rate uses the number of children below either deprivation threshold divided by the total number schooling and learning of children in the age category. This ratio, learning poverty, is extremely simple and clear for policymakers to inter- In a recent paper, Azevedo (2020) complements other pret, given the observable nature of school enrollment COVID-19–related simulations by looking at the and the use of an agreed common standard of proficiency learning poverty measure. The main drivers of these defined in the context of the SDGs. simulations are school closures, mitigation and remedi- ation effectiveness, and the income shock to economies Countries can improve this measure by reducing the and households, which affect two main transmission chan- learning deprivation as they raise proficiency levels for nels—learning losses and student dropouts at primary age. children below the minimum proficiency threshold, or they can reduce schooling deprivation by expanding Learning poverty is defined through deprivations coverage and bringing their out-of-school population of schooling and of learning. Each requires a specific into the system. threshold or standard from the education domain. The deprivation of schooling is ordinal and has enrollment as However, the learning poverty headcount ratio has its threshold. Its measurement is simple, since children limitations. It seems plausible that children or educa- attending school are directly observable, and the measure tion systems with lower scores among the poor are is dichotomous, since a child can be in only one of two worse off, other things being equal, but the poverty states—in school or out of school (figure 1). headcount ratio (the share of children in learning poverty) cannot capture that. The learning poverty gap The deprivation of learning is more complicated. or learning deprivation gap, are measures that capture It cannot be directly observed and is measured as a the average learning shortfall among students below cardinal latent variable using large-scale standardized the minimum proficiency level. This measure indicates assessments, which are used to derive a measure of the average increase in learning required to eliminate minimum proficiency based on a desired and agreed set learning poverty (figure 2). 2 LEARNING POVERTY IN THE TIME OF COVID-19: A CRISIS WITHIN A CRISIS Figure 2: Learning deprivation and poverty gap and The learning poverty headcount ratio is suitable for severity (shaded area) countries and regions with average to lower levels of learning poverty. The learning poverty gap and the learning deprivation severity are particularly relevant for high learning poverty settings, such as Sub-Saharan Africa and the low-income country group.3 Since the second and third measures are sensitive to changes in learning that might happen below minimum proficiency, they are the only measures that can capture the effects of shocks in countries where most of the students are already learning deprived. The simulations add three main contributions: the focus on learning at the end of primary, the inclusion of school enrollment effects due to the household income shock, and the inclusion of a remediation effectiveness compo- nent. Mitigation is the effectiveness of government But any average gives an incomplete picture in an responses while schools are closed, considering what the unequal world. By construction, the gap measure cannot government is offering and the ability of households to capture the changes in the learning inequality among the take up what is on offer, given the availability of connec- learning poor or deprived students. To tackle this limita- tivity assets such as radios, televisions, mobile phones, tion, the learning poverty severity or learning deprivation computers, and the internet. Remediation reflects poli- severity measures are introduced (figure 2). This measure cies that might be implemented when schools reopen. captures the inequality of learning among the learning It is assumed to be equally effective across all country poor population and can indicate how flexible the educa- income levels. tion system must be to both identify student needs and offer appropriate learning opportunities. Understanding In what follows we summarize the results from Azevedo such heterogeneity can be of critical importance for an (2020) in terms of three scenarios that are illustrative of effective strategy to reduce learning poverty. potential global and regional increases in learning poverty. So, simulating the effect of COVID-19-related school • In the optimistic scenario about 60 percent of the closures on learning poverty requires simulating the school loss will be fully remediated and about 40 effects on both learning and schooling deprivations. The percent of the school loss while schools are closed simulation results here are based on three complemen- will be fully mitigated in high-income countries, but tary measures: the learning poverty headcount rate, the in the developing world, 30 percent. learning poverty gap, and the learning poverty severity • In the intermediate scenario about 30 percentage are presented. The learning poverty headcount ratio is points of the 70 percent school loss will be fully the share of 10-year old who are not in school (schooling remediated and about 20 percent of the school loss deprived) or are below the minimum proficiency level while schools are closed will be fully mitigated in (learning deprived) (figure 1). The learning poverty gap high-income countries, but in the developing world, is the distance of the average student from minimum 15 percent. proficiency (figure 2). The learning deprivation severity, a measure sensitive to the learning inequality, is the gap • In the pessimistic scenario, there is no remedi- squared in relation to the minimum proficiency squared ation and about 10 percent of the school loss (figure 2). This measure, by providing greater weight to while schools are closed will be fully mitigated students with the largest learning gaps, is able to differen- in high-income countries, but in the developing tiate the distribution of learning among the learning poor. world, 7 percent. 3 One important empirical question remains. Are these But this is not the whole story. It’s also important to look complementary measures empirically relevant? That at inequality or learning poverty severity. For example, depends on whether countries with: learning poverty severity in Nicaragua is almost 10 times that in the Philippines, suggesting a far greater level of • The same learning poverty level have different learning heterogeneity among the learning poor students in the poverty gaps (figure 3, panel A). latter. This finding supports the empirical relevance of • The same learning poverty gaps have different learning the measures and the importance of clarity on which poverty severity (figure 3, panel B). specific properties are needed when choosing one. For policy, the strategies to reduce learning poverty could Figure 3 illustrates those points using the latest avail- differ considerably if the levels of the learning poverty able data from 99 countries in the learning poverty gap or learning poverty severity are drastically different. database with indicators available for the learning poverty gap and learning poverty severity.4 The figure Strategies to reduce learning poverty could differ consid- shows a wide range of learning poverty gaps among erably if the levels of learning poverty gap or learning the poor in countries with similar levels of learning poverty severity are drastically different. Countries with poverty (panel A): Several countries have around 70 the same level of learning poverty but higher learning percent learning poverty, but the Philippines’s learning poverty severity will need far greater flexibility in learning poverty gap among the poor is almost three times (and schooling) strategies to adapt to the needs of chil- Nicaragua’s. This suggests that the effort required to dren with a wider range of learning (and schooling) needs tackle learning poverty in the Philippines might be than countries with the same level of learning poverty greater than in Nicaragua. but a higher learning poverty gap. Figure 3. Relationships between learning poverty, the learning poverty gap, learning poverty severity, the learning deprivation gap, and the learning severity gap A.Countries where students are at the same level of B.Countries that require the same average effort (learning learning poverty, require very different levels of effort poverty gap); have very different levels of learning (learning poverty gap). poverty inequality among students below the MPL. Note: Learning deprivation gap and learning deprivation severity refer to measures computed exclusively from information from the learning dimension of the indicator. Learning poverty gap and learning poverty severity also take into consideration out-of-school information. Each point represents one country assessment (N = 99). Source: Azevedo (2020) http://hdl.handle.net/10986/34654 4 LEARNING POVERTY IN THE TIME OF COVID-19: A CRISIS WITHIN A CRISIS 2 The learning poverty severity measure is also directly rele- vant to policy debates on effective strategies to address the challenge of tacking large learning deficits accompanied by Protecting the learning significant heterogeneity in within-grade student learning levels.5 Just like computer assistant learning data can be of the most vulnerable extremely powerful to characterize the mean and variance in grade-level preparation of students,6 measures such as learning poverty severity or learning deprivation severity can characterize both the mean and variance in grade-level Most governments and development partners are working preparation of students at the educational-system level. on identifying, protecting, and supporting the learning By being sensitive to both the level and changes in the of the most vulnerable members of the COVID-19 learning heterogeneity among low performing students, generation, such as children in the bottom of income this measure can help align incentives for educational distribution, with less access to assets and connectivity, systems to deploy and monitor the effectiveness of inter- or who were already in learning poverty prior to the ventions designed to tackle this challenge. pandemic. Countries’ initial conditions matter, and the size of the learning distribution does not say much about As school systems reopen, it will be critical to meet how vulnerable the youth of different countries are. A students where they are on learning and to monitor headcount measure, such as learning poverty, provides a changes in the learning distribution among the learning focus on the base of the distribution, which is critical for poor, given that evidence suggests that a significant prioritizing actions to support those who were suffering source of inequality is within groups. For that, learning the most before the COVID-19 learning crisis. But that poverty severity is the appropriate measure. The use of measure does not say how much learning is being lost by these complementary measures is supported by both their the children already experiencing learning poverty—for properties and empirical relevance. that, the learning poverty gap is an important measure. Then, as school systems reopen, it will be critical to meet student needs and to monitor changes in the learning distribution among the learning poor—for that, learning poverty severity is the appropriate measure. Figure 4: Learning poverty simulation results Note: All underlying numbers can be found in annex table A.1 in Azevedo (2020) Source: Author’s calculations. 5 The pessimistic scenario assumes no remediation where students are on average the farthest behind in the and very low mitigation effectiveness in low- and minimum proficiency level, with a learning deprivation middle-income countries, learning poverty increases gap of approximately 20 percent (figure 5, horizontal by 10 percentage points, from 53 percent to 63 percent axis). This rate is double the global average (10.5 percent), (figure 4). Sub-Saharan Africa and Europe and Central four times that in East Asia and Pacific (5 percent), and Asia have the smallest absolute increase of learning more than ten times that in Europe and Central Asia (1.3 poverty, 5 percentage points, while South Asia has percent). This average learning gap is equivalent to what the largest (17 percentage points), followed by Latin students are expected to learn, in the respective regions, America (12 percentage points). Sub-Saharan Africa in a full academic school year. In the pessimistic scenario, also has the smallest relative increase, 5 percentage the learning gap for the average student in low- and points, while East Asia and the Pacific and Europe and middle-income countries could increase by 30 percent; Central Asia have the largest (more than 30 percentage and in East Asia and the Pacific and Latin America, points), suggesting that the children in the upper-mid- the regions with the largest relative increase, close to dle-income and lower-middle-income countries are 40 percent. likely to become the new learning poor. This result only reinforces the understanding that Sub-Saharan Africa The gap measures are not distribution-sensitive and was already experiencing a massive learning crisis before cannot distinguish between an increase in the learning COVID-19 in which children were not learning as much gap driven by students near the threshold and one driven when schools were still open. by those at the very bottom of the learning distribution. The Sub-Saharan and Latin American increases in the A complementary and relevant set of measures are the learning gap might be qualitatively different if, in one set gap and the severity. To avoid the confounding of policies of countries, the pandemic were pushing many children required to improve schooling, in what follows we focus marginally below the deprivation threshold and in the on the learning deprivation. Sub-Sharan Africa and other, it were further deepening the deprivation of those the Middle East and North Africa are the two regions already far below the threshold. Figure 5: Pre-COVID, the Middle East and North Figure 6: Post-COVID, the learning gap might widen Africa and Sub-Saharan Africa had the greatest by about the same in several regions, but the challenge, as both the gap and severity of severity could increase the most in the Middle learning are the highest East and North Africa and in Sub-Saharan Africa Bigger Bigger and more and more heterogenous heterogenous problem problem Smaller Smaller and more and more homogenous homogenous problem problem Note: Variances in the types of Arab script pose specific challenges to teaching and assessing learning in the Middle East and North Africa. All underlying numbers can be found in annex table A.2 in Azevedo (2020). 6 LEARNING POVERTY IN THE TIME OF COVID-19: A CRISIS WITHIN A CRISIS Severity measures can distinguish among these qual- in Europe and Central Asia. These results suggest an itatively different types of impacts. Results for Latin increase in the complexity and the cost to tackle the America, for the Middle East and North Africa, and for learning crisis in the continent. Sub-Saharan Africa suggest that on average, students in those regions are experiencing a similar increase in the Countries with the same level of learning poverty but learning gap, of approximately 2.5 percentage points a higher learning poverty gap will need a far greater (figure 5, horizontal axis). But the learning gap fails to effort to bring children above the minimum proficiency take into account the inequality of learning among the level. At the same time, countries with the same learning learning deprived. This means that a hypothetical reduc- poverty gap but different learning poverty severity will tion of 20 learning points is fully equivalent, whether need far greater flexibility in learning (and schooling) for students just below the minimum proficiency level strategies to better align their education systems with or at the very bottom of the learning distribution. This student needs.7 They can accomplish this by setting can hide significant differences in the complexity of the clear goals, instructional coherence, teacher support challenge (figure 6, vertical axis). and contextual salience. Both Sub-Saharan Africa and the Middle East and North Africa seem to have the Through the lens of learning severity, a measure sensitive biggest and more complex challenges in terms of learning to differences in learning poverty severity, two distinct deprivation, and those are also the regions where both groups emerge: Latin America, with a learning severity the learning deprivation gap and learning deprivation increase of 0.5 percentage points, and the Middle severity could increase the most. East and North Africa, with an increase greater than 1 percentage point (figure 6, vertical axis). The results This finding suggests that COVID-19 could qualitatively support the idea that the consequence of this crisis is change the learning crisis in the African continent, since qualitatively different for Sub-Saharan Africa and the students will come out of this pandemic in a much deeper Middle East and North Africa than for Latin America. learning crisis than before, falling farther behind the So, students in the latter group might fall much farther, minimum proficiency levels established under SDG 4. relative to the MPL, than those in the former group. This greater depth of the learning crisis in Sub-Saharan Africa and elsewhere will require qualitatively different 3 policy responses of far greater complexity and cost. Governments, development partners, teachers, students, Strategies for reducing and parents must work together to deploy effective learning poverty mitigation and remediation strategies to protect the COVID-19 generation’s future. School reopening, when safe, is critical, but not enough. The simulation results show major differences in the distribution of learning. In absolute terms, Sub-Saharan Africa and the Middle The big challenge will be to rapidly identify and respond East and North Africa remain the two regions that face to each individual student’s learning needs in a flexible the greater challenge to reduce learning deprivation, and adaptive way and to build back educational systems given the magnitude of the problem and the heteroge- more resilient to shocks, using technology effectively to neity of their respective learning deprived students. In enable learning both at school and at home. Latin America, the new learning poor seem to have fallen much closer to the minimum proficiency level. Moreover, the depth of learning deprivation in Sub-Saharan Africa could increase three times more than the number of new learning deprived children in the region. This is almost three times the global average, and four times more than 7 References Instruction in India.” American Economic Review, 109(4):1426-60. Rodriguez-Segura, Daniel, Cole Campton, Luis Crouch, Azevedo, João Pedro. 2020. “Learning Poverty: Measures and Timothy Slade. 2020. “Learning inequalities in and Simulations.” Policy Research Working Paper developing countries: evidence from early literacy No. 9446. World Bank, Washington, DC. http://hdl. levels and changes.” RISE Programme working paper. handle.net/10986/34654 World Bank. 2019. “Ending Learning Poverty: What Azevedo, João Pedro and Diana Goldember. 2020. Will It Take?” Washington, DC: World Bank. http:// “Learning for All: Within-country learning inequality.” hdl.handle.net/10986/32553 Published at Education for Global Development series of the World Bank Blogs. November 12th 2020. https://blogs.worldbank.org/education/ learning-all-within-country-learning-inequality Endnotes 1 Azevedo et al (2020). Azevedo, João Pedro, Amer Hasan, Diana Goldemberg, 2 World Bank (2019). Syedah Aroob Iqbal, and Koen Martijn Geven. 2020. 3 For more details see Azevedo (2020). 4 All learning assessments used are anchored in a standard deviation of 100  “Simulating the Potential Impacts of COVID-19 points, this should be sufficient to have the FGT-class of measures to be School Closures on Schooling and Learning minimally comparable. Of course, within country temporal comparisons, Outcomes: A Set of Global Estimates.” World assuming temporal comparability of the assessments, are the ideal case. All gap measures are relative to the test-specific minimum proficiency level Bank Policy Research Paper 9284. https://doi. (MPL). One interesting aspect is that once the gap conversion is made, the org/10.1596/1813-9450-9284. measure becomes test-independent, and can be presented independently of any scale. One important assumption when doing cross country compari- Hwa, Y., Kaffenberger, M. and Silberstein, J. 2020. sons, which is shared with global poverty monitoring, is that the learning “Aligning Levels of Instruction with Goals and the (income) marginal sensitivities of the cardinal variable are the same. So, Needs of Students (ALIGNS): Varied Approaches, improving one learning point, is equally hard (or equally well captured) across all assessments (or different measures of income and consumption). Common Principles.” RISE Insight Series. 2020/022. Muralidharan et al. (2019), Rodriguez-Segura et al (2020), and Azevedo 5  https://doi. org/10.35489/BSGRISE-RI_2020/022 and Goldemberg (2020) use a range of different standardize learning assessments from developing and developed countries to show that most Muralidharan, Karthik, Abhijeet Singh and Alejandro learning variation are within school or even classrooms. J. Ganimian. 2019. “Disrupting Education? 6 Muralidharan et al. (2019). Experimental Evidence on Technology-Aided 7 Hwa et al (2020). Acknowledgments This brief was prepared by João Pedro Azevedo, under the overall guidance of Jaime Saavedra (Global Director, Education Global Practice) and Omar Arias (Practice Manager, Education Global Practice). This brief benefited to specific suggestions from Luis Crouch, Amer Hasan, and Silvia Montoya. I would like to thank Paola Ballon, Paul Corral, Diana Goldemberg, Shabana Singh, Amer Hassan, Ambar Narayan, Yevgeniya Savchenko, and Umar Serajuddin for providing comments to the working paper “Learning Poverty: Measures and Simulations” from which the main findings of this brief are derived. I would also like to thank Julia Azevedo for her support in preparing the diagrams, and Bruce Ross-Larson for editorial comments. This work would not have been possible without the work of the producers of all the learning data used— the national governments and the teams at the International Association for the Evaluation of Educational Achievement (Progress in International Reading Literacy Study and Trends in International Mathematics and Science Study), United Nations Educational, Scientific and Cultural Organization (Laboratorio Latinoamericano de Evaluación de la Calidad de la Educación), Conférence des Ministres de l’Éducation des États et Gouvernements de la Francophonie (Programme d’analyse des systèmes éducatifs), and Southern and Eastern Africa Consortium for Monitoring Educational Quality, as well as the Organisation for Economic Co-operation and Development—and the staff of the education management information systems departments of all ministries of education, without whom none of this work would have been possible. The findings, interpretations, and conclusions expressed in this paper are entirely those of the author. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Cover photo: “A young girl in class” by GPE/Carine Durand, license: CC BY-NC-ND 2.0 8