Measuring Women and Men’s Work Summary of Main Findings and Recommendations from a Joint ILO and World Bank Study in Sri Lanka Introduction Between 2017 and 2019, the International Labour survey (MLSS). The study was supported by Organization (ILO) and World Bank completed Data2X and the William and Flora Hewlett a multistage pilot study, in collaboration with Foundation under the umbrella of the Women’s the Department of Census and Statistics (DCS) Work and Employment Partnership. of Sri Lanka, with the objective of developing guidance, for various types of household The findings and their implications have been surveys, on good practices in the measurement described in a joint ILO and World Bank report. of women and men’s work, as defined in the A selection of the main findings is highlighted standards adopted by the 19th International in Box 1 below and further elaborated in the Conference of Labour Statisticians (ICLS). following sections. It focuses on some of The Sri Lanka pilot study was designed to enable the key lessons that should be considered by a comparison of the outcomes of two types of anyone seeking to design a household survey to household surveys, namely, the labour force measure participation in work in line with the survey (LFS) and the multitopic living standards latest standards. Measuring Women and Men’s Work Main Findings from a Joint ILO and World Bank Study in Sri Lanka Box 1 Summary of main findings and recommendations  The Sri Lanka study highlighted planned on this issue to study the important differences most efficient methods to capture between the LFS and MLSS good quality information on time in the measurement of key spent doing unpaid household labour-related indicators, e.g. service work. participation in various forms of paid and unpaid work. These  All surveys should emphasize differences were reduced through good translation and national relatively minor changes in adaptation, as well as interviewer questionnaire content (e.g. by training and supervision. adding recovery questions) and survey implementation.  The harmonization of questionnaire content may be  The risks of misclassification or a way to improve consistency measurement difficulties were in measurement, but it cannot concentrated among people be assumed that absolute engaged in casual, low-hours consistency between the LFS jobs or people helping on family and other household surveys can farms or in businesses. This is be achieved or that the need for highly relevant from a gender a national process of adaptation perspective because these types and testing can be ignored. of activities were more common among women respondents in the  The findings presented in this pilot study, which is also likely to report are being used to update be true in many other settings. the guidance, tools and support of the ILO for LFSs and the World  In addition, the measurement Bank for MLSSs. The model of working time in unpaid work questionnaires, guidance and appears to be highly sensitive to tools are excellent reference the measurement approach (e.g., points for those facing the task the use of one vs. two questions of designing a questionnaire to to capture hours per week). capture work- and labour-related This was especially evident in issues through a household survey unpaid care work. Further work is in line with the latest standards. 2 Measuring Women and Men’s Work Main Findings from a Joint ILO and World Bank Study in Sri Lanka 1 Summary of findings: identification of employment Measurement of important prerequisites to achieving employment – key messages comprehensive coverage of employment in line with the latest standards.  The measurement of employment, as defined within the 19th ICLS standards, has been shown The 19th ICLS standards (adopted in 2013) to be sensitive to survey design and content. established an updated definition of The Sri Lanka study demonstrates that there employment that is focused on work done in is a higher risk of the underidentification of return for pay or profit, a narrower definition certain types of employment activities, such than the previous definition established at the as helping without pay in family businesses 13th ICLS (1982). or farms or casual jobs and small income- generating activities. Women are more likely The Sri Lanka study showed that there is a clear to be involved in these types of activities risk of undercounting various types of working than men. This creates a risk that the scale activities or of misclassifying employment and of gender gaps in employment may be the own-use production of goods among those misrepresented. These findings are consistent engaged in own-account farming. In wave 1 of data with earlier pilot studies by both the ILO and collection, the LFS identified 22 per cent more the World Bank. employed women than the MLSS, equivalent to an 8.1 percentage point difference in the measured  Experience, including that gained from the employment to population ratio. The LFS also study in Sri Lanka, has shown that these identified approximately 3 per cent more employed risks can be reduced with well-targeted, men, a 2.4 percentage point difference in the well-worded “recovery” questions that more employment to population ratio. This resulted in a directly reference the types of activities at gap of 10 per cent overall between the surveys, a risk of undercount in the national context.1 5.5 percentage point difference in the employment to population ratio (see Figure 1). In-depth analysis  Good adaptation to national context and of the data led to the conclusion that the gap translation to local languages are also emanated from the fact that the MLSS, which, unlike the LFS, did not initially include any recovery questions, did not fully capture people engaged 1 Recovery questions are here defined as questions whose purpose is in more casual, low-hours jobs, helpers in family to “recover” persons who were not classified as employed during the businesses and farms or others in informal working core questioning designed to capture employment, even though they were engaged in activities that count as employment. activities, primarily women. 3 Measuring Women and Men’s Work Main Findings from a Joint ILO and World Bank Study in Sri Lanka Figure 1 Employment to population ratio (% of working-age population), by sex, wave of data collection and survey 57.0 Wave 1 TOTAL 51.5 57.4 Wave 2 53.9 72.4 Wave 1 70.0 MALES 73.5 Wave 2 68.9 44.1 Wave 1 36.0 FEMALES 43.8 Wave 2 41.3 0 10 20 30 40 50 60 70 80 LFS MLSS Source: Joint DCS, ILO, and World Bank pilot study in Sri Lanka, Wave 1 and Wave 2, March–October 2019. Changes to address these issues were successful These conclusions support the development in significantly narrowing the gap in wave 2 of of guidance on good measurement practices, data collection (a 6 per cent gap among both men such as the need for recovery questions, careful and women, resulting in a 3.5 percentage point wording and translations into local languages, difference in the ratios of total employment to to ensure that people with “small” jobs or population in the two surveys). people helping in family businesses or farms are identified as employed in the survey. It was This LFS-MLSS comparison highlights the also notable that the revisions to the MLSS key groups and types of workers at risk of instrument, while important in the measurement misclassification and supports earlier findings from of employment, also improved the measurement ILO, World Bank and academic studies that these of own-use production work in agriculture risks are greater among women than among men. (described below).2 2 In the MLSS, a common set of questions was used to identify employment in agriculture (that is, agricultural work for pay or profit) and own-use production work in agriculture (that is, for own or family consumption). The distinction between these two concepts is fleshed out in subsequent questions, which seek information on the intended use of the agricultural outputs (for pay or profit or for own or family consumption). Any revisions that improve the ability of the MLSS to capture employment in agriculture will thus enhance the ability of the survey to measure own-use production in agriculture. 4 Measuring Women and Men’s Work Main Findings from a Joint ILO and World Bank Study in Sri Lanka 2 Summary of findings: measuring other working activities Measurement of the range of different paid and unpaid working other working activities – activities in which people engage. This can key messages reveal women and men’s full working contribution and close important gender data gaps.  As with employment, the measurement of participation in other working activities The Sri Lanka study also included questions is highly sensitive to survey content and on a selection of unpaid working activities. context. For own-use production of goods, Specifically, work to produce goods for own- the questions targeting farming work consumption was covered (called own-use need to be chosen with great care. The production of goods in the standards), which experience of the study demonstrates includes subsistence farming, as well as unpaid that it is important to use locally familiar work to provide services to the household (called terms and to mention a range of relevant own-use provision of services in the standards), activities in the question text. such as housework, childcare and other activities that are predominantly carried out by  A particular sensitivity is required in the women because of gendered social norms. The measurement of the time spent performing other forms of work defined in the standards, unpaid household service work, most namely, unpaid trainee work and volunteer work, notably the unpaid care of children were not covered in the Sri Lanka pilot study. and adults. Differences in wording and implementation that may be considered In wave 1 of data collection, the LFS recorded a minor (such as a greater emphasis on active higher prevalence of both own-use production versus passive caregiving) led to substantial of goods and own-use provision of services than differences in the estimates of working time. the MLSS. The difference was concentrated in Furthermore, to develop guidance on good crop farming, with relatively lower differences measurement practices, a dedicated study in other types of activities. This reflects the fact of methods is needed on the measurement that (as described above) the MLSS identified of the time spent in unpaid care and fewer family helpers and fewer other marginal domestic work. workers in farming. The updates undertaken after wave 1 reduced the recorded gap, with a The 19th ICLS standards established a forms relatively small difference observed in wave 2 of work framework as a basis for identifying (see Figure 2). This suggests that the additional 5 Measuring Women and Men’s Work Main Findings from a Joint ILO and World Bank Study in Sri Lanka Figure 2 Participation rate (% of working-age population) in the own-use production of goods, by sex, wave of data collection and survey 45.0 Wave 1 TOTAL 37.7 38.7 Wave 2 39.8 38.2 Wave 1 30.4 MALES 33.1 Wave 2 31.6 50.8 Wave 1 43.9 FEMALES 43.5 Wave 2 46.6 0 10 20 30 40 50 60 LFS MLSS Source: Joint DCS, ILO, and World Bank pilot study in Sri Lanka, Wave 1 and Wave 2, March–October 2019. questions and wording updates (mentioned a descriptive text to be read aloud by LFS above) were successful in narrowing the gap interviewers to respondents). Conversely, there between the LFS and the MLSS. was no explicit emphasis on active caregiving in the MLSS. As a consequence, the MLSS estimate Even more notable was the sensitivity of data for caregiving activities in wave 1 was nearly three on hours worked in the own-use provision of times the LFS estimate (43.8 versus 16.1). During services. While the MLSS identified fewer the wave 2 training, both sets of interviewers people engaged in these activities in wave 1, it were instructed to read the additional text; the did show a substantially higher average number impact on the results was clear. The LFS result of hours worked per week (34.2 versus 24.8 – in wave 2 was relatively consistent with the see Figure 3). Analysis narrowed this down to a results in wave 1, while the MLSS estimate fell substantial difference between the two surveys by half, leaving a much smaller gap gap in the in the reported hours spent on childcare and on overall estimate of the time spent in the own-use care for dependent adults. A review of practices provision of services in wave 2 (26.1 hours per identified the source of the disparity to be a week in the MLSS and 25.3 hours in the LFS). difference in implementation between the two surveys. While the two surveys used similar The study also shows that the measured weekly questions to identify individuals engaged in hours spent on the own-use provision of services care work among adults and children, the LFS are significantly lower if the survey relies on only emphasized active caregiving (and included one question (seeking information on the number 6 Measuring Women and Men’s Work Main Findings from a Joint ILO and World Bank Study in Sri Lanka Figure 3 Average hours actually worked in the own-use provision of services, by sex, wave of data collection and survey 24.8 Wave 1 34.2 TOTAL 25.3 Wave 2 26.1 12.8 20.8 Wave 1 16.4 29.4 MALES 12.0 23.4 Wave 2 11.9 24.1 33.6 Wave 1 45.7 FEMALES 35.4 Wave 2 36.0 0 10.0 20.0 30.0 40.0 50.0 LFS MLSS GENDER GAP Source: Joint DCS, ILO, and World Bank pilot study in Sri Lanka, Wave 1 and Wave 2, March–October 2019. Note: Averages were calculated only for those respondents who reported that they had carried out some own-use provision of services during the reference period. The red diamond indicates the gender gap in working time in the activities covered. The diamond is included on the bar of the gender with lower working time. If it is included on the bar for women, it shows the amount by which the average working time of women in the activity was less relative to men and vice versa if it is shown on the bar for men. 7 Measuring Women and Men’s Work Main Findings from a Joint ILO and World Bank Study in Sri Lanka of hours worked during the previous week) rather the two-question approach leads to a relative than two questions (seeking information on the overestimation relative to the one-question number of days worked during the previous week approach. However, while the direction and scale and the average number of hours per day). In wave of the impact are quite consistent, which of the 2, both the LFS and MLSS surveys administered two sets of results is more valid is not certain. half the samples the one-question approach and the other half the two-question approach. The The study covered many other issues, the results of both surveys were highly consistent. analysis of which enhances our understanding The two-question approach yielded a number of of good practices in the measurement of work, weekly hours spent on the own-use production employment and labour underutilization, as framed of services that was approximately 30 per cent by the 19th ICLS standards. A variety of additional higher than the number of weekly hours captured findings is highlighted in the main report. Perhaps by the one-question approach (see Figure 4). a general summary is that, as highlighted above, This pattern was repeated among both men the measurement of work may be sensitive and women albeit with slightly different gaps. to questionnaire design and implementation, A possible explanation for this last outcome and the study has allowed us to identify where is that the rounding of daily averages in using misclassification risks appear greatest. Figure 4 Average hours actually worked in the reference week by own-use providers of services, by sex, survey and type of questions used to capture working time One question 22.0 LFS Two questions 28.8 TOTAL One question 22.7 MLSS Two questions 29.7 One question 10.2 LFS Two questions 13.8 MALES One question 11.0 MLSS Two questions 12.9 One question 31.0 LFS Two questions 39.7 FEMALES One question 31.0 MLSS Two questions 41.0 0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0 LFS MLSS Source: Joint DCS, ILO, and World Bank pilot study in Sri Lanka, Wave 1 and Wave 2, March–October 2019. Note: Each of the two survey samples was divided into two random groups. The questions about number of hours worked were asked using only one question to one of the groups (that is, “How many hours did you spend doing this last week?”) and with two questions among the other group (that is, “Last week, on how many days did you do this work?” and “And, on average, how many hours per day did (you/NAME) spend doing this last week?”). 8 Measuring Women and Men’s Work Main Findings from a Joint ILO and World Bank Study in Sri Lanka 3 Other general points of note The findings of the studies have been or will be smaller samples that reflect these objectives incorporated into tools and guidance provided and design choices. Surveys with substantially by the ILO for the LFS and by the World Bank for different overall measurement objectives and the MLSS. The various tools available are a useful designs should not be expected to generate reference point for anyone engaged in the design comparable estimates of key labour market– of a household survey that seeks to measure work and work-related concepts. However, the and labour market engagement in line with the experience of the Sri Lanka study shows that 19th ICLS standards. However, in deciding on the careful design can improve the quality of most appropriate approach to adopt in any given the estimates generated by distinct types of household survey, various issues will need to be household surveys and reduce the level of the considered such as the following: differences observed.  What is the objective of the measurement of  What is the socio-economic context of the work and labour market engagement within country? The tools provided by the ILO include the survey? For example, is the motivation a multiple approaches that can be adopted to a desire to generate estimates of the prevalence country’s context. For instance, differences in of participation and time spent in work, or is it the approach to the measurement of labour- the generation of a variable that can be used related indicators may be justified in countries as an explanatory variable in the analysis for with a high prevalence of subsistence poverty or other issues? Another consideration farming versus countries with smaller is the desired range of the indicators on work agricultural sectors. In settings characterized (for example, the characteristics of jobs, the by substantial subsistence farming, the types of work to be covered) and the levels of coverage of both employment and the own- disaggregation targeted. Dedicated LFSs are use production of goods in each household typically designed to achieve a wide coverage survey with a focus on labour will be critical, of a range of work-related indicators and while this may be less important in countries often rely on larger sample sizes than other with less prevalence of these activities. household surveys, thereby enabling greater However, even these countries may wish to precision in the labour market indicators and measure such indicators periodically. Such allowing more extensive disaggregation (within differences in context are reflected in the the limits of the sample size and design). By various versions of the tools provided by the contrast, other household surveys with a ILO for the LFS as well as the short sequences less central focus on labour, but nonetheless proposed for the population census. Similar seeking to cover selected labour-related considerations are relevant in other types issues, typically dedicate fewer questions to of household surveys seeking to cover work the topic and often (though not always) rely on and labour market engagement. For instance, 9 Measuring Women and Men’s Work Main Findings from a Joint ILO and World Bank Study in Sri Lanka MLSSs conducted in settings with substantial both at the international level in the activities subsistence agriculture need to ensure that of international agencies and at the national the own-use production of agricultural goods level among national statistical compilers. In the is covered along with employment by the absence of appropriate testing, the degree of survey instrument. Otherwise, such surveys sensitivity of measurements may never become risk underestimating the importance of apparent. This leaves open the possibility that the agriculture in rural livelihoods, undercounting statistics generated may not be representative women’s work in agriculture and potentially of the true situation among the underlying leading to biases in the estimation of sectoral population, for example, by obscuring important labour productivity. These considerations differences between women and men’s working have informed the new Living Standards lives. Activities at the international level can Measurement Study (LSMS) Guidebook for provide a major support to countries, but not measuring labor in MLSS-type surveys. entirely replace the need for the effective translation and adaptation of questionnaires to A general point is the critical need for good national contexts, a process that needs to be questionnaire development and testing practices supported by qualitative and quantitative testing to put surveys on a sound footing. This is true at the national level to the extent possible. 10 Measuring Women and Men’s Work Main Findings from a Joint ILO and World Bank Study in Sri Lanka Summary Conclusions  The first key conclusion is that the field  The Sri Lanka study highlighted experiment undertaken in Sri Lanka has important cross-survey differences in generated a wealth of rich data that may the measurement of key labour-related be used to identify good practices in variables, particularly participation in questionnaire design and apply the latest various forms of paid and unpaid work. standards in the domain of work statistics. The impact of the changes undertaken in The findings of the study are enabling the MLSS before wave 2 of data collection existing guidance on good practices in the suggests that some of the differences can measurement of women and men’s work to be reduced, if not removed entirely, through be extended to surveys other than the LFS, relatively minor changes in questionnaire including MLSS-type surveys. The depth and content or survey implementation. breadth of the conclusions generated by the study could not have realistically been  A variety of other sensitivities could be generated through another mechanism. identified, such as the sensitivity of the measurement of working time in unpaid  From a gender perspective, it is difficult to work to the measurement approach (for overstate the value of the study. As highlighted example, the use of the one-question in the report on the findings, a much larger method or the two-question method or other part of women’s work relative to men’s work differences). This was especially evident tends to be invisible, at risk of underreporting, in unpaid care work. In addition, all surveys or not measured at all in official statistics. should emphasize good translation and The value of the data generated as the new national adaptation, as well as interviewer standards are applied is demonstrated in training and supervision to promote recently published reports. It is clear that consistency in measurement. pilot studies such as this one are crucial to improving the measurement of both paid and  The risks of misclassification or measurement unpaid work, thereby playing a key role in difficulties were concentrated among people ensuring that the potential of the standards engaged in certain types of activities. For is achieved. These enhancements have been example, higher risks of misclassification the outcome of a long, ongoing process, or undercounting were seen in the case of which gained significant momentum with the people engaged in casual, low-hours jobs adoption of the 19th ICLS standards. This has or people helping on family farms or in been the main focus of the ILO and World Bank businesses. This is highly relevant in gender agenda to operationalize these standards and studies because these types of activities were refine survey methods in the measurement of more common among women respondents in work and labour market engagement under the the pilot study, which is also likely to be true in Women’s Work and Employment Partnership. many other settings. 11 Measuring Women and Men’s Work Main Findings from a Joint ILO and World Bank Study in Sri Lanka  The questionnaires were successful in differences in surveys and across countries capturing a range of paid and unpaid working mean that questionnaires should be adapted activities. This unlocks a wide range of to context and fully tested to improve the analytical potential, such as in obtaining quality of the statistics generated. A starting a deeper understanding of gender gaps point of any survey process should be clear in working activities and labour market discussion and clarification of the key engagement. An important future objective objectives of the survey. This can form the arising from this study and related guidance basis of the choices that need to be made on measurement practices is the promotion about questionnaire content, matching the of the mainstreaming of the measurement of survey objectives and country context. unpaid working activities, alongside labour market engagement, to enable this type of  The findings presented in this report are a analysis on a regular, wide-scale basis. subset of the many findings possible from the studies. The findings are being used to update  The harmonization of questionnaire content the guidance, tools and support of the ILO for may be a way to improve consistency in LFSs and the World Bank for MLSSs. The model measurement, but it cannot be assumed that questionnaires, guidance and tools are excellent absolute consistency between the LFS and reference points for those facing the task of other household surveys can be achieved designing a questionnaire to capture work- and or that the need for a national process of labour-related issues through a household adaptation and testing can be ignored. The survey in line with the latest standards. 12