90878 Purchasing Power Parity and Education Productivity Analysis: Preliminary Draft by Richard Ashford Consultant for: Academy for Educational Development Presented to International Comparative Program Regional Coordinator’s Meeting held in Washington, DC April, 14, 2010 1 1.0 Introduction Over the past several decades, new ways of comparing global development levels across nations have been developed to overcome deficiencies in the use of nominal exchange rates. One method which has received considerable attention and success is the purchasing power parity. "Purchasing power parity (PPP) is a disarmingly simple theory which holds that the nominal exchange rate between two currencies should be equal to the ratio of aggregate price levels between the two countries, so that a unit of currency of one country will have the same purchasing power in a foreign country" (Taylor and Taylor, 2004p1). Since 1968, the International Comparison Program (ICP), a partnership between the World Bank, OECD, the UN system and many other development agencies in collaboration with (in 2011) 181 countries, has been tasked with developing practical applications for applying PPPs to compare output of economies and the welfare of their people in real terms. Calculating PPPs requires a common volume and price measurement system which act as a conversion factor to adjust or measure GDP, GNI, etc. There have been some criticisms of PPP, but with recent advances in theory and methodological application, some variant of PPPs have become accepted as a means to calculate real exchange rates (Rogoff, 1996). Although initially designed to serve monetary policies with regard to exchange rates, PPP has come to be used as a weighting mechanism for a variety of other areas by many international organizations: "international poverty headcount index (World Bank), comparing relative sizes of economies and estimating weighted averages of regional growth rates (IMF), allocation of structural and cohesion funds (European Commission), Human Development Index (UNDP), gender empowerment measures (UNDP), health inequality assessment (World Health Organization), assessing per capita expenditures in education (U.N. Educational, Scientific and Cultural Organization), monitoring the welfare of children (U.N. Children’s Fund) and designing effective aid programs (International Organizations)" among others (ICP, 2010). Although the theory of PPP has come to be generally accepted, several technical issues still exist that require redress. One major problem has been and continues to remain that non-market services generally and education services specifically are difficult to measure. Generally, the PPP literature, especially with regard to education, refers to problems in distinguishing between nominal and real per capita expenditure (World Bank, 2008). Education is "comparison resistant" (World Bank, 2007) and in past ICP rounds, volumes were compared directly while prices were obtained implicitly (World Bank, 1993). As of the 2005 round of the ICP, education was measured through an input-price approach. Problems exist with the PPP methods for determining education PPPs because inputs are taken as outputs and productivity is largely ignored (World Bank 1993). Inputs (expenditures) cannot be used as a proxy for outputs because the ratio of input to output differs so greatly between countries. In the World Bank review of the 2005 round of the ICP, education was found to have the greatest variation in price levels across countries, showed the greatest difference between nominal and real expenditure and the greatest difference between nominal and real expenditure per capita (World Bank, 2007). The question is: why? Some have suggested that the problem is methodological and that a great deal of variation may be explained by inconsistencies, data omissions, etc. (Barro, 1997). Still others suggest that input approaches or expenditure approaches are inappropriate for measuring education productivity 2 (Fraumeni et. al., 2008; Hanushek & Kim, 1995; Lequiller, 2006). Expenditure approaches are also problematic in disaggregating data (OECD, 2006). Input approaches are theoretically unsound and have produced unacceptably high degrees of variation (Stiglitz, Sen & Fittoussi, 2009, World Bank, 2008). Thus, rather than using an input approach to measuring the contribution of educational services to national production, some argue that the solution to problematic national accounting for comparison resistant education is to use an output approach (Stiglitz, Sen & Fitoussi, 2009) and measure educational output (Lequiller, 2006) while adjusting for educational quality (Atkinson, 2005). Some go further and advocate the use of examinations as a proxy for quality (Atkinson, 2005) while others advocate using examination scores to measure outputs generally (Deaton & Heston, 2009). Scholars and practitioners are proposing to devise better ways to capture volume and price measures for education. In order to reliably add education production to national productivity estimates for international comparative purposes, issues dealing with approach, methodology and measurement should be studied. A single recommendation for calculating PPPs for education is beyond the scope of this paper. Rather, we propose to review approaches involved in calculating PPPs for education, examine the issues and debates and evaluate some of the requirements for particular approaches. This paper draws on several bodies of literature to better frame these issues, review the literature and come to some consensus on logical routes to resolve current problems of comparison- resistance in order to improve methodological approaches to educational production in the upcoming 2011 ICP process. Generally, the most common suggestions advocate refining volume measures to include pupil hours of schooling and to use international examinations as a proxy for quality in order to impute price measures. 2.0 Using Purchasing Power Parity Although this paper is concerned primarily with addressing problems with measuring and comparing educational productivity, it is important to briefly review the literature surrounding the use of PPPs generally not only to contextualize this paper's findings, but also to point toward potential theoretical and methodological issues germane to the application of PPPs to educational services. PPP requires that a basket of common goods (and services) be established with which to compare across countries. This basket acts as the norming mechanism for prices given the same 'volume' of goods or services. When applying PPPs to adjust GDP, GNI or poverty indices, one needs to be careful in selecting basket goods and over time, the basket has grown and contracted, adding new goods, while removing others. Due to the difficulty in calculating prices, in past years, non-market services were not included in GDP calculations (ICP, 2010). It is important to note that GDP, GNI and poverty indices all rely not only on goods but on services. Further, much productive activity takes place outside of typical market transactions. That is, much productivity occurs through non-market services sometimes referred to as non-profit institutions that serve households (NPISHs) within sectors such as health and education. The 3 primary purpose of this paper is to review literature on ways to refine educational productivity measurement in order to more accurately apply PPPs for various purposes. 2.1 Purchasing Power Parity Applications Initially developed to better understand international trade and the macro-economic influences of exchange rates and inflation, PPP relies on the "law of one price" which suggests that in an efficient market, all identical goods should have the same price1. Although key theoretical and methodological issues are described in Officer's (1982) work on the development and use of PPP in relation to exchange rates, the application of PPP to other areas of economic activity and policy making are gaining ground. Many have contributed to the development of PPP theory and application (see Kravis & Lipsey, 1991; Taylor & Taylor, 2004). Besides exchange rate applications, PPPs are now used to compare GDP productivity by establishing a GDP deflator (Eurostat, 2001), to compare standards of living (Fenstra, et. al., ), and more recently to compare relative poverty by calculating the relative poverty level (ADB, 2008). There are multiple critiques of the use of (absolute) PPPs and the law of one price which point out problems of transaction costs (Davutyan & Pippenger, 1990), relative technology levels (Balassa, 1964), and in incorporating non-tradable goods (Samuelson, 1994) and non-market services (Bullock & Minot, 2006; Kigyossi-Schmidt, 1989) into PPP calculations. These primarily technical critiques do not detract from the promise that PPPs hold in acting as price relatives for comparing GDP. It should be noted, however, that productivity growth rates may differ by country attributes such as level of development2. 2.2 Purchasing Power Parity Approaches and Methods The use of baskets of diverse goods and services produces a more reliable measure of comparability. However, two novel indices have been developed over the past several years that as they point out differences in how we might use PPP for education: the BigMac and IPod indices. Certain goods like the McDonald's Big Mac, or Apple's IPod can be found in markets across the globe and tend to be somewhat identical in composition. These two indices were developed for these products allowing the creation of a standard comparative capacity of pricing. By capturing the average price of these goods in different countries, one can calculate the purchasing power of the national currency vis- à-vis a standard international currency. However, one cannot do the same for education and difficulties arise for education in determining both volume and price measures. Fraumeni et al. (2008) state, "quality-adjustments continue to be the most challenging aspect of decomposing nominal expenditures for government-provided education into price and quantity components" (p1). That is, education quality differs greatly within countries and among countries. This will be discussed later as we review literature surrounding school effectiveness and is the crux of this document - how to account for differing levels of educational quality in order to establish a national aggregate standard price or value for purchasing power parity. 1 The law of one price assumes that if price differences between goods in different markets exist, arbitrageurs will buy low and sell high until price levels reach equilibrium. 2 Arneberg and Bowitz (2006) find this is important in estimating educational inputs in countries with higher and lower levels of development. 4 The ICP uses three sets of indices to create comparable real expenditures. These are indices of real expenditures at the level of GDP, real expenditure per capita and price level indices. The first two (volume indices) are used to corroborate data and together with the price level indices are used to calculate a measure of price level differences. The Laspeyres, Paasche, Marshal-Edwards and Fisher formulas are four competitive index formulas for establishing overall measurement of real prices. Because we are focused on methodological improvements for measuring and comparing education productivity, this paper will not comment on the use or technical specifics of these3. Also, due to complexities and numerous calculation and estimation processes, this paper will not review all issues dealing with PPP methodologies except those that are relevant to our discussion of education. Barro (1997) provides a good overview of some technical problems associated with the PPP comparison project. 2.3 Non-Market Services The OECD describes non-market services as those services provided to communities or individuals either free of charge or at 50% price reduction (OECD, 2010). Measuring non-market services like education continues to offer challenges and Dean (2002) argues that we should be "exceptionally modest" in claims to be able to compare non-market services internationally. Both health and education have been singled out for improved measurement because of their 'market share' of national production. For example statistics show that education spending in G8 countries accounts for between 4 and 7 % of GDP (OECD, 2006) while representing on average nearly 16% of government spending (UNDP, 2009). Despite representing a large portion of government spending (along with the health sector the "principle market services purchased by government" ICP, 2010) - challenges still exist in measuring the productive contribution of education to overall GDP (OECD, 2007). Dean (2002) presents nine separate approaches to the estimation of non-market services. The nine approaches are: 1. Direct collection of price data for detailed services: “direct pricing of outputs”. 2. Direct collection of data on outputs of the non-market service sectors: “direct output measurement”. 3. Adoption of price parities for market services as the price parities for non-market sector services: the “borrowed price parities approach”. 4. The approach to indirect price estimation described in Kravis, Heston and Summers (1982): the “KHS 1982” method. 5. An approach described in OECD (1998) and used for Group I countries in the 1996 European Comparison Programme: the “ECP Group I” method. 6. An approach described in OECD (1998) and Sergueev (1998) and used for Group II countries in 1993 and several earlier ECP rounds: the “ECP Group II” method. 3 For information on indices, methodological issues, and calculations, see the Eurostat Manual (OECD, 2006). 5 7. Estimation of output ratios by adjusting the ratio of labor inputs in a sector by a labor productivity ratio taken from outside that sector: the “labor productivity indicator” approach. 8. Estimation of output ratios by weighting labor inputs with labor compensation weights: the “compensation weights approach”. 9. Estimation of output ratios by using labor inputs and coefficients from wage equations: the “wage equation approach”. (p29) As one can see, these outline three approaches, an input approach, an output approach and a mixed-approach (weight estimation). Up to now, the World Bank has been using what it calls the input- price approach for estimating PPPs for non-market services. Input approaches to calculating educational PPPs have used expenditure data (inputs) to estimate outputs (OECD, 2007). Investments and expenditure were also seen as a proxy for educational quality and price (OECD, 2007). This approach is unsound because input approaches ignore productivity gains and improvements in services (Stiglitz, Sen & Fitoussi, 2009). As the OECD (2007) notes for education, due to input approaches, productivity gain had not been measured. Most recently, the ICP and others have suggested the use of a mixed approach based on output estimates referred to as the input-price approach (ICP, 2010). Although Dean disregards several approaches, he suggests that a major concern for approach choice will be data availability. This will also be one of our concerns as we move forward. 2.4 National Accounts and Government Services Input and output approaches to measuring PPPs are related to how we calculate national accounts. Less than two decades ago, the system of national accounts (System of National Accounts - SNA 1993) was developed by the United Nations which is currently used to measure economic activity internationally. Prior to this the SNA 1968 was used in which the productivity of government services generally and education specifically were estimated through costing exercises. The SNA 1993 recommended distinguishing between inputs and outputs of productivity for non-market services, but did not provide details on how to do this. In order for input approaches to accurately reflect outputs in PPP processes, the ratios of inputs to outputs across countries must be the same. In the past, education has been measured through pricing estimates (Sergueev, 1998), but clearly for education ratios of inputs to outputs differ greatly by country (Eurostat, 2001). Thus we cannot use the ratio of one country as identical to that of another. We should, however be able to calculate the ratio of expenditures to outcomes. Another problem may arise in distinguishing between private and public expenditures. In this case, Nordhaus (2004) argues for "augmented" accounting of both market and non-market economic activity. In the Handbook on Price and Volume Measures in National Accounts developed by the European Commission (EC) (Eurostat, 2001) it is suggested that the lack of prices for non-market services may be overcome with either deflating inputs or direct volume measurement (p31). According to the handbook, a variety of measures could be used to estimate volume including inputs, activities, outputs and outcomes (Eurostat, 2001). Because these services are non-market and no prices exist, the 6 value of volumes may be determined through quality proxies. Three suggestions are made: to directly measure the quality of output, to measure the quality of inputs, or to use outcomes. One issue that we hope to clarify in this document on educational productivity is to be more precise about what entails educational quality. Essentially, this handbook reinforces the general understanding of the difficulty in measuring non-market services like education and presents three broad possible approaches to improving on previous input-price approaches. The EC goes on to suggest various methods for calculating volume and price levels which must meet two criteria: complete or near complete coverage and stratification by educational level or category (Eurostat, 2001:116). Additionally, it suggests following Nordhaus's (2004) call for accounting for market and non-market services separately. From the accounting perspective then, educational productivity must first combine public and private productivity measures at each level/stage of education and then roll up to national for use in various PPP calculations. 2.5 Summary To summarize, education is a non-market (or near market) service which are notoriously difficult to measure. Input approaches tend to miss increases in productivity, while output approaches lack conceptual clarity. Education differs so greatly in quality (not a unique item such as a Big Mac or IPod), that quality adjustment appears to be the primary hurdle in allowing one to use education as a basket item for establishing PPPs. Further, education expenditure has both private and public expenditure components. As we will see, the benefits to educational expenditure may not be measured strictly by educational outputs (often viewed by examination scores) but perhaps part of its contribution to labor productivity. 3.0 PPPs & ICP The International Comparison Program has been in existence since 1968 when it was part of the United Nations Statistical Division. The development of the ICP coincides with the desire to create better and more standardized means of aggregating economic production globally (Pant, 2004). Every several years (most recently in 1993 and again in 2005), the ICP collects data and uses PPP to better compare economic development and various economic development indicators across nations. Issues with health and education have been acknowledged as difficult to measure for the past 40 years (Kravis & Lipsey, 1991). Yet, despite recommendations by Hill (1975) and others (Barro, 1997), little progress has been made in refining the input-price approach. In the most recent round, the input- price approach was used to measure educational productivity resulting in what may be considered overly high degrees of variation in price levels (World Bank, 2008). 3.1 2005 ICP The latest round of PPP calibration took place in 2005 with 146 participating economies. An input-price approach was used to calculate education productivity, but as noted above, while the 7 education expenditure per capita across counties had little variation, the variation in educational prices levels was high (World Bank, 2008). Education (and health) also showed the greatest differences between nominal and real expenditure (150%) (ibid). This has prompted a renewed focus on developing refined methods for measuring educations contribution to GDP. 3.2 ICP development since 2005 Nearly two years have passed since the results of the 2005 ICP process have been published and a good deal of refinement is currently underway within the ICP process generally. For example, the 'ring' method of regional ICP data collection is being replaced by a 'core list' approach. Health and education methodological refinements are key as are improvements in survey frameworks and instruments. Rather than using an input-approach which may have difficulty capturing productivity and service improvements which displaying wide variation between country price levels, an mixed-approach using outputs (test scores) as a proxy for quality is advocated. Several scholars and practitioners have focused on the issue of refining educational methodologies for the calculation of PPPs (Arneberg & Bowitz, 2006; Barro, 1997; Eurostat, 2001; Fraumeni, et. al., 2008; Gallais, 2006; Murray, 2007; OECD, 2007; Schreyer and Lequiller, 2007). In a draft of a section of the handbook Gallais (2006) describes the need remain consistent in approaching market and non-market education measurement if services are similar, to ensure appropriate stratification of education data, the consideration of alternatives to output indicators and to take care in differing weighting and statistical analyses. In looking at how spatial and temporal deflators (country comparisons and inflation) are calculated for PPPs in education, Arneberg and Bowitz (2006) find that "estimates of the trend in real spending on education are highly vulnerable to the deflator being used" (p43). In an attempt to construct a model for capturing educational productivity in primary and secondary education, Fraumeni, et. al., (2008) also point toward the importance of different approaches and methodologies which will lead to different measurements of education. In particular, they note that price changes tend to be higher than quantity changes over time and that measuring quality change over time is difficult. They caution that quality adjustments should be made to volume measurements. What these sources (and many others) tell us is that there remains much variety over how to approach education and non-market service calculations - much of which depends on data availability and data quality (Gallais, 2006). Even though an output approach or a mixed approach may have several different methods which require different types of information to calculate (Dean, 2002), all approaches have at the core a requirement to calculate or estimate volume and price measures. In other words a good process to refinement is to identify proposals, and then compare and contrast their promise and viability. For the rest of this section, I will briefly examine suggestions on how to better account for educational volume and price. 3.2.1 Proposals to Improve Volume Measures A review of proposed refinements in quantitative measures reveals that there is some trend toward refining quantitative measures of education. 8  # of pupil hours (or pupils) (Konijn & Gallais, 2006)  Pupil hours - teaching received (Schreyer, 2009)  Pupil hours adjusted for quality (Eurostat, 2001)  # of pupils/# of pupil hours differentiated by level of education (OECD, 2007)  # days of learning opportunity (Schuh Moore, et al., 2010)  hours of pupil attendance (Lequiller, 2006)  Pupil hours of instruction (Hill, 1975)  Student years of education (Fraumeni, et al, 2008)  Real earnings growth (Atkinson, 2005)  # of pupils (Gallais, 2006) It is widely acknowledged that many educational indicators do not reflect educational productivity. For example, although both gross and net enrollment figures can tell us something about how many children may be enrolled, it does not tell us how often children come to school or how often students attend (Atkinson, 2005; Hill, 1975, OECD, 2007). Thus we cannot be certain how much education students have received. On the other hand, Gallais (2006) argues that # of pupils or enrollment figures offers a better opportunity to qualify education based strictly on a simple equation of (number of pupils) * (the change in test scores). This would avoid double counting of negative influences on achievement scores. We want to look more closely at Fraumeni et. al. (2008) aggregation of pupil hours later in the Indicators Section. Essentially, we need to match quantitative measures (# of pupil hours/ pupil years) with qualitative measures of test scores. That is, we need to ensure that these are the same populations and at least for the PISA, this may not necessarily be the case. Of these suggestions above, only Schuh Moore, et. al., (2010) offer some insight into how we might calculate a # of pupil hour or days of school open for learning (if we pursue this as an option) and perhaps more importantly, how we might collect the data. Their research of over 100 schools in four countries calculated the number of days that teachers and students were in classrooms. Their findings indicate that more than half of the school year was lost due to school closures, teacher absences, student absences, late starts, prolonged breaks and other reasons (Schuh Moore, et. al., 2010). The findings support other research in the field and common sense that suggests that time spent in the classroom impacts learning (e.g Abadzi, 2007; Woessmann, 2005). Some literature describes this as the "education boundary" (Hill, 1975; Schreyer and Lequiller, 2007) - the productive exchange between teacher and student that is at the core of education service production. Schuh Moore, et. al., (2010) use several instruments to collect their data including " Concepts about Print (CAPs); Early Grade Reading Assessments (EGRA); Stallings Classroom observation protocols; school observations; and interviews with teachers and principals" (p1). This data collection process is time consuming, but there is no reason why education management information systems (EMISs) or inspection systems could not collect some of this information going forward. In previous approaches, time spent teaching may have been seen as an input and teacher's salaries were an important factor for calculating educational production. Expenditures were used for both volume and to adjust for price. However, here teaching is the key service being exchanged and 9 pupils presumable gain greater cognitive capacity. The amount of time spent 'actually' teaching is being used as an output measure for individual learning and thus in the next section will describe how to qualify prices. In calculating volume output measures, we must be careful that we capture not only the total pupil hours for each child who reaches the end of an educational stage (primary, secondary, tertiary, etc.), but we must also account for drop-outs and repetition - if we are using pupil enrollments as a means to calculate pupil contact hours. In other words, if we calculate volume measures for education, we should take care in the aggregation process. OECD (2007) offers the most likely stratified collection suggestions by noting the importance of collecting pupil hours by level of education or grade. If we assume that teaching quality is relatively similar throughout an educational system given teacher certification programs, then we may wish to explore the value of differing levels of education as well. Literature from both cognitive psychology and educational economics may shed light on the amount of learning that takes place at certain ages (and hence the value of human capital accumulation for the individual) and the returns to additional years of education as has been argued for using future earnings measures. Essentially, due to drop-outs and repetition, we need to take care in measuring total pupil hours directly and ensure that this qualifies either an average year of student education or a yearly average rolled up to the entire educational system. The problem with an average is that this is often so low as to exclude higher levels of education and quality/value assumptions at that level. This will be important in matching volume with price (in the next section) as we know that higher levels of education have greater value to individuals and governments. 3.2.2 Suggestions to Improve Quality Measures Suggestions to improve price measures through quality proxies are far more diverse than those to improve volume measures. Suggestions include:  Quality adjustments based on PISA (Konijn & Gallais, 2006)  Quality adjustments based on PISA corrected for SES (Schreyer, 2009)  Quality adjusted PISA scores (OECD, 2007)  Use school inspection data (Lequiller, 2005)  School inspections (Eurostat, 2001)  Lesson quality based on inspector reports (Pritchard, 2002)  Quality adjustments based on teacher-pupil ratio (Hill, 1975)  Quality adjustments based on incremental earnings (Fraumeni et. al, 2008)  Quality adjustment based on local housing costs/taxes (Fraumeni, et al, 2008 referring to Black, 1998)  High school drop-out rate (Fraumeni, et al, 2008)  College enrollment (Fraumeni, etal, 2008)  Pupil attainment (Atkinson, 2005)  Pupil progress (Atkinson, 2005)  Future real earnings (for tertiary students) (OECD, 2007)  Expected future earnings (Murray, 2007)  Changes in examination scores (Eurostat, 2001) 10 We can place these suggestions into four categories: those that want to use examinations (the PISA) as a proxy for education quality, those that want to use actual quality data from inspections, those that want to use alternative output data such as drop-out rates and those that advocate for using outcome data such as future real earnings. Within each of these suggestions is an implied approach. The first and most common suggestion is to use international test scores to qualify teaching through student test scores and thus the value of education which could then be priced. This output approach must be addressed as the potentially next step from input-approaches of using expenditures as a proxy for outputs/prices. Eurostat (2001) sums up the procedure of estimating productivity changes based on changes in the proxy of examination scores. However, there are several more output-oriented suggestions and one outcome-oriented suggestion (in this list above). The outcome-oriented suggestions focus on using future earnings as a proxy for the level of human capital developed during one’s educational experience. Although there is considerable research in to effects of education on economic development (Hanushek, 1996; Woessman, 2007), there is less conclusive comparative work on the relationship of education to employment. It is not clear whether education acts as a screening device, whether the credentials signal some level of capacity, whether human capital accumulation may also produce job acquisition skills or some combination of these and other factors. For example, in one comparative study in East Africa, Knight and Sabot (1990) find that personal contacts are more likely to have a positive employment result than educational credentials. All of these suggestions will move us away from input approaches, but taken together these quality measure improvements beg the question - how do we measure educational quality generally? That question in turn requires that we move away from more economically oriented literature and examine the literature surrounding school effectiveness and educational quality. This will provide us with a stronger understanding of what actually constitutes educational quality and thus educational value. These suggestions also beg two additional questions: why do we need quality measures and what do these do for us? Both of these will be examined in the next section on education and education systems. The use of examinations (PISA) as a qualifier for prices matched with pupil hours as a measure of volume represents a selection bias. Keeves (2000) refers to a sampling selection bias for international exams that do not account for children who do not reach the age for testing. Fuller (1987) confirms that some correction for sampling bias may be necessary. In this case, both drop-out rates and repetition rates will impact the volume measure of output approaches and correction measures could be included. It is a distribution issue and we need to know more about the characteristics of the student population who take the PISA and those who do not take the PISA (or any national/international examination). 3.3 Summary Up to 2005, education PPPs were constructed using input approaches where expenditures were used as a proxy for production outputs. For 2011, the ICP has continued to recommend the use of an input-price approach. At the same time it is proposing the development of an output approach. In evaluating the proposals for new volume and price/value proxy measures, several problems are noted. 11 Regarding the suggestion to revise volume measure with # of pupil hours, there is a great deal of research on time spent in classrooms, but little comprehensive data. Regarding the suggestions to revise price and value measures with international test scores, the ICP coverage may be lacking. PISA examinations are not offered in all the ICP countries. We will need to either impute PISA scores as some have done (Crouch and Fasih, 2004; Mingat, et. al., 2004), utilize other international examinations data such as TIMSS or PIRLS (see Annex B for international examination country coverage), or perhaps use other commonly used international examinations such as the SAT or ACT, the International Baccalaureate Higher Level Exam or perhaps the Cambridge Examinations. Other options include imputing missing test scores with an understanding of the relationship between quality indicators and outcomes in similar countries. 4.0 Education & Educational Systems Early on we asked why it is so difficult to capture volume and price components of education as a non-market service. Why is education comparison resistant? Part of this question is organizational - the way that public education is structured, delivered, planned and another part is the nature of education in creating human capital. Education is a long term complex process that may not exhibit outward accumulation until applied to some activity or evaluated at some point. 4.1 Overview of Education Systems Educational systems are complex bureaucratic organizations which serve many functions. Investments in education are made at both public and private levels not only for some return, but also for intrinsic value - for the enjoyment of learning. Investments in educational services such as art or music may not have a strategic monetary goal. The value of educational services and outcomes has to do in part with non-monetary individual or system-wide preferences which thus makes pricing of these services quite difficult without an understanding of some preference orientation or ranking. Formal educational systems4 have many organizational components: curriculum, instruction, physical premises, examinations, human resources and teachers, etc. Together these components or inputs, when interaction takes place between teacher and student, create some output (often referred to as human capital) which is most frequently measured though some form of assessment or test. Standardized state, national or international tests are often designed to capture cognitive development, but education may also impact affective and behavioral outcomes (which may also impact labor 4 When we think of education, we often think of formal public education, but there are many other forms of education and ways to categorize educational activities (e.g. general public education, general high cost/low cost private education, special education, non-formal education, information education, on-the-job training, etc.). Many, including Hill (1975) acknowledge the multiple dimensions of education productivity. For the most part, this paper is discussing ways of refining public or non-market educational services because private and shadow (out of school tutoring) educational services could estimate both volume and price and because we believe that we should move toward education PPP methodological refinements in small measured stages. 12 productivity). For the most part, we will leave non-cognitive outputs aside as beyond the immediate scope of this paper. In assessing which factors may lead to better cognitive outputs, an explicit or implicit education production function is commonly used. Quite simply this assumes that various inputs, when acted upon in schools though a teaching/learning process, create cognitive outputs - a simple input-output or input- process-output model. Education production function research is ubiquitous (e.g. Case and Deaton, 1999; Hanushek, 1979; Krueger, 1999; Psacharopoulos and Patrinos, 2004) in part because there are numerous applications to understanding which factors or variables impact educational outcomes. In their introduction the handbook on school effectiveness, Reynolds, et. al. (2000) note how various paradigms have contributed to refinements in the development of the production function as it applies to what is now termed "school effectiveness". Chapman, et. al., (2005) also note how educational production function research has multiple uses by multiple actors. A major function has been to determine which variables significantly impact educational outcomes in order to make policy decisions about where to invest resources or alleviate systemic problems (e.g. Pritchett and Filmer, 1999). This does not mean that production function methodology does not also require careful consideration (Klein, 2007), but production function research is mentioned here to point out both that a methodology for analyzing quality variable impact on educational outcomes already exists and that a number of factors or variables that influence educational outcomes have already been studied. The results of these studies when examined together may help us better evaluate which quality measures may be identified for corroborative efforts in PPPs methodological refinement. 4.2 School Effectiveness, School Quality, and Education Production If we assume that we will be moving forward with the proposal advocating the use of PISA scores as a proxy for quality, we need to better understand and evaluate how that proxy functions. Stepping back, this section should start with a brief overview of out-of-school factors which also impact educational outcomes. For several decades we have known that socio-economic status, community variables, family make-up and other issues have a strong influence on educational outcomes as measured by examination scores. Two seminal works by James Coleman and others (Coleman, et. al, 1966) and Christopher Jencks and others (Jencks, et.al., 1972) found that in the US, SES impacts student achievement. Many international studies have confirmed these findings (e.g. Heyneman and Loxley, 1983) and it is widely recognized that educational production is not only influenced by school factors. This recognition is problematic for isolating the contribution of providers for educational services (Fraumeni, et al., 2008). The ability to remove non-school factors from output measures of education have not yet been developed (ibid), but some research has developed methods for controlling for family background (Case & Deaton, 1999; Lee & Barro, 2001). The PISA currently utilizes the Economic, Social and Cultural Status (ESCS) index to control for non-school factors. This does not imply that schools have no influence over student outcomes and the majority of school effectiveness research focuses on the relationship of school factors to outcomes (Rivkin, 13 Hanushek & Kain, 2005). Just like quality, school effectiveness is a relative term. Schools (and school systems) are relatively more or less effective (Teddlie & Reynolds, 2000) compared to each other and over time. School effectiveness must define the desired outputs, indicators, etc, in order to remain comparative in nature (Scheerens, 2000). As part of defining objectives, desired outputs and indicators to measure effectiveness, quality becomes an essential dimension of effectiveness. School quality and the plans and operations for quality attainment differ widely across countries. Research indicates that educational outcomes are affected by organizational and political dimensions. Research into the effects of other administrative structures such as decentralized systems (Bray, 1994) suggest that differences in how educational systems are structured and managed impacts school performance. In a recent study using PISA data, Fuchs and Woessman (2004) found that institutions account for roughly 1/4 of student achievement variation. Woessman (2003) found that differences in TIMSS scores between countries were related to institutional factors such as examination systems or school autonomy. A commonly mentioned study of institutional factors in the US is Chubb and Moe (1990) who argue that schools with more autonomy perform better. If there is a sufficient level of devolution, we cannot assume that educational quality is equal between administrative systems. Atkinson (2005) suggests that this is important in determining if the output results for England can be taken to represent the entire UK. So prior to even investigating school quality issues, we must address other factors that impact educational success as measured by examinations. This has implication for how we calculate national education PPPs and the ICP currently has a working group to propose specific recommendations. Educational quality variables are in one sense intuitive, but research in the area of school effectiveness or school quality may be expressed in various types of research. What we have looked for in the literature is any and all possible measures that may be explored to improve our understanding of educational quality as it impacts price or value. Educational quality variables include but are not limited to: expenditure (Chapman, et. al. 2005), class size (Krueger, 1999) teacher characteristics (Rivkin, Hanushek & Kain, 2005), capital investments (OECD, 2010), learning materials (Heyneman, et. al., 1981), and others (Mayer, et. al., 2000). Several reviews of studies confirm relationships between educational quality variables and outcomes (Pennycuick, 1993; Scheerens, 2004; Teddlie and Reynolds, 2000; Yu, 2007). Quality variables are sometimes characterized as inputs. Quality Variable Studies Expenditures Diawara, 2009; Fuller and Clark, 1994; Hanushek, 1989; Hedges, et. al. 1994 Institutional Structures Chubb and Moe, 1990, Fitz-Gibbons & Kochan, 2000; Woessman, 2003 Teacher Characteristics DeJaeghere, et. al., 2006, Glewwe and Kramer, 2006; Lee and Barro, 1999; Rivkin, Hanushek & Kain, 2005 Learning Materials Fuller, 1986; Heynemann et. al., 1981; Schuh Moore et. al., 2010 Class size Akerheilm, 1995; Glass & Smith, 1979; Kreuger, 1999; Hanushek, 1999; Mosteller, 1995 14 Educational Content Chapman, et. al., 2005; Ferguson and Womack, 1993; UNESCO, 2005 Management Bossier, 2004; Chapman, 1998; Fuller and Clark, 1994; Pennycuick, 1993 This table does not capture all educational studies that identify and utilize quality variables, but rather is designed to identify some of the variables used for examining the quality of education. Each study is unique in its identification and definition of variables, research design, methodology, and conclusions. Despite what appears to be conclusive evidence in some studies, others find little or no significant effect of these variables on test scores. The direction and order of effects may vary as well from study to study. For example, research indicates that class-size is a quality indictor that significantly impacts educational achievement (e.g. Glass & Smith, 1979). In a meta-analysis, Hanushek (1999), however, concludes that class size demonstrates a weak correlation to achievement. As a related but perhaps separate construct, pupil-teacher ratios are frequently used as quality indicators in research and international education data collection systems. In another example, one would assume that greater levels of educational expenditure lead to higher educational functional productivity and greater educational outputs in the form of higher test scores. However, as Postlethwaite (2004) notes, there is a diminishing rate of return for educational investment. At a certain point, additional funding does not lead to greater outcomes. Additionally, research drawing on both PISA and TIMSS data indicates that countries with higher educational expenditure do not perform better in cross-country comparisons (Woessmann, 2007). Inevitably, Scheerens (2000) notes, some research will be conducted which results in inconclusive findings. Even Hanushek (1999) notes that just because the data does not currently demonstrate that a certain variable exhibits significant influence on educational achievement, does not mean that it has no impact on educational performance or that policy decisions (such as increasing class size) should result from any particular research study. Creemers and Reynolds (1996) note that each of the many variables influencing educational achievement account for only a small portion of variance and errors in approach, methodology, measurement or analysis may not find expected results. Thus, we need not get overly bogged down in debates over educational quality variables unless it is decided to use specific variables to corroborate test score proxies. It is important to note however, that multiple factors influence the production of education and research to date has not found a single indicator or index for educational quality. For example, the Opportunity to Learn index uses a set of 12 indicators to assess students learning, but cautions that each should be seen as separate indicators (Schuh Moore, et. al., 2010) Still, most quality variables can be converted to indicators. For example, Chapman, et. al., (2005) use the public education expenditure as a % of GDP to represent one facet of educational quality. Similarly, UNESCO uses % of trained teachers as an indicator of teacher quality. Because the PPP process requires operationalization of variables, we will review indicators in the next section. 15 As a final note to this section, education production functions are typically set up as the influence of educational variable/factors on test scores. However, there is another body of research that examines the rates of return to investments in education to economic productivity (Hanuskek and Kimko, 2000; Psacharopoulos and Patrinos, 2004). Although the focus may still be on various school effectiveness or quality variables, rates of return analysis may actually get us closer to our interest in education as an intermediate service/good and its impact on economic productivity in the labor market - though Dean (2002) argues that we would want to use earnings coefficients rather than rates of return analysis. It still requires that we understand which quality variables impact the production of educational services, but only in so far as they impact overall economic productivity. This conceptual distinction is one of the many reasons that education and non-government services have been so difficult to accurately measure. Education is an open system with dynamic feedback loops (Fraumeni, et. al., 2008) that reinforce educational attainment/achievement and potential levels of human capital accumulation. 4.3 Summary Education is a comparison resistant non-market service which differs widely in its quality and by service type. Education is an open system that is influenced by non-production external factors such as student characteristics. Although research is not entirely conclusive, several factors stand out as important to educational quality (though each contributes only a small portion to achievement score variance). These factors can be operationalized to create indicators. As a final note, outcome indicators may represent an opportunity to expand PPP calculations in the future. 5.0 International Educational Indicators and Data Availability When moving toward education PPP refinements, a pragmatic consideration is the availability of data. There are a variety of quantitative and qualitative measures currently in use in international education circles. The collection and storage of these data are done by UNESCO, World Bank, OECD, Eurostat, Education Policy and Data Center (EPDC), international household survey series, (DHS, MICS) and others. 5.1 Indicators All indicators currently used can be found in Annex #1, but here it is important to describe several key indicators that may prove useful in alternative calculations of education PPPs. In terms of quantitative indicators, these include:  Gross Enrollment Numbers (for primary, secondary and tertiary education)  Gross Enrollment Rates (for primary, secondary and tertiary education)  Net Enrollment Numbers (for primary, secondary and tertiary education)  Net Enrollment Rates (for primary, secondary and tertiary education)  Secondary School Graduates 16  Tertiary Graduates  Adult and Youth Literacy Rates  Number of Teachers (Primary, Secondary and Tertiary)  Vocational Education Enrollments  Educational Expenditure  Expenditure on Government Teachers Salaries  Average Number of Hours of Teaching per year Adams (1993) defines nearly 50 different definitions of quality. Despite these various approaches to defining educational quality (noted above) several sets of indicators have been created which are now commonly used to compare educational quality.  Average Years of Schooling  Drop-out Rate  Repetition Rate  Current Educational Expenditure as % of GDP  Current Educational Expenditure as % of Government Expenditure  Current Educational Expenditure on Educational Materials as % of Educational Expenditure  Expected Primary (and Secondary) Completion Rates  Expenditure on Teachers Salaries as % of Educational Expenditure  Percent of Trained Teachers  Pupil to Teacher Ratio  Class Size  Teacher Salaries For the purpose of evaluating data availability for the purpose of constructing alternative PPP calculations, we have embedded a brief matrix of data (Annex #3). As can be seen in the preliminary analyses of this annex, a high percentage of data is now available from a variety of sources. Organizations like the EPDC have both access and technical capacity to quickly compile sets of indicators for weighting PPP quality measures. 5.2 Time on Task, Opportunity to Learn, and Refining Education Volume Measures As noted above, the Opportunity to Learn instrument is comprised of several quantitative and qualitative measures. Major findings in this area are that less than 50% of the school year is actually spent teaching students (Schuh Moore, et. al., 2010). The research was conducted in just over 100 schools across four countries. Research like this conducted by the organizations like the Academy for Educational Development (AED) and the Research Triangle Institute (RTI)5 is time and resource intensive. Direct school observations as were conducted in the Schuh Moore et. al., (2010) study may not be feasible for the ICP. In looking for other possibilities, one indicator may prove useful. The D4 indicator (how much time do teachers spend teaching) from the Education at a Glance program, has been 5 Parts of the OTL index were drawn from research conducted by RTI. 17 collected with data from two separate years (OECD, 2009). Listed as net contact hours, the glossary seems to indicate that this is formal policy time that a teacher is supposed to be in the classroom and thus may not prove useful. The tools designed by AED and RTI may be employed to collect the type of data needed for meeting the proposed quantity measure refinements suggested by the ICP. Although their collection techniques are intensive, we believe that some form of data collection could be developed to ensure reliable and accurate data collection - perhaps by utilizing inspectorate or Education Management Information System (EMIS) services. For example, Chaudhary, et. al., (2006) used direct school observations alongside inspectorate visits to measure teacher absence. 5.3 What is missing? We do not have an overall quality measure for education on the aggregate for each country. Indeed, as noted above this issue continues to be highly controversial. Defining a multi-faceted concept like educational quality let alone measuring and comparing is difficult at best. Still we do have a set of quality indicators that can be divided up into categories which can and have been examined extensively and we plan on exploring how a test score like the PISA will function to capture all quality inputs. As mentioned in the discussions on school effectiveness, because we have to roll up quality indicators to the sub-national and national levels, we need to think about educational measurement aggregation and systems quality. Several issues come to mind that may be useful in defining the relation of inputs to outputs and these have to do with delivery methods. Systems efficiencies (input- output models of expenditures or inputs to test scores or outputs) may differ from system effectiveness and since we are looking at productivity issues and not only inputs. In order to understand educational productivity, some of the quality indicators may have to do with system wide administration or service delivery. USAID has produced a list of some indicators that may be useful in disentangling the input- output process (USAID, 2005). These include:  Number of Trained Administrators  Number of Parent-Teacher Organizations per School  Number of Repaired Classrooms  Number of Textbooks or other Materials Available  Number of External Institutional Links Because we do not have a single indicator to measure educational quality nor indeed a single type of education, we typically utilize a set of quality indicators to create a battery as was done with the OTL index. However, because PPPs create a single factor for educational quality and we must caution that due to the complexity of variables, and differences in correlation within country and between countries, a single quality proxy, even a highly valid and reliable proxy like the PISA test, may not produce the results we envision. 5.4 ISCED and Stratification of Educational Services The International Standard Classification of Education (ISCED) was designed by UNESCO and the current version used is ISCED 1997. The ISCED classification is well-developed, regularly maintained and 18 acts as a standard around which sets of data from multiple levels and programs are organized. In terms of aggregation techniques, we need to decide whether we should use the full set of ISCED categories or look measure educational production as Lequiller (2006) suggests by simpler categories such as pre- primary, primary, secondary, higher education. ISCED offers a unique stratification system, but some note that it does not perfectly reflect all educational systems (Schneider and Müller, 2009). 5.5 Data Availability & New Data Collection Issues We have been asked to review the availability of data and the need for new data collection systems. In considering any data requirements, we must understand data availability and current collection techniques. 5.5.1 Secondary Data Sets As demonstrated in the list of indicators above and through the links in Annex #1, secondary data sets from organizations such as UNESCO, World Bank, the EDPC, ILO, and others may prove useful for some analyses. We anticipate that these sources will be our primary source of data. The data is cleaned, comparable and much easier to obtain than many national sources. 5.5.2 Household Surveys Household surveys can provide data on: student enrollment, student background (SES), graduation rates, public v. private or religious school enrollments, and perhaps information on private household expenditures. National household surveys are common globally and available for almost all ICP countries6. Given the current methodology of ICP for the 2011 PPP round, data from this source will be useful in capturing household expenditure 5.5.3 Education Management Information System (EMIS) EMIS systems can provide information on pupils, teachers, teacher training, number of schools, school facilities, length of school year, days schools are open, etc. Many countries have functional EMISs that collect, compile, analyze and disseminate data for educational purposes. EMIS often compile various sets of data to create both quantity and quality indicator reports. Some research however, shows that EMISs do not always deliver on the promise of appropriate, accurate and timely data (Chapman and Mählck, 1993). Each country will differ in its capacity to collect, compile, process and deliver data. 5.5.4 School Inspection Data School inspection data provide us with the opportunity to gather direct qualitative data on the educational system. School Inspection data differs by country educational system. There is no standard set of activities, measures, collection and compilation protocols or quality control. Research shows that in many countries school inspections are limited to a narrow set of activities, that inspections are focused on monitoring those activities instead of providing technical advice, or in some cases that inspections do not occur at all (Kemmerer, 1993). Some educational systems do not have formal inspection units but rely on alternative means of monitoring and providing technical assistance. 6 It is not clear whether ICP can access household surveys or whether ICP has funds to extract the data from a large number of them. 19 However, if it is possible to collect in a planned and well managed fashion, school inspection systems may offer a source of data for evaluating educational quality. 5.4.5 Census data Census data can corroborate household surveys in finding level of education for the general public. However, this would not be the primary source of data. 5.5.6 Ad-hoc Surveys Ad-hoc surveys can fill in the gaps of data not found in other sources such as time on task, subject instructional time, or facilities quality estimates (if these are needed). Given the requirement to refine both quantitative and qualitative in the short term, we may need to develop ad-hoc surveys for pilot countries. Utilizing those countries where AED is already working makes the most sense - especially if those countries have already used the OTL index and those indicators will be employed. 5.6 Summary Much if not all the data to match the indicators is already currently available. In the next phase we will identify which indicators to use in piloting new methods for calculating education PPPs based on available data. We may wish to run concurrent methods to test which works best with particular sets of data. Given the available data, we may also want to compare data sets from different sources. The two main issues noted above will be the availability of # pupil/hours data (which incorporates missing classroom opportunity) and international assessment availability which we will look at in the next section. 6.0 Comparative Educational Assessment "In the 2011 ICP the OECD will make estimates of educational output based on students at various educational levels with some quality adjustment based on standardized tests" (Deaton and Heston, 2009:39). This section is designed to explore the viability of using standardized test scores as a possible quality adjustment for educational measurements and as such will review currently used international assessments and issues dealing with their use as a quality adjustment mechanism. The International Association for the Evaluation of Education Achievement (IEA) has been studying international education outcomes since the 1950s. Over the past several decades, the IEA studies have examined mathematics, reading, science, computer literacy, civics education and more. The IEA studies are well respected, continue to develop and revisit important educational issues in their assessment programs, and form the basis for most international assessments today. Several international examinations are currently being administered. 6.1 Progress in International Reading and Literacy Study - PIRLS PIRLS tested 4th grade students for reading and literacy skills in 2006. In 2006, 38 countries participated in the PIRLS and the next round is scheduled for 2011. Currently housed at Boston College, the PIRLS assesses a range of reading processes. The results ranked countries and provided interesting analyses of school curriculum, student's affective and behavioral data, home activities, and school contexts (Mullis, et al., 2006). Also, the PIRLS utilized a questionnaire designed to capture some 20 background information about students home and school experiences. The PIRLS studies continue to build on previous experience and the next round in 2011 may collect usable information not only dealing with reading and literacy, but with school, family and student background. 6.2 Program for International Student Assessment - PISA PISA tests science, math and reading skills of 15-year olds and was last given in 2009. Organized and managed by the OECD, the PISA is designed to capture and compare cognitive development across countries. The PISA goes beyond typical input-output studies to relate cognitive skills to economic growth. Findings from previous PISA studies indicate that small improvements in a nations labor force skills can have "very large impacts on future well-being" (OECD, 2010). The PISA has a diversity of academic content often not found in other international assessments and includes the collection of student and family demographic data which allows for the control of non-school factors during analysis. As such, the PISA offers an excellent opportunity to help qualify education PPPs. Problematic is the difference between the 65 countries undertaking the PISA in 2009 and the 181 participating countries listed on the ICP website. Another problem is that the PISA is given only to 15 year olds (average) from selected schools within countries. This sample does not account for the loss of productivity due to drop- out or repetition. As noted above the PISA also utilizes the ESCS index to control for non-school factors influencing achievement. 6.4 TIMSS In the most recent (4th) round of TIMSS, 4th and 8th grade students were assessed in mathematics skills. Administered by IEA, 48 countries participated in the 2007 TIMSS. Mullis et. al., (2008) found association between test results and parental level of education, the language of the test frequently found at home, positive attitudes toward math and computer using students performed better. For the purpose of this paper, it is perhaps more important to note that research has been conducted using the TIMSS to compare achievement across assessments (US Department of Education, 2004), to estimate human capital quality (Altinok and Murseli, 2006), and even as a quality proxy for learning (Crouch and Fasih, 2004). Moreover, Crouch and Fasih (ibid) take advantage of an 'overlapping' method to calculate test scores for countries missing data. 6.5 National Assessments Most national exams may not be comparable except under Bologna process expansion. In Dept of Education (2004), major challenges are found in comparing PISA and TIMSS to the US national test - NAEP. Others suggest that the challenges of using national exams may be preferred because "while PISA may provide a possibility to follow student scores over time, national exams are generally to be preferred over international comparisons, in particular when they systematically control for socio- economic variables and for the educational level at the entry into the school system" (OECD, 2007:53). Atkinson (2005) also advocates using changes in national exams for individual country (England) to determine important potential changes in productivity. 6.6 Assessment Issues 21 Both PIRLS and TIMSS test based on grade level where age differs between countries. These two assessments also test for reduced curricular content - reading and math. The PISA assessment tests children of the same age and covers three subjects reflecting more curricular diversity. In a comparison between TIMSS and PISA (and NAEP), the US Dept of Education notes important differences between content student population, sampling and comparability with national examinations (Dept. of Education, 2004). Still, PISA offers the most diverse content for capturing possible economic productivity skills. Using international examination scores as a proxy for educational quality holds some promise. Because these examinations have already been tested and held to be valid and reliable, development issues will not be problematic. However, the PISA is not administered in all ICP countries (see Annex 2). The PISA is offered in offered in just over 1/3 of the ICP countries. This will require some creative estimations if one wants to use the PISA as a proxy for quality. One option to capture PISA scores is to impute test scores based on other international assessments as suggested by Crouch and Fasih (2004). Another method would be to construct test scores from quality variables, using similar quality and quantity variables. The imputation methods used by Crouch and Fasih (2004) and similar methods used by others (Hanushek and Kimko, 2000; Mingat, et. al., 2004; Woessman, 2005) offer an enticing opportunity that should be further explored and developed. Another novel approach may be to use alternative tests that are more widely administered internationally, though to a reduced population. It may be possible to use ACT/SATs, International Baccalaureate Higher Exams or Cambridge Exams as a quality proxy. We acknowledge that there is a bias among those who take these exams but if we can determine the distribution of scores for these alternative tests, we could then chart a similar distribution against other quality proxy exams such as national exams. A final approach could be to use the correlations found in the PISA between quality indicators, SES, quantity and the actual scores to impute a PISA score in countries with similar levels of development, economic diversity, and demographics. We will need to address the issue of absent PISA scores in many countries if we are to move forward with using the PISA as a quality proxy. 7.0 Insights and Conclusions There are several insights that can be drawn from this paper. First, the literature reviewed for this paper supports the suggestions to move toward output-based approaches to measuring educational production. Input price approaches have not produced the desired results and testing new measures for volume and price holds promise. Second, the suggestions for improving both volume and price/value proxies tend to converge around similar issues and measures. For a volume measure, we find that the # of pupil hours of instruction more closely represents the volume of education services delivered than enrollment figures. 22 We still need to decide how to best define that measure - whether through # of pupil hours of learning throughout the system or rolled up to the year. For a price measure, using examination scores as a proxy for quality/value and thus price offers a sensible solution to the inability to measure prices in non- market services. Alternative output measure proxies like future earning require more research into the influence of credentials on employment acquisition across countries. We are cautiously optimistic that comparisons may prove valuable for ICP purposes if certain steps are taken to minimize problems. Third, several problems must be overcome in order for volume and price measures to be refined. For the volume measure, current intense collection methods for gathering information on actual # of contact hours (e.g. Schuh Moore, et. al., 2010) need to be addressed. Perhaps selective sampling with corroborative data could be developed to estimate sub-national or national figures. For the price measure, there are two major drawbacks. The selection bias must be accounted for by either incorporating dropout/repetition rates into the volume measure or adjusting PISA/test scores to account for retention "yield" (Keeves, 2008). We must also find ways for creating internationally comparable test scores for countries which are not currently administering the PISA. As mentioned above, there are several options including extrapolating a PISA outputs for those countries missing and not offering the PISA, using (normed) national examinations to calculate the PISA score or perhaps a norming process for national exams (see discussion above). Essentially, if we are to use output measures qualified by an assessment proxy, we want a standard proxy or some method for calculating a standard for comparison. We need to ask whether we are finally at the stage in terms of data collection, analysis and availability, along with basic understandings of both the internal education production function and the relationship of education as an intermediate service/good to economic production to apply direct output approaches to the calculation of PPPs for educational production? Currently the answer is no. We are using several levels of proxies and estimating productive capacities. However, the suggestions above may bring us closer to acceptable levels of education price variation. 23 References Abadzi, H. (2007) Instructional time loss and local-level governance. Prospects. 37(1) 13-16. Adams, D. (1993) Defining Educational Quality. Improving Educational Quality Project. USAID: Washington, DC. Akerhielm, K. (1995) Does Class Size matter? Economics of Education Review. 14(3) 229-41. Altinok, N. and H. Murseli. (2006) International Database on Human Capital Quality. IREDU Working Paper. September. Arneberg, M. and E. Bowitz. (2006) Who is the big spender? Price index effects in comparisons of educational expenditures between countries and over time. Peabody Journal of Education. 81(2) 33-46. Asian Development Bank (2008) Research Study on Poverty-Specific Purchasing Power for Selected Countries in Asia and the Pacific. ADB: Manila Atkinson, A. (2005) The Atkinson Review: Final Report. Macmillan: Houndmills Balassa, B. (1964) The Purchasing Power Parity Doctrine: A Reappraisal. Journal of Political Economy. 72 (December). 584–596. Barro, L. (1997) International Education Expenditure Comparability Study: Final Report vol. 1. US Department of Education: Washington, DC. Bray, M. (1994) Centralization/decentralization and privatization/publicization: Conceptual issues and the need for more research. International Journal of Educational Research. 21(8) 817-824. Bullock, D. and N. Minot (2006). On measuring the value of a non-market goods using market data. American Journal of Agricultural Economics. 88 (4) 961-973. Chapman, D., Weidman, J. Cohen, M. and M. Mercer. (2005). The search for quality: A five country study of national strategies to improve educational quality in Central Asia. International Journal of Educational Development. 25 (4) 514-530. Chapman, D. and L. Mählck. (1993). From Data to Action: Information Systems in Educational Planning. UNESCO: Paris Chaudhary, N., Hammer, J., Kramer, M., Muralidharan, K. and F. H. Rogers. (2006) Missing in action: Teacher and health worker absence in developing countries. Journal of Economic Perspectives. 20(1) 91- 116. Chubb, J. and T. Moe. (1990). Politics, Markets and America's Schools. The Brookings Institution: Washington, D.C. 24 Coleman, J. S., Campbell, E. Q., Hobson, C. J., McPartland, J., Mood, A. M., Weinfeld, F. D., et al. (1966). Equality of Educational Opportunity. U.S. Government Printing Office: Washington DC Creemers, B. and D. Reynolds. (1996). Issues and implications of international effectiveness research. International Journal of Educational Research. 25(3) 257-266. Crouch, L. and Fasih (2004) Patterns in Educational Development: Implications for Further Efficiency Analysis. Unpublished manuscript. Davutyan, N and J. Pippenger (1990) Testing purchasing power parity: Some evidence of the effects of transaction costs. Economic Reviews. 9(2) 211-240 Dean, E. (2002) Purchasing Power Parity for nonmarket services. Paper presented at ICP conference March 2002, Washington DC. Deaton, A. and A. Heston (2009). Understanding PPPs and PPP based national accounts. NBER Working Paper # 14499. NBER: Cambridge DeJaeghere, J., Chapman, D. and Mulkeen, A. (2006). Increasing the Supply of Secondary Teachers in Sub-Saharan Africa: A Stakeholder Assessment of Policy Options in Six African Countries. Journal of Education Policy. 21(5) 515-533. Diawara, B. (2009) Can Spending on Education by Donors and National Governments Help Enhance Education Performance in Africa? International Journal of African Studies. (2) 31-46. Eurostat (2001) Handbook on Price and Volume Measures in National Accounts European Commission. European Commission: Brussels. Ferguson, P., & Womack, S .T. (1993). The impact of subject matter and education coursework on teaching performance. Journal of Teacher Education. 44 (1) 55-63. Fitz-Gibbons, C. and S. Kochan. (2000) School effectiveness and education indicators. In C. Teddlie and D. Reynolds (eds). The International Handbook of School Effectiveness Research. Falmer Press: London. Fraumeni, B., Reinsdorf, M., Robinson, B., and M. Williams. (2008) Price and Real Output Measures for the Education Function of Government: Exploratory Estimates for Primary and Secondary Education. NBER Working Paper #14099. NBER: Cambridge. Fuller, B. (1986). Raising school quality in developing countries: what investment boosts learning? World Bank Discussion Papers. Washington: World Bank Fuller, B. and P. Clark (1994) Raising school effects while ignoring culture? Review of Educational Research. 64(1) 119-157. Gallais, A. (2006) Consultation Notes on Measuring Educational Output and Quality. unpublished document accessed 4/1/2010 at: www.ons.gov.uk/.../measuring.../response-to-establishing-the- principles-from-alain-gallais.pdf 25 Glass, G. and M. L. Smith. (1979) Meta-Analysis of Research on Class Size and Achievement. Educational Evaluation and Policy Analysis 1(1) 2-16. Glewwe, P. and M. Kremer (2006), Schools, Teachers, and Education Outcomes in Developing Countries. in E. Hanushek, and F. Welch (Eds.), Handbook on the Economics of Education, Amsterdam: Elsvier: 945- 1017. Hanushek, E. (1989) The impact of differential expenditure on school performance. Educational Researcher. 18(4) 45-65. Hanushek, E. (1999) The evidence on class size. In S. Mayer and P. Peterson (eds) Earning and Learning: How Schools Matter. Brookings Institution press: Washington DC. Hanushek, E., and D. Kim (1995) Schooling, Labor Force Quality, and Economic Growth. National Bureau of Economic Research paper 5399. NBER: Cambridge Hanushek, E., and D. Kimko (2000) Schooling, labor force quality and the growth of nations. American Economic Review. 90(5) 1184-1208. Hedges, L., Laine, R., and R. Greenwald (1994) Does Money Matter? A meta-analysis of the effects of differential school inputs on student outcomes. Educational Researcher. 23(3) 5-14. Heyneman, S. Farrell, J. and M. Sepulveda-Stuardo. (1981) Textbooks and Achievement in Developing Countries: What We Know. Journal of Curriculum Studies. 13(3)227-246. Hill, P. (1975) The production boundary and nature of output (with special emphasis on health and education). accessed April 1, 2010 at: http://siteresources.worldbank.org/ICPEXT/Resources/ICP_2011.html ICP (2010) Website Homepage accessed 4/1/2010 at: http://siteresources.worldbank.org/ICPEXT/Resources/ICP_2011.html Keeves, J. (2000) Errors: What are they and how significant are they? International Education Journal. 1(3) 164-180 Klein, C. (2007) Efficiency versus Effectiveness: Interpreting Education Production Studies. Department of Economics and Finance Working Paper April. accessed 4/1/2010 at: http://frank.mtsu.edu/~berc/working/Klein2007b.pdf Konijn, P, and A. Gallais. (2006) Non-market Services: Overview of Eurostat and OECD activities. Powerpoint Presentation for OECD conference in London, October. Accessed April 1 at: www.oecd.org/dataoecd/43/2/37470016.ppt Knight, J. and R. Sabot (1990). Education, Productivity and Inequality: The East African Natural Experiment. World Bank: Washington, DC. 26 Kravis, I. and Lipsey, R. (1991) The International Comparison Program: Current Status and Problems. In P. Hooper and J.D. Richardson (eds) International Economic Transactions: Issues in Measurement and Empirical Research. University of Chicago Press: Chicago. 437-468. Kreuger, A. (1999) Experimental estimates of education production functions. Quarterly Journal of Economics. 114(2) 497-532. Lee, J-W and R. Barro (2001) School Quality in a Cross-Section of Countries. Economica. 68 465-488. Lequiller, F. (2006) Measurement of Non-Market Output. Paper presented as the Fourth meeting of the Advisory Expert Group on National Accounts. February. Frankfurt Mayer, D. J. Mullens, M. Moore, & J. Ralph. (2000) Monitoring School Quality: An indicators report. US Dept of Education. Washington DC Mingat, A., Ramahatra R., and V. Mapto Kengne. (2004) La dynamique des scolarisations au Niger. Evaluation pour un développement durable. World Bank: Washington, DC. Mosteller, F. (1995). The Tennessee study of class size in the early school grades. The Future of Children 5 (2), 113-127 Mullis, I. Martin, M. Gonzales, E. and A. Kennedy. (2008) Mathematics and Science Achievement in the Final Year of Secondary School. Murray, R. (2007) Developing a quality adjusted output measure for the Scottish educational system. OECD: Paris. Nordhaus, W. (2004) Principles of National Accounting for Non-market Accounts. Prepared for CRIW conference. OECD (2006) Eurostat-OECD Methodological Manual on Purchasing Power Parity. OECD: Paris OECD (2007) Toward Measuring Education and Health Volume Output: A Handbook. OECD, Paris OECD (2010) The High Cost of Low Educational Performance. OECD: Paris. Officer, L. (1982) Purchasing Power Parity and Exchange Rates: Theory, Evidence and Relevance. Jai Press. Greenwich Pant, B. (2004) Purchasing Power Parity and the International Comparison Program in a Globalized World. ADB: Manila Pennycuick, D. (1993) School Effectiveness in Developing Countries. DIFD: London. Postlethwaite, T.N. (2004) Monitoring Educational Achievement. UNESCO. Paris Pritchard, Alwyn. 2002. Measuring Productivity Change in the Provision of Public Services. Economic Trends. 582 (May) 20-32. 27 Psacharopoulos, G. and H. Patrinos, (2004) Returns to investment in education: A further update. Education Economics. 12(2) 111-134. Reynolds, D. & C. Teddlie (2000) The future agenda for school effectiveness research. In D. Reynolds & C. Teddlie (eds) The International Handbook of School Effectiveness Research. Falmer Press: London Rivkin, S. Hanushek, E. and J. Kain. (2005) Teachers, schools and economic achievement. Econometrica. 73(2) 417-458 Rogoff, K. (1996). The Purchasing Power Parity Puzzle. Journal of Economic Literature. 34 (5) 647-688. Samuelson, P. (1994) Facets of Balassa-Samuelson Thirty Years Later. Review of International Economics. 2 (3) 201-226 Scheerens, J. (2000) Improving School Effectiveness. UNESCO. Paris Schneider, S. and M. Müller. (2009) Measurement in EU-SILC: Preliminary Evaluation of Measurement Quality. EdQualSoc Working Paper #5. Schreyer. P. (2009) Update on PPP program and new data set for health expenditure. Power point presentation for working party on national accounts held in November and accessed April 1, 2010 at: www.oecd.org/dataoecd/2/2/44043181.ppt Schreyer, P. and F. Lequiller (2007) Measuring Health and Education Volume Output – Draft Chapter 1. OECD: Paris. Schuh-Moore, A., DeStefano, J. and E. Adelman. (2010) Opportunity to Learn as a Measure of School Effectiveness in Ethiopia, Guatemala, Honduras, and Nepal. Equip 2 Working Paper. AED: Washington, DC Sergueev, S. (1998) Comparison of Non-market Services at Crossroads. Paper presented at conference of European statisticians. June. Vienna. Stiglitz, J. Sen, A., and J-P. Fitoussi (2009) Stiglitz, Sen Fitoussi Commission Findings. Republique Francaise: Paris. Taylor, A. and Taylor, M. (2004). The Purchasing Power Parity Debate. Journal of Economic Perspectives. 18 (4) 135-158. UNESCO. (2005) Education for All: The Quality Imperative. UNESCO: Paris. Woessman, L. (2003) Schooling resources, educational institutions and student performance: The international evidence. Oxford Bulletin of Economics and Statistics. 65(2) 117-170. Woessman, L. (2005) Educational production in Europe. Economic Policy. (July) 445-504 28 Woessman, (2007) International evidence on expenditures and class size. Brookings Papers on Educational Policy. World Bank (1993) Purchasing power of currencies: comparing national incomes using ICP data. World Bank: Washington, DC. World Bank (2007) 2005 International Comparison Program Preliminary Results. World Bank, Washington, DC. World Bank (2008) Global Purchasing Power Parities and Real Expenditures. World Bank: Washington, DC Yu, Guoxing (2007) Research Evidence from School Effectiveness in Sub-Saharan Africa. EdQual Working Paper #2 DFID: London 29 Annex 1 - International Education Indicators - Links World Bank - EdStats http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTEDUCATION/EXTDATASTATISTICS/EXTEDSTAT S/0,,menuPK:3232818~pagePK:64168427~piPK:64168435~theSitePK:3232764,00.html OECD - Education at a Glance http://www.oecd.org/document/24/0,3343,en_2649_39263238_43586328_1_1_1_1,00.html UNESCO - UIS http://www.uis.unesco.org/ev.php?ID=2867_201&ID2=DO_TOPIC EU - Eurostat http://epp.eurostat.ec.europa.eu/portal/page/portal/eurostat/home/ US-NCES http://nces.ed.gov/ 30 Annex 2 - Country Participation in International Assessments Country ICP PIRLS PISA TIMSS SACMEC LLECE/ (2011) (2006) (2009) (2007) (2007) SERCE (2007) Albania X X Algeria X X Angola X Antigua-Barbuda X Argentina X X X Armenia X X Australia X X X Austria X X X X Azerbaijan X X Bahamas X Bahrain X X Bangladesh X Barbados X Belarus X Belgium X X* X Belize X Benin X Bhutan X Bolivia X Bosnia and Herzegovina X X Botswana X X X Brazil X X X Brunei-Darussalam X Bulgaria X X X X Burkina Faso X Burundi X Cambodia X Cameroon X Canada X X* X Cape Verde X Central African X Republic Chad X Chile X X X China X Columbia X X X Comoros X Congo, Democratic X Republic 31 Congo, Republic X Coast Rica X X Cote D'Ivoire X Croatia X X Cuba X X Cyprus X X Czech Republic X X X Denmark X X X X Djibouti X Dominica X Dominican Republic X X Dubai (UAE) X Ecuador X X Egypt X X El Salvador X X X England X* X Equatorial Guinea X Eritrea X Estonia X X Ethiopia X Fiji X Finland X X France X X X Gabon X The Gambia X Georgia X X Germany X X X X Ghana X X Greece X X Granada X Guatemala X X Guinea X Guinea-Bissau X Guyana X Haiti X Honduras X Hong Kong -China X X X X Hungary X X X X Iceland X X India X Indonesia X X X X Iran, Islamic Republic of X X Iraq X Ireland X X X Israel X X X X Italy X X X X 32 Jamaica X Japan X X X Jordan X X Kazakhstan X X X Kenya X X Kiribati X Korea, Republic X X X Kuwait X X X Kyrgyz Republic X X Lao PDR X Latvia X X X X Lebanon X X Lesotho X X Liberia X Libya X Liechtenstein X Lithuania X X X X Luxemburg X X X Macao-China X X Macedonia, Republic of X X Madagascar X Malaysia X X Malawi X X Maldives X Mali X Malta X X Mauritania X Mauritius X X Mexico X X X Micronesia X Moldova, Republic of X X Mongolia X X Montserrat X Montenegro, Republic X X of Morocco X X X Mozambique X X Myanmar X Namibia X X The Netherlands X X X X Nepal X New Zealand X X X X Nicaragua X X Niger X Nigeria X Norway X X X X 33 Oman X X Pakistan X Panama X X X Papua X Paraguay X X Peru X X X Philippines X Poland X X X Portugal X X Qatar X X X X Romania X X X X Russian Federation X X X X Rwanda X Samoa X Sao Tome and Principe X Saudi Arabia X X Scotland X* X Senegal X Serbia, Republic of X X X Seychelles X X Sierra Leone X Shanghai-China X Singapore X X X X Slovak Republic X X X X Slovenia X X X Solomon X South Africa X X X Spain X X X Sri Lanka X St. Kitts and Nevis X St. Lucia X St. Vincent and X Grenadines Sudan X Suriname X Swaziland X X Sweden X X X X Switzerland X X Syrian Arab Republic X X Chinese Taipei (Taiwan) X X X X Tajikistan X Tanzania X X** Thailand X X X Timor-Leste X Togo X Tonga X 34 Trinidad & Tobago X X X Tunisia X X X Turkey X X X Uganda X X Ukraine X X United Arab Emirates X (Dubai?) United States X X X X Uruguay X X X United Kingdom X X Vanuatu X Venezuela X Vietnam X West Bank/Gaza X X Yemen Republic X X Zambia X X Zimbabwe X X *PIRLS data may be sub-national **Tanzania is split between mainland and Zanzibar for SAQMEC 35 Annex # 3 - Indicator Availability Quantity Inputs Quality S.E.S. Descriptive Governance Structure # Days/Hours of instruction /year Adult Average Years of Schooling % Instructional time on reading Public expenditure per student Public expenditure as % of GNI % of students with textbooks Facilities quality indicator GER Tertiary (ISCED 5&6) School Life Expectancy Teacher salary level Teacher experience Pupil-Teacher Ratio % Trained teachers Completion rates Repetition rates Parents Income GER Secondary Drop-out rates GER Primary Class-size Albania X X X X X X X X X X Algeria X X X X X X X X Angola X X X X Antigua and X X X X X X Barb. Argentina X X X X X X X X X X Armenia X X X X X X X X X X X Australia X X X X X X X Austria X X X X X X X X X X Azerbaijan X X X X X X X X X X X Bahamas X X X X X Bahrain X X X X X X X X X Bangladesh X X X X X X X X X X Barbados X X X X X Belarus X X X X X X X X X X Belgium X X X X X X X X X X Belize X X X X X X X X X X X X Benin X X X X X X X X X X X X X X Bhutan X X X X X X X X X Bolivia X X X X X X X X X X X Bosnia and X X X X X X X X Herz. Botswana X X X X X X X X X X X Brazil X X X X X X X X X X X Brunei X X X X X X X X X Darussalam Bulgaria X X X X X X X X X Burkina Faso X X X X X X X X X X X X X Burundi X X X X X X X X X X X Cambodia X X X X X X X X X X X X Cameroon X X X X X X X X X X X X X Canada X X X X X X X X 36 Quantity Inputs Quality S.E.S. Descriptive Governance Structure # Days/Hours of instruction /year Adult Average Years of Schooling % Instructional time on reading Public expenditure per student Public expenditure as % of GNI % of students with textbooks Facilities quality indicator GER Tertiary (ISCED 5&6) School Life Expectancy Teacher salary level Teacher experience Pupil-Teacher Ratio % Trained teachers Completion rates Repetition rates Parents Income GER Secondary Drop-out rates GER Primary Class-size Cape Verde X X X X X X X X X X X Central Afr. X X X X X X X X X X Rep. Chad X X X X X X X X X X X X Chile* X X X X X X X X X X China X X X X X X X Colombia X X X X X X X X X X X Comoros X X X X X X X X X Congo, Dem. X X X X X X X X X X X X Rep. Congo, Rep. X X X X X X X X X X X X X Costa Rica X X X X X X X X X X X Cote d'Ivoire X X X X X X X X X X X X X Croatia X X X X X X X X X X Cuba X X X X X X X X X X X Cyprus X X X X X X X X X X Czech X X X X X X X X X Republic Denmark X X X X X X X X X Djibouti X X X X X X X X X X Dominica X X X X X X X Dominican X X X X X X X X X X X X Rep. Ecuador X X X X X X X X X X Egypt* X X X X X X X X X X X El Salvador X X X X X X X X X X X Equatorial X X X X X X X X X X Guinea Eritrea X X X X X X X X X X X X Estonia X X X X X X X X X Ethiopia X X X X X X X X X X X X X Fiji X X X X X X X X X X Finland X X X X X X X X X X France X X X X X X X X X 37 Quantity Inputs Quality S.E.S. Descriptive Governance Structure # Days/Hours of instruction /year Adult Average Years of Schooling % Instructional time on reading Public expenditure per student Public expenditure as % of GNI % of students with textbooks Facilities quality indicator GER Tertiary (ISCED 5&6) School Life Expectancy Teacher salary level Teacher experience Pupil-Teacher Ratio % Trained teachers Completion rates Repetition rates Parents Income GER Secondary Drop-out rates GER Primary Class-size Gabon X X X X X X X X Gambia, The X X X X X X X X X X X X X Germany X X X X X X X Ghana X X X X X X X X X X X X X Greece X X X X X X X X X X Grenada X X X X X X X Guatemala X X X X X X X X X X X Guinea X X X X X X X X X X X X X X Guinea- X X X X X X X X X X Bissau Guyana X X X X X X X X X X X Haiti X X X X Honduras X X X X X X X X Hong Kong, X X X X X X X X X X China Hungary X X X X X X X X X X Iceland X X X X X X X X X India X X X X X X X X X X X Indonesia X X X X X X X X X X Iraq X X X X X X X X X Ireland X X X X X X X X X X Israel X X X X X X X X X X Italy X X X X X X X X X X Jamaica X X X X X X X X X X X Japan X X X X X X X X Jordan X X X X X X X X Kazakhstan X X X X X X X X X X Kenya X X X X X X X X X X X X X Kiribati X X X X X X Korea, Rep. X X X X X X X X X X Kuwait X X X X X X X X X X X Kyrgyzstan X X X X X X X X X X X X Laos X X X X X X X X X 38 Quantity Inputs Quality S.E.S. Descriptive Governance Structure # Days/Hours of instruction /year Adult Average Years of Schooling % Instructional time on reading Public expenditure per student Public expenditure as % of GNI % of students with textbooks Facilities quality indicator GER Tertiary (ISCED 5&6) School Life Expectancy Teacher salary level Teacher experience Pupil-Teacher Ratio % Trained teachers Completion rates Repetition rates Parents Income GER Secondary Drop-out rates GER Primary Class-size Latvia X X X X X X X X X Lebanon X X X X X X X X X X Lesotho X X X X X X X X X X X X X X Liberia X X X X X X X X X X Libya X X X X Lithuania X X X X X X X X X Luxembourg X X X X X X X Macao, China X X X X X X X Macedonia X X X X X X X X X X X Madagascar X X X X X X X X X X X X X Malawi X X X X X X X X X X Malaysia X X X X X X X X X Maldives X X X X X X X Mali X X X X X X X X X X X X X Malta X X X X X X X X X X Mauritania X X X X X X X X X X Mauritius X X X X X X X X X X X Mexico* X X X X X X X X X X X Micronesia X X X X X X Moldova X X X X X X X X X X X Mongolia X X X X X X X X X X Montenegro Montserrat Morocco X X X X X X X X X X Mozambique X X X X X X X X X X X X X Myanmar X X X X X X X X X X Namibia X X X X X X X X X X X X X Nepal X X X X X X X X X X X X Netherlands X X X X X X X X New Zealand X X X X X X X X Nicaragua X X X X X X X X X X X X Niger X X X X X X X X X X X X X 39 Quantity Inputs Quality S.E.S. Descriptive Governance Structure # Days/Hours of instruction /year Adult Average Years of Schooling % Instructional time on reading Public expenditure per student Public expenditure as % of GNI % of students with textbooks Facilities quality indicator GER Tertiary (ISCED 5&6) School Life Expectancy Teacher salary level Teacher experience Pupil-Teacher Ratio % Trained teachers Completion rates Repetition rates Parents Income GER Secondary Drop-out rates GER Primary Class-size Nigeria X X X X X X X X X X X X Norway X X X X X X X X X Oman X X X X X X X X X X Pakistan X X X X X X X X X X X Panama X X X X X X X X X X X X Papua N. X X X X Guinea Paraguay X X X X X X X X X X Peru X X X X X X X X X X X X Philippines X X X X X X X X X X X Poland X X X X X X X X X X Portugal X X X X X X X X X Qatar X X X X X X X Romania X X X X X X X X X Russian Fed. X X X X X X X X X Rwanda X X X X X X X X X X X X X X Samoa X X X X X X X X X Sao Tome X X X X X X X and Pr. Saudi Arabia X X X X X X X X Senegal X X X X X X X X X X X X X Serbia X X X X X X X X X Seychelles X X X X X X X Sierra Leone X X X X X X X X X X X X X Singapore X X X X Slovakia X X X X X X X X X X Slovenia X X X X X X X X X X Solomon X X X X Islands South Africa X X X X X X X X X Spain X X X X X X X X X X Sri Lanka X X X X X X St. Kitts and X X X X X X X X Nevis 40 Quantity Inputs Quality S.E.S. Descriptive Governance Structure # Days/Hours of instruction /year Adult Average Years of Schooling % Instructional time on reading Public expenditure per student Public expenditure as % of GNI % of students with textbooks Facilities quality indicator GER Tertiary (ISCED 5&6) School Life Expectancy Teacher salary level Teacher experience Pupil-Teacher Ratio % Trained teachers Completion rates Repetition rates Parents Income GER Secondary Drop-out rates GER Primary Class-size St. Lucia X X X X X X X X X X St. Vinc. and X X X X X X X X Gren. Sudan* X X X X X X X X X Suriname X X X X X X X Swaziland X X X X X X X X X X X X Sweden X X X X X X X X Switzerland X X X X X X X X X Syria X X X X X X X X Taiwan X Tajikistan X X X X X X X X X X X Tanzania X X X X X X X X X X Thailand X X X X X Timor-Leste X X X X X X X X X Togo X X X X X X X X X X X X X Tonga X X X X X X X X X Trin. and Tob. X X X X X X X X X X Tunisia X X X X X X X X X X Turkey X X X X X X X X X X X U.A.E. X X X X X X X X X X Uganda X X X X X X X X X X X X X X Ukraine X X X X X X X X X X X United X X X X X X X X Kingdom United States X X X X X X X X X Uruguay X X X X X X X X X Vanuatu X X X X X X X X X Venezuela X X X X X X X X X Vietnam X X X X X X X X X X West Bank & Gaza Zambia X X X X X X X X X X X X X Zimbabwe X X X X X X X X X X X 41 42