Textbooks and Aichievement: What We Know SWP298 World Bank Staff Working Paper No. 298 October 1978 The views and interpretationis in this document are those of the authors and should not be attributed to the World Bank, to its affiliated organizations, or to any individual acting in their behalf. Prepared by: Stephen P. Heyneman Joseph P. Farrell (Consultant) Manuel A. Sepulveda-Stuardo (Consultant) Education Department Copyright © 1978 The World Bank 1818 H Street, N.W. Washington, D.C. 20433, U.S.A. PUB HG 3881.5 .W57 W67 no. 298 The views and interprel:ations in this document are those of the authors and should not be attributed to the World Bank, to its affiliated organizations or to any individual acting in their behalf. WORLD BANK Staff Working Paper No. 298 October 1978 TEXTBOOKS AND ACHIEVEMENT: WHAT WE KNOW This paper sets out to review the published evidence from less industrialized societies on the relationship between textbook availability and academic achievement. Data are available from twelve countries. Though there were differences in the way these studies were designed, and in the way the data were handled, the importance of books vis-a-vis other school investments justifies attention. From the evidence so far, the availability of books appears to be the most consistent school factor in predicting academic achievement. It is positive in 15 of the 18 statis- tics (83%). This is, for example, more favorable than the 13 of 24 (54%) reported recently for teacher training. The consistency of these positive results does not mean that we know all the reasons why; indeed we do not. Books do not have an impact anywhere near uniform. The paper discusses four areas for future research and evaluation: (i) analysis of existing sets of survey information; (ii) collection of new data from intervention experiments; (iii) studies of book production and use in classrooms; and (iv) the exploration of ques- tions on distribution, equity, and costs. If we had more knowledge on these subjects, then the coming increase in textbook investments would be all the more secure. Nevertheless, given the inconsistent results from other pedagogical variables, these findings imply that investments in reading materials hold a distinct advantage when maximizing cognitive achievement. Prepared by: Stephen P. Heyneman Joseph P. Farrell (Consultant) Manuel A. Sepulveda-Stuardo (Consultant) Education Department Copyright ® 1978 The World Bank 1818 H Street, N.W. Washington, D.C. 20433 U.S.A. Summary 1. This paper set out to review the published evidence from 12 less industrialized countries on the relationship between textbook availability and academic achievement. The task was less straight-forward than anticipated. There is a surprising array of measures, methodologies, and reporting proce- dures. Studies focused on differing achievement indicators: math, science, reading, language and othaers. One study also included progress in school and a child's ambition. 2. Books too were counted in different ways. Some were counted by asking the headmaster how many he thought were contained in the school library, or were in school classrooms; or how many were owned by students, etc. In other cases books were counted by putting a question directly to pupils them- selves to answer in a questionnaire. Only one study used an actual school- by-school count by the author. 3. There were also differences in the way these studies were designed, whether by experiment or survey; and in the way the data were analyzed, whether by regression, through a correlation or with a difference among means. Results in one study may have been unjustifiably influenced by the problems in sampling (Iran); another study was plagued with problems attendent with a muddy intervention definition (Ecuador). All data, moreover, which are derived from areas which lack efficient processing procedures and familiarity with research in general, should be treated with some caution. These caveats would characterize any findings from LDC's; and no summary is better than the data on which the information is based. 4. Despite these cautions, the importance of books vis-a-vis other school investments justifies our attention. From the evidence we have so far, the availability of books appears to be the single most consistently positive school factor in predicting academic achievement. In 15 of the 18 statistics (83%) it is positive; this is, for example, more favorable than the 13 of 24 (54%) recently reported by Husen (1978) for teacher training. 5. The consistency of these positive results does not mean that we know all the reasons why; indeed we do not. Books do not have an impact anywhere near uniform. To help boost our understanding the paper discusses four areas for future research and evaluation: (i) analysis of existing sets of survey information; (ii) collection of new data from intervention experiments; (iii) studies of book production and use in classrooms; and (iv) the exploration of questions on distribution, equity, and costs. If we had more knowledge on these subjects, then the coming increase in textbook investments would be all the more secure. Nevertheless, given the inconsistent results from other pedagogical variables, these findings imply that investments in reading materials hold a distinct advantage when maximizing cognitive achievement. US_ OF BOOKS AND PEDAGOGICAL STYLE 1. Usually the term educational "technology" refers to recent, mecha- nically sophisticated didactic aids, those produced with large amounts of capital such as radio, television, and computer-assisted instruction. Because the delivery mechanism (not content) was new, they have been thought of as innovative. Not so with textbooks. Using a textbook has frequently been portrayed as pedagogically constraining, and the phrase "textbook teaching" as though it were pejorative. 2. Perhaps this is changing. Recent evaluations of project inter- ventions have indicated that what in the 1960's had been championed as "innovative"--new math, open classrooms, individualized instruction--are frequently found to be less effective than "traditional" strategies, those which had emphasized set information and highly structured presentations (Anderson et al., 1977; U.S. Office of Education, 1976; Smith and Tucker, 1977).1/ While one cannot of course automatically transfer a finding from the U.S. to other nations, we note that when radio has succeeded in assisting formal instruction, as it has in the teaching of mathematics to Nicaraguan children, it has succeeded because of intensive curriculum planning--a tight structure and strong doses of oral repetition, review, and memorization (Searle, B., Friend, J., Suppes, 1976). 3. Many now recognize that there is no single "best" style or universal standard of pedagogy for all children or in all subjects; and furthermore, that structure and memorization can support and not diminish curiosity and inductive reasoning (Bennett et. al., 1976). The re-emergence of using more structure and memory in the classroom has made the use of textbooks all the more acceptable. 4. On occasion less industrialized societies, after a time lag, have been know to follow the educational fashions of wealthy societies. In this instance rediscovering "structure" may come at a convenient time. LDC's may be able to avoid having textbook investments rejected because they encourage the use of memorization, only to have the use of memorization re-emerge as acceptable pedagogy a decade later. This may save them from wasting time and energy on ambiguous pedogogical assumptions about what constitutes good teaching style. 5. We should not forget what every rural primary school teacher in Africa or Asia already knows: that the textbook is an educational technology too. Though not new, books have never been widely diffused in less-wealthy societies. When first introduced they can stimulate profound changes. Books have the capacity to deliver massive amounts of new information to the most remote locations. To operate they depend neither upon electriciy nor consum- able supplies. If the content is not understood, books can be studied and 1/ Anderson et al. argue that a "basic skills" approach is more effective than all other pedogogical styles. House et. al. (1978) dispute this. But all sides agree, however, that the effectiveness of pedagogical style varies so widely from one school district to the next and from one teacher to the next, that no single pedogogy is effective as policy. -2- again and again; if quickly understood, individuals can read ahead. Ideally books can be delivered to all children equally, urban, rural, rich and poor alike. 6. With all this potential it would not be unreasonable to assume an abundance of reliable information on the effectiveness of so basic an educa- tional implement. But compared to other influences (pupil SES, teacher training etc.), the amount of data on textbooks is exceedingly scanty. This is true because techniques for empirically assessing the power of school characteristics was perfected in industrial societies in the 1960's. When later applied to non-industrialized societies the statistical methodology had to be transferred. 1/ But in many cases also transferred were industrial society preconceptions about which characteristics to measure (Heyneman, 1978a). Relatively low-cost, widely-distributed items such as blackboards, window glass, ditto machines and textbooks frequently were not realized as being important; science laboratories, classroom T.V., and access to a school librarian were. The latter were measured carefully; the former were not. Add this perceptual blind-spot to the fashionable tendency to down-play textbooks as pedagogy and one can understand why, in most education surveys in non- industrial societies, textbook measurements have been given only peripheral treatment. The following describes what few sources of information are available, and what we have learned from them so far. WHAT DO WE KNOW? 7. In a recent review of the impact of teacher training on academic achievement, Torsten Husen et al. (1978) report equivocal results: out of 24 studies in lesser industrialized societies, 13 (54%) report positive relationships of varying magnitudes; but 9 (37%) report null relationships; and two report negative relationships. Between books and achievement the relationships are more consistent and more clear (Table 1). There have been studies in ten countries. From these have emerged 18 separate statistical relationships between reading and achievement - that is, achievement of some kind. Fifteen of the 18 are measures of academic achievement--math, science, reading and language. In addition Farrell and Schiefelbein (1974) studied the relationships between textbooks and educational aspirations, prestige of secondary school entered, and the propensity to remain in secondary school. Of the 18, 16 measured the availability of textbooks in the classroom; one measured the availability of a school library (Beebout, 1972); and one measured the amount of daily time spent reading (Simmons and Askoy, 1972). Out of the 18, two relationships are negative and one relationship is null. 1/ This transfer would have been considerably more accurate had researchers listened closely to the advice of C. Arnold'Anderison. "If I were sent", he said, "to inspect a sample of school systems, rating them from 'promising' to 'sterile', my basic question would be: What percentage of your elementary pupils have no textbooks" (1970:5). - 3 - As suggested below, these may be spurious. Fifteen of the 18 relationships (83%) are positive. 8. This does not mean that we know all we need to know. What it does mean is that compared to other commonly measured characteristics such as teacher training, class size, teacher salaries, boarding facilities, grade repetition etc. The ava:ilability of books appears so consistently associated with higher achievement :Levels that as an instrument for affecting learning they represent a reasonable choice. In short, they are worthy of more experi- mentation and close scrul:iny. 9. Data on pupil achievement and access to reading materials in school are available from Ghana., Thailand, Uganda, India, Iran, Chile, El Salvador, Brazil, Ecuador and Malaysia. In two studies (Schiefelbein and Farrell's study of Chile and Haron's study of Malaysia), textbook influences are explored in detail. In others they are considerecd as only one among a large number of variables in a regression or an A.I.D. procedure. 1/ Thus we have differing statistics: correlations, regression coefficients, and simple mean differences between experimental and control groups. The IEA Studies: India, Iran and Chile 10. Under the auspLces of the International Association for the Evalua- tion of Educational Achievement (IEA) a vast and comprehensive study has been conducted of academic achievement in six subject areas (science, reading com- prehension, literature, English as a foreign language, French as a foreign language, and civic education) and the factors that affect those levels of achievement. The study was conducted in twenty-one countries, among student populations ranging from ten years of age to those in the last year of secondary school. Our concern here is with the results with respect to the two subject areas, science and literature, where the effect of textbooks on achievement is reported in three developing nations: India, Iran and Chile (Comber and Keeves, 1973 and Purves, 1973). 2/ 1/ AID - Automatic Intieraction Detection (El Salvador). For an explanation of technical terms please refer to the appendix. 2/ Precisely because they were collected in developing countries, the data from Iran, India and Chile are not as "clean" as the data from the richer nations included in the IEA survey. The interested reader can consult the originaL sources or Peaker (1976) for careful discussions. However the problems in the data are not in our judgement sufficient to invalidate the fLndings; they are problems which plague all research on developing areas.. Table 1 Availability of Reading Material and its Relationship to Student Achievement Source Country School Level Criterion Variable Relationships + 0 1) Fuller and Chantavarish (1977) Thailand Primary G 3 Language and Arithmetic + (1) 2) Schiefelbein and Farrell (1974) Chile Primary 6-8 Mathematics and Language + (1) 3) " Educational Aspirations + 4) " " Type of Secondary School 5) " Entered + Survival to End of Secondary + (1) 6) Heyneman (1976) Uganda 2293 7th graie students from Mathematics, General Knowledge, a random sam?le of 67 schools and English. + in 5 districts and 3 urban areas. 7) IEA (1973) Chile Primary (11 vears old) Science + 8) IEA India ' Science + 9) IEA Iran " Science 10) lEA Chile Primary (14 -ears old) Literature + 11) IEA Iran Literature 12) Ministry of Grades 2, 3, 5, 6, 8 and 9 Spanish, Mathematic! , Social Education (1976) El Salvador and Natural Science. + 13) Ministry f Malaysia National prol ability sample Bahasa Malaysia, En;lish, Math, + (1) Education and of 6,000 12-)3 year old Science, History anc Geography. Haron (19 7) youths. 14) Lynch (1974) Ecuador 88 randomly selected first- Pre and post-tests over a one- 0 grade classrooms. year period in math, reading and science. 15) Smart (1978) Ghana 40 rural sch.)ols at grades Pre and post tests ,ver a two- + 8 and 9. year period in read-ng comprehension. 16) Simmons aid Tunisia 44 students 'rom a village Arabic, French, Ariahmetic. + Askoy (19 '2) and 80 stude,lts from an urban suburb grades 4-8. 17) Beeboxit ( .972) Malaysia 7674 student grades 10-11; Amount learned between second- + random sampli of 89 schools. ary entrance and secondary leaving examinations. 18) Wolff (19.0) Brazil Random selection of 5% of Language competence in + (1) the first grade students in Portuguese as assessed by Rio Grande dc Sul (N = 20 grade one teacher. thousand). (1) - Stronge in students with low SES. - 5 - 11. In science the data are taken from the 10-year old population, most of whom were in primary grades four or five (60% of 2731 students in India, 97% of 708 students in Iran, and 79% of 1510 students in Chile). The achieve- ment instrument was a 40 item test of general knowledge. The textbook vari- able was measured by determining if each student had a textbook available for learning science. There is no indication whether having a textbook available meant that each individual student had a book, only whether there was a book in the classroom which students could use. In all three countries surprisingly high proportions of students reported having a textbook "available": 69% in Chi]e and 81% in both India and Iran. Simple correla- tions of textbook availability with test score are reported at both the between-student and between-school levels for all three countries. 1/ (See Table 2) Table 2 /1 Country Between-Students r Between-School r India .21 .34 Iran .10 -.26 Chile .02 .21 /1 No significance levels are reported for these IEA data, but givern the size of the samples involved the prob- abilities of even very weak r's having arisen by chance is fairly remote. Source: Comber and Keeves, 1973, p. 248 and 205. Regression coefficients for individual variables would permit one to deter- mine the effect of textbook availability in relation to the effects of the large number of other variables measured in these studies. But these are not reported. The coefficients which are reported, at both student and school level, are blocks of school variables. In both Chile and India a "learning conditions" blc,ck, which includes textbook availability, accounts for more of the variance in test score than does a block which includes home circumstances and student age and sex. This is the case at both the student and school level. In Iran, however, the pattern is reversed, although even there the "learning conditions" block accounts for a noticeable portion of the variance (6% at the student level and 13% at the school level). 12. In literature the data are taken from a 14-year old population: 1058 students in Chile and 1254 in Iran (data from India are not reported). The achievement test was a 17 to 19 item instrument measuring the ability to 1/ For a discussion of between-student and between-school analysis, please refer to the annexes. read a literary text critically. This included both comprehension and inter- pretive skills. The textbook variable was measured in two ways: (1) by asking whether teachers used textbooks as a prime teaching material, and (2) by asking how frequently texts were used. Again, neither of these indicate whether each student had a text, nor how many texts students had. Results using these measures are reported only at the between-student level. 13. Simple correlations between literature achievement and use of texts as a prime teaching material are reported for both countries (Chile = .09, Iran = .10). Here too regression coefficients are not reported for this measure of text availability, but the variable is included in an "instruc- tional variables" block. In Chile this block explains 9.0% of total achieve- ment variance. This is less than the 14.4% explained by home background and age and sex. In Iran 12.1% of variance is accounted for by instructional variables, compared to 4.5% by home background, age and sex. For Chile the unique contribution (from commonality analysis) of frequency of use of text- books is reported (0.9%). (Purves, 1973) 1/ Thailand 14. The data from Thailand are derived from a 1973 stratified probability sample of 23,555 grade three pupils and 987 teachers from all types of schools and regions throughout the country. (Fuller and Chantavanish 1977). Achieve- ment was measured by a test covering arithmetic concepts and problem solving, reading comprehension, vocabulary and spelling. Textbook availability was measured as the number of books available to pupil. The correlation coeffi- cient between these variables is positive but modest (r = .11). In this study regression coefficients are reported so that one can assess the "independent" effect of textbook possession. This is particularly important since one can assume that in many societies possession of a textbook is highly correlated with the socio-economic status of a student. Table 3 presents the regression results for the total sample. Textbook possession clearly has some indepen- dent effect upon achievement but, relative to other variables included in the analysis, not a large effect. It is worth noting that when socio-economic status is excluded from the analysis, the regression coefficient associated with textbook possession does not change markedly, nor do any of the other coefficients. Of particular interest in this study is an analysis by type of school, since different types of schools in Thailand cater to different social strata. Four types of schools are considered. Private schools are urban and serve relatively wealthy people. Ministry of Education schools are urban and serve primarily middle class families. Municipal schools are also urban but appear to serve slightly lower social status families. Provincial schools are generally rural and serve pupils of relatively low economic status. (Fuller and Chantavanish, 1977: 41-42). In the two types of schools which serve primarily middle and upper class students (private and Ministry of Education) textbook possession has no observable effect upon achievement. In the two types of schools which primarily serve lower status students (municipal and provincial) there is an observable textbook effect. (Municipal schools: b = .06--5th of 10 variables reported; Provincial schools: b = .04--12th of 14 variables). 1/ See Appendix for a discussion of commonality analysis. TABLE 3 SCHOLASTIC ACFiIEVEMENT AS A FUNCTION OF SELECTED PUPIL SCHOOL AND SOCIOEC'ONOMIC BACKGROUND CHARACTERISTICS IN TIAILAND - Equation 1 Equation 2 Variables Socioeconomic background included Socioeconomic background excluded B Coefficient Cumulative R2 B Coefficient Cumulative R2 School Size .21 .20 .26 .20 Social Class(father's occupation) .18 .27 -- Attended Kindergarten .13 .29 .18 .25 Did Not Repeat Any Grade .13 .31 .13 .28 Teacher's General View Learning Ability of Children .09 .32 .10 .29 Pupil Weight .08 .32 .09 .31 Pupil/Teacher Ratio .07 .33 .08 .30 Teacher Qualification .06 .33 .05 .32 Pupil Absence -.05 .34 -.06 .31 Teacher's Years School- incq .04 .34 .05 .32 Distance Home to Schiool -.06 .34 -.07 .30 Availability Textbooks .02 .34 .03 .32 Aqt2 -.04 .34 -.05 .32 Teaclher's Absence Record .01 .34 .01 .32 Conl:stcint 19.27 25.34 a/ All reported values are statistically significant (p < .001). SwArc%. FulLer arnd Chnntavanish, 1977: 33. -8- Uganda 15. The Ugandan data were derived from a random sampling of 67 primary schools in five districts and three urban areas (Heyneman, 1976; 1977). Information was collected from each of 2,293 children on parental occupation, education and household possessions. There is information on school facili- ties and teacher characteristics. Data on learning were collected by mea- suring the child's performance on the Primary Leaving Examination, containing sections on math, science, English, geography and history. Book measures were obtained as a result of the author personally counting each book available to the students in grades one and seven. No effort was made to distinguish old books from new, inappropriate from appropriate, or English from vernacular. The measure consisted of a count of the total number of textbooks and readers available/child in the highest and lowest grade levels in the primary school. 16. Access to textbooks in grade one appears to have an equivocal rela- tionship to grade seven academic performance, averaged by school (r = .01). But access to grade seven textbooks does not. There the relationship which emerges with performance averaged by school (r = .24 ]p. > .05]) is the third largest among the physical facilities, and remains consistent irrespective of socio-economic status controls. 1/ Chile 17. A more detailed analysis of the effect of textbook availability is found in a separate longitudinal study of Chile (Schiefelbein and Farrell, 1973, 1974, 1977). The original student cohort, sampled in 1970, consisted of 3,530 students in the 8th grade (the last year of primary schooling) from throughout Chile. The achievement measure consisted of a 100 item national test administered to all 8th grade students in Spanish and mathematics. Textbook availability was measured by asking students in a questionnaire whether they had a text for their personal use in each of the five academic subjects in the 8th grade curriculum (Spanish, mathematics, social science, natural science and foreign language - usually English). From these data a total score was produced-for each student. 18. In Table 4 the results of regression analysis on 8th grade test scores are reported. Here textbook possession has a regression coefficient of .06, the eighth highest of the twelve variables included in the equation. Of those variables which lend themselves to policy manipulation, textbook possession is essentially equal in effect with "average age of teachers", interpreted as a rough proxy for teacher experience, and more powerful than the other, "school size". 1/ In October, 1977 the Research Committee approved a proposal to examine these Ugandan data in more detail (RPO 671-60). When completed this study can be expected to control for "previous ability". Only three other studies have attempted to do this. The Ecuadorian and Ghanaian experiments used a pre and post-test design; while Beebout's study measured the amount of change between the secondary school entrance and leaving examinations. All other studies have attempted to account for differences in measured ability among individuals and not differences in the amount learned by individuals over time. Table 4 Sl:andardized Regression Coefficients In Order oi- Size, For Each of 12 Regressor Variab)es on Total Score on National Test for the Eighth Grade- Variable Coefficient: Average grade 8 mark per class .18312 SD per class on verbal score national test (inverted) .16788 Hrs. per week in free reading by individual student .15286 Average possession of TV sets per class .11864 Socio-economic status of' individual student .09556 Liking for academic subjects .06910 Average age of teachers .06688 Textbook availability in class .06264 SD per class on hours spent in free reading .05067 Average inherited status of teachers .03860 Student height .03626 School size .03390 /a All reported values are significant at the p < .01 or p < .05 levels. Source: Schiefelbein and Farrell, 1974: 22. 19. Having discovered that textbooks have some significant overall relationship with achievement independent of other variables, the Schiefelbein and Farrell study goes on to specify the effect of textbook possession in relation to several other variables: a student's sex, school type (private or public) and education of the father (as a measure of SES). The results are noted in Table 5. 20. Textbook availability is divided into three categories - few texts, average texts, and many texts. The cutting points between the categories are one-half of' a standard deviation above and below the mean on the text- book availability scale. To interpret these data two facts must be taken into account. First, in Chile texts are usually not provided by the school; they are bought by the ;tudent or his parents. Thus one would expect a high correlation between number of textbooks possessed by a student and his socio- economic status. That i:his is in fact the case makes all the more impressive the relatively strong eEfect of textbook availability, independent of socio- economic status. - 10 - Table 5 Textbook Availability and Academic Performance: Means and Standard Deviations By Sex of Student, Type of School, and Education of Student's Father /a Few Texts Normal Texts Many Texts Variable M SD N M SD N M SD N Total Students 49.89 14.59 974 53.51 14.76 1225 58.49 14.52 879 Boys 50.65 14.53 456 53.44 14.70 547 60.71 14.55 432 Girls 49.21 14.53 518 53.02 14.83 678 56.33 14.49 430 In public schools 49.70 14.73 799 53.16 15.03 807 56.64 14.19 430 In private schools 50.70 13.95 175 53.28 14.27 418 60.25 14.83 449 Father's educ. Prim.or less(FEP) 48.39 14.37 719 50.83 14.51 636 54.22 14.58 264 Sec.or more (FES) 54.10 14.10 255 56.38 14.41 589 59.35 14.06 615 Boys in public sch. FEP 48.77 14.36 296 50.44 14.07 218 57.35 11.38 66 FES 55.07 14.13 90 56.81 14.87 145 58.83 14.30 126 Boys in priv.sch. FEP 47.05 14.64 39 53.84 14.77 56 56.50 17.98 44 FES 60.24 12.66 31 57.41 14.47 128 64.00 14.35 197(N.S.) Girls in pub.sch. FEP 48.71 14.81 321 51.88 15.21 265 53.54 14.90 112 FES 50.96 15.07 92 55.44 15.00 179 56.84 14.40 126 Girls in priv.sch. FEP 45.81 11.98 63 47.16 13.46 97 48.71 14.42 42(N.S.) FES 54.38 12.96 42 56.20 13.10 137 59.72 13.28 166 /a In a table such as this, one could make many comparisons between mean test scores, for each of which a level of statistical significance may be cal- culated. The means, standard deviations and N's are reported if a reader wishes to calculate the statistical significance of a particular com- parison. Perhaps the most telling comparisons are between the categories "few texts" and "many texts" in each row. All of these comparisons are significant at least at the .01 level except for the two rows marked n.s. Source: Schiefelbein and Farrell, 1974, p. 24. - 11 - 21. Second, textbooks are rather widely available to grade 8 students in Chile, (at least by the standards of most developing nations): 71.4% of the students reported that they had a Spanish text and 74.6% stated that they had a text for the foreign language they were studying; some 54.7% had a social science text, 43.9% had a mathematical text, and 42.8% - the lowest figure - had a natural science text. Eleven percent of the sample reported that they had no book of any kind in any of these five subject areas. Thus there is sufficient variation in this indicator to make its analysis useful. 22. Referring to Table 5, one first notes an almost nine-point differ- ence in test score between students with few texts (48.89) and many texts (58.49) considering no other variables. Such a result is to be expected, given the relatively high correlation of text availability with test score. The difference between the extreme categories is somewhat more pronounced among boys thani among girls. The differences are also more pronounced among students in private rather than public schools. 23. The finding that text availability has a stronger relationship to academic performance among children in private schools has an interesting implication. Private schools in Chile cater mainly (though not exclusively) to children of wealthier, higher-status, parents. One may be observing here what has come to be called the "headstart" effect, which seems to appear with most educational "treatments"; i.e., that programs designed to improve the academic performance of all children tend to benefit most those who are already advantaged, thus increasing rather than decreasing the "learning gap" between high and low-status children. In societies that are trying to reduce educational inequalities based on non-school factors, and at the same time trying to use education as one of many levers to reduce social inequalities in the society as a whole, the existence of this pattern could be quite important. 24. However, again from Table 5, the magnitude of difference in test scores between students with few and many texts is nearly identical for students with high-education and low-education fathers. Among boys however greatest textbook effect is found among lower-class students in private schools where the difference between those with few and many texts is 9.45 points. The textbook effect among higher-class boys (3.76 points) is much smaller, and identical in public and private schools. Moreover the difference between lower-class boys with few and many texts is almost as great in public schools as in private schools (8.58), Among girls, by contrast, the textbook effect is greater for those with higher levels of paternal education, in both public and private schools. However for both levels of father's education, the textbook effect is greater in public than in private schools. It appears that if the "headstart effect" operates at all, it is only among girls in private schools, and there only slightly. Indeed what is most striking in these data is the very strong reversal of this headstart effect among boys. 25. It seems clear that textbook availability has a positive relation- ship to academic performance over many different types of Chilean students. Most important:Ly, this variable seems to be one of the few discovered which - 12 - reverses the "headstart" effect and has its greatest effect precisely among those children who are most in need of educational assistance - the children of the lower social strata. 26. Being longitudinal,.the Schiefelbein and Farrell study also produced data on the effects of textbook possession on other key elements in a student's educational career: educational aspirations at the eighth grade level, the kind of secondary school a studeret enters, and the probability of a student surviving to the end of secondary school. The most powerful predictors, using either regression analysis or weighted net percentage differences (see Spady, 1970) are reported in Table 6. 1/ Here the variables reported are only the few from several hundred candidate predictors which have survived a series of screenings to eliminate all but the most important. In each case texts are one of the most important predictors. One additional specification analysis is reported by Schiefelbein and Farrell with respect to the probabilities of survival to the end of secondary school (see Table 7). Here again, textbooks have their most important effect upon children of lower social status, par- ticularly among lower class students who had high achievement levels at the end of the eighth grade. El Salvador 27. In 1974 El Salvador administered a series of national achievement tests in mathematics, social science, natural science, and Spanish to a sample of 55,000 pupils in grades 2, 3, 5, 6, 8 and 9. A sub-sample of 2,600 pupils also completed an 80-item questionnaire which noted their social status, attitudes and numerous other characteristics, including whether students had pencils, notebooks and books. 28. The former, it was found, were universally distributed - every student had a pencil and a notebook. In urban schools however, 15% of the children had no books; in rural schools 55% had no books. The published report which discusses this variable notes that "average scores for students without books were in general significantly lower than scores for students with books..." (ODEPOR, February, 1977: 43); but there is no attempt to explore this further. One task with high priority will be to take these data and apply one of the methodological techniques used elsewhere to them, thus creating a meaningful comparison. Ecuador 29. In the mid-1960's the Ecuadorian Ministry of Education decided that the style of teaching in primary school classrooms needed to be changed; and that the best way to change teaching was to design new textbooks with "modern" content, quickly train teachers to use the books, and then distribute them widely. This they did over a six-year period commencing in 1967. 1/ See Appendix. 2/ This will be one of the.tasks proposed for stage II of the World Bank's research project on "Textbook Availability and Educational Quality" (RPO 671-60). TABLE 6 STRONGEST PREDICTORS OF FOUR STAGES OF EDUCATIONAL RESULTS IN CHILE Educational Test First Year Secondary Survival to Aspirations Score Destination 4th Year Secondary (WNPD) (Regression Coeff.) (WNPD) (WNPD) SES 7.3% 8th grade mark .18 Aspirations 13.6% 8th grade mark 28.5% Texts 5.8% Verbal score S.D. .17 Pedagog. Exc. 11.3% Aspirations 10.0% Test Score 3.9% Free reading .15 SES 6.2% Pedagog. Exc. 9.4% Comm. Size 3.3% TV possession .12 Teacher Trg. 4.3% Teacher Training 7.0% Sex 1.7% SES .10 Texts 2.8% SES 4.3% Liking acad. Texts 1.9% subjects .07 Teacher age .07 Texts .06 Source: Schiefelbein and Farrell, 1977. - 14 - 30. First a set of "objectives",was decided upon. Then new textbooks were designed. The content and layout of each book had been specifically constructed to correspond to "modern pedagogy". In constructing these books in this way it was believed that such a pedagogy was definable; that such a pedagogy was correct; and that it could be transplanted from the U.S. to Ecuador and within Ecuador successfully. This was despite the fact that few Ecuadorian teachers had previously studied the concepts of new math of the "whole word method" of teaching reading, and despite the fact that these new methods were significant breaks with past local experience (Lynch, 1974). 31. In late 1970 these new first grade textbooks in math, reading and science were distributed to Ecuadorian schools. If pupils wanted them, they had to pay $1.20 (US), slightly more than a day's wages for a laborer. An experiment was designed to monitor changes in learning in the following way. A random selection of 88 schools was taken and divided into three groups: (A) those schools which had been offered new textbooks, teacher guides and teacher orientation to the teacher guides; (B) those schools which had been offered new textbooks and teacher guides but no teacher orientation; and (C) those which had access to old textbooks at whatever level of quality and Table 7 WNPD: Present in 4th Year Secondary or Not Present, By Three Predictors in a Cross-Tabulatj7n of SES and Average 8th Grade Mark - 8th Grade Mark SES High LOW WPND WPND Text availability -1.4% Text availability -2.5% Educ. aspirations 11.6% Educ. aspirations 1.0% HIGH Pedagogical exc. 8.7% Pedagogical exc. 10.0% Pre-service training Pre-service training of teachers 7.9% of teachers 8.1% Overall % surviving: 59.4% Overall % surviving: 20.6% Text availability 5.8% Text availability 1.7% Educ. aspirations 11.7% Educ. aspirations 8.4% Pedagogical exc. 7.3% Pedagogical exc. 13.5% LOW Pre-service training Pre-service training of teachers 8.8% of teachers 3.1% Overall % surviving: 36.4% Overall % surviving: 13.0% 1/ WNPD: Weighted Net Percentage Differences (See Appendix). Source: Schiefelbein and Farrell, 1977. - 15 - quantity which had existed previous to the experiment. Pre- and post-test scores were collected in each of the three school categories, and in each of the three subjects: math, science and reading. The experiment was limited to the first grade and l:o a duration of one school year only. In the experi- mental group therefore, school teachers had unfamiliar books, based upon unfamiliar beliefs abou't how to teach well; and to succeed, they had to adjust to these unfamiliar characteristics on the first try, that is, in year one. 32. Post-test scores at the end of the year were signficantly higher in all three subjects. This was to be expected. The issue was whether the learning gain in the classrooms with new textbooks was significantly higher than the learning again in classrooms without new textbooks. The results show that they were higher; but not significantly higher. This is illustrated by Table 8 below. The difference in mean achievement gain is less than one point on a 50 point scale, ancl not sufficient to say with certainty that the differ- ence wasn't due to chance. 33. These data from Ecuador raise more questions than they answer. One cannot be certain that the new textbooks did not create the anticipated effect because they were poorly written; there is no way to tell whether they were distributed in any larger quantity than the books already available; 1/ nor is it certain that the teachers were committed to the teaching changes required as the purpose of the experiment. Table 8 Mean Achievement Gain Between Pre- and Post-Tests in Three Types of Ecuadorian First-Grade Classrooms 1/ 2/ 3/ A B C B Minus C (N = 31) (N = 27) (N = 30) Reading 4/ 15.09* 15.56* 15.42* + .14 Math 4/ 12.73* 12.74* 12.11* + .63 Science 4/ 9.87* 9.93* 9.20* + .73x * p < .01 x not statistically significant 1/ A = Classrooms with new texts, teacher guides and teacher orientation. 2/ B = Classrooms with new texts and teacher guides but without teacher orientation. 3/ C = Classrooms without new texts. 4/ Range = 0-50; no display of means or standard deviation in Lynch (1974). 1/ Few first-grade classrooms had math or science books previous to the experiment; but many had readers. - 16 - Ghana 34. The Densu Times project began as an intervention to combat illi- teracy. It is relevant for our purposes because its effectiveness was evaluated on the basis of reading comprehension gains among school children. In October, 1974 members of a research team from the Ministries of Information, Education, and the University of Ghana picked 40 rural Eastern Region schools. Students in Forms II and III (grades 8 and 9) were divided into experimental and control groups of 20 classrooms each, and were pretested for reading com- prehension. The experimental groups were given access to a weekly newspaper for use in their classrooms. 35. The newspaper contained stories on local items of interest, many of which were contributed by readers. Seventy-five percent of the teachers in the experimental schools claimed to have used the newspaper in their class- rooms "every week without fail". "Readers clubs" were formed. In the course of the two-year experiment period the newspapers' editors received over 3,000 written contributions (about 35/week). The impact on reading comprehension, as reported by Smart (1978), can be found in Table 9. 36. Students in the 20 classrooms receiving the Densu Times gained between 5 and 6 more points (on a 37 point test) of reading comprehension; the difference is statistically significant. It is not entirely clear which particular element in the experiment can account for the difference. It might have been the excitement generated by having readers respond to newspaper stories with stories of their own; or perhaps it was the personal flavor of the news itself; or the scarcity of other available printed materials. Which- ever, it is clear that having a weekly newspaper did affect the ability of 8th and 9th grade Ghanan children in learning how to read. Table 9 Mean Gain in Reading Comprehension Between Pre and Post-Tests in 40 Ghanan Schools (October 1974 - June 1976) A I/ B 2/ C Gain for the Gain for the Experimental Group Control Group Difference (20 schools) (20 schools) (A-B) Form II 18.9 13.9 + 5.0* Form III 15.5 9.6 + 5.9* Total (N = 1659) 17.2 11.8 + 5.4* * Test range = 37; p <.001 1/ Schools receiving a weekly newspaper. 2/ Schools not receiving a weekly newspaper. - 17 - Malaysia 37. In 1972 a probab:[lity sample was taken of approximately 5% of the children born in 1960. A sub-sample of 6,056 youths (2,389 out-of-school) answered a questionnaire designed by the Ministry of Education's division of of Educational Research and Planning. Parental background data were obtained from the 1970 census records. Pupil data were collected from teachers and school records as well as the pupils themselves. These included pupil per- formance on national standard five examination results from 1972 in Bahasa Malaysia, English, math, science, history and geography. Data about schools were gathered by talking 1:o school principals. Included, among many other items, were measures of school size, pupil-teacher ratio, central library, class libraries and the nnuber of books in them, teacher qualifications and the percentage of children with individual textbooks (as near as tlle principal could estimate). 38. In 25% of the urban and 65% of the rural schools less than 80% of the pupils had individual textbooks. Having an individual textbook was related to pupil socio-economic status (r = .2); to teacher experience (r = .2); to school size (r = .3); and (negatively) to teacher qualifica- tions (r = -.18) (Haron, 1977: Tables 25-26). 39. Neither the number of volumes in the school libraries nor the pro- portion of pupils with individual textbooks appear at first to be related to pupil performance ( r z= .081 and r = .082). But through both regression analysis and specification of sub-samples for rural and urban location, high, medium, and low social status, and ethnic group, the intervening influence of a child's background is controlled and the influence of having a book emerges. Pearson coefficients are highest for urban Indians (r = .25), but are positive and statistically significant for each geographical, SES and ethnic category. (Chinese living in rural areas are the only exception). Furthermore gains in achievement associated wit:h having access to a textbook are stronger and more definitive among children of the less privileged within the Chinese and Indian ethnic groups. The author, however, is cautious not to assume that the dif- ferences in achievement can be explained by books alone, and speculates on the influence which might be attributable to parental motivation (Huron, 1977: 165). This query could be raised with respect to each of these sets of data (see paras 53 and 63). Brazil 40. In 1968 the Cent:er for Educational Research and Orientation (CPOE) of the Secretariat of Education in Rio Grande do Sul collected information on each primary school in the state, including information on each principal, teacher, and first grade student. The amount of data generated was somewhat burdensome, to say the least; and the decision to analyze only 5% of the sample (20,120 students) in no way detracted from the quality of representation. 41. The teacher questionnaire included a request for information on the number of textbooks owned by each student and whether students received a - 18 - passing Portuguese grade after their year in first grade. Questionnaires were returned by mail. Results were analyzed for the sample as a whole, and individually within four categories: Urban State Schools, Rural State Schools, Municipal Schools, and Private Schools (Wolff, 1970). 42. The author is quick to point out that the data contain two flaws which-make the results difficult to interpret: one as to the meaning of book ownership, another, and more serious, as to the meaning of a first grade pass. Some ambiguity arose as to whether number of books owned was a proxy for the influence of socio-economic status. It is intercorrelated with parental education (r = .28); but was not collapsed into a general SES scale. The influence of books, therefore, can be separated from measures of other SES influences. 43. This is illustrated in Table 10,which displays the proportion of children who receive a passing grade in Portuguese in each category of primary school, at each level of parental education, and number of books. Thus in Urban State schools there were 238 children whose parents had no education at all and who owned no books. Thirty-four percent of these re- ceived a passing grade. On the other hand, there were 97 children whose parents also had no schooling but who owned two books, or more. Of this group 64.9 percent received a passing grade, a proportion of 1.9 times more. - 19 - Table 10 Pass in Language and the Number of Textbooks Owned by the Student, by Level of Parental Education and Type of School (Rio Grande do Sul, 1968) 1/ Parents' Education Secondary Number of Incomplete Complete and Books None Primary Primary University Urban State Schools % (N) % (N) % (N) % (N) 0 34.0 (238) 40.0 (402) 44.3 (194) 75.6 (41) 1 52.8 (547) 58.9 (1,273) 65.3 (995) 81.5 (205) 2 + 64.9 (97) 76.4 (301) 88.7 (337) 81.0 (174) Rural State Schools 0 24.4 (135) 42.2 (185) 60.6 (33) 2/ 1 49.9 (381) 53.7 (806) 61.0 (254) 2 66.6 (63) 70.4 (152) 72.6 (95) Municipal Schools 0 39.1 (527) 48.2 (577) 58.9 (112) 2/ 1 50.6 (1,895) 57.8 (3,580) 65.3 (767) 2 + 62.2 (225) 71.0 (521) 78.6 (168) Private Schools 0 36.8 (19) 38.5 (26) 80.0 (15) 60.0 (5) 1 53.6 (69) 61.5 (182) 75.7 (185) 76.6 (64) 2 + 61.0 (41) 72.7 (165) 87.6 (275) 94.2 (206) 1/ Source: Wolff (1970). 2/ (N) is too small for comparisons. In RuraL State schools, those children whose parents had no education had a two-thirds chance of receiving a passing grade if they owned two books; but only a one-fourth chance if they owned no book. Those whose parents finished primary school had a 72 percent chance with two books but only a 60 percent chance with no book. What this demonstrates is that books are consistently effective within each category of parental status. - 20 - 44. There is another trend evident in Table 10: the effectiveness of books is stronger if one's parents had no schooling. The degree to which this is true is portrayed in Table 11. In Urban-State schools the advantage of having two books, over none, is 1.9 for children whose parents have no education, but only. 1.1 for children whose parents have had secondary or university education. In Municipal Schools too, it is significant (1.6:1.3); but of all categories of schools the difference in textbook impact on low status children is greatest in Rural-State Schools, where it is more than double the impact on high status children (2.7:1.1). Table 11 Degree of Advantage From Having Two or More Books Over Having None, by Level of Parental Education and Type of School (Rio Grande do Sul, 1968) 1/ Secondary No Incomplete Complete and Education Primary. Primary University Urban-State Schools 1.9 1.9 2.0 1.1 Rural-State Schools 2.7 1.7 1.1 Municipal Schools 1.6 1.5 1.3 Private Schools 1.7 1,8 1.1 1.6 1/ Source: Wolff (1970). 45. This substantial textbook effect is evident when textbook ownership is entered into a regression (Table 12). Here the influence of textbooks can be compared to other influences. In Rural State Schools, in Urban State Schools, and in Municipal Schools, of all the variables measured, the number of textbooks owned by students is the most powerful predictor of a passing grade in Portuguese language. In private schools and for all schools, the influence is only slightly less; in the latter case, equivalent to the impact of mother's education (Wolff, 1970). In sum these data indicate that textbook ownership has a powerful effect on passing Portuguese in grade one; and though owning texts certainly may be more likely in the higher social status groups and within the higher social status schools, the effect of owning textbooks is not a proxy for the influence of social status. In fact the lower the social status the larger impact textbooks seem to have. Table 12 Most Important Beta Coefficients in the Step-Wise Multiple Regression with Grade in Portuguese Language (Rio Grande do Sul, 1968) 1' Type of School _(N =20,120) All Urban Rural Variable* Schools _ta -iiin--i-al t .4 .tnSa Mother's educational level .14 .13 .06 .16 Father's educational level .07 .10 ,06 .10 Number of textbooks owned by students .14 .15 .16 .11 .10 Father's occupational level .05 Attendance in kindergarten .04 .10 NumDer of teachers in the class .07 .11 .07 .07 Teacher's experience in first grade .06 Area per student -.06 -.06 Numbers of hours in the school day .05 Principal's experience in primary education -.14 Principal's experience in the job .08 Number of specialized personnel in school .05 Presence of agricultural club -.09 Supervisor's level of training .05 *Criterion for inclusion: improvement in the squared multiple correlation (R2) of .002 or more; for private schools, of .003 or more. 1/ Source: Wolff (1970). - 22 - 46. On the other hand, the author suggests that these data are flawed in another way; and in this case the flaw is more serious. These data from grade one derive from a teacher-generated criteria of "quality". Because teachers differ in judgement,one cannot be certain that the same charac- teristics are being predicted in every case. Some teachers for example fail larger numbers of students than do other teachers. A student of the same quality might receive a "pass" in one context, but a "fail" in another.l/ The dependent variable in this case is not as a clear measure of Portuguese achievement as it is an indicator of who passes a teacher's judgement. Knowing this, all one can say is that those children whom the teachers report as owning more textbooks stand a significantly greater chance of ''passing" a teacher-administered grade one assessment; and that the influence of owning more textbooks, independent of other influences, is strongest on those teachers in Rural State schools and for those children in Rio Grande do Sul whose parents have had less education. Summary Tendencies 47. While there is a positive association between textbook possession and academic achievement reported in most of these studies (the deviant cases being Iran and Ecuador) the results are not conclusive. The Schiefelbein and Farrell study of Chile indicates that textbook possession has its most impor- tant effect upon achievement among children of lower socio-economic status. Although the findings are less straightforward, the Thailand and the Malaysian studies point to the same conclusion. If texts have their greatest impact on children from lower social backgrounds, then the "deviant" negative findings from Iran may be significant. On only two variables does Iran differ notably from other developing nations included in the IEA studies. In Iran all sampled schools were urban, as opposed to India (21%) and Chile (58%). Twenty percent of the students in the Iranian sample had fathers with managerial or professional occupations. This again contrasts with India (8%) and Chile (4%). Thus the Iranian sample is predominantly urban disproportionately high status and from this sample emerges a negative correlation between textbook possession and academic achievement. Were the achievements of lower status Iranian students to have been cross-tabulated, the question is whether the results would differ markedly from the results of Chile, Thailand and India.2/ In one way or another the latter suggest that textbook possession has its strongest effect upon academic achievement among lower status children. 1/ Despite the fact that teachers are supposed to follow state-wide criteria for deciding who should fail, there is a tendency to confuse strict teachers with good teachers. According to Wolff (1970) this may be one explaination why negative relationships emerge between pass-rates and teacher quality. 2/ This is being considered as one task for stage II of the World Bank's research project RPO 671-60. The IEA data have already been acquired for this and for other purposes as part of the project. - 23 - 48. It is interestiag to note that where textbooks and achievement were measured at both the individual and school level (IEA India and Chile), the correlation at the school level is markedly higher (again, Iran is a deviant case). This parallels tlae Schiefelbein and Farrell findings where both correlation and regression coefficients relating to achievement were higher at the classroom level. When aggregated to the school or classroom level what may be captured is not so much the effect on the individual student's learning of his own possession of a textbook, but the effect of a teacher in using them. When few (or no) students have textbooks, the teacher has to use the textbook as a guide, and rely on the blackboard or on oral dictation; and students have to copy material into their own notebooks, thus producing their own texts. This wastes the time and energy of both teacher and student. Where most or all students have texts,, a teacher may have an option of working with small groups, of assigning and of engaging in other teaching practices. If this is the case then textbooks may be important not simply as learning aids for individual students who have them, but as a necessary condition for teachers to use more effective teaching techniques. Research on this is needed. If correct, thea there are implications for textbook distribution strategies and for the kLnds of pre-service and in-service teacher training programs that might accompany programs of textbook production and distribution. 49. No apparent connection emerges between geocultural area (Latin America, Asia, Africa, tne Middle East), and the strength of the relationship between books and achievement. On the other hand, each society considered here is developing. Comparisons with similar studies done in industrialized nations provide striking patterns of two kinds. First, achievement levels in all subjects studied are approximately 50% lower in poorer nations. The correlation between amount learned in science and reading and national GNP is approximately +0.7. (Inkeles, 1977). Second, the factors which predict those achievement levels are markedly different. Put briefly, in developing nations the proportion of achievement variance accounted for by in-school variables is much greater. This can be seen by systematically plotting the variance accounted for by home background and in-school factors against a nation's level of development (Heyneman, 1976). The more developed a nation, the greater is the proportion of achievement variance accounted for by home background factors [r = .60 (p < .01)] and the smaller is the propor- tion of variance accounted for by in-school factors [ r = .57 (p < .01)]. In India for example, sex alnd family social status has only 10% as much influence on science achievement than they do in in other (mostly industrial) IEA coun- tries. On the other hand, the impact of Indian schools is three times more in science and almost 4 times more in reading (Shukla, 1974). This is illus- trated by Table 13. What this implies is that a school investment in New Delhi can be three to four times more likely to account for a difference in achievement than would a similar investment, say in Chicago (Heyneman, 1978b). - 24 - Table 13 Contributions to the Variance in Learning by Four Blocks of Variables, in India and in other IEA Countries * Science Reading Median for Fraction Median for Fraction 18 nations India (B + A) 15 nations India (B * A) A B C A B C 10-year olds Sex and SES 14.9 1.3 (.08) 14.0 1.6 (.11) Curriculum track 0.4 0.1 (.25) 0.4 0.4 (1.0) School influences 6.7 19.8 (3.0) 3.9 14.9 (3.8) Attitudes 5.8 8.1 (1-4) 6.8 13.9 (2.4) * Source: Shukla, 1974: Table 7 and Table 8. 50. One hypothesis for this is that school variables, measured by cross-sectional data, vary less in developed societies and therefore cannot account for as much variance in achievement scores. In industrialized socie- ties it can be assumed that every pupil will have a complete set of textbooks for all subject areas, indeed often complemented by a variety of alternative texts, workbooks, classroom libraries, and so forth. In rich societies investigators must look for subtle and complicated differences in quality of texts. But these are difficult to define, let alone measure. 51. The little we know about the effect of textbooks upon academic achievement suggests that they are the most consistently important variable, but that the degree of their effect varies. The potential value of very detailed analyses of existing data sets is indicated by the results from the Haron and Schiefelbein and Farrell studies, both of which suggest that text- books have their most significant impact upon the education of lower status students. WHERE DO WE GO FROM HERE? 52. In the following paragraphs we suggest possible lines of additional research. These comments draw as heavily upon our understanding of the general educational literature and our experience in schools in both devel- oped and developing areas, as they do upon the more formalized data briefly described above. - 25 - Detailed Analysis of Existing Data Sets 53. We should devote effort to sets of data already created. The ECIEL, IEA, El Salvador and other data sets have valuable information on access to and use of reading materials. These need to be pursued systematically. Two styles of analysis are particularly important. First, we need more detailed regression analyses. In many societies access to and use of reading material is closely associated wit:h socio-economic status. Without the results of multiple regression or similar multivariate analyses, it is impossible to tell if correlations between reading materials and achievement are spurious, proxies for social status, or represent an independent effect. We also can profit Erom more specification analysis, one which carefully considers the interaction between textbook possession and other variables: teacher train- ing, class size, in-service training, community type (urban vs. rural), ethnicity, and social background. Particularly important will be to inves- tigate what Kifer (1977) calls "process variables--whether parents assist with homework, whether ptupils like to read in their spare time and others. These, he discovers, are consistently more important in predicting learning than is socio-economic st:atus; and though the level of impact of socio- economic status varies dramatically between industrial and non-industrial societies, the influence of these "process" variables does not. These may be particularly relevant for understanding how reading materials get utilized effectively in one social context but not in another. Future Intervention Evaluations 54. One way to find out whether having more reading materials available will increase learning is to distribute them and record the level of learning beforehand and afterward. Because books can be distributed in a manner approaching uniformity, by their very nature they make particularly good candidates for interventions. This can be done in two ways. 55. One is through straight (non-experimental) project evaluation. The Philippines is a good example. With the assistance of a World Bank loan (1224-PH), the Covernmenl: is developing, revising, printing, and distributing textbooks for elementary and secondary schools. The quantitative objective is to reduce the ratio oi- students per textbook per subject from 10:1 to 2:1. The evaluation component will compare the improvement in learning in grades one and two, in Filipino., science, and mathematics. The project is national; there is no status quo control group because the object is to distribute enough textbooks so that all children will have an equal opportunity for benefiting from them. 56. A second way is through an experimental intervention. With an experiment there is a random assignment of "treatment conditions" to the units of observation (be they pupils or classrooms). One is currently under- way in Nicaragua. There., 40 classrooms, 20 in grade 1, 20 in grade 4, are receiving enough to provide one math book for each child. First-graders will receive workbooks (non-reusable); fourth-graders will receive textbooks. In addition, each teacher will receive a short training course on the use of the texts, a teacher'S guide, and a copy of the math text for the grade - 26 - above, in the event that there are children who are quick and who could profit from more advanced exercises. The teachers are used to the textbook; and they like it. It is a book common to most Nicaraguan schools but because of financial constraints, rarely before available to each child. 57. The experiment has two purposes. The first is to see whether children's mathematics achievement can be increased by giving each child access to a high-quality, attractive mathematics textbook. Measures have been taken of mathematics ability before the experiment and will be taken again at the end of the experiment. These in turn will be compared to pre- and post-test scores of children in a randomly selected control group whose access to texts and other school resources has not been altered from the norm. 58. The second purpose is to compare the achievement gains of children with new textbooks to the achievement gains of children who, in the same period, will be exposed to mathematics by radio. At the end of the school year in December 1978, there will be a three-way comparison in grades 1 and 4, as follows (Table 14): Table 14 Mathematics: Grades 1 and 4 Net Pre-Test Post-Test Difference 1. Control Group 1-a 1-b 2. Experimental Text Group 2-a 2-b 3. Experimental Radio Group /a 3-a 3-b /a At the first grade level there will be two variations of the radio group, generating a four-way comparison at that level. 59. These experiments in the Philippines and Nicaragua will provide information as to the actual robustness of textbook interventions. But the number of case studies are hardly sufficient to generalize to the variety of milieus where the introduction or improvement of textbooks is under consider- ation. Information could be maximized were future intervention projects to be supplied with the pre- and post-test evaluation capacities that have been built into the Philippines project. In this way each could become a new source of information on what materials work in which context. Studies of Book Production and Use in Classrooms 60. In terms of usage, both surveys and intervention experiments are only rough maps of what actually goes on in classrooms. Their primary function is to tell us whether reading materials get to classrooms, in what amounts, and what effect they have. They supply precious little information about the extent or the ways in which reading materials are actually used. For this we - 27 - need a few good observational studies. For example one important question is whether there is a "critical mass" before supplies have an effect. If shared, is half the students having texts an equivalent to all having them? Is there a threshold--a proportion of students who have to have books before their impact can be multiplied? We do not know what that threshold might be, and for poor nations, this would be a key datum. 61. Does the importance of reading material differ by grade level, or by subject area? Obviously', it is difficult to teach reading skills without books. But once initial skills have been established, could the mechanics of reading be maintained or improved through the use of non-specialized reading matter-- newspapers, magazines, wall posters, comic books? This is what the Ghanian study would suggest. Conversely, in mathematics one may be able to teach the basic algorithms and numnber facts without a text, but in higher grades, is it more difficult to present problems and teach the elements of mathematical theory without textbooks? Given the fact that many societies will not finan- cially be able to supply books for all subjects in all grades for a long time, some systematic thinking needs to be done on this question. 62. When speaking of "textbooks" people tend to have a standard image in mind--a set of pages bound together permanently, mass produced with stand- ard content for all chiLdren. Yet one suspects that the key feature of text- book availability is simply that learners have material to read and to work with. Perhaps equally effective material could be printed on newsprint or produced by mimeo machine. For some subject areas and for some grades it may be better to have packages of locally-printed learning materials than to produce a single standard text. Very little is known about the costs and benefits associated witli these alternative means of providing printed matter. 63. There is frequently an implicit assumption that a textbook is a textbook is a textboolc--that content and presentation don't matter. The insignificant results from the Ecuadorian experiment would contradict this. Content can be differeni:ly organized and this can affect learning. Material can be deduced from: (1) universal notions of how children learn; (ii) the internal logic of the subject matter; or (iii) learning patterns specific to the child. Fierce debai:es exist on which style of organization is most appro- priate. In North America the debate is particularly strong over teaching grammar, science, and, as always, reading; but similar debates occur in less industrialized nations. In Chile for example there are now four distinct "syllabarios" in use--a:Ll basic reading texts. Each uses, and implies for the teacher, a different style of teaching reading: phonetic method, globalized semantic methocl, word recognition method, etc. Similarly mathematics texts can vary in the extent lo which they embody notions of "new math". Science texts may emphasize memory or experimentation. Despite enormous amounts of research in developed nations, there is relatively little agreement about which kinds of content are most useful (or counter-productive) for which kinds of children with which kcinds of teachers. In the varying cultures encompassed in the "developing" wor:Ld the answers are no less uncertain. For example, are books less important for privileged children because they have so many other learning resources, incLuding printed matter, at their disposal? Or are books - 28 - more effective when they are reinforced by other cultural clues and media. This review would indicate that texts are more effective with children for whom the textbook is one of the only resources available; but this is not assured. 64. In the final analysis one can assume that the effectiveness of text- books as didactic aids depends upon the use made of them by teachers. Beeby (1962) once suggested that at the early stages of the educational development, when teachers frequently know only a little more than their more advanced students, a textbook may be absolutely essential; it is the only "rock" they have to cling to. As a system develops, teachers may need texts less and can deviate from their content more. Although this suggestion has considerable intuitive appeal, it has never been verified. 65. One might assume that pre- and in-service teacher training are important to maximize the effect of textbooks, but except for the Ecuadorian experiment, we have very little empirical evidence about how effective this interaction has been, despite the fact that very large sums are spent on teacher training. When "modern math" textbooks were massively distributed in Chile, and national tests were constructed in relation to their content, large numbers of teachers knew nothing about this new notion of math nor how to teach it. A massive in-service training program was implemented but a teacher's exposure to in-service training had no consistent relationship with student scores on the mathematics portion of the national test. In spite of this, possession of a mathematics text was related to student scores on the mathematics sub-test (r = .10) (Schiefelbein and Farrell, 1971). 66. Books are frequently treated as a single intervention; very little is understood about their impact in conjunction with other instructional aids, such as radio, television, or ditto machines. Is a textbook a substitute for the use of science laboratories, or does each make the other more effective? Are reading materials more effective in large classes? Without texts, pre- sumably a teacher in a large class has to rely upon the lecture/group recita- tion method. There is some unpublished evidence from Chile which suggests that in order for teachers in large classes to work with small groups, the presence of textbooks is necessary, but not a sufficient condition of effec- tiveness. 67. Answers to these questions will require detailed and carefully designed- observation studies and in-class experiments. On the other hand, the long tradition of micro-studies of teaching behavior in the U.S. has produced little certainty in the design of teacher training programs. Perhaps it would be useful to explore a relatively new model of micro-teaching devel- oped by Orme (1970), which in experimental in-service training applications in the U.S. and Canada, has produced substantial changes in teacher behavior and improvements in student learning. Exploring Questions of Distribution, Equity and Costs 68. More information is needed on the costs of book production and dis- tribution, costs of differing subject specializations, differing scales of - 29 - operation, and differing systems of pricing and ownership. It is often assumed that school books should be produced and distributed by the state and that central distribution would be more equitable than private purchase by schools or by individuals. This may or may not be the case. 69. Among school re!sources, books are one of the simplest to purchase, distribute, and maintain. Books are prime candidates for providing increments in equity; they can be reliably provided in greater amounts to individuals (or schools) who currently have least of them. But to increase equity the distributing authority has to perform better than individual (or local) pur- chase of books in two different ways. First it has to more intelligently decide which kind of books to distribute. This is not an impossible task since central authorities have access to wider experience and more scientific information on what works and why. But central authorities also have to actually deliver books more equitably than books had been acquired in the past. If books are distributed in a less equitable fashion, then the central- ization of distribution can be said to have worked both against efficiency and against equity. 70. This mismanagemtent has appeared in Uganda after the purchasing of textbooks was centralized. In Table 15 two groups of correlations are dis- played: (1) the number-of classroom textbooks for each grade seven student acquired by schools throtugh their own purchase prior to 1972; and (2) the percentage of each school's 1972 order of (approved and prepaid) textbooks which actually arrived from the central distribution agency. Both figures are correlated with measures of school socio-economic status: paternal educational attainment and the number of consumer possessions to be found in pupils' homes. It is not surprising to discover that the number of textbooks/ child tends to be higher in the schools with pupils from more privileged social backgrounds--this inequity, after all, was one of the justifications for having a central authority distribute school textbooks in the first place. What is surprising is to discover that the relationships were strengthened rather than diminished as a result of centralization of school textbook dis- tribution. Between access to textbooks and paternal education, the relation- ship increased from r = .125 to r = .535; between access to textbooks and home possessions from r = .285 to r = .638. The lesson to be drawn is not that individual purchase is necessarily superior to central distribution. The lesson is that the criteria of success are not intentions; and that a central- ized system of distribution does not necessarily work in favor of equity. - 30 - Table 15 Relationships Between Acess to School Textbooks and Ugandan Primary School Socio-Economic Status /1 (N=36) Number of Textbooks Percentage of Textbooks Per Grade 7 Child Ordered from Central Accumulated Before Authority2Which Actually 1972 Arrived - Father's Educational Attainment .125 .535** Home Possessions Scale /3 .285* .638** * p < .05 ** p < .001 /1 Extracted from Heyneman, (1975). /2 Range = 0 to 229%. /3 Bed, newspaper, bicycle, radio, clock, car, television. 71. Chile's experience in the late 60's and early 70's may also be instructive. Approved textbooks were privately produced and distributed, but state subsidization kept the cost fairly low. For the private entre- preneur they were still a profit-making item. In 1971 one could go to very remote and poor villages and find a local kiosk with textbooks for sale. The question is whether it is better to have texts theoretically free to all but unavailable because of inefficiencies of the distributing monopoly, or avail- able to all but requiring an investment which though small, may still put them out of the range of the poorest students. Perhaps there is a way to combine the distributional efficiencies of the private sector with the poten- tial equity advantages of a more centralized intervention. - 31 - CONCLUSION 72. In sum the evidence available suggests that an investment in text- books will produce learning gains; and that this is more likely to occur as a result of a textbook investment than it is as a result of other educa- tional interventions such as teacher training. We do not know the exact conditions under which textbook effects occur. We do not know the relative rates of financial return for an investment in texts as compared to an invest- ment in other scholastic improvements, nor how to maximize that payoff under varying conditions. Efforts to gather more precise information would pay off. Because investment programs in textbooks are likely to increase, if we were to have more pre- and post-tests, and more information on what kind of teachers and other materials already existed in the affected schools, this would later allow us to conclude, in a less ambiguous fashion, that both the amount and the substance of our investments were wise choices. - 32 - REFERENCES Anderson, C. Arnold. "The Sorcerer's Apprentice: Education in Developing Nations," Comparative Education 6 (March, 1970), 5-18. Anderson, Richard B. et. al. Educational Experimentation: A Planned Variation Model. Cambridge, Mass.: Abt Associates, 1977. Anderson, Richard B., St Pierre, Robert G., Proper, Elizabeth C., and Stebbins Linda B. "Pardon Us, But What was that Question Again?: A Response to the Critique of Follow Through Evaluation." Harvard Educational Review 48 (May, 1978) 161-170. Beeby, C. E. "Stages in the Growth of a Primary Education System", Comparative Education Review, 6 (1), June 1962. Beebout, Harold S. "The Production Surface for Academic Achievement: An Economic Study of Malaysian Secondary Education". Unpublished Ph.D. dissertation, University of Wisconsin at Madison, 1972. Bennett, Neville; Jordan, Joyce; Long, George; and Wade, Barbara. Teaching Styles and Pupil Progress, Cambridge: Harvard University, 1976. Cisin, Ira H., Miller, Judith D., and Wirtz, Philip W. "Discontinuing Drug Use" Washington D. C. Social Research Group, George Washington University, February, 1976. Comber, L. C. and Keeves, J. P. Science Education in Nineteen Countries, New York: Wiley, 1973. Fuller, W. P. and Chantavanish, A. A Study of Primary Schooling in Thailand: The Final Report. Factors Affecting Scholastic Achievement of the Primary School Pupils. Bangkok: Office of the National Education Commission. 1977. Haron, Isahak bin, "Social Class and Educational Achievement in a Plural Society: Peninsular Malaysia", Ph.D. dissertation, University of Chicago, Chicago, Illinois, December, 1977. Heyneman, S. P. "Changes in Efficiency and Equity Accruing from Government Involvement in Ugandan Primary School Education", African Studies Review, April, 1975. Heyneman, S. P. "Differences in Construction, Facilities, Equipment and Academic Achievement among Ugandan Primary Schools", Inter- national Review of Education. (1977) - 33 - REFERENCES (Continued) Heyneman, S. P. "Influences on Academic Achievement: A Comparison of Results from Uganda and more Industrialized Societies", Sociology of Education, July, 1976. Heyneman, S. P. Plannin,g the Equality of Educational Opportunity Between Regions, Paris: International Institute of Educational Planning, (1978a). Heyneman, S.P. "Investment in Indian Education: Uneconomic?" Washington: International Bank for Reconstruction and Development (1978b) (mimeographed). House, Ernest R., Glass, Gene V., Mclean, Leslie D., and Walker, Decker F. "Critiquing a Follow - Through Evaluation." Phi Delta Kappan March, 1978,, p. 473-4. House, Ernest R., et. al. "No. Simple Answer: Critique of the Follow Through Evaluation," Harvard Educational Review 48 (May, 1978) 128-160. Husen, T. and others. "Teacher Training and Student Achievement in Less Developed Countries", Washington, D.C. The World Bank, January, 1978 (mimeographed). Inkeles, Alex "The International Evaluation of Educational Achievement" Proceedings of the National Academy of Education 4 (1977) 139-200. Johnson, Paul D. Predicting Adolescent Smoking Behavior with Stepwise Multiple Regression, Moderator Analysis and Interaction Detection Techniques, Unpublished Doctoral Dissertation, University of Minnesota, 1970. Oficina de Planeamiento y Organizacion (ODEPOR). The Relations of Social, Economic, and Cultural Factors and National Achievement Exam Results, El Salvador, Ministry of Education, February 1977. Orme, M. Effects of Modifying Teacher Interaction Skills on Student Behavior. Indiana University Institute for Child Study, 1970. Peaker, G.F. An-Empirical Study of Education in Twenty-One Countries: A TechnicaL Report, New York: Halsted Press, 1976. Purves, Alan C., Literature Education in Ten Countries. New York: Wiley, 1973. Schiefelbein, E. and Farrell, J.P. FFactores y Resultados del Proceso Educativo Chileno. Santiago: Ministerio de Educacion, 1971. - 34 - REFERENCES (Continued) Schiefelbein, E. and Farrell, J. P. "El Efecto de la Disponibilidad de Textos en el Rendimiento Escolar," Education Hoy: Perspectivos Latino-Americanos, 3 (13), 1973. Schiefelbein, E. and Farrell, J. P. "Expanding the Scope of Educational Planning: The Experience of Chile", Interchange 5 (2), 1974. Schiefelbein, E. and Farrell, J. P. "Social and Pedagogical Factors Affecting Survival in the Schools of Chile", Paper read at the annual meeting of the Comparative and International Education Society of Canada, Fredericton, New Brunswick, June 1977. Schiefelbein, E. and Farrell, J. P. "Selectivity and Survival in the Schools of Chile," Comparative Education Review 22 (2), 1978. Searle, B., Friend, J. and Suppes, P., "The Radio Mathematics Project: Nicaragua 1974-75", Stanford University: Institute for Mathematical Studies in the Social Sciences, 1976. Shukla, S. "Achievements of Indian Children in Mother Tongue (Hindi) and Science", Comparative Education Review" 18 (1974) 237-247. Simmons, John and Askoy, Sumru. "Schooling for Development? Students and Workers in Tunisia", Harvard University Economics Dept., 1972, (unpublished manuscript). Smart, M. Neff, "The Densu Times - Self-Made Literacy", Development Communication Report 21 (January, 1978). Smith, Marshall and Tucker, Mark. "A Research Perspective on Educational Improvement", Washington, D.C., Department of Health, Education and Welfare (October, 1977) (mimeographed). Sonquist, J. A. and Morgan, J. N. The Selection of Interaction Effects Monography Number 35, Ann Arbor: Survey Research Center, Institute for Social Research, University of Michigan, 1964. Spady, W. "Simple Techniques for Multivariate Analysis", Interchange 1 (3), 1970. Thorndike, Robert L. Reading Comprehension in 15 Countries, New York: Halsted Press, 1973. United States Office of Education. The Impact of Educational Innovation on Student Performance, Washington, D.C.: USHEW, 19776. Wolff, Laurence, "Why Children Fail First Grade in Rio Grande do Sul: Implications for Policy and Research," Rio de Janeiro: USAID, October, 1970 (mimeographed). APPENDIX Page 1 NOTE ON TECIINICAL TER41INOLOGY We will necessarily use a number of statistical terms. We assume most readers will be familiar with correlation coefficients (r) and means. To Some Regression coefficients may be less familiar. Briefly (and simply) in a regression analys:is one is attempting to assess the effect on student achievement of a number of variables, each independent of the effect of the other variables. The indicators of the strength of each predictor variable are variously labelled b coefficients, Beta coefficients, or regression coefficients. The differences between these terms are not important for present purposes. In each case, the higher the coefficient, the greater is the independent effect of that variable on student achievement2 In 2one table reported here regression results are presented as cumulative R . R is a measure of the proportion of all variance in student achievement accounted for by a2particular variable. Cumulated, they can be interpreted as follows. If the R for the first listed variable is .20, this means that viriable accounts for 20% of the achievement variance. If the cumulative R for the second variable is .27, then together the first two variables account for 27% of the achievement variance, and the second variable has added an additional 7% to the explanatory power of the first. Automatic Interaction Detector (AID) is a method of scoring and dividing predictors configurally. Unlike multiple regression which predicts a simple dependent variable while attempting to hold others in control, AID selects the best predictors, and then selects the best predictors of those predictors. At each step AID uses a variance procedure as the criterion for selection. The result looks like a pine tree laid on its side. The top, on the left, consists of the dependent variable, academic achievement or whatever. To the right of that lie the branches, the two best predictors, the four best predictors of those two, the eight best predictors of those four, etc. Listed with each predictor is the number of cases in that sub- category, and the mean score of those sub-category cases on the dependent variable. Developed originally by Sonquist and Morgan (1964), it has suc- cessfully illustrated how various categories of adolescents do (or do not) begin to smoke (Johnson, 1970), or use drugs (Cisin, Miller and Wirtz, 1976). In "between-student" analyses the student is the unit of analysis and each student's questionnaire responses and test scores are individually considered. In the words of Comber and Keeves by contrast, "In the between- school analyses the responses were scaled and then combined together using the school as the unit of analysis by averaging the data for individuals" (p. 193). If it happens that individuals within a school resemble each other more than they resemble members of the sample as a whole on some character- istic (i.e. if there is a clustering effect, as frequently happens for example with family background characteristics--poor children go to school with poor APPENDIX Page 2 children, rich children with rich children) then the between-school r's for that characteristic are likely to be higher than the between-student r's. This is known as the ecological effect. We have no evidence that this ecolo- gical effect is operating to any significant degree with respect to textbooks. In any event we shall treat the differences in between-school and between- student r's as "real". Commonality analysis is a modification of standard regression anal- ysis designed to correct for problems in interpretation of results created when predictor variables are not only associated with achieved scores but are themselves highly intercorrelated. Weighted net percentage differences (WNPD's) can best be thought of as an analogue of regression analysis when the criterion variable is not continuous (like test scores) but is categorized (e.g. either survived to 4th year secondary or did not). The higher the WPND for a given predictor, the greater its independent effect on the criterion. RECENT PAPERS IN THIS SERIES: No. TITLE OF PAPER AUTiOR 265 India's Population Policy: History and Future R. Gulhati 266 Radio for Education and Development: P. Spain, D. Jamison Case Studies (Vols. I & II) E. McAnany 267 Food Insecurity: Magnitude and Remedies S. Reutlinger 268 Basic Education and Income Inequality in J. Jallade Brazil: The Long-Term View 269 A Planning Study of the Fertilizer Sector A. Choksi, A Meeraus in Egypt A. Stoutjesdijk 270 Economic Fluctuations and Speed of Urbanization: B. Renaud A Case Study of K,orea 1955-1975 271 The Nutritional and Economic Implications of L. Latham, M. Latham Ascaris Infection in Kenya S. Basta 272 A System of Monitoring and Evaluation of M. Cernea, B. Tepping Agricultural Projects 273 The Measurement of Poverty Across Space: V. Thomas The Case of Peru 274 Economic Growth, 'Foreign Loans and Debt G. Feder Servicing Capacity of Developing Countries 275 Land Reform in Latin America: Bolivia, Chile, S. Eckstein, G. Donald Mexico, Peru and Venezuela D. Horton, T. Carrol (consultants) 276 A Model of Agricultural Production and Trade R. gorton. in Central America C. Cappi, L. Fletcher, C. Pomareda, M. Wainer (consultants) 277 The Impact of Agricultural Price Policies on M. Osterrieth, Demand and Supply, Incomes and Imports: E. Verrydt, J. Waelbroeck An Experimental Model for South Asia (consultants) 278 Labor Market Segmentation and the Determination D. Mazumdar of Earnings: A case Study M. Ahmed 279 India - OccasionaL Papers M. Ahluwalia, J. Wall S. Reutlinger, M. Wolf R. Cassen (consultant) No. TITLE OF PAPER AUTHOR 280 Educational Effects of Class Size W.D. Haddad 281 Relieving Traffic Congestion: The Singapore P.L. Watson Area License Scheme E.P. Holland 282 World Trade and the International Economy: B. Balassa Trends, Prospects and Policies 283 Urban Land Policy Issues and Opportunities H.B. Dunkerley, A.A. Walters J.M. Courtney. W.A. Doebele, D. Shoup, M.D. Rivkin (consultants) 284 Pakistan: Forestry Sector Survey S.A. Draper, A.J. Ewing. J. Burley, G. Grayum (consultants) 285 The Leisure Cost of Electric Power Failures M. Munasinghe 286 Shadow Pricing and Power Tariff Policy M. Munasinghe, J. Warford 287 Wages, Capital Rental Values and Relative S. Guisinger (consultant) Factor Prices in Pakistan 288 Educational Reform in the Soviet Union: I. Blumenthal, Implications for Developing Countries C. Benson (consultants) 289 Petroleum and Gas in Non-OPEC Developing R. Vedavalli Countries: 1976-1985 290 M-ajor Reforms of. the Swedish Education System A. Heidenheimer (consultant) 291 Industrialization, Technology and Employment - T.G. Rawski (consultant) China 292 Development and Income Distribution - Zambia C. Blitzer 293 World Potash Survey W.F. Sheldrick, H. Stier 294 The Economic Dimensions of Malnutrition in M. Selowsky Young Children 295 The Technology of Rural Development J.P. McInerney (consultant) 296 The Financial Cost of Agricultural Credit: C.D. Datey (consultant) A Case Study of Indian Experience 297 Agricultural Sector Planning Models: A.C. Egbert A Selected Summary and Critique PUB HG3881.5 .W57 W67 no.298 Heyneman, Stephen P. ,Textbooks and achievement : what we know / PUB HG3881.5.W57 W67 no.298 Heyneman, Stephen P. Textbooks and achievement : what we know /