World Bank Reprint Series: Number 335 REto3v Stephen P. Heyneman, Dean T. Jamison, and Xenia Montenegro , / 3 Textbooks in the Philippines Evaluation of the Pedagogical Impact of a Nationwide Investment FILE COPYj Reprinted with permission from Educational Evaluation an lolicy Anali voL 6, no. 2 (Smmer 1984) pp. 139-50, published by the American Educational Research Association and University of Califomnia, Los Angeles. .5.o@| - - ; - World- Bank Repn t No. 294. Barend A. de Vries, 'International Ramifications of the External Debt Situation," The AMEX Bank Review Special Papers No. 295. Rakesh Mohan, "The Morphology of Urbanisation in India," Economic and Political Weekly No. 296. Dean T. Jamison and Peter R. Moock, '"armer Education and Farm Efficiency in Nepal: The Role of Schooling, Extension Services, and Cognitive Skills," MArld Development No. 297. Sweder van Wijnbergen, 'The 'Dutch Disease': A Disease after All" The Economic journal No. 298. Ame Drud and Wafik M. Grais, 'Macroeconomic Adjustment in Thailand: Demand Management and Supply Conditions," Journal of Policy Modeling No. 299. Shujiro Urata, "Factor Inputs ,nd Japanese Manufactuning Trade Structure," .The Review of Economics and Statistics No. 300. Dipak Mazumdar, 'The Rural-Urban Wage Gap Migration and the Working of Urban Labor Market: An Interpretation Based on a Study of the Workers of Bombay City," Indian Economic Review No. 301. Gershon Feder and Roger Slade, "Contact Farmer Selection and Extension Visits: The Training and Visit Extension System in Haryana, india," Quarterly Journal of Interna- tional Agriculture No. 302. James Hanson and Jairne de Melo, 'The Uruguayan Experience with Liberalization and Stabilization, 1974-1981," Journal of Interamerican Studies and Wbrld Affairs No. 303. Nancy Birdsall and Dean T. Jamison, 'Income and Other Factors Influencing Fertility in China," Population and Development Review No. 304. Graham Donaldson,'Food Security and the Role of the Grain Trade," American Journal of Agricultural Economics No. 305. William F. Steel and Yasuoki Takagi, "Small Enterprise Development and the Employment-Output Trade-Off," Oxford Economic Papers No. 306. Oi Havrylyshyn and Engin Civan, 'Intra-Industry Trade and the Stage of Develop- ment: A Regression Analysis of Industrial and Developing Countries," Intra-Industry Trade: Empirical and Methodological Aspects No. 307. Mateen Thobani, 'A Nested Logit Model of Travel Mode to Work and Auto Ownership," Journal of Urban Economics No. 308. Johannes Bisschop and Alexander Meeraus, "On the Development of a General Algebraic Modeling System in a Strategic Planning Environment," A4athematical Programming Study No. 309. Reynaldo Martorell, Joanne Leslie, and Peter R Moock, "Characteristics and Deter- minants of Child Nutitional Status in Nepal," The American Jounal of Clinical Nutrition No. 310. Robert H Litzenberger and Jacques Rolfo, 'An Intemational Study of Tax Effects on Government Bonds," The Joural of Financ. No. 311. Jle R. Behrman and Nanicy Birdsall" The Quality of Schoolng Quantity Alone is Misleading, Amecan Economic Review No. 312. Bela Balassa, 'Adjustment Policies in Developing Countries: A Reassessment," Wbrbd D-velopment No. 313. Keith Marsden, "Services for Small Finns: The Roles of Goverment Prog es and Market Networks in Thailand," Ienatial Labour Reiew. Educational Evaluation and Policy Analysis Summer, Vol. 6, No. 2, 1984, pp. 139-150 Textbooks in the Philippines: Evaluation of the Pedagogical Impact of a Nationwide Investment Stephen P. Heyneman and Dean T. Jamison The World Bank and Xenia Montenegro Arlington (VA) The contribution of learning to eco- education increased from 48% of the nomic growth has long been recognized, school-age population in 1960 to 70% in although precise measurement of its ef- 1977. There are, moreover, about 50% fect is still subject to debate. Parental de- more children enrolled in grade 1 than in mand for educational opportunity, cou- grade 4, so this 70% enrollment figure pled with the economic and political in- significantly understates the proportion of cer lives for authorities to supply it, ac- children who begin school. count for the recent expansion of formal Despite the significant advances made schooling to unprecedented levels. Uni- in providing "places," primary schools in versal primary school enrollment has now the developing countries generally have been achieved in 35 of the richer devel- poor resources and therefore are ineffec- oping countries since World War II, in- tive in passing on the complex cognitive cluding Argentina, Gabon, Malaysia, and skills required of school leavers. Inade- Trinidad and Tobago. In the 36 poorest quate teaching environments result in countries (whose per capita gross national part from the fact that about 95% of the product [GNP] was US$265 a year or less funds available for primary education is in 1975), average enrollment in primary allocated to salaries. Paying a teacher to copy a wornout textbook onto a black- board and to supervise its memorization by 50 students is an ineffective use of, in This analysis is part of the World Bank's research relative terms, expensive talent. (This is project "Textbook Availability and Educational now widely referred to as the "copy-copy" Quality." The data were collected as part of the -situation.) The question is, What level of Philippine government's evaluation of the Third Ed- classroom resources should teachers have ucation Project, assisted by the World Bank (Loan avibltocmeenthrtmenp? 1224-PH). The authors are indebted to Augusto Ten- available to complement their time input? matay, Pic Aprieto, and Jesus Ochave for valuable In industrial countries today, 14% of pri- advice concerning the textbook project and its eval- mary school recurrent costs are allocated uation. and to Kalpana Nandwani for assisting with to classroom resources (books, maps, vis- computer analysis. However, the views and inter- ual aids, furniture, etc.) and 86% is spent pretations expressed here are those of the authors . . . and do not necessarily reflect the views of the World on salaries. The average in Asia IS 9% and Bank or the Philippine governnient. 91 % for salaries, and in Africa 4% and 139 140 Heyneman and Jamison 96% for salaries.' One policy 'dilemma primary school construction and/or qual- that developing countries are now con- ity are being made in Columbia, Para- fronting with regularity is determining guay, the Philippines, Thailand, Brazil, what is the minimum standard of re- and Indonesia as well as Malawi, Sierra sources a primary school should have be- Leone, Papua New Guinea, Bangladesh, fore new teachers are taken on. Experi- and Pakistan. The one conclusion we are ence has suggested that, at a minimum, able to draw from this experience is that expenditures on teaching tools should ac- high rates of school attendance are only count for approximately 10% of public the first among many steps toward provid- recurrent expenditures. But the question ing a sufficient human capital base; as remains: Given sharp limitations on the long as there are substantial differences availability of the requisite capital, what across countries in the quality of school are the most efficacious approaches to classrooms there will be a need (and de- improving school quality? mand) for improvements in school quality (Heyneman, 1983). The Demand for Investments in School Quality The international demand for invest- Research on School Quality ments in school quality has risen mark- Investments edly over the last decade. For example, School quality research in developing not one of the 31 education projects ap- countries has been handicapped in three praised by the World Bank between 1963 ways. Reviews of the literature have had and 1969 contained support specifically to be limited to the research findings cur- for classroom materials. Since 1976, how- rently available, which are unacceptably ever, the provision of classroom materials scarce and often of low quality. A careful (electronic or print) has become a major search for studies that used correlations, component in 2 out of 10 projects; inser- regressions, or differences in means (i.e., vice training has become a major compo- those that had potential for comparison), nent in 1 out of 10 projects; and preservice located fewer than 30 published sources training in 1 out of 2 projects. The demand on the cognitive impact of teacher train- has been particularly pronounced with . . . respect to primary education. The portion igi o n ideicm onre ofeWorlBank education led pn dorte over the last 20 years. This stands in com- to primary education wa only %e- parison with almost 400 such studies pub- o p lished in the United States in 1 year alone. tween FY70-74. It rose to 14% between Similar imbalances exist with respect to FY74-78 and is projected to be 24% be- textbooks, duplicating machines, visual tween FY79-83. This represents an in- . vetetinraeo from US$8 milio aids, radio, television-in fact, with re- vestment bncrease of Yrom Ua8 mFllion spect to all school resources. Second, what pr yr b e FY adata have been available have been al- US$144 million per year between FY79- most without exception cross sectional. 83. The source of this demand, further- Conclusions have to be based on differ- more, is not at all confined to the lowest ences within static samples. Recommen- income borrowers. Major investmen'ts in dations for investment have to be based on assumed changes in achievement and/ or earning that could occur rather than on changes observed as a result of actual 'A comparison of percentages does not reflect investment. Third, where there has been real differences in the level of investment per stu- experimentation and useful analysis of dent. For example, before the Third Education Proj- gain scores as a result of an intervention, ect was appraised, the average pupil:book ratio in the Philippines was 10:1. According to the National for reasons of cost and logistics the exper- Center for Educational Statistics a similar figure for iments have been confined to relatively the United States at the same grade level was about small samples (Heyneman, Farrell, & Se- 1:14; that is, 14 library books (this is in addition to pulveda-Stuardo, 1978; Heyneman & Ja- textbooks and reference books) for each pupil. Thus . 18 T the typical pupil in the United States had, at a mison, 1980). This leaves open the ques- minimum, 140 times the reading material available tion of whether this effectiveness could in school. be replicated on a nationar scale even Textbooks in the Philippines 141 within the same country where experi- Because books were expected to be in use mentation occurred. Our purpose in this for up to 5 years, binding and stitchiiig paper is to repiort the results of an evalu- were of high quality. Distribution was ation of a large-scale investment in school made first to i-egional warehouses, then quality improvement: the Philippines to municipalities, then to schools. Arrival Textbook Project. The results illuminate time and textbook condition in the na- not only the strong positive impact of the tion's 100,000 schools were monitored by project, but also the possibility of assess- computer in the capital. The success of ing quantitatively the impact of a major the project was due not to simply having educational intervention. new textbooks, but rather to having new textbooks of high quality, reasonably on The Philippinle Textbook Project time, well understood, and well used by The Philippine government, with the teachers. assistance of the World Bank, launched The Evaluation Section of the Textbook the $US37 million Textbook Project to Project was formed primarily to investi- alleviate the lack of textbooks in the na- gate the effects of this sizable investment tion's public schools. Prior to this invest- on student achievement. One year before ment, there was an average of 10 pupils the first sets of textbooks were distrib- for every book available in a given sub- uted, an evaluation plan was designed, ject. During the first year of implementa- and instruments were developed to meas- tion, the Textbook Project produced ap- ure these effects. The plan was imple- proximately 20 million first and second mented during the first year of textbook grade textbooks in science, mathematics, distribution and has been a continuing and Pilipino. These were distributed na- component of the Textbook Project. The tionwide in the 1977-78 school year at a results presented here were obtained ratio of two pupils per book. In the course from the first-year evaluation data, oth- of its 5-year duration, the Textbook Proj- erwise known as Phase I. The effects of ect would produce 97 million books cov- textbooks on achievement in conjunction ering all subject areas from first grade with pupil, home, school, and environ- through high school and would distribute mental variables were examined, the pri- enough so that there would be only two mary concern being the increment in the students per book at each grade level in achievement of pupils, given the use of each subject. Books cost an average of the project textbooks, while other influ- US$.55 each (for an average length of 180 encing factors were controlled. pages); the overall program increased per Methodology student costs by about 1% (Orivel, 1979). Teacher training in the use of the books Treatment and Comparison Conditions was part of the project. These and other The design for the evaluation of the aspects of implementation are described Textbook Project called for a comparison in detail by Aprieto (1983). of two groups: those with textbooks and Textbook content was geared closely to those without. Simultaneous nationwide the national curriculum and tightly distribution of books in each subject at screened for clarity of presentation. Photo . each grade precluded assignment of stu- layouts and colors were used judiciously. dents to contemporaneous experimental and control groups. The closest control available was the previous year's stu- dents. A random sample of grade 1 and 2In a study related to the one reported here (Ja- grade 2 students selected one school year mison, Searle, Galda, & Heyneman, 1981) a small before textbook distribution served as a group of Nicaraguan primary schools was randomly assigned to receive textbooks, to receive radio in- control group, referred to as the nonpro- struction, or to serve as controls. The subject was ject group. Another sample of grade I and mathematics. Random assignment and careful mon- grade 2 students was selected during the itoring of interventions allowed a particularly tight school year of textbook distribution, re- evaluation. What remains is to see if similar gains ferred to as the project group. The aca- can be acquired through the implementation of a project in a large and geographically diverse national demic achievement of sample project setting. group pupils in science, mathematibs, and 142 Heyneman and Jamison Pilipino at the end of the project year was ect at the grade 2 level was redluced by compared with the achievement of sam- 12% due to problems in textbook distri- ple nonproject group pupils the year be- bution. Nevertheless, we are confident fore that. Thus, the project group is the that the nonproject sample roughly rep- treatment group, and the nonproject resents the typical grade 1 and grade 2 group is the comparison group. child in the Philippines prior to the mas- There were two treatment conditions sive governmental investment in text- for both grade levels of the project group. books, and that the project group roughly The first followed the general distribution represents the typical grade 1 and grade 2 of a 2:1 pupil textbook ratio; the second child 1 school year after that investment. derived from a randomly selected sub- sample of project schools provided with Instruments books on a 1:1 pupil/textbook ratio so that In both grades, achievement tests were student performance on the two treat- administered in science, mathematics, ment conditions could be compared. and Pilipino. The grade 1 test consisted of 25 items, and the grade 2 test consisted of Sampling 30 items. The curriculum objectives of the A two-stage procedure was used to ob- Ministry of Education and Culture are tain-the sample for both project and non- uniform throughout the Philippines, and project groups. Specifically, two school these tests were developed to encompass divisions were randomly selected from these objectives. (For a more detailed de- each of the 12 geographical regions in the scription of test development, see Text- Philippines (excluding metropolitan Ma- book Board Secretariat, 1978.) In addition nila). Within divisions, schools were strat- to the achievement tests, a questionnaire ified according to central and barrio cat- was designed to gather pupil-specific in- egories, and random samples of 30% and formation, but because pupils were too 70% were then obtained. The number of young to answer questions on their own, schools selected from each region was de- other arrangements had to be made. termined in proportion to the size of the Qnestionnaires were first translated into region. The clustering of the sample nine local languages. Classroom teachers within two divisions per region resulted sent the forms home with the children, from a concern for logistics and manage- whose parents or guardians were asked to ment. (Definitions for all variables, such respond. In some instances teachers sup- as central and barrio schools, can be found plied the required information from in Appendix 1.) school records. Teachers also were asked A random sample of 32 pupils was se- for information separately. Occupations lected from all sections (homerooms) in of the father and mother were divided grades 1 and 2 in the sample schools. This into 16 categories; parents' educational reduced bias introduced by homogeneous level was obtained by requesting the high- homeroom groupings. Where schools had est school grade completed. Family in- only one section in the grade level, the come was categorized into nine levels, entire class was administered the instru- ranging from below 250 pesos per month ments. In small sample schools an entire to above 2,000 pesos per month. Informa- class may have had less than 32 students. tion on the language spoken in the pupil's In the first grade sample 1,547 project home also was gathered. group pupils from 52 schools and 1,652 Appendix 1 contains definitions of the nonproject group pupils from the same variables included in the regression anal- number of schools were tested. Of the yses presented here. The variables de- grade 1 project group only 1,204 pupils scribing treatment condition, region, and 'were administered the Pilipino test be- language are all indicator variables that cause there were 11 sample schools that for each student take on the value 1 if the did not receive Pilipino textbooks in time. student is in the indicated treatment con- There were 1,191 project group pupils dition (region, language group) and take from 46 schools and 1,634 nonproject on the value 0 otherwise. Means and -group pupils from 52 schools in the grade standard deviations are shown in Appen- 2 sample. The school sample for the proj- dixes 2 and 3 (grades 1 and-2, respec- Textbooks in the Philippines 143 tively). The percentages for various sub- ficients comparing project to nonproject samples may not add up to the total sam- group were statistically significant at the ple because of missing data. Table I dis- .001 level. Table I presents the full mul- plays the regression results for science tiple regression results for science, and achievement using two specifications. Table II presents the overall pattern of Model 1 includes the project group in results. Table II demonstrates how many comparison with the nonproject group,3 standard deviations better on the posttest alcng with other pupil and region/ a student could be expected to perform if language indicator variables. Model 2 he or she were in the Textbook Project. breaks tne project group into two catego- (Table II shows the coefficient of project ries, one that had textbooks at a 2:1 pupil/ effectiveness. This is the ratio of the textbook ratio (project group 2), and one change in the number of problems an- with a 1:1 pupil/textbook ratio (project swered correctly for being in the project group 1). group to the standard deviation in the Results posttest score.) In sum, the overall effect of the first Our analysis addressed two questions. year investment in textbooks was to raise The first was the degree to which student the national level of academic achieve- achievemenit was altered as a result of ment by one-third of a standard deviation exposure to new textbooks, a level of ex- in grade 1 Pilipino, by a similar magnitude posure that reduced the ratio of pupils per in grades 1 and 2 mathematics, and by book per subject from an average of 10:1 almost one-half of a standard deviation in to 2:1. In addition to the effects of the grades 1 and 2 science. This is a sizeable standard intervention, we discuss the dif- impact. The .51 change in standard devia- ferential effects of a 2:1 student-io-text- tion in first grade science, for example, book ratio compared to the 1:1 ratio. implies that the mean score achieved by Our second interest was the question of 50% of the population was achieved by who benefits most from the introduction 69% of the population a year later. In of new textbooks. Do new textbooks have seconcgrade mathematics, the .32 change the most pronounced cognitive impact on in standard deviation implies that what those students whose homes are more ad- was achieved by 50% of the population vantaged and who are more familiar with without the new textbooks was achieved the use of educational materials; or do by 63% of the population a year later. they have the most pronounced effect on These learning gains represent some of students from homes of relative poverty the highest we have observed; moreover, and who have had the least previous ex- they reflect not a few experimental class- perience with reading materials? rooms, but gains achieved by 8 million students in the nation's schools. Meas- General Intervention Effects ured in units of standard deviation, the All achievement test scores in science, alteration. in science achievement in the mathematics and Pilipino were strongly Philippines is twice the impact of what influenced by membership in the Text- Glass and Smith (1979) conclude would book Project. For all six of the project to be the result of reducing the size of the nonproject comparisons (three subjects average class in North America from 40 and two grade levels), the regression coef- students to 10. Further improvements were not ob- served when the 1:1 pupil-textbook ratio 'In the initial analysis the project group sample and the 2:1 treatment condition were was subdivided into sample students (25%) who had compared. In science, the 2:1 condition access to the books for only half a school year (21 regressin sciene, wee slitly weeks or fewer) and those who were able to use the books for longer than 21 weeks. The differences in higher than the 1:1 condition at both length of exposure to the textbooks was an unfore- grades. In mathematics, the reverse was seen event caused by problems in textbook distri- true at the first grade, although the differ- bution. When length of textbook use was included ence was small (1.58 and 1.13). In second in the regressions, the effects of this variable turned ence was small (1.58tan 1.13). In rse out to add little to the analysis, and was thus ex- grade mathematics, slightly higher regres- cluded from further consideration. sion coefficients (1.646 vs. 1.365) were ob- 144 Heyneman and Jamison tained for the 2:1 condition. In Pilipino, science books were better; (b) the absence both first and second grade pupils in the of science books may have been more 2:1 condition did better than those in the detrimental to learning than the absence 1:1 conidition. In fact, the regression coef- of mathematics or Pilipino books; and (c) ficient for the 1:1 condition in grade 2 there was better use of science books by Pilipino was not statistically significant. teachers, that is, teachers may have been Whatever the cause, the results provide better trained or motivated in the use of no indication that the 1:1 ratio of pupils the science books. to books added any significant impact There is some evidence, at least in the over the 2:1 ratio in the first and second case of Pilipino, that the pupils may have grades, despite the near doubling of costs had difficulty in the use of the new books, that the 1:1 ratio would entail. which could have hindered their per- The strongest overall effects were seen formance. A small survey of 64 randomly in science at both the grades. The weakest selected grade 2 teachers was conducted project effects were seen in grade 2 Pili- with the purpose of getting their assess- pino. Several hypotheses may be ad- ment of the new books after having used vanced to explain the better response of them. Comments were largely positive, pupils to new textbooks in science: (a) The but more teachers gave unfavorable com- TABLE I Multiple Regression kesults: Unstandardized Regression Coefficients for Science Achievement in Grades 1 and 2 Grade 1 Grade 2 Model 1 Model 2 Model 1 Model 2 Varable (N = 2,246) (N = 2,141) (N = 2,158) (N = 2,098) Project group 2.39... 2.46"'. (13.57) (11.10) Project group 1 2.33' -* (8.51) (6.40) Project group 2 - 2.39"** - 2.74"-* (11.45) (10.68) Pupil sex -.12 -.17 .37 .25 (.74) (1.03) (1.864) (1.29) Pupil age .10 .13 -.13 -.49 (1.12) .(1.34) (1.307) .51 Parent education .24*" .24..' .25"*' .24"*' (8.83) (8.46) (7.463) (7.49) Regions" 1 (constant) 2 2.18-" 2.16*"' .71 .50 (4.85) (4.82) (1.329) .94 3 4.90"* 4.88"* 3.14**" 2.98**' (9.69) (9.68) (5.251) (5.02) 4 1.53" 1.57** -.65 -.14 (2.78) (2.89) (.095) (.17) 5 6.32*" 7.27*"* -.65 -.83 (3.97) (4.29) (.390) (.51) 6 -1.43' -1.01 -1.13 -1.32 (2.15) (1.53) (1.378) (1.62) 7 -1.90"* - 1.58*" -1.35 -1.48 (2.65) (2.15) (1.488) (1.64) 8 2.15' 2.36* -4.83*** -4.94*** (2.25) (2.45) (3.774) (3.93) 9 4.27*" 4.28**' .95 .79 (6-85) (6.93) (1.184) (1.00) Textbooks in the Philippines i 45 ments regarding the Pilipino than the this national assessment of students mathematics or science books (36% for emerged the fact that parental income and Pilipino, 2% for mathematics, and 10% for education- and student achievement were science). The teachers reported that the positively and highly intercorrelated. Pilipino books were too difficult (24%), The SOUTELE Survey, of course, is not had long stories that confused the stu- the only source to suggest that a child's dents (9%), or were inappropriate for the socioeconomic status affects learning. The grade level (Textbook Board Secretariat, fact is that socioeconomic status is per- 1978). haps the single best predictor of achieve- c oment in Europe and North America, and Effects on Children with Different is close to becoming an expected tendency Social Backgrounds in the social sciences. Recently, however, The most recent and perhaps most com- it has been noticed that the strength of prehensive survey of student achieve- the relationship seems to diminish in par- ment in the Philippines at the time of the allel fashion with national per capita in- Textbook Project Evaluation Study was come. It ranges from being strong and the Survey of the Outcomes of Philippine consistently positive in high income Education (SOUTELE, 1974-1977). From countries to being null or even slightly TABLE 1 (Continued) Grade 1 Grade 2 Model 1 Model 2 Model 1 Model2 Variable (N = 2,246) (N - 2,141) - (N = 2,1-8) (N = 2,098) 10 37 .63 3.22'** 300' (.55) (.93) , (3.747) (3.49) 11 1.88... 1.86* -.16 -.24 (2.73) (2.57) ,(.184) (.27) 12 4.08"' 1.66 3.04"* 2.72w (5.26) (1.78) (2.910) (2.61) Languages' 9 (constant) - - 1 -4.90*" -5.95**' 1.84 1.82 (3.18) (3.60) (1.174) (1.19) 2 .83 .42 -.63 -.59 (1.47) (.71) (.910) (.88) 3 .17 -.12 .41 .39 (.03) (.00) (.616) (.61) 4 .59 .51 -.98 -.93 (1.24) (1.05) (1.810) (1.75) 5 -2.45**" -.2.43"'. -3.51 "'- -3.65** (3.66) (3.67) (4.686) (4.96) 6 -1.63 -1.81 (1.83) (2.02) 7 -1.28 1.85 5.05*** 5.03**" (1.16) (1.02) (4.591) (4.68) 8 -1.67 -.82 -3.02* -2.95' (1.69) (.75) (2.518) (2.43) 10 -1.73* -1.70*" -.44 -.45 (4.66) (4.71) (.978) (1.01) Note. t values are in parentheses. " All regions and languages have been assigned random numbers. 'p <.05. p < .01. p < .001. 146 Heyneman and Jamison TABLE If Summary of the Impact of the Textbook Project Grade level Subject 1 2 Science .51 .46 Mathematics .30 .32 Pilipino .32 .18 Note. Entries in the table show the number of standard deviations improvement in posttest score expected to result from a student's being in the Textbook Project rather than not. negative in low income countries (Heyne- ularly evident in mathematics and sci- man, 1976a, 1976b, 1980; Heyneman & ence and less so with respect to Pilipino. Loxley, 1982, 1983). Among nonproject students, the correla- From these Textbook Evaluation data it tion between family income and science appears that mother's education, father's achievement is .16. Among project stu- education, and family income do emerge dents, the correlation drops to .05. In with positive zero-order relationships mathematics it drops from .14 to .08, and with pupil (posttest) scores in mathemat- in Pilipino from .19 to .09. The size of the ics, science, and Pilipino. Zero-order cor- drop is not uniform across subjects or relations range from r = .14 to .24 (p < measures of social status. It is least uni- .001). These relationships are stronger form with respect to changes in Pilipino than what one might expect to find in performance and parental education. Pa- Africa or India; weaker than similar rela- ren'tal education appears to be equally tionships in Western Europe, and approx- associated with Pilipino performance in imately the same to what is found in both project and nonproject groups. The Indonesia or Thailand. drop is most noticeable and most consis- The question is whether the interven- tent across different measures of social tion has more effect on children of privi- status in the case of science achievement. leged social backgrounds or more effect Textbooks appear to have had their most on children of impoverished social back- pronounced effect on science scores of grounds. There are hypotheses to support low status children. either. Children from privileged back- Second, there are substantial differ- grounds might take advantage of the in- ences in the predictive power of socioec- tervention more efficiently because they onomic status depending on whether the might be more familiar with books and dependent variable is cross sectional ac- how to use them. Alternatively, children ademic performance or a gain in academic from impoverished backgrounds might performance.4 A gain in performance, it take advantage of the intervention more must be noted, was due to more than the efficiently because they are more de- textbook intervention. Gains are due to prived and because educational materials are more scarce in their lives, and conse- quently they may appreciate more the Only grade 2 data were included because only privilege of having them. the grade 2 students had p subsample in the project The -evidence suggests that the inter- group that was pretested, thus making gain scores vention was more effective on children available on the same individuals. At the same time, whose back grounds were impoverished. there did not seem to be any dramatic difference First, the strength of intercorrelations be- between the grade 1 and grade 2 groups in terms of tween achievement and social back- the influence of pupil, home, school, and other var- tweeniables on academic achievement. Definitions for pre- ground is significantly weaker for chil- and posttest scores used to calculate gains in dren in the project group. This is partic- achievement can be found in Appendix 1. Textbooks in the Philippines 147 good teaching, to exposure to radio, in ing itself is scarce, where classrodms are some cases television, and in all cases to impoverished, where the motivation for a change in physical and mental maturity education is high from both privileged between the beginning and end of the and underprivileged families, a physical school year. Nevertheless, the influence alteration in the quality of a school class- of a child's socioeconomic status, a}- room can have sizeable effects. This ap- though a powerful predictor of cross sec- pears to be true in the Philippines. More- tional performance, appears to be mini- over. there the effect of a school quality mal, at best, in the prediction of gain intervention appears to be the most pro- scores of students within the project nounced among the children who are group. The only significant influence of most imnpoverished and whose home parental education on gain scores, con- backgrounds are the most underprivi- trolling for other influences, is that on leged. science and 'there the coefficient is slightly negative (beta = -.10, p < .01), 5Parental education (measured as the sum of both that is, higher parental education is parents' level of education) was made the measure slightly indicative of lower science gain of socioeconomic status because it correlated highly scores.' This anomoly is substantiated by with family income and other home variables. Initial cross tabulations between gain scores in regression runs have shown that when antered into science and father's education. Children the equation before other SES variables, parental fromhoms were he athr neer in- education accounted for most of the SES variance in from homes where the father never fin- the dependent variables. ished primary school gained twice as " Are these results subject to "ceiling effects"? much in science (4.0 points) as did chil- *That is, are they a result of pupils who begin with dren whose fathers finished college (1.97 high scores and who cannot markedly improve be- cause of their original standings? If this were the points)." case, pupils who scored low on pretests should then only equal those who scored high on the pretests. Summary But low pretest-scoring pupils surpassed many high pretest-scoring pup'ils on the posttest. Furthermore, Ambiguity can be found in North of the 30 items employed, the highest school mean America as to whether an improvement scores only went to 23. still below the maximum, in physical resources can affect signifi- though there were pupils who did obtain a score of cantly the quality of educational output. 30. This suggests that the substantial effort that went t te q o into pretesting ultimately insured that an appropri- Such ambiguity should not pertain to de- ate range of difficult levels was contained in the veloping countries. There, where school- tests. APPENDIX 1 Definitions of Variables Achievement test scores Grade 1 science Scores on 25-tem science test Grade 1 mathematics Scores on 25-itern mathematics test Grade 1 Pilipino Scores on 25-itern Pilipino test Grade 2 scice Scores on 30-item sciew test Grade 2 mathematics Scores on 30-tem mathematics test Grade 2 Pilipino Scores on 30-item Pilipino test Treatment conditions Nonproject group Grades 1 and 2 pupi sample during the school year immediately preceding textbook distribution Project group Grades 1 and 2 pupil sample during the school year of textbook distribution Project group 1 Project group pupils provided books on a 1:1 pupi to textbook ratio Project group 2 Project group pupils provided textbooks on a 2:1 pupil to textbook ratio Regional and language indkators All regions and languages have been assigned random numbers. Other variables Schoo location 0 = Central Main school in a town 1 = Barrio School outside a town, usualy in an outying area or in barrio 148 Heyneman and Jamison APPENDIX 1 (Continued) Pupil sex 1 = Male, 0 = Female Pupil age Measured in years Parental Education Average number of years of combined father's and mother's education (range = 1 to 17) Grade 2 pretest Achievement tests administered to a subsample of grade 2 students in the project group at the beginning of the school year. Grade 2 posttest Achievement tests administered to the full sample of grade 2 students in the project group at the end of the school year. Grade 1 posttest Achievement tests administered to the full sample of grade 1 students in the project group at the end of the school year. APPENDIX 2 Variable Means and Standard Deviations for Grade 1 Nonproject Project group Project group Total sample Project group group 2:1 1:1 Variable X SD X SD X SD X SD X SD Science score 13.11 4.70 14.10 5.06 12.18 4.13 13.93 5.08 13.21 3.90 Math score 12.33 4.64 12.82 4.81 11.87 4.43 12.21 4.75 12.24 4.04 Pilipino score 13.49 5.31 14.57 5.59 12.72 5.08 14.09 6.76 14.33 4.97 Project membership .40 Project group = 1 Nonproject group = 0 Project group .13 1:1 = 1 Other = 0 Project group 2 .23 2:1 = 1 Other = 0 Pupil sex (0 = female) (1 = Male) .49 .52 .47 .53 .50 Pupil age (in years) 8.18 1.0 8.17 .98 8.19 1.01 8.22 .96 8.19 .98 School location (0 = central) (1 = barrio) .36 .38 .35 .31 .48 Average parental education (1 to 17 years) 6.85 3.28 6.97 3.3 6.75 3.24 6.70 3.02 7.16 3.50 APPENDIX 3 Variable Means and Standard Deviations for Grade 2 Nonproject Project group Proct group Total sample Project group group 2:1 1:1 Variable X SD X SD X SD X SD X SD Science score 16.89 5.39 18.28 6.23 15.88 4.42 17.39 5.38 19.82 6.35 Math score 14.03 5.77 14.21 7.07 13.88 4.46 13.80 6.12 14.35 7.88 Textbooks in the Philippines 149 * APPENDIX 3 (Continued) Nonproject Project group Project group Total sample Project group group 2:1 1:1 Variable X SD X SD X SD X SD X SD Pilipino score 15.07 6.22 14.87 7.26 15.23 5.25 14.96 6.46 14.73 8.09 Project membership .38 Project group = 1 Nonproject group =0 Project group .13 1:1 = 1 Other = 0 Project group 2 .24 2:1 = 1 Other = 0 Pupil sex (0 = female) (1 = male) .51 .51 .50 .50 .50 Pupil age 9.01 1.15 8.67 .98 9.25 1.20 8.62 1.04 8.65 .89 (in years) School location (O = central) (1 = bamo) .32 .30 .34 .38 .28 Average parental Education (1 to 17 6.78 3.21 6.92 3.31 6.67 years) 3.14 7.36 3.510 6.74 3.13 References academic achievement across high and low in- come countries: A reanalysis of IEA data. Soci- ology of Education, 55, 13-21. APRIETO, P. (1983). The Philippine Textbook Project. HEYNEMAN, S. P., & LOXLEY, W. (1983). The impact of Prospects, 13, 351-359. primary school quality on academic achieve- GLASS. G. V, & SMITH. M. L. (1979). Meta-analysis of ment across 29 high and low income countries. research on class size and achievement. Educa- American Journal of Sociology, 88, 1162-1194. tional Evaluation and Policy Analvsis, i, 2-15. HEYNEMAN, S. P., FARRELL, J. P., & SEPULVEDA- GRANT, W. V., & EIDEN, L. J. (1982). Digest of educa- STUARDO, M. (1978). Textbooks and achieve- tion statistics. Washington, DC: National Center ment: What do we know? (Staff Working Paper for Educational Statistics. No. 298). Washington, DC: World Bank. HEYNEMAN, S. P. (1976a). A brief note on the rela- HEYNEMAN, S. P. (1983). Improving the quality of tionship between socioeconomic status and test education in the developing countries. Finance performance among Ugandan primary school and development, 20, 18-21. children. Comparative Education Review, 20, JAMISON, D. T., SEARLE, B., GALDA, K., & HEYNEMAN, 42-47. S. P. (1981). Improving elementary mathematics HEYNEMAN, S. P. (1976b). Influences on academic education in Nicaragua: An experimental study achievement: A comparison of results from of the impact of textbooks and radio on achieve- Uganda and more industrialized societies. Soci- ment. Journal of Educational Psychology, 73, ology of Education, 49, 200-211. 556-567. HEYNEMAN, S. P. (1980). Differences between devel- OORIVEL, F. (1979). The cost of the Philippines Text- oped and developing countries: Comment on book Project (MMT Working Paper). Paris: Simmons and Alexander's determinants of UNESCO. school achievement. Economic Development SOUTELE. (1974-1977). The outcomes of Philippine and Cultural Change, 28, 403-405. elementary education: A report of the major find- HEYNEMAN, S. P., & JAMISON, D. T. (1980). Student ings of project SOUTELE Phase I and II. Makati, learning in Uganda: Textbook availability and Philippines: Educational Development Projects other factors. Comparative Education Review, Implementing Task Force. 24, 206-220. Textbook Board Secretariat. (1978). Evaluation sec- HEYNEMAN, S. P., & LOXLEY, W. (1982). Influences on tion report. Makati, Rizal, Philippines2 150 Heyneman and Jamison Authors Department, The World Bank, Wash- STEPHEN P. HEYNEMAN, Senior Soci- ington, D.C. 20433. Specializations: ologist, Education Department, The Economic analysis of mass media for World Bank, 1818 H Street, N.W. use in education. Washington, D:C. 20433. Specializa- XENIA P. MONTENEGRO, Independent tiong: Quality of education in devel- Research and Evaluation Consultant, oping countries. 6301 Miller Drive, Alexandria, VA DEAN T. JAMISON, Senior Projects Offi- 22310. Specializations: Educational cer, Population, Health and Nutrition research and evaluation projects. No. 314. Gunter Schrnm , "Te Changing World of Natural Gas Utilization, Natural Resourcs journal No. 315. Bela Balassa and Carol Balassa, "Irdustrial Protection in the Developed Countries," The Mlrld Economy No. 316. Nancy Birdsall and Jere R. Behrman, "Does. Geographical Aggregation Cause Overestimates of the Returns to Schooling?" Oxford Bulletin of Economics and Statistics No. 317. Jan Peter Wogart antd Jose Silverio Marques, 'Trade Liberalization, Tariff Redundancy, and Inflation: A Methodological Exploration Applied to Argentina," tAltwirt- schafiliches Archiv No. 318. Sweder van Wijnbergen, "Credit Policy, Inflation, and Growth in a Financially Repressed Economy," Journal of Development Economics No. 319. David M. Newbery, "Commodity Price Stabilization in Imperfect or Cartelized Markets," Econometrica No. 320. George Psacharopoulos, 'The Contribution of Education to Economic Growth: International Comparisons," with a Comment by Theodore W. Schultz, International Comparisos of Productivity and Causes of the Slowdown No. 321; Bela Balassa, 'Trade between Developed and Developing Countries: the Decade Ahead," OECD Economic Studies No. 322. Nicholas Prescott and Jeremy Warford, "Economic Appraisal in the Health Sector," The Economics of Health in Developing Countries No. 323. William I. Jones, 'Agriculture's Changing Role in International Trade and Aid: Tastes and Techniques," Annales d'Etudes Infemnationales No. 324. Ricardo Martin and Marcelo Selowsky, "Energy Prices, Substitution, and Optimal Borrowing in the Short Run: An Analysis of Adjustment in Oil-Importing Developing Countries," Journal of Development Economics No. 325. A. Drud, W. Grais, and G. Pyatt, "The Transaction Value Approach: A Systematic Method of Defining Economywide Models Based on Social Accounting Matrices," Dynamic Modelling and Control of National Economies 1983 No. 326. Avishay Braverman and J. Luis Guasch, "Capital Requirements, Screenin& and Interlinked Sharecropping and Credit Contracts," Journal of Developent Economics No. 327. Sang-Chuel Choe and Byung-Nak Song, "An Evaluation of Industrial Location Polices for Urban Deconcentration in Seoul Region," Journal of Environmental Studies No. 328. Mohan Munasinghe, "Energy Strategies for Oil-Importing Developing Countries," Natural Resources Joumal No. 329. Abdun Noor, "Shard Control of Education: Theory, Practice, and Impressions from Education Projects Assisted by the WMorld Bank' Managing International Developmn No. 330. Larry E. Westphal and Yung W. Rhee, "Choices of Technology in Industry," International Journal of Development Banking No. 331. George Psachropoulos, "Assessing Training Priorities in Developing Countries: Current Practice and Possible Altenatives," International Labour Review No. 332. Abdun Noor, '1Proposing Another Dimension in Intemational Adult Educatio Availability of Basic Education," Graduate Studies Jomrnal. No. 333. Sweder van Wijnbergen, 'Inflation, Employment, and the Dutch Disease in Oil- Exporting Countries," Quarterly Journal of Ewnoics No. 334. Gerald T. YMara and John H. Duloy, "Modeig Efficient Water oation in a Conunctive Use Regime The Indus Basin of Pakistan," VWter Raeoc Resach 1ane of the World Bantl Reprint Series are available free of charge from the address on the bottom of the back cover. THE WORLD BANK Headquarters 1818 H Street, N.W. Washington, D.C. 20433, U.S.A. Telephone: (202) 477-1234 Telex: WUI 64145 WORLDBANK RCA 248423 WORLDBK Cable address: INTBAFRAD WASHINGTONDC European Office 66, avenue d'Iena 75116 Paris, France Telephone: (I) 723-54.21 Telex: 842-620628 Tokyo Office Kokusai Building 1-1, Marunouchi 3-chome Chiyoda-ku, Tokyo 100, Japan Telephone: (03) 214-5001 Telex: 781-26838 The l range of Wold Barnk publications, both free and for sale, s described in the MrId Bank Catalog of Publicaflon, and of the continuing research program of the World Bank, in VIbrld Bank Resah Pogn: Abtracs ofCuumnt Ssdies. The most recent edition of each is- available without chrge f PUBLICATIONS SALES UNIT THE WORLD BANK lJls H STREET, NW. WASHINGTON, D.C 20433:- . US.A. ISSN 0253-2131