89276 GUYANA’S HINTERLAND COMMUNITY-BASED SCHOOL FEEDING PROGRAMME MINISTRY OF EDUCATION / WORLD BANK IMPACT EVALUATION 2007-2009 Latin America and the Caribbean Region The World Bank Report prepared by: Suraiya Ismail, Public Health Nutritionist, Director, Social Development Inc. Christian Borja-Vega, Economist, The World Bank Angela Demas, Senior Education Specialist, The World Bank Edward Jarvis, EFA-FTI Program Coordinator, Guyana Ministry of Education July 13, 2012 Abbreviations BMI Body Mass Index DMP Daily Meal Programme (India) EFA-FTI Education for All - Fast Track Initiative FPD Food Policy Division GDP Gross Domestic product GoG Government of Guyana GPRS Guyana Poverty Reduction Strategy HAZ Height for age z score NAS National Assessment Scores NCERD National Centre for Educational Resource Development NCHS National Centre for Health Statistics (USA) R1 Round 1 Survey (baseline) R2 Round 2 Survey (midterm) R3 Round 3 Survey (final) SDI Social Development Inc SF 0 Schools where no feeding had started by Round 3 (control schools) SF 1 Schools where feeding had starting by Round 3 (treatment schools) SFP Community-based School Feeding Program SPSS Statistical Package for the Social Sciences WB World Bank WFP World Food Program WHO World Health Organization ii Acknowledgements This study presents the findings of a three year impact evaluation that was financed by the Guyana Education for All-Fast Track Initiative and the World Bank. The impact evaluation was developed in partnership with the World Bank team, Social Development Inc., and the Government of Guyana team, including impact evaluation design, field work, analysis, and writing of the study. Overall guidance and editing were provided by Angela Demas, Sr. Education Specialist at the World Bank, and valuable inputs were provided by Evelyn Hamilton, Chief Planning Officer of the Guyana Ministry of Education. Alonso Sánchez was instrumental in the analytical work for Round 2, and he was a main contributor to the study. We are grateful for the contributions of the enumerators, partnering agencies including staff at the Ministry of Education, Health, Agriculture, Amerindian Affairs, and Local Government. We thank the Regional Education Offices and the officers from the Department of Education at Bartica, Mabaruma and Moruca. To the EFA-FTI team from the Ministry of Education, led by Mr. Edward Jarvis, EFA-FTI Coordinator, we would like to acknowledge their excellent support and express sincere gratitude for their efforts in making the impact evaluation a possibility. We would also like to extend our deepest thanks to the training officers from the Ministry of Health (Food Policy Division), the Ministry of Education and the Social Development Inc., (SDI), with a special thanks to Mrs. Howard and Dr. Suraiya Ismail for so skillfully ensuring high standards in measurement and data collection. We wish to extend special thanks to Miss Claudette Phoenix of NCERD for the efficiency and speed with which she made the national assessment scores available to us. Lastly, we would like to thank the hinterland school communities for their willingness and enthusiasm to participate in the impact evaluation of their Community-Based School Feeding Program. To the above persons and to anyone else who contributed in any way to the successful implementation of the study but whose names we cannot now remember, we would like to extend our sincere thanks and gratitude for your inputs in this exercise which seeks to improve access by pupils to quality primary education by attending to their nutritional needs. iii Table of Contents Abbreviations .................................................................................................................................. ii Acknowledgements ........................................................................................................................ iii Executive Summary ........................................................................................................................ 1 1 Introduction ............................................................................................................................. 5 1.1 Guyana and its people ...................................................................................................... 5 1.2 Guyana’s educational system ........................................................................................... 5 2 Education for All – Fast Track Initiative (EFA-FTI): Program description ........................... 6 2.1 Program rationale and components .................................................................................. 6 2.2 The impact evaluation of the community-based school feeding program ....................... 7 2.3 Context and Locations of treatment and control schools ................................................. 8 3 Methodology ......................................................................................................................... 13 3.1 Study design ................................................................................................................... 13 3.2 Survey instruments ......................................................................................................... 15 3.3 Training and fieldwork ................................................................................................... 16 3.4 Data management and analysis ...................................................................................... 16 3.5 The samples .................................................................................................................... 17 4 Descriptive Statistics and Impact Estimation ....................................................................... 18 4.1 Outcomes: The students ................................................................................................ 19 4.1.1 School attendance ................................................................................................... 19 4.1.2 Students’ nutritional status ...................................................................................... 23 4.1.3 Students’ academic performance ............................................................................ 26 4.1.4 How students behave .............................................................................................. 30 4.2 Outcomes: The communities......................................................................................... 33 4.2.1 Parental participation in school activities ............................................................... 33 4.2.2 Safety net and price shocks ..................................................................................... 34 4.3 Impact analysis: What was the impact of the school feeding program? ....................... 44 4.3.1 Health Indicators ..................................................................................................... 45 4.3.2 School Enrollment .................................................................................................. 47 4.3.3 Students’ attendance ............................................................................................... 50 4.3.4 Educational Attainment .......................................................................................... 51 4.3.5 Households’ food consumption and Food Price Volatility Impacts ....................... 55 5 The schools and their communities....................................................................................... 61 5.1 The learning environment .............................................................................................. 61 5.1.1 The schools and their staff ...................................................................................... 61 5.1.2 The students ............................................................................................................ 63 5.2 The parents, their households and communities ............................................................ 64 5.2.1 Socio-demographic profiles of parents and their households ................................. 64 5.2.2 Household possessions and access to services and amenities ................................ 66 5.2.3 How children get to school ..................................................................................... 67 6 Conclusions ........................................................................................................................... 68 7 REFERENCES ..................................................................................................................... 71 ANNEX 1...................................................................................................................................... 76 ANNEX 2...................................................................................................................................... 78 iv ANNEX 3...................................................................................................................................... 79 ANNEX 4...................................................................................................................................... 83 ANNEX 5.................................................................................................................................... 123 List of Tables, Maps and Figures Table 1: Enrollment and Date of Entry to Comparison Groups by School ................................. 14 Table 2: The samples by region and school feeding status in Round 3 (2009) ........................... 17 Table 3: The samples in all rounds .............................................................................................. 17 Table 4: Perceived impact of school feeding – attendance .......................................................... 19 Table 5: Attendance in two weeks preceding survey day ............................................................ 20 Table 6: Reasons for absence in two weeks preceding survey day ............................................. 23 Table 7a: Guyana’s health indicators for coastal and hinterland regions ..................................... 24 Table 7: Nutritional status of students ......................................................................................... 25 Table 8: Grades 2 and 4 - National Assessment Scores (expressed as percentages) ................... 29 Table 9: Grade 6 - National Assessment Scores (expressed as percentages) – Round 3............. 29 Table 10: Perceived impact of school feeding - behavior............................................................ 30 Table 11: Class participation and signs of disconnect ................................................................. 32 Table 12: Participation of parents in school feeding-related activities ........................................ 33 Table 13: Guyana’s SFP compared to SFPs around the World ................................................... 38 Table 14: Guyana SFP Costs vs. India’s Day Meal Program ...................................................... 39 Table 15: Food groups and scores by survey round and by group .............................................. 41 Table 16: Children’s breakfasts on day of interview: parent and student responses .................. 44 Table 17: School Enrollment in 2007 and 2009 .......................................................................... 48 Table 17a: Effects of SFP on different Academic Subjects ......................................................... 53 Table 17b: Propensity Score Matching Results ............................................................................ 54 Table 17c: Diet Diversity Scores and Households’ Characteristics ............................................. 58 Table 17d: Poverty Impact of the Food Price Crisis and the SFP Safety Net ............................. 61 Table 18: Schools – enrollment ................................................................................................... 62 Table 19: Classes ......................................................................................................................... 62 Table 20: Student profiles ............................................................................................................ 63 Table 21: Profile of parents and their households, by survey round ........................................... 65 Table 22: Literacy (Round 3 only, students’ responses) ............................................................. 65 Table 23: Access to services ........................................................................................................ 66 Table 24: Amenities and possessions: parents’ and students’ responses .................................... 66 Table 25: Students’ access to schools .......................................................................................... 67 Map 1: Region 1 Agricultural Employment Intensity ................................................................... 9 Map 2: Region 7 Agricultural Employment Intensity ................................................................. 10 Map 3: Region 1 Location of Treatment and Control Schools .................................................... 12 Map 4: Region 7 Treatment and Control Schools Location ........................................................ 12 Figure 1: Poverty in Guyana, by Region ..................................................................................... 11 Figure 2: Guyana’s hinterland Community-Based School Feeding Program: conceptual framework ..................................................................................................................................... 18 Figure 3: School-level mean percent attendance by survey round and group ............................. 20 v Figure 4: Percent of sample with >50% attendance in two weeks .............................................. 21 Figure 5: Sample for Round 3 - absence and reasons for absence............................................... 22 Figure 6: Mean height for age z-scores by group and by survey round ...................................... 26 Figure 7: Stunted and severely stunted children - ........................................................................ 26 Figure 8: National Assessment Scores: Grades 2 and 4 by group and survey round ................. 28 Figure 9: Mean participation scores by survey round and group................................................. 31 Figure 10: Mean disconnect scores by survey round and group .................................................. 31 Figure 11: Food Crises - Safety Nets in Low and Middle Income .............................................. 35 Figure 12: Food Deficit and Average Dietary Requirement in Guyana ...................................... 36 Figure 13: Availability of selected foods and food groups in Guyana 1990-2009 ...................... 37 Figure 14: Mean diet diversity scores by survey round and group .............................................. 41 Figure 15: Mean food frequency scores by survey round and group .......................................... 42 Figure 16: Treatment and Control (T/C) Ratios of Frequency of Food Consumption ................ 43 Figure 17: Estimation of SFP impacts on height-for-age z-scores .............................................. 46 Figure 18: Treatment and Control Groups Students’ Height Comparison (centimeters) ............ 46 Figure 19: Treatment and Control Groups Students’ Weight Comparison (kilograms) .............. 47 Figure 20: Estimated SFP impact on Enrollment by Region, 2007-2009.................................... 49 Figure 20a: Estimated SFP impact on Enrollment by Gender, 2007-2009.................................. 50 Figure 21: SFP impact coefficients on Attendance Rates 2007-2009 ......................................... 51 Figure 21a: Average School Attendance and Academic Performance, 2007-2009..................... 51 Figure 21b: Propensity Scores for Treated and Untreated (common support) for all individuals with at least one score reported ..................................................................................................... 54 Figure 22: Impact of SFP on Food Frequency Scores (from Baseline) ....................................... 55 Figure 23: Food Frequency Scores Prediction by SFP status ...................................................... 56 Figure 24: Diet Diversity scores, SFP participation and Household Variables ........................ 59 Figure 24a: Partially employed, no SFP ...................................................................................... 59 Figure 24c: Fully employed, no SFP ........................................................................................... 59 Figure 24b: Partially employed, with SFP ................................................................................... 59 Figure 24d: Fully employed with SFP ......................................................................................... 59 Figure 25: Food Prices Shocks in Areas Without a Safety Net (SFP) ......................................... 60 vi Executive Summary Guyana’s Hinterland Community-Based School Feeding Program (SFP) began in 2007 with the objective of building more community participation in schools and improving children’s human development outcomes, such as student enrollment and attendance, nutritional status and learning outcomes. In addition, the program supports improvement of schools’ organization of primary level in Regions 1, 7, 8 and 9. In order to participate in the program, schools and their associated communities are required to submit school feeding proposals, undergo training in basic financial bookkeeping, food hygiene and nutritious meal preparation, using locally produced foods whenever possible. Communities must also ensure school kitchens meet the requirements and guidelines of the Ministry of Health, ensuring an adequate safe-water supply. To evaluate the program, the Government of Guyana and the World Bank collected survey data from schools, students, teachers and parents in three rounds 2007, 2008 and 2009 in Regions 1 and 7. This report shows the findings and impacts of SFP using all survey rounds. Regions 1 and 7 are characterized by high poverty levels and agricultural labor intensity. Both factors highlight the potential of organizing SFP around local producers, so that a regular supply of low-cost food can be guaranteed to the local schools and children living in precarious conditions. Seventeen of the sixty-four schools participating in the impact evaluation are in Region 7, and the rest are in Region 1. Randomization for the selection of comparison groups was not achieved, due to the participation rules. However, sample selection correction methods were used to correct for observable and unobservable biases. The conceptual framework to evaluate the SFP addresses the following questions: Quantitative • Has the hinterland school feeding program had a positive impact on students’ enrollment, attendance, learning outcomes, and nutritional status? • Has the program provided a safety net against food price increases or shocks? Qualitative • Has the program led to improvements in students’ classroom participation and behavior, school performance, and parental participation in school activities? • Has the program improved diets of households as well as food frequency intakes? The following findings are separated by topic: Enrollment and Attendance Findings • The total increase in enrollment observed in three rounds of survey data equals 338 students. Around 53 percent (178 students) of the observed change in enrollment between 2007 and 2009 can be attributed to the SFP. • In percentage terms, the treatment group increased enrolment by 16 percent between 2007 and 2009, while for the control group enrolment declined 7.5 percent. • SFP increased average attendance by 4.3 percent between 2007 and 2009. 1 Nutritional Status Findings • Control schools showed consistently higher levels of stunted children because i) treatment schools enjoy better economic conditions (selectivity) and / or ii) because control groups may have a higher proportion of Amerindian children, who have, in turn, consistently higher levels of poverty. • Children in treatment areas grow 0.8 centimeters more compared to children attending schools without school feeding. Children at risk of wasting in the treatment group improved their Body Mass Index (BMI) between 1.07 to 1.34 standard deviations compared to the control group. • The SFP contributed to preserving frequency of food consumption and diet diversity, particularly in a period of food price volatility. The lowest food diversity scores with relatively large variability are seen when children in the household do not receive a school meal, and when the household hosts an extended family, regardless of whether the household head is employed or not. Students’ classroom behavior and school performance • Almost two-thirds of head teachers and class teachers consistently noted that the behavior of students changed positively with the SFP. However, this perception declined in each survey round. • Survey data showed that SFP schools had higher levels of class participation. Students’ of disconnect and distraction from classroom activities showed lower incidence in treatment schools. Average student participation in control schools declined compared to increases shown in treatment schools over the period of 2007 to 2009. Student disconnect in classroom activities showed a reduction in treatment schools and a relative increase in control schools. Academic Performance by Students • These impacts are expected to manifest in the longer-term measured through nationally standardized test-scores. However, in Round 3 there is already evidence that the treatment group is beginning to perform better than the control group in English, math and social studies. This last subject showed higher scores in treatment groups than control groups, although this test was only applied to 6th graders. • Using Propensity Score Matching techniques (PSM) to correct for differences in students’ characteristics, showed significant impacts in Math and English. Social studies showed modest impacts. Reading and Science showed non-significant impacts. Parental/Community Participation • Survey data shows that parents are actively participating in cooking, cleaning and serving meals. Other important activities in which parents participate are food provision and commercialization. 2 • An important contribution of the program is to actively engage communities and parents in the school feeding program. Parents and school teachers showed high levels of involvement in fund raising and other activities that improve the quality of the program. Household Diets • The diet of rural and Amerindian communities often lacks nutrient diversity. In poor and Amerindian communities diets are low in vegetables, fruits and dairy products. Treatment areas kept the frequency of consumption relatively steady. During the food price shocks the gap in food consumption frequency and diet diversity between control and treatment groups increased substantially. Food Prices Shocks and SFP as a Safety Net • SFP implementation coincided with a period of uncertainty and high volatility in food and agricultural commodities prices. A daily meal to children living in a poor household represents a safety net mechanism by keeping household’s purchasing power relatively constant from adverse food prices shocks. An increase in food prices harms the spending capacity of households which in turn affects food consumption, nutritional intake and poor nutrition outcomes. • On average, before the food prices shocks a household in control areas consumed US$2.40 less per month on food, compared to treatment areas. During and after the food prices shocks control areas consumed US$9.00 less per month worth on food. Control areas would have 150 more children at risk of falling into poverty, compared to treatment areas, before the food price shocks. During and after the food price shocks 510 more children in control areas would be at risk of falling deeper into poverty. • The financial returns of the SFP can be substantial if it expands. It can also bring a safety net mechanism to regions with economic adversities. Compared to other large- scale programs SFP has a relatively low cost per child enrolled. The evaluation findings and the experiences of the SFP raise a number of important issues that need the attention of the Ministry of Education: • Although attendance was indeed better in treatment schools, absenteeism remains a serious concern. More than 59 percent of all children included in the evaluation were absent for at least one day in the two weeks preceding the survey application. Of those children, more than 22 percent gave labor of some kind as a reason for their absence. Labor included care of younger siblings, work on the farm or at their homes. • A further 6.6 percent of children with absences in the two weeks preceding the survey were absent because of the household’s economic condition: no food, no uniform, no cash or no stationery. Interestingly, of those who gave lack of food as the reason for absence, all children except one attended control schools where no lunch was offered. 3 • The short-term hunger which occurs if a child receives an inadequate breakfast and/or has a long and arduous journey to school, is a major contributor to poor classroom behavior and academic achievement (see Annex 3). Single parents were least likely to offer a full breakfast to their children. An important finding of the evaluation was that students attending treatment schools were significantly less likely to receive a full breakfast. Are parents of treatment school children withholding a full breakfast because they know the child will receive a substantial lunch? If this is indeed the case, then the training offered by the school feeding program needs to stress the importance of an adequate breakfast, even when a full lunch is to be offered at noon. • Sustainability of the community-based school feeding program when external funding ends is an issue for urgent attention. A formal consultation with SFP communities to garner their opinions of the program and ideas for sustainability produced actionable items and recommendations, some of which have been implemented and others require government attention. When considering the obvious educational and nutritional benefits of the Program, the Government should take into account the important role of the Program in providing a safety net, as well as aspects that address poverty in hinterland communities. Thus, for example, the Program offers guaranteed markets to farmers for their produce and employment for women as cooks. • While the SFP is a cost-effective program, the Ministry may wish to consider ways by which the Program’s costs may be maintained or reduced so as to increase the likelihood of sustainability. These include establishing school gardens to provide produce for the meals, encouraging home gardens, seeking partnerships with the private sector, and finding ways in which the school kitchens could be used on weekends and during school holidays to raise funds through small business enterprises. These are possibilities that have been discussed with community members and could be explored further. Finally, the SFP’s achievements with regard to improving community participation in school activities can make a key contribution in finding solutions to many of the challenges faced by the Ministry of Education for service delivery and quality education in rural, remote areas. 4 1 Introduction 1.1 Guyana and its people Guyana is situated on the north eastern shoulder of South America, bordering Suriname on the East, Venezuela on the West, and Brazil on the South and Southwest. Its population of approximately 700,000 persons consists of five main ethnic groups: East Indian, African, Amerindian, Chinese and European. The majority of the population (97%) lives in the urban and rural coastal areas and in communities bordering Guyana’s major rivers. The remaining 3 percent of Guyana’s population, largely the Amerindians who are the indigenous population of the country, live in the hinterland which consists of rainforest, savannah, deep riverine and mountainous topography. Much of the country’s agricultural produce is cultivated in rural areas because of the presence of rich alluvial soil. Administratively, Guyana is divided into ten regions. Regions 1, 7, 8 and 9 constitute the hinterland regions and are inhabited largely by Amerindians, although some Amerindian communities are also found in other regions. 1 Many of the communities in the hinterland regions are small and remote, and access to these communities is difficult. The country’s capital, Georgetown, is located on the coast in Region 4. The three main ethnic groups (East Indian, African and Amerindian) of Guyana present different patterns of malnutrition. In particular, the Amerindian population generally presents a much higher prevalence of chronic malnutrition (stunting) and a lower prevalence of acute malnutrition (wasting). The few studies available indicate that malnutrition among Amerindian children is linked to deficiencies in key micronutrients which are probably caused by lack of diet diversity rather than general food insecurity. These issues are discussed in further detail in Annex 2 (Nutrition Notes). 1.2 Guyana’s educational system Guyana’s post-independence educational development included the expansion of access to education from the pre-primary to the university level. However, many factors militated against the Government’s achieving the goal of providing quality education to all of its citizens. The topography of Guyana and the remote location of segments of the population in the hinterland regions created a major challenge for good quality service delivery. Children from many of the most remote communities travel for up to two hours by boat or a combination of land and river to reach their schools. Most of the larger educational institutions are found on the populous coastland, thus requiring greater budgetary allocations. This resulted in the educational service offered on the coastland 1 Many Amerindian communities are now communities of mixed ethnicities. 5 being of a higher quality than that offered in the interior (hinterland). The Ministry of Education, with the assistance of bilateral and multilateral agencies, has attempted to grapple with the issue of providing equitable and quality education to all, as is articulated in its Education Strategic Plan, 2008-2013. Primary school, where attendance is compulsory, provides six years of education (Grades 1 – 6). Academic performance at the primary school level is assessed in Grades 2, 4 and 6. Performance at the Grade 6 level (with a small contribution from the Grade 4 assessment) determines which, if any, secondary school the child is to attend. 2 Education for All – Fast Track Initiative (EFA-FTI): Program description Guyana was one of the first of seven countries to apply for funding from the Education for All Fast Track Initiative (EFA-FTI) 2. Their proposal for covering the period 2003 to 2015 was endorsed by the donor partnership in November 2002 and awarded US$12.0 million in 2004 to cover the first three years of implementation. The aim of the funds is to assist the government in its effort to reach the EFA-FTI goals for primary education by 2015. This first allocation of funding helped to design, launch and maintain the community-based school feeding program which is a sub-component within Guyana’s overall EFA-FTI program. In 2008, upon successful execution and progress, Guyana’s EFA-FTI program was re-endorsed and received funding for another three years for a total amount of US$32.5 million. 2.1 Program rationale and components Guyana’s EFA-FTI program encompasses three initiatives: • Initiative I - Improving the Quality of the Teaching Force in the Hinterland; • Initiative II – Enhancing the Teaching/Learning Environment in Primary Schools; and • Initiative III – Strengthening School and Community Partnerships. These three main initiatives are consistent with Guyana’s Poverty Reduction Strategy (GPRS), the National Development Strategy, the Education Strategy Development Plan and the EFA-FTI goals for primary education, all of which seek to achieve quality education and produce a literate and numerate society. Guyana’s Hinterland Community-Based School Feeding Program (SFP) is a key sub- initiative of Initiative III. The objectives of the SFP include building more community participation in schools, and improving school attendance, school performance and the nutritional status of primary school children in Region Nos. 1, 7, 8 and 9. Implementation of Guyana’s SFP for these 2 The Education For All – Fast Track Initiative is a global initiative supported by donor partners, and it is now known as the Global Partnership for Education (GPE). 6 four hinterland regions began in 2007. In order to participate in the program, schools and their communities are required to submit project proposals, undergo training in basic bookkeeping, food hygiene and good food handling practices, basic nutrition and nutritious meal preparation. Communities that prepare their own meals on premises are also required to ensure that school kitchens meet the requirements and guidelines of the Ministry of Health, passing certifications and providing a safe and adequate water supply. 2.2 The impact evaluation of the community-based school feeding program In 2007, the Ministry of Education, with funding and technical assistance from the World Bank, began the impact evaluation of the SFP in Regions 1 and 7. The first survey round covering 64 schools was conducted in June 2007, the second in May / June 2008 and the third and final round in May / June 2009. At the start of the evaluation, five schools had started school feeding, but no more than a month before data collection for the first survey round. By Round 3, nineteen schools were providing school meals in the two regions. This report presents the final results of the three surveys (2007, 2008, and 2009) of the impact evaluation of the SFP in hinterland areas. Additional details of the impact evaluation methods are presented in Annex 4. Given the evaluation design and they way in which the SFP was structured, full randomization for the allocation of the SFP to treatment and control groups was not achieved. Treatment and control groups arose largely from a process of self-selection. Preparatory activities of the SFP comprised three phases of activities prior to the provision of grants to establish community-run kitchens in schools. The first phase included a promotion campaign and awareness-raising sessions with parents, teachers, village councils, and other members of communities. Overall this phase enjoyed high levels of participation from school and community members. During phase 2, training was provided to elaborate a proposal to request grants in order to create the conditions for food preparation, agricultural production, financial management and administration of the school feeding program. A standard template for proposals was developed by the EFA-FTI team which was used during the training to allow communities to become familiar with the contents and structure their proposals with ease. The sessions were carried out with school representatives and community members interested in implementing the SFP in their communities. This phase was very important because it led to the production of a quality proposal to access funding for the provision of school meals. In parallel, phase 3 included a certification of cooks and the facilities essential to launch the program. Without certification, the SFP could not start. While the process through which schools and communities were enabled to submit proposals was open and on a rolling basis, it could have created selectivity into the comparison groups. This selectivity biases impact estimators. In particular, because phase 2 involves schools’ proposal submission as a prerequisite to the time when a school enters into the program, it may result in better organized schools entering more rapidly while less organized schools take longer to reach approval and ultimately the grant. Thus, school and community characteristics play a 7 role in the timing of program entry. Intuitively, the characteristics that may affect this sample selection have to do with observed and unobserved attributes of the schools and communities, as well as regional and sub-regional characteristics. Thus, it was essential to incorporate the sample selection bias correction in the regression analyses. An aspect that also may bias results relates to the student’s absence when the survey was applied. If this aspect modified the sample substantially it will bias impacts particularly in the educational performance outcomes (e.g. students did not report their scores). 2.3 Context and Locations of treatment and control schools The surveys conducted to evaluate the community-based school feeding program were applied in Regions 1 and 7 in Guyana. Barima-Waini (Region 1) is located in the northwest of the country. It covers an area of 20,339 km² with a population of around 26,000 inhabitants in 2007. It borders the Atlantic Ocean to the north, the region of Pomeroon-Supenaam to the east, the region of Cuyuni-Mazaruni to the south and Venezuela to the west. Major settlements include the regional capital Mabaruma, Port Kaituma, Matthew's Ridge, Morawhanna, Towakaima, Koriabo, Hosororo, Arakaka and Moruca. Barima and Waini are the two main rivers that flow through the densely forested and sparsely populated region. Region 1 is ethnically diverse and has a low population density. The region is predominantly forested highland, bordered at the north by a narrow strip of low coastal plain. Both the region’s two main rivers impede transportation access so many households depend upon subsistence agriculture. Many schools do not have electricity, and lack of other social facilities, which combined with the high cost of living, makes education inaccessible for many children. Other than agriculture, Region 1 communities are engaged in fishing, mining and logging activities. Region 7 is Cuyuni-Mazaruni which borders the Barima-Waini, Essequibo Islands-West Demerara and Pomeroon-Supenaam to the north, the region of Upper Demerara-Berbice to the east, the region of Potaro-Siparuni and Brazil to the south and Venezuela to the west. Its capital is Bartica. Other major settlements include Issano, Isseneru, Kartuni, Peters Mine, Arimu Mine, Kamarang, Keweigek, Imbaimadai, Tumereng and Kamikusa. Region 7 covers an area of 47,213 km² with a population of almost 18,000 people according to the last estimates of the Statistics Bureau in 2007. Region 7 is also ethnically diverse and has the lowest population density in the country. The population in this area consists mainly of Amerindians and Coastlanders who work as miners, and Government employees (public servants). The Regional Administration's main office is at Bartica and a subsidiary office is located in Kamarang in the Upper Mazaruni. The inhabitants of Region 7 villages are engaged in subsistence agriculture, mainly ground provisions, cassava and peanuts. In addition to agriculture, many communities are engaged in logging and mining and, to a lesser extent, fishing. The distribution of agricultural intensity of subsistence farming in Regions 1and 7 is shown in Maps 1 and 2. Region 1 has high agricultural intensity in its northeastern sub-regions. The southern sub-regions have lower agricultural intensity owing to facilitation of trade and 8 households’ engagement in the home-based manufacture of non-agricultural products. In Region 7 employment in agriculture is highest in Bartica (its capital) located in the far east of the region and in the Kamarang district, predominantly of seasonal crops and rotational agriculture of tropical fruits and grains. The agricultural intensity profile of Regions 1 and 7 is important to the communities’ ability to organize the SFP around local producers, so that a low-cost supply of food can be guaranteed to the schools’ kitchens. Map 1: Region 1 Agricultural Employment Intensity p g g p y Mabaruma/Kumaka/ Hosororo Barima/Amakura Mathews Ridge/ Waini Arakaka/Port Kaituma Rest of Region Very High High Medium Low Very Low 9 Map 2: Region 7 Agricultural Employment Intensity Jawalla, Bartica Kubenang Kamarang River Arau Agatash Paruima Waramadan Karambaru Lower Mazaruni Very High High Medium Low Very Low Source: Bureau of Statistics, Guyana and Ministry of Agriculture. 2006. Regions 1 and 7 are characterized by high poverty levels as shown in Figure 1. Although regions 1 and 7 contain only 2.8 and 2.2 percent of the country’s population respectively, more than half of their populations are below the poverty threshold. Both regions also have a large proportion of children who do not attend school or attend irregularly or who leave school with an incomplete education 3. Low rates of access to education have reinforced the vicious circle of poverty in the last years, particularly in regions 1, 7, 8 and 9. The SFP provides poor households in remote rural areas with an incentive to send their children to school and save the cost of a daily meal, which in some cases represents a considerable proportion of disposable income. 3 See Guyana’s Poverty Assessment. World Bank. 2006. 10 Figure 1: Poverty in Guyana, by Region 100 90.04 90 83.68 80 76.13 70 64.08 Percentage Poor 60 52.15 50 42.53 40.09 40.27 40 28.45 30 25.23 20 10 0 Cuyini -Mazaruni Mahaica- Berbice E. Berbice- Potaro- Siparuni Supenaam Demerara- Upper Demerara- Corentyne Barima-Waini Pomerron- Upper Essequibo Essequibo Island- Upper Takuntu - Mahaica W. Demerara Berbice Region Source: Own estimates using Household Budget Survey data 2006 Seventeen of the sixty-four schools participating in the impact evaluation are in Region 7, and the rest are in Region 1. The 21 schools that had started providing school meals by Round 3 are designated treatment schools, while the rest served as the control or comparison group. Further details of the treatment and control groups are given in Section 3.3. The full list of schools is provided in Annex 1. Map 3 shows the locations of the treatment and control schools in Region 1. For each treatment school there are two to four control schools in each cluster. The location of the treatment group is located in the eastern and northern sub-regions, which are characterized by high agricultural intensities. 11 Map 3: Region 1 Location of Treatment and Control Schools • Treatment schools • Comparison schools Map 4: Region 7 Treatment and Control Schools Location •Treatment schools •Comparison schools 12 Map 4 shows the location of the treatment and control groups in region 7. Region 7 has a lower population density than Region 1 and high dispersion of its communities across the whole region. The central part of Region 7 has very few communities so all treatment and control schools are located in the east (Bartica) and in the west. Although Region 7 has far fewer schools than Region 1, Region 7 has demographic, social and economic indicators representative of Guyana’s hinterland regions. The geographic distribution of schools in treatment and control schools is fairly even in both regions. Nevertheless, the sample of schools in Region 1 was distributed over a wider range of geographical coverage than Region 7 in order to gain statistical representativeness. 3 Methodology 3.1 Study design All state primary schools in Regions 1 and 7 that had not started providing school meals by the start of the third term of the 2006 – 2007 school year were included in the impact evaluation. These 64 schools were assigned to treatment and comparison groups according to whether they had or had not started school feeding by the end of the second term of the 2008 – 2009 school year. Some schools had only single-grade classes, others only multi-grade classes, and the remainder had a mix of single and multi-grade classes. Three survey rounds were conducted, during May and June of 2007, 2008 and 2009. In Round 1 (baseline), information was gathered on Grades 2, 3 and 4; in Round 2, on grades 2, 3, 4 and 5; and in Round 3 on grades 2, 3, 4, 5 and 6. A grade was added each year to enable the impact evaluation to follow and include the original students as they progressed to the next grade level. These students made up the longitudinal group. The impact evaluation design did not entail a randomized assignment of treatment and control groups, for reasons elaborated in Section 2.2. As a result of the self-selection process, some characteristics at baseline for control and treatment groups were dissimilar (see section 4.1). The impact analysis tried to correct for selectivity in the treatment and control group assignment, but selectivity cannot be addressed fully using correction techniques. Table 1 shows control and treatment school enrollment and dates of entry into the SFP. The dates of entry of schools vary depending upon the submission of proposal and completion of training phases to obtain certification for the start of school feeding. Surveys were stratified in six clusters to gain internal validity. Representativeness was achieved at the regional level. 13 Table 1: Enrollment and Date of Entry to Comparison Groups by School REGION 7 Cluster and School 2007 2009 SFP status (until Round 3) Round 1 Round 3 Group Date of entry Cluster 1: Lower Mazaruni 1 Agatash 99 102 Treatment May 15 2007 2 Batavia 65 68 Control Sept 10 2009 3 Butukari 13 25 Control NA 4 Karrau Creek 40 26 Control Sept 10 2009 5 St. John the Baptist 665 610 Control Sept 10 2009 6 Wineperu 64 41 Control NA 8 Itaballi 147 152 Treatment March 23 2007 9 72 Miles 67 72 Treatment June 21 2007 10 Two Miles 175 181 Treatment June 21 2007 11 St. Anthony’s (Bartica) 532 545 Treatment May 18 2007 12 Kartabo 63 65 Treatment March 23 2007 13 Holy Name 163 169 Treatment June 21 2007 14 St. Mary’s 56 45 Treatment May 18 2007 Cluster 2 Middle Mazaruni 15 St Martin 76 83 Control Sept 10 2009 16 St Martin’s Annex 59 78 Treatment June 1 2008 17 Kurupung 68 54 Treatment Feb 4 2008 18 Isseneru 57 58 Treatment Feb 4 2008 REGION 1 Cluster 4 Matarkai 21 Port Kaituma 849 943 Treatment June 4 2007 22 Arakaka 119 126 Treatment June 10 2007 23 Baramita 90 103 Control Sept 10 2009 24 Matthews Ridge 214 226 Control Sept 10 2009 25 Sebai 119 136 Control Sept 10 2009 26 Falls Top 48 37 Control NA Cluster 5 Mabaruma 27 Almond Beach 40 34 Control Sept 10 2009 28 Aruka Mouth 72 76 Control NA 29 Barabina 126 132 Control NA 30 Black Water 38 39 Control NA 31 Hobedia 89 95 Control NA 32 Hosororo 343 346 Control Sept 10 2009 33 Hotoquai 124 114 Control NA 34 Kamwatta (Mabaruma) 70 53 Control NA 35 Lower Kaituma 37 50 Control NA 36 Lower Waini 16 54 Control NA 37 Mabaruma 584 579 Treatment June 9 2007 38 Peter & Paul 65 47 Control NA 39 Sacred Heart 116 119 Control NA 40 St. Anselm’s 96 91 Control NA 41 St. Anthony’s 98 85 Control Sept 10 2009 42 St. Cyprian’s 47 55 Control NA 43 St. Dominic’s 34 38 Control NA 44 St. John’s 53 28 Control Sept 10 2009 45 St. Margaret’s 72 80 Control NA 14 Cluster and School 2007 2009 SFP status (until Round 3) Round 1 Round 3 Group Date of entry 46 St. Mary’s 26 15 Control NA 47 St. Ninian’s 97 72 Control NA 48 Unity Square 18 22 Control NA 49 Wauna 499 597 Treatment Jan 14 2008 50 Yarakita 203 207 Control Sept 10 2009 67 White Water 136 143 Control NA Cluster 6 Moruca 51 Assakata 77 73 Treatment June 4 2008 52 Kamwatta (Moruca) 249 222 Treatment June 4 2008 53 Karaburi 260 246 Treatment Dec 3 2007 54 Kokerite 100 86 Control NA 55 Kwebana 178 203 Control Sept 10 2009 56 Santa Cruz 65 90 Control Sept 10 2009 57 Santa Rosa 633 700 Treatment June 4 2007 58 St. Bede’s 37 53 Control NA 59 St. Nicholas 446 468 Control Sept 10 2009 60 Wallaba 30 29 Control Sept 10 2009 61 Waramuri 344 340 Treatment June 4 2008 62 Warapoka 138 157 Control Sept 10 2009 63 Kokerite Annexe 18 21 Control NA 64 Father’s Beach 34 92 Control NA 65 Chinese Landing 28 31 Control NA 66 Waramuri Annex 148 143 Control NA Note: Kokerite annex is now considered part of Kokerite school. The total number of schools adds up to 64. No schools were surveyed in Cluster 3. Schools that began school feeding after Round 3 were kept in the control group. Source: SFP 3.2 Survey instruments Five questionnaires were used in the surveys: Head teacher questionnaire: collected information on the school, on parental participation in school-related activities, and on the head teachers themselves, including their academic and professional qualifications, and their views on the impact of school feeding on student attendance and behavior. School-level average attendance was added in later. Class teacher questionnaire: were completed by class teachers of the grades covered in each survey round, these questionnaires gathered information on grades taught and the numbers enrolled in each grade, numbers of repeaters, the teacher’s academic and professional qualifications, and their views on the impact of school feeding on student attendance and behavior. Class observation: conducted in each participating grade by the enumerators, these recorded classroom conditions (state of desks, number of books), numbers of students present, subjects taught, methods of teaching, signs of participation in class activities, and signs of disconnect from class activities. 15 Parent questionnaire: administered to up to ten parents per school by the enumerators, to gather information on the household’s socio-economic and demographic data, on participation in school-related activities, access to services and amenities, access to schools (distance and mode of travel), and frequency of consumption of specific food groups. Student data form: used to record the student’s date of birth, attendance, weight and height, family literacy and composition, access to schools (distance and mode of travel), selected household possessions, and breakfast consumption. National assessment scores were added later as they became available from the Ministry of Education. Student data forms were completed for all students present on the survey day in one class of each grade covered in survey rounds. Minor adjustments and additions were made to the questionnaires after Rounds 1 and 2. 3.3 Training and fieldwork Enumerators from Regions 1 and 7 were trained prior to each survey round. The training took the form of presentations by training officers, demonstrations, role playing and question and answer sessions. Role play was used to practice the administration of the parent questionnaire. The measurements of weight and height were practiced at local primary schools, and a standardization exercise was conducted to identify weak measurers. Topics covered included: • Purpose of the exercise • How to conduct an interview / meeting • Weighing and measuring children. • Correct use and care of the equipment • How to complete the questionnaires • How to conduct focus groups • Logistics and fieldwork instructions. Fieldwork was carried out by the trained enumerators during a period of four to five weeks in the third term of each school year. In addition to administering questionnaires, weighing and measuring children, observing classes and arranging for the completion of head teacher and class teacher questionnaires, enumerators conducted focus group discussions in each community in which the school was located. 3.4 Data management and analysis Data were coded, and then entered into computer files by trained data entry clerks using EpiInfo 6. A system of duplicate data entry followed by validation of the two entries and corrections where necessary was employed. National assessment scores (NAS) were entered later, when these were available. After data entry, validation and correction, epidemiological and nutrition indicators were calculated in EpiInfo. EpiInfo and SPSS-Macro use the new WHO reference standards which is 16 needed to calculate the nutrition indicators, height for age and body mass index (BMI) for age (see Annex 2, Nutrition Notes). Basic analyses were done using SPSS v12.0. An initial frequency was run prior to further analysis. Data analysis involved simple frequencies and distribution statistics, and bivariate analyses (chi square tests, t-tests, analysis of variance and linear correlation). A p value less than 0.05 was used to define statistical significance. More elaborate regression analyses were carried out using STATA v10. Details of these analyses are given in Annex 4. 3.5 The samples Table 2 gives the number of schools and students included in treatment and comparison groups by region. Details of the number of questionnaires and data forms achieved at each survey round are shown in Table 3. Table 2: The samples by region and school feeding status in Round 3 (2009) School feeding Region Data Totals status Region 1 Region 7 Comparison No. of schools 37 6 43 schools No. of students 1968 255 2223 Intervention No. of schools 10 11 21 schools No. of students 1038 616 1654 No. of schools 47 17 64 Totals No. of students 3006 871 3877 Source: Own estimations using Guyana IE surveys, 2007-2009 Table 3: The samples in all rounds Categories Number in category # of schools/communities with no data Round 1 Round 2 Round 3 Round 1 Round 2 Round 3 Schools 64 62 58 - 2 6 Head teachers 64 61 58 - 3 6 Teachers 112 140 162 7 3 3 Class observations 105 172 170 5 3 2 Parents 581 0 583 569 2 0 Food frequencies 580 1 Students – total 2430 3006 3877 0 2 0 No assessment scores: 8 9 5 No anthropometry: 4 3 0 Number of students with Tests Round 1 Round 2 Round 3 Grade 2 776 907 933 Grade 3 842 676 809 Grade 4 773 792 776 Grade 5 N/A 612 617 Grade 6 N/A * N/A ** 742 *Note: 39 Students without test data ** Note: 19 students without test data Source: Own estimations using Guyana IE surveys, 2007-2009 17 4 Descriptive Statistics and Impact Estimation Section 4.0 begins by qualitatively and quantitatively describing the data gathered, by survey round and by group (treatment and control), as appropriate (Sections 4.1 and 4.2) cross-examines comparison group composition and outcomes at the individual and community level, respectively. Section 4.3 concludes by presenting the results of the impact analysis, summarizing the results obtained with different estimation methods. Figure 2 illustrates the conceptual (logic) framework for the school feeding program. This section addresses the following questions: • Has the hinterland school feeding program had a positive impact on students’ attendance, nutritional status, classroom behavior and school performance? • Has the program encouraged parental participation in school activities? • Has the program improved diets of children and household members? • Has the program provided a safety net against food price increases? Figure 2: Guyana’s hinterland Community-Based School Feeding Program: conceptual framework INPUTS Training, funding, purchase of food products from local farmers OUTPUTS Increased food production, community organization, employment, school meals OUTCOMES: THE STUDENTS OUTCOMES: THE COMMUNITY Improved: student attendance, Community and parental classroom behaviour, nutritional participation in school activities, status, student performance improved household diets, safety net Social Development Food diversity and safety net By Product By Product Mechanism (against shocks) 18 4.1 Outcomes: The students 4.1.1 School attendance School attendance is an important outcome indicator to assess the impact of a school feeding program. We obtained both school-level and individual-level attendance data. We also asked head teachers and class teachers whether they had noticed any change in attendance since school feeding had started (Table 4). Most respondents agreed that school feeding was associated with increased attendance, although head teachers in Round 3 had a relatively lower rate of positive perception 4. Table 4: Perceived impact of school feeding – attendance Percent of respondents Factor Round 1 Round 2 Round 3 Increases attendance Head teacher agrees 87.3 72.1 57.9 Class teacher agrees 84.2 90.8 80.4 Source: Own estimations using Guyana IE surveys, 2007-2009 Figure 3 shows mean school-level attendance by treatment and control groups for each school year covered by the impact evaluation. Between round 1 and round 3 (school years 2006-07 and 2008-09) mean attendance fell marginally in the control groups, but rose in the treatment schools. Figure 3 illustrates the school-level differences between comparison groups in attendance rates, which can certainly bias the impact estimators. However, both groups show similar trends. 4 One possible explanation may be due to unawareness of SFPs presence in schools by new or incoming head teachers that began to work by the time round 3 was conducted. 19 Figure 3: School-level mean percent attendance by survey round and group 75 73 71 Percent attendance 69 67 Treatment 65 Control 63 61 59 57 55 R1 R2 R3 Source: Own estimations using Guyana IE surveys, 2007-2009 Table 5 provides information on individual-level attendance by survey round and by group. At baseline, full attendance (all days) is fairly similar and the difference between the comparison groups widens as time progresses. In addition, low attendance rates (<50%) between comparison groups is quite similar at baseline. Given that low attendance rates are relatively large and affect comparison groups unevenly, the impacts could by biased by selectivity in the sample of children. This issue is addressed in Section 4.3. Table 5: Attendance in two weeks preceding survey day Percent of students Attendance Round 1 Round 2 Round 3 (% of days present) Control Treatment Control Treatment Control Treatment < 50% 9.2 9.1 15.0 13.1 9.8 5.8 50 – 70 17.9 12.8 21.3 19.3 17.2 13.1 70 – 90 28.0 25.8 34.0 32.8 29.7 33.6 90 - <100 20.6 25.4 8.1 10.0 7.9 7.0 Present all days (100%) 24.3 26.8 21.6 24.8 35.4 40.6 Source: Own estimations using Guyana IE surveys, 2007-2009 Figure 4 illustrates the percent of the sample with high attendance in the two weeks preceding the survey, for Rounds 1 and 3, by treatment and control groups. The figure shows that while the percent fell slightly for the control group, it rose sharply in the treatment group. This change in the sample of students reinforces the point that selectivity can become an issue when estimating impacts. 20 Figure 4: Percent of sample with >50% attendance in two weeks preceding survey day, by survey round and group % of sample with >50% attendance in 2 weeks Treatment Control 95 94 preceeding survey 93 92 91 90 89 2007 2009 Source: Own estimations using Guyana IE surveys, 2007-2009 Figure 5 is a flow chart describing the individual-level absence data collected for the survey day and the two weeks preceding the survey day, in Round 3. Table 6 gives the reasons for absence in Rounds 2 and 3 for the two weeks preceding the survey day. In both rounds around 20 percent of students were kept at home to provide labor for the family: housework, farm work or child care. In Round 2, 72 children (3.7%) were absent for economic reasons (no food, no money, no uniform, no stationery). The figure rose to 127 (6.6%) in Round 3. All children except one who stated they were absent because of lack of food in Round 3 were from the control schools where no lunch was provided. Because the survey did not capture child employment and detailed information about reasons for absence it is not possible to control for the bias with observed characteristics. 21 Figure 5: Sample for Round 3 - absence and reasons for absence Not absent in previous two weeks Present on survey 1315 day Absence > 0 in Reason Total sample previous two for weeks absence 4786 missing (3877 with 2181 response) 249 Reason for absence coded (1-6) No reason given Reason for 1749 153 absence Absent on given survey day Reason for Reason Known to have 1932 absence not 1290 transferred or coded given (“other”) (of which 381 sat dropped out the National 1137 Assessment test) 362 22 Table 6: Reasons for absence in two weeks preceding survey day Round 2 Round 3 No. of Percent of No. of Percent of Reason respondents respondents respondents respondents Illness 976 50.6 1111 57.5 Climate 233 12.1 26 1.3 House or farm work 227 11.8 251 13.0 Care of younger sibling 148 7.7 181 9.4 Transportation 143 7.4 112 5.8 No food 32 1.7 68 3.5 Other reasons (see below) 169 8.8 183 9.5 Total 1928 100 1932 100 Other reasons for absence: Uniform dirty or wet, no footwear, 33 53 boots wet Traveling, out of village 17 45 Didn’t want to go, too late to go 16 30 Attend funeral or wedding, visit 14 9 relative, run errand Child doesn’t know, parent decision 14 26 School tour 13 0 Parent absent, mother ill, injured or 10 14 died, domestic problem Recreation 9 0 No exercise book, no stationery 4 4 No finance 3 2 Teacher was absent 2 0 Tide 2 0 Source: Own estimations using Guyana IE surveys, 2007-2009 4.1.2 Students’ nutritional status It is well-accepted that poor linear growth (stunting) can be the result of a number of factors, largely related to the environment, socioeconomic factors and education. Nutrition, infections and mother-infant interactions have been cited as ‘environment’ factors having the greatest influence on child’s growth (Waterlow & Schürch, 1994). Other authors (Martorell et al, 1988) highlight the link between stature and poverty. In the context of nutrition, poor stature is associated with diets of poor quality, namely that are deficient or marginal in micronutrients or 23 protein rather than in energy. Specific micronutrients mentioned in various studies include zinc, calcium and vitamin A, but there may be others that have not yet been studied. Skoufias et al. (2009) estimated income elasticities in rural Mexico for a variety of macro- and micro-nutrients, showing positive elasticities for vitamins (A,C) and calcium. The authors used fixed-effects and instrumental variables to correct from biases resulting from measurement errors associated to the outcome variable. While breastfeeding practices and mother-infant interactions are generally excellent in Amerindian communities in Guyana, diets are often monotonous and low in vegetables, dairy products, proteins, legumes and fruits. A study conducted in Regions 8 and 9 compared the stature of children of two tribes: the Wapishana in Region 9 and the Patamona in Region 8 (Dangour, 2001). The Wapishana villagers were substantially wealthier and had access to a more diverse diet than the Patamona villagers, who lived in relatively remote areas. The study found that Wapishana children were on average more than 3 cm taller than their Patamona counterparts. The descriptive statistics on stunting and wasting indicators based on Guyana’s Community- Based School Feeding Program surveys reiterate findings of earlier studies: high levels of stunting and a low prevalence of wasting (less than 3%). The results show that even at baseline, the prevalence of stunting (height-for-age less than -2 z-scores) was slightly higher in control schools (20%) than in intervention schools (17%). By the third round of the evaluation (2009), the prevalence of stunting had fallen by nearly three percentage points among intervention children, and risen by more than three percentage points among the children attending control schools. The differences in the prevalence of stunting between the two groups had thus risen from four percentage points in 2007 to ten in 2009. Table 7a highlights differences between coastal and hinterland regions in key health and nutrition indicators. Table 7a: Guyana’s health indicators for coastal and hinterland regions Coastal Hinterland Indicator regions regions Prevalence of stunting (<5yrs; <-2 SD) 8.7% 22.7% Prevalence of wasting (<5yrs; <-2 SD) 9.7% 3.7% Prevalence of anemia (<2yrs; <11g/dl) 63.5% 74.6% Low birth weight (<2500g) 18.6% 24.2% Infant mortality (per 1000 live births) 38 52 Under–five mortality (per thousand) 47 68 Sources: Multiple Indicator Cluster Survey (Guyana 2006); Evaluation of the Basic Nutrition Program (Social Development Inc, Guyana 2009). Table 7, and Figures 6 and 7 show the information on one of the key outcome indicators, the students’ nutritional status. Annex 3 provides a brief discussion of these indicators and the relevant cutoffs used to estimate the impacts on children at risk. In Amerindian communities stunting levels are high. If data on ethnicity had been gathered on SFP’s surveys, we would have seen even higher levels of stunting among children of Amerindian descent, when compared to the non-Amerindian children living in these communities. Thus, for 24 example, Guyana’s Basic Nutrition Program found a prevalence of almost 31 percent of stunting among Amerindian children aged 12 – 24 months and 12 percent among children of other ethnicities living in Amerindian communities. 5 A study of nursery school children reported a prevalence of 26.6 percent stunting among Amerindian children aged 4 – 6 years. 6 Table 7 and Figure 6 show that even at Round 1, stunting was a much greater problem among students attending the control schools: 20.9 percent were stunted or severely stunted in the control group, as compared to 14 percent in the treatment schools. There are three possible explanations for this finding: either treatment schools were in more organized communities with better economic conditions (selectivity), or treatment schools had a higher proportion of non- Amerindian children, or a mixture of both situations. Table 7: Nutritional status of students Percent of students Levels of Indicator* Round 1 Round 2 Round 3 nutritional status C T C T C T Not stunted 42.9 53.1 44.3 53.4 42.6 54.0 At risk of stunting 36.2 32.8 35.2 32.5 34.2 32.9 1. Height for age Stunted 17.6 12.4 17.1 12.9 18.9 11.5 Severely stunted 3.3 1.7 3.3 1.2 4.3 1.6 Not wasted 89.0 84.7 85.8 84.6 89.7 84.6 2. Body Mass At risk of wasting 9.1 12.5 11.2 13.2 9.0 13.6 Index Wasted 1.6 2.3 2.1 1.9 1.1 1.6 (BMI) for age Severely wasted 0.2 0.5 0.9 0.2 0.2 0.2 Source: Own estimations using Guyana IE surveys, 2007-2009 * See Annex 2 (Nutrition Notes) for an explanation of these indicators Not stunted / wasted: > -1.0 s.d.; At risk of stunting/wasting:-1.0 to -2.0 s.d.; Stunted / wasted:-2.0 to -3.0 s.d; Severely stunted / wasted: < -3.0 s.d. 5 S. Ismail and T. Roopnaraine. The impact evaluation of the GoG / IDB Basic Nutrition Program: Integrated report (July, 2009) 6 Food Policy Division, Ministry of Health, Guyana. Nursery school Sentinel site surveillance report (February, 2010) 25 Figure 6: Mean height for age z-scores by group and by survey round -0.6 R1 R2 R3 -0.7 Mean height-for age z-score -0.8 -0.9 Treatment Control -1 -1.1 -1.2 -1.3 Source: Own estimations using Guyana IE surveys, 2007-2009 Figure 7: Stunted and severely stunted children - Mean height for age z-scores by group at survey rounds 1 and 3 -1.65 R1 R3 -1.67 -1.69 Mean height for age z-score -1.71 -1.73 Treatment -1.75 Control -1.77 -1.79 -1.81 -1.83 -1.85 Source: Own estimations using Guyana IE surveys, 2007-2009 The difference between control and treatment schools in relation to stunting continued in rounds 2 and 3. However, Figure 6 shows that while the mean height-for-age fell slightly for children attending control schools, it rose among children attending treatment schools. Figure 7 illustrates the differences of the school feeding on children who were stunted or severely stunted at Round 1. As found in all other studies (see footnotes 3 and 4), the prevalence of wasting in Amerindian communities is lower than in the coastal communities of Guyana. 4.1.3 Students’ academic performance Guyana’s Ministry of Education assesses school performance of children in Grades 2, 4 and 6. In Grades 2 and 4, mathematics, English and Reading are the assessed subjects, and for Grade 6, Mathematics, English, Science and Social studies are assessed. These National Assessment 26 Scores (expressed as percentages) are presented in Tables 8 and 9. Figure 8 shows results for Grades 2 and 4 by group, for survey rounds 1 and 3. Looking at significant differences in the descriptive statistics (average scores) between treatment and control groups in the three survey rounds, we find the following: • Mathematics: no significant differences between groups in any grade or survey round; • English: no significant differences at either baseline (Round 1) or Round 2, but the treatment schools perform significantly better at round 3 in all grades; • Reading: in Round 1, the control group performs better than the treatment group for grade 4, but there is no significant difference in the performance of grade 2 children. In Round 2 there are no significant differences in either grade, but by round 3, performance is significantly better in both grades among children of the treatment schools; • Science: no significant differences between groups; • Social studies: the treatment school children perform better than the control school children. In short, it would appear from these basic analyses that while virtually no differences between the treatment and control groups existed at Round 1 (except for Grade 4 reading when the control group did better), by Round 3 the treatment group is performing better than the control group in English, reading and social studies. The impact of SFP on educational attainment outcomes (subject scores) used matching to balance individual and household characteristics between comparison groups in order to explore causality. The results in section 4.3.4 are only estimated for the Average Treatment Effect of the matched groups, so that generalized impact results cannot be drawn from this exercise. 27 Figure 8: National Assessment Scores: Grades 2 and 4 by group and survey round 60 55 Percent scores 50 45 R1 R3 40 35 30 25 Treatment Control Treatment Control Treatment Control Maths English Reading Source: Own estimations using Guyana IE surveys, 2007-2009 28 Table 8: Grades 2 and 4 - National Assessment Scores (expressed as percentages) Maths English Reading Grade 2 Round 1 Round 2 Round 3 Round 1 Round 2 Round 3 Round 1 Round 2 Round 3 % of students scoring 4.3 0.5 5.3 3.7 0.3 5.6 19.7 8.4 23.7 zero Mean % score ± SD 48.8 ± 22.4 55.9 ± 22.5 52.5 ± 25.9 42.4 ± 21.7 44.3 ± 19.3 39.4 ± 21.6 32.0 ± 25.9 38.8 ± 25.7 30.5 ± 28.6 Median of % score 50 56 54 41 44 36 26 37 22.9 Grade 4 Round 1 Round 2 Round 3 Round 1 Round 2 Round 3 Round 1 Round 2 Round 3 % of students scoring 1.1 0.3 4.7 1.6 0.2 4.1 7.1 5.6 12.9 zero Mean ± SD 42.5 ± 22.4 43.2 ± 20.3 38.0 ± 21.7 39.2 ± 19.1 41.9 ± 18.8 38.0 ± 21.9 47.5 ±27.5 52.5 ± 27.5 46.9 ± 30.8 Median 40 42 36 40 40 34 49 57 51.4 Source: Own estimations using Guyana IE surveys, 2007-2009 Table 9: Grade 6 - National Assessment Scores (expressed as percentages) – Round 3 Maths English Science Social Studies C T C T C T C T % scoring zero 0 0 0 0 4.6 4.4 3.7 4.8 Mean ± SD 58.7 ± 10.7 60.1 ± 12.2 58.8 ± 12.4 60.9 ± 13.0 57.9 ± 14.0 59.6 ± 14.4 61.2 ± 13.5 63.7 ± 16.3 Median 59.2 61.2 59.6 61.6 59.2 61.2 61.5 65.0 Source: Own estimations using Guyana IE surveys, 2007-2009 29 4.1.4 How students behave Both head teachers and class teachers of schools where school meals were offered were asked if they had noted any change in the behavior of the students (Table 10). 7 While the majority responded that behavior did change, the proportion of head teachers giving this response was highest in Round 1 while the proportion of class teachers was highest in Round 3. Table 10: Perceived impact of school feeding - behavior Percent of respondents Factor Round 1 Round 2 Round 3 SFP Changes behavior? Head-teacher agrees 87.3 60.7 63.2 Class-teacher agrees 72.5 83.2 83.3 Source: Own estimations using Guyana IE surveys, 2007-2009 Details of classes observed by the enumerators are provided in Annex 5: grades observed, subjects taught and methods of teaching used (Table A5-3), the state of the desks, number of books, class enrolment and number of students present (Table A5-4). A primary objective of the class observation exercise was to record the numbers of children who demonstrated signs of active participation in classroom activities, and the numbers of students who showed visible signs of disconnect with these activities. The results of this exercise are presented in Table 11. Since improved class participation is a key outcome indicator for the impact of school feeding, results are separated by treatment and comparison groups for each round. Participation and disconnect scores were calculated for each observed class using the percentage of children in class who showed the sign, then calculating the mean disconnect and participation scores for classes in treatment and control schools. Figures 9 and 10 illustrate these findings. Figure 9 shows that, while the mean participation score for treatment schools was lower than that for control schools in Round 1, by Round 3, treatment schools had substantially higher mean scores. The reverse is seen in Figure 10 for disconnect scores: in control schools, the mean fell marginally from 5.3 to 4.7, the fall in treatment schools was substantially greater, from 6.8 to 2.1. 7 For the teachers’ survey, they reported meals that were not necessarily offered exclusively through the Ministry of Education’s EFA – FTI Program. In some cases, other meals could have been provided by NGOs or religious organizations as well. 30 Figure 9: Mean participation scores by survey round and group 24 23 22 Mean participation score 21 20 Control Treatment 19 18 17 16 15 R1 R2 R3 Source: Own estimations using Guyana IE surveys, 2007-2009 Figure 10: Mean disconnect scores by survey round and group 8 7 6 Mean disconnect score 5 Control 4 Treatment 3 2 1 0 R1 R2 R3 Source: Own estimations using Guyana IE surveys, 2007-2009 31 Table 11: Class participation and signs of disconnect Respondents as percent of children present Percent of classes with zero responders Observation Categories Round 1 Round2 Round 3 Round 1 Round 2 Round 3 C T C T C T C T C T C T 1. Answered 53.8 50.4 47.9 50.9 41.9 49.8 15.9 7.1 14.1 9.6 21.0 12.3 when asked Raising hands 19.8 21.8 16.4 19.1 15.4 22.0 46.0 26.2 44.4 37.0 47.6 20.0 2. Demonstrated willingness to Standing 6.9 5.7 5.1 12.8 10.2 10.5 73.0 69.0 68.7 53.4 62.9 44.6 participate by: Leading in group 8.8 6.2 6.4 9.3 8.1 7.0 76.2 61.9 71.7 64.4 66.7 63.1 3. Responded 36.7 25.5 25.1 27.6 27.5 20.9 19.7 9.5 22.2 16.4 30.5 23.1 when prompted Respondents showing sign as percent of Percent of classes with no children showing children present sign Round 1 Round2 Round 3 Round 1 Round 2 Round 3 C T C T C T C T C T C T Resting heads 4.4 6.2 1.7 1.9 4.0 1.8 76.2 54.8 82.8 79.5 70.5 75.4 Sleeping 0.7 0.1 - - 0.4 0.2 96.8 97.6 100.0 100.0 96.2 96.9 4. Signs of disconnect Showing fatigue 4.2 5.8 4.0 4.1 5.6 2.3 79.4 54.8 71.7 74.0 71.4 76.9 Unrelated activity 11.9 14.5 8.0 5.0 8.8 4.0 58.7 28.6 59.6 67.1 54.3 67.7 Source: Own estimations using Guyana IE surveys, 2007-2009 32 4.2 Outcomes: The communities 4.2.1 Parental participation in school activities To establish and sustain a successful school feeding program, the EFA-FTI requires active participation by the community. In preparation, community members are asked to submit a proposal, undertake training, and raise funds and provide labor for the building of the school kitchen. Once the provision of meals starts, community farmers are required to provide commodities for the meals; local cooks provide nutritious meals; and an active committee member manages the program. In the head teachers’ questionnaire, the head teacher was asked to state whether parents assisted with specific school feeding activities, if the school had a school feeding program. The responses indicated a very high degree of involvement: nearly 80 percent of head teachers said that parents helped with cooking, cleaning and serving, 30 percent said parents helped to provide foods, and nearly 60 percent that they helped to raise funds. The surveys asked parents to assess their own level of involvement in school feeding-related activities, and head teachers to assess parents’ involvement in such activities. The results are given in Table 12. While there are differences in head teachers’ and parents’ perceptions, in all survey rounds, more responses from treatment school parents and head teachers indicated a high level of involvement in fund raising and school feeding activities than those from control schools. It should be noted that many control schools are actively preparing for the SFP, hence their positive responses to involvement in fund-raising and school-feeding activities. Table 12: Participation of parents in school feeding-related activities Percent indicating a high level of involvement Round 1 Round 2 Round 3 C T C T C T Parents’ perceptions Fund-raising activity 31.8 37.6 26.4 34.6 21.7 28.8 School feeding activity 16.4 31.8 19.3 34.1 18.7 22.0 Head teachers’ perceptions Fund-raising activity 40.9 57.9 38.1 42.1 30.8 57.9 School feeding activity 15.9 52.6 16.7 42.1 30.7 52.6 Source: Own estimations using Guyana IE surveys, 2007-2009 33 4.2.2 Safety net and price shocks 4.2.2.1. Food prices shocks and SFP as a safety net SFPs around the world have been limited in finding benefits to better understand how they can serve as a safety net mechanism for poor families, particularly when linking social protection, consumption of nutritious foods (frequency and diversity) and uninterrupted education. SFPs can provide and/or maintain diversity/balance in food consumption and micronutrients to complement, and not compete, with other nutrition programs. Little has been explored about the interrelation between food delivery modalities and SFPs effectiveness. At the local level, SFPs may contribute to creating new markets for agricultural produce. Guyana’s SFP was implemented in a period of unprecedented increases in food prices (2007 and 2008). This gives the program a unique timing to be evaluated from a safety net perspective in the face of food price shocks. A meal for poor families can represent a safety net mechanism to counter adverse shocks in prices and production of agricultural commodities. For instance, the food crises faced around the globe produced average increases of 130 percent in commodity prices (de Hoyos and Lessen, 2008). Commodities such as corn, wheat, rice and soybeans rose by 190, 162, 318 and 246 percent respectively (Lustig, 2009). The value of transfer of in-school meals appears to fall in the range of transfers common to other safety net programs. Guyana is largely independent of agricultural imports: around 32 percent of its GDP depends on agriculture and agricultural exports represent around 37 percent of export earnings (Ministry of Agriculture, 2008). Since food represents a relatively large share of developing countries’ consumption baskets, inflationary pressures are common, with a negative impact on the living standards of poor net consumers. In the hinterland areas of Guyana, the SFP provides incentives to continuous local food production which can affect prices paid by poor households. In this way, governments can use SFP safety nets to protect the poor from rising food prices. Such broad school-based interventions may be even more effective in targeting the poor than food subsidies or import restrictions to stabilize food prices. Safety net programs implemented around the world can be arranged in four categories: Cash Transfers, Food for Work, Food rations/stamps and School Feeding (see Figure 11). These programs have been shown to prevent steep and sudden declines in poor people’s income in the advent of economic crises and price shocks. According to Lustig (2009), 19 (out of 49) low- income and 49 (out of 95) middle-income countries do not have safety net programs of any kind. Given the characteristic of the adverse shock—i.e., an increase in the price of a basic good takes up a substantial portion of a poor person’s budget—the most adequate safety net is to compensate the affected population for their loss in purchasing power in cash or in kind. In the absence of cash transfer programs, countries could resort to school feeding programs. While they will not compensate the poor for the loss of purchasing power associated with higher 34 food prices, school feeding programs can insulate (at least in part) children of poor households from suffering a cut in their nutritious food intake. Figure 11 shows how school feeding programs are widely prevalent among low and middle income countries, compared to other safety net programs. Since vulnerable populations are most affected by food price shocks, the presence of school feeding programs provide enormous non- tangible benefits by i) investing early in the health and education of children, and ii) shielding partially from economic shocks the disposable income of households, at least in the short-run. Figure 11: Food Crises - Safety Nets in Low and Middle Income Countries (number of programs) Cash Transfer 37 Food for Work Food ration/stamp 31 School Feeding 24 21 16 12 8 8 Low Income Middle Income Source: Lustig, 2009. In Guyana, Regions 1 and 7 are rural areas where many families depend on local agricultural markets to preserve their purchase power on food consumption. Other adjacent regions within Guyana contribute to meeting the agricultural and food needs of regions 1 and 7, but subsistence farming is common. In Guyana, food deficit and dietary requirements rise and fall, respectively, in the period where food prices increases took place (see Figure 12) 8. Evidently, food price increases may have reduced the consumption capacity of poor households on food products. As a consequence the food deficit among the population increased or, at least remained constant. The food deficit can also be explained by income and substitution effects (changes in relative prices and spending capacity on food products). Although Guyana has improved substantially in terms of nutritional status of its population 9 the pace nutrition improvements could have more rapid if dietary changes would have been more steady. 8 Although production data in Guyana has several shortcomings, the FAO estimations based on Food Balance Sheets provide an approximation of the diet trends in the country given the supply and demand side levels. 9 According to FAO (2008) Guyana is one of the five countries in LAC that have relatively high rates of income growth, and strong productivity in the agricultural sector (along with Argentina, Chile, Peru and Uruguay). In Guyana the proportion and numbers of undernourished decreased over the 1990s and 2000s. Incidentally, the numbers and proportions undernourished are derived from statistics using the DES figures, and not on any food consumption or nutrition surveys. 35 Figure 12: Food Deficit and Average Dietary Requirement in Guyana Food Deficit Undenourished Population (kcal/person/day) Average Dietary Energy Requirement (kcal/person/day) 300 2360 250 2340 200 2320 150 2300 100 50 2280 0 2260 1990-1992 1995-1997 2000-2002 2004-2006 2008-2009 Source: FAO, 2009; Guyana Market Corporation, 2010. Generally speaking, if the percent of calories (dietary energy supply, DES) from fat in a country rises (Figure 13), it means that people may be eating “more expensive” diets but not necessarily more nutritious ones. Going along with this, based on the FAO Guyana country profile, the percent of DES from carbohydrates decreases (from 76% to 70%). Apparently, this may improve the nutritional balance of households. However, the marked shift over the past 20 years to diets high in saturated fat, sugar and refined foods, trends in Guyana might pose a double burden on households: to health and purchasing power. Over the long run, changing dietary patterns and lifestyles - spurred by urbanization, the liberalization of markets, demographic shifts and declining levels of physical activity - may contribute to overweight and chronic diseases, along with nutrition deficits. In this way, school feeding programs have the objective of preserving healthy diets among children and prevent this double burden that is present in countries like India, Philippines, Mexico, South Africa, China and Egypt where both under- and over-nutrition have seriously posed a significant burden in their health care systems (FAO, 2008). Given that Guyana is a net food exporter, recent increases in food prices provide an incentive to Guyana’s export market to expand supply, which could possibly shrink local supply. This local scarcity can in turn increase prices. A reduction in purchasing power of the rural poor population takes place. This leads to an increase in the internal demand for substitute food groups (see Graph 13). 36 Figure 13: Availability of selected foods and food groups in Guyana 1990-2009 Rice Wheat Sugars Chicken % major food products Consumed Fruits Other (oils and fats) 50 40 30 20 10 0 1990-1992 1995-1997 2000-2002 2004-2006 2008-2009 Source: FAO, 2009 An increase in food prices can not only harm the diet and nutritional intake of poor households, it can also reduce spending capacity which can bring a household to below the poverty threshold. To measure the price shock transmission to household poverty, price change simulations in treatment and control areas were estimated based on food consumption (see Annex 4). With price and consumption differences, the SFP safety net contribution to preventing households and individuals to falling below the poverty threshold can be estimated. If an SFPs is seen as a transfer program with specific safety net mechanisms towards poor households (Alderman, 2009), then not only education and health outcomes need to be assessed but also more general welfare effects should be explored. When comparing school meals to transfer programs then cost per meal compared to alternatives including take home rations and snacks are relevant. The main advantages of Guyana’s SFP compared to the general experience of other SFP around the world are adequacy, equity, cost effectiveness and sustainability, which position it as a program with high potential for welfare protection program conditioned to a national expansion. 37 Table 13: Guyana’s SFP compared to SFPs around the World Measurement General SFP Experience in the Guyana’s Case World Appropriateness Relatively easy to scale in Already implemented isolated crisis/or once piloted rural areas Adequacy Good (>10% of HH income) Good for rural areas/ Amerindian population Equity Moderate (lowest where ED low) Improve some EDU and Health indicators of poor Cost-effectiveness Poor to Moderate Moderate (20-40% non-transfer costs) Avoiding dependency Good, could be Very Good Very Good: Community/Schools and Parents Involved Sustainability Questionable Depends on medium-term funding as ECD/CCT Dynamism Usually not Not Source: Adapted from Alderman (2009) Costs and Sustainability of the SFP Safety Net The Guyana EFA-FTI SFP has also financial returns. Beyond the benefits of increasing school enrollment, height of children, and providing a safety net mechanism to poor rural households, the program shows financial viability. To illustrate the financial viability of the program it is important to estimate the unit average costs and ratios as benchmarks for comparison with other similar school feeding programs. The program costs can be estimated using the costs per child, the fixed start up costs, and the variable costs (see Annex 4 for cost details by schools). The recurrent cost is G$175 per school day (approximately US$ 0.90 per child enrolled). This covers all running costs: food, cooks' salaries, cooking fuel, water etc. The academic year comprises 192 schooldays. In addition, each school receives a one-off start-up sum. This is equal to one-third the annual recurrent costs: G$175 x 64 schooldays x number of children enrolled. With these costs we can estimate the average costs per day per school. Also by using actual data on the costs absorbed by the government of Guyana and the EFA-FTI funding it is possible to compare program expenditures with other programs. Table 14 shows the costs indicators for the Guyana SFP program and the Day Meal Program (DMP) in India, which is co-financed by the government of India and the World Food Program (WFP). Although the DMP has expanded substantially to cover 5,700 schools, it is arranged in similar fashion to Guyana’s SFP operation. 38 Table 14: Guyana SFP Costs vs. India’s Day Meal Program GUYANA INDIA Day Meal Cost Categories SFP (EFA- Ratio Program (WFP-GOI) FTI-GOG) USD Average Cost Per Day Per School 228.5 192.6 0.84 Average Number of Students per 261.2 168.4 0.64 school Number of Schools 137 5,700 41.6 Average Unit Cost (USD per 0.9 1.14 1.31 student /day) (In USD Millions) EFA-FTI Expenditure * 2.33 3.8 *** 1.64 Government Expenditure SFP *** 4.05 3.20 0.79 Total Expenditure **** 6.37 7.00 1.10 Costs for Guyana program accumulated for 2007-2009 period. Costs for India program accumulated for 2008-2009 period. * Includes community grants and training ** Includes food expenditures *** WFP contribution **** In the case of Guyana all costs are incorporated by EFA-FTI and Government. In the case of India there are also state contributions that increase the total expenditures. For comparability issues, only the Government expenditures and EFA-FTI Expenditures included in each country. This is why, although India's SFP has higher unit costs and more schools, total expenditures are marginally above Guyana's total expenditures. Sources: Own Estimations based on Guyana's SFP Program; National Mid-Day Meal Program; Akshaya Patra Foundation, 2008. The average SFP cost is around US$230 per school per day 10. The DMP has a lower average cost per day of around US$193 11. However, the DMP expanded to 5,700 schools which allowed the program to have economies of scale based on a large numbers of schools. This reduces substantially the sunk costs to start the program, since they are averaged to the total number of schools. Conversely, the SFP has a cost of $0.90 USD compared to the $1.14 USD that cost the DMP per student per school-day. The Guyana SFP has lower costs than a large scale program in India which sets it in a good benchmark against costs increases. The unit cost ratio between the DMP and the SFP is 1.31 indicating that the SFP unit cost is 30 percent lower than the DMP. 10 The number of schools covered by the EFA-FTI impact evaluation is 64. However, we consider all primary schools in Regions 1, 7, 8 and 9, covering a total of 137 schools. 11 Based on all costs of the program, including state’s/UT contributions. 39 The average cost per day per school shows a ratio of 0.84 between the DMP and the SFP, which is partially a result of the large scale presence of the DMP program 12. Expansion not only can bring benefits to a target population with reasonable program costs, it can also serve as means to achieve a financially sustainable program that takes advantage local agricultural markets to stimulate local/communities’ economies. As such, local purchase of food for school feeding is seen as a multiplier, benefiting children and the local economy at the same time. 4.2.2.2. Household diets Household diets vary considerably by ethnic group in Guyana. The diet of Amerindian communities, especially the more remote/rural ones, often lacks diversity. A diet that is monotonous risks being nutritionally unbalanced, and it is likely to be deficient in one or more micronutrients (vitamins and minerals). In Amerindian communities, diets are low in vegetables, fruit, legumes and dairy products. The reason is likely two-fold: on the one hand meat and/or fish and cassava are culturally the most acceptable commodities, and on the other hand, vegetables, fruit, legumes and dairy products are scarce and expensive. A primary objective of the nutrition training provided as part of the hinterland school feeding program is the promotion of nutritionally balanced diets with the increased consumption of vegetables, fruits and legumes. This objective is linked to the promotion of community development and participation: schools are encouraged to procure all commodities needed to provide nutritious meals from local farmers, and farmers are encouraged to grow the products for this guaranteed market, thereby increasing their incomes. The impact evaluation surveys included a limited food frequency questionnaire as a sub- component of the parents’ questionnaire. Parents were asked how often certain commodities had been consumed in their households during the previous week. Table 15 shows the results of this dietary assessment. 13 We also developed two scores: a food frequency score which is based on the frequency of consumption of each item included in the food list (1 lowest, 21 highest), and a diet diversity score which simply sums consumption/non-consumption of each food as 0/1 variables (1 lowest, 8 highest consumption), and takes no account of frequency of consumption. The means of these scores by survey round and group are shown in Table 15 and are presented as graphs in Figures 14 and 15. 12 Other local costs, procurement and program organization can play an important role in reducing costs. 13 Foods included: meat of any kind, including poultry, seafood, legumes, milk, cheese, eggs, vegetables, fresh fruits and juices. Specifically excluded were root vegetables, called ground provisions locally, which include the staple cassava. This was not intended to be a full food frequency questionnaire which would have been too time consuming to administer and would have required substantially more training. 40 Table 15: Food groups and scores by survey round and by group Percent of respondents Frequency of Once only or not at all in previous Once a day or more in previous consumption: week week C T C T Food groups R1 R2 R3 R1 R2 R3 R1 R2 R3 R1 R2 R3 Protein foods 4.4 5.2 1.9 5.3 1.5 3.7 67.1 53.5 61.1 67.9 65.9 65.1 Vegetables 14.1 20.8 21.1 11.1 18.0 12.7 50.6 37.5 40.2 56.0 47.3 53.5 Fruits & juices 32.8 40.4 49.3 29.1 31.4 27.5 13.8 8.1 9.6 16.9 20.1 19.0 Dairy products 18.7 47.1 28.5 10.5 26.8 19.0 37.9 19.3 30.4 46.8 41.8 47.6 Means ± SD C T Score R1 R2 R3 R1 R2 R3 Food frequency 9.0 ± 4.2 6.7 ± 3.4 7.4 ± 3.6 9.8 ± 4.4 8.6 ± 3.9 9.1± 3.9 score Diet diversity 5.4 ± 1.9 4.6 ± 2.0 4.8 ± 1.9 5.7 ± 1.8 5.2 ± 1.9 5.3 ± 1.7 score Source: Own estimations using Guyana IE surveys, 2007-2009 Figure 14: Mean diet diversity scores by survey round and group 5.8 5.6 5.4 Mean diet diversity scores 5.2 5 Treatment Control 4.8 4.6 4.4 4.2 4 R1 R2 R3 Source: Own estimations using Guyana IE surveys, 2007-2009 41 Figure 15: Mean food frequency scores by survey round and group 10 9.5 Mean food frequency scores 9 8.5 Treatment 8 Control 7.5 7 6.5 6 R1 R2 R3 Source: Own estimations using Guyana IE surveys, 2007-2009 The treatment group began at baseline with higher food frequency and diet diversity scores, in line with the higher wealth indicators noted in Section 5.2. Food prices rose substantially in Guyana in the period between rounds 1 and 2. Both the figures and the table show that while food frequency and diet diversity fell between rounds 1 and 2, they fell more sharply in the control group. By Round 3, the gap between the groups had increased substantially. These findings suggest that the school feeding program was able to at least partially cushion the schools in the treatment group against the increase in food prices. Another way to look at this last point is to build food frequency ratios between treatment and control areas. Figure 16 shows the ratios for 2007 through 2009 by food group categories. At baseline the gap between comparison groups was relatively small for all food groups. During 2008 the ratios of all food groups increased, particularly for fruits and dairy products. This means that the treatment areas kept the frequency of consumption relatively steady while the control group areas reduced food consumption frequency, which in turn is manifested in larger gaps between comparison groups. Section 4.3 presents SFP impacts on diet diversity and food frequencies using regression analysis. Food frequency and diet diversity scores are defined exactly as defined in this section. 42 Figure 16: Treatment and Control (T/C) Ratios of Frequency of Food Consumption Protein Foods Vegetables 3.0 Treatment to Control Proportion Ratio Fruits and Juices Diary Products 2.5 2.0 1.5 1.0 0.5 0.0 2007 2008 2009 Year Source: Own estimations using Guyana IE surveys, 2007-2009 4.2.2.3. Breakfast consumption Consuming an adequate breakfast helps a child stay alert and to benefit more fully from the education offered during the morning hours. We asked both parents and students in Rounds 2 and 3 about the kind of breakfast consumed by the student on the survey day. Table 16 shows their responses. In Sections 5.1.2 and 5.2.3 we note the facts that more than 30 percent of students with an arduous travel to school each morning received at most a light breakfast, and that single parents (especially fathers) were least likely to provide a full breakfast. 43 Table 16: Children’s breakfasts on day of interview: parent and student responses Percent of respondents Factor Category Round 1 Round 2 Round 3 All C T C T No 4.7 2.2 4.4 1.8 3.5 Yes 95.3 - - - - Students’ responses Drink only 1.1 1.5 0.9 1.8 Light breakfast 28.8 46.0 25.3 48.6 Full breakfast 67.8 48.1 72.0 46.1 C T C T None 1.0 1.5 0.3 0 Drink only 1.0 3.1 1.6 1.1 Parents’ responses Light breakfast 38.0 55.7 25.8 43.2 Full breakfast 59.9 39.7 72.3 55.8 Source: Own estimations using Guyana IE surveys, 2007-2009 Table 16 shows that in rounds 2 and 3, students attending treatment schools were significantly less likely to receive a full breakfast. This is potentially a serious concern. Are parents of treatment schools withholding a full breakfast because they know the child will receive a substantial lunch? If this is indeed the case, then the training offered by the school feeding program needs to stress the importance of an adequate breakfast, even when a full lunch is to be offered at noon. 4.3 Impact analysis: What was the impact of the school feeding program? Regression analysis can shed light on some of the impacts of SFPs on children and schools while controlling for demographic, social and economic characteristics. In addition, the data collected in three rounds is rich enough to even allow controlling for observed factors that may have enhanced comparison group differences and sample selectivity. The regression analysis considered the following outcome variables: 1) Health Indicators: Stunting (height-for-age) and Wasting (BMI) 2) School enrollment 3) Attendance 4) Educational attainment 5) Household food consumption The regression analysis cannot be limited to reporting on simple estimators. Because the groups were not selected randomly and because SFP implementation was delayed in some schools, there is sample selectivity in the treatment group, which may bias the estimates. To control for sample selectivity a technique was used to correct bias in the SFP estimates (see Annex 4 for a detailed description of the methodology). 44 4.3.1 Health Indicators Stunting (height-for-age) Stunting, measured by the height-for-age z-score (HAZ), is an important variable for the program. It allows us to explore whether the program is actually supporting early childhood development. The SFP will be highly effective if it can produce changes at this level, independently of which subgroups (age, gender) concentrate the larger share of impacts. The long term consequences of reducing stunting among children in Guyana’s hinterland regions will produce better cognitive abilities of children and higher probabilities of completion rates of primary school (Alderman, Hoddinott and Kinsey, 2006). As mentioned before, selectivity in unobserved variables might be present between comparison groups, given the baseline difference. In order to estimate the impacts and to highlight the sensitivity of impact estimators, fixed effects and sample selection (2-stage) models were run. Figure 17 shows the effects of SFP on height-for-age z-scores of children 14. There are two important aspects to highlight. First, by not correcting for sample selectivity impacts are overestimated. We expect a positive coefficient, which is the case, to infer that the SFP causes an increase in z-score mean. Second, by separating the effects between girls and boys it is found that the selectivity is predominant among girls, leading to a higher overestimation of SFP impacts for girls rather than boys. Figure 18 shows the differences in height for the children sample in 2007, 2008 and 2009. For 2008 and 2009 the mean height for children is fairly similar. However, for the baseline year (2007) the height of children is higher for the treatment group than the control group. This intuitively implies that girls selected in the treatment group at baseline were taller than control group girls. Overall, girls included in the sample were taller at baseline that boys. 14 For full description and results of the models see Annex 4. 45 Figure 17: Estimation of SFP impacts on height-for-age z-scores 0.18 Impact coefficient from Regressions Simple Panel (F.E.) Regression 0.16 Sample Selectivity correction 0.14 0.12 0.1 0.08 0.06 0.04 0.02 0 Total Boys Girls Source: Own estimations using Guyana IE surveys, 2007-2009 Figure 18: Treatment and Control Groups Students’ Height Comparison (centimeters) 140.00 Control Group Treatment Group 136.00 132.00 128.00 124.00 120.00 2007 2008 2009 Source: Own estimations using Guyana IE surveys, 2007-2009 SFP impacts on the height-for-age scores are small but not negligible. The impacts are larger for boys than girls. Overall, the mean impact coefficient of the SFP on HAZ is around 0.09. Because the mean height of children is 1.26 meters in our sample, and because our HAZ score is standardized to a normal distribution, the coefficient will be translated in a proportional shift to one standard deviation. Because the standard deviation of children’s height is 8.7 centimeters, the impact coefficient of 0.09 will imply on average an increase of 0.8 centimeters (8.7 cm x 0.09) of children that participate in SFP compared to children that do not participate in the program. An increase in 0.8 centimeters in children’s height might be small, but the long term consequences of such a height increase will be enormous in terms of human capital gains. 46 Subsample of Children at-risk of Stunting and Wasting Annex 3 shows the cutoff points to define children at-risk of stunting based on height-for-age z- scores. Based on these cut-off points, two models were run containing an interacted term between the SFP dummy and the at-risk-of-stunting dummy (see Annex 4). The results of these models with full controls show no significant impact of SFP for children at-risk of stunting, even after using the PSM matched sample. Although it was shown that the average treatment effect is small and statistically significant, the impacts for the vulnerable subgroup (at-risk children) are not significant. Wasting, on the other hand, showed a different story. The impacts were close to zero and insignificant even after sample selection correction 15. Baseline characteristics were fairly similar between treatment and control groups, but BMI’s composition is affected by the height differences shown at baseline. Figure 19: Treatment and Control Groups Students’ Weight Comparison (kilograms) Control Treatment 35 30 25 20 15 10 5 0 2007 2008 2009 Source: Own estimations using Guyana IE surveys, 2007-2009 Nevertheless, the estimated models for at-risk children of wasting (Annex 4) show that SFP increases the BMI from 1.07 (0.704/0.65 16) to 1.34 (0.874/0.65) standard deviations compared to non-participants. The average treatment effect for the complete sample of children that reported BMI was insignificant, but it was significant for the subgroup of children that are at higher risk of wasting, according to the WHO cutoffs. 4.3.2 School Enrollment Table 17 shows the number of children enrolled in schools. A typical effect of SFP program would be on enrollment because of the economic incentive it creates for parents to enroll their 15 Results of regressions not included in the paper. 16 0.65 is the standard deviation of wasting (BMI) z-score for the at-risk subpopulation. 47 children in school in order to receive a free meal. Many evaluations target school enrollment as an important outcome of SFPs. Table 17: School Enrollment in 2007 and 2009 Status 2007 2009 Difference Treatment 4,442 5,173 731 Region 1 3,640 3,841 201 Region 7 1,302 1,521 219 Control 5,290 4,897 -393 Region 1 3,480 3,648 168 Region 7 923 853 -70 Total 9,732 10,070 338 Source: Own estimations using Guyana IE surveys, 2007-2009 Enrollment at baseline was higher for control schools. More schools were incorporated in the treatment group in 2008 and 2009 making the total number of children enrolled in treatment schools higher than in control schools. The changes in enrollment can be confounding because comparison groups were dissimilar at baseline. This is suggestive of sample selection in the outcome variable and would lead to biased program impact estimates. Figure 20 shows the differences in school enrollment estimated through a simple model. We cannot infer if the program caused an increase in enrollment because simply the groups have different enrollment levels at baseline and randomization was not achieved. The impacts might be driven come from those intrinsic differences rather than from SFP participation. These estimations can be carried out by modeling the proportion of enrollment present in each region and the composition of girls and boys present in each school. The impact estimators measure the increase in the number of enrolled children by the program 17. With sample selection correction we can reduce the bias from the intrinsic enrollment differences shown at baseline. Simple OLS estimates show that the SFP increases enrollment by 200 children; with the sample selection correction model there are only around 178 children enrolled because of the SFP. The total increase in enrollment observed in the survey data equals 338 students. This means that around 53 percent of the observed change in enrollment between 2007 and 2009 could have been induced by the SFP 18. 17 See Annex 4 for a detailed description of the models used. 18 The differences in enrollment for the control group are the same between 2007 and 2009 from survey data and from official data from the Ministry of Education. It is worth to mention that Enrollment data may have measurement errors even from Administrative records from the Ministry of Education. The degree of the error is unknown but can indeed bias estimates. 48 The predictions in enrollment by region are shown in Figure 20. These includes the estimations of the mean enrollment after running the two models one with sample selection correction and the other with simple fixed-effects regression. Selectivity is present predominantly in Region 1 and total enrollment had higher predicted values for Region 7. Primary enrollment in Region 1 is higher than Region 7. A marginal increase in enrollment in Region 7 relative to Region 1 will be better captured in the model because of the higher enrollment gap existing in Region 7 between primary-age population and primary enrollment. Figure 20: Estimated SFP impact on Enrollment by Region, 2007-2009 Without sample selection 300 Sample Selection Correction Number of enrolled children 260 220 180 140 100 Total Region 1 Region 7 Source: Own estimations using Guyana IE surveys, 2007-2009 Figure 20a shows the model estimations in the number of students enrolled in SFP schools compared to non-SFP schools. The overestimation of impacts, given by selectivity present in SFP participation variable that affect the enrollment outcomes, was predominantly biased towards girls in the predictions. This is because the program had a higher effect on increasing girls’ enrollment. On average, the program creates 196 newly enrolled girls compared to an average of 161 newly enrolled boys. 49 Figure 20a: Estimated SFP impact on Enrollment by Gender, 2007-2009 Without sample selection 220 Number of newly enrolled children Sample Selection Correction 200 180 160 140 120 100 Total Boys Girls Source: Own estimations using Guyana IE surveys, 2007-2009 4.3.3 Students’ attendance As noted in Section 4.1.1, attendance rates show differences at baseline between treatment and control groups. Again, self-selection is present and thus it is necessary to correct with an econometric technique to reduce bias on impact estimates of SFP on attendance. The impact coefficients for the fixed-effects model and the two-stage sample selection 19 correction are shown in Figure 21 (full regressions in Annex 4). A positive coefficient indicates that children exposed to the program attend school more days on average. Treatment schools had higher attendance rates than control schools at baseline (sample selection), but control schools catch up with higher growth rates of attendance between 2007 and 2009. Both factors bias the effects of the program. The estimated impacts of SFP in attendance rates are reported in Figure 21, showing that SFP increased average attendance by 4.3 percent between 2007 and 2009. With the selectivity correction estimates it was found that underestimation of impacts was driven by boys. Boys showed an increase in attendance of 6.7 percent compared to an increase in girls’ attendance of 1.6 percent using the 2 stage sample selection correction. Girls have higher attendance rates than boys and non-reported attendance was primarily skewed towards boys. Boys often contribute with labor at early ages to enhance household incomes, so the economic incentive of the SFP within beneficiary households might be reflected in these findings. 19 The influence of self-selection effects must be assessed in order to accurately interpret outcome data. The analysis then should directly model the process of entry into a program and incorporating information on the factors affecting self-selection into estimation of program effects. The two-stage sample selection models are designed to address such situations. 50 Figure 21: SFP impact coefficients on Attendance Rates 2007-2009 Percent Increase in Attendance by SFP 8 Without sample selection Sample Selection Correction 7 6 5 4 3 2 1 0 Percent Increase in Boys Girls Attendance Source: Own estimations using Guyana IE surveys, 2007-2009 Figure 21a: Average School Attendance and Academic Performance, 2007-2009 Control Treatment 150 total assessment score (escore+mscore+rscore) 100 50 0 .4 .6 .8 1 .4 .6 .8 1 School-level Average of Attdrate (2007-2009) Source: Own estimations using Guyana IE surveys, 2007-2009 4.3.4 Educational Attainment As noted in Section 4.1.3, the surveys captured academic scores in multiple subjects. Regressions were run to explore the effects of the SFP on the scores reported from National 51 Assessments in grades 2, 4 and 6. In this case, the issue of non-random selection of comparison groups can be corrected using propensity score matching on students’ characteristics 20. For grade 6 there are two additional subjects tested: science and social sciences. However, the scores of 6 graders were only collected in the third round so it is not possible to draw conclusions of SFP impacts without baseline data 21. As shown in section 4.1.3 the education performance outcomes of students show ambiguous results in terms of the simple average differences between 2007 and 2009 (Table 8). The causal impact of the program can be measured only if the comparison groups can be matched to student characteristics are similar and preserve fairly similar scores for the matched subsample of treatment and control groups. Sample selection correction is not enough since the scores of students can be modeled to gain comparability of students in terms of educational outcomes. In this case, it provides better statistical justification to perform matching methods to explain the existence of any difference in the educational outcomes induced by the SFP. The techniques available for matching will only allow the identification of Average Treatment on the Treated (ATT) effects 22. To verify the importance of estimate educational performance impacts using Propensity Score Matching (PSM) method (see Annex 4), given that scores are observed, table 17a shows the effects estimated using standard fixed effects and sample selection correction (unobserved attributes explaining treatment and control allocation). In the case of Science and Social Studies (only for 6th graders) there is indeed an impact of the program. The main issue with these estimations, as previously noted, is that it is possible to increase the accuracy of the estimations by matching students with similar characteristics and scores into the treatment and control groups. In the cases of the English, Math and Reading PSM estimates cannot be compared to the F.E. and SSC models given that the coefficients for both methods are statistically insignificant and the source of bias unobserved. 20 All estimates using with sample selection correction, showed no significant impacts in Reading, English and Mathematics for grades 2 and 4. 21 However, the data will be useful to evaluate educational outcomes in future survey rounds. 22 The explanation to estimate and interpret ATT is in the annex. 52 Table 17a: Effects of SFP on different Academic Subjects Summary Tables: Educational Attainment Impacts Simple Model Model with Sample Subject (F.E.) Selection Correction Science 0.037 1.96 Significant Yes Yes Social Studies 0.221 3.79 Significant Yes Yes Significant at 10 percent level or less. Model with Sample Subject Simple Model Selection Correction English -1.13 -1.95 Significant No No Math -3.36 -4.10 Significant No No Reading 0.865 1.62 Significant No Yes Significant at 10 percent level or less. Source: Author’s estimates GY SFP Surveys The PSM common support distribution is shown in Figure 21a. The balancing of both groups with the characteristics and the performance scores make both distributions fairly similar. This depicts the level of comparability gained by estimating the scores for each block of characteristics. All other observations where there cannot be a match are dropped from the PSM estimation. However, the number of observations that remained after the PSM was enough to proceed to the next stage of the estimations 23. Table 17b shows the PSM ATT results with the kernel matching. The estimators are expressed in percent increase (decrease) from the average scores. English and Math scores show statistically significant increases of scores: 28.5 and 29.8 percent, respectively. Reading and Science scores showed statistically significant increases and close to zero. The Social Studies scores increased marginally and statistically significant, with an ample confidence interval which makes the results less robust that Math and English scores. This may be due to the fact that Social Sciences is only evaluated for 6th graders, while English and Math scores are measured for a larger sample of individuals: 2nd, 4th, and 6th graders. 23 Propensity score matching was not performed by school grade because of the small number of observations remaining in each grade subsample. 53 Figure 21b: Propensity Scores for Treated and Untreated (common support) for all individuals with at least one score reported 0 .2 .4 .6 .8 Propensity Score Untreated: Off support Untreated: On support Treated Note: Dependent Variable of the pscore model Dummy of Program Participation. Several control variables added at the individual, school and household levels. See annex for detail on propensity score (probabilistic) regression results. Source: Author’s estimates GY SFP Surveys Table 17b: Propensity Score Matching Results Average Treatment on the Treated (only for comparison sample) Grades 2, 4 and 6 Only 6th Grade Social English Reading Science Indicator Math (2007-2009) Studies (2007-2009) (2007-2009) (2007-2009) (2007-2009) Observations 2193 2193 2193 1018 1020 Treatment 410 410 410 278 284 Control 1783 1783 1783 740 736 Average Treatment on the Treated \a 28.48 *** 29.77 *** -0.03 0.54 3.15 * Standard Error \b 6.078 4.19 0.757 1.33 1.65 Confidence Interval [16.26 -- 40.70] [21.34 -- 38.20] [-1.55 -- 1.49] [-2.12 -- 3.21] [-0.16 -- 6.47] Bias 0.40 0.847 -0.158 0.32 -0.018 Average Score Treatment 40.9 50.82 18.20 60.1 64.2 Control 36.7 42.74 13.43 57.9 60.9 \a ATT (Average Treatment on the Treated). Propensity score applied to match students with similar individual and household characteristics; kernel matching was used to estimate the ATT. Impact Estimators in %. \b Estimated with bootstrapping method Source: Author's estimations Performing simple propensity score matching estimators to assess impacts at the educational attainment level increased the accuracy of the estimates. The small coefficients shown with the conventional econometric estimations could be explained by confounding factors that played a role in determining SFP impacts. The incentive to keep children in school by providing a daily 54 meal represents important gains in some educational attainment outcomes. Other outcomes are still showing very small impacts. 4.3.5 Households’ food consumption and Food Price Volatility Impacts The parent’s survey has important information regarding the diet diversity and food frequency of households. With this information regressions were estimated to explore if SFP participation enhances the diet diversity and food frequency scores 24. In principle, because the parent surveys were drawn randomly from the sample of eligible schools (treatment and control) we would not expect to find significant bias from sample selectivity. However, both simple regression and sample selection correction regressions are reported (see Annex 4). The food frequency score has a mean in 2008 of 7.3 and rose to 7.9 in 2009. A higher food frequency score implies that the household consumed more food products rich in nutrients at a higher frequency on average per week. Figure 22 shows the impact coefficients from sample selection correction regressions 25. The food frequency score is the outcome variable 26. The coefficients show that compared to 2007 SFP increased the food frequency score on average 2.2 points in 2008 and 1.3 in 2009. The standard deviations from the food frequency scores for 2008 and 2009 are fairly similar, 3.7 and 3.8 respectively. Figure 22: Impact of SFP on Food Frequency Scores (from Baseline) 3 SFP (=1 Participated) Increase in food frequency scores Water Availabiltiy 2.5 Electricity Availability Head of HH Education 2 1.5 1 0.5 0 2008 2009 Source: Own estimations using Guyana IE surveys, 2007-2009 24 Food frequency is recoded as 0 or 1, where 1= consumption frequency in last week of at least 2-3 times. The frequency is summed by food group to build the diversity score. The food frequency ranges from 0 to 25 (being 25 the highest frequency) and the diversity score ranges from 1 to 8, being 8 the highest diversity in consumption of different food groups (protein, vegetables, etc.). 25 All coefficients significant at least at 10 percent level, see Annex 4 for regression results. 26 The results were very similar to the linear regression, which implies that random selection of interviewed parents was relatively successful. See annex 4 for a detailed description of the models. 55 Other factors that contribute to an increase in the food frequency scores over baseline are proxies of household access to services and human capital. Having a household with electricity or water access increases the food frequency index by an average of 1.1. Similarly, the education level of the head of household also increases the food frequency scores. High frequency of consumption of nutritious food products enables a nutritionally balanced dietary intake which produces long term health and education benefits, particularly to children at early ages. Figure 23 shows the predictions of the food frequency scores from the regressions and food frequencies by food group. Overall, after controlling for socio-demographics and other characteristics, the predicted scores are higher for the SFP group compared to non-SFP. Regardless of the number of children in each household, SFP is an important contributor in keeping higher food consumption frequencies, thus preserving adequate child nutrition. Figure 23: Food Frequency Scores Prediction by SFP status Food Frequency Score (prediction) 12 SFP non-participants SFP Participants 10 8 6 4 2 0 1 2 3 4 5 6 7 8 9 10 Number of Children in HH Source: Own estimations using Guyana IE surveys, 2007-2009 Control Group Treatment Group 2007 (Control) 2008 (Control) 2009 (Control) Index Frequency of Consumption 100 (=100 Daily Consumption) 80 60 40 20 0 Protein Foods Vegetables Fruits and Juices Diary Products Protein Foods Vegetables Fruits and Juices Diary Products Source: Own estimations using Guyana IE surveys, 2007-2009 Another important outcome is diet diversity. Using the diet diversity score (defined in section 4.2.2.2.) it is possible to qualitatively assess how the diversity score changes based on household characteristics. This is important because school feeding programs are considered important 56 safety nets whereby households alleviate food consumption pressure that rises from adverse economic and social conditions. We computed box plots of the diversity scores with three variables (Figure 24). The first variable is SFP participation which shows higher average diversity scores than non-SFP participants. The second variable is employment status of the head of household. 27 This is relevant because when the head of household is not fully employed there are consumption restraints, particularly on food. The third variable included is “family type”: whether the household is composed of a single parent, both parents, or is an extended family. More strain in consumption would be present in single parent households (which generally have a single source of income at best). Extended family households can present a mixed picture, depending on how many adults are employed and how many family members are supported by the household’s income. First it is important to note in Figure 24 that households without the SFP have the lowest mean diversity scores and higher variability in these scores, regardless of the employment status of the head of the household (Figures 24a and 24c). Households with SFP show higher diversity scores, again regardless of the employment status of the head of the household (Figures 24b and 24d). In this way, the SFP acts as a mechanism to keep diet diversity high and relatively equal among families that have higher propensity of consumption restraints. The lowest food diversity scores with relatively large variability are seen when children in the household do not have SFP, and when the household is an extended family. This combination represents the worst case for children because they do not have access to a safety net and the household structure increases the likelihood of restraining food consumption diversity. On the other hand, families where the head of household works and children have access to SFP have higher levels of the food diversity scores in all types of family structure (single parent, both parents, extended family28), but are especially high among extended families. But these types of households show higher variability in the scores compared to those households where head does not work and children have access to SFP. If the head of household is partially or totally unemployed, household income is very low. Moreover, unless the employed household head is a single parent, the care of the kids is generally the responsibility of the mother, who is very often a housewife or only partially employed (Table 17c). Both frequency of food consumption and diet diversity are important indicators to evaluate SFPs from a social safety net standpoint. School feeding programs are often used for social protection purposes as much as or more than for education goals. 27 The questionnaire asked simply if the household head was fully employed. If the answer was no, the s/he could be either partially employed or unemployed. The former is the more likely scenario. 28 Extended families can mean more incomes – it all depends on the child to adult ratios, but most of the time more adult members in the households tend to disproportionally consume more, leaving lower levels of disposable income. 57 Table 17c: Diet Diversity Scores and Households’ Characteristics Unemployed Employed Unemployed Employed Household No SFP No SFP Have SFP Have SFP type Figure 25A Figure 25C Figure 25B Figure 25D One parent 4.67 ± 2.27 5.53 ± 1.68 5.86 ± 1.35 5.35 ± 1.69 2-parent 4.82 ± 1.83 4.82 ± 1.89 5.26 ± 1.65 5.21 ± 1.69 Extended 4.1 ± 1.97 4.11 ± 2.03 5.0 ± 1.41 6.2 ± 2.39 All types 4.71 ± 1.9 4.8 ± 1.9 5.36 ± 1.58 5.32 ± 1.77 Note: Extended family types have small sample size. Means and Standard Deviations Reported Source: Own estimations using Guyana IE surveys, 2007-2009 58 Figure 24: Diet Diversity scores, SFP participation and Household Variables Figure 24a: Partially employed, no SFP Figure 24b: Partially employed, with SFP 8 8 7 7 6 6 5 5 diverse diverse 4 4 3 3 2 2 1 1 One parent Both parents Extended family One parent Both parents Extended family hhtype hhtype Figure 24d: Fully employed with SFP Figure 24c: Fully employed, no SFP 8 8 7 7 6 6 5 5 diverse diverse 4 4 3 3 2 2 1 1 One parent Both parents Extended family hhtype One parent Both parents Extended family hhtype Source: Own estimations using Guyana IE surveys, 2007-2009 59 Figure 25 shows the results of the simulations of price shocks on the consumption of four food categories between treatment and control groups. On average, in control group areas there was a reduction of food consumption worth GY$2,000 per household per month in protein food groups, compared to treatment areas. Fruits and vegetables food groups averaged shocks in consumption are worth GY$1,700 and GY$2,300 respectively. On average, before the food prices shocks a household in control areas consumed US$2.4 less per month (all food categories) compared to treatment areas. With the food price increase from mid-2007 to the beginning of 2009, control areas consumed US$9 less per month worth of food compared to treatment areas. The SFP shielding mechanism in preserving the diet balance and the purchasing power of nutritious food can become even more pronounced in those households that have more than two children as program beneficiaries. Figure 25: Food Prices Shocks in Areas Without a Safety Net (SFP) Protein Dairy Vegetables Fruits 500 Average Monthly GY$ 0 -500 -1,000 -1,500 -2,000 -2,500 Before Food Price Shocks * Price Shock Period ** Notes: * includes period from January 2006 to June 2007; ** includes period from July 2007 to December 2009 Source: Own estimations using Poverty Figures (BOS); Guyana Central Bank data and GMC Prices. Table 13 shows the estimation of poverty rate and population at risk of falling into poverty before and after the food prices crisis between treatment and control areas. Using the national poverty rates in rural areas and assuming that the program would operate in all regions, rural areas without the program would have 6 percent more poverty prior to the food price crisis. Because the SFP contributes to developing local agricultural production, prices will be less volatile than in those areas without the SFP which are more likely to import agricultural products from other regions. In consequence, the period of food price increases (between the second half of 2007 and the beginning of 2009) increases by 21.5 percent the population at risk of falling into poverty in control areas. Although it is hard to conclude whether these poverty risk changes are dominated by price differentials prior to the food crisis, the food crisis itself, or other local contextual factors, it is illustrative of the risk gaps present in areas with a safety net mechanism compared to areas that lack these types of programs. These relationships can be estimated for the child population at risk, based on the proportion of child population. Around 2,500 more children in all regions in rural areas without a safety net fall in the poverty risk category before the food prices crisis. 60 After the food prices crisis, the number of children at risk of falling into poverty will rise to 9,000. Based on the proportion of children that live in regions 1 and 7, control areas would have had 150 more children at risk of falling into poverty compared to treatment areas, before the food price crisis. After the food price crisis more than 510 children in control areas would be at risk of falling into poverty. This illustrates under modest and conservative assumptions (see Annex 4) that safety nets can provide solid mechanisms against adverse shocks that limit poor households spending power, particularly those households with young children. Table 17d: Poverty Impact of the Food Price Crisis and the SFP Safety Net Before Food Price Shock Poverty Change from Price Difference Price Shocks * Period ** Rural Poverty Change (%) all regions 6.0 21.5 Population at risk rural areas (#) all regions 12,595 44,124 Rural Children at risk of poverty 2,572 9,011 Regions 1 and 7: Children at Risk of Poverty 148 517 * Includes from January 2006 to June 2007 ** From July 2007 to December 2009 Source: Own estimations using Poverty Figures (BOS); Guyana Central Bank data and GMC Prices. 5 The schools and their communities 5.1 The learning environment 5.1.1 The schools and their staff Table 18 shows the distribution, ranges and medians of school enrolment in each survey round using the school-level and teaching staff survey. Median school enrollment rose steadily with survey round. Enrollment was substantially larger in the treatment schools. For control schools, the median rose from 70 to 80, and for treatment schools it rose from 132 to 152. 61 Table 18: Schools – enrollment Percent of schools Categories Round 1 Round 2 Round 3 3. Enrollment ≤ 50 students 24.1 23.0 27.8 51 – 100 41.4 32.8 29.6 101 – 200 15.5 23.0 22.2 201 – 500 12.1 13.1 14.8 > 500 6.9 8.2 5.6 Range 13 - 841 15 - 976 15 - 943 Median 76 79 84 Source: Own estimations using Guyana IE surveys, 2007-2009 Table 19: Classes Factor Categories Percent of teachers responding Round 1 Round 2 Round 3 Multigrade 51.8 52.9 49.4 1. Class type Single grade 48.2 47.1 50.6 2. Repeaters None 66.4 72.9 65.1 1–5 20.5 15.0 21.4 ≥6 13.1 12.1 13.5 Round 1 Round 2 Round 3 Categories Median Range Median Range Median Range 2. Enrollment: Boys 6.0 1 – 25 11.5 2 – 39 11 1 – 41 multigrade Girls 7.2 1 – 21 10.5 2 – 38 9.5 1 - 49 classes Total enrollment 12.0 3 – 41 21.5 7 - 73 21 4 - 90 3. Enrollment: Boys 17.0 4 – 52 16 3 – 41 15 3 – 52 single grade Girls 14.0 3 – 62 16 0 – 51 14 2 – 54 classes Total enrollment 33.0 10 - 114 31.5 3 - 92 30 5 - 106 Source: Own estimations using Guyana IE surveys, 2007-2009 Schools had single grade classes, multigrade classes or a mix of the two. Of the 64 schools surveyed, twelve had single grade classes only, 31 had multigrade classes only and 21 had both types of classes. Table 19 provides data on the classes covered by the survey, using the class teacher questionnaire. Median enrolment was higher in single grade classes. Profiles of head teachers and class teachers are provided in Annex 5 (Tables A5-1, A5-2) 62 5.1.2 The students Table 20 provides some basic information on the students included in the sample. As grade coverage increased with survey round, so too mean age and mean number of siblings rose with survey round. Table 20: Student profiles Percent of students Factors Categories Round 1 Round 2 Round 3 Male 50.8 49.4 50.2 1. Sex Female 49.2 50.6 49.8 ≤7 7.8 6.8 17.0 8 25.3 24.2 17.4 2. Age (years) 9 29.8 21.3 19.8 10 24.9 23.9 17.7 ≥ 11 12.3 23.8 28.2 2 32.5 30.4 24.1 3 35.2 22.6 20.9 3. Grades 4 32.3 26.5 20.0 5 20.5 15.9 6 19.1 ≤2 17.8 16.7 14.2 4. Number of 3–4 30.2 30.7 30.5 siblings 5–6 28.5 28.4 29.2 ≥7 23.6 24.1 26.1 With both parents 76.1 73.4 74.7 5. Living With mother only 10.5 13.1 12.6 arrangements With father only 6.8 3.8 3.5 Other 6.2 9.8 9.3 Means ± SD Round 1 Round 2 Round 3 Age (years) 9.1 ± 1.2 9.4 ± 1.4 9.9 ± 1.7 Number of siblings 4.8 ± 2.5 4.9 ± 2.5 5.1 ± 2.5 Source: Own estimations using Guyana IE surveys, 2007-2009 A significantly larger proportion of students in the treatment group lived in single parent homes: 20.6 percent in the treatment group as compared to 12.6 percent in the control group. We examined whether single parent children differed significantly from those living with both parents with regard to key outcome indicators, in both treatment and control groups. There was no significant difference in school attendance, but children living with their mothers only were less likely to be stunted than all other children, and children living with their father alone were more likely to obtain higher mathematics scores in the National Assessments. 63 Students were asked also if they had consumed a breakfast on the day of the survey, and if so whether it had been a beverage only, a light or a full breakfast (see also Sections 4.2.2.3 and 5.2.3). Single parent families were least likely to provide a full breakfast when compared to families with both parents present, in both the treatment and the control groups. Of the single parent families, students with only the father present fared worst in relation to breakfast. 5.2 The parents, their households and communities One important aspect of the hinterland school feeding program is the promotion of the communities’ participation in all the program's activities. Equally important, especially for the impact analysis of the survey data, is the extent to which treatment schools and their communities differ from control schools and their communities, in terms of the households’ wealth, the amenities they enjoy and their access to services. The surveys therefore gathered information on the schools’ communities and their households. This section presents information on the parents and their households, the parents’ participation in school activities, and their children’s ease of access to their schools. More detailed information is provided in Annex 5, Tables A5-5 and A5-6. 5.2.1 Socio-demographic profiles of parents and their households Table 21 presents a profile of the parents interviewed and their households. Details of the occupations of the mothers and fathers are given in Table A5-5 (Annex 5). Focusing on Round 3 data, we see significant differences between communities served by treatment schools and those served by control schools. In treatment communities there are: • More single parent families; • More female household heads; 29 • Fewer household heads in full employment; • Fewer children; • Mothers and fathers with higher education levels. In Round 3 only, students were asked if they were able to read a full page from a storybook, if there were any books or newspapers in their homes, and if they had ever heard their mother and father read a book or newspaper. Responses are given in Table 22. A significantly higher proportion of students in the treatment group stated that they had heard their mother read a book or newspaper. 29 In some cases, the household head may have been a male absent due to employment in mining or logging. 64 Table 21: Profile of parents and their households, by survey round Percent of respondents Factor Category Round 1 Round 2 Round 3 C T One-parent 13.2 13.2 7.9 19.0 1. Type of household Two-parent 71.2 75.4 81.9 72.5 Extended 15.6 11.4 10.2 8.5 2. Sex of household Female 17.7 31.5 20.2 32.1 head Male 82.3 68.5 79.8 67.9 3. Relationship of Parent 93.4 93.2 92.5 91.7 respondent to child Guardian / other 6.6 6.8 7.5 8.3 Female 66.3 70.3 59.6 79.3 4. Sex of respondent Male 33.7 29.7 40.4 20.7 ≤ 29 years 24.9 18.8 18.0 18.8 30 – 39 36.0 38.1 35.5 38.7 5. Respondent’s age 40 – 49 28.7 31.9 31.2 27.2 ≥ 50 10.3 11.2 15.3 15.2 6. The household head Yes 63.0 65.5 25.3 40.3 employed full-time? No 37.0 34.5 74.7 59.7 1–3 32.2 29.1 25.1 43.5 7. Number of children 4–5 31.7 30.8 29.7 28.0 in household 6 – 16 36.1 40.1 45.2 28.5 None 6.5 17.5 17.4 9.0 Primary 73.5 67.9 72.9 65.6 8. Mother’s education Secondary 15.0 12.6 7.8 23.3 Tertiary / other 5.0 1.9 1.9 2.1 None 2.3 14.8 17.3 9.9 Primary 68.8 66.2 66.6 59.3 9. Father’s education Secondary 22.7 15.8 13.7 24.7 Tertiary / other 6.1 3.2 2.5 6.0 Source: Own estimations using Guyana IE surveys, 2007-2009 Table 22: Literacy (Round 3 only, students’ responses) Control (SF 0) Treatment (SF 1) Child can read a page 63.3 63.9 Has books or newspapers in home 73.5 68.7 Has heard mother read book or newspaper 75.4 80.9 Has heard father read book or newspaper 71.7 70.7 Source: Own estimations using Guyana IE surveys, 2007-2009 65 5.2.2 Household possessions and access to services and amenities Table 23 gives information on households’ access to services. Although the responses for all services differ substantially between survey rounds, in each round respondents from communities with the treatment schools have better access to piped water, electricity and roads. Table 23: Access to services Percent of respondents Round 1 Round 2 Round 3 C T C T C T Piped water 34.9 47.6 15.5 21.5 10.6 24.0 Electricity 20.5 40.1 23.5 37.9 38.3 51.6 Roads 5.1 17.1 2.6 9.7 3.2 9.9 Source: Own estimations using Guyana IE surveys, 2007-2009 Table 24 summarizes information for households’ amenities and possessions in Round 3 only. While information on household possessions was obtained at each survey round, the rest of the information given in Table 24 was obtained only in Round 3. Full details of household possessions for each survey round are given in Table A5-6 (Annex 5). Table 24: Amenities and possessions: parents’ and students’ responses (Round 3 only) Categories Control Treatment A. Parents’ responses Bathroom Yes 37.3 59.9 None 15.8 0.7 Toilet Pit latrine 81.2 89.3 Flush toilet 3.0 10.1 0 – 3 items 37.9 18.8 Possession score 4 – 6 items 41.1 46.9 7 – 10 items 21.1 34.4 B. Students’ responses None 9.6 0.3 Toilet Pit latrine 82.3 87.3 Flush toilet 8.1 12.3 None / thatch 10.2 1.9 Home’s walls Wood, concrete or both 89.8 98.1 Yes 8.0 9.2 Livestock No 92.0 90.8 Source: Own estimations using Guyana IE surveys, 2007-2009 66 As with access to services, both parents and students from communities with treatment schools indicate significantly better amenities and a greater number of possessions than those from control schools. In short, the information shown in Tables 23 and 24 indicates strongly that communities served by treatment schools are wealthier and have better access to services than those served by the control schools. 5.2.3 How children get to school In Rounds 2 and 3 only, we asked both parents and students how the child goes to school and to estimate the time taken to travel to school. Their responses are given in Table 25. A large proportion of children walk to school, or travel by paddle boat. Of those who travel by paddle boat, more than 25 percent take an hour or more to reach school. Of those who walk, more than 15 percent take thirty minutes or more to reach school. Of these children with long and arduous travel to school, more than 30 percent leave homes with at most a light breakfast to sustain them. These data are based both on parents’ and students’ responses. Table 25: Students’ access to schools Percent of respondents Parents’ responses Categories Round 2 Round 3 < 15 mins 41.2 47.0 1. Time taken for child 15 – 30 mins 29.8 26.1 to travel to school 30 – 60 mins 16.2 18.1 > 1 hour 12.8 8.9 Walks 63.8 69.0 2. How child gets to Bicycle 0.7 3.5 School Paddle boat 30.2 24.0 Speed boat + other 5.4 3.5 Students’ responses < 15 mins 38.1 34.1 3. Time taken for child 15 – 30 mins 27.2 30.8 to travel to school 30 – 60 mins 22.9 22.7 > 1 hour 11.8 12.4 Walks 65.6 70.1 4. How child gets to Bicycle 2.6 5.2 School Paddle boat 23.5 21.4 Speed boat + other 5.2 3.3 Source: Own estimations using Guyana IE surveys, 2007-2009 67 6 Conclusions The impact evaluation of Guyana’s hinterland school feeding program was successfully conducted in sixty-four schools in Regions 1 and 7. Survey data were collected from schools, students, teachers and parents in three rounds in 2007, 2008 and 2009. Out of the 64 schools included in the surveys, twenty one had started school feeding before Round 3 and constituted the treatment group. The rest formed the control group. Because of the procedures followed to assign communities and schools into treatment and control groups, self-selection in the sample produced uneven treatment and control group characteristics, in some cases differing significantly at baseline. In addition, sample selectivity was manifested in student samples because of absence in the day when the survey and the tests were applied. Parametric and non- parametric corrections in both cases were carried to reduce, not eliminate, these sources of bias in the estimates. Primary level education poses challenges to the Ministry of Education in its efforts to improve academic achievement in the hinterland regions, mainly due to: • Poor school attendance; • High levels of stunting, associated with a poor diet and with important consequences to cognitive development; • Poverty and limited access to essential resources and services impose constraints to increase nutritionally-balanced diets. The conceptual framework to evaluate the SFP addresses the following questions: • Has the program had a positive impact on students’ attendance, nutritional status and school performance? • Has the program led to improvements in students’ classroom behavior and parental participation in school activities? • Has the program provided a safety net against food price increases? • Has the program improved diets of households? The evaluation’s most robust findings are listed as follows: • The SFP increased average attendance by 4.3 percent between 2007 and 2009. • Control schools showed consistently higher levels of stunted and severely stunted children in all survey rounds, because of their poorer economic status (selectivity) and perhaps also because they had more children of Amerindian ethnicity than treatment schools. Children in treatment schools grew 0.8cm more than children attending control schools, a small but significant difference. • Children’s classroom behavior improved with the introduction of a school meal. By Round 3, students in treatment schools had higher levels of class participation than those in control schools. Conversely, students’ disconnect and distraction from classroom activities showed a lower incidence in treatment schools. 68 • SFP had impacts on academic performance for Math and English scores. However, further academic performance will take time to achieve, especially for Reading and Science subjects. Better attendance, improved student participation in classroom activities, and better nutritional status, when combined with the EFA-FTI’s efforts to improve schools’ human and physical resources, must ultimately promote the Ministry’s aim of improving academic achievement in the hinterland regions. • Community participation in school feeding-related activities has been achieved: parents actively participate in cooking and serving meals, and in growing and providing food commodities. • SFP implementation coincided with a period of uncertainty and high volatility in food and agricultural commodities’ prices. A daily meal to children in poor household represents a safety net mechanism from adverse price shocks. An increase in food prices harms the spending capacity of households which in turn affects food consumption, nutritional intake and poverty levels. • The diet of rural and Amerindian communities often lacks diversity. In poor and Amerindian communities, diets are low in vegetables, fruits, dairy products and legumes. The SFP improved diet diversity and frequency of food consumption in treatment communities as compared to control communities, despite higher food prices. During the food price shocks the gap in food consumption frequency and diet diversity between control and treatment groups increased substantially. The SFP has thus successfully provided a safety net against food price increases. • The financial returns of the SFP can be substantial if it expands considerably. Expansion can also bring a safety net mechanism to regions and communities facing economic hardship. Compared to other large-scale programs SFP has a relatively low cost per child enrolled. Expansion to all rural schools could drop the unit cost by half. The evaluation findings and the experiences of the SFP raise a number of important issues that need the attention of the Ministry of Education: • Although attendance was indeed better in treatment schools, absenteeism remains a serious concern. More than 59 percent of all children included in the evaluation were absent for at least one day in the two weeks preceding the survey day. And of these children, more than 22 percent gave labor of some kind as a reason for their absence. Labor included care of younger siblings, work on the farm or in the home. • A further 6.6 percent of children with absences in the two weeks preceding the survey were absent because of the household’s economic condition: no food, no uniform, no cash or no stationery. Interestingly, of those who gave lack of food as the reason for absence, all children except one attended control schools where no lunch was offered. • The short-term hunger which occurs if a child receives an inadequate breakfast and/or has a long and arduous journey to school, is a major contributor to poor classroom behavior and academic achievement (see Annex 3). Single parents were least likely to offer a full breakfast to their children. An important finding of the evaluation was that students attending treatment schools were significantly less likely to receive a full breakfast. Are parents of treatment school children withholding a full breakfast because they know the child will receive a substantial lunch? If this is indeed the 69 case, then the training offered by the school feeding program needs to stress the importance of an adequate breakfast, even when a full lunch is to be offered at noon. • Sustainability of the hinterland school feeding program when external funding ends is clearly a serious issue and one that needs urgent attention. When considering the obvious educational and nutritional benefits of the Program, the Ministry needs also to take into account the important role of the Program in providing a safety net, as well as aspects that address poverty in hinterland communities. Thus, for example, the Program offers guaranteed markets to farmers for their produce and employment for women as cooks. • While the SFP is a cost-effective program, the Ministry may wish to consider ways by which the Program’s costs may be reduced so as to increase the likelihood of sustainability. These include modest parental cash contributions, a reduction in the number of days when a meal is provided, establishing school gardens to provide produce for the meals, encouraging home gardens, seeking partnerships with the private sector, and finding ways in which the school kitchens could be used on weekends and during school holidays to raise funds through small business enterprises. These are possibilities that should be discussed with community members. Finally, the SFP’s achievements with regard to improving community participation in school activities can make a key contribution to finding solutions to many of the challenges facing the Ministry of Education. 70 7 REFERENCES Adelman, S. Gilligan, D. and Lehrer, K. 2009. How Effective are Food For Education Programs? A Critical Assessment of the Evidence from Developing Countries. IFPRI. Food Policy Review No. 9. Alderman, H. (2009) School Feeding as a Transfer Program: New Perspectives, New Evidence. SFP Presentation World Bank, Washington D.C. Alderman, H., de Walque, D., and Kazianga, H. 2009. Educational and Health Impacts of Two School Feeding Schemes. PRWP 4976. World Bank. Angrist, J.D. and Guryan, J. 2005. Does Teacher Testing Raise Teacher Quality? Evidence from State Certification Requirements. IZA. Dp. No. 1500. Germany. Angrist, J.D. and Krueger, A. 2001. Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments. The Journal of Economic Perspectives, Vol. 15, No. 4. pp. 69-85. Ahmed, A. 2004. Impact of Feeding Children in School: Evidence from Bangladesh. IFPRI, Washington D.C. Banerjee, Abhijit, Cole, Shawn, Duflo, Esther and Linden, Leigh. (2007) “Remedying Education: Evidence from Two Randomized Experiments in India.” Quarterly Journal of Economics, August, 122(3), pp. 1235-1264. Bank of Guyana. 2008. Annual Report. Georgetown, Guyana. Behrman, J. and Todd, P. 1999. Randomness in the Experimental Samples of Educational, Nutrition and Health Program (PROGRESA). IFPRI. Washington D.C. Behrman, J. and King, E. 2001. Household schooling behaviors and decentralization. Economics of Education Review 20 321–341. Behrman, J., Ross, D. and Sabot, R. 2008. Improving quality versus increasing the quantity of schooling: Estimates of rates of return from rural Pakistan. Journal of Development Economics 85 94–104. Behrman, J. and Wolfe, B. 1987. Investments in Schooling in two generations in pre- revolutionary Nicaragua. The Roles of Family Background and School Supply. Journal of Development Economics. 27 397-419. North Holland. Behrman, J., Cheng, Y., and Todd, P. 2004. Evaluating preschool programs when length of exposure to the program varies: a non parametric approach. The Review of Economics and Statistics. Vol 86 No. 1. pp. 108-132 71 Binder, Melissa. “Community Effects and Desired Schooling of Parents and Children in Mexico.” Economics of Education Review, vol. 18 (1999), pp. 311-325. Bundy, D. et al. 2009. Rethinking school feeding : social safety nets, child development, and the education sector. WFP-World Bank. Budd, J. 1993. Changing food prices and rural welfare: A nonparametric examination of the Côte d’Ivoire, Economic Development and Cultural Change 41 (April): 587–603. Barrett, C., and P. Dorosh. 1996. Farmers’ welfare and changing food prices: Nonparametric evidence from rice in Madagascar, American Journal of Agricultural Economics 78 (August): 656–669. Connelly, Rachel and Zhenzhen Zheng. “Determinants of School Enrollment and Completion of 10 to 18 Year Olds in China.” Economics of Education, vol. 22 (2003), pp. 379-388. Cuddeback, G. et al. 2004. Detecting and Statistically Correcting Sample Selection Bias. Journal of Social Service Research, Vol. 30(3) Dangour A.D. (2001) Growth of upper- and lower-body segments in Patamona and Wapishana Amerindian children (cross-sectional data). Annals of Human Biology, Vol. 28, No.6, 649-663 Deaton, A. 1989. Rice prices and income distribution in Thailand: A non-parametric analysis. Economic Journal. 99 (395) (Supplement): 1–37 Deaton, A. 1997. The Analysis of Household Surveys. A Microeconometric Approach. World Bank. De Hoyos, R. and Lessen, R. 2008. Food Shares in Consumption: New Evidence Using Engel Curves for the Developing World. World Bank. Mimeo. Den Berg, I. 2008. Impact of the Ghana School feeding program on Nutritional and Health status of Ghanaian pupils, Central Region. Wageningen University. WP 801215. Deolalikar, A. 1997. The Determinants of Primary School Enrollment and Household Schooling Expenditures in Kenya: Do They Vary by Income? World Bank Duflo, E., Hanna, R., and Ryan, S. 2008. Monitoring Works: Getting Teachers to come to school. CEPR. DP 6682. Duflo, E., Dupas, P., and Kremer, M. 2008. Peer Effects and the Impact of Tracking: Evidence from a Randomized Evaluation in Kenya. J-Pal. Mimeo. FAO. 2009. Country Profile: Food Security Indicators, Guyana. 72 FAO. 2008. Assessment of the double burden of malnutrition in six case study countries. Rome, Italy. FAO, 2008a. Undernourishment around the world. High food prices: another 75 million hungry. FAO State of the Food Insecurity around the World in 2008. Glewwe, Paul W., Kramer, Michael and Moulin, Sylvie. (2007). “Many Children Left Behind? Textbooks and Test Scores in Kenya.” National Bureau of Economic Research (Cambridge, MA) Working Paper No. 13300. Glick, P. and Sahn, D. 2000. Schooling of girls and boys in a West African country: the effects of parental education, income, and household structure. Economics of Education Review 19 63–87. Greenleaf Market. 2007. School Breakfast Program Cost/Benefit Analysis. UW-Extension Family Program. Guyana Market Corporation. 2010. Quarterly Local Price and Commodity Data: 2007-2009. GMC Guyana. Guyana Ministry of Education. 2008. Guyana Education Strategic Plan, 2008-2013. Guyana Ministry of Education. September 30, 2002. Education for All – Fast Track Initiative Country Proposal, Credible Plan. Hall, A. et al. 2007. An evaluation of the impact of a school nutrition program in Vietnam. Public Health Nutrition: 10(8), 819–826 Hannum, E. and Buchham, C. 2005. Global Educational Expansion and Socio-Economic Development: An Assessment of Findings from the Social Sciences. World Development Vol. 33, No. 3, pp. 333–354 Holmes, J. 2003. Measuring the determinants of school completion in Pakistan: analysis of censoring and selection bias. Economics of Education Review 22 249–264/ Ivanic, M. and W. Martin. 2008. Implications of higher global food prices for poverty in low- income countries. World Bank Policy Research Working Paper No 459 Kremer, M., and Miguel, E. 2004. The Illusion of Sustainability. NBER, WPS 10324. Kremer, M. and Miguel, E. Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities. Econometrica, Vol. 72, No. 1 pp. 159-217 73 Lloyd, C., Mete, C., and Grant, M. 2009. The implications of changing educational and family circumstances for children’s grade progression in rural Pakistan: 1997–2004. Economics of Education Review 28 152–160 Lustig, N. 2009. Coping with Rising Food Prices: Policy Dilemmas in the Developing World. Center for Global Development. Working Paper No. 164. Macours, K., Schady, N., and Vakis, R. 2008. Cash Transfers, Behavioral Changes, and Cognitive Development in Early Childhood. PRWP 4759. World Bank. Martorell et al (1988). Poverty and stature in children. In Linear Growth Retardation in Less Developed Countries ed J.C. Waterlow (NY: Vevey / Raven Press) Osterbeek, H., Ponce, J., and Schady, Norbert. 2008. The Impact of Cash Transfers on School Enrollment. PRWP 4645. World Bank. Schultz, T. Paul. Final Report: Impact of PROGRESA on School Enrollments. Washington, D.C.: International Food Policy Research Institute, April 2000. –––––. Impact of PROGRESA on School Attendance Rates in the Sampled Population. Washington, D.C.: International Food Policy Research Institute, February 2000. –––––. "School Subsidies for the Poor: Evaluating the Mexican Progresa Poverty Program." Center Discussion Paper No. 834. New Haven, CT: Economic Growth Center, Yale University, August 2001. Skoufias, E., Di Maro, V., González-Cossío, T. and Rodríguez-Ramírez, S. (2009) "Nutrient consumption and household income in rural Mexico," Agricultural Economics, International Association of Agricultural Economists, vol. 40(6), pages 657-675. Skoufias, E. 2005. Guyana EDMI Methodology. World Bank. Ravallion, Martin. The Mystery of Vanishing Benefits: Ms. Speedy Analyst’s Introduction to Evaluation. 2153. Washington, D.C.: Poverty and Human Resources, Development Research Group, The World Bank, July 1999. Taulman, L. and Marshall, M. 2007. Limited Resource Families and Child Health Status: A Case Study in Montero, Bolivia. AAEA 2007 Annual Meeting Selected Poster Presentation. Vailas LI, Blankenhorn DH, Johnson RL, Selzer RH. 1987. A computerized quantitative food frequency analysis for the clinical setting: use in documentation and counseling. Journal of American Diet Association. Nov;87(11):1539-43. Waterlow, J.C. & Schürch, B (eds.) Causes and Mechanisms of Linear Growth Retardation. European Journal of Clinical Nutrition (1994) Vol 84 (Suppl. 1), page S210 Wodon, Q. and Zaman, H. 2008. Rising Food Prices in Sub-Saharan Africa: Poverty Impact and Policy Responses. World Bank WPS 4738. 74 Wishik, S. and Van der Vynckt. 1979. The Use of Nutritional 'Positive Deviants' to Identify Approaches for Modification of Dietary Practices. AJPH Vol. 66, No. 1 Wise and Birell, 2002. Design and analysis of food frequency questionnaires - review and novel method. International Journal of Food Sciences and Nutrition, Volume 53, Issue 3, pages 273 – 279 World Bank. 2006. Guyana Poverty Assessment. Zezza, A., et al. 2008. The Impact of Rising Food Prices on the Poor. ESA Working Paper No. 08-07. FAO 75 ANNEX 1 LIST OF SCHOOLS SURVEYED (Schools in bold are treatment schools) School ID School name REGION 7 Cluster 1: Lower Mazaruni 1 Agatash 2 Batavia 3 Butukari 4 Karrau Creek 5 St. John the Bap. 6 Wineperu 8 Itaballi 9 72 Miles 10 Two Miles 11 St. Anthony’s 12 Kartabo 13 Holy Name 14 St. Mary’s Cluster 2: Middle Mazaruni 15 St. Martin 16 St. Martin’s Annex 17 Kurupung 18 Isseneru REGION 1 Cluster 4: Matarkai 21 Port Kaituma 22 Arakaka 23 Baramita 24 Matthews Ridge 25 Sebai 26 Falls Top Cluster 5: Mabaruma 27 Almond Beach 28 Aruka Mouth 29 Barabina 30 Black Water 76 School ID School name 31 Hobedia 32 Hosororo 33 Hotoquai 34 Kamwatta (Mabaruma) 35 Lower Kaituma 36 Lower Waini 37 Mabaruma 38 Peter & Paul 39 Sacred Heart 40 St. Anselm’s 41 St. Anthony’s 42 St. Cyprian’s 43 St. Dominic’s 44 St. John’s 45 St. Margaret’s 46 St. Mary’s 47 St. Ninian’s 48 Unity Square 49 Wauna 50 Yarakita 67 White Water Cluster 6: Moruca 51 Assakata 52 Kamwatta (Moruca) 53 Karaburi 54 Kokerite 55 Kwebana 56 Santa Cruz 57 Santa Rosa 58 St. Bede’s 59 St. Nicholas 60 Wallaba 61 Waramuri 62 Warapoka 63 Kokerite Annexe 64 Father’s Beach 65 Chinese Landing 66 Waramuri Annex Note: Kokerite annex is now considered part of Kokerite school. The total number of schools adds up to 64. No schools were surveyed in Cluster 3. Schools that began school feeding after Round 3 were kept in the control group. Source: SFP 77 ANNEX 2 IMPACT EVALUATION TEAM Project organizers 30 Edward Jarvis, EFA/FTI Program Manager, Ministry of Education Angela Demas, Sr. Education Specialist, The World Bank Trainers Norma Howard, Director, Food Policy Division, Ministry of Health Suraiya Ismail, Public Health Nutritionist, Director, Social Development Inc. Anthony Hunte & Edward Jarvis, Ministry of Education Enumerators Region 1 Ellis Alphonso Francine Bennet James Carlos Rodrigues Seeta Antonio Oliver John Jacqueline Rodrigues Sharon Atkinson Isabella Moonsammy Genevieve Rufino Pablino Cupido Cecily Phillips Magdalyn Simon Lawrence David Kelly Pierre Rosana Smith Stanley Fredericks Alex Ragwen Helen Thomas Ignatius Gomes Karon Roberts Enan Williams Natalie Henry Melissa Robinson Region 7 Thea Chase Diannah Fredericks Deanne Jones Felicity Da Silva Onica James Keisha King Winston Ferreira Youlanda James Beverly Shewram Data management Patricia Roopnaraine & Lalita Murphy – data entry (Ministry of Education) Sharon James & Nini Osaze – data entry (SDI) Suraiya Ismail – coding, file management, data analysis (SDI) Christian Borja – data analysis (World Bank) Report preparation Suraiya Ismail Christian Borja Edward Jarvis Angela Demas 30 Advisory inputs were also provided by Mrs. Evelyn Hamilton, Chief Planning Officer, MOE and by staff of the World Bank. Dr. Nandram Persaud, Procurement Officer, MOE, and his staff organized the speedy delivery of the anthropometric equipment. 78 ANNEX 3 NUTRITION NOTES Nutritional status indicators Nutritional status is assessed by measuring the weight and height of the individual, and then comparing these quantities to international reference standards obtained from healthy well- nourished populations. For this study, the latest WHO reference standards were used, published in 2007. Using weight and height, two indicators are calculated for this sample of primary school children: • Height for age – a measure of stunting or chronic malnutrition. This is calculated by comparing the height of the individual to the reference height of an individual of the same age and sex. • Body Mass Index (BMI) for age – a measure of wasting (acute malnutrition) or obesity. BMI is calculated first 31, then the BMI is compared to the reference BMI of an individual of the same age and sex. Nutritional status of the individual is then defined by means of cut-off points below the individual’s reference median: Table A3-1 Cutoff Points World Health Organization Cut-offs below the reference Nutritional status median Stunting (height for age) and wasting (BMI for age) < -3 standard deviations (SD) Severely Stunted or wasted < -2 standard deviations (SD) Stunted or wasted ≥ -2 SD and < -1 SD At-risk of stunting or wasting ≥ median Not stunted or not wasted Note: WHO classification and cutoffs for defining the severity of stunting and wasting changed recently. For an update and description of cutoffs see: http://www.who.int/nutgrowthdb/about/introduction/en/index5.html a) Stunting Stunting or chronic malnutrition is generally considered to be the consequence of a diet lacking diversity and / or a high level of disease. A monotonous diet is likely to be deficient in a number of nutrients. A stunted child may be receiving adequate quantities of energy (calories), protein and fat, but yet be lacking in vitamins and minerals. Consequences of stunting include poor 31 BMI = weight height2 79 mental development and poor school achievement. Other consequences may be linked to deficiencies of specific vitamins and minerals. Stunting has been the subject of the classic “nature vs nurture” debate i.e. whether it is the result of a person’s genetic make-up or because of environmental insults suffered during a child’s growth periods, insults such as high morbidity or poor nutrition. In favor of the nurture position is the fact that ethnic groups, such Indian, Chinese and Japanese, traditionally considered to be of short stature, are now achieving heights comparable to those found in European and North American populations. This is especially so when economic status improves or when these groups migrate to the States or Europe. A PhD study conducted in Guyana 32 compared the heights of two groups of Amerindians: the group with the better economic status and a more diverse diet was taller than the other. The diet of Amerindian communities, especially the more remote ones often lacks diversity. Specifically, as found in this baseline survey, consumption of vegetables is low. Consumption of dairy products and legumes may also be low. It is important to appreciate that what matters with stunting is not the fact of being short, but rather the failure to achieve one’s full growth potential, and why this failure has occurred. Graph A3-1 Stunting Distribution SFP Students 2007-2009 (Height-for-Age) 4 Height-for-Age z-score Second Follow up Seve Stunting Moderate At risk (stunting) Stunting -2 0 -4 2 -4 -3 -2 -1 0 1 2 3 4 Height-for-Age z-score Baseline Source: Own estimations using Guyana IE surveys, 2007-2009 b) Wasting 32 Alan Dangour (1998) Growth of body proportions in two Amerindian tribes in Guyana. PhD thesis, University College, London. 80 Wasting, or acute malnutrition is the result of an inadequate intake of energy (calories) and / or a recent illness. The diet is primarily deficient in energy, but going along with what is essentially an insufficient food intake are deficiencies in many other nutrients. Extreme thinness, or severe wasting, of the form seen in famine conditions, is associated with a high risk of disease and, ultimately, death. Graph A3-2 Wasting Distribution SFP Students 2007-2009 (Body Mass Index) 4 Severe Wasting Moderately Wasting At risk (Wasting) BMI Z-score Second Follow-up -2 0 -4 2 -4 -3 -2 -1 0 1 2 3 4 Baseline BMI Z-score Source: Own estimations using Guyana IE surveys, 2007-2009 c) Short-term hunger During the course of the day, a child may receive an adequate quantity of nutrients, and thus his/her nutritional status may be satisfactory. If, however, a child arrives at school with no breakfast or an inadequate breakfast, the child will experience short-term hunger until it is offered a snack or a meal. Studies have shown that short-term hunger is associated with a short attention span and with poor school achievement, even if the child’s nutritional status is good. d) Diet diversity Diet diversity is linked to the diet’s nutritional adequacy. Simply put, a diet that is monotonous and deficient in one or more food groups is likely to be deficient in specific vitamins and minerals, and result in sub-optimal linear growth (height). This manifests itself in children as stunting or short stature. Guyanese Amerindian diets are reportedly monotonous, and the prevalence of stunting in Amerindian children is significantly higher than that found in children of other ethnicities. A major thrust of the EFA-FTI hinterland school feeding program is to promote the production and 81 consumption of a wider range of foods than is traditionally the case. Thus the program has stressed in the course of community training exercises the importance of including a range of fruits, vegetables and legumes, and of encouraging the production of these items on local farms. The food frequency questionnaire for the impact evaluation was restricted to a selected number of foods. The choice of foods was based on deficiencies identified anecdotally and from earlier studies. 33 The questionnaire did not ask about the frequency of consumption of staples, mainly cassava in these communities. Studies on Amerindian diets are limited, and it is important to recall that food consumption patterns differ from tribe to tribe and according to the ecology of the area. Graph A3-3 Food Frequency by Comparison Groups and Food Groups 2007 (Control) 2008 (Control) 2009 (Control) 2007 (Treatment) 2008 (Treatment) 2009 (Treatment) 80 Food Freq. Consumption (=100 daily) 70 60 50 40 30 20 10 0 Protein Foods Vegetables Fruits and Juices Diary Products Source: Own estimations using Guyana IE surveys, 2007-2009 33 Vegetables were subdivided into three categories (green leafy, yellow/orange, and other) because different vitamins and minerals are supplied by these categories. For example, orange and yellow vegetables are rich sources of β-carotene, a precursor of Vitamin A, while green leafy vegetables are important sources of iron. 82 ANNEX 4 ESTIMATION PROCEDURES AND METHODS Program Evaluation Framework Program monitoring provides ongoing information on the direction and the magnitude of change in outputs or outcomes of the project. Monitoring is then critical to know whether the project is moving in the right direction. Program monitoring, however, is not a tool that provides information to determine if the observed changes in specific outcome indicators are the direct consequence of the intervention. The main purpose of an impact evaluation study is to provide convincing and reliable evidence that the changes in the outcome indicators are attributed exclusively to intervention and not to other factors. In order to be able to establish causality, the impact evaluation designs a credible “counterfactual” that describes what would have happened had the project never taken place. For example, in the case of the impact evaluation of the school feeding program the counterfactual consists of what participants would have experienced had their communities not participated in the SFP. The central problem in the evaluation of any program is the fact that communities participating in the program cannot be simultaneously observed in the alternative state of no treatment. At a first glance, one has to resort to statistical methods to address this problem (e.g., see Heckman, LaLonde and Smith, 1999). But when the impact evaluation is planned prospectively, the process of selecting the unit of analysis fitted into the design can be understood and randomized methods to select counterfactuals may provide enormous advantages to infer causal linkages. The idea to evaluate a program lies on constructing a suitable counterfactual outcome in the untreated state conditional on receiving treatment. If T=1, the state that denotes participation, then the treatment parameter can be express as: The equation above shows the counterfactual that is impossible to estimate. We can observe, however, the average outcome in the untreated states conditional on similar characteristics: Since in the equation above, 83 The evaluation problem arises when finding an accurate estimate that makes both elements of the equation above to be closer. Formally, “randomization provides a mechanism to derive probabilistic properties of estimates without making further assumptions.”(Rubin, p 693) Randomized trials of participants in the intervention may also be useful for incorporating causal effects. Holland (1985) reminds to us that randomized trials for participation can be a powerful aid in investigating causal relations. Randomly assigning individuals or communities into treatment and control groups, solves the evaluation problem by using information from communities or households in the control group to construct an estimate of what participants would have experienced had they not participated in the program. Therefore, impact evaluation with randomized participation criteria focus attention on impacts across persons with certain similar features. Fixed-Effects Regression Non-experimental data can get much closer to the virtues of a randomized experiment. Specifically, by using the fixed effects methods it is possible to control for all possible characteristics of the individuals in the study—even without measuring them—so long as those characteristics do not change over time. There are two key data requirements for the application of a fixed effects method. First, each individual in the sample must have two or more measurements on the same dependent variable. Second, for at least some of the individuals in the sample, the values of the independent variable(s) of interest must be different on at least two of the measurement occasions. In panel data analysis, the term fixed effects estimator (also known as the within estimator) is used to refer to an estimator for the coefficients in the regression model. If we assume fixed effects, we impose time independent effects for each entity that are possibly correlated with the regressors. Formally, the model is specified as: yit = β0 + Xitβ + Ziγ + αi + uit, where yit is the dependent variable observed for individual i at time t, Xit is the time-variant regressor, Zi is the time-invariant regressor, αi is the unobserved individual effect, and uit is the error term. αi could represent motivation, ability, genetics (micro data) or historical factors and institutional factors (country-level data). The two main methods of dealing with αi are to make the random effects or fixed effects assumption: 1. Random effects (RE): Assume αi is independent of Xit,Zi or E(αi | Xit,Zi) = 0 84 2. Fixed effects (FE): Assume αi is not independent of Xit,Zi. To get rid of individual effect αi, a differencing or within transformation (time arranging) is applied to the data and then β is estimated via Ordinary Least Squares (OLS). The most common differencing methods are: Sample Selection Correction The Heckman correction (the two-stage method) is any of a number of related statistical methods developed by James Heckman in 1976 through 1979 which allow the researcher to correct for selection bias. The Heckman correction, a two-step statistical approach, offers a means of correcting for non-randomly selected samples. Heckman discussed bias from using nonrandom selected samples to estimate behavioral relationships as a specification error. He suggests a two-stage estimation method to correct the bias. The correction is easy to implement and has a firm basis in statistical theory. Heckman’s correction involves a normality assumption, provides a test for sample selection bias and formula for bias corrected model. The Heckman correction takes place in two stages. First, the researcher formulates a model, based on economic theory, for the probability of participating in a program. The canonical specification for this relationship is a probit regression of the form: where D is an indicator variable (D = 1 if the respondent is employed and D = 0 otherwise). Z is a vector of explanatory variables, γ is a vector of unknown parameters, and Φ is the cumulative distribution function of the standard normal distribution. Estimation of the model yields results that can be used to predict this probability for each individual. In the second stage, the researcher corrects for self-selection by incorporating a transformation of these predicted individual probabilities as an additional explanatory variable. which leads to, 85 The model for sample selection that allows treatment and control comparability needs further specifications. The model obtains formal identification from the normality assumption when the same covariates appear in the selection equation and the equation of interest, but identification will be tenuous unless there are many observations in the tails where there is substantial nonlinearity in the Inverse Mills Ratio. Generally, an exclusion restriction is required to generate credible estimates: there must be at least one variable which appears with a non-zero coefficient in the selection equation but does not appear in the equation of interest, essentially an instrument. If no such variable is available, it may be difficult to correct for sampling selectivity. In our models we included many selection variables which are uncorrelated with the error but that explain the selectivity of the sample and the comparison groups. First by including the days taken to implement SFPs in schools we have a variable that controls exogenous conditions that modified entrance into the treatment group. Because many factors such as school organization/performance dealt with the delays in SFP implementation, the variable captures both the organizational skills of schools to submit SFP applications, and other institutional aspects that played a role in postponing the planned date of entrance of schools to the SFP. Another important variable included in the selection equation are the clusters, regions and schools. This can be thought of as fixed effects that play a role in determining the selection of schools into the treatment and control groups. In addition it corrects for all sample limitations derived from attrition rates from the longitudinal sample. When data are collected over two or more points in time, it is common for some participants to drop out of the study prematurely. The attrition of the original sample can occur in longitudinal research as well as in experimental designs that include pretest, posttest, and follow-up data collection. In longitudinal research, which often lasts many years, some participants move between data points and cannot be located. Others, especially older persons, may die or become too incapacitated to continue participation in the study. In clinical treatment studies, there may be barriers to continued participation in the treatment program, such as drug relapse or lack of transportation. Attrition of the original sample represents a potential threat of bias if those who drop out of the study are systematically different from those who remain in the study. The result is that the remaining sample becomes different from the original sample, resulting in what is known as attrition. In our sample the mean attrition rate is of 25 percent for all schools in treatment and control groups. Still attrition rates where very variable by school. Graph A4-1 shows the attrition rates and the number of matched students in survey rounds by each school included in the sample. Although the variability shown in the attrition, only in few cases attrition was high enough to drop the majority of children followed in the longitudinal sample. 86 Graph A4-1 Attrition Rates and Matched Students in Longitudinal Sample by School R1-R3 matched (#) Attrition Rate (%) 60 100.0 50 80.0 40 60.0 30 40.0 20 20.0 10 0 0.0 Batavia primary Father's Beach St. John the Baptist Itaballi Primary St. Anselm's Primary St. Anthony's Primary St. Cyprian's Primary St. Margaret's Primary St. Martin's Annex St.Mary's Primary Kamwatta,(Mabaruma) Almond Beach Chinese Landing Kurupung Lower Kaituma Peter and Paul 72 Miles Primary Aruka Mouth Primary Barabina Primary Holy Name Primary Hotoquai Primary Karrau Creek Primary Kokerite Annex Mabaruma primary Sacred Heart Primary Santa Rosa Primary St. Nicholas Primary Unity Square Primary Yarakita Primary Waramuri Annexe Warapoka Primary White Water Primary Source: Own estimations using Guyana IE surveys, 2007-2009 Counterfactual analysis enables evaluators to attribute cause and effect between interventions and outcomes. The ‘counterfactual’ measures would have happened to beneficiaries in the absence of the intervention, and impact is estimated by comparing counterfactual outcomes to those observed under the intervention. The key challenge in Impact Evaluation is that the counterfactual cannot be directly observed, but must be approximated with reference to a comparison group. There are a range of accepted approaches to determining an appropriate comparison group for counterfactual analysis, using either prospective (ex ante) or retrospective (ex post) evaluation design. Prospective evaluations begin during the design phase of the intervention, involving collection of baseline and end-line data from intervention beneficiaries (the ‘treatment group’) and non-beneficiaries (the ‘comparison group’), and may also involve selection of individuals or communities into treatment and comparison groups. Retrospective evaluations are usually conducted after the implementation phase, and may exploit existing survey data, although the best evaluations will collect data as close to baseline as possible, to ensure comparability of intervention and comparison groups. There are five key principles relating to internal validity (study design) and external validity which rigorous Impact Evaluations should address: confounding factors, selection bias, spillover effects, contamination, and impact heterogeneity. Confounding occurs where certain factors, typically relating to socio-economic status, are correlated with both exposure to the intervention and, independent of exposure, are causally related to the outcome of interest. Confounding factors are therefore alternate explanations for an observed (possibly spurious) relationship between intervention and outcome. Because nutrition status and educational outcomes might be influenced by many factors, the SFP may capture confounding factors. This is why the regressions include fixed-effects and the sample selection correction. 87 Selection bias occurs where intervention participants are non-randomly drawn from the beneficiary population, and the criteria determining selection are correlated with outcomes. Unobserved factors, which are associated with access to or participation in the intervention, and are causally related to the outcome of interest, may lead to a spurious relationship between intervention and outcome if unaccounted for. In the case of the SFP, Self-selection occur where, for example, more able or organized individuals or communities, who are more likely to have better outcomes of interest, are also more likely to participate in the intervention. Endogenous program selection occurs where individuals or communities are chosen to participate because they are seen to be more likely to benefit from the intervention. Ignoring confounding factors can lead to a problem of omitted variable bias. In the special case of selection bias, the endogeneity of the selection variables can cause simultaneity bias. All these types of bias can be addressed substantially with sample selection correction models. Impact evaluation designs are identified by the type of methods used to generate the counterfactual and can be broadly classified into three categories – experimental, quasi- experimental and non-experimental designs – that vary in feasibility, cost, involvement during design or after implementation phase of the intervention, and degree of selection bias. White (2006) and Ravallion (2008) discuss alternate Impact Evaluation approaches. Under experimental evaluations the treatment and comparison groups are selected randomly and isolated both from the intervention, as well as any interventions which may affect the outcome of interest. These evaluation designs are referred to as randomized control trials (RCTs). In experimental evaluations the comparison group is called a control group. When randomization is implemented over a sufficiently large sample with no contagion by the intervention, the only difference between treatment and control groups on average is that the latter does not receive the intervention. Random sample surveys, in which the sample for the evaluation is chosen on a random basis, should not be confused with experimental evaluation designs, which require the random assignment of the treatment. The experimental approach is often held up as the ‘gold standard’ of evaluation, and it is the only evaluation design which can conclusively account for selection bias in demonstrating a causal relationship between intervention and outcomes. Randomization and isolation from interventions are seldom practicable in the realm of social policy, and may also be ethically difficult to defend, although there may be opportunities to utilize natural experiments. Bamberger and White (2007) highlight some of the limitations to applying RCTs to development interventions. Methodological critiques have been made by Scriven (2008) on account of the biases introduced since social interventions cannot be triple blinded, and Deaton (2009) has pointed out that in practice analysis RCTs falls back on the regression-based approaches they seek to avoid, and so are subject to the same potential biases. Other problems include the often heterogeneous and changing contexts of interventions, logistical and practical challenges, difficulties with monitoring service delivery, access to the intervention by the comparison group and changes in selection criteria and/or intervention over time. Thus, it is estimated that RCTs are only applicable to 5 per cent of development finance. 88 Quasi-experimental approaches can remove bias arising from selection on observables and, where panel data are available, time invariant unobservables. Quasi-experimental methods include matching, differencing, instrumental variables and the pipeline approach, and are usually carried out by multivariate regression analysis. If selection characteristics are known and observed then they can be controlled for to remove the bias. Matching involves comparing program participants with non-participants based on observed selection characteristics. Propensity score matching (PSM) uses a statistical model to calculate the probability of participating on the basis of a set of observable characteristics, and matches participants and non-participants with similar probability scores. Regression discontinuity design exploits a decision rule as to who does and does not get the intervention to compare outcomes for those just either side of this cut-off. Difference-in-differences or double differences, which use data collected at baseline and end-line for intervention and comparison groups, can be used to account for selection bias with under the assumption that unobservable factors determining selection are fixed over time (time invariant). Instrumental variables estimation accounts for selection bias by modelling participation using factors (‘instruments’) that are correlated with selection but not the outcome, thus isolating the aspects of program participation which can be treated as exogenous. The pipeline approach (stepped-wedge design) uses beneficiaries already chosen to participate in a project at a later stage as the comparison group. The assumption is that as they have been selected to receive the intervention in the future they are similar to the treatment group, and therefore comparable in terms of outcome variables of interest. However, in practice, it cannot be guaranteed that treatment and comparison groups are comparable and some method of matching will need to be applied to verify comparability. Propensity Score Matching In the evaluation literature, data often do not come from randomized trials but from (non- randomized) observational studies. In seminal work, Rosenbaum and Rubin (1983) proposed propensity score matching as a method to reduce the bias in the estimation of treatment effects with observational data sets. These methods have become increasingly popular in medical trials and in the evaluation of economic policy interventions. Since in observational studies assignment of subjects to the treatment and control groups is not random, the estimation of the effect of treatment may be biased by the existence of confounding factors. Propensity score matching is a way to partially “correct” the estimation of treatment effects controlling for the existence of these confounding factors based on the idea that the bias is reduced when the comparison of outcomes is performed using treated and control subjects who are as similar as possible. Since matching subjects on an n-dimensional vector of characteristics is typically unfeasible for large n, there are new methods that propose to summarize pre- treatment characteristics of each subject into a single-index variable (the propensity score) which makes the matching feasible. The propensity score is defined by Rosenbaum and Rubin (1983) as the conditional 89 Probability of receiving a treatment given pre-treatment characteristics: where D = {0, 1} is the indicator of exposure to treatment and X is the multi dimensional vector of pre-treatment characteristics. Rosenbaum and Rubin (1983) show that if the exposure to treatment is random within cells defined by X, it is also random within cells defined by the values of the mono-dimensional variable p(X). As a result, given a population of units denoted by i, if the propensity score p(Xi) is known the Average effect of Treatment on the Treated(ATT)can be estimated as follows: For practical ends, the methodology for propensity score matching used first an equation that included individual and household characteristics and student test scores (grades) to explain participation in the treatment dichotomous variable. Given the relatively small number of observations included in the individual-level sample the command pscore in Stata v 10 was used to include common support blocks to minimize the dropout of observations. After the propensity scores are estimated, they are used to balance the sample and estimate the Average Treatment on the Treated effects (ATT) for each type of test (Reading, English, Math, etc.). The method to match the scores is based on the kernel distribution. This facilitates a robust distributional configuration of the scores along the subsample derived from the propensity scores. The method estimates the standard errors via bootstrapping so that estimators are efficient as well. The ATT effects should be interpreted carefully. They do not represent results beyond the sample in question, given the non-random assignment. But they do represent impacts at the survey sample level, which is relevant at least from the program perspective. The procedure to match treatment and control subsamples encompasses 3 steps. First, a simple probit model is estimated with a set of observable characteristics, where the outcome variable is the dummy of program participation. The most relevant variables for the probit model step are shown in the following table: 90 Table A4-1 Propensity Score Matching: Selection into Treatment (observables) Dependent Variable (=1 SFP) Coefficient S.E. Gender (=1 male) 0.037 0.069 Age -0.382 0.065 Grade (Primary) 0.514 0.098 Both parents in HH -0.067 0.042 Child's Score (NAS) -0.228 0.087 Attendance Rate (School) 0.007 0.002 Z-score Body Mass Index -0.266 0.053 Constant 2.145 0.554 Observations 1516 LR Chi2(8) 111.6 Log of Likelihood -898.177 Source: Author’s estimation using pscore command Stata v.10. with SFP Survey data The second step is to estimate the propensity score distribution with the blocks that identify “common support” characteristics between the groups: Table A4-2 Estimated propensity score in region of common support Percentiles Smallest 1% 0.045 0.0291 5% 0.105 0.0291 10% 0.158 0.0291 Obs 1508 25% 0.246 0.0291 Sum of Wgt. 1508 50% 0.4237 Mean 0.423406 Largest Std. Dev. 0.120458 75% 0.5070 0.7747 90% 0.5777 0.7747 Variance 0.01451 95% 0.6157 0.7747 Skewness -0.05825 99% 0.7583 0.7747 Kurtosis 2.763906 Source: Author’s estimation using pscore command Stata v.10. with SFP Survey data 91 The final step is to estimate the inferior block that needs to be rebalanced. The results shown in the following table contains the inferior blocks with relatively low observations compared to the total PSM estimates. This is why the pscore graphs have satisfied balancing properties 34. Table A4-3 Inferior Block of Pscore Score Control Treatment Total 0.029 98 8 106 0.125 78 6 84 0.188 162 48 210 0.450 630 378 1,008 0.600 50 50 100 Total 1,018 490 1,508 Source: Author’s estimation using pscore command Stata v.10. with SFP Survey data Graph A4-2 Distributions of Normalized Scores vs. Health Indicators .4 .3 Density .2 .1 0 -4 -2 0 2 4 z-score scale Normalized General Standardized Score Matched Height for Age z-score treatment Matched Height for Age z-score control Matched BMI z-score treatment Matched BMI z-score control 34 Although the scores could match at higher propensities, the results are enough to correct for observed biases in SFP participation, particularly those resulting from the student education and health outcomes. 92 .5 .4 .4 .3 Density .3 Density .2 .2 .1 .1 0 0 -4 -2 0 2 4 -4 -2 0 2 4 Height-for-age z-score Body Mass Index z-score Source: Author’s estimation using pscore command Stata v.10. with SFP Survey data Why SFP participation has Selectivity Issues? In non-experimental studies like this one, researchers often try to approximate a randomized experiment by statistically controlling for other variables using methods such as linear regression, logistic regression, or propensity scores. While statistical control can certainly be a useful tactic, it has two major limitations. First, no matter how many variables are controlled for, there is always room for arguing that selectivity still remains. Given this evaluation design and they way in which the SFP participation was structured, fully randomization for the allocation of the program to participant schools was not achieved. The program comprised 3 phases of training prior to the provision of grants to establish community- run kitchens in schools. The first phase included a promotion campaign and awareness sessions with parents, teachers, village councils, and members of communities. Overall this phase have high levels of participation from schools and dwellers. During phase 2, training was provided to elaborate a proposal to request financing grants in order to create conditions to food preparation, agricultural production, financial management and administration of the school feeding program. The sessions were carried with school representatives and community members interested in implementing the SFP in their communities. This phase was very important because it involved producing a quality proposal to receive the award. In parallel, phase 3 included a certification of cooks, and readiness of facilities to launch the program. Without certification SFP could not start. Based on this design and steps to select schools, there are sample selection issues that may bias the impact estimators of the SFP participation to the above mentioned outcomes. In particular, phase 2 contributes to enhance self selectivity. Schools and communities that are better organized and show higher commitment to implement an SFP will tend to have higher probabilities of success in elaborating high quality proposals and, as a consequence, receive the grant to launch the SFP. Intuitively, the characteristics that may affect this sample selection have to do with attributes at the school, region and sub-region levels. In addition, community 93 organization will be fundamental to submit on time the proposal and to timely implement the SFP. Fortunately the richness of the information contained in the surveys provides important observable characteristics to reduce the bias generated by the aforementioned selection process reflected in the outcome variables. Other unobservable characteristics may be corrected through statistical techniques. Correcting for both observable and unobservable characteristics of sample selection between comparison groups will minimize the bias, therefore improving statistical inference of SFP’s impacts. Non randomness in the assignment of treatment and control groups can be partially corrected if we have observed factors that could determine the selection bias that is manifested in the outcome variables. Graph A4-2 shows the relationship between the delay in days of SFP implementation and attendance rates. It is worth noticing that there is a clear relationship between the outcome variable (attendance) and the delay in days. The delays in SFP implementation was caused by multiple factors that range from school organizational level to budgetary and grant allocation delays. However, this variable captures intrinsic characteristics of schools and other contextual characteristics that produced higher quality schools to enter in the treatment group in the initial phase of the SFP. Because of this reason, the delay in days to implement SFP was used as a control within the selectivity equation of Heckman’s two stage procedure. Graph A4-3 Relationship between Delay in SFP Implementation and Attendance Rates 90 Attendance Rate 80 70 60 No Delay First Quintile Second Third Quintile Forth Quintile Fifth Quintile Quintile Quintiles of Delay in Days of SFP Implementation Source: Own estimations based on Guyana’s SFP IE Surveys In order to verify how strong is the variable “days of delay in SFP implementation” in terms of capturing school quality attributes, graph A4-3 shows the relationship between the days of delay in SFP implementation and the standardized z-scores. Schools with no delays in implementation show higher average z-scores than those schools that presented at least one day of delay in SFP implementation. More days of delay in SFP implementation is associated with consistently lower total standardized z-scores. 94 Graph A4-4 Relationship between Delay in SFP Implementation and Total Standard Z-scores (Academic) 0.8 Standarized Z-scores 0.6 0.4 0.2 0 -0.2 No Delay First Second Third Forth Fifth Quintile Quintile Quintile Quintile Quintile Quintiles of Delay in Days in SFP Implementation Source: Own estimations based on Guyana’s SFP IE Surveys Sample selection correction included this variable to increase accuracy in the observable attributes that may contribute to increase sample selection bias. In addition, other regional, school and cluster unobserved attributes were included in the selection equation to improve the accuracy in the estimations. Although the results reported are consistent, they should be interpreted with caution because sample selectivity will always be present in non-random assignments. However, the models allow approximate a fairly accurate estimate of SFP’s impacts. Overall Sample Changes and Children At-risk (stunting and wasting) subsample results The school datasets (Round 1, 2 and 3) contain rich information of programs and activities undertaken in each school. The longitudinal database was constructed based upon face-to-face interviews to head teachers. Baseline and the first follow up surveys show a perfect matching of 64 schools. However, when the third round is merged, 6 schools do not match with baseline and round 2 data. This represent 90 percent of information kept at the school level for all three rounds (58 schools). Student level data was also merged for baseline, round 2 and round 3. The merge of baseline and round 2 showed 3,590 students 35 that were perfectly matched. In other words, the sample of children that was followed in baseline and the subsequent round was of 3,590 students. Out of these children, 616 belong to the treatment group (considering the dates of entrance to the program) and 2,974 were in control areas. Using the third round data, the sample increased to 6,581 students, but 3,590 were perfectly matched. This means that at the student level, there was not a single child that did not matched between the baseline-round2 merged data and the round 3 35 The total sample of children is 5,405 students but the sample was reduced due to missing data on children and unsuccessful follow up of children. 95 data. Out of the 3,590 children with data for the three rounds 36, 1,982 belong to the control group and 1,608 to the treatment group. This means that from the baseline-round2 merged data there was an increase of 992 children (which compensated the decrease from 2,974 to 1,982 children in the control group). The treatment and control groups show the following distribution of children by region: Table A4-4 Number of Children Total Sample with Data* (Rounds 1, 2 and 3) Total Region Treatment Control Sample 1 1,006 1,728 2,734 7 602 254 856 Total 1,608 1,982 3,590 *With observed data on NAS and Health Indicators Longitudinal Sample of Children (Baseline and First Follow up) Baseline Region and First Treatment Control Round 1 1,367 323 1,044 7 428 293 135 Total 1,795 616 1,179 Longitudinal Sample of Children (Baseline and Two-Follow ups) Panel 3 Region Treatment Control Periods 1 1,494 503 991 7 428 301 127 Total 1,922 804 1,118 With the longitudinal sample the analysis can be done at the student level controlling for student, school and region characteristics. Additional controls for school/community quality proxies (absenteeism, days taken to implement SFP and test scores). Long-term health outcomes cannot be assessed at this point, although marginal effects can be computed for changes in BMI and HAZ. Two additional models that included dummies to identify the children at-risk of stunting and wasting were run to verify if the program had any these impacts across these vulnerable subgroups (see Table below). The first model included a simple regression (OLS) with matched 36 Absenteeism data was also merged from a separate database and all 3,590 students merged successfully. 96 sample using PSM and the second an Instrumental Variable model where the instrument used was the delay in days in implementing the program (see Graphs A4-2 and A4-3), as it is exogenous from participation but it can change the stunting and wasting distribution by adding most disadvantaged children at later points in time. Table A4-5 AT-RISK CHILDREN: STUNTING (HAZ) (1) (2) (3) (4) Height-for-age z-score (% Change) SFP (=1 Treatment) 0.003 0.010 0.128 0.031 s.e. 0.016 0.088 0.029 Dummy At-Risk (=1 WHO Cutoff) 0.006 0.031 0.038 s.e. 0.018 0.058 0.033 Interaction term=Atrisk*SFP -0.017 -0.036 -0.031 s.e. 0.030 0.058 0.038 Constant -0.004 -0.007 -0.284 0.043 s.e. 0.009 0.014 0.429 0.157 Observations 2642 2642 1134 1432 R-squared 0.01 0.01 0.03 0.03 CHILDREN AT-RISK: WASTING (BMI) (1) (2) (3) (4) BMI z-score (% Change) SFP (=1 Treatment) 0.094 0.051 0.060 0.208 s.e. 0.020 0.018 0.115 0.029 Dummy At-Risk (=1 WHO Cutoff) -0.574 -0.708 -0.596 s.e. 0.050 0.236 0.078 Interaction term=Atrisk*SFP 0.314 0.411 0.290 s.e. 0.066 0.246 0.102 Constant -0.077 0.017 0.523 0.206 s.e. 0.012 0.010 0.373 0.157 Observations 3518 2642 1132 1436 R-squared 0.01 0.01 0.19 0.15 Note: Controls for student, school, community characteristics were included in models (3) and (4) 97 (1) OLS, robust standard errors (2) OLS, robust standard errors (3) OLS, clustered errors school level. Matched sample PSM. (4) IV regression using delay in days in implementing the program as instrument On merging the “parents” data there are several points worth mentioning. For the baseline survey, parents and food frequency data were collected separately. Merging both datasets caused a reduction in 27 observations. This represents around 95 percent (567 observations) of data kept in the baseline where food frequency and parent data is available. For the second round data (first follow-up) the parent and food frequency data was built in the same dataset. By merging both, the baseline parents-food frequency and the second round parents-food frequency data there were left 546 observations (58 observations were lost). After merging the final Round (3) to the merged panel of parents for baseline (R1) and follow-up (R2), the number of observations preserved were 506. In the R1 and R2 data there were 376 parents interviewed in control and 130 in treatment areas. Subsequently, R3 added 47 new parents from the control to the treatment group ending up with 329 and 177, respectively for control and treatment groups. The regional distribution of parents in treatment and control areas for the longitudinal sample is as follows: Table A4-6 Number of Parents in Complete Panel Sample (Rounds 1, 2 and 3) Total Region Treatment Control Sample 1 77 292 369 7 100 37 137 Total 177 329 506 School Enrollment (2007 and 2009) Region 2007 2009 Difference Region 7 2,409 2,374 -35 Cluster 1 2,149 2,101 -49 Cluster 2 260 273 13 Region 1 7,323 7,696 373 Cluster 4 1,439 1,571 132 Cluster 5 3,099 3,171 72 Cluster 6 2,785 2,954 169 Total 9,732 10,070 338 Status 2007 2009 Difference Treatment 4,442 5,173 731 Region 1 3,640 3,841 201 98 Region 7 1,302 1,521 219 Control 5,290 4,897 -393 Region 1 3,480 3,648 168 Region 7 923 853 -70 Total 9,732 10,070 338 Table A4-7 SFP impacts on enrollment 2007-2009 Sample Fixed Effects Dependent: School Enrollment Selection Model \1 Model nd School Feeding Program*2 Follow Up 201.4 *** 178.3 *** Standard Error 10.9 13.0 Controls GRADE \2 -31.7 *** -31.7 Standard Error 6.64 27.17 Age 1.03*** 1.46 *** Standard Error 0.39 0.42 Proportion Communities with Electricity 60.5 *** 57.4 *** Standard Error 8.0 8.4 Proportion Households with Radio -1.7 12.11 * Standard Error 8.29 6.23 Dummy Households without Both parents -5.8 -2.68 Standard Error 5.5 5.2 Average Numbers of Siblings -14.9 *** -12.9*** Standard Error 2.4 2.5 Time kid takes to go to School (minutes) 1.67 9.7 ** Standard Error 4.0 3.9 Time Controls Yes Yes Selection Variables on SFP: Reported Absenteeism (Baseline) -0.03 Standard Error 0.020 Days to implement SFP 0.002*** Standard Error 0.000 Cluster (6) -.0.144 Standard Error 0.11 Region -0.218*** Standard Error 0.065 School -0.028 *** Standard Error 0.005 Rho 0.04 Lambda (Mills) 8.10** 99 R2/Wald Chi 0.228 780.6 Number of Observations 1,534 1,534 \1 Robust corrected standard errors reported \2 Grades 2, 3, 4 included * Significance at 10 percent level. ** Significance at 5 percent level. *** Significance at 1 percent level. Source: Own estimations based on Guyana’s SFP IE Surveys Table A4-8 Height for Age Z-scores 2007 and 2009 Category 2007 2009 Difference Total -1.075 -0.972 0.103 *** s.e (1.086) (1.141) Girls -0.983 -0.890 0.092 *** s.e. (1.091) (1.201) Region 1 -1.193 -1.113 0.080 ** s.e (0.019) (0.024) Region 7 -0.557 -0.531 0.026 * s.e. (0.038) (0.047) Treatment -0.851 -0.721 0.126 *** s.e. (0.035) (0.037) Control -1.236 -1.097 0.138 *** s.e. (0.031) (0.027) * Significance at 10 percent level. ** Significance at 5 percent level. *** Significance at 1 percent level. ttest mean differences Source: Own estimations based on Guyana’s SFP IE Surveys Graph A4-5 Distribution of HAZ Scores by Region 100 Table A4-9 SFP Impacts on Stunting 2007-2009 Sample Fixed Effects Dependent: Height-for-Age (Stunting) Selection Variable Normalized in Z-Score Model \1 Model nd School Feeding Program*2 Follow Up 0.135 ** 0.089 * Standard Error 0.067 0.05 Controls GRADE \2 0.271 *** 0.324 *** Standard Error 0.076 0.037 Age -0.026*** -0.026 *** Standard Error 0.003 0.002 Proportion Communities with Electricity 0.334 *** 0.315 *** Standard Error 0.059 0.044 Proportion Households with Radio 0.061 0.097 ** Standard Error 0.067 0.041 Dummy Households without Both parents -0.037 -0.011 Standard Error 0.040 0.025 Average Numbers of Siblings -0.079 *** -0.064 *** Standard Error 0.014 0.012 Time kid takes to go to school (minutes) -0.109 *** -0.091 *** Standard Error 0.030 0.020 Time Controls Yes Yes Selection Variables on SFP: Reported Absenteeism (Baseline) -0.0813 *** Standard Error 0.011 Days to implement SFP 0.001 ** Standard Error 0.000 101 Cluster -0.184 ** Standard Error 0.072 Region -0.072 ** Standard Error 0.034 School 0.007 ** Standard Error 0.003 Rho -0.71 Lambda (Mills) -0.857 *** R2/Wald Chi 0.15 481.2 Number of Observations 1,498 1,513 \1 Robust corrected standard errors reported \2 Grades 2, 3, 4 included * Significance at 10 percent level. ** Significance at 5 percent level. *** Significance at 1 percent level. Source: Own estimations based on Guyana’s SFP IE Surveys Table A4-10 SFP Impacts on School Attendance 2007-2009 Sample Fixed Effects Dependent: School Attendance Selection Model \1 Model nd School Feeding Program*2 Follow Up 2.75** 4.33 ** Standard Error 1.13 1.38 Controls GRADE \2 2.38 2.38 *** Standard Error 1.47 0.95 Age -0.411*** -0.404 *** Standard Error 0.068 0.056 Proportion Communities with Electricity 2.19 * 2.66 *** Standard Error 1.18 1.12 Proportion Households with Radio 0.89 1.06 Standard Error 0.97 1.07 Dummy Households without Both parents -0.85 -0.77 Standard Error 0.66 0.64 Average Numbers of Siblings -0.21 -0.32 Standard Error 0.322 0.312 Time kid takes to go to school (minutes) -1.18 *** -1.62 *** Standard Error 0.494 0.050 Time Controls Yes Yes Selection Variables on SFP: Reported Attendance (Baseline) -0.714 *** Standard Error 0.117 Days to implement SFP 0.000 Standard Error 0.000 102 Cluster -0.049 Standard Error 0.074 Region -0.039 Standard Error 0.036 School 0.001 Standard Error 0.003 Rho 1.00 Lambda (Mills) -35.48 *** R2/Wald Chi 0.167 112.5 Number of Observations 1,358 1,527 \1 Robust corrected standard errors reported \2 Grades 2, 3, 4 included * Significance at 10 percent level. ** Significance at 5 percent level. *** Significance at 1 percent level. Source: Own estimations based on Guyana’s SFP IE Surveys Table A4-11 SFP Impacts Using Parents Data 2008: Food Frequency Sample FE Model Dependent: Food Frequency Selection Model \1 Model School Feeding Program (Baseline/2008) 2.35*** 2.25 *** Standard Error 0.44 1.38 Controls Head sex (1=male) -0.47 -0.43 Standard Error 0.45 1.20 Employment (1=employed) -0.22 -0.24 Standard Error 0.32 0.92 Number of Children in HH 0.009 2.66 *** Standard Error 0.069 1.12 Education of Head 0.278 * 0.272 ** Standard Error 0.22 0.14 Water access 1.12 *** 1.11 ** Standard Error 0.35 0.64 Electricity 1.15 *** 1.12 Standard Error 0.43 0.312 Roads 0.45 0.409 Standard Error 0.69 1.80 Selection Variables on SFP: Cluster (6) -0.032 Standard Error 0.117 School effects -16.62 * Standard Error 10.13 103 Rho 1.00 Lambda (Mills) 9.53 * R2/Wald Chi 0.168 81.28 Number of Observations 462 461 \1 Robust corrected standard errors reported \2 Grades 2, 3, 4 included * Significance at 10 percent level. ** Significance at 5 percent level. *** Significance at 1 percent level. Source: Own estimations based on Guyana’s SFP IE Surveys Table A4-12 SFP Impacts Using Parents Data 2009: Food Frequency Sample FE Model Dependent: Food Frequency Selection Model \1 Model School Feeding Program (2008/2009) 1.359 ** 1.275 ** Standard Error 0.597 0.641 Controls Head sex (1=male) 0.257 0.424 Standard Error 1.018 3.06 Employment (1=employed) -0.186 -0.266 Standard Error 0.512 1.92 Number of Children in HH 0.005 -0.029 Standard Error 0.101 0.341 Education of Head 1.05 * 1.08 ** Standard Error 0.317 0.541 Water access 1.31** 1.35 *** Standard Error 0.65 0.64 Electricity 1.24 *** 1.30 *** Standard Error 0.54 0.64 Roads 0.73 0.32 Standard Error 1.36 4.24 Selection Variables on SFP: Cluster (6) -0.31 * Standard Error 0.189 School effects -5.02 * Standard Error 2.52 Rho 1.00 Lambda (Mills) -13.80 * R2/Wald Chi 0.24 25.36 104 Number of Observations 240 238 \1 Robust corrected standard errors reported \2 Grades 2, 3, 4 included * Significance at 10 percent level. ** Significance at 5 percent level. *** Significance at 1 percent level. Source: Own estimations based on Guyana’s SFP IE Surveys Table A4-13 Math Scores Regressions with Fixed Effects and Sample Selection Fixed Sample Selection Effects Model Model SFP 3.36 * 4.10 * (1.27) (1.33) Grade 34.04 *** 33.26 *** (1.20) (0.84) Age 0.282 *** 0.278 *** (0.10) (0.06) Electricity 0.855 0.843 (1.34) (1.04) Radio in HH -2.96 ** -2.09 * (1.23) (0.97) Living without 2 parents 0.16 0.025 (0.86) (0.59) Number of Siblings -0.735 -0.454 (0.52) (0.30) Fixed Effects Time Yes Region Yes Selection Variables Time to assign program 0.003 Cluster 0.134 Region 0.075 School -0.007 Constant -63.45 (9.06) R-squared 0.748 Lambda Mills 0.993 F(g, k-1) 361.5 Number of Observations 1532 3254 Standard errors in Parenthesis * Significance at 10 percent. ** Significance at 5 percent. *** Significance at 1 percent. Source: Own estimations based on Guyana’s SFP IE Surveys 105 Table A4-14 Reading Scores Regressions with Fixed Effects and Sample Selection Fixed Sample Selection Effects Model Model SFP 0.865 1.62 (1.18) (1.65) Grade -4.31* -3.04 (1.81) (1.87) Age -0.138** -0.136** (0.056) (0.058) Electricity 0.157 0.807 ( 1.09) (1.17) Radio in HH -0.772 -0.162 (0.85) (1.04) Living without 2 parents 0.823 0.205 ( 0.862) (0.675) Number of Siblings -0.173 -0.185 (0.335) (.335) Fixed Effects Time Yes Region Yes Selection Variables Time to assign program -0.01 Cluster 0.059 Region 0.029 School -0.004 Constant 41.18 -206.43 ( 6.71) (608.33) R-squared 0.625 Lambda Mills 30.26 F(g, k-1) 127.17 Number of Observations 854 3424 Standard errors in Parenthesis * Significance at 10 percent. ** Significance at 5 percent. *** Significance at 1 percent. Source: Own estimations based on Guyana’s SFP IE Surveys 106 Table A4-15 English Scores Regressions with Fixed Effects and Sample Selection Fixed Sample Selection Effects Model Model SFP 1.13 1.95 ( 1.285) (1.17) Grade 34.86 *** 34.45 *** (1.40) (0.91) Age 0.172* 0.149** (0 .107) (0.06) Electricity -0.873 -0.716 ( 1.62) (1.12) Radio in HH -0.815 0.334 (1.479) ( 1.06) Living without 2 parents -0.053 -0.238 (0.969) (0.639) Number of Siblings -1.05 -0.801 (0.549) (0.324) Fixed Effects Time Yes Region Yes Selection Variables Time to assign program 0.003 Cluster 0.134 Region 0.075 School -0.007 Constant -53.82 (10.00) R-squared 0.7191 Lambda Mills 0.253 F(g, k-1) 232.11 Number of Observations 1532 3254 Standard errors in Parenthesis * Significance at 10 percent. ** Significance at 5 percent. ***Significance at 1 percent. Source: Own estimations based on Guyana’s SFP IE Surveys 107 Table A4-16 Science Scores Regressions with Fixed Effects and Sample Selection Fixed Sample Selection Effects Model Model SFP 0.037 1.96 ( 1.81) (1.95) Grade 6.62 *** 6.64 *** ( 2.47) (1.39) Age -0.535 -0.623 (0.244) (0.109) Electricity -1.69 -0.429 (2.33) (1.33) Radio in HH 2.33 2.78 (2.33) (1.34) Living without 2 parents -1.48 -1.08 (1.11) (0.73) Number of Siblings -2.14*** -1.35 *** (0.887) (0.39) Fixed Effects Time Yes Region Yes Selection Variables Time to assign program 0.03 Cluster 0.087 Region 0.04 School -0.004 Constant 141.06 1185.45 ( 26.13) (919.23) R-squared 0.1037 Lambda Mills -8.89 F(g, k-1) 2.52 Number of Observations 678 3422 Standard errors in Parenthesis * Significance at 10 percent. ** Significance at 5 percent. *** Significance at 1 percent. Source: Own estimations based on Guyana’s SFP IE Surveys 108 Table A4-17 Social Studies Scores Regressions with Fixed Effects and Sample Selection Fixed Sample Selection Effects Model Model SFP 0.221 3.79 * (1.96) (2.05) Grade 2.56 4.64 ** (2.28) (1.37) Age -0.412* -0.459* (0.223) (0.108) Electricity 1.34 1.28 (1.80) (1.32) Radio in HH 1.74 1.68 (2.24) (1.32) Living without 2 parents -1.36 -1.06 (1.28) (0.73) Number of Siblings -1.36 -1.89 (0.85) (0.385) Fixed Effects Time Yes Region Yes Selection Variables Time to assign program 0.003 Cluster 0.087 Region 0.044 School -0.004 Constant 140.9 3387.9 (26.18) (1907.34) R-squared 0.103 Lambda Mills F(g, k-1) 2.75 -21.47 Number of Observations 678 3422 Standard errors in Parenthesis * Significance at 10 percent. ** Significance at 5 percent. *** Significance at 1 percent. Source: Own estimations based on Guyana’s SFP IE Surveys 109 Table A4-18 Food Frequency Regressions from Graph 22 Diff 2007-2008/2007-2009 110 Simulation: Price Shocks and SFP’s Role as Safety Net Given a change in producer and consumer staple prices, the net effect on household welfare depends on the household’s condition as net seller or net buyer. If staple prices increase, the household will experience a welfare gain in the short run if it is a net seller or a welfare loss if it is a net buyer. To quantify this change in welfare in an intuitive manner a useful notion is that of compensating variation, which equals the gain/loss to the income/monetary transfer needed to restore the household to the position it was before the (price) shock occurred. In this paper the compensating variation is expressed as a percentage of the initial welfare level. The methodology used in this paper has several antecedents, starting with Deaton (1989), and many other empirical applications thereafter, including Budd (1993), Barrett and Dorosh (1996), Minot and Goletti (2000) and, recently, Ivanic and Martin (2008). A change in the staple food price is the original income (here proxied by total consumption expenditure) of each household with the original price of the staple. Conceptual Framework for SFP role as a Safety Net Mechanism Microeconomic baseline Price Levels Time series by conditions (poverty, Product and Geographical income, HH Region characteristics) Community Based School Feeding Program Changes in relative supply and demand of Food Products Changes in Households’ Disposable Income Price level reallocation Reallocation of Poverty Incidence and household consumption Source: Adapted from Colombo (2010). Linking CGE and Microsimulation Models: A Comparison of Different Approaches. International Journal of Microsimulation (2010) 3(1) 72-91 111 The three rounds of SFP’s impact evaluation surveys are based mostly on the same sample of households, and can be combined with the household budget and expenditures survey (HBE), with additional socioeconomic characteristics of individuals and households. Additionally, the prices are calibrated with GMC data 37. When the two data sets are combined and observations with missing sampling weights are dropped. Weights from national HBE survey are used for the merged dataset with slight calibration based on the sample in each cluster of the evaluation surveys. One must ensure that outcomes from the micro-simulation model are consistent with the aggregate results from the poverty estimates both before and after the price shock for each comparison area. This includes aggregating the simulation results at the community level using average socioeconomic characteristics, prices, purchases and food groups by SFP participant and non-participant communities. The model requires information on household consumption expenditure (C), the agricultural intensity of communities and a dummy variable that indicates whether the household receives SFP (Ict), and time-series price levels by food groups. The household survey provides information on the poverty line. The model simulates the impact of higher commodity prices on households by simulating attendant changes in consumption and the poverty line. We assume that simulated and baseline consumption by individual j in food group i has the following formula: This implies that the simulated percentage change in consumption (abstracting from changes in SFP status) is identical for every household in a community. Further, the change in SFP status over time is assumed to be unconditional. An increase in commodity prices will increase the cost of the poverty basket. Baseline and simulated increases in the real poverty line (PL) are calculated as follows: The simplicity of this equation leads to extensive use in the analysis of the poverty impact of increases in food prices. However, it is worth reminding that bias exists from its use. Individual food prices rarely increase by the same percentage or proportion. An individual would typically substitute away from a good whose relative price has increased. Thus this assumption results in a substitution bias that over-estimates the poverty impact of the increase in food prices. Because GMC prices measurement error tends to under-estimate prices, there could be an under- estimation of impact, which in combination with the substitution bias, both biases may cancel- out (based on their magnitudes). Given the short-run orientation of this simulation model, it is possible that this bias may not be substantial. Let Ipj be a dummy variable indicating whether household j is poor: 37 Prices from the Guyana Market Corporation may have substantial measurement error as communities become more isolated and rural. 112 The national poverty incidence for the baseline and simulation outcomes (Hb and Hs respectively) are given by: To build the model and estimate how many children in SFP areas did not fall below the poverty threshold with an increase in food prices, different data was used and several assumptions were made. First, monthly prices of food and food groups from 1990 to 2010 were used to explore the price index changes and trends. Based on frequencies of food group consumption and local food prices in sub-regions 38 the estimated price changes for the main food staples/groups were listed in frequencies. The estimations were then separated according to the price indices reported in markets adjacent or close to treatment and control areas 39. With food prices changes estimated for treatment and control areas it is possible to calculate the differences shown in the period before and after the food price shocks by each food group. The price shocks reduce spending power and increases the likelihood of a household to fall into the poverty line. Guyana´s extreme and moderate poverty lines were estimated through a household consumption and expenditure survey collected nationally, reported in the Guyana Poverty Assessment. The thresholds are reported in Table A4-19. Table A4-19 Poverty Line Thresholds in Guyana Figures in Guyanese Dollars (2005) Year Poverty Line 1992 1999 2006 2009* Extreme poverty 2,929 5,463 7,959 8,312 Moderate poverty 3,960 7,639 11,143 13,274 Source: Guyana Poverty Assessment (1994), and BOS 2009. *Estimated Price differences between treatment and control areas changes relative households’ food expenditures and income levels, so it is possible to estimate changes in poverty levels as well. Since the differences in prices are expressed in percentage between the price levels paid in 38 The Guyana Market Corporation (GMC) provided useful quarterly data from 2007 to 2009 on all prices paid in local markets for 50 products that included fruits, meats and proteins, legumes, ground and roots, vegetables, seasonings, starch and nuts. 39 Local markets can provide food products to both treatment and control areas particularly when they are close enough. Prices were imputed to the comparison group that had schools close to the local market. 113 treatment and control areas, it is feasible that, with the incentive to provide additional meals with the SFP, local market production and food frequency-intake stability could lead to lower price levels in treatment areas. Graph A4-4 shows that the simulations produced an increase in the gap of comparison group food prices due to the assumptions made and the presence of the SFP. Graph A4-20 Price Levels in Treatment and Control areas Protein control 170.0 Protein treatment Price Index (2002-2004=100) Price Shock 130.0 90.0 50.0 1/2000 1/2002 1/2004 1/2006 1/2008 1/2010 300 Vegetables Control Vegetables Treatment Price Index (2002-2004=100) 250 Price Shock 200 150 100 50 1/2000 1/2002 1/2004 1/2006 1/2008 1/2010 Source: Own estimations using Guyana Central Bank data and Guyana Market Corporation Prices. The estimations of price differences between treatment and control areas for the period 2000 to 2010 is shown in graph A4-20 (for proteins and vegetables). All food groups showed price stability and small differences in the period previous to the food price crisis in both treatment and control areas. During and after the period of food price increases (mid-2007) the price differences for all food groups become higher. When the price difference is negative it indicates lower price levels for treatment areas. 114 Graph A4-21 Price Differences (Treatment-Control) by Food Groups (2000-2010) Difference Protein Difference Dairy Difference Vegetables Difference Fruits 20.00 Difference Price Index 10.00 0.00 -10.00 -20.00 -30.00 -40.00 -50.00 1/2000 1/2001 1/2002 1/2003 1/2004 1/2005 1/2006 1/2007 1/2008 1/2009 1/2010 Source: Own estimations using Guyana Central Bank data and Guyana Market Corporation Prices. Average yearly price differences, expressed in percentages, are imputed into the poverty line thresholds, assuming that any percent change in food prices will proportionally affect household expenditure. Because the poverty thresholds were built from expenditure data based on price and quantities of good consumed in the household, this is not an unreasonable assumption. Once price shocks are translated into poverty line changes it is possible to estimate the changes in the total number of poor people and estimate the number of people that did not fall into poverty because of the presence of the program. To do so, we need to further assume that the distribution of the children and region’s population remains constant from 2006-2009 (see Figure A4-1). Figure A4-1 Steps for Estimation of Price Shocks and Poverty Changes Estimation of Number of Price Change Price Price difference children that did not fall (by Food differences (%) creates change into poverty because Group) between T&C in poverty line program kept poverty line (GY$) higher than control areas 2006 to mid 2007; and mid 2007 to 2009 Rural poverty change in regions 1 and 7 Price estimated using simulation population Control and proportions and Treatment poverty line changes Based on food frequency The final step is to estimate actual poverty changes. To do so, we need the total poor population distributed by area (urban, rural) and the proportion of population distributed by region. In addition, poverty rates for 2009 were estimated using a conservative rate of population increase and poverty changes. Table A4-2 shows the total number of poor people and poverty rates in 115 2006 and 2009. For 2009, the difference in prices was used to change the poverty rate threshold 40. In consequence it could be estimated the difference in the number of people that fall below the poverty threshold according to the price shock differential in treatment and control areas. In other words, without the program food prices increases would have caused an upward shift in the poverty threshold and consequently in the number of poor people. Table A4-22 Total poor population by area and Poverty Rates 2006 and 2009 2006 2009 Estimates Number of poor Population Rate Number of poor Population Rate Extreme Poverty Extreme Poverty National 124,637 681,977 18.3 National 121,160 665,234 18.2 Urban 13,684 193,509 7.1 Urban 13,089 198,020 6.6 Rural 110,953 488,468 22.7 Rural 109,455 467,215 23.4 Moderate Poverty Moderate Poverty National 244,088 681,977 35.8 National 237,278 665,234 35.7 Urban 35,784 193,509 18.5 Urban 34,227 198,020 17.3 Rural 208,304 488,468 42.6 Rural 205,492 467,215 44.0 Source: Own estimates using Household Budget Survey data 2006 Estimates based on annual population growth average and poverty rate change (2000-2006) The predictions based on the conservative scenarios using the baseline population of the Census 2002. The population growth is declining in Guyana in recent years therefore the 2009 estimates in Table A4-22 show an actual decrease in rural population and nationally. But overall there is an increase in both extreme and moderate poverty in rural areas, whereas urban areas experience a decline over the baseline and simulated year. Graph A4-23 shows the UN Population predictions for Guyana from 2005 to 2050 under three fertility scenarios. This shows the tendency of population reduction in this country. 40 Because the difference is expressed between price levels in treatment areas minus the price levels of control areas, the changes in poverty will be a result of an increase in the poverty threshold when prices in the control group are higher. 116 Source: UN http://ic.ltrr.arizona.edu/ic/nats101c/quiz/guyana05_50.png Regions 1, 7, 8 and 9 rank highest among the regions with access to public services and facilities such as schools, hospitals, electricity, water and roads. The EDMI is a marginality index built at the NDC level. The higher the score the least access to services and facilities by the community. Graph A4-24 EDMI by NDC and Region in Guyana 4.000 3.000 2.000 EDM Index 1.000 0.000 -1.000 -2.000 I II III III IV IV V V VI VI VI VII VIII IX X Regions Enumerator District Marginality Index (Based on Skoufias, 2006) The simple estimations of the changes in poverty due to food price changes relied on several assumptions, which are enlisted below: 117 Assumptions 1. Price changes entirely affect consumption (all income effect); no confounding factors of consumption and poverty change considered. This assumption does not consider changes and shift in patterns and preference of consumption. Although it does consider the frequency of food groups to approximate changes in consumption, a better model will incorporate percent changes in consumption or price elasticities of the various products included in the consumption basket to estimate poverty lines. 2. Poverty and Population Growth in the 2006 to 2009 period is constant. Population growth in Guyana’s hinterland rural areas is decreasing, according to Guyana’s Bureau of Statistics. Poverty has experience modest decreases in early 2000s. From 2006 and 2009 modest and conservative changes in population and poverty changes were considered. 3. Poverty rates change proportionally to the poverty line threshold. This is a strong assumption. However, in Guyana’s context, relatively steady and low levels of inflation makes this assumption reasonable. This assumption will be unreasonable in countries that experience sharp changes in inflation rates. 4. Regional distribution of children population is proportional to the regional distribution of poor children. This is an assumption that prevents overestimating the poverty effects of the children that are at risk of falling into poverty in the presence of food price shocks. In other words, this represents a lower bound estimate since the number of children that may become poor is estimated by multiplying the total change in poor people in rural areas and the proportion of children dwelling in regions 1 and 7. In reality, in hinterland areas the proportion of poor children is higher than the proportion of children population. 5. Food groups and categories used determine the consumption basket. The consumption basket used to estimate the poverty thresholds comprises the same food groups but more products. Based on the methodology and assumptions described above, Table A4-3 shows the results in poverty changes and the potential children that could fall below the poverty line in the absence of a safety net program. We use four main food group categories and estimate the treatment and control price differences before and during the food prices shock. Since food prices are reported in indices (constant terms) the changes are expressed in percentages. For instance, the protein food group showed a marginal 4 percent difference between treatment and control areas during 2006 and mid-2007. The difference for the same food group in the food price shock period escalated to 15.1 percent. The price shocks or percentage differences are then expressed in terms of changes in the poverty line threshold. The moderate poverty line of GY$11,143 per month (USD 1.85 daily) would have increased on average G$445.4 per month (USD 0.07 daily) more in control areas for protein products in the period before the food price crises. In the food price crisis period there was around GY$2,000 per month (USD 0.33 daily) shock for control areas. 118 These estimates can then be used to estimate the average change in poverty using the poverty line and the population under the poverty threshold. Table A4-23 Change in poverty in the absence of a safety net Before Price Food Price Shock Shocks * Period ** Mean Price Shock (treatment minus control) (%) Protein -4.00 -15.08 Dairy 0.59 -7.72 Vegetables -8.10 -17.66 Fruits -5.77 -13.32 Price shock change in Poverty Line (GY$, T-C) Protein -445.4 -2,002.0 Dairy 65.6 -1,024.9 Vegetables -902.4 -2,344.2 Fruits -642.8 -1,768.5 Poverty Change from Price Difference (population) Rural Poverty 6.0 21.5 Change (%) Rural Poverty 12,595 44,124 Change (#) Total Children at risk of 2,572 9,011 poverty Regions 1 and 148 517 7 Children * Includes from January 2006 to June 2007 ** From July 2007 to December 2009 119 SFP Costs Table A4-24 SFP Costs No. of Start-up costs Total Variable School No. of Total Cost Average teaching (Fixed Cost) Cost 2007-2009 name cooks 2007-2009 Cost per Day staff USD USD Agatash 2 4 5,628 50,652 56,280 87.9 Itaballi 7 5 8,372 75,348 83,720 130.8 72 Miles 3 3 3,892 35,028 38,920 60.8 Two Miles 6 7 9,968 89,712 99,680 155.8 St. 17 13 30,156 271,404 301,560 471.2 Anthony’s Kartabo 2 2 3,584 32,256 35,840 56.0 Holy Name 6 5 9,296 83,664 92,960 145.3 St. Mary’s 2 2 2,828 25,452 28,280 44.2 St. Martin’s 1 3 3,836 34,524 38,360 59.9 Annex Kurupung 2 3 3,416 30,744 34,160 53.4 Isseneru 1 2 3,220 28,980 32,200 50.3 Port 24 19 50,176 451,584 501,760 784.0 Kaituma Arakaka 4 5 6,860 61,740 68,600 107.2 Mabaruma 20 14 32,564 293,076 325,640 508.8 Wauna 8 12 30,688 276,192 306,880 479.5 Assakata 3 4 4,200 37,800 42,000 65.6 Kamwatta 6 7 13,188 118,692 131,880 206.1 (Moruca) Karaburi 5 7 14,168 127,512 141,680 221.4 Santa Rosa 23 14 37,324 335,916 373,240 583.2 Waramuri 9 9 19,152 172,368 191,520 299.3 TOTAL 151 140 292,516 2,632,644 2,925,160 228.5 Note: Only treatment schools with data and grant awards included Source: SFP Data 120 2005 2006 2007 2008 2009 Daily Amount Cost Cost Cost Cost Cost Food Basket Unit (Oz) Sept. (G$) Sept. (G$) Sept. (G$) Sept. (G$) Sept. (G$) $ per $ per $ per $ per $ per Oz. Daily Oz. Daily Oz. Daily Oz. Daily Oz. Daily Cereals & cereals Products $ per Rice (white) Oz. 2.32 2.65 6.14 2.89 6.70 3.23 7.56 3.89 8.42 4.21 9.12 $ per Flour Oz. 2.32 3.83 8.89 3.90 9.06 4.07 9.54 4.67 10.61 5.06 11.49 $ per Macaroni Oz. 2.32 8.16 18.93 8.24 19.12 8.50 19.93 9.85 22.19 10.67 24.03 Pulses & Pulse products $ per Split peas Oz. 1.48 4.04 5.98 4.29 6.34 4.65 6.95 5.66 7.73 6.13 8.37 $ per Blackeye peas Oz. 1.48 8.96 13.26 10.96 16.23 13.71 20.51 15.58 22.82 16.87 24.71 Meat, Fish & eggs $ per Stew Beef Oz. 0.83 12.26 10.18 13.57 11.26 15.34 12.87 18.84 14.32 20.40 15.51 $ per Chicken (frozen) Oz. 0.83 12.93 10.73 15.00 12.45 17.80 14.93 22.06 16.61 23.89 17.99 $ per Banga mary (fish) Oz. 0.83 8.31 6.90 6.55 5.44 5.28 4.42 6.33 4.92 6.85 5.33 $ per Shrimp,fresh Oz. 0.83 7.27 6.03 6.25 5.19 5.49 4.61 6.25 5.13 6.77 5.55 $ per Shoulder pork Oz. 0.83 14.25 11.83 18.52 15.37 24.60 20.64 31.31 22.97 33.91 24.87 Milk & Milk products $ per Fresh milk Oz. 0.83 2.38 1.98 3.97 3.30 6.77 5.68 7.86 6.32 8.51 6.84 Oil & Fats (exc. Butter) 121 $ per Fry oil Oz. 0.74 6.99 5.18 9.77 7.23 13.95 10.43 16.19 11.61 17.53 12.57 $ per Margarine Oz. 0.74 13.31 9.85 12.96 9.59 12.90 9.65 14.97 10.74 16.21 11.63 Vegetables & vegetables Products $ per Cassava Oz. 3.99 3.18 12.67 3.28 13.07 3.45 13.92 4.01 15.49 4.34 16.78 $ per Eddoes Oz. 3.99 2.78 11.08 3.23 12.87 3.83 15.43 4.44 17.17 4.81 18.60 $ per Pumpkin Oz. 1.98 3.39 6.72 2.23 4.41 1.49 2.98 1.73 3.32 1.87 3.60 $ per Plantains Oz. 3.99 2.94 11.72 3.41 13.62 4.05 16.34 4.70 18.18 5.09 19.69 $ per Callable Oz. 1.98 1.16 2.30 1.76 3.47 2.71 5.43 3.15 6.04 3.41 6.54 $ per Ochra Oz. 1.98 3.89 7.70 4.25 8.42 4.75 9.50 5.51 10.57 5.97 11.45 $ per Tomatoes Oz. 1.98 9.23 18.27 12.15 24.06 16.35 32.72 18.98 36.42 20.55 39.43 Fruits & fruit Products $ per Oranges Oz. 1.71 5.50 9.41 4.25 7.27 3.36 5.80 4.15 6.46 4.50 6.99 $ per Bananas Oz. 1.71 3.03 5.18 3.95 6.75 5.26 9.09 6.51 10.12 7.05 10.96 $ per Papaw Oz. 1.71 3.80 6.50 5.40 9.24 7.85 13.56 9.71 15.09 10.52 16.34 $ per Watermelon Oz. 1.71 5.86 10.03 3.86 6.60 2.60 4.49 3.22 5.00 3.49 5.42 $ per Sugar (dark ) Oz. 2.27 2.46 5.58 2.84 6.45 3.36 7.70 4.15 8.57 4.50 9.28 Total (Daily) 45.38 223.01 243.49 284.68 316.83 343.10 Total (Monthly) 6913.40 7548.31 8824.97 9821.58 10635.9 Source: Caribbean Food and Nutrition Institute, GMC and Guyana Bureau of Statistics 122 ANNEX 5 SUPPLEMENTARY TABLES Table A5 – 1: Head teachers’ profile Percent of head teachers Factor Categories Round 1 Round 2 Round 3 Head teacher 40.7 54.1 53.4 Senior or Senior assistant 29.7 25.2 25.9 teacher or assistant teacher 1. Designation Temporary qualified or 20.4 13.1 15.5 unqualified teacher Pupil teacher or teacher aide 9.4 6.5 5.2 or acting teacher or other ≤ 29 years 18.3 9.8 8.6 30 – 39 years 20.0 31.1 27.6 2. Age 40 – 49 years 45.0 36.1 34.5 ≥ 50 years 16.7 23.0 29.3 Female 52.5 57.6 53.4 3. Sex Male 47.5 42.4 46.6 None 18.2 0 1.9 4. Academic SSPE Part 2 N/A 11.9 7.5 qualifications Foundation upgrading course N/A 32.2 41.5 (*) CXC / O levels 40.0 33.9 28.3 Other 41.8 22.0 20.8 None 15.0 11.9 12.3 CPCE/Trained teacher 5. Professional N/A 72.9 71.9 certificate qualifications Diploma in education 66.7 10.2 8.8 (*) BEd 6.7 3.4 8.8 Other 11.7 1.7 5.3 (*) Round 1 categories for academic and professional qualifications are not comparable to those of Rounds 2 and 3 – see text. 123 Table A5 – 2: Class teachers’ profile Factor Categories Percent of teachers Round 1 Round 2 Round 3 Head teacher, Senior or Senior assistant teacher or assistant 23.4 27.4 34.4 teacher 1. Temporary qualified or Designation 41 37.8 33.8 34.4 unqualified teacher Pupil teacher or teacher aide or 38.7 38.9 31.3 acting teacher or other ≤ 20 years 12.5 10.1 7.5 20 - 29 years 49.1 49.6 43.8 2. Age 30 – 39 years 18.8 25.2 25.4 ≥ 40 years 19.7 15.1 23.8 Female 75.5 75.0 72.7 3. Sex Male 24.5 25.0 27.3 None 11.1 1.5 3.2 4. Academic SSPE Parts 1 and 2 N/A 11.2 10.1 Foundation upgrading course N/A 18.7 21.5 qualifications CXC / O levels 56.5 61.2 53.2 Other 32.4 7.4 12.1 None 48.6 59.3 51.9 5. Professional Trained teacher’s certificate 29.7 28.6 36.8 Certificate in education 2.7 3.6 3.0 qualifications Other 18.9 8.5 8.3 41 The designation of teachers is as follows: Senior Assistant-a certified teacher with 7 years of experience; Senior- a certified teacher with 5 years of experience; Assistant-a certified teacher; Temporary Qualified-teacher that has met the basic requirements but is not trained; Temporary Unqualified-a teacher who has not met the basic requirements and is not trained; Pupil-an untrained teacher under 17 years of age; Teacher Aide- an untrained teacher who assists teachers; Acting- an untrained teacher employed while a qualified teacher is found. 124 Table A5-3: Observed classes Percent of observations Variable Category Round 1 Round 2 Round 3 1. Grades 2 28.7 19.9 18.8 observed 3 28.7 15.2 12.4 4 24.1 14.6 11.2 5 N/A 14.6 8.2 6 N/A N/A 11.2 > one grade present 18.4 35.7 38.2 2. Subjects taught Maths 36.9 32.0 37.6 English 42.7 21.9 12.7 Science 10.7 8.3 10.9 Social studies 9.7 10.7 10.3 Other - 27.2 28.5 3. Teaching Discussion 12.5 8.9 14.3 method Group work 14.4 14.9 15.5 Student directed 5.8 4.2 2.4 Teacher directed 53.8 69.6 62.5 Other 13.5 2.4 5.4 Table A5 – 4: Observation details Round 1 Round 2 Round3 Variable Median Range Median Range Median Range No. of good desks 8 0 – 24 8 0 - 75 9.5 2 – 35 No. of bad desks 0 0 – 15 0 0 - 14 0 0 – 10 No. of books 18 0 – 255 48.5 0 - 948 125 0 – 4132 Time allocated to subject 30 15 – 70 30 15 – 99 35 15 – 75 Time utilized for subject 25 0 – 65 20 7 - 70 30 10 – 70 Class enrollment 17 2 – 110 20 1 - 115 22 3 – 98 No. present during observation 14 2 - 95 16 1 - 71 16 1 - 73 125 Table A5-5: Occupation of respondents % of heads of Occupations Percent of respondents households R1 R2 R3 R2 R3 A. Most frequently mentioned occupations Farmer 40.8 37.0 45.2 46.3 48.6 Housewife (woman), Unemployed (man) 25.6 34.8 25.7 6.3 4.0 Teacher 8.0 N/A 0.2 0.4 0.9 Miner 4.5 4.9 3.2 10.4 12.5 Health professional (medex, nurse, physiotherapist) 3.5 3.3 3.2 2.5 2.7 Caterer 3.3 4.3 4.0 3.6 1.6 Fisherman, seaman 2.6 2.3 3.2 5.2 5.1 Laborer 1.7 4.7 2.0 4.7 3.1 Carpenter 0.9 3.9 0.4 3.9 2.9 B. Less frequently mentioned occupations Office workers, police/security persons, salespersons, 3.1 2.3 2.2 2.7 2.5 hairdresser Engineer, gov. employee, businessman, accountant, IT 3.0 2.4 2.8 5.9 5.4 officer, supervisor, contractor, ship’s captain, social worker Handicraft / factory worker, logger, plumber, mechanic, 1.9 1.3 3.6 5.2 7.9 welder, electrician, driver, machinist, fireman, painter Domestic, gardener, vendor, porter 0.9 3.4 2.0 2.3 1.3 Self-employed (occupation not specified) 1.6 0.5 2.4 0.5 1.6 Table A5 – 6: Household possessions - parents’ and students’ responses Percent of respondents (positive responses only) Item Round 1 Round 2 Round 3 A. Parents’ responses Books 80.9 85.9 89.2 Newspapers / magazines 70.7 71.2 71.6 Radio 61.7 52.0 45.7 Dictionary 53.3 50.8 52.0 Calculator 49.3 44.8 45.9 Television 32.1 33.3 42.2 Boat 25.7 57.1 44.3 Telephone 21.2 50.6 57.0 Refrigerator 19.1 16.3 20.8 Computer 2.1 4.3 4.2 B. Students’ responses Radio 51.0 Television 57.8 Refrigerator 34.4 Boat, car or tractor 21.7 Livestock 8.5 126 127