Kiribati Early Grade Reading Assessment (KiEGRA) RESULTS REPORT 1 Table of Content Table of Content ................................................................................................................. 2 List of Tables ...................................................................................................................... 3 List of Figures ..................................................................................................................... 5 Executive Summary ............................................................................................................ 6 Summary of EGRA results and findings ........................................................................ 7 Factors contributing to greater fluency and comprehension in Kiribati ......................... 8 Recommendations ........................................................................................................... 8 Chapter 1: Introduction ..................................................................................................... 13 Chapter 2: Instrument Design and Implementation .......................................................... 15 2.1. KiEGRA Survey Implementation ..................................................................... 15 2.2. Sample Design .................................................................................................. 15 2.3. Development of the KiEGRA Instrument......................................................... 15 2.4. Enumerator training .......................................................................................... 16 2.5. Data collection .................................................................................................. 16 2.6. Reliability of the Instrument ............................................................................. 16 Chapter 3: EGRA Results ................................................................................................. 18 3.1. Sub-test 1 – Letter Name Knowledge ............................................................... 18 3.2. Sub-test 2 – Letter Sound Knowledge .............................................................. 20 3.3. Sub-test 3 – Initial Sound Identification ........................................................... 22 3.4. Sub-test 4 – Familiar Word Reading ................................................................ 24 3.5. Sub-test 5 – Nonword Reading ......................................................................... 26 3.6. Sub-test 6 – Oral Passage Reading ................................................................... 28 3.7. Sub-test 7 – Reading Comprehension ............................................................... 30 3.8. Sub-test 8 – Listening Comprehension ............................................................. 35 3.9. Sub-test 9 – Dictation........................................................................................ 37 3.10. Summary of assessment results ........................................................................ 40 Chapter 4 – Performance in reading fluency and reading comprehension ....................... 44 Chapter 5- Analysis of Student and Teacher Factors Associated with Better Reading Outcomes .......................................................................................................................... 49 5.1. Association of student characteristics to student reading outcomes ................. 49 5.2. Association of teacher characteristics to student reading performance ............ 51 5.3. Association of teacher expectations to student reading performance ............... 52 5.4. Association of teacher training and guides to student reading performance .... 54 5.5. Association of Classroom Environment to Student Performance .................... 54 5.6. Association of Reading Materials to Student Performance .............................. 56 5.7. Association of Teacher Instructional and Assessment Methods to Student Performance .................................................................................................................. 56 Chapter 6 - Conclusions and Next Steps........................................................................... 61 ANNEX 1 / TABLES ....................................................................................................... 66 ANNEX 2 / INSTRUMENTS .......................................................................................... 78 2 List of Tables Table 1: EGRA Sample by Region, Year and Gender...................................................... 15 Table 2: Reliability of the EGRA Sub-tests ...................................................................... 17 Table 3: Sub-test 1 Letter Name Knowledge: Results by Year and Gender .................... 19 Table 4: Letter Name Results by Region and Year .......................................................... 19 Table 5: Sub-test 2 Letter Sound Identification: Results by Year and Gender ................. 20 Table 6: Letter Sound Identification: Results by Region and Year .................................. 21 Table 7: Sub-test 3 Initial Sound Identification: Results by Year and Gender ................. 22 Table 8: Initial Sound Recognition: Results by Region and Year .................................... 24 Table 9: Sub-test 4 Familiar Word Reading: Results by Year and Gender ...................... 25 Table 10: Familiar Word Reading: Results by Region and Year (without zero scores)... 25 Table 11: Sub-test 5 Nonword Reading: Results by Year and Gender............................. 27 Table 12: Nonword Reading: Results by Region and Year (without zero scores) ........... 27 Table 13: Sub-test 6a Oral Passage Reading: Results by Year and Gender ..................... 29 Table 14: Oral Passage Reading: Results by Region and Year ........................................ 29 Table 15: Sub-test 6b Reading Comprehension: Results by Year and Gender ................ 31 Table 16: Correct Answers by Questions in sub-test 6b ................................................... 33 Table 17: Distribution of Correct Answers ....................................................................... 34 Table 18: Reading Comprehension: Results by Region and Year .................................... 34 Table 19: Sub-test 7 Listening Comprehension: Results by Year and Gender................. 35 Table 20: Listening Comprehension Percentage of Correct Answers by Question.......... 36 Table 21: Listening Comprehension Results by Region and Year ................................... 36 Table 22: Sub-test 9 Dictation: Results by Year and Gender ........................................... 38 Table 23: Dictation: Results by Region and Year (without zero scores) .......................... 40 Table 24: Distribution of percent correct scores for reading comprehension by sub-test per year ............................................................................................................................ 44 Table 25: Percent of students reading with 80% or more comprehension by Year .......... 45 Table 26: ORF score by reading comprehension score .................................................... 46 Table 27: Distribution of ORF Scores for students meeting and not meeting the 80% benchmark ................................................................................................................. 47 Table 28: Student background characteristics .................................................................. 49 Table 29: Association of student characteristics to Oral Reading Fluency (ORF) scores 50 Table 30: Profile of teachers in EGRA ............................................................................. 51 Table 31: Association of teachers' characteristics to Oral Reading Fluency Score (ORF) ................................................................................................................................... 52 Table 32: Association of teacher expectations to oral reading fluency (ORF) scores ...... 53 Table 33: Teacher training and guides .............................................................................. 54 Table 34: Association of training and guides to student oral reading fluency (ORF) scores ................................................................................................................................... 54 Table 35: Average Number of Classroom Displays and Materials Observed .................. 54 Table 36: Frequency and Type of Classroom Displays/Resources Available .................. 55 Table 37: Association of Classroom Environment to Student ORF Scores ..................... 55 Table 38:Average Number of Reading Materials and Books in Classroom ..................... 56 Table 39: Association of Reading Materials to Student ORF Scores ............................... 56 3 Table 40: Frequency of methods used during reading instruction .................................... 57 Table 41: Association of teacher instructional and assessment methods to student performance .............................................................................................................. 57 Table 42: Impact of student characteristics on Oral Reading Fluency (ORF) scores ....... 72 Table 43: Effect of teachers' characteristics on Oral Reading Fluency Score (ORF) ....... 73 Table 44: Effect of teacher expectations on oral reading fluency (ORF) scores .............. 73 Table 45: Effect of training and guides on student oral reading fluency (ORF) scores ... 74 Table 46: Effect of Classroom Environment on Student ORF Scores.............................. 74 Table 47: Effect of Reading Instructional Resources on Student ORF Scores................. 74 Table 48: Effect of teacher instructional and assessment methods on student performance ................................................................................................................................... 75 4 List of Figures Figure 1: Letter Name Results by Region and Year ......................................................... 20 Figure 2: Letter Sound Identification: Results by Region and Year ................................. 22 Figure 3: Initial Sound Recognition: Results by Region and Year ................................... 24 Figure 4: Familiar Word Reading: Results by Region and Year (without zero scores) ... 26 Figure 5: Nonword Reading: Results by Region and Year (without zero scores) ............ 28 Figure 6: Oral Passage Reading: Results by Region and Year ......................................... 30 Figure 7: Reading Comprehension Results: Distribution of percentage of correct answers by year ....................................................................................................................... 32 Figure 8: Reading Comprehension Results: Distribution of percentage of correct answers by gender ................................................................................................................... 32 Figure 9: Reading Comprehension: Results by Region and Year .................................... 34 Figure 10: Listening Comprehension Results by Region and Year .................................. 37 Figure 11: Dictation Results: Percentage of Correct Answers by Item and Year ............ 39 Figure 12: Dictation Results: Distribution of correct answers by gender ......................... 39 Figure 13: Dictation: Results by Region and Year (without zero scores) ........................ 40 Figure 14: Summary Results: Number correct answers for timed sub-tests by year ........ 41 Figure 15: Summary Results: Percent correct for untimed sub-tests by year ................... 42 Figure 16: Summary Results: Percentage of Zero Scores by Sub-test and Year .............. 42 Figure 17: Summary Results: Percentage of Zero Scores by Sub-test and Gender .......... 43 Figure 18: ORF distribution by reading comprehension level.......................................... 45 Figure 19: Distribution of ORF scores for students reading with at least 80% comprehension .......................................................................................................... 46 Figure 20: Distribution of ORF scores for students meeting and not meeting 80% benchmark ................................................................................................................. 47 5 KIRIBATI EARLY GRADE READING ASSESSMENT Results Report Executive Summary This report summarizes the results of an Early Grade Reading Assessment (EGRA) conducted in Kiribati from September 21 to October 27, 2016. With funding from the Global Partnership for Education (GPE), the World Bank and Education Technology for Development (Et4d) carried out the assessment in collaboration with the Kiribati Ministry of Education (MOE). The overall purpose of the EGRA was to inform education policymakers of students’ basic reading skills in years 1-3 of primary school and to identify factors that contribute to reading development in Kiribati. The findings are expected to assist policymakers with designing effective early grade reading interventions to improve school performance and literacy outcomes. This activity is part of the Pacific Early Age and Readiness Program (PEARL), which was established to improve the school readiness and literacy outcomes of children throughout the Pacific region. The Kiribati EGRA (KiEGRA) was administered to a nationally representative sample of students enrolled in Years 1, 2 and 3. A total of 1,363 students (691 girls and 672 boys) participated in the assessment. The EGRA tool consisted of six reading skills tests and two reading-related tests (listening comprehension and dictation). Unlike most EGRAs, which primarily test reading and listening skills, the KiEGRA included a short dictation exercise to assess early writing skills. In addition, a teacher and student survey that collected information on characteristics associated with reading outcomes was administered to identify factors contributing to reading fluency. The assessors also carried out a classroom observation in each school visited to assess the classroom environment and teaching resources available. The analysis of KiEGRA data included descriptive statistics (means and standard deviations) to measure average levels in basic reading skills; an analysis of variance to determine statistical significance of gender and regional differences; and regression analysis to estimate the association of a given teacher, student or classroom characteristic and reading fluency outcomes. The key findings, factors associated with reading performance, and recommendations are presented below. 6 Summary of EGRA results and findings • Students show significant gains in word reading skills from Years 1 to 3. There was measurable progress between Years 1-3 across all sub-tests, especially for the familiar word and oral reading fluency sub-tests. Scores on the familiar word subtest increased by 15 words between Years 1-2 (from 9 to 24 words per minute), and an additional 13 words between Years 2-3 to an average of 37 correct words per minute. On the oral reading passage sub-test, students’ scores improved by 18 words between Years 1 and 2 (from 10.9 to 29) and an additional 18 words between Years 2 and 3 (from 29 to 47). Thus, students in Year 3 are on the cusp of becoming fluent readers. • Students are not yet fluent readers: In order to understand a simple passage, students must read fast enough to retain the words in short-term memory. Research1 suggests a minimum fluency rate of 45-60 words per minute, depending on the complexity of the language. Overall, reading fluency scores averaged 29 correct words per minute. While Year 3 students read an average of 47 words per minute, only 20 percent could comprehend 80 percent of the text. • Students’ struggle with decoding. To read text fluently, students must be able to decode unfamiliar words by sounding out individual letters and syllables. Students scored low across three sub-tests that measure these skills: letter sounds, initial sounds, and nonwords. Overall, students identified an average of 27 letter sounds, 3 initial sounds, and 13 nonwords. • Reading comprehension levels are below the recommended benchmark. Over 80 percent of students in Years 1 and one-third in Year 2 have zero reading comprehension. In Year 3, comprehension improves where nearly half of students could comprehend at least 60 percent of text. However, the majority of students in Years 1-3 are reading below the recommended 80 percent comprehension benchmark. Only 14 percent of all students tested met the benchmark and about one-third of students in Year 3. The breakthrough point for reading fluency and comprehension is likely much later than Year 3 for the majority of students. An analysis of fluency scores falling in the 80 percent comprehension range revealed that students must be able to read between 40-69 words per minute with accuracy. • Girls have higher reading fluency and comprehension . Girls had a lower percentage of zero scores and scored equal to or better than boys across all sub- tests. The difference in girls’ and boys’ performance was statistically significant. • Regional differences were not statistically significant. Students in the Linnux regions performed better than other regions. Yet, the differences were insignificant. 1 Abadzi, H. (2011). Reading Fluency Measurements in EFA FTI Countries: Outcomes and Improvement Prospects. Education for all Fast Track Initiative 7 Factors contributing to greater fluency and comprehension in Kiribati The following factors 2 were positively associated with better oral reading fluency, measured by the number of correct words read in the reading passage, per minute (cwpm): Student characteristics: • Student speaks Kiribati at home (+10 cwpm) • Student likes to read (+8 cwpm) • Student receives help with homework from mother (+5 cwpm) • Student reads to himself/herself at home (+5 cwpm) Teacher expectations: • When teachers expect students to reach the competency later than the expected grade level, students tend to have higher scores (+2-14 cwpm) Classroom environment: • Maintaining folder with student work and student info/student profile (+6 cwpm) Teaching pedagogy and assessment: • Teacher guided reading 1-2 days a week (+5 cwpm) • Teaching reading comprehension (+2-5 cwpm) • Child reading aloud or silently (+3-4 cwpm) Recommendations Based on the findings and conclusions, the following recommendations are presented to improve the quality of early grade reading instruction in Kiribati schools: 1. Set reading fluency and comprehension benchmarks to guide and support teacher professional development to reach reading targets: Reading benchmarks bring attention to the learning expectations both students and teachers should be held accountable to support children in becoming readers. They serve a useful function when they are coupled with strategies that will strengthen the capacity of schools and teachers to help students reach them. Without benchmarks, there is no pressure in the system to achieve learning outcomes and no fear of failure. At the same time, simply setting standards and benchmarks without evidence that schools and teachers are failing in their job and the support mechanisms to improve, benchmarks will create fear and frustration among school administrators, teachers and parents. Reading benchmarks should be established to guide the development of basic reading skills in each of the early grades both in terms of oral reading fluency and comprehension; and to guide the mechanisms that are required for schools to succeed in achieving them. As a first step, stakeholders should decide on the level of comprehension required to understand grade level text 2 Note that the average effects of the factors are not cumulative. In other words, students who possess more than one characteristic do not have double the effect by adding the values of both factors. 8 (e.g., 60 percent, 80 percent) and then review the fluency scores that fall within that range. Tonga, for instance, has set the reading fluency benchmark at 50 cwpm and a reading comprehension of at least 75%. Students achieving these levels can be considered to have learned to read well, have the basic reading skills needed to develop their literacy skills, and have the ability to comprehend more complex text in upper grades. If stakeholders agree with the 80 percent benchmark, then an acceptable fluency range may be 60-64 cwpm. Given that the current mean ORF score is 29 cwpm, that range may seem too high. Thus, policymakers may decide to lower the benchmark to 60 percent and the fluency benchmark to 53 cwpm. Once the benchmark is decided, the next step is to consider the targets. Currently, 14 percent of students are meeting the 80 percent reading comprehension benchmark. How many should meet the benchmark in one year or five years? Once the targets are set up, schools and educators should be supported to achieve these benchmarks. The Kiribati MEYS should then conduct a national reading campaign to sensitize all stakeholders of the new benchmarks and then monitor and report progress towards achieving the targets at all levels (national/regional MEYS, school and communities). 2. Conduct a follow-up study on teaching practices and assessment methods to identify strengths and weaknesses. Although 56 percent of teachers were trained on reading instruction in the past three years, teaching decoding skills and assessment were associated with lower ORF scores. It is important to understand why these interventions have an adverse relationship. It could be that teachers were not adequately supported to master the new teaching methodologies or that training is not aligned with best practices in reading instruction. Additionally, teachers may not understand how to accurately assess students and utilize the results for reflection and lesson planning. The analysis also showed that classrooms with teachers who maintained a record of students’ work had higher ORF scores. A follow-up study is recommended to investigate the strengths and weaknesses of current assessment and teaching methods vis-à-vis the results observed in different classrooms. It is also important to understand what types of information the teacher records and whether it is related to tracking student progress. 3. Strengthen teacher training for reading instruction with a focus on vocabulary, decoding skills and reading comprehension. Results indicate that students are weak in recognizing letter sounds and unfamiliar words, and have low reading comprehension skills. Hence, instruction in these areas should be strengthened to increase overall reading scores. In the case of Kiribati, the large shares of zero scores observed in various reading subtasks included in KiEGRA suggest inadequate instructional time in the classroom due to student absenteeism, school closures, or even inadequate use of instructional time -i.e. pedagogy or content-- during regular school days. Additionally, training should incorporate curriculum expectations and provide teachers with the specific methods, classroom activities and assessment methods required to achieve results. 9 4. Increase the number of books in the classroom library/reading corner and the use in teachers’ instruction. In Kiribati, 75 percent of classrooms have reading corners, but only 1.97 books are used during classroom instruction. One possible explanation is that there are not enough books in the classroom. The International Reading Association (IRA) recommends that classroom libraries start with at least seven books per child and purchase two additional new books per year. The optimal number of books in a classroom library is 300-600, depending on the grade level and number of copies3. The number of books teachers should expect children to read during the school year is 100-125 picture books by the end of Year 1 and 50- 75 chapter books by the end of Year 2. There is strong evidence that reading skills grow and develop to the extent that children get actual practice reading. This is particularly difficult in countries with a modest literary tradition where the amount of titles in the local language may be limited. Recent innovations in technology now enable the production of reading materials in almost any language, provided there are guidelines for avid writers to follow. Educators in Kiribati have experience producing additional reading materials for children and their efforts should be further supported to ensure beginning readers have a reasonable variety of stories to practice and enjoy. With a corpus of reading large enough to provide sufficient reading practice, a low-cost option is to provide e-readers in low-resource countries where publishing may be expensive and there are high teacher-pupil ratios. E- readers allow students and teachers to choose from a variety of genres, it is portable so students can read from home or school, and its read aloud features provides additional support for emergent readers45. In addition to provision of an increased number of hard and soft copy books, teachers should be trained on how to better integrate materials into their instruction and on how to develop attractive reading corners6. 5. Increase time-spent reading. The more time children spend reading the better and more fluent readers they become. In order to increase students’ reading fluency skills, teachers should ensure that students are spending sufficient time reading every day through teacher-led, parent-led or self-guided reading activities. Research recommends that children read between 20-40 minutes per day. The results also showed that bringing books home on a regular basis was associated with lower ORF scores. It is important to engage parents in reading activities to ensure that students and parents are spending the recommended time reading at home. 6. Address the issue of non-readers in Years 1-2. Stakeholders should conduct classroom level assessments to identify non-readers in Years 1 and 2, diagnose the 3 Neuman, S. (undated). The importance of the classroom library. Available at: http://teacher.scholastic.com/products/paperbacks/downloads/library.pdf 4 Adams, A. & van der Gaag, J. (2011). First Step to Literacy: Getting Books in the Hands of Children Available at: https://www.brookings.edu/research/first-step-to-literacy-getting-books-in-the-hands-of- children/ 5 UNESCO (2014). Reading in the mobile era: A study of mobile reading in developing countries. http://unesdoc.unesco.org/images/0022/002274/227436E.pdf 6 Neuman, S. (undated). The importance of the classroom library. 10 causes, and design specific activities to address deficiencies before children complete Year 2. Teachers in Years 1 and 2 have the greatest responsibility in ensuring children finish the first two grades of primary education with the necessary skills to read and understand text. In this sense, identification of struggling readers and knowledge of ways in which they need to be supported should make the core material of in-service training for Year 1 and Year 2 teachers. For instance, teachers may group students according to ability and provide remedial activities and appropriately levelled text. This “catch-up approachâ€? is being used by UNICEF in Zambia based J-Pal’s research in India, which demonstrated that students grouped by ability is more effective than mixed-ability grouping. UNICEF will assess students in all grades and group them according to reading levels (non-readers, those who can read letter sounds, syllables, words, passages, etc.). Grouping students by level rather than grades has produced dramatic results in India, Kenya and Ghana. Teachers and school administrators should further determine whether non-readers have learning disabilities (e.g., dyslexia) and design relevant interventions for special needs students. 7. Teach students how to develop reading comprehension skills in the early grades. Given only 14 percent of students are able to comprehend 80 percent or more of grade level text, teachers should focus on increasing reading comprehension levels for all years. Research shows that teachers are often the ones to ask questions orally and that students are expected to respond, but rarely are students asked to develop their own questions as they read (think aloud) or to make predictions and then check their predictions. Students need to be empowered to make meaning from text and to use an inquiry-based approach to reading. Teachers could ask students to make predictions before, during and after reading, to retell stories, or to identify problems within the story and provide possible solutions. Teachers should use various strategies and frameworks to develop students’ reading comprehension skills from as early as kindergarten or Year 17. 8. Develop activities that specifically focus on raising boys’ performance and interest in reading. The results illustrated that boys consistently performed lower than girls. There may be cultural or gender barriers that affect boys’ interest and engagement in reading activities. In addition to designing strategies to address low competencies of boys and girls, stakeholders at all levels should discuss the potential challenges specific to boys and design strategies to improve boys’ reading achievement. Successful strategies that have worked in other countries include developing gender-sensitive materials that attract boys’ attention (such as sports, 7 Sample reading comprehension activities can be found in the following guides: Ontario Ministry of Education. (2003). A Guide to Effective Instruction in Reading: Kindergarten to Grade 3 (Available at: http://eworkshop.on.ca/edu/resources/guides/Reading_K_3_English.pdf) Institute of Education Sciences (IES). (2010). Improving Reading Comprehension in Kindergarten through 3 rd Grade. What Works Clearninghouse (Available at: https://education.ohio.gov/getattachment/Topics/Early-Learning/Third- Grade-Reading-Guarantee/Third-Grade-Reading-Guarantee-Teacher-Resources/Improving-Reading- Comprehension-in-Kindergarten-Through-3rd-Grade.pdf.aspx) 11 science fiction, fantasy, comic books, digital text, and stories that are humorous) and integrating reading into extracurricular activities (e.g., sports, health clubs, student government). 12 Chapter 1: Introduction Kiribati is a country consisting of 33 coral atolls in the Pacific Ocean located about halfway between Hawaii and Australia. It is one of the smallest and most remote countries in the world with a population of 106,925 (July 2016 est.)8. Education is free and compulsory from age 6 to 14. Primary education consists of Years 1 to 6.9 Among the total official school age population (6-14 years old), 96percent were enrolled in primary school in 201410. Including those over and under-age, the gross enrollment rate was 113percent11. Despite high enrollments and alarmingly low literacy rates throughout the region, the country has performed well on regional literacy assessments. According to the 2015 Pacific Islands Literacy and Numeracy Assessment (PILNA), 93 percent of Year 4 students are meeting or exceeding the expected level for literacy and 72 percent are performing at or above the expected level in Year 6. This is a marked improvement from 2012 where 66 percent were meeting or exceeding the benchmark in Year 4 and 43 percent in Year 6. The government of Kiribati has instituted a new curriculum and implemented curricular reforms under the Kiribati Education Improvement Program (KEIP). The overarching goal of the KEIP is that, by 2020, all Kiribati children achieve functional literacy and numeracy after six years of basic education12. In order to achieve this goal, learning how to read in the early grades is critical. The Pacific Early Age and Readiness Program (PEARL) was established to improve the school readiness and literacy outcomes of young children throughout the Pacific region. The World Bank, in collaboration with the Kiribati government and local education stakeholders, are collaborating in this joint effort to undertake an Early Grade Reading Assessment and to design effective interventions to improve school readiness and reading development in Kiribati. The Early Grade Reading Assessment (EGRA) measures basic reading skills in the first three grades of primary school. The EGRA tool has been administered in over 100 languages in more than 60 countries around the world, including the Pacific islands of Tonga, Vanuatu, Papua New Guinea and Timor Leste. The EGRA tool was adapted to the Kiribati context and administered nationally from September 21 to October 27, 2016. The Kiribati EGRA (KiEGRA) aimed to achieve three objectives: 1. To develop survey of basic reading skills and temporary reference standards to monitor reading performance in schools and system wide; 8 CIA World Factbook. (2016). Kiribati population. Available at https://www.cia.gov/library/publications/the-world-factbook/geos/kr.html 9 In the Kiribati education system, grade levels are called years. Year 1 is a synonym for Grade 1. 10 World Bank. (2014). Kiribati gross enrollment ratio, primary, both sexes. Available at http://data.worldbank.org/indicator/SE.PRM.ENRR 11 Ibid. 12 Kiribati Education Improvement Program (KIEP) Evaluation Report. (2014). Commissioned through the Australian government Education Resource Facility. 13 2. To build local capacity to replicate early grade reading assessments in the future; and 3. To work with local education stakeholders to interpret EGRA findings and analyze their policy and sector investment implications. The overall purpose of the KiEGRA is to inform education stakeholders of reading skills students are struggling with and to identify factors that appear to contribute to reading development in Kiribati. The findings will help policymakers to design responsive strategies to improve early grade reading instruction and assessment. In particular, the EGRA results may assist Kiribati policymakers with building the capacity of teachers, allocating budgetary and human resources, and with developing reading indicators and benchmarks. The ultimate goal of the KiEGRA is to ensure that all children develop the skills needed to become fluent readers. Towards this aim, the assessment was designed to respond to the following questions: 1. What are the basic reading skills of students in Years 1, 2, and 3 in Kiribati? 2. What is the year in which students in Kiribati “break intoâ€? reading fluently with comprehension? 3. How does student performance in Years 1, 2, and 3 compare to curriculum expectations in Kiribati language for these grades? 4. What are the main skills students struggle with? What are the main skills students have strengthened? 5. What are the differences in performance between boys and girls? 6. What are the student and teacher factors associated with strong and weak early grade reading outcomes? a. What are teacher expectations around when students should develop key basic reading skills? How different are these from curricular expectations in the early grades? b. What are the resources available in classrooms to support the reading development of children? c. What are the pedagogical practices supporting reading that are prevalent in the early grades in Kiribati classrooms? The report is divided into six chapters beginning with this introductory section in Chapter 1. Chapter 2 will discuss the KiEGRA survey design and implementation. Chapter 3 presents overall results of the reading assessment per sub-test, gender and year as well as a summary of the overall strengths and weaknesses responding to questions 1, 4 and 5. The correlation between oral reading fluency and comprehension (question #2) is discussed in Chapter 4. As there are no established standards for reading fluency, the report used 80 percent reading comprehension benchmark as the basis for determining fluency scores. Chapter 5 provides an analysis of student and teacher factors associated with improved reading outcomes, which corresponds with question #6 (6a, 6b, and 6c). Finally, the conclusions and policy recommendations to improve literacy outcomes are presented in Chapter 6. 14 Chapter 2: Instrument Design and Implementation 2.1. KiEGRA Survey Implementation In order to build local capacity to replicate early grade reading assessments, Kiribati Ministry of Education staff were directly involved in the EGRA design and implementation. From August 31 to November 1, 2016, the World Bank provided in- country support to selected Ministry staff to undertake all aspects of the assessment including, sampling, adapting the KiEGRA instruments into the i-kiribati language, facilitating the training of enumerators and supervisors, coordinating survey logistics during the pilot and fieldwork, and administering the EGRA using Tangerine software. 2.2. Sample Design With guidance from the World Bank, the Kiribati MOE chose a nationally representative sample for students in Years 1, 2, and 3. The target population was defined as students enrolled in Years 1 to 3 in primary schools implementing the official curriculum in Kiribati. A sample of 49 schools was selected based on enrollment data from the Kiribati Education Management Information System (KEMIS). To ensure all regions had an equal probability of being selected despite their actual distribution in the country, a stratified random design with proportional allocation was applied. The final sample consisted of 1,363 students, 691 girls and 672 boys (see Table 1). Table 1: EGRA Sample by Region, Year and Gender Year 1 Year 2 Year 3 Region Total Girl Boy Total Girl Boy Total Girl Boy Total Central 48 55 103 40 35 75 30 31 61 239 Linnux 29 30 59 32 25 57 31 27 58 174 North 85 68 153 72 75 147 73 65 138 438 South 32 29 61 32 41 73 29 33 62 196 South 54 53 107 54 52 106 50 53 103 316 Tarawa Total 248 235 484 230 228 458 213 209 422 1363 All statistical analysis perform take the complex survey design into consideration. For instance, all standard errors were adjusted using the Taylor series variance estimates adjusted for the design effects resulting from the multi-stage cluster sampling. Sampling weights were also computed to correct for the unequal probability selection resulting from the multi-stage cluster sampling. Thus all results presented are weighted results. 2.3. Development of the KiEGRA Instrument Due to differences in language, culture, and expectations about learning outcomes, the EGRA tool was adapted to the Kiribati context and piloted. From August 31 to September 7, 2016, a team of language experts, curriculum officers and teachers attended an adaptation workshop focused on learning the research foundations and structure of the EGRA tool in order to the develop the KiEGRA instrument in the Kiribati language. Before 15 the workshop, a language analysis was carried out to gain a good understanding of what orthography and language issues needed to be taken into account when developing the EGRA tools. Also a letter frequency lists, word frequency list and possible non-word list were developed. The letter names and letter sound grid had been drafted prior to the workshop. During the workshop, participants worked in small groups to develop a number of potential listening and reading texts with comprehension questions. Participants also translated the instruments into Kiribati. Two versions of the instruments were field-tested on September 5,2016. The results from the pre-test were shared and feedback from workshop participants and ministry staff were incorporated into the instrument. The revised instrument was piloted on September 7, 2016, and a few minor changes were applied to some of the comprehension questions. 2.4. Enumerator training The Enumerator Training Workshops for KiEGRA were held in Tarawa on September 26 to 30, 2016, and on Christmas Island on October 26 to 29, 2016. Pre-selected enumerators and supervisors attended the training workshop to learn about the scope and purpose of the survey and the manipulation of tablets for data collection. Assessors were trained three days in a classroom environment, and they had the chance to administer a practice EGRA in a nearby primary school, where the instruments had been previously piloted. An Assessor Accuracy Measurement 13 (AAM) was conducted three times in order to familiarize the enumerators with the process and examine their accuracy level. Results were used to select 16 enumerators and 4 supervisors. All candidates' results were above 90 percent overall. 2.5. Data collection Data collection took place in Kiribati between October 4 and November 9, 2016. Over a period of six weeks, teams of enumerators and supervisors visited sample schools in the country. Three teams based in Tarawa visited schools in the South, Central and North districts, and one team focused on schools in Linnux district. The data was uploaded at least once a week, and checked by the survey coordinators. 2.6. Reliability of the Instrument Cronbach’s alpha was computed for every task of the EGRA instrument to estimate the level of reliability of each sub-test. Results showed strong internal consistency for almost every sub-test except Reading Comprehension and Listening Comprehension. As a rule of thumb, an alpha coefficient of 0.80 is considered good while a 0.70 is the minimum 13 The AAM evaluates the degree of agreement among enumerators administering the same test at the same time to the same student. This type of measurement determines the trainees’ ability to accurately administer the EGRA. A script prepared beforehand, complete with deliberate errors, becomes the gold standard. After a practice exercise, the enumerators’ responses are compared against the gold standard. The benchmark is set at 90% agreement between the gold standard and the enumerator’s responses. Therefore, if the enumerators’ responses are in agreement with at least 90% of the responses on the gold standard, the assessor is considered qualified to administer the EGRA. The results are used to address any weaknesses identified during the enumerator training. 16 acceptable result. The lower values of alpha coefficients for Reading and Listening comprehension could be explained by a smaller number of items (five) in those sub-tests since the number of items in a sub-test influences the value of the Cronbach’s alpha. Table 2: Reliability of the EGRA Sub-tests Sub-tests Number of items Number of Alpha respondent Letter Name Knowledge 100 1,363 0.981 Initial Sound Identification 10 1,363 0.811 Letter Sound Knowledge 100 1,363 0.962 Familiar Word Reading 50 1,363 0.983 Non Word reading 50 1,363 0.968 Oral reading 60 1,363 0.988 Reading Comprehension 5 1,363 0.757 Listening Comprehension 5 1,363 0.606 Dictation 31 1,165 0.942 Correlations between tasks are presented in Annex. 17 Chapter 3: EGRA Results This section reports the results of the EGRA per sub-test. There were eight sub-tests comprised of the following: 1. Letter Name Knowledge 2. Letter Sound Knowledge 3. Initial Sound Identification 4. Familiar Word Reading 5. Nonword Reading 6. Oral Passage Reading 7. Reading Comprehension 8. Listening Comprehension 9. Dictation The average scores of the above sub-tests are provided and disaggregated by year and gender. Mean scores are presented for the entire sample as well as for the sample that was able to read at least one item (minus zero scores). The overall results show the actual reading performance of the entire population, including those who can and cannot read. The results minus zero scores show the average score of the population who can read at least one item, which addresses research concerns that the inclusion of a large percentage of zero-scores could underestimate the result of the population who can read. Results disaggregated by region are also provided at the end of each section to highlight any regional differences in average reading performance. 3.1. Sub-test 1 – Letter Name Knowledge14 The test of letter name knowledge is the most basic assessment of reading skills. It measures students’ ability to read the names of the letters of the alphabet with accuracy. Automaticity and fluency of letter name knowledge is a predictive skill for later reading success. During the EGRA, students were given a page of 100 randomly distributed upper- and lowercase letters and asked to say the names of as many letters as possible within one minute. The test included the full Kiribati alphabet and randomization was used to prevent students from reciting a memorized alphabet. The test is scored by the number of letters that students correctly name in one minute (correct letters per minute—clpm). Table 3 shows overall scores for sub-test 1 by year, gender and minus zero scores. Overall, students were able to identify 37 letters in one minute. Excluding the zero scores, the results increase slightly to 39. Students identified 28 letters in Year 1, 44 letters in Year 2 and 41 in Year 3. The greatest improvement in letter recognition was in Year 2, where students identified 16 additional letters. Students in Year 3 scored four points lower than students in Year 2. However, assessors observed that Year 3 students were beginning to provide letter names in English rather than Kiribati. Overall, girls scored higher than boys, 14 The sub-test descriptions are based on the Early Grade Reading Assessment Toolkit (RTI, 2009). 18 identifying 40.3 letters compared to 34.1 for boys. Some girls exceeded the maximum score of 100 by nine points (109). Table 3: Sub-test 1 Letter Name Knowledge: Results by Year and Gender Sub-test 1 – Number of correct letter names N Mean SD Min Max identified per minute (CLPM) CLPM – overall 1,363 37.2 23.2 0 109.1 CLPM – minus zero score students 1,318 38.6 22.5 1 109.1 Year CLPM – overall 483 27.8 20.2 0 102.4 Year 1 CLPM – minus zero score students 457 29.2 19.7 1 102.4 CLPM – overall 458 44.0 22.7 0 107.1 Year 2 CLPM – minus zero score students 448 45.2 21.9 1 107.1 CLPM – overall 422 40.5 23.4 0 109.9 Year 3 CLPM – minus zero score students 413 41.9 22.6 1 109.1 Gender CLPM – overall 691 40.3 24.1 0 109.9 Girls CLPM – minus zero score students 679 41.9 23.2 1 109.9 CLPM – overall 672 34.1 21.8 0 100 Boys CLPM – minus zero score students 648 35.3 21.2 2 100 The highest-ranking district for letter name scores was the Southern region. The Central, Linnux and Northern regions performed similarly. The South Tarawa region had the lowest overall mean score. Table 4: Letter Name Results by Region and Year Central Linnux North South South Tarawa Year 1 32.6 30.9 28.2 28.9 25.8 Year 2 48.8 44.2 48.6 52.7 40.6 Year 3 40.6 42.9 39.1 45.6 39.9 Overall 39.7 39.2 38.4 42.9 35.4 19 Figure 1: Letter Name Results by Region and Year Central Linnux North Year 1 Year 2 South Year 3 South Tarawa 0 10 20 30 40 50 60 Number of correct letters per minute 3.2. Sub-test 2 – Letter Sound Knowledge In order to read fluently, students must understand the alphabetic principle (that words are made up of sounds) and that letters (e.g., graphemes) are symbols that represent sounds. Thus, knowledge of how letters correspond to sounds is a critical skill children must master to become successful readers. The letter sound knowledge test is administered similarly to the letter name knowledge subtest. Students were provided a page of 100 randomly distributed upper- and lowercase letters of the Kiribati alphabet and asked to provide the sounds (not the names) of as many letters as they could identify within a one-minute period. The results of this sub-test by year, gender and with and without zero scores are shown in Table 5. The overall results for the letter sound sub-test indicate that students had more difficulty with identifying letter sounds than letter names. Students were able to identify 27 letter sounds on average (10 fewer than letter names). Year 1 students accurately identified 22 letter sounds. Students in Year 2 could name an additional eight letter sounds, for a total mean score of 30. Similar to the letter name sub-test, scores decreased by two points in Year 3, from 30 letters in Year 2 to 28 in Year 3. Girls read four more letter sounds than boys, but boys achieved a higher maximum score of 95 compared to 85 for girls. Table 5: Sub-test 2 Letter Sound Identification: Results by Year and Gender Sub-test 2 – Number of correct letter sounds N Mean SD Min Max identified per minute (CSPM) CSPM – overall 1,361 26.5 16.9 0 95.1 CSPM – minus zero score students 1,286 28.3 16.1 1 95.1 Year Year 1 CSPM – overall 482 22.4 15.2 0 77 20 CSPM – minus zero score students 443 24.4 14.3 1 77 CSPM – overall 457 29.7 16.7 0 84 Year 2 CSPM – minus zero score students 436 31.2 15.7 1 84 CSPM – overall 422 27.7 18.2 0 95.1 Year 3 CSPM – minus zero score students 407 29.2 17.4 1 95.1 Gender CSPM – overall 689 28.8 17.3 0 85 Girls CSPM – minus zero score students 657 30.4 16.3 1 85 CSPM – overall 672 24.2 16.4 0 95.1 Boys CSPM – minus zero score students 657 30.4 16.3 1 85 Student performance on the letter sound sub-test was related to the type of the letter presented. The percentage of correct answers for each of the letters tested showed that students have a good knowledge of vowel sounds but are unable to correctly identify the sounds of consonants. Sample students were able to correctly identify vowel sounds in over 80 percent of the cases. Meanwhile, performance decreased to 8 percent - 31 percent for consonants, and was particularly low for the letters w, k, t, r and b. Nasal consonants (n, m, ng) were easier to pronounce and were correctly identified in over 70 percent of the cases presented. The Central region scored 3-5 points higher than other regions. Linnux performed the lowest averaging 25 correct letter sounds per minute (See Table 6). Table 6: Letter Sound Identification: Results by Region and Year Central Linnux North South South Tarawa Year 1 28.7 24.3 22.9 20.8 20.6 Year 2 35.6 24.8 31.0 34.4 28.8 Year 3 27.5 27.1 24.7 27.9 28.9 Overall 30.6 25.4 26.2 27.9 26.0 21 Figure 2: Letter Sound Identification: Results by Region and Year Central Linnux North Year 1 Year 2 South Year 3 South Tarawa 0 10 20 30 40 Number of correct letters per minute 3.3. Sub-test 3 – Initial Sound Identification Initial sound identification assesses phonemic awareness (the ability to hear, identify and manipulate individual sounds –phonemes- in spoken words). In this test, the assessor read aloud a word and then asked the student to identify the first sound, or phoneme, in the word. The test is comprised of 10 words and is administered orally. The test is untimed and is measured according to the number of items answered correctly. Overall, students could identify 30 percent of initial sounds attempted (3 out of 10 items). Excluding the zero score students, the average score increases to four. Identifying initial sounds was a particularly difficult task for all students. Students identified an average of 2.3 initial sounds in Year 1, with a slight increase to 3.7 in Year 2, and practically no change (4) in Year 3. There was no difference in performance for boys and girls. Table 7: Sub-test 3 Initial Sound Identification: Results by Year and Gender Sub-test 3 – Number of correct initial sounds N Mean SD Min Max identified Overall 1,363 3.3 2.5 0 10 Minus zero score students 1.033 4.3 2.0 1 10 Year Overall 483 2.3 2.4 0 10 Year 1 Minus zero score students 279 3.6 2.0 1 10 Overall 458 3.7 2.4 0 10 Year 2 Minus zero score students 379 4.4 1.9 1 100 Overall 422 4.0 2.5 0 10 Year 3 Minus zero score students 375 4.7 2.0 1 10 22 Gender Overall 691 3.5 2.6 0 10 Girls Minus zero score students 541 4.5 2.1 1 10 Overall 672 3.0 2.4 0 9 Boys Minus zero score students 492 4.1 1.8 1 9 Based on the results by region in Table 8, there was very little variation in the performance of students across regions. Linnux demonstrated the highest scores on initial letter sound while the Central and Southern regions scored the lowest. 23 Table 8: Initial Sound Recognition: Results by Region and Year Central Linnux North South South Tarawa Year 1 2.2 2.3 2.1 1.4 2.4 Year 2 3.3 4.3 3.7 3.7 3.6 Year 3 3.7 5.2 4.2 4.2 3.6 Overall 2.9 3.9 3.3 3.1 3.2 Figure 3: Initial Sound Recognition: Results by Region and Year Central Linnux North Year 1 Year 2 South Year 3 South Tarawa 0 1 2 3 4 5 6 Number of correct sounds 3.4. Sub-test 4 – Familiar Word Reading The familiar word sub-test measures students’ ability to read familiar words with fluency and accuracy, both of which are necessary to become fluent readers. For the familiar-word reading test, students were given a list of 50 familiar words (selected from readers available in the classroom) with instructions to read as many as they could in one minute. Familiar- word reading is a timed test scored by the number of correct words read per minute (cfwpm). Table 9 shows the results of the familiar word reading sub-test by year, gender and with and without zero scores. The overall mean score was 23 correct words per minute and 27 excluding zero scores. The test was most difficult for Year 1 students who could only read an average of 8.6 words correctly per minute. Performance improves by 15 points in Year 2, in which students read an average of 23.7 words per minute, and another 13 points in Year 3 where the overall mean score was 37 words per minute. While these scores might be too low to lead to reading comprehension, there is steady progress towards reading fluency. Girls scored eight points above boys (27 compared to 19 words per minute for boys). The maximum score for girls was nearly 20 points higher than boys (98 compared to 78.9 respectively). 24 Table 9: Sub-test 4 Familiar Word Reading: Results by Year and Gender Sub-test 4 – Number of correct familiar words read N Mean SD Min Max per minute (CFWPM) CFWPM – overall 1,362 22.7 20.9 0 98 CFWPM – minus zero score students 1,171 26.7 20.1 1 98 Year CFWPM – overall 482 8.6 11.9 0 88.2 Year 1 CFWPM – minus zero score students 338 12.4 12.6 1 88.2 CFWPM – overall 458 23.7 17.8 0 81.7 Year 2 CFWPM – minus zero score students 428 25.3 17.2 1 81.7 CFWPM – overall 422 37.0 21.4 0 98 Year 3 CFWPM – minus zero score students 405 39.8 19.5 1 98 Gender CFWPM – overall 690 26.6 23.2 0 98 Girls CFWPM – minus zero score students 615 30.4 22.4 1 98 CFWPM – overall 672 18.6 17.3 0 78.9 Boys CFWPM – minus zero score students 556 22.6 16.5 1 78.9 On the familiar word reading sub-test, regional averages were highest in the Linnux region and lowest in the Central region (See Table 10). Table 10: Familiar Word Reading: Results by Region and Year (without zero scores) Central Linnux North South South Tarawa Year 1 10.9 13.6 8.7 5.2 7.2 Year 2 23.2 30.8 23.6 26.6 22 Year 3 35.9 41.8 38.9 44.1 34.8 Overall 20.9 28.6 23.2 25.5 21.2 25 Figure 4: Familiar Word Reading: Results by Region and Year (without zero scores) Central Linnux North Year 1 Year 2 South Year 3 South Tarawa 0 10 20 30 40 50 Number of correct words per minute 3.5. Sub-test 5 – Nonword Reading To be able to comprehend unfamiliar text, children must be able to decode new unfamiliar words. Nonword reading measures students’ ability to decode words that follow the linguistic rules of the Kiribati language but that do not exist in real life. Unlike familiar word reading in which students can read from memory or sight recognition, on the nonword reading sub-test students must sound out the words; thus, it is a purer measure of decoding ability. Students were provided with a table of 50 made-up words and instructed to read as many as they could within one-minute. The test is timed and is measured by the number of correct nonwords read per minute (cwpm). Overall, students read an average of 13 nonwords correctly in one minute (See Table 11). When the zero scores are removed, the mean score jumps to 20 correct words per minute. The difference in these scores indicates that there is a significant proportion on non-readers on this sub-test, especially in Year 1. Year 1 readers could read an average of 13 nonwords, but when zero scores are included the mean score drops to 5 correct nonwords per minute (8 point difference). There was a 3-4 point difference between readers and non-readers in Years 2 and 3; thus, the percentage of students able to decode nonwords increases with each level. Overall mean scores do indicate progress from Years 1-3, with performance improving from 5.2 correct words in Year 1 to 14 correct in Year 2 and 22 correct in Year 3. On average, girls read five more words than boys (15.9 and 10.9, respectively), which is also evident in the higher maximum score for girls (49 versus 45). 26 Table 11: Sub-test 5 Nonword Reading: Results by Year and Gender Sub-test 5 – Number of correct nonwords read per N Mean SD Min Max minute (CUWPM) CNWPM – overall 1,362 13.4 12.8 0 49 CNWPM – minus zero score students 951 19.9 10.7 1 49 Year CNWPM – overall 482 5.2 8.7 0 47 Year 1 CNWPM – minus zero score students 209 12.8 9.3 1 47 CNWPM – overall 458 13.9 11.4 0 43 Year 2 CNWPM – minus zero score students 354 18.3 9.5 1 43 CNWPM – overall 422 21.9 12.4 0 49 Year 3 CNWPM – minus zero score students 388 25.1 9.8 1 48 Gender CNWPM – overall 690 15.9 13.8 0 49 Girls CNWPM – minus zero score students 512 21.7 11.7 1 49 CNWPM – overall 672 10.9 11.2 0 45 Boys CNWPM – minus zero score students 439 17.8 8.9 1 45 Based on Table 12 results by region, Linnux region scored the highest with a mean score of 19.1 correct nonwords per minute and the Central region had the lowest average score (11.5 correct nonwords per minute). Table 12: Nonword Reading: Results by Region and Year (without zero scores) Central Linnux North South South Tarawa Year 1 6.1 10.8 5 2.4 4.2 Year 2 13.2 20.3 13.8 16.1 12.4 Year 3 19.1 26.4 23.3 25.1 20.5 Overall 11.5 19.1 13.7 14.7 12.3 27 Figure 5: Nonword Reading: Results by Region and Year (without zero scores) Central Linnux North Year 1 Year 2 South Year 3 South Tarawa 0 5 10 15 20 25 30 Number of correct nonwords per minute 3.6. Sub-test 6 – Oral Passage Reading The best single measure of a child’s reading proficiency in the primary grades is oral reading fluency (ORF). Oral reading fluency measures students’ ability to read a short passage with sufficient speed and accuracy. It encompasses all of the previous reading skills plus the skills needed for comprehension -- the ability to translate letters into sounds, unify sounds into words, process connections, relate text to meaning, and make inferences. 15 ORF has been shown to be a powerful predictor of overall reading competence and comprehension16. The oral passage reading sub-test is a timed test. In order for students to understand a simple passage, they must be able to read it fast enough to retain the words in short-term memory. Research 17 suggests a minimum fluency rate of 45-60 words per minute, depending on the complexity of the language. In this sub-test, students were asked to read a very short story comprised of 60 words in one minute. After one minute, the assessor stopped students and recorded the number of words read correctly (cwpm). If the child could not read any words correctly in the first line, the assessor stopped the test early and the child received a score of zero. The overall mean score for this sub-test was 29 correct words per minute, well below the 45-60 cwpm standard. Among those who could read at least one word (excluding zero scores), students read 32.4 words per minute (See Table 13). While the overall mean score 15 Hasbrouck, J., & Tindal, G. A. (2006). “Oral reading fluency norms: A valuable assessment tool for reading teachers.â€? The Reading Teacher, 59(7), 636–644. 16 Ibid 17 Abadzi, H. (2011). Reading Fluency Measurements in EFA FTI Countries: Outcomes and Improvement Prospects. Education for all Fast Track Initiative 28 was quite low, the bulk of the low scores were in Year 1, where students read an average of 10 correct words per minute. Students improved significantly as they progressed to Year 2, reading 29 words per minute, and Year 3, where students read an impressive 47 words per minute. Students read an additional 18-19 words each year. Girls read 11 more words than boys (34 compared to 23. The maximum score for girls were 129 words per minute while boys read a maximum of 103 words per minute. Table 13: Sub-test 6a Oral Passage Reading: Results by Year and Gender Sub-test 6a – Number of correct words read per N Mean SD Min Max minute (CWPM) CWPM – overall 1,358 28.6 25.4 0 128.6 CWPM – minus zero score students 1,199 32.4 24.6 1 128.6 Year CWPM – overall 478 10.9 13.6 0 100 Year 1 CWPM – minus zero score students 361 14.2 14.0 1 100 CWPM – overall 458 29.2 20.3 0 94.7 Year 2 CWPM – minus zero score students 428 31.2 19.5 1 94.7 CWPM – overall 422 47.2 26.4 0 128.6 Year 3 CWPM – minus zero score students 410 49.6 24.8 1 128.6 Gender CWPM – overall 689 34.0 28.2 0 128.6 Girls CWPM – minus zero score students 629 37.5 27.3 1 128.6 CWPM – overall 669 23.1 20.7 0 102.9 Boys CWPM – minus zero score students 570 26.8 19.9 1 102.9 Average results by region illustrated in Table 14 indicate that the Linnux region had the highest fluency rates (35.7 cwpm) while the Central region (26 cwpm) and South Tarawa region (27 cwpm) had the lowest. Table 14: Oral Passage Reading: Results by Region and Year Central Linnux North South South Tarawa Year 1 13.5 16.7 10.5 7.4 9.7 Year 2 30.1 38 29.6 33.2 26.7 Year 3 42.4 53 50.7 55.2 44.6 Overall 25.9 35.7 29.6 32.1 26.9 29 Figure 6: Oral Passage Reading: Results by Region and Year Central Linnux North Year 1 Year 2 South Year 3 South Tarawa 0 10 20 30 40 50 60 Number of correct words per minute 3.7. Sub-test 7 – Reading Comprehension The reading comprehension subtask measures the ability to answer several comprehension questions based on the passage read. There are a total of five questions for this sub-test, consisting of direct, fact-based questions and at least one question requiring inference from the passage read. Students were asked questions only up to the point where they had stopped reading. For instance, if the child read the first sentence (6 words), s/he was asked one question. If s/he read half of the text (27 words), s/he was asked three questions; and, if s/he read all five sentences (54 words), s/he was asked all five comprehension questions. Similarly, if learners did not read any of the text, they were not asked any questions and received a score of zero. This sub-test is scored by the number of questions answered correctly. The scores are also calculated to determine the percentage of questions answered correctly out of a total possible (five). The overall results are shown in Table 15. On average, students could accurately respond to 1.3 comprehension questions. Readers (excluding zero scores) comprehended the double the text of non-readers achieving 50 percent correct (2.5 out of 5 questions asked). The highest percentage of zero scores were in Year 1, where the overall mean score was 0.2. Students showed significant gains from Years 1-3. Overall mean scores improved by 30 percent from Year 1 to Year 2 with mean scores increasing from 0.2 to 1.5. From Year 2 to Year 3, scores increased nearly 20 percent from 1.5 to 2.4. Among readers (minus zero scores), scores increased an average of 11 percent each year. 30 Table 15: Sub-test 6b Reading Comprehension: Results by Year and Gender Sub-test 6b – Number of questions answered N Mean SD Min Max correctly Number of correct answer – overall 1,363 1.3 1.5 0 5 Number of correct answer – minus zero 757 2.5 1.2 1 5 score students Year Number of correct answer – overall 483 0.2 0.7 0 5 Year 1 Number of correct answer – minus zero 78 1.8 1.1 1 5 score students Number of correct answer – overall 458 1.5 1.4 0 5 Year 2 Number of correct answer – minus zero 310 2.3 1.2 1 5 score students Number of correct answer – overall 422 2.4 1.5 0 5 Year 3 Number of correct answer – minus zero 369 2.9 1.2 1 5 score students Gender Number of correct answer – overall 691 1.6 1.6 0 5 Girls Number of correct answer – minus zero 416 2.7 1.2 1 5 score students Number of correct answer – overall 672 1.1 1.4 0 5 Boys Number of correct answer – minus zero 341 2.3 1.2 1 5 score students Figure 7 below shows the percentage of correct answers by Year. About 85 percent of students in Year 1 showed zero percent comprehension of the passage read. Only one percent of Year 1 students could comprehend 60 percent or more of the text. The results for Year 2 are better as the majority of the students comprehended some of the text and nearly 30 percent scored 60 percent or above. Year 3 students had more distribution across the higher percentage categories. Half of Year 3 students could comprehend 60 percent or more of the text and 27 percent could comprehend at least 80 percent of the text. Overall, the majority of students in Years 1 and 2, and half in Year 3 have reading comprehension levels below 60-80 percent. 31 Figure 7: Reading Comprehension Results: Distribution of percentage of correct answers by year 100% 90% 80% 70% %students 60% 50% 40% 30% 20% 10% 0% 0% 20% 40% 60% 80% 100% Reading comprehension (% correct responses) Class1 Class2 Class3 Although girls read 11 more words in the passage than boys, the point difference between boys and girls’ reading comprehension scores was trivial. Table 15 shows that girls on average correctly responded to 0.5 more questions than boys (1.6 versus 1.1, respectively). However, the distribution of responses by percentage of correct answers illustrated in Figure 8 shows that girls had a higher percentage of correct answers in most categories and a lower percentage of zero scores compared to boys. Figure 8: Reading Comprehension Results: Distribution of percentage of correct answers by gender 60% 50% 40% %students 30% 20% 10% 0% 0% 20% 40% 60% 80% 100% Reading Comprehension (% correct responses) Girls Boys 32 The average results by year and gender according to the type of question asked and the order in which it was presented is provided in Table 16. All five questions were direct factual questions; thus the answers were embedded in the text. The easiest question was Question 1, in which 38 percent of students accurately responded. Questions 2-4 had fairly equal levels of difficulty with correct answers averaging 30 percent. Question 5 was the most difficult with only five percent of students who attempted the question able to answer correctly. Table 16: Correct Answers by Questions in sub-test 6b Questions Year 1 Year 2 Year 3 Total Girl Boy Total Girl Boy Total Girl Boy Total Girl Boy Total 1. Who went 12% 6% 9% 53% 41% 47% 67% 53% 60% 43% 33% 38% fishing with Tooma? 2. Where did 8% 6% 7% 43% 35% 39% 56% 48% 52% 35% 29% 32% Tooma and his father put their fishing line? 3. What kind of 6% 3% 4% 33% 22% 28% 68% 47% 57% 34% 23% 29% fish was Tooma’s catch? 4. How did Tooma 6% 1% 3% 34% 26% 30% 72% 38% 55% 36% 21% 29% feel when he hooked the catch? 5. How did 1% 0% 1% 5% 1% 3% 16% 10% 13% 7% 3% 5% Tooma’s catch get away? The percentage and number of questions students answered correctly based on those attempted are presented in Table 17. The first column shows number and percent correct for the entire sample. Columns 0-5 show the number of questions that students attempted and scored correctly. The results indicate that the more text students read, the more they were able to comprehend. This is evident in the decreasing percentage of zero scores. Of students who did not comprehend any of the text (196), 100 percent scored zero. Meanwhile, of students who read enough to be asked two questions, nearly 50 percent scored zero. Only 7 percent of all students attempted and correctly responded to four of the questions, and 2.4 percent attempted and correctly responded to five questions. 33 Table 17: Distribution of Correct Answers Number of Total Questions correct Number of questions attempted Correct # 0 1 2 3 4 5 0 606 196 184 195 13 13 5 44% 100% 93.4% 48.5% 14.6% 4.9% 2.4% 1 195 13 123 19 26 14 14% 6.6% 30.6% 21.4% 9.7% 6.6% 2 198 84 33 52 29 14% 20.9% 37.1% 19.5% 13.7% 3 181 24 94 63 13% 26.9% 35.2% 29.7% 4 144 82 62 10% 30.7% 29.3% 5 39 39 2% 18.4% Total (N) 1,363 196 197 402 89 267 212 Scores were similarly low on this sub-test across all regions and years. The Linnux region performed slightly better than the rest with a mean comprehension score of 1.7 and the Central and South Tarawa regions had the lowest comprehension scores (1.2). Table 18: Reading Comprehension: Results by Region and Year Central Linnux North South South Tarawa Year 1 0.5 0.3 0.3 0.2 0.2 Year 2 1.6 1.9 1.6 1.6 1.3 Year 3 2.1 2.9 2.6 2.7 2.2 Overall 1.2 1.7 1.4 1.5 1.2 Figure 9: Reading Comprehension: Results by Region and Year Central Linnux North Year 1 Year 2 South Year 3 South Tarawa 0 1 2 3 4 Number of questions answered correctly 34 3.8. Sub-test 8 – Listening Comprehension The purpose of the listening comprehension assessment is to measure whether the student can listen to a short passage being read aloud and then answer several questions correctly with a word or a simple statement. Poor performance on a listening comprehension tool would suggest that children simply do not have the basic vocabulary that the reading materials expect, or that they have difficulty processing what they hear. The assessor read a short fable to students and then asked five comprehension questions. Students had 15 seconds to respond to each question. As this was an untimed test, all students heard the entire story and responded to all five questions. For this reason, scores are based on the percentage of questions answered correctly. The average mean score for listening comprehension was 46 percent correct and 52 percent without zero scores (See Table 19). The results indicate progressive improvement in listening comprehension levels by at least 6-7 percent with each Year. By the end of Year 1, students could comprehend about 40 percent of the passage read aloud. Year 2 students understood 47 percent and Year 3 students comprehended 53 percent. Boys and girls had relatively similar comprehension skills (46 percent for girls and 47 percent for boys). Table 19: Sub-test 7 Listening Comprehension: Results by Year and Gender Sub-test 7 – Percentage of overall listening N Mean SD Min Max comprehension Percentage of correct answer – overall 1,363 46.2% 28.2 0 100 Percentage of correct answer – minus 1,253 51.9% 24.5 20 100 zero score students Year Percentage of correct answer – overall 483 39.1% 27.5 0 100 Year 1 Percentage of correct answer – minus 422 46.7% 23.4 20 100 zero score students Percentage of correct answer – overall 458 46.9% 26.5 0 100 Year 2 Percentage of correct answer – minus 426 51.6% 23.1 20 100 zero score students Percentage of correct answer – overall 422 53.4% 28.8 0 100 Year 3 Percentage of correct answer – minus 405 57.4% 25.7 20 100 zero score students Gender Percentage of correct answer – overall 691 45.8% 27.9 0 100 Girls Percentage of correct answer – minus 633 51.0% 24.5 20 100 zero score students Percentage of correct answer – overall 672 46.7% 28.5 0 100 Boys Percentage of correct answer – minus 620 52.8% 24.4 20 100 zero score students 35 Table 20 shows average performance per question type. All questions were fact-based questions (answers can be found within the story). While all questions were of similar level of difficulty, students performed highest on questions 1 and 3, and lowest on questions 4 and 5. It should be noted that students only heard the passage read aloud once. Table 20: Listening Comprehension Percentage of Correct Answers by Question Questions Year 1 Year 2 Year 3 Total Girl Boy Total Girl Boy Total Girl Boy Total Girl Boy Total 1. Who went 67% 72% 70% 82% 80% 81% 87% 84% 86% 79% 79% 79% with her father to get some fresh air? 2. Where was a 39% 41% 40% 49% 49% 49% 48% 47% 47% 45% 46% 45% bird perched? 3. Why did Anna 51% 63% 57% 65% 61% 63% 71% 72% 71% 62% 65% 64% climb the tree? 4. What was 14% 19% 16% 27% 20% 23% 32% 36% 34% 24% 25% 24% Anna warned by her father about birds? 5. Why then was 13% 12% 13% 21% 16% 18% 26% 31% 28% 20% 19% 20% Anna happy? In terms of regional performance, the highest scores were in the Linnux region where students correctly responded to 3.3 questions and the lowest performance was in South Tarawa where students accurately responded to 2 questions. Table 21: Listening Comprehension Results by Region and Year Central Linnux North South South Tarawa Year 1 2.2 2.8 2 1.6 1.7 Year 2 2.2 3.4 2.6 2.2 2.1 Year 3 3.1 3.7 2.7 2.8 2.4 Overall 2.4 3.3 2.4 2.2 2.0 36 Figure 10: Listening Comprehension Results by Region and Year Central Linnux North Year 1 Year 2 South Year 3 South Tarawa 0 1 2 3 4 Number of questions answered correctly 3.9. Sub-test 9 – Dictation The dictation sub-test measures students’ alphabet knowledge and ability to hear and distinguish individual letter sounds in words and to spell words correctly. For this sub-test, the assessor read aloud a short sentence of 15 words and asked students to write down what they had heard. The assessor read the sentence three times, once before students began writing and twice while they were writing. The test was untimed and scored by the percentage of items written correctly. The dictation sub-test assessed four basic writing skills: spelling, spacing, capitalization and punctuation (use of full stop). The dictation sentence contained 10 words, consisting of a compound sentence that included a compound noun comprised of three morphemes. In pilot testing, the sentence was found to provide a good discrimination of scores between grades. The scores were calculated using weights to arrive at a maximum score of 100 percent. The spelling items received a weight of 60 percent while other components (spacing, capitalization, comma and full stop) received a weight of 40 percent. Given the focus of the EGRA is to assess reading ability, the spelling component received more weight. Students performed exceptionally well on this writing task. The maximum score was 100 percent for all three classes. The average score for the entire sample was 71 percent and 72 percent excluding zero scores. Students’ writing skills progressively increased from 51 percent in Year 1 to 76 percent in Year 2 to finally 82 percent in Year 3. Girls scored six percentage points higher than boys (74 percent compared to 68 percent). 37 Table 22: Sub-test 9 Dictation: Results by Year and Gender Sub-test 9 – Percentage of overall early writing N Mean SD Min Max skills) Percent Correct Average percent of correct answer – 1,165 70.5% 26.4% 0% 100% overall Average percent of correct answer – 1,153 71.7% 25.0% 2.1% 100% minus zero score students Year Average percent of correct answer – 332 50.5% 27.5% 0% 100% overall Year 1 Average percent of correct answer – 323 52.8% 25.9% 2.1% 100% minus zero score students Average percent of correct answer – 425 76.2% 20.4% 0% 100% overall Year 2 Average percent of correct answer – 424 76.3% 20.2% 4.2% 100% minus zero score students Average percent of correct answer – 408 82.1% 20.8% 0% 100% overall Year 3 Average percent of correct answer – 406 82.8% 19.3% 4.2% 100% minus zero score students Gender Average percent of correct answer – 608 74.2% 25.7% 0% 100% overall Girls Average percent of correct answer – 602 75.6% 23.8% 2.1% 100% minus zero score students Average percent of correct answer – 557 66.6% 26.7% 0% 100% overall Boys Average percent of correct answer – 551 67.6% 25.6% 4.2% 100% minus zero score students As presented in Figure 11 and mentioned above, the dictation test measured spelling, spacing, capitalization and full stop. Students scored highest on spelling. The greatest improvements were with use of the full stop between Years 1 and 2 (an increase of 26 percent). However, Year 3 students struggled the most with use of the full stop scoring zero 38 correct. Girls scored an average of 12 percent higher than boys across all writing categories (Figure 12). Figure 11: Dictation Results: Percentage of Correct Answers by Item and Year 100% 90% Percentage of correct answers 80% 70% 60% 50% 40% 30% 20% 10% 0% Spelling Spacing Capital Full stop Writing skills Year 1 Year 2 Year 3 Figure 12: Dictation Results: Distribution of correct answers by gender 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Spelling Spacing Capital Full stop Girls Boys 39 Regional results for the dictation sub-test are presented in Table 23. Students fared well across all regions with average scores between 71 percent and 77 percent correct. The Southern region scored the highest with 78 percent correct. The lowest scoring region was South Tarawa where students achieved 68 percent correct. Table 23: Dictation: Results by Region and Year (without zero scores) Central Linnux North South South Tarawa Year 1 59.5 46.4 56.3 52.9 48 Year 2 83.1 80.8 74.4 80.1 74.2 Year 3 85.5 85.6 85.3 89 78.9 Overall 74.5 71.4 74.2 77.7 67.8 Figure 13: Dictation: Results by Region and Year (without zero scores) Central Linnux North Year 1 Year 2 South Year 3 South Tarawa 0 20 40 60 80 100 Dictation precent correct score 3.10. Summary of assessment results The overall EGRA results show that there is significant progress from Years 1 to 3 and that students have basic reading and writing skills. Students possess good letter recognition, familiar word reading and oral passage reading skills. There was measurable progress between Years 1-3 across all sub-tests, especially on the familiar word and oral reading fluency sub-tests. Scores on the familiar word subtest increased by 15 cwpm between Years 1-2 (from 9 to 24 words per minute), and an additional 13 between Years 2-3 to an average of 37 correct words per minute. On the oral reading passage sub-test, students’ scores improved by 18 points between Years 1 and 2 (from 10.9 to 29) and Years 2 and 3 (from 29 to 47). Thus, students in Year 3 are on the cusp of becoming fluent readers. However, students are still struggling with identifying letter sounds and decoding unfamiliar words. 40 Listening comprehension skills were higher than reading comprehension for Years 1 and 3, but at similar levels for Year 3. The reading comprehension sub-test had the highest percentage of zero scores across all years as well as the lowest mean scores (See Figures 16 and 17). The highest percentage of zero scores was in Year 1, where 84 percent of students could not accurately respond to one comprehension question. Students did, however, show significant improvement as they progressed throughout the levels. By Year 3, students read an average of 47 correct words per minute and could accurately respond to 47 percent of the comprehension questions asked (See Figures 14 and 15). With regards to gender, there were more female readers than male readers, and girls read equal to or better than boys in all reading skills tested. The Linnux region scored highest on six of eight reading sub-tests and lowest on Letter Sound Knowledge. Conversely, the Central Region scored lowest across five sub-tests and highest (along with the Southern region) on Letter Sound Knowledge. Figure 14: Summary Results: Number correct answers for timed sub-tests by year 50 Number of correct answers per minute 45 40 35 30 25 20 15 Year 1 10 Year 2 5 Year 3 0 Letter Familiar Oral Letter Name Sound Nonword Word Passage Knowledge Identificatio Reading Reading Reading n Year 1 27.8 22.4 8.6 5.2 10.9 Year 2 44 29.7 23.7 13.9 29.2 Year 3 40.5 27.7 37 21.9 47.2 41 Figure 15: Summary Results: Percent correct for untimed sub-tests by year 60% 50% Percentage of correct answer 40% 30% Year 1 Year 2 20% Year 3 10% 0% Reading Comprehension Listening Comprehension Year 1 5.0% 39.1% Year 2 29.4% 46.9% Year 3 47.3% 53.4% Figure 16: Summary Results: Percentage of Zero Scores by Sub-test and Year 90% 80% 70% Percent of Zero Scores 60% 50% 40% 30% 20% 10% 0% Oral Reading Listening Letter Initial Letter Familiar Nonword Passage Compreh Compreh Dictation Name Sound Sounds Word Reading ension ension Year 1 5% 42% 8% 30% 57% 24% 84% 13% 3% Year 2 2% 17% 5% 7% 23% 7% 32% 7% 0% Year 3 2% 11% 4% 4% 8% 3% 13% 4% 1% 42 Figure 17: Summary Results: Percentage of Zero Scores by Sub-test and Gender Dictation Listening Comprehension Reading Comprehension Oral Passage Reading Nonword Familiar Word Letter Sounds Initial Sound Letter Name 0% 10% 20% 30% 40% 50% 60% Boys Girls 43 Chapter 4 – Performance in reading fluency and reading comprehension According to the second edition of the EGRA toolkit18, there are two steps to identifying a reading fluency benchmark. The first is to identify the level of reading comprehension that is expected for the grade level. In most countries, this level is set at 80 percent or higher (4 correct answer out of 5 questions) for Years 1-3. This threshold will be used for the purposes of this EGRA study. Once the reading comprehension benchmark is set, the second step is to use EGRA data to show the range of ORF scores obtained by students able to achieve the desired level of comprehension. Because students may be able to achieve the 80 percent comprehension benchmark at different fluency rates, and there could be a wide range of scores, we have added a third step to the analysis. The third step is to identify the range of scores with the highest proportion of students meeting the comprehension benchmark. With this information, then stakeholders may decide on the value within the fluency range that should be put forward as the reading fluency benchmark. Table 24 shows the actual distribution of percent correct scores for reading comprehension. Overall, 11 percent of students scored 80 percent correct and 3 percent achieved 100 percent. Thus, the total percentage of students achieving 80 percent or more reading comprehension is 14 percent. Table 24: Distribution of percent correct scores for reading comprehension by sub-test per year Comp. Score Year 1 Year 2 Year 3 Overall Mean 0% 84% 32% 13% 44% 20% 8% 21% 14% 14% 40% 5% 19% 21% 15% 60% 1% 15% 26% 13% 80% 1% 11% 20% 11% 100% 1% 1% 7% 3% Total 100% 100% 100% 100% As mentioned above, as a first step, we decided to set the level of comprehension at 80 percent as an indicator that students demonstrate full understanding of the text. Table 25 shows the percentage of students by year and gender that comprehended 80 percent or more of the text read. The distribution of scores by Year is: 2 percent in Year 1, 12 percent in Year 2 and 27 percent for Year 3 students. In terms of gender, 16 percent of female students and 10 percent of male students reached the 80 percent benchmark in reading comprehension. 18 RTI International. (2016). Early Grade Reading Assessment (EGRA) Toolkit, Second Edition. Washington, DC: United States Agency for International Development. 44 Table 25: Percent of students reading with 80% or more comprehension by Year 80% or more reading comprehension Mean (%) SD (%) N Overall 14% 34.1% 1363 Year 1 2% 13.6% 483 Year 2 13% 33.3% 458 Year 3 27% 44.7% 422 Female 16% 37.1% 691 Male 10% 30.3% 672 For the second step, we need to look at the distribution of fluency scores of students who reached the threshold level in reading comprehension. Figure 18 shows the distribution of ORF scores for each level of reading comprehension. Globally, we can see that an increase in the level of reading comprehension is associated with an increase in ORF scores. Table 26 confirms that mean scores in ORF were higher when students had higher levels of reading comprehension. However, it also shows that some students who did not reach the 80 percent reading comprehension benchmark achieved higher ORF scores than students who did reach the benchmark. For example, students who have 80 percent in reading comprehension could have a fluency score as low as 25 correct words per minute. Meanwhile, some students who were unable to answer a single reading comprehension question correctly could read up to 6519 correct words per minute. Figure 18: ORF distribution by reading comprehension level 19 Note that ORF scores may exceed the total number of words in a passage since it is a timed test. If students could read the entire passage within the 60 seconds allowed, the fluency score would take into account the time remaining from the 60 seconds. This explains why some students present such high value as 129 correct words per minute although the passage had only 60 words. 45 Table 26: ORF score by reading comprehension score Reading Mean Fluency SD Min Max Sample (n) Comp. Score Score 0% 11.9 8.8 1 65 442 20% 29.5 17.4 5 109 195 40% 39.5 18.7 9 129 198 60% 53.3 15.0 16 100 181 80% 59.9 16.6 25 120 144 100% 72.5 15.4 46 129 39 Distribution of ORF scores for students who reach 80 percent in reading comprehension Figure 19 shows the distribution of ORF scores for the 183 students who reached the level of 80 percent correct in reading comprehension. The majority of the students who met the benchmark read between 40 and 69 correct words per minute. However, scores ranged between 40 and 129 words per minute. Scores above 60 indicate that students finished the paragraph in less than one minute while scores below 60 imply that students who did not finish the paragraph were able to infer enough to correctly respond to all questions. Figure 19: Distribution of ORF scores for students reading with at least 80% comprehension The histogram in Figure 20 compares the distribution of ORF scores for students who did and did not meet the reading comprehension benchmark. As illustrated, the number of 46 students who reached the benchmark is quite small compared to those who did not and there is overlap between the distributions of the two groups. The benchmark set for reading fluency should ensure that a maximum number of students who did reach the 80 percent level in reading comprehension would be identified as fluent readers but it should also ensure not to include too many students who did not reach 80 percent in reading comprehension. This means that setting the benchmark value for fluency at 45 correct words per minute could include too many students who didn’t reach the 80 percent reading comprehension level while setting it at 75 correct words per minute would exclude too many students who reached the benchmark. Figure 20: Distribution of ORF scores for students meeting and not meeting 80% benchmark Table 27 shows that the fluency rate of 60 cwpm has a greater proportion of students meeting rather than not meeting the benchmark. Policymakers may decide on an acceptable range (e.g., 60-64 cwpm) that can be considered proficient or acceptable. If the benchmark for reading comprehension were lowered to 60 percent, then the fluency scores would also be lower (closer to 50 cwpm). Table 27: Distribution of ORF Scores for students meeting and not meeting the 80% benchmark Correct number of % meeting 80% % not meeting words per minute RC benchmark 80% RC (ORF) benchmark 45-49 36.7% 63.3% 47 50-54 27.9% 72.1% 55-59 43.1% 56.9% 60-64 57.4% 42.6% 65-69 47.1% 52.9% 70-74 36.8% 63.2% 75 and more 53.4% 46.6% In summary, greater oral reading fluency is associated with higher levels of reading comprehension with only 14 percent of students who are identified as being able to comprehend 80 percent of what they read. More females than males reached that level of comprehension. ORF scores for students achieving the benchmark shows great variability. Stakeholders should discuss the number of correct words per minute that could qualify a student as a fluent reader. The decision should be based on the distribution of scores for students who reached the reading comprehension benchmark. 48 Chapter 5- Analysis of Student and Teacher Factors Associated with Better Reading Outcomes This section reports the results of regression analyses conducted to explore the effect of student and teacher factors (independent variables) on oral reading fluency scores (dependent variable). The factors are organized into seven categories: 1) student demographics; 2) teacher characteristics; 3) teacher expectations; 4) classroom environment; 5) teacher training and guides; 6) reading materials; and, 7) teacher instructional and assessment methods. The results of the analyses directly respond to research question #7. 5.1. Association of student characteristics to student reading outcomes General background characteristics and reading activities were collected in the student questionnaire. The factors as shown in Table 28 include whether the student attended preschool, ate breakfast before arriving to school, language spoken at home, parents’ literacy, whether students receive help with homework, availability of reading materials and whether students read or are read to at home. Over 90 percent of students surveyed attended pre-school, speak Kiribati at home, eat breakfast before arriving to school and state that they like to read. Eighty-five percent of students read at home. About half of students have reading materials in Kiribati. Table 28: Student background characteristics Student Characteristics % of cases SE* N Student attends preschool before Year 1 94% 0.7% 1336 Student speaks Kiribati at home 96% 0.9% 1361 Student eats before arriving to school 92% 0.9% 1360 Student has someone who can read at home 82% 0.01% 1361 Student receives help with homework from the 25% 1.8% 1361 mother Student receives help with homework from the 17% 1.5% 1361 father Student receives help with homework from the 15.4% 1.1% 1361 sibling Student receives help with homework from any 5.5% 0.9% 1361 other person Someone asks student about what he/she did in 75% 1.3% 1316 school Student tells someone at home when he/she gets 77% 1.5 % 1344 good marks Student has books, newspapers or other things to 59% 2.2% 1335 read at home Student has books, newspapers or other things to 48% 1.3% 1361 read at home in Kiribati Student has books, newspapers or other things to 14% 0.9% 1361 read at home in English 49 Someone reads to student at home 59% 1.4% 1361 Student reads aloud to someone at home 42% 2.6% 1361 Student reads to himself/herself at home 85% 1.0% 1361 Student reads on a computer or mobile device at 69% 2.3% 731 home Student likes to read 96% 0.6% 1348 20 *SE = Standard Error The two factors that had the greatest positive impact on ORF scores (with an increase of 7 to 10 correct words per minute) were: whether students speak Kiribati at home and whether the student likes to read. Two additional factors that resulted in an increase of about 5 correct words per minute were: whether students received help with their homework from the mother and whether students read to himself at home. Parents’ literacy presented a positive association with ORF scores. When both parents are literate, students read an additional 8.15 words per minute. Students who had a literate father read almost twice as many words as those with a literate mother (3.86 more words compared to 1.99 more words). However, students who received help with their homework from their mother read 5 more words and the difference was statistically significant while the help received from their father or sibling was not significant. Table 29: Association of student characteristics to Oral Reading Fluency (ORF) scores Student Characteristics Change in SE Rsqr ORF Score (+/-) Student attends preschool before Year 1 4.37 3.82 0.40 Student eats breakfast before arriving to school -1.11 2.92 0.39 Student speaks Kiribati at home 10.38* 2.26 0.39 Student has a literate mother 1.99 2.24 0.39 Student has a literate father 3.86 2.03 0.39 Student has both parents literate 8.15* 3.78 0.40 Student has a literate sibling 1.10 1..47 0.39 Student receives help with homework from 5.07* 2.36 0.40 the mother Student receives help with homework from 2.31 1.71 0.39 father Student receives help with homework from -2.53 2.16 0.39 sibling Someone asks student about what he/she did in 1.65 1.83 0.39 school Student tells someone at home when he/she gets 0.21 1.45 0.39 good marks Student has books, newspapers or other things to 0.47 1.31 0.39 read at home 20 SE is a measure of uncertainty caused by sampling error. It is used to build a confidence interval around the sample mean 50 Student has books, newspapers or other things to 1.60 0.93 0.39 read at home in Kiribati Student has books, newspapers or other things to 3.84 2.25 0.39 read at home in English Student reads aloud to someone at home -3.29 1.65 0.39 Student reads to himself/herself at home 4.93* 2.25 0.40 Someone reads to student at home -1.79 1.12 0.39 Student reads on a computer or mobile device at -0.94 1.25 0.39 home Student likes to read 7.64* 2.86 0.40 5.2. Association of teacher characteristics to student reading performance The association between teachers’ characteristics and student reading performance on the ORF sub-test was explored using data collected in the teacher interview questionnaire. Teacher profiles are presented in Table 30. Teachers in Kiribati have an average of 15 years of experience. Half of their experience seems to be on the island they are currently teaching. Nearly all primary school teachers (90 percent) are female and hold a primary school teaching certificate. Seventy-seven percent of teachers have a reading corner in the classroom and 74 percent have met with their students’ parents. Finally, less than half of teachers have not been absent from school in the last term. Table 30: Profile of teachers in EGRA Teacher demographics Mean (sd) N Age of the teacher 43.4 (9.5) 132 Number of years of teaching experience 15.4 (9.1) 132 Number of years spent teaching on the island 7.4 (7.7) 132 Number of minutes from home to school 8.8 (11.9) 132 Teacher characteristics % of cases N Female 90% 132 Has a primary teaching certificate 98% 132 Has a reading corner in the classroom 77% 130 Has not been absent from school in the last term 43% 131 Has met with parents of his/her students 74% 132 In order to identify the teacher characteristics associated with better student reading outcomes, separate regression analysis was conducted for each teacher characteristic (See Table 31). The dependent variable was the average number of words read per minute in the oral reading passage (scores from sub-test 6a) and the independent variables were the teacher characteristics described above (Table 30). Gender and year level were considered in the regression analysis to control for these two variables. The results in Table 31 indicate that no factor had a statistically significant association with ORF scores. Whether teachers held a primary school teaching certificate or had a reading corner showed the greatest effect but those two factors are not statistically significant. 51 Table 31: Association of teachers' characteristics to Oral Reading Fluency Score (ORF) Teacher Characteristics Change in SE Rsqr ORF Score (+/-) Has a primary teaching certificate 2.88 6.78 0.74 Has a reading corner in the classroom 3.41 1.94 0.74 Has not been absent from school in the last term 0.81 1.66 0.74 Has met with the parents of his/her students 1.04 1.74 0.74 Age of the teacher -0.08 0.09 0.74 Number of year of experience in teaching -0.01 0.10 0.74 Number of year of experience in teaching on 0.03 0.11 0.74 this island Number of minute from home to school -0.11 0.07 0.74 5.3. Association of teacher expectations to student reading performance Teachers were asked to report what grades they expected students to meet certain reading competencies (e.g., recognize letters, understand stories read aloud). Table 32 provides results of regression analyses that show the relationship of five indicators of teacher expectations on student oral reading fluency scores (number of words read correctly per minute). Each regression analysis included gender and year as control variables. The reference level for each regression is the median expectation of all teachers for each indicator. The dash (-) represents the expected year for that competency to be reached. For example, Year 1 is the reference category in the first regression because the median expectation of teachers for students to recognize and say letter names was by the end of Year 1. The positive coefficient of 8.48 in Year 2 indicates that students whose teachers expected them to recognize and say letter names by Year 2, a year later than the median expected time, read 8.48 more words relative to students whose teachers expected them to achieve the competency by Year 1. This pattern was also observed in the second regression. Most teachers expected their students to understand stories read aloud by the teacher by the end of Year 2. Teachers who expected their students to achieve this competency by the end of Year 3 read 3.74 more words per minute, and for those with expectations for Year 4 students, the ORF score increased by 1.82 words per minute. When teachers had higher expectations than the median teachers, reading performance decreased. This was observed for the second indicator. Students were expected to read aloud to the teacher by the end of Year 2. Teachers who expected students to achieve these competencies by the end of Year 1 saw a decrease of 2.72 correct words per minute. 52 Table 32: Association of teacher expectations to oral reading fluency (ORF) scores Teacher expectations of student Change in SE Rsqr reading performance ORF score (+/-) Grade students should recognize and say letter names Year 1 - - 0.77 Year 2 8.48* 1.96 Year 3 2.19 2.40 Year 4 3.91 4.12 Grade students should understand stories read aloud by the teacher in the classroom Year 1 -2.72 2.01 0.75 Year 2 - - Year 3 3.74 2.10 Year 4 1.82 3.29 Grade students should sound out words using phonics Year 1 -0.56 1.72 0.78 Year 2 - - Year 3 6.43* 2.14 Year 4 14.25* 4.13 Grade students should read aloud to teacher and other students Year 1 -3.61 2.72 0.76 Year 2 -0.47 2.08 Year 3 - - Year 4 5.86* 2.40 Grade students should understand simple texts that they read in class Year 1 -4.40 2.60 0.75 Year 2 1.09 1.99 Year 3 - - Year 4 4.00 2.69 Based on the results in Table 32, students whose teachers expected them to reach the reading competency later than expected consistently read more words per minute than students who met the competency as expected. The coefficient was statistically significant when teacher expected students to recognize and say letter names in Year 2. In this situation, students were able to read 8 more correct words per minute. It is also the case for when students should sound out words using phonics or when students should read aloud to teacher and other students. Teachers who expected students to sound out words using phonics in Year 4 read 14 more cwpm, and teachers who expected students to read aloud to teacher and students in Year 4 read 5.86 more cwpm. However, teachers who expected students to reach the competency sooner than the expected grade level actually read fewer correct words per minute. When teachers expected students to read aloud in Year 1, students read 3.6 fewer cwpm. Furthermore, when teachers expected students to understand simple texts they read in class in Year 1, they read 4.4 fewer cwpm. 53 5.4. Association of teacher training and guides to student reading performance We also sought to explore the relationship between teaching materials and student reading performance. Teachers were asked whether they had a teaching syllabus and whether they had received any training on how to teach reading in the last three years. As shown in Table 33, over 60 percent of teachers had a syllabus. Eighty-nine percent of teachers reported having a teacher guide. Finally, about half of teachers have received training on reading instruction. Table 33: Teacher training and guides % of cases N Teacher has a syllabus 64% 132 Teacher has a teacher guide 89% 132 Teacher have receive training on how to teach reading in the 56% 132 last three years Results of the regression analysis for teaching resources and its relation to student reading performance, measured by a change in ORF score are presented in Table 34. Whether the teacher had a syllabus or a teacher guide did not have any effect on ORF scores. Training also did not show statistically significant association with ORF scores. Table 34: Association of training and guides to student oral reading fluency (ORF) scores Change in SE Rsqr ORF score Teacher have a syllabus -1.72 2.29 0.74 Teacher has a teacher guide -2.91 2.63 0.74 Teacher have receive training on how to teach 1.85 1.63 0.74 reading in the last three years 5.5. Association of Classroom Environment to Student Performance Through classroom observations, we collected data on the classroom environment and types of resources available in the classroom. Assessors recorded whether they observed the following classroom displays: the alphabet, a calendar, days of the week, numbers and the weather. Classrooms observed had an average of 3.58 classroom displays. The observation also sought to determine the types of printed materials used in instruction, such as newspapers, magazines, flashcards, food wrappers and packaging, prepaid cards, objects in treasure boxes and any other materials. There was an average of 1.10 printed material used in the classroom. Table 35: Average Number of Classroom Displays and Materials Observed Classroom environment Mean (sd) SE N Classroom displays 3.58 0.10 155 Print materials used in instruction 1.10 0.09 155 54 As shown in Table 36, 67 percent of classrooms have spelling/vocabulary words displayed, 15 percent have songs/hymns/stories written on the blackboard and 54 percent have them written on charts or posters, and 69 percent have student work displayed. Additional factors related to the classroom environment that were observed included whether there was sufficient space for organizing group activities, whether there was a reading corner in the classroom, whether teachers maintained folders with students’ work (student profiles) and the seating arrangement. The results showed that 75 percent of classrooms have a reading corner, 81 percent have sufficient space for organizing group work, 68 percent have student profiles and 77 percent of classroom students sit on the floor. Table 36: Frequency and Type of Classroom Displays/Resources Available Types of classroom displays/resources available % of classes SE N Spelling/vocabulary displayed 67% 0.04 155 Song/hymns/stories displayed on blackboard 15% 0.03 155 Song/hymns/stories displayed on charts/posters 54% 0.04 155 Student work displayed 69% 0.04 155 Sufficient classroom space for organized group 81% 0.03 155 activities Reading corner in the classroom 75% 0.03 155 Student profiles (folder with student work and student 68% 0.04 155 info) Students sitting on floor 77% 0.04 155 Among the different classroom environment variables, the presence of student profiles had the most positive relation with ORF scores. When teachers maintained student profiles, the students read an average of 6.37 more words per minute. This relation is statistically significant. On the other hand, having student work displayed was adversely related to ORF scores. Classrooms where student work was displayed had lower overall ORF scores. Table 37: Association of Classroom Environment to Student ORF Scores Classroom Environment Change in SE Rsqr ORF Score (+/-) Classroom displays 0.38 0.54 0.75 Spelling/vocabulary displayed 1.10 1.56 0.75 Song/hymns/stories displayed on blackboard 3.51 2.04 0.75 Song/hymns/stories displayed on charts/posters 0.08 1.50 0.75 Student work displayed -3.28* 1.55 0.76 Print materials used in instruction 0.88 0.63 0.75 Sufficient classroom space for organized group -2.23 1.89 0.75 activities Reading corner in the classroom 2.92 1.71 0.76 Student profiles (folder with student work and 6.37* 1.53 0.78 student info) Students sitting on floor 0.75 1.76 55 5.6. Association of Reading Materials to Student Performance The classroom observation further aimed to identify teaching and learning aids used for reading instruction. The instrument included 13 different reading instructional materials that could be observed in the classroom (Instructional readers, big books, posters/charts with poems, posters/charts with songs, posters/charts with phonics, A5 letter cards, small cards, mathematics charts, community studies posters, environmental science posters, healthy living charts, climate change story book, and any other reading materials). The assessors reported a mean of 8.26 different instructional resources used in the classrooms. Additionally, there was an average of 1.97 reading books observed during the lesson. Table 38:Average Number of Reading Materials and Books in Classroom Reading Materials Mean (sd) SE N Reading instructional materials in classroom 8.26 (2.80) 0.23 152 Number of reading books used during the lesson 1.97 (17.5) 1.44 148 According to the regression results in Table 39, the relation between the number of reading books used during the lesson and ORF scores of students is statistically significant. When there are more books used in the classroom, the ORF scores were higher. Table 39: Association of Reading Materials to Student ORF Scores Change in SE Rsqr ORF Score (+/-) Reading materials in classroom 0.43 0.26 0.74 Number of reading books used during the 0.11* 0.04 0.74 lesson 5.7. Association of Teacher Instructional and Assessment Methods to Student Performance The final set of regression analysis observed the relationship between instructional and assessment methods and student performance in ORF. Table 41 presents results of regressions analysis of the frequency in which students and teachers performed seventeen instructional and assessment methods within the course of the week. The dependent variable is the mean ORF score of the students and the independent variables are the instructional methods. The mean ORF score of students who were never exposed to an instructional or assessment method was compared to the mean ORF score of students who were exposed 1-2 days, 3-4 days or 5 days a week. Results from those regression analyses were analyzed to determine if being exposed to a specific method is associated with increased or decreased ORF scores and if so, whether the effect is the same for all rates of exposure (e.g., 1-2 days, daily, etc.). All regression models include gender and Year for controlling for those two characteristics. Table 40 shows the teaching methods used during reading instruction and the frequency in which it is applied. The instructional methods used most frequently were teaching the meaning of new words/vocabulary and listening comprehension. The most common 56 classroom activities were shared reading, reading comprehension activities, drawing or writing sentences. Less than 10% of teachers evaluate students’ oral reading fluency on a regular basis (3-5 days a week). Table 40: Frequency of methods used during reading instruction Never 1–2 3–4 5 days N days days Teaching of Listening 5% 30% 19% 46% 132 Comprehension Children Practice Letter Name 20% 11% 9% 61% 132 Children orally retell a story that 18% 42% 25% 15% 132 they have read Children learn new letter sounds 18% 17% 15% 50% 132 Children sound out unfamiliar 52% 23% 11% 14% 132 words using knowledge of letter sounds Children learning meanings of new 11% 20% 33% 36% 132 words/vocabulary Shared reading 7% 29% 39% 26% 132 Group Guided reading 19% 48% 20% 13% 132 Listening to a child read aloud 30% 45% 14% 11% 132 Students readings on their own 27% 14% 7% 52% 132 silently Reading comprehension activities 12% 24% 30% 33% 132 Children take books home to read 92% 4% 2% 2% 132 with their parents Evaluating student’s oral reading 57% 36% 2% 5% 132 with running records or any other method Teachers works on word building 22% 23% 20% 35% 132 with students Students read and draw 13% 20% 31% 36% 132 Students working on spelling 20% 32% 20% 28% 132 words in exercise books Students writing sentences 10% 24% 23% 43% 132 Table 41: Association of teacher instructional and assessment methods to student performance Teaching instructional and assessment Change in SE Rsqr methods ORF score Teaching of Listening Comprehension 0.74 1-2 days 3.52 3.86 3-4 days 4.57 3.99 5 days 2.87 3.74 Teaching Letter Names 0.75 1-2 days -2.12 3.08 3-4 days -5.62 3.36 57 5 days -6.44* 2.50 Asking children to orally retell a story 0.74 that they have read 1-2 days 0.74 2.29 3-4 days -1.97 2.52 5 days -0.85 2.82 Teaching new letter sounds 0.77 1-2 days -10.10* 2.68 3-4 days -10.16* 2.81 5 days -6.9* 2.36 Asking children to sound out 0.74 unfamiliar words using knowledge of letter sounds 1-2 days -0.32 2.00 3-4 days 0.58 2.74 5 days -4.41 2.47 Teaching meaning of new vocabulary 0.74 words 1-2 days 0.27 3.08 3-4 days -1.27 2.82 5 days -0.88 2.78 Shared reading 0.74 1-2 days 2.57 3.48 3-4 days 2.61 3.39 Daily 3.01 3.59 Group Guided reading 0.74 1-2 days 5.20* 2.20 3-4 days 4.29 2.56 Daily 2.75 2.92 Listening to a child read aloud 0.74 1-2 days 3.94 1.88 3-4 days -0.31 2.71 Daily 3.55 2.83 Students readings on their own silently 0.74 1-2 days 2.35 2.65 3-4 days 4.36 3.46 Daily 3.69 1.93 Reading comprehension activities 0.74 1-2 days 5.10 2.83 3-4 days 4.71 2.74 Daily 2.62 2.70 58 Children take books home to read with 0.74 their parents 1-2 days -1.27 4.29 3-4 days -3.32 6.69 Daily 0.97 5.50 Evaluating student’s oral reading with 0.74 running records or any other method 1-2 days 0.66 1.74 3-4 days -4.24 5.51 Daily -1.57 3.76 Teacher works on word building with 0.74 students 1-2 days -2.25 2.41 3-4 days -3.75 2.54 Daily -3.36 2.30 Students read and draw 0.74 1-2 days -2.45 2.90 3-4 days -3.29 2.66 Daily -5.48* 2.63 Students work on spelling words in 0.74 exercise books 1-2 days 6.14* 2.25 3-4 days -0.01 2.48 Daily 1.31 2.33 Students writing sentences 0.74 1-2 days -0.83 3.09 3-4 days 2.19 3.10 Daily 0.14 2.87 Results of the regression analysis presented in Table 41 show that some instructional methods used in reading classroom are associated with higher levels of ORF scores. Reading comprehension activities were linked with higher scores between 2-5 words per minute. When students do group guided reading 1-2 days a week, they were able to read an average of 5.20 more words. Also, when students work on spelling words in exercise books 1-2 days a week, they were able to read an average of 6.14 more words. On the other hand, several teaching methods show a negative association in students’ ORF scores. Students who learned new letters sounds 1-4 days a week read an average of 10 fewer words while students who were taught letter sounds on a daily basis read 6.9 fewer words. Additionally, when students practiced letter names, read or drew 1-5 days week, 59 there were lower ORF scores. Evaluating students’ oral reading skills three or more days a week showed lower scores compared to evaluating once or twice a week. This could be because less time was spent on actual teaching. 60 Chapter 6 - Conclusions and Next Steps The KiEGRA study shows that students are making significant gains in literacy between Years 1-3, but more improvement is needed for them to become fluent readers. Results indicate that while students have good letter recognition, familiar word reading and oral passage reading skills, they struggle with identifying letter sounds and decoding unfamiliar words. As a result, the majority of students are unable to read fluently with comprehension. The majority of students in Year 1 and one-third in Year 2 have zero reading comprehension skills. In Year 3, comprehension improves where nearly half of students could comprehend at least 60 percent of text. However, the majority of students in Years 1-3 are reading below the 80 percent comprehension benchmark. Overall, 14 percent met the benchmark. In Year 3, about 80 percent of students did not meet the benchmark. An analysis of the relationship between oral reading fluency and reading comprehension scores to determine fluency ranges and percentage of students reaching the 80 percent reading comprehension benchmark found that greater oral reading fluency is associated with higher levels of reading comprehension. However, there is great variability in fluency ranges that fall within the 80 percent benchmark. Scores ranged from 25 to 129 words per minute. Most students meeting the benchmark read 40-69 correct words per minute. Only 14 percent of students across all years were able to comprehend 80 percent of what they read. Across all sub-tests, girls tended to perform equal to or better than boys. More females than males met the 80 percent reading comprehension benchmark. Students in the Linnux region performed better than other regions. The analysis identified a number of student characteristics, instructional methods, resources and teacher expectations that are associated with better reading outcomes. Students who speak Kiribati at home, have literate parents, and liked to read scored an average of 7-10 more words on the ORF sub-test. Teaching reading comprehension seemed to have a positive influence on students’ ORF scores. Guided reading was also found to be very important for increasing students’ fluency rates. When teachers maintained records of students’ work, ORF scores increased by 6 words per minute. Not surprisingly the availability of reading materials in the classroom or used in teaching instruction was positively related to ORF scores. Interestingly, students tended to have higher ORF scores when teachers had low expectations for achievement of reading competencies. This could be because teachers provided more time for practice or reviewed lower-level skills with students. Teachers with high expectations yielded a small decrease in ORF scores. This could be because they spent less time reviewing lessons. In this analysis teacher expectations were measured according to whether they were higher or lower than the median expectation of teachers. In the future, teacher expectations should be measured against curricular standards. Remarkably, teaching letter sounds and letter names and assessment methods were negatively related to student ORF scores. One reason could be if teachers are not closely 61 following the teacher guide. Overemphasizing decoding skills or assessment and not utilizing results to inform instruction can hamper student performance. More research is necessary to identify the true cause behind these results. Based on the findings and conclusions, the following recommendations are presented to improve the quality of early grade reading instruction in Kiribati schools: 1. Set reading fluency and comprehension benchmarks to guide and support teacher professional development to reach reading targets: Reading benchmarks bring attention to the learning expectations both students and teachers should be held accountable to support children in becoming readers. They serve a useful function when they are coupled with strategies that will strengthen the capacity of schools and teachers to help students reach them. Without benchmarks, there is no pressure in the system to achieve learning outcomes and no fear of failure. At the same time, simply setting standards and benchmarks without evidence that schools and teachers are failing in their job and the support mechanisms to improve, benchmarks will create fear and frustration among school administrators, teachers and parents. Reading benchmarks should be established to guide the development of basic reading skills in each of the early grades both in terms of oral reading fluency and comprehension; and to guide the mechanisms that are required for schools to succeed in achieving them. As a first step, stakeholders should decide on the level of comprehension required to understand grade level text (e.g., 60 percent, 80 percent) and then review the fluency scores that fall within that range. Tonga, for instance, has set the reading fluency benchmark at 50 cwpm and a reading comprehension of at least 75%. Students achieving these levels can be considered to have learned to read well, have the basic reading skills needed to develop their literacy skills, and have the ability to comprehend more complex text in upper grades. If stakeholders agree with the 80 percent benchmark, then an acceptable fluency range may be 60-64 cwpm. Given that the current mean ORF score is 29 cwpm, that range may seem too high. Thus, policymakers may decide to lower the benchmark to 60 percent and the fluency benchmark to 53 cwpm. Once the benchmark is decided, the next step is to consider the targets. Currently, 14 percent of students are meeting the 80 percent reading comprehension benchmark. How many should meet the benchmark in one year or five years? Once the targets are set up, schools and educators should be supported to achieve these benchmarks. The Kiribati MEYS should then conduct a national reading campaign to sensitize all stakeholders of the new benchmarks and then monitor and report progress towards achieving the targets at all levels (national/regional MEYS, school and communities). 2. Conduct a follow-up study on teaching practices and assessment methods to identify strengths and weaknesses. Although 56 percent of teachers were trained on reading instruction in the past three years, teaching decoding skills and assessment were associated with lower ORF scores. It is important to understand why these interventions have an adverse relationship. It could be that teachers were not adequately supported to master the new teaching methodologies or that training 62 is not aligned with best practices in reading instruction. Additionally, teachers may not understand how to accurately assess students and utilize the results for reflection and lesson planning. The analysis also showed that classrooms with teachers who maintained a record of students’ work had higher ORF scores. A follow-up study is recommended to investigate the strengths and weaknesses of current assessment and teaching methods vis-à-vis the results observed in different classrooms. It is also important to understand what types of information the teacher records and whether it is related to tracking student progress. 3. Strengthen teacher training for reading instruction with a focus on vocabulary, decoding skills and reading comprehension. Results indicate that students are weak in recognizing letter sounds and unfamiliar words, and have low reading comprehension skills. Hence, instruction in these areas should be strengthened to increase overall reading scores. In the case of Kiribati, the large shares of zero scores observed in various reading subtasks included in KiEGRA suggest inadequate instructional time in the classroom due to student absenteeism, school closures, or even inadequate use of instructional time -i.e. pedagogy or content-- during regular school days. Additionally, training should incorporate curriculum expectations and provide teachers with the specific methods, classroom activities and assessment methods required to achieve results. 4. Increase the number of books in the classroom library/reading corner and the use in teachers’ instruction. In Kiribati, 75 percent of classrooms have reading corners, but only 1.97 books are used during classroom instruction. One possible explanation is that there are not enough books in the classroom. The International Reading Association (IRA) recommends that classroom libraries start with at least seven books per child and purchase two additional new books per year. The optimal number of books in a classroom library is 300-600, depending on the grade level and number of copies21. The number of books teachers should expect children to read during the school year is 100-125 picture books by the end of Year 1 and 50- 75 chapter books by the end of Year 2. There is strong evidence that reading skills grow and develop to the extent that children get actual practice reading. This is particularly difficult in countries with a modest literary tradition where the amount of titles in the local language may be limited. Recent innovations in technology now enable the production of reading materials in almost any language, provided there are guidelines for avid writers to follow. Educators in Kiribati have experience producing additional reading materials for children and their efforts should be further supported to ensure beginning readers have a reasonable variety of stories to practice and enjoy. With a corpus of reading large enough to provide sufficient reading practice, a low-cost option is to provide e-readers in low-resource countries where publishing may be expensive and there are high teacher-pupil ratios. E- readers allow students and teachers to choose from a variety of genres, it is portable so students can read from home or school, and its read aloud features provides 21 Neuman, S. (undated). The importance of the classroom library. Available at: http://teacher.scholastic.com/products/paperbacks/downloads/library.pdf 63 additional support for emergent readers2223. In addition to provision of an increased number of hard and soft copy books, teachers should be trained on how to better integrate materials into their instruction and on how to develop attractive reading corners24. 5. Increase time-spent reading. The more time children spend reading the better and more fluent readers they become. In order to increase students’ reading fluency skills, teachers should ensure that students are spending sufficient time reading every day through teacher-led, parent-led or self-guided reading activities. Research recommends that children read between 20-40 minutes per day. The results also showed that bringing books home on a regular basis was associated with lower ORF scores. It is important to engage parents in reading activities to ensure that students and parents are spending the recommended time reading at home. 6. Address the issue of non-readers in Years 1-2. Stakeholders should conduct classroom level assessments to identify non-readers in Years 1 and 2, diagnose the causes, and design specific activities to address deficiencies before children complete Year 2. Teachers in Years 1 and 2 have the greatest responsibility in ensuring children finish the first two grades of primary education with the necessary skills to read and understand text. In this sense, identification of struggling readers and knowledge of ways in which they need to be supported should make the core material of in-service training for Year 1 and Year 2 teachers. For instance, teachers may group students according to ability and provide remedial activities and appropriately levelled text. This “catch-up approachâ€? is being used by UNICEF in Zambia based J-Pal’s research in India, which demonstrated that students grouped by ability is more effective than mixed-ability grouping. UNICEF will assess students in all grades and group them according to reading levels (non-readers, those who can read letter sounds, syllables, words, passages, etc.). Grouping students by level rather than grades has produced dramatic results in India, Kenya and Ghana. Teachers and school administrators should further determine whether non-readers have learning disabilities (e.g., dyslexia) and design relevant interventions for special needs students. 7. Teach students how to develop reading comprehension skills in the early grades. Given only 14 percent of students are able to comprehend 80 percent or more of grade level text, teachers should focus on increasing reading comprehension levels for all years. Research shows that teachers are often the ones to ask questions orally and that students are expected to respond, but rarely are students asked to develop their own questions as they read (think aloud) or to make 22 Adams, A. & van der Gaag, J. (2011). First Step to Literacy: Getting Books in the Hands of Children Available at: https://www.brookings.edu/research/first-step-to-literacy-getting-books-in-the-hands-of- children/ 23 UNESCO (2014). Reading in the mobile era: A study of mobile reading in developing countries. http://unesdoc.unesco.org/images/0022/002274/227436E.pdf 24 Neuman, S. (undated). The importance of the classroom library. 64 predictions and then check their predictions. Students need to be empowered to make meaning from text and to use an inquiry-based approach to reading. Teachers could ask students to make predictions before, during and after reading, to retell stories, or to identify problems within the story and provide possible solutions. Teachers should use various strategies and frameworks to develop students’ reading comprehension skills from as early as kindergarten or Year 125. 8. Develop activities that specifically focus on raising boys’ performance and interest in reading. The results illustrated that boys consistently performed lower than girls. There may be cultural or gender barriers that affect boys’ interest and engagement in reading activities. In addition to designing strategies to address low competencies of boys and girls, stakeholders at all levels should discuss the potential challenges specific to boys and design strategies to improve boys’ reading achievement. Successful strategies that have worked in other countries include developing gender-sensitive materials that attract boys’ attention (such as sports, science fiction, fantasy, comic books, digital text, and stories that are humorous) and integrating reading into extracurricular activities (e.g., sports, health clubs, student government). 25 Sample reading comprehension activities can be found in the following guides: Ontario Ministry of Education. (2003). A Guide to Effective Instruction in Reading: Kindergarten to Grade 3 (Available at: http://eworkshop.on.ca/edu/resources/guides/Reading_K_3_English.pdf) Institute of Education Sciences (IES). (2010). Improving Reading Comprehension in Kindergarten through 3 rd Grade. What Works Clearninghouse (Available at: https://education.ohio.gov/getattachment/Topics/Early-Learning/Third- Grade-Reading-Guarantee/Third-Grade-Reading-Guarantee-Teacher-Resources/Improving-Reading- Comprehension-in-Kindergarten-Through-3rd-Grade.pdf.aspx) 65 ANNEX 1 / TABLES Annex 1.A: Reading skills tested Annex 1.B: Correlations between tasks Annex 1.C: 95 percent Confidence interval for EGRA Task Annex 1.D: Tuvalu Regression with Reading Comprehension as Outcome Annex 1.A: Reading Skills Tested Test Measured Timed Letter Name Knowledge Ability to read alphabet letters with accuracy and Y fluency Letter Sound Knowledge Phonics – the ability to identify sounds of letters with Y accuracy and fluency Initial Sound Phonemic awareness – the ability to identify sounds in N Identification spoken words Familiar Word Reading Ability to read familiar words with fluency and Y accuracy Nonword Reading Ability to decode linguistically sound invented words Y Oral Passage Reading Ability to read a short passage with fluency and Y accuracy Reading Comprehension Ability to respond to several comprehension questions N based on passage Listening Comprehension Ability to comprehend a short story read aloud N Dictation Alphabet knowledge, listening and writing skills N 66 Annex 1.B: Correlations between tasks Letters Correct Phonemic Sounds Correct Correct words Correct Oral Reading Reading Listening Dictation Per Minute awareness Per Minute Per Minute Invented Words Fluency Comprehension Comprehension (D) (LCPM) (PA) (SCPM) (CWPM) Per Minute (ORF) (RC) (LC) (CIWPM) LCPM 1 PA 0.483** 1 SCPM 0.688** 0.407** 1 CWPM 0.527** 0.572** 0.384** 1 CIWPM 0.477** 0.573** 0.355** 0.910** 1 ORF 0.508** 0.555** 0.359** 0.943** 0.896** 1 RC 0.455** 0.519** 0.294** 0.782** 0.735** 0.819** 1 LC 0.233** 0.331** 0.144** 0.333** 0.329** 0.306** 0.354** 1 D 0.427** 0.533** 0.306** 0.656** 0.636** 0.670** 0.607** 0.249** 1 67 Annex 1.C: 95 percent Confidence interval for EGRA Task 95 percent Confidence Interval for mean at Task 1: Letter Name Knowledge by Year and Gender 95% Confidence Interval Lower Upper bound Bound Overall 35.2 39.3 Minus zero score students 36.7 40.6 Year 1 Overall 24.9 30.7 Minus zero score students 26.4 32.1 Year 2 Overall 41.0 47.1 Minus zero score students 41.9 48.5 Year 3 Overall 36.8 44.2 Minus zero score students 38.5 45.4 Girls Overall 37.6 43.1 Minus zero score students 39.5 44.3 Boys Overall 31.8 36.3 Minus zero score students 33.2 37.4 95 percent Confidence Interval for mean at Task 2: Letter Sound Identification by Year and Gender 95% Confidence Interval Lower Upper bound Bound Overall 25.2 27.8 Minus zero score students 26.9 29.6 Year 1 Overall 20.7 24.1 Minus zero score students 22.6 26.3 Year 2 Overall 27.9 31.5 Minus zero score students 29.3 33.1 Year 3 Overall 25.2 30.3 Minus zero score students 26.5 31.9 Girls Overall 26.5 31.0 Minus zero score students 24.7 27.4 Boys Overall 22.7 25.8 Minus zero score students 28.2 32.6 68 95 percent Confidence Interval for mean at Task 3: Initial Sound Recognition by Year and Gender 95% Confidence Interval Lower Upper bound Bound Overall 3.1 3.5 Minus zero score students 4.1 4.5 Year 1 Overall 2.0 2.5 Minus zero score students 3.4 3.9 Year 2 Overall 3.4 4.0 Minus zero score students 4.2 4.7 Year 3 Overall 3.6 4.4 Minus zero score students 4.4 5.0 Girls Overall 3.3 3.7 Minus zero score students 4.3 4.7 Boys Overall 2.8 3.2 Minus zero score students 3.9 4.3 95 percent Confidence Interval for mean at Task 4: Familiar Word Reading by Year and Gender 95% Confidence Interval Lower Upper bound Bound Overall 20.2 25.1 Minus zero score students 24.5 28.9 Year 1 Overall 6.7 10.4 Minus zero score students 10.4 14.4 Year 2 Overall 20.3 27.1 Minus zero score students 21.6 28.9 Year 3 Overall 32.9 41.1 Minus zero score students 37.0 42.7 Girls Overall 23.9 29.4 Minus zero score students 20.5 24.8 Boys Overall 16.1 21.1 Minus zero score students 27.4 33.3 69 95 percent Confidence Interval for mean at Task 5: NonWord Reading by Year and Gender 95% Confidence Interval Lower Upper bound Bound Overall 11.8 15.0 Minus zero score students 18.8 21.1 Year 1 Overall 3.9 6.5 Minus zero score students 11.4 14.1 Year 2 Overall 11.5 16.2 Minus zero score students 16.3 20.4 Year 3 Overall 19.9 24.0 Minus zero score students 23.9 26.2 Girls Overall 14.1 17.7 Minus zero score students 19.9 23.4 Boys Overall 9.4 12.4 Minus zero score students 16.8 18.9 95 percent Confidence Interval for mean at Task6a: Oral Passage Reading by Year and Gender 95% Confidence Interval Lower Upper bound Bound Overall 25.8 31.5 Minus zero score students 29.7 35.1 Year 1 Overall 8.8 13.1 Minus zero score students 11.8 16.6 Year 2 Overall 25.5 32.9 Minus zero score students 27.5 35.0 Year 3 Overall 42.4 52.1 Minus zero score students 45.9 53.3 Girls Overall 31.0 37.1 Minus zero score students 34.5 40.6 Boys Overall 20.2 26.0 Minus zero score students 23.9 29.8 95 percent Confidence Interval for mean of number of items answered correctly at Task6b: Reading Comprehension by Year and Gender 70 95% Confidence Interval Lower Upper bound Bound Overall 1.18 1.47 Minus zero score students 2.45 2.63 Year 1 Overall 0.17 0.32 Minus zero score students 1.47 2.06 Year 2 Overall 1.21 1.73 Minus zero score students 2.15 2.46 Year 3 Overall 2.12 2.61 Minus zero score students 2.75 3.00 Girls Overall 1.40 1.71 Minus zero score students 2.54 2.83 Boys Overall 0.92 1.27 Minus zero score students 2.23 2.46 95 percent Confidence Interval for mean percentage of correct answer at Task7: Listening Comprehension by Year and Gender 95% Confidence Interval Lower Upper bound Bound Overall 43.2 49.3 Minus zero score students 49.2 54.3 Year 1 Overall 35.5 42.6 Minus zero score students 43.6 49.7 Year 2 Overall 43.3 50.6 Minus zero score students 48.6 54.5 Year 3 Overall 49.5 57.3 Minus zero score students 53.8 60.9 Girls Overall 42.0 49.5 Minus zero score students 48.0 54.0 Boys Overall 43.1 50.3 Minus zero score students 50.0 55.6 95 percent Confidence Interval for mean at Task8: Dictation by Year and Gender 71 95% Confidence Interval Lower Upper bound Bound Overall 67.8% 73.2% Minus zero score students 69.2% 74.3% Year 1 Overall 45.6% 55.4% Minus zero score students 48.3% 57.3% Year 2 Overall 73.1% 79.2% Minus zero score students 73.2% 79.4% Year 3 Overall 78.7% 85.4% Minus zero score students 79.4% 86.2% Girls Overall 71.9% 76.5% Minus zero score students 73.7% 77.4% Boys Overall 62.7% 70.5% Minus zero score students 63.5% 71.7% Annex 1.D: Tuvalu – Regression Analysis with Reading Comprehension as Outcome Table 42: Impact of student characteristics on Oral Reading Fluency (ORF) scores Student Characteristics Change in SE Rsqr RC score (+/- ) Student attends preschool before Year 1 0.41* 0.10 0.36 Student eats breakfast before arriving to school -0.16 0.16 0.34 Student speaks Kiribati at home 0.71* 0.12 0.34 Student has a literate mother 0.18 0.13 0.34 Student receives help with homework from the 0.25* 0.10 0.34 mother Student receives help with homework from father 0.13 0.08 0.34 Student receives help with homework from -0.03 0.14 0.34 sibling Someone asks student about what he/she did in 0.08 0.11 0.34 school Student tells someone at home when he/she gets -0.05 0.08 0.34 good marks Student has books, newspapers or other things to -0.03 0.06 0.34 read at home Student has books, newspapers or other things to 0.05 0.05 0.34 read at home in Kiribati 72 Student has books, newspapers or other things to 0.26* 0.12 0.34 read at home in English Student reads aloud to someone at home -0.21* 0.09 0.34 Student reads to himself/herself at home 0.38* 0.09 0.35 Someone reads to student at home -0.11 0.08 0.34 Student reads on a computer or mobile device at -0.07 0.07 0.34 home Student likes to read 0.51* 0.15 0.35 Table 43: Effect of teachers' characteristics on Oral Reading Fluency Score (ORF) Teacher Characteristics Change in SE Rsqr RC Score (+/- ) Has a primary teaching certificate 0.34 0.47 0.65 Has a reading corner in the classroom 0.21 0.13 0.65 Has not been absent from school in the last term 0.09 0.11 0.65 Has met with the parents of his/her students 0.10 0.12 0.65 Age of the teacher -0.01 0.01 0.66 Number of year of experience in teaching -0.01 0.01 0.65 Number of year of experience in teaching on 0.001 0.01 0.65 this island Number of minute from home to school -0.01 0.01 0.65 Table 44: Effect of teacher expectations on oral reading fluency (ORF) scores Teacher expectations of student Change in SE Rsqr reading performance RC score (+/- ) Grade students should recognize and say letter names Year 1 - - 0.68 Year 2 0.39* 0.14 Year 3 0.05 0.17 Year 4 0.09 0.30 Grade students should understand stories read aloud by the teacher in the classroom Year 1 -0.15 0.14 0.66 Year 2 - - Year 3 -0.01 0.14 Year 4 0.14 0.23 Grade students should sound out words using phonics Year 1 -0.09 0.13 0.67 Year 2 - - Year 3 0.21 0.16 Year 4 0.42 0.30 Grade students should read aloud to teacher and other students 73 Year 1 -0.19 0.19 0.66 Year 2 0.06 0.15 Year 3 - Year 4 0.19 0.17 Grade students should understand simple texts that they read in class Year 1 -0.25 0.18 0.67 Year 2 0.11 0.14 Year 3 - - Year 4 -0.07 0.19 Table 45: Effect of training and guides on student oral reading fluency (ORF) scores Change in RC SE Rsqr score Teacher have a syllabus 0.18 0.23 0.65 Teacher has a teacher guide 0.03 0.18 0.65 Teacher have receive training on how to teach 0.31* 0.11 0.66 reading in the last three years Table 46: Effect of Classroom Environment on Student ORF Scores Classroom Environment Change in SE Rsqr RC Score (+/- ) Classroom displays 0.04 0.04 0.71 Spelling/vocabulary displayed 0.01 0.06 0.71 Song/hymns/stories displayed on blackboard 0.15 0.10 0.71 Song/hymns/stories displayed on charts/posters 0.08 0.06 0.71 Student work displayed 0.02 0.06 0.71 Print materials used in instruction 0.02 0.13 0.71 Sufficient classroom space for organized group 0.07 0.04 0.71 activities Reading corner in the classroom 0.21 0.11 0.71 Student profiles (folder with student work and 0.38* 0.10 0.73 student info) Students sitting on floor 0.10 0.11 0.71 Table 47: Effect of Reading Instructional Resources on Student ORF Scores Language use in classroom Change in RC SE Rsqr Score (+/-) Reading materials in classroom 0.03 0.01 0.73 Number of reading books used during the 0.01 0.002 0.73 lesson 74 Table 48: Effect of teacher instructional and assessment methods on student performance Teaching instructional and assessment Change in SE Rsqr methods RC score Teaching of Listening Comprehension 0.66 1-2 days 0.07 0.27 3-4 days 0.22 0.28 5 days 0.10 0.26 Teaching Letter Names 0.66 1-2 days 0.18 0.22 3-4 days -0.04 0.24 5 days -0.10 0.18 Asking children to orally retell a story 0.67 that they have read 1-2 days -0.04 0.15 3-4 days -0.36* 0.17* 5 days -0.23 0.19 Teaching new letter sounds 0.67 1-2 days -0.44* 0.19* 3-4 days -0.46* 0.20* 5 days -0.24 0.16 Asking children to sound out 0.67 unfamiliar words using knowledge of letter sounds 1-2 days -0.05 0.14 3-4 days 0.30 0.19 5 days -0.18 0.17 Teaching meaning of new vocabulary 0.66 words 1-2 days -0.12 0.21 3-4 days -0.24 0.19 5 days -0.17 0.19 Shared reading 0.67 1-2 days 0.11 0.24 3-4 days 0.27 0.23 Daily 0.36 0.24 Group Guided reading 0.67 1-2 days 0.28 0.15 3-4 days 0.16 0.18 Daily -0.0001 0.20 75 Listening to a child read aloud 0.67 1-2 days 0.20 0.13 3-4 days 0.44* 0.19 Daily 0.19 0.20 Students readings on their own silently 0.66 1-2 days -0.05 0.19 3-4 days 0.09 0.24 Daily 0.04 0.14 Reading comprehension activities 0.67 1-2 days 0.36 0.19 3-4 days 0.43* 0.19 Daily 0.23 0.19 Children take books home to read with 0.66 their parents 1-2 days 0.24 0.30 3-4 days -0.36 0.46 Daily 0.07 0.38 Evaluating student’s oral reading with 0.66 running records or any other method 1-2 days 0.12 0.12 3-4 days -0.32 0.38 Daily -0.20 0.26 Teacher works on word building with 0.66 students 1-2 days 0.12 0.12 3-4 days -0.32 0.38 Daily -0.20 0.26 Students read and draw 0.66 1-2 days -0.07 0.17 3-4 days -0.01 0.18 Daily -0.16 0.16 Students work on spelling words in 0.68 exercise books 1-2 days -0.31 0.20 3-4 days -0.20 0.18 Daily -0.36* 0.18 Students writing sentences 0.67 1-2 days 0.35* 0.16 3-4 days -0.02 0.17 Daily -0.003 0.16 76 77 ANNEX 2 / INSTRUMENTS Annex 2.A: EGRA Instrument Annex 2.B: Student Questionnaire Annex 2.C: Head Teacher Questionnaire Annex 2.D: Teacher Questionnaire Annex 2.E: Classroom Observation 78 Annex 2.A: EGRA Instrument Early Grade Reading Assessment: Student Response Form Administrator Instructions and Protocol General Instructions It is important to establish a playful and relaxed rapport with the children to be assessed, via some simple initial conversation among topics of interest to the child (see example below). The child should perceive the following assessment almost as a game to be enjoyed rather than a severe situation. It is important to read ONLY the sections in boxes aloud slowly and clearly. Mauri n te ingaabong aei. Arau bon _____________ ao I maeka i _______________, ao n na kabwarabwaraai riki teutana nakoim. [Taekin baika ko kan kaakaraoi n aron te nakonako, kanam, ke am man ae ko taatangiria.] Ao ngkoe ko kona ni kabwarabwarako teutana nakoiu? 1. Teraa te bwai ae ko kan kaakaraoia ngkana ko aki reirei? [Wait for response; if student is reluctant, ask question 2, but if they seem comfortable continue to verbal consent]. 2. Baikara am takaakaro aika e maamate nanom iai? Verbal Consent • Bukin menau n te tabo aei bwa I mwakuri ma te Botaki n Reirei. Ti rang kan karekea ootara n aron reken te wareware iroun te teei. • Ti rang n tangira am ibuobuoki. Ma ngkana ko aki tauraoi ao akea te kaangaanga. • Ti nang takaakaro ngkai i aon te wareware. N na butiiko bwa ko na wareki manin te koroboki, taeka ao te karaki ae e uareereke ni kabuuburaa bwanaam. • N na kabongana te ‘tablet’ aei n taua maanin am wareware. • Tiaki te ukeuke aio ae e na rootaki iai am bwi n te reirei. • Iai naba au titiraki ibukin am utuu, n aron te taetae are kam kakabonganaa ma kaain am utuu, ao tabeua aia bwai aika iai irouia. • N na kaokia riki, akea te kangaanga ngkana ko aki tauraoi ni ira te bwai ae e karaoaki. Ngkana ti waaki ao iai te titiraki ae ko aki kan kaekaa ao akea naba te kaangaanga iai. Iai riki am titiraki? Ko tauraoi ni waaki? Check box if verbal consent is obtained: YES (If verbal consent is not obtained, thank the child and move on to the next child, using this same form) A. Date of assessment G. Which years/grades are in the class? â—‹ â—‹ â—‹ 1 2 3 â—‹ â—‹5 â—‹6 4 B. Enumerator’s name H. Student’s grade â—‹ â—‹ 1st Grade 2nd Grade â—‹ 3rd Grade C. Name of school I. Student’s age â—‹6 â—‹7 â—‹8 â—‹ 9 â—‹ 10 â—‹ 11 â—‹ 12+ D. Island of school J. Student’s gender â—‹ Female â—‹ Male E. Unique School code K. Class name or section F. Child first name L. Time Started: _____ :_____ am / pm 79 Early Grade Reading Assessment: Student Response Form MOD 1 Module 1. Letter Name Knowledge Show the child the sheet of letters in the student stimuli booklet. Say: Noora te beeba aio, e onrake ni manin ara Taetae ni Kiribati. N na butiiko bwa ko na tuangai ARAIA maan aika ko kona ni warekii mai ikai. Tiaki TANGIIA maan bwa ARAIA. Aio te katootoo, aran te man aei [Point to the first example letter] bon ‘m’. Ti na kataneiai moa. Tuangai aran te man aei. [Point to the second example letter] [If the child responds correctly] E tikiraoi, aran te ma aei bon ‘wi.’ [If the child responds incorrectly] Aran te man aei bon ‘wi’. Ti na kataa riki teuana. Tuangai aran te man aei. [Point to the third example letter] [If the child responds correctly] E tikiraoi, aran te ma aei bon ‘u’ [If the child responds incorrectly] Aran te man aei bon ‘u’. Ngkana I kaangai, ‘e nako’ ao ko a moanna naba ikai [Point to the first letter in the table] ni karokoa banen tokin te rain n te taibora [Point to the last letter in the table]. Kotei taian man teuana i mwiin teuana ao takaarua n tuangai araia. Wareware raoi ma ni kawitiiko. Ngkana ko roko n te man ae ko aki ataia ao manga nakon te man are i mwiina. Kotea te moan man n tabonibaim. Ko a tauraoi? Ti nang waaki. Start the timer when the child reads the first letter. Follow along with your pencil and clearly mark any incorrect letters with a slash ( / ). Count self-corrections as correct. If you’ve already marked the self-corrected letter as incorrect, circle the letter and go on. Stay quiet, except when providing answers as follows: if the child hesitates for 3 seconds, provide the name of the letter, point to the next letter and say “Please go on.â€? Mark the letter you provide to the child as incorrect. If the student gives you the letter sound, rather than the name, provide the letter name and say: [“Please tell me the NAME of the letterâ€?]. This prompt may be given only once during the exercise. AFTER 60 SECONDS SAY, “stop.â€? Mark the final letter read with a bracket ( ] ). Early stop rule: If the child does not give a single correct response on the first line, say “Thank you!â€?, discontinue this exercise, check the box at the bottom, and go on to the next exercise. Example: m w U 1 2 3 4 5 6 7 8 9 10 O k A I e n w U R b (10) M T ng K a Ng i E B r (20) t W N u O m T b e w (30) k U R n M I o ng A E (40) W m O t u Ng i N B a (50) r K o w M T k e ng I (60) U A R n b O E t a u (70) Ng B K r N W i m B T (80) m A I o ng E w n E u (90) K t a k a r i A N i (100) Time remaining on stopwatch at completion (number of SECONDS) : Tick this box if the exercise was discontinued because the child had no correct answers in the first line. E tikiraoi am kekeiaki! Ti a manga nakon are i mwiina. 80 81 82 83 Early Grade Reading Assessment: Student Response Form MOD 5 Module 5. Invented Word Decoding Show the child the sheet of invented words in the student stimuli booklet. Say: Aikai tabeua taeka aika a karioaki n te Taetae ni Kiribati. Taiaoka ni wareki ake ko kona ni wareki. Ko na AKI TIBEERINGIIA te taeka ma ko na WAREKIA. Te katootoo, aio te taeka ae e karioaki [point to the first example word] “tanguâ€?. Ti a karaoa moa te katootoo. Taiaoka ma wareka te taeka ae e karioaki aei. [Point to the second example word] [If the child responds correctly] E tikiraoi, te taeka ae karioaki aio bon “nireâ€?. [If the child does not respond correctly] Te taeka ae karioaki aio bon “nireâ€?. Ti a noora riki aio. Taiaoka ma wareka te kariotaeka aio. [Point to the third example word] [If the child responds correctly] E tikiraoi, te taeka ae karioaki aio bon “keimâ€?. [If the child does not respond correctly] Te taeka ae karioaki aio bon “keimâ€?. Ngkana I kaangai, ‘e nako’ ao ko a moanna naba mai ikai [point to the first word] ao ko a warekia nako naba [point to the last word]. Kotekotei taian taeka aikai ao ko warekii n takaarua. Warekia raoi ni kawiitiiko are ko kona ni warekia. Ngkana arona bwa ko roko n te taeka ae ko aki ataia, ao wareka are i mwiina. Katokaa tabonibaim n te moan taeka. Ko a tauraoi? Ti nang waaki. Start the timer when the child reads the first word. Follow along with your pencil and clearly mark any incorrect words with a slash ( / ). Count self-corrections as correct. If you’ve already marked the self-corrected word as incorrect, circle the word and go on. Stay quiet, except when providing answers as follows: if the child hesitates for 3 seconds, provide the word, point to the next word and say “Please go on.â€? Mark the word you provide to the child as incorrect. AFTER 60 SECONDS SAY, “stop.â€? Mark the final word read with a bracket ( ] ). Early stop rule: If the child does not give a single correct response on the first line, say “Thank you!â€?, discontinue this exercise, check the box at the bottom, and go on to the next exercise. Example: tangu nire keim 1 2 3 4 5 toru ei ako kira imi (5) naku tam om kin noro (10) oroai urom eie araaka otamo (15) nam miao ongu auri erero (20) konka eoneke ngita bengam oie (25) kei amaro bwarim noa mibu (30) iobo ngoba koai onge mwei (35) kari engai omwei tomo aema (40) muou wima aria binanai natarei (45) nomo mwaua kunati arong karaarito (50) Time remaining on stopwatch at completion (number of SECONDS not used) : Tick this box if the exercise was discontinued because the child had no correct answers in the first line. E tikiraoi am kekeiaki! Ti a manga nakon are i mwiina. 84 Early Grade Reading Assessment: Student Response Form MOD 6-7 Module 6. Oral Passage Reading Module 7. Oral Passage Comprehension Show the child the story in the student stimuli booklet. Say: When 60 seconds are up or if the child finishes reading the passage in less than 60 seconds, REMOVE the passage from in front of the child, and ask the Ko na wareka te karaki ae e uareereke aei nakoiu ni kabuubura first question below. bwanaam ma ni kawiitiitiiko. I mwiin am wareware ao N na titirakiniko tabeua titiraki. Ngkana I kaangai, ‘E nako’ ao ko a wareka naba te Give the child at most 15 seconds to answer the question, mark the child’s karaki anne. Ngkana ko kaaitibo ma te taeka ae ko aki ataia, ao response, and move to the next question. nakon are i mwiina. Katokaa tabonibaim n te moan taeka. Read the questions for each line up to the bracket showing where the child Ko a tauraoi? Ti nang waaki. stopped reading Start the timer when the child reads the first word. Follow along with your pencil and clearly mark any incorrect words with a slash (/). Count self- corrections as correct. Stay quiet, unless the child hesitates for 3 seconds, in which case provide the word, point to the next word and say “Please go on.â€? Mark the word you provide to the child as incorrect. AFTER 60 SECONDS SAY, “stop.â€? Mark the final word read with a bracket ( ] ). Early stop rule: If the child does not give a single correct response on the Ngkai N nang titirakiniko tabeua titiraki ibukin te karaki are ko a tia ni warekia. first line, say “Thank you!â€?, discontinue this exercise, check the box at the Kataia kaekai titiraki aikai nakon am kabanea ni konaa. Ko kona ni kaekai bottom, and go on to the next exercise. titiraki aikai n am taetae are ko bon tangiria. Correct Incorrect No Response N te bong teuana ao a iangoa te akawa Tooma ma tamana. (12) Antai ae e nako n akawa ma Tooma? [Tamana; an papa; papana; daddy; ana karo] A kabwakaa waaia, a katokai aia bwai n akawa (21) A katikui iaa aia bwai n akawaTooma ma tamana? [I aon te waa / i nanon te waa] ao a bweenako. Inanon aia tai n akawa ao e a teke Tera konan Tooma te ika ae e teke? konan Tooma te ati ae e buubura. (40) [Te ati ae e buubura / Te ati] E rootia te kukurei (44) Tera ana namakin Tooma ngke e teke konana? [E kukurei] ao e kabaitia n katikia. E karina n moti ana ao, E kanga ni bwaka konan Tooma? ao e a birinako konana. (60) [E moti ana ao.] Time remaining on stopwatch at completion (number of SECONDS) : Check this box if the exercise was discontinued because the child had no correct answers in the first line. E tikiraoi am kekeiaki! Ti a manga nakon are i mwiina. 85 Early Grade Reading Assessment: Student Response Form MOD 8 Module 8. Listening Comprehension This is NOT a timed exercise and THERE IS NO STUDENT SHEET. Read the following passage aloud to the child ONLY ONE TIME, slowly (about 1 word per second). Then allow 15 seconds for each question. Say: N na wareka te karaki ae e uarereke aei nakoim ao I a manga titirakiniko tabeua titiraki. Taiaoka kakaauongo raoi ao ni kaekai titiraki ake ko kona ni kaekai. Ko kona ni kaekai titiraki n am taetae are ko bon tangiria. Ko a tauraoi? Ao ti a waaki. N te bong teuana ao a nako n kaukiang Anna ma tamana n te atimwakoro. A noori mannikiba aika a mwaiti i aon tenaan mwaanga ni kai. E tamwarakea te aroka Anna bwa e na rawea ana man ma e aki reke bwa e kibanako. E taku tamana, “Anna, e bon taburoroko raweaia maan.â€? E kukurei Anna bwa a tia n kamanoia mannikiba. Antai ae e nako ni kaukiang ma [Anna] â—‹ â—‹ â—‹ No tamana? Correct Incorrect Response A tiku iaa tenaan mannikiba? [I aon mwaangan kaai] â—‹ â—‹ â—‹ No Correct Incorrect Response Bukin tera e tamwarakea te aroka [E na rawea ana mannikiba] â—‹ â—‹ â—‹ No Anna? Correct Incorrect Response Tera tuangakin Anna iroun tamana [E taburoroko ke e tabuaki ibukiia mannikiba? raweakiia mannikiba.] â—‹ â—‹ â—‹ No Correct Incorrect Response [Ibukina bwa e a tia ngaia Bukin tera ngkai e kukurei Anna? ma tamana ni kawakiniia â—‹ â—‹ â—‹ No ma ni kamanoia mannikiba.] Correct Incorrect Response 86 Early Grade Reading Assessment: Student Response Form MOD 9 Module 9. Dictation Turn this student response form to the last, lined page for writing, and place it in front of the student. Take the student stimulus sheet and turn to the last page, where you will find the same instructions as below. Say: Kakaauongo raoi bwa N na wareka te kibuntaeka aei nakoim. N na warekia teniua te tai. N na warekia moa teuana te tai. I mwiina N na manga warekia riki te kauoua n tai ni kamwakorokoroa, teutana i mwiin teutana ao ko na kaewea are ko ongo. N na manga warekia te kateniua n tai bwa ko aonga n noora mwiin are ko kaewea. Ko oota n te bwai ae ko na karaoia? The student will write the dictation sentence on the lined page of the response form. Read the following sentence aloud ONCE at about 1 word per second. A toka n te ka ataei bwa a na nako n te marae ni wanikiba. Then give the child a pencil, and repeat a SECOND time, grouping the words. Wait 10 seconds after each group, allowing the student to write. A toka - n te ka ataei - bwa a na nako - n te marae ni wanikiba (biriari) Then repeat the sentence a THIRD time while the child is writing. A toka n te ka ataei bwa a na nako n te marae ni wanikiba. Give the child some time (up to around 15 seconds) to complete writing after the third reading. 87 Annex 2.B: Student Questionnaire Kiribati Student Questionnaire Taiaoka! Kaekai titiraki aikai nakon am kabanea n tamaroa Taioaka maroro ma te ataei n te aro ae e na tamaroa ao n iraorao. Taiaoka kamronrona te reke are ena anganiko te ataei ao man aki anganiia te reke n aron akana a oti iaon te beeba n reke. Uringa taua mwiin ana reke te teei are e anganiko. 1 What grade were you in last year? Did not attend school………………………. 0 Ko koraki iraua n te ririki ae e nako? Year 1 ……………………………………………… 1 Year 2……………………………………………… 2 Year 3……………………………………………… 3 Pre-school…………………………………………… 7 Do not know/ No response ……………… 88 2 Did you go to a pre-school (center-based)? No…………………………………………………….. 0 Ko reirei n aia reirei ataei imwaain rinim n te moan Yes……………………………………………………. 1 rinan? Do not know/ No response………………. 88 3 Did you eat any food before you arrived at school No……………………………………………………. 0 today? Yes……………………………………………………. 1 Ko a amarake imwaain manangam nakon te reirei? Do not know/ No response………………. 88 5 What language (s) do you speak at home? Tuvaluan Fijian Tera te taetae ae kam kabonganaa ni mwengami? English Chinese Tick all responses given. Kiribati Other 6 What language(s) does your teacher speak in the Tuvaluan Fijian classroom? English Other Tera te taetae ae kabonganaa am tia reirei n te Kiribati umwanreirei? Tick all responses given. 7 What does the teacher say or do when someone Neutral response answers a question correctly in class? Teacher says or does nothing ……………. 0 Tera te taeka / bwai ae karaoia te tia reirei ngkana e Positive response eti ana kaeka te ataei? Praise students (good job/well done).... 1 Tick all responses given. Teacher asks the class to clap ……………. 2 Teacher is happy ………………………………. 3 Teacher smiles ………………………………….. 4 Other…………………………………………………… 7 Don’t know/ No response………………… 88 8 What does the teacher say or do when someone Positive response answers a question incorrectly in class? Teacher says good effort/ good try .……… 1 Tera te taeka/bwai ae karaoia te tia reirei ngkana e Teacher corrects student.………………………2 bure ana kaeka te ataei? Explains question again/says try again…. 3 Tick all responses given. Neutral response Teacher says or does nothing ……………. 4 Asks another student/says go sit down .. 5 Negative response Teacher hits the student ……………………. 6 Teacher insults or yells at student ……… 7 Teacher says no……………………………………..8 Punishes student in some other way …. 9 Don’t know/No response ………………… 88 1 88 9 Does anyone know how to read at your home? No .…………………………………………………… 0 Iai ae ko ataia ae rabakau ni wareware ni mwengam? Yes.……………………………………………………. 1 Do not know/ No response .…………….. 88 10 [If yes to question 9] Who is that? Mother ……………………………………………… 1 Ngkana iai ao antai? Father ……………………………………………….. 2 Tick all responses given. Sister …………………………………………………. 3 Brother ……………………………………………… 3 Any other person ………………………………. 4 Identify that person: _______________ Do not know/ No response ………………. 88 11 Does the teacher give you homework? No …………………………………………………….. 0 Iai am bwai n reirei ae anganganiko am tia reirei bwa Yes ……………………………………………………. 1 kona karaoia ni mwengam? Do not know/ No response ………………. 88 12 Does anyone help you do your homework? No …………………………………………………….. 0 Iai ae e buobuokiko n karaoi am bwai n reirei ni Yes .…………………………………………………… 1 mwengam? Do not know/ No response ………………. 88 13 [If yes to 12, then ask: Who is that? Mother ……………………………………………… 1 Ngkana eng ao antai? Father ………………………………………………… 2 Tick all responses given. Sister …………………………………………………. 3 Brother ……………………………………………….. 3 Any other person? Identify that person: _________________ Do not know/No response.……………… 88 14 When you come home from school, does someone in No ……………………………………………………. 0 the home ask about what you did at school? Yes .………………………………………………….. 1 Ngkana ko oki man te reirei, iai kaain mwengam ae Do not know/ No response .……………… 88 titirakiniko bwa tera ae ko karaoia n te reirei? 15 When you get a good mark in school, do you tell No …………………………………………………….. 0 someone at home? Yes…………………….…..……………………………. 1 Ngkana e tamaroa am bwai n te reirei, iai kaain mwengam ae ko tuangnga? 16 If yes to #15, then ask: They do nothing …..……………………………….1 What do they do? They congratulate or encourage me…….. 1 Ngkana iai, ao tera ae a karaoia? They give me a treat …………………………… 1 Other …….…………………………………………… 1 Don’t know/No response .………………… 88 17 Do you have time to read books on your own in your No ……………………………………………………… 0 classroom? Yes .…………………………………………………… 1 E rereke am tai ni wareware I bon iroum n am Do not know/No response ……………… 88 umwanreirei? 18 Do you have a school library here?. Point it out for me. No ….………………………………………………… 0 Okay, Good. Yes .…………………………………………………. 1 Iai ami tabo n tango boki “libraryâ€? ikai? Koteia Do not know/No response ..……………. 88 nakou. E tamaroa. 19 Do you have time to read books in your school library? No ..…………………………………………………… 0 E rereke am tai n wareware n ana tabo n tango boki Yes ..…………………………………………………… 1 te reirei? Do not know/No response .……………… 88 20 Do you take books home from school to read? No .……………………………………………………. 0 Ko uouotii rikaaki booki man te reirei ni wareware iai Yes .…………………………………………………… 1 ni mwengam? Do not know/No response ……………….. 88 2 89 21 Are there other books, newspapers or other things to No ………………………………………………………. 0 read at your house? Yes ……………………………………………………… 1 Iai riki booki, beba ni kaongora ke bwaai riki tabeua Do not know/No response ………………… 88 ake ko kona n warekii n am auti? 22 [If yes to #21, then ask: Kiribati ………………………………………………… 1 What language is used in these books? English ..………………………………………………. 2 Any other language .…………………………… 3 Baikara taetae aika a kabonganaaki ni booki aikanne? List: ______________________________ Do not know/ No response ……………… 88 23 Do you read aloud to someone at home? Yes ………………………………………………….. 1 Ko wawareware n takarua nakon temanna ni No …………………………………………………… 0 mwengam? 24 How often? Sometimes or everyday Sometimes ……………………………………….. 1 Iraua te tai? N Tabetai ke ni katoabong Every day …………………………………………… 2 Don’t know/Refuse ……..……………………. 88 25 Do you read just by yourself and to yourself at home? Yes ………………………………………………….. 1 Ko wawareware I bon iroum ke bon nakoim ni No …………………………………………………… 0 mwengam? 26 How often? Sometimes or everyday Sometimes .……………………………………….. 1 Iraua te tai? N tabetai ke ni katoabong Every day …………………………………………… 2 Don’t know/ Refuse …..……………………. 88 27 Does someone read to you at home? Yes ………………………………………………….. 1 Iai ae wawareware nakoim ni mwengam? No …………………………………………………… 0 28 How often? Sometimes or every day? Sometimes .……………………………………….. 1 Iraua te tai? N tabetai ke ni katoabong? Every day …………………………………………… 2 Don’t know/ Refuse .…..……………………. 88 29 Do you have access to a computer or mobile device? Yes ………………………………………………….. 1 Iai am tarebooon, kombinta ke tablet n mwengam? No …………………………………………………… 0 30 If yes to #29, then ask: Yes ………………………………………………….. 1 Do you read on it? No …………………………………………………… 0 Ko kabongana ni wareware iai? 30b If yes to #was, then ask: Sometimes .……………………………………….. 1 How often Every day …………………………………………… 2 Iraua te tai? Don’t know/ Refuse .…..……………………. 88 31 Do you like reading? No…………………………………………………….. 0 Ko tatangira te wareware? Yes……………………………………………………. 1 Do not know/ No response………………. 88 Thank you very much for talking with me! Ko bwati n rabwa ibukin te maroroo! 3 90 Annex 2.C: Head Teacher Questionnaire Kiribati Ministry of Education Early Grade Reading Assessment (EGRA) Head Teacher Questionnaire The Ministry of Education of Kiribati is conducting a study to better understand how children learn to read. We would like to ask you some questions about your school. • The information obtained in this questionnaire will be used by the Ministry of Education to help identify areas where additional support may be needed. • The name of your school, the grade level and class you teach will be recorded, but only so that we can correctly link school, class and student data in order to analyze relationships between children’s learning and the characteristics of the schools in which they learn. Your school’s name will not be used in any report or presentation. • I will read you the consent statement below and mark ‘X’ in the “Yesâ€? box if you agree. Please respond to the questions I ask you as completely and accurately as you can. It should not take you more than 30 minutes for us to complete this questionnaire. • You do not have to complete the questionnaire if you do not want to. CONSENT STATEMENT: I understand and agree to participate in this reading study by filling out this questionnaire with you as completely and accurately as possible. YES SECTION 1: YOUR SCHOOL FACILITY AND THE COMMUNITY We would like your views on your school facility and the community’s role in your school. Name of Island Name of School Age Years____________ Male…………………………………………......... 1 Gender Female………………………………………........ 2 1 None …………………………………………....... 0 JSC (Junior Secondary Certificate) (Form 3)... 11 What is your highest level of KNC (Kiribati National Certificate) (Form 5). .. 12 qualification? KSSC (Kiribati Senior Secondary Certificate) . 13 (PSSC or Form 6)…………..………….…......... 2 SPFSC (South Pacific Form Seven Certificate) 4 USP Foundation (equivalent to Form 7)............. 1 FSLC (Fiji School Leaving Certificate)…............ 3 Certificate in Primary Teaching (KTC) .............. 5 91 Diploma in Primary Education (KTC)................. 6 Diploma in Primary Education Upgrading (KTC) 7 Bachelor’s degree in Education (primary)......... 8 Bachelor’s degree in Education (secondary)... 14 Master’s degree in Education….…..…............. 9 2.b Other (specify) ..……………………………. 10 2 What is your role? HT……………………….…………………......…… 1 Teacher ……………………….…………….......… 2 Other ……………………….……………….....…… 0 3 [IF HT]How many years have you been a head teacher? Years_______________ 4 IF HT]How many years were you a teacher before becoming a head teacher? Years _______________ 5 No…………………………………………...........…. 0 Is there a school committee at this school made up of parents and Yes………………………………………….............. 1 teachers? Don’t know……………………………….........…… 2 6 Once a week………………………………....…….. 1 Once a month……………………………...………. 2 How often did the school committee Once a term……………………………………....… 3 meet in the past year? Once a year……………………………………...…. 4 Skip if #5= No or Don't know Other……………………………………………...…. 5 Don’t know……………………………………....….. 6 6c No………………………………………….........…. 0 Is there a school improvement plan committee at this school made up of Yes……………………………………….........…... 1 parents and teachers? Don’t know…………………………………..…….. 2 6d Once a week………………………………...…….. 1 Once a month………………………………...……. 2 How often did the school Once a term……………………………….…..…… 3 improvement plan committee meet Once a year…………………………………..……. 4 in the past year? Other………………………………………..………. 5 Don’t know…………………………………..…...… 6 7 Is there a kauntira member on the No………………………………………........……. 0 school committees? Yes………………………………………........…... 1 Skip if #5= No or Don't know 8 Is there a clean, safe water supply No…………,,,,,………………………….……..…. 0 available on the school premises? Yes…………..…………………………………..... 1 Sometimes………………………………….…….. 2 92 9 No…………,,,,,…………………………..….…..…. 0 Does the school have electricity? Yes…………..……………………………..…..…... 1 Sometimes…………………………………...….… 2 10 No…………,,,,,…………………………..…….…... 0 Does the school have girls’ and Yes, separate toilets……………………..….…….. 1 boys’ toilets? Yes to toilet, but same toilet for girls and boys.… 2 11- 11. Parent or community member checking on 15 student attendance Once a week………………………………..…….. 1 Once a month……………………………….……. 2 Once a term………………………………….…… 3 Once a year………………………………………. 4 Never………………………………………..…….. 5 Other………………………………………………. 6 12 Parent or community member visiting you to resolve problems Once a week………………………………..…….. 1 Once a month……………………………….……. 2 Once a term………………………………….…… 3 Once a year………………………………………. 4 Never………………………………………..…….. 5 Other………………………………………………. 6 13 Parent or community member helping you at the How many times during the last term school did a parent or member of the Once a week………………………………..…….. 1 community come to your school to Once a month……………………………….……. 2 do any one of the following things. Once a term………………………………….…… 3 [Please read each type of activity Once a year………………………………………. 4 and indicate the response of the Never………………………………………..…….. 5 teacher.] Other………………………………………………. 6 14 Parent or community member being guest speaker at your school Once a week………………………………..…….. 1 Once a month……………………………….……. 2 Once a term………………………………….…… 3 Once a year………………………………………. 4 Never………………………………………..…….. 5 Other………………………………………………. 6 15 Parent or community helps clean up the school or the school grounds…………………….. 93 Once a week………………………………..…….. 1 Once a month……………………………….……. 2 Once a term………………………………….…… 3 Once a year………………………………………. 4 Never………………………………………..…….. 5 Other………………………………………………. 6 16 Did a parent or member of the No………………………………………….........……. 0 community come to your school to Yes…………………………………………............... 1 do for any other reason? Don’t know/No response……………….....……. 99 16b if yes, specify 17 Semi-permanent (concrete walls, thatch roof OR iron roof, coconut walls)………………………… 1 Which description is most Made from all local materials (coconut walls, appropriate for your school? thatch roof)………………………………………. 2 Permanent (concrete walls, iron roof – all imported materials)……………………………… 3 18- How many teaching days ______ days in the last term 19 (instructional days in which teachers ______ teaching days that actually occurred last were face to face with the students) term (eliminate days such as sports, cultural, MAD, were there in the last term? exams, weather, and other days that were not instructional days in which you were face to face with Please refer to the school calendar to students) answer this question. 20 What are the most common reasons Weather (cyclone, rain, tsunami, etc.)……........... 1 for school to be closed (apart from Official holidays………………………….......…..... 2 school holidays already announced Village celebrations…………………….......…...... 3 on the school calendar)? (Multiple responses possible.) Sports and cultural events……………............…. 4 No electricity/power outage……………...........… 5 No water…………………………………..........…. 6 Fire………………………………………...........…. 7 Other………………………………….....…......….. 8 School doesn’t close……………….....……...….. 9 94 Annex 2.D: Teacher Questionnaire 95 1e Married……………………………………........… 1 Divorced……………………………………......... 2 What is your marital status? Single (never married)…………………….......... 3 Widow/widower……………………………......... 4 2 None …………………………………………....... 0 JSC (Junior Secondary Certificate) (Form 3)... 11 KNC (Kiribati National Certificate) (Form 5). .. 12 KSSC (Kiribati Senior Secondary Certificate) . 13 (PSSC or Form 6)…………..………….…......... 2 What is your highest level of SPFSC (South Pacific Form Seven Certificate) 4 qualification? USP Foundation (equivalent to Form 7)............. 1 FSLC (Fiji School Leaving Certificate)…............ 3 Certificate in Primary Teaching (KTC) .............. 5 Diploma in Primary Education (KTC)................. 6 Diploma in Primary Education Upgrading (KTC) 7 Bachelor’s degree in Education (primary)......... 8 Bachelor’s degree in Education (secondary)... 14 Master’s degree in Education….…..…............. 9 2.b Other (specify) ..……………………………. 10 How many years have you been teaching? Years_______________ 4 How many years have you been teaching on this island? Years _______________ 5 Normally, how long does it take Record response: you to get from your home to (Answer in minutes - 0 to 120 minutes) your school? 6 Does your school provide you No……………………………………………........ 0 with housing? Yes………………………………………….......... 1 7 Your school………………………………............ 1 The ministry of education of Kiribati…..........…. 2 Who provides your transport to Island council (kauntira)…………………........... 3 school? You pay transport for yourself……………......... 4 You walk or bike to school…………………........ 5 SECTION 2: LIBRARY, READING CORNERS AND RESOURCES 8 Does your school have a library No…………………………………………….….... 0 that students can use? Yes…………………………………………....…... 1 8b Is the library accessible to No……………………………………………….…. 0 students? Yes…………………………………………….…... 1 8c If yes How often do you use the Every day……………………………………….… 1 school library with your Two or three times a week………………….…... 2 students? Once a week………………………………….….. 3 Less than once a week……………………….…. 4 2 96 Never…….…………………………………..……. 5 9 If yes, do you supervise your No…………………………………………….……. 0 students when they use the Yes…………………………………………...……. 1 library? Skip if #8 or #8a = No Don’t know/No response………………………. 99 10 Do you have a reading corner No……………………………………….……….….0 (or classroom library) in your Yes…………………………………………...……. 1 classroom? 11 Every day…………………………………………. 1 If yes, how often do your Two or three times a week………………...……. 2 students use the reading Once a week………………………………..……. 3 corner? Skip if #10 = No Less than once a week…………………........…. 4 Never…………………………………….......…… 5 12 What kind of reading books and Instructional readers……………..…………........ 1 classroom reading resources in Big Books ………………………….………......... 2 Kiribati language for students do you have at your school (Can Posters/Charts with poems…………..……....... 3 be in a box or in use in the Posters/Charts with songs…………………....... 4 classroom) Posters/Charts with phonics………………........ 5 [Tick ALL that apply] A5 Letter Cards………………………….........… 6 (You can provide list of titles under Small cards – phonics cards for word building each category to assessors). (sets of 3)……………………....................…..… 7 12b Others (specify) ……………………………. 8 How often do you use the following books and resources in Kiribati reading lessons? Book or resource type Never Every 2 1 or 2 days 3 or 4 Daily weeks a week days a week 13 Instructional readers 1 2 3 4 5 14 Big Books 1 2 3 4 5 15 Posters/Charts with poems 1 2 3 4 5 16 Posters/Charts with songs 1 2 3 4 5 17 Posters/Charts with phonics 1 2 3 4 5 18 A5 Letter Cards 1 2 3 4 5 19 Small cards – phonics cards 1 2 3 4 5 for word building (sets of 3) 19 Use blackboard for reading 1 2 3 4 5 c lessons 20 Others (specify) 1 2 3 4 5 Comments: 3 97 SECTION 3: TEACHING AND LEARNING I will ask you some questions about the activities you did with your students over the last five days of school. Please tell me if you did each activity daily, 3 or 4 days a week, 1 or 2 days a week, not in the past 5 days, but you do it sometimes, or never Activity Never Not in the 1 or 2 3 or 4 Daily last 5 days a days a days, but week week sometimes 21 Listening comprehension 1 2 3 4 5 (children listen to a story and answer questions) 22 Children practice letter names 1 2 3 4 5 23 Children orally retell a story 1 2 3 4 5 that they have read 24 Children practice letter sounds 1 2 3 4 5 25 Children sound out unfamiliar 1 2 3 4 5 words using knowledge of letter sounds 26 Children learn meanings of new 1 2 3 4 5 words/vocabulary 27 Shared reading (read as a 1 2 3 4 5 whole class with the teacher) 28 Group guided reading (teacher 1 2 3 4 5 listens to children read in small groups) 29 Listening to a child read aloud 1 2 3 4 5 to you one on one 30 Students reading on their own 1 2 3 4 5 silently 31 Reading comprehension 1 2 3 4 5 activities (orally or in writing) 32 Children take books home to 1 2 3 4 5 read with their parents 33 Evaluating students’ oral 1 2 3 4 5 reading with running records or any other method 33c Teacher works on Word 1 2 3 4 5 building with students 33d Students do Read and draw 1 2 3 4 5 33e Students working on spelling 1 2 3 4 5 words in exercise books 33f Students writing sentences 1 2 3 4 5 34 Other activities (please 1 2 3 4 5 describe): 4 98 SECTION 4: READING IN THE CURRICULUM Listed below are five important early reading skills that students must learn. For each reading skill, circle the class level when you think students are able to: Reading skill Year 1 Year 2 Year 3 Year 4 Year 5 Not important 35 Recognize and say letter 1 2 3 4 5 0 names 36 Understand stories read 1 2 3 4 5 0 aloud by the teacher in the classroom 37 Sound out words using 1 2 3 4 5 0 phonics 38 Read aloud to teacher 1 2 3 4 5 0 and other students 39 Understand simple texts 1 2 3 4 5 0 that they read in class Comments: SECTION 5: IN-SERVICE TRAINING AND TEACHER SUPPORT MATERIALS We would like your views on teacher support for reading. 40 No………………………………………… 0 Do you have a Year 1 and 2 syllabus Yes, paper copy……………….…..….….. 1 for Kiribati? Yes, copy on computer…………………… 2 Don’t know/no response…………….…… 3 40b Not very useful…………………………….. 1 If yes to Question 40, how useful do Quite useful……………………………… 2 you find the syllabus? Very useful………………………………… 3 41 No………………………………………….. 0 Do you have a Year 3 and 4 syllabus Yes, paper copy………………….……..... 1 for Kiribati? Yes, copy on computer…………………... 2 Don’t know/no response……………….... 3 41b Not very useful……………………………. 1 If yes to Question 41, how useful do Quite useful……………………………….. 2 you find the syllabus? Very useful………………………………… 3 42 None…………………………………......… 0 Which Teacher Guides for Kiribati Year 1 Term 1 - 2 - 3 - 4 ......................... 1 Year 1, 2, or 3 do you have? Year 2 Term 1 - 2 - 3 - 4.......................... 2 (Tick all that apply). Year 3/4 : Te- Kiribati……………............. 3 42b Not very useful……………………………. 1 If yes to Question 42, how useful do Quite useful……………………………...... 2 you find the teacher guides? Very useful………………………………… 3 44 Have you received any training on No…………………………………………... 0 how to teach reading in the last three Yes………………………………………..... 1 years? 45- If yes to Question 44, indicate the Name of training__________ 5 99 46 name of the training and how many hours of reading training you have Total hours__________ received in total (approximately). 47 If yes to Question 44, what was the most useful aspect of this training? __________________________ SECTION 5: PARENTS AND THE COMMUNITY We would like your views on the community’s role in your school. 48- 48 Have a parent-teacher meeting with you 54 Once a week……………………………...................….……….. 1 Once a month………………………………….......................…. 2 Once a term………………………………...................….……… 3 Once a year………………………………...................….………. 4 Never…………………………………...................….……..…….. 5 Other……………………………………...................….…………. 6 49 Parent or community member checking on student attendance……. How many Once a week……………………………...................….……….. 1 times Once a month………………………………….......................…. 2 during the Once a term………………………………...................….……… 3 last term Once a year………………………………...................….………. 4 did a parent Never…………………………………...................….……..…….. 5 or member Other……………………………………...................….…………. 6 of the community 50 Parent or community member visiting classroom to resolve come to problem your class Once a week……………………………...................….……….. 1 to do any Once a month………………………………….......................…. 2 one of the Once a term………………………………...................….……… 3 following Once a year………………………………...................….………. 4 things. Never…………………………………...................….……..…….. 5 [Please Other……………………………………...................….…………. 6 read each type of 51 Parent or community member helping you in the classroom activity and Once a week……………………………...................….……….. 1 indicate the Once a month………………………………….......................…. 2 response of Once a term………………………………...................….……… 3 the Once a year………………………………...................….………. 4 teacher.] Never…………………………………...................….……..…….. 5 Other……………………………………...................….…………. 6 52 Parent or community member being guest speaker in your class Once a week……………………………...................….……….. 1 Once a month………………………………….......................…. 2 Once a term………………………………...................….……… 3 Once a year………………………………...................….………. 4 Never…………………………………...................….……..…….. 5 Other……………………………………...................….…………. 6 6 100 53 Parent or community helps clean up the school or the school grounds Once a week……………………………...................….……….. 1 Once a month………………………………….......................…. 2 Once a term………………………………...................….……… 3 Once a year………………………………...................….………. 4 Never…………………………………...................….……..…….. 5 Other……………………………………...................….…………. 6 54 Did a parent No…………………………………………….….................……. 0 or member Yes……………………………………………....................……. 1 of the Don’t know/No response……………………...................……. 99 community come to your school to do for any other reason? 54b if yes, specify SECTION 5: TEACHER AND STUDENT ATTENDANCE 55 How many days were you absent None……………………………………...... 1 from school in the last term? 1-5 days……………………………………. 2 6-10 days………………………………..… 3 Over 11 days……………………………… 4 Don’t remember…………………………... 5 56 Weather (cyclone, rain, tsunami, etc.)…. 1 You are sick ……………………………… 2 Caring for a sick relative………………… 3 What are the most common reasons Family functions (Funeral, Wedding, for you to be absent from school? Birthdays)………………………………..... 4 (Apart from official holidays) Travel……………………………………… 5 Personal business………………………... 6 Other (specify)………….…………………. 7 56b If other, please specify 57- Go to the attendance register for the __________total # of boys enrolled in the 60 5th week of the term. class Record the average number of boys _________average # of boys who attended and average number of girls who in Week 5 attended school on week 5 and record the total number of boys and total __________total # of girls enrolled in the number of girls enrolled. class __________average # of girls who attended in Week 5 7 101 61 What are the most common reasons Weather (cyclone, rain, tsunami, etc.)… 1 for students to miss school? Student sickness………………………… 2 [Tick ALL that apply] Caring for siblings………………………. 3 Family problems………………………… 4 Family finances…………………………. 5 Family function (Funeral, Wedding, Birthdays)………………………………..… 6 Travel…………………………………..….. 7 No clean uniform……………………..…… 8 No lunch…………………………………... 9 No money for school materials………… 10 Internet addiction (went to Internet café)……………………………………..... 11 Student didn’t want to go……………..… 12 Parent didn’t care………………..………. 13 Student scared of the teacher………..… 14 Haven’t done their homework………....… 15 Bullying……………………………………. 16 No transport/nobody brings them to school……………………………………. 17 Other……………………………………... 18 62 Is there any other comment or information you would like to ________________________________ provide me? Thank you very much for taking the time to complete this questionnaire with me! 8 102 Annex 2.E: Classroom Observation Kiribati Early Grade Reading Assessment Classroom Observation A. Date of observation: B. District North - Central - South - Tarawa South - Linnux C. Island D. School name E. School Code: F. Observer Name G. Year observed: If combined classes, indicate which classes have been combined o 1= o 2= o 3= o 4= o 5= o 6= H. Subject observed o 1 = Language (Reading) o 2 = Language (Other) o 3 = Other subject I. Time observation started: LJ Time observation finished: _____ : ____ (circle AM or PM) AM / PM ____ : ___ (circle AM or PM) AM / PM INSTRUCTIONS 1. Have the required number of observation guides on your clipboards before you travel to the schools. 2. Sharpen two pencils and attach to your clipboard before you travel to the schools. 3. Carry a pencil sharpener at all times. 4. Observe a class that has a teacher present on the day of the classroom observation. 5. Make a point of introducing yourself to the teacher before the observation session. 6. Ask the teacher if you can observe a reading lesson. If this is not possible and this is the only classroom to observe, proceed with the observation. 7. The whole classroom observation should last one hour. Keep to this time allocation at all times. 8. Collect a random sample of 5 language exercise books to assess. Take about 10 minutes at the end of the observation and return the books to the teacher before you leave. 9. Blend into the classroom environment during the observation sessions. 10. Please complete ALL sections of this form, including the comments section. It is important to provide as much detail as possible. 11. Thank the teacher at the end of the lesson. 1 103 1. CLASS ENROLLMENT (You can get this at the close of the observation). Write down the total number of students enrolled in the class observed (look in the roll book for this information). If combined classes, record the enrollment for each class. Class Male Female Total 1 2 3 4 5 6 Comments: 2. ATTENDANCE (Count students when you sit down to begin the observation quietly from your seat). Write down the number of students who are actually in class today (count the children in class). If combined classes, record the attendance for each class. Class Male Female Total 1 2 3 4 5 6 Comments: 3. CLASSROOM ENVIRONMENT Tick the appropriate response to show what you can actually see in the classroom. Tick ALL sections. Resources Yes No 1. Kiribati alphabet displayed in classroom 2. Classroom has Kiribati print in common teaching aids and wall displays ¨ calendar ¨ days of week ¨ alphabet ¨ numbers ¨ weather 2 104 3. List of Kiribati spelling/vocabulary words written on black board 4. List of Kiribati spelling/vocabulary words written on charts/posters 5. Kiribati stories written on the black board 6. Kiribati hymns/songs written on the black board 7. Kiribati stories written on charts 8. Kiribati hymns/songs/prayers written on charts 9. Recent students’ written work (from this school year) displayed around classroom 10. Student written work from previous terms displayed around classroom 11. Sufficient classroom space for organized group activities 12. Other print materials are used in instructional activities ¨ newspapers ¨ magazines ¨ flash cards ¨ food wrappers and packages (for example, sugar, biscuits, rice) ¨ prepaid cards ¨ objects in treasure boxes in classroom ¨ Other materials ¨ None 12b. Other, specify ______________________________________ 13. Student reading corner in the classroom 14. Student Profiles (folder with student work and student information) 15. Students sitting on floor 16. Students sitting in desks or chairs Comments: 4. TEACHING AND LEARNING Tick the appropriate response to show what you actually see going on in the classroom. Include as much detail as you can in the comments section. Activity Yes No 17. Teacher having students practice letter sounds (for example: m, b, k, t, a, e, etc.) 18. Teacher having students practice syllable sounds (for example: ma, me, mi, mo, mu) 19. Students learning the meaning of new words/vocabulary 20. Students singing song with the teacher. 21. Teacher leading students in activity or song with letters and words in it and body movements that go with it 22. Teacher reading story/text aloud to students 23. Students reading story/text aloud with teacher 24. Teacher asking comprehension questions when reading story/text 25 Students assisting peers to read (buddy reading) 26 Students reading to each other in groups 27. Students reading silently on their own 3 105 28. Varied reading activities during lesson. ¨ Spelling ¨ Students reporting out about a story or text ¨ general discussions of teacher and students about a book or story ¨ building new words ¨ creating new sentences & stories ¨ Other ¨ None 28b Other activities, specify Comments: 5. LANGUAGE USED BY TEACHERS AND STUDENTS IN CLASSROOM Tick the boxes to show what you observed about the language used in the classroom. Language Kiribati English Other 29. Language(s) of instruction used by teacher 30. Language(s) students use to communicate with each other during classroom activities 31. Language(s) students use to communicate with the teacher. 32. Language(s) written on the black board to teach content 33. Language(s) written on charts, posters, classroom displays and other teaching aids. Comments: 6. TEACHER’S USE of KIRIBATI LANGUAGE (Tick the appropriate response.) Activity Yes No 34. Does the teacher switch languages when teaching in Kiribati class? (Write examples in the Comments section below.) 35. Are there errors in the teacher’s written text in Kiribati on the blackboard? (Write examples in the Comments section below.) 36. Two different writing systems have been used for Kiribati. One writing system uses the standard letters: For example: a, e, i, o, u, f, g, aa, uu, oo, bwa, mwa, etc. Does the teacher use this writing system when teaching or writing Kiribati on the blackboard? 37. Tick here the letters the teacher ¨ u ¨ b is teaching today: ¨ k ¨ m ¨ a ¨ n ¨ t ¨ e ¨ w ¨ ng ¨ i ¨ r ¨ Not teaching letter ¨ o 38. The other writing system includes other symbols like: For example: Ä?, Å«, b’a, m’a, etc. Does the teacher use these additional symbols when teaching or writing Kiribati? Comments: 4 106 7. RESOURCES Write the number of sets of different titles in the appropriate column to show what resources you actually see in the classroom. Please comment if there are books stored elsewhere and not in the classroom. Resources in classroom None 1 to 5 6 to 10 11 to 20 20 or more 39. Instructional readers 40. Big Books 41. Posters/Charts with poems 41b. Posters/Charts with songs 41c. Posters/Charts with phonics 42. A5 Letter Cards 43. Small cards – phonics cards for word building (sets of 3) 44. Mathematics charts 45. Community studies posters 45b. Environmental science posters 45c. Climate change story book 45d. Healthy living charts 46. Others Comments: 47. Count up all the reading books for students that you can see being used during the lesson. Write the total in the box below: 8. DO THE FOLLOWING AT THE END OF THE OBSERVATION. Ask the teachers for a sample of five students’ language exercise books. Areas to take note of in students’ language exercise books (tick ALL sections): Activity Ex book 1 Ex book 2 Ex book 3 Ex book 4 Ex book 5 48. Filling in missing words, sentence beginnings and endings, etc. 49. Short sentences 50. Short stories (2-5 sentences) 51. Reading comprehension activities 52. Labeling things or matching 54. Regular written work in students’ language exercise books 55. Exercises in students’ language books marked by the teacher regularly 56. Written corrections and feedback given by teacher 5 107 7. RESOURCES Write the number of sets of different titles in the appropriate column to show what resources you actually see in the classroom. Please comment if there are books stored elsewhere and not in the classroom. Resources in classroom None 1 to 5 6 to 10 11 to 20 20 or more 39. Instructional readers 40. Big Books 41. Posters/Charts with poems 41b. Posters/Charts with songs 41c. Posters/Charts with phonics 42. A5 Letter Cards 43. Small cards – phonics cards for word building (sets of 3) 44. Mathematics charts 45. Community studies posters 45b. Environmental science posters 45c. Climate change story book 45d. Healthy living charts 46. Others Comments: 47. Count up all the reading books for students that you can see being used during the lesson. Write the total in the box below: 8. DO THE FOLLOWING AT THE END OF THE OBSERVATION. Ask the teachers for a sample of five students’ language exercise books. Areas to take note of in students’ language exercise books (tick ALL sections): Activity Ex book 1 Ex book 2 Ex book 3 Ex book 4 Ex book 5 48. Filling in missing words, sentence beginnings and endings, etc. 49. Short sentences 50. Short stories (2-5 sentences) 51. Reading comprehension activities 52. Labeling things or matching 54. Regular written work in students’ language exercise books 55. Exercises in students’ language books marked by the teacher regularly 56. Written corrections and feedback given by teacher 5 108