Report No: ACS9460 West Bank and Gaza Quality of Teaching in PA School Learning from Local Practices: Improving Student Performance in West Bank and Gaza June 12, 2014 MNSHE MIDDLE EAST AND NORTH AFRICA Document of the World Bank Standard Disclaimer: This volume is a product of the staff of the International Bank for Reconstruction and Development/ The World Bank. The findings, interpretations, and conclusions expressed in this paper do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Copyright Statement: The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The International Bank for Reconstruction and Development/ The World Bank encourages dissemination of its work and will normally grant permission to reproduce portions of the work promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA, telephone 978-750-8400, fax 978-750-4470, http://www.copyright.com/. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA, fax 202-522-2422, e-mail pubrights@worldbank.org. 2 P143564 Final Report Learning from Local Practices: Improving Student Performance in West Bank and Gaza Noah Yarrow, Husein Abdul-Hamid, Manal Quota and Ernesto Cuadra Human Development Sector MNSHD Middle East and North Africa Region 3 TABLE OF CONTENTS EXECUTIVE SUMMARY .......................................................................................................... 5 INTRODUCTION ........................................................................................................................ 6 CONTEXT ..................................................................................................................................... 7 KEY FINDINGS ........................................................................................................................... 8 I. METHODOLOGY .................................................................................................................... 9 II. GENERAL OVERVIEW OF SCHOOL PERFORMANCE IN THE PALESTINIAN TERRITORIES.......... 12 III. HOW DO TEACHER CLASSROOM PRACTICES VARY ACROSS HIGH AND LOW-PERFORMING CLASSROOMS? ............................................................................................................................ 18 IV. SCHOOL LEADERSHIP, TEACHERS AND SCHOOL FACILITIES AND THEIR INFLUENCES ON STUDENT LEARNING OUTCOMES ................................................................................................. 24 V. CONCLUSION: POLICY IMPLICATIONS AND NEXT STEPS ....................................................... 31 ANNEX 1 – LOGISTIC AND FACTOR ANALYSIS RESULTS ......................................... 35 ANNEX 2 MODELS OF TEACHER LEARNING AND BEHAVIORAL CHANGE ........ 42 4 Executive Summary 1. We use the positive deviance approach to identify classrooms in the Palestinian territories with above average student performance as measured by TIMSS and national assessments. We investigate teacher classroom practices and school management practices through classroom observations and survey data collection. When comparing high-performing and low-performing classrooms, we find important differences in both how teachers use their time, and how schools are run. Specifically, more effective teachers tend to use more student discussion in the classroom, have a higher proportion of students engaged in learning, receive greater support and conduct more outreach to parents. Effective management practices included contributions to the school culture, school facilities and the school mission.1 Overall, we found statistically significant relationships between classroom performance and the following parameters: teacher accountability, school culture2, parental engagement, school facilities, school mission and student recognition, student involvement and student-centered pedagogy3. 2. Based on these findings, the team calls for a renewed focus on teachers, instructional practice and monitoring of student learning outcomes in the upcoming Education Sector Development Plan. Specifically, assistance to low-performing schools could include support programs anchored on nearby schools that serve similar students with higher results. Peer learning programs could support lower-performing classrooms to improve, while additional support can be provided through training programs, school management and the teacher supervisors. These recommendations are similar to those in the companion report from the MoEHE. 3. The purpose of the study is to provide timely information to the MoEHE for the development of the Education Strategy 2014-2018, a draft of which was published in January 2014. It was not possible to collect student-level data or more detailed information on teacher practices. Therefore, the conclusions of the study are correlational links between teacher practices and learning impacts. Additional, in-depth investigations of specific teacher practices at different grade levels, for different subjects, and for different student genders are of interest to policy makers in the Palestinian Territories. 1 Please see Annex 1, Logistic and Factor Analysis Results for a complete definition and description. 2 As defined in the factor analysis (see Annex 1). 3 As defined in the factor analysis (see Annex 1). 5 Introduction 4. The motivation for this study is to contribute to the preparation of the new Palestinian Education Strategy by shedding light on the school and classroom level factors that influence student learning, and to identify good practices that can be generalized from high-performing classrooms to those that need improvement. While most Palestinian children are in school, performance on assessments indicates that many of them are not learning as much as they could. This represents not only inefficiency in the use of public resources, but also a lost opportunity for individual students and the society as a whole. The current study was carried out by the Assessment and Evaluation Department (AED) of the Palestinian Ministry of Education and Higher Education (MoEHE) with technical and financial support from the World Bank. The analysis presented in this paper was prepared by the World Bank team as a complement to a previous paper prepared by a team of experts from AED.4 5. A broad base of research shows that teacher quality is a significant determinant of student learning. Income, health, parent education and other factors contribute to learning outcomes, but the single most important school-based determinant of student learning is teacher quality (e.g. Hanushek, 2010 and Rivken et al., 2005). This study was designed to enable the MoEHE to better understand the factors behind low levels of student learning and to identify the better practices that lie behind the performance of positive outliers, or ‘positive deviants.’ Combined with practices form other contexts that have a strong evidence base, this study will inform MoEHE policy and investment choices to improve student learning outcomes through a program of school management and classroom teaching change built on locally based success. 6. For the purposes of this study, schools were classified by student performance in TIMSS 2011 and the 2012 Palestinian national exams. Classroom and school-based tools were then used to gather information from both high and low-performing schools. A total of 122 public, private and United Nations Relief and Works Agency for Palestine Refugees in the Near East (UNRWA) schools were surveyed using four different instruments: (i) Stallings classroom observations; (ii) school leadership survey; (iii) teacher survey; and, (iv) school facilities survey, which are all provided in Annex 1. 7. The study follows the positive deviance approach, which favors local problem solving and local research (Zeitlin et al., 1990, Marsh et al., 2004, Pascale et al. 2010). Positive deviance is the observation that some individuals in a population will implement certain practices and consequently experience better outcomes than other members of the population with similar resources (following Berggren and Ray, 2002 in Marsh et al., 2004). This approach is used here to address the challenges of improving student learning in the Palestinian education system. By gathering local evidence to increase the knowledge base, the team seeks to enable a dialogue and public debate both at the level of policy makers and of practitioners. The working hypothesis is ‘better practices’ taken from local contexts are more easily transferred and have a higher 4 Under Secretary of General Education Dr. Basri Saleh and the AED Team Dr. Mohammed Matar, Dr. Sahar Ouda, Mr. Khalid Bsharat, Ms. Sajeda Atallah, Mr. Waheed Qutnah, Mr. Samer Alkhatib, Mr. Khalid Dian and Mr. Admad Abu Obaid, including data collected by Dr. Samir Safi and Dr. Eid Dahadha “Lessons Learned from High Achieving Schools in Palestine” (MoEHE 2013). The authors would also like to thank Halsey Rogers, Yyannu Cruz Aguayo, Brian Blankespoor and Emma Paulette Etori for their comments and contributions. 6 likelihood of implementation than those originating exclusively from outside the Palestinian context. Examining this proposition constitutes the next step of the investigation. Context 8. Education is highly valued across the Palestinian Territories. School aged children as well as parents eagerly express their commitment to education despite economic and other constraints. This is reflected in the rates of enrollment, 94.4 percent for primary and 82.8 percent secondary (Gross Enrollment Ratio, Total, 2012, World DataBank). However, achieving access to schooling is only one step in the process of creating an educated and productive population able to effectively survive in a highly complicated political climate. Despite high enrolment rates and educational expenditures of approximately 4.9 percent of GDP (Caussen et. al, 2013)5, results from national and international assessments indicate that student learning outcomes in the Palestinian Territories are modest when compared to other countries with similar per-capita GDP (figure 1). Almost half the student population does not meet the “low” international benchmark in TIMSS and only 69.6 percent of students pass the high school Tawjihi exam6. Low student learning outcomes can be addressed both by increasing average student performance and by decreasing the achievement gap between students. This study focuses on quality instruction and learning, an important aspect of effective education reform. Data from EdStats in current USD, except for Palestinian GDP per capita, which comes from the Palestine Central Bureau of Statistics; in order to improve visibility, not all countries are labelled 5 Claussen et al. also report that education accounts for 15.7 percent of total public expenditure in 2012, at approximately $680 USD per student per year 6 UNESCO, World Data on Education, 7th edition 2010/2011 retrieved on 28 April 2014, http://www.ibe.unesco.org/fileadmin/user_upload/Publications/WDE/2010/pdf-versions/Palestine.pdf. 7 Key Findings 9. Using the positive deviance methodology, we find that classroom teaching practices and school leadership activities are strong differentiators that are correlated to different levels of student learning at the classroom level. Teachers in high-performing classrooms use a different mix of instructional practices and are managed differently than teachers in low-performing classrooms. In terms of instructional practice, while time devoted to instruction was very high across Palestinian classrooms compared to international standards, differences in the type and quality of activities by classroom performance were identified. Higher performing classrooms have, among other things: (i) more interactive learning and less teacher-focused lecturing, for example more interactions during class discussions, questions and answers etc., (ii) lower levels of unengaged students than low-performing classrooms, (iii) teachers with greater levels of lesson planning and observable motivation, and (iv) a higher level of utilization of classroom resources such as visual aids, learning materials and ICT. It is also important to note that the majority of low-performing classrooms were composed of all male students. 10. Analyzing the data using the Principal Components Method, we find the strongest differentiators7 associated with student achievement is the level of student participation in the classroom (significant at 0.016), along with other factors such as student centered pedagogy (significant at 0.002) and school culture (significant at 0.001) (Annex 1). For example, having a high proportion of uninvolved students in a classroom reduces the chance of being a high- performing classroom by 98 percent. Relatedly, employing a high proportion of student-centered pedagogy including discussion and debates more than doubles the chance of being a high- performing classroom. Classrooms in schools with a strong and positive school culture that includes high teacher expectations of student performance and parent participation more than doubles the probability of being a high-performing classroom, based on results from the Principal Components analysis. Please see Annex 1 for details. 11. We use these findings to provide evidence from Palestinian data for a series of policy recommendations and actions: i) classroom pedagogic changes to improve student learning outcomes, particularly the use of active discussions focused on student engagement and participation; ii) school leadership focused on student learning, teacher support and parent engagement and not only administrative management; and iii) related policy interventions to re- enforce the Ministry’s goals of providing quality education to all citizens. Sharing of practices between teachers and schools could occur, as many tools to improve Palestinian student performance are known and used in a few high-performing Palestinian classrooms. When these existing local “positive deviant” practices are combined with what has been shown to be effective in other contexts, it is plausible that student learning can improve as measured by both national and international assessments without large increases in educational funding levels. 7 Based on the value of the standardized coefficients for those variables with statistical significance in the logistic regression; Annex 1. 8 I. Methodology 12. The positive deviance approach originated in the context of nutrition, but is now used in many other domains (Pascale and Sternin, 2005). The methodology focuses on identifying individuals (in this case, teachers) who get above average results with the same inputs as other members of the group being observed (Marsh et al., 2004). Logical deductions are then used to select the most probable practices that account for the better-than-average success rate. We have adapted this methodology here and added a second layer of analysis that involves grouping different practices into categories for which we do find statistical significance. We consciously chose this approach in an effort to anticipate the challenges of scale-up and transmission of likely successful behaviors, while recognizing the outcome has more limited scientific validity than other possible approaches. 13. Classroom performance was differentiated based on students’ average achievement on the 2011 international assessment (TIMSS 2011) and 2012 Palestine National Assessment at the classroom level. TIMSS is taken by a nationally representative sample of classrooms in each participating country. Individual classrooms that performed at or above the second TIMSS international benchmark (schools achieving an average score of 475 or higher) or equivalent level in the national assessment were selected as top performers. Classrooms performing at an average of half a standard deviation or greater below TIMSS lowest international benchmark (lower than an average of 350) or in the bottom 20 percent in terms of performance in math, science, or Arabic language in the National Assessment were identified as low performing. 14. Using this criterion, a total of 122 schools were selected to study classroom instructional practices, school leadership, teacher’s conditions and perceptions, and school environment. Of these schools, 78 were identified as high-performing schools and 44 were identified as low- performing schools, of about 150 schools which participated in TIMSS. Ninety of the schools selected for our study are located in the West Bank while 32 are located in Gaza. Table 1 shows the general sample characteristics: Table 1: School Sample Characteristics (add more details) School Type Number of Classrooms Percentage of total Private 22 18 Public 84 69 UNRWA 16 13 Total sample 122 100 15. Classroom teacher practices were observed using the Stallings classroom observation methodology at each school in the sample. The subjects observed included Math, Science and Arabic at the 4th, 8th and 10th grade levels. Each teacher who taught a class in the tested subject at a sampled school was observed once. In the case of the 8th grade, this was the same teacher who completed the TIMSS 2011 questionnaire. In the case of 4th and 10th grades, these were teachers of classes that participated in the national exams. 16. Based on research pointing to the centrality of teachers in determining student learning outcomes (e.g. Hanushek & Rivkin, 2010; Rockoff, 2004), the current study collected data using the Stallings classroom observation tool (Table 2). This low-inference instrument generates 9 information about teacher and student time use that are directly comparable across different types of schools and country contexts. Classroom observers can be trained in about a week’s time to achieve a 0.8 or higher level of inter-rater reliability (Bruns et al. 2014). The tool captures basic information about how teachers are teaching in classrooms and what the students and teachers are doing, but it does not capture micro-level information about what is being taught. The time-based survey method provides information about how teachers use class time in a math class, for example, but does not capture information on the level of mathematics taught, the difficulty of questions asked during student discussions or the strength or weakness of student responses. The amount of time spent on academic activities is likely a necessary but insufficient factor to explain the complex relationship between teacher classroom practices and student learning. The quality of the use of that time is also important, as we will see below. Table 2: Stalling Observation Instrument Categories for Teacher and Student Time Use Use of Time Specific Practice Read Aloud Academic Activities Demonstration/Lecture Student Discussion/Question and Answer Practice and Drill Classwork/Assignment Copying Verbal Instruction Classroom Management Student Social Interaction Students Uninvolved Discipline Classroom Organization Administrative Tasks Teacher Teacher Social Interaction or Teacher Uninvolved Off-Task Teacher Out of Room 17. The Stallings observation instrument8 contains three sections of which only the Classroom Snapshot was used for reasons of time and cost. The Classroom Snapshot consists of: (i) identification of classrooms, number of adults and students present, the duration of the class, the grade level, and the subject taught; (ii) classroom observations during class time to obtain information on how the teacher spends his or her time and with whom, how the aides spend their time and with whom, and the activities that occur in the classroom, and (iii) classification of the different groupings of students during the class (single, pairs, small groups, large groups, whole class etc). The tool consists of a series of frames in which each behavior/interaction is recorded in four categories of: who, to whom, what, and how. 18. The classroom snapshot data was recorded ten times during a given class period by a trained observer who was not aware of the performance ranking of the classroom being 8 The description of the Stallings method is adapted from the “Stallings Observation System” manual retrieved on www.personal.psu.edu/wuh113/blogs/.../SOS_Manual-New.doc‎ . Retrieved 26 December 2013. 10 observed. The schools were informed that they would be visited, but the date and classroom to be visited were not disclosed in advance. 19. The assessment of school leadership was done using an adapted and translated version of the OECD leadership toolkit and the TALIS school director questionnaire, as well as questions from TIMSS 2011 to create a context-sensitive school leadership assessment.9 This modified tool was subjected to an expert review by Ministry staff and then piloted at three schools prior to implementation (see Annex 1). 20. The tool to investigate teacher perceptions are based on the World Bank SABER- Teachers framework and TIMSS instruments.10 Teachers were asked questions of their perceptions related to the main teachers’ policy areas: leadership, qualifications, training and development, motivation, and support level. Data was also collected on working conditions and school facilities informed by the SABER and TIMSS survey instruments as well as specific items from the School Snapshot of Management Effectiveness11. Student and parent interviews were not conducted as part of the study for reasons of cost. 21. Two teams of researchers, deployed in Gaza and the West Bank, were trained on the use of the Stallings observation and the three survey instruments to conduct the field work. The surveys were administered through the face-to-face interviews. Data was collected during the months of March and April in the second term of school year 2012/2013. Data was cleaned by AED and preliminary analysis was conducted by AED staff with support from World Bank staff. 22. In order to identify factors to differentiate schools, data on teachers, school leadership, and school infrastructure were first analyzed using the Principal Components method (following Filmer 2001, among others). Principal Components Analysis (PCA) is used for two objectives: (1) reducing the number of variables comprising the questionnaire while retaining the variability in the data and (2) identifying hidden patterns in the data and classifying them based on the amount of the information that can be attributed to them. For the purposes of this calculation, we take all principal components to be orthogonal. The data is modeled as in equation (1): f(x)=b1 F1+b2F2+b3 F3+b4 F4 …. (1) where b represents different factor loadings and F represents the factor scores 23. We used the Principal Components methodology to identify factors that explained variation in the data of each instrument (classroom observation, school director interview, teacher interview, facilities survey). Each factor was composed of several items from the specific instrument, selected based on similarities to each other and their overall contribution to the variation in the data set from the instrument. We then selected the five factors that contributed 9 The OECD toolkit can be consulted at: http://www.oecd.org/education/school/44339174.pdf. Retrieved on 26 December 2013, the TALIS Principal Questionnaire at http://www.oecd.org/edu/school/43081362.pdf. Retrieved on 17 January 2014, and the TIMSS School Questionnaire at https://nces.ed.gov/timss/pdf/T11_Gr8_SchQ_USA_final.pdf. Retrieved on 17 January 2014. 10 SABER-Teachers http://wbgfiles.worldbank.org/documents/hdn/ed/saber/supporting_doc/Background/TCH/Rubrics_Teachers.pdf. Retrieved on January 17, 2014. TIMSS http://timssandpirls.bc.edu/timss2011/downloads/T11_TQM_8.pdf. Retrieved on January 17, 2014 11 Snapshot for School Management Effectiveness https://www.eddataglobal.org/documents/index.cfm?fuseaction=pubDetail&id=353. Retrieved on January 17, 2014 11 the most to that instrument as a way of summarizing the elements of the instruments with the highest explanatory power. Essentially, it is splitting the variation into separate parts and then ranking the parts by importance for each instrument. 24. Once the principal factors were identified, logistic regression was used to identify the magnitude of the potential explanatory power that each of the factors have on determining how low-performing classroom could move to the high-performing category. Logit(p)= log( )= β0+β1X1+ β2X2+ β3X3+ β4X4+ β5X5…(2) where p is the probability of being a high-performing classroom 25. The coefficient Exp(β) from the logistic regression quantifies the positive or negative probability that a given characteristic will be found in a high-performing classroom. For example, the value of Exp(β) is 3.41 for interaction in the classroom. This means that high- performing classrooms are 3.4 times more likely to have a high level of interaction than low- performing classrooms. 26. There are a number of limitations to our study. One is that teachers were observed only once, and only one teacher per school was observed. There is no estimation of changes in individual teacher behavior across time (good days vs. bad days) or variation of teacher practices within a school. Since this study focuses primarily on classroom level dynamics, we don’t consider the lack of school-wide data as a major constraint. Another limitation of the study is that student-level data and the motivational factors for teacher classroom practices were not collected (e.g. why does one teacher use lecture almost exclusively and another use primarily student discussion?). Understanding behavioral motivations would support attempts at influencing them. For example, does the teacher not know how to engage students in a discussion, or do they believe that discussion is an ineffective pedagogic practice? In terms of student-level information, we expect that students are not assigned completely randomly to teachers or schools and that this has some influence on both teacher practice and student performance. We do not have student-level data that would allow a comparison of students to ensure that high and low-performing classrooms are fully comparable. Another significant limitation is the lack of information about the quality of specific teacher practices. Teacher A lectured for the majority of the classroom time, but was the lecture relevant to the curriculum, was it well presented, was it content information rich? Teacher B used student dialogue and discussion, but was this a high-quality discussion about a relevant topic? The qualitative nature of the activities is important, since it may mask important links to student learning. This study was conducted to respond to the need of the Ministry for real-time information about teacher classroom practices and so it was not possible to gather information on all optimal variables which limited further econometric analysis at the student and within school related variables. II. General overview of school performance in the Palestinian Territories 27. Student performance in the Palestinian Territories showed improvement between 2007 and 2011 as measured by the respective rounds of TIMSS. Despite this improvement, the performance level of Palestinian students lags behind some of their peers in the MENA region and significantly below the internationally constructed average. Analysis of the variation of learning outcomes across schools in the Palestinian Territories has uncovered classroom 12 practices that are associated with higher student learning. However, those are limited to a modest number of schools, and instead, a large number of students in the Palestinian Territories have poor learning outcomes. For example, in eighth grade math, about 48 percent of students scored less than the “low” benchmark and only around 7 percent performed at the “high” or “advanced” TIMSS 2011 benchmarks.12 28. Many countries perform poorly on TIMSS and other international exams, particularly developing and lower and middle income countries. Putting aside the Palestinian’s relative performance, let us consider absolute performance. The “low” benchmark, which almost half of Palestinian students did not meet, is “Students have some knowledge of whole numbers and decimals, operations, and basic graphs.” (TIMSS 2011). An example question on 8th grade math on TIMSS is: 42.65 +5.748 = _____ 29. A bare majority of Palestinian students (56 percent) got this question correct. A more challenging question asking students to create a chart showing the favorite sports of a group of people was answered correctly by only 30 percent of students. The comparisons to other countries are helpful to situate performance in an economic and developmental context, but ultimately the Palestinian Territories needs to significantly improve the absolute level of student achievement. 30. Student learning outcomes in the Palestinian Territories can be characterized along three dimensions: (i) performance level: overall low student learning levels when compared to international averages; (ii) equity: high variation in student learning outcomes between schools; and (iii) variation over time: fluctuations in test scores between different administrations of TIMSS. 31. Performance level: Poor student performance in the Palestinian territories is a persistent issue. Although Palestinian students improved their learning outcomes between 2007 and 2011 as measured by TIMSS, eighth grade Palestinian students performed below the international standardized average of 500 and below the 2011 participating countries’ averages in science (473) and math (462). Student outcomes in the Palestinian Territories are at or below the lowest International Benchmark in math and science,13 which puts them at a level of performance comparable to Syria, Qatar, Oman and Saudi Arabia. MENA countries such as Jordan, Lebanon and the UAE have higher student performance, yet they also do not reach the average international benchmark. 12 The TIMSS low benchmark measure students who are able to show some knowledge of whole numbers and decimals, operations and basic graphs. Student that reach the TIMSS high benchmark are able to apply knowledge and understanding in a variety of relatively complex situations while those in the advanced level are able to reason, draw conclusions, make generalizations and solve linear equations. 13 TIMSS defines the Low International Benchmark in science as students are: capable of recognizing some basic facts from life and physical science. In math it is defined as: students have some knowledge of numbers and decimals, operations, and basic graphs. 13 32. Equity: Another issue faced by the MoEHE in the Palestinian Territories is high levels of variance in student performance, a type of score dispersion, which raises serious concern as it indicates a lack of equity of outcomes among students. Education is potentially an equalizing force, allowing all students access to similar levels of achievement. However, students often begin school with inequitable levels of achievement,14 and these differences can be either augmented or lessened by the school system. In the Palestinian Territories, the magnitude of difference between the top 10 percent and the lowest 10 percent in prior years were large for both math and science student scores and are large compared both within the region and internationally (see Graph 1). Graph 1 – Score Dispersion TIMSS 2007 Science 350 High Dispersion & Qatar High Dispersion & High Scores Difference in mean score between top and low 10 percent Low Scores 300 Palestinian Territories Ghana Singapore Egypt 250 Botswana Romania Malaysia 200 Saudi Arabia Japan Korea El Salvador 150 100 Low Dispersion & Low Dispersion & Low Scores High Scores 50 300 350 400 450 500 550 600 Mean score 14 In 2011 only 38% of eligible students attended some form of preschool in the Palestinian Territories (PT) and most of them attended private institutions staffed by inexperienced teachers with only secondary education credentials. The MoEHE only runs 3 preschools in the PT while the private sector runs the rest (1,132). ANERA, 2013. 14 33. Equity can be conceived of as the absolute difference in performance and the distribution of students between the two extremes. Eighth grade math results for TIMSS 2011 show that about 48 percent of students score “below the low” benchmark and about 7 percent perform at the “high” or “advanced” benchmark (Graph 2). In 8th grade science about 41 percent of students score “below the low” benchmark, 27 percent score at the “low” benchmark, 23 percent at the “intermediate,” and only 10 percent at the “high” and “advanced” international benchmark. It is of concern that almost half of the student population is performing at “below the low” international benchmark. This suggests that high-performing schools are exceptions and that the majority of students in the Palestinian territories are participating in an education system that is not equipping them with the needed knowledge and skills to compete and in perform in the labor market or to participate and contribute to local and national institutions. 34. Other countries in the region have similar equity challenges. For example, in Jordan, 45 percent of students scored “below the low” benchmark and 6 percent at the “high,” none reached the “advanced” benchmark. In Lebanon, the figures were 27 percent for the “below the low” benchmark, and a combined 9 percent reached the “high” and “advanced” benchmark. This contrasts with high-performing education systems that produce students who perform at the advanced or high benchmarks, and have schools with much less variation in their ability to produce high achieving students. 35. Variation over time: The examination of Palestinian TIMSS results on three different rounds of student assessment shows large fluctuations on test results between each round (Figure 2). In TIMSS 2003, the performance of 8th grade Palestinian students in science was among the highest in MENA (average 435 points). However, the performance on 8th grade math (average 390 points) put Palestinian student performance second to the bottom of the range. On TIMMS 2007, outcomes of Palestinian students went down in both 8th grade subjects, decreasing to an average score of 404 in science and 367 in math. In 2011, Palestinian student achievement levels went up to an average of 420 in science and 404 in math. While some variation is expected, and indeed improvements are hoped for as a result of different interventions, this level of variation is considered high. 15 Figure 2: Palestinian student's mean performance in eighth grade 450 400 350 300 2003 2007 2011 Math Science Figure 3: TIMSS: Mean performance on the mathematics scale for eighth grade students 500 400 300 200 100 0 Bahrain Algeria Egypt, Jordan Lebanon Oman Qatar Saudi Syrian Tunisia United West Arab Arabia Arab Arab Bank Rep. Republic Emirates and Gaza 2003 2007 2011 36. While student socio-economic status plays a role in learning outcomes, it is not determinative. The study sample was stratified based on urban and rural school locations, which are reflective of poverty levels and socioeconomic status. This stratification reduces the chances that measured differences in student performance are due to socioeconomic status rather than school-based factors. In addition, low and high performing schools were selected across the strata and across districts on population size and number of schools. 37. For reasons of time and cost, this study did not gather information from individual students and parents to control for socio-economic status. We did ask School Directors to rate the poverty level of their students using the same methodology as TIMSS 2011. Using this subjective measure, there is only a very modest change in the values and significance level of our findings (Annex 1). Further, a related analysis of the macro factors influencing student performance in Jordan and the Palestinian Territories shows that contrary to expectations, socio- economic factors do not play a determinative role in student achievement and can be overcome with effective instructional practices and school support (Patrinos et al, forthcoming). 16 38. To further isolate the role that income plays in student achievement for this sample, we have mapped 104 of the 120 schools for which GIS data was available on a poverty map of West Bank and Gaza, using data published in “Seeing Is Believing: Poverty in the Palestinian Territories” (World Bank, 2014) (Figure 4). As can be seen from the map, there is a dispersion of classrooms, with some high-performing classrooms in high-poverty areas and some low- performing classrooms in low-poverty areas, and vice-versa. While student-assignment to individual teachers and classrooms is likely to be non-random and there may be student income variations between schools in a given locality, the map provides additional evidence for the importance of teachers and schools in determining student learning outcomes in addition to economic considerations. Figure 4: School Mapping The locations of schools with high performing classrooms are represented by half-darkened circles; schools with low performing classrooms are represented by darkened squares within a square. The “Poverty Headcount Rate” or HCR is the proportion of individuals in each locality belonging to households that are poor. Higher concentrations are in darker colors. 17 III. How do teacher classroom practices vary across high and low-performing classrooms? 39. While there are many important non-school components to student achievement, the overall quality of a country’s education system cannot exceed the quality of its teachers .15 There is a strong body of evidence for teachers as the most important school-based factor for student learning. Teacher classroom practices are a determinant factor in how and how much student learning takes place. Results from a range of studies indicate that students tend to perform better when their teachers spend more class time on subject matter related academic activities and when teachers are able to keep them engaged in these activities as compared to teachers who do not (e.g. Kane, et al. 2010; Lavy 2012; OECD 2012; Stallings 1985, World Bank 2010). These findings align well with the intuitive conclusion that in order to learn in school, students and teachers must be engaged in meaningful learning processes. (i) Putting Palestinian teacher time use into context: regional comparisons 40. Overall, Palestinian classroom time-on-task compares very favorably to other non- MENA countries for which data is available (Figure 5). Indeed, at 90 percent across all types of schools surveyed, Palestinian teachers have the highest proportion of time devoted to academic activities recorded in the Stallings observation literature reviewed by the team. The total average teacher time dedicated to social interactions or time outside the classroom was one percent. However, these results are less exceptional when compared with other similar observations carried out in the region. For example, in Jordan, authors found that teachers were off task just over 5 percent of the time, and in Morocco the figure was slightly lower, with both reporting that “at almost no time were teachers observed to be outside the classroom.”16 Relatedly, Abadzi reports teacher engagement in a sample of classrooms for an average of 87 percent of the class time in Tunisia and 83 percent in Morocco (Abadzi 2009). 15 Original quote: “The quality of an education system cannot exceed the quality of its teachers,” from an un -named South Korean education official quoted in Barber and Mourshed, 2007 16 Student Performance in Reading and Mathematics, Pedagogic Practice, and School Management in Jordan, RTI 2012 and Student Performance in Reading and Mathematics, Pedagogic Practice, and School Management in Doukkala Abda, Morocco, RTI 2012. 18 Figure 5: Classroom Time-on-Task17 100% 90% Percent of Total Class Time 90% 85% 80% 70% 64% 64% 65% 61% 60% 52% 55% 50% 39% 40% 32% 28% 27% 25% 30% 24% 20% 13% 15% 9% 9% 11% 10% 12% 9% 10% 1% 0% 0% Palestine Mexico Peru Jamaica Brazil Honduras Colombia Stallings D.F. Good Practice Indicator Academic Activities Classroom Management Teacher Off-Task 41. The Hawthorne effect, whereby individuals perform better when they are being observed, likely contributes to these high results. However, the amount of academic time use found in the results for the Palestinian Territories and other countries in the region is so much greater than those for other observations that were also subject to the Hawthorne effect as to suggest a true underlying difference. These results indicate that teachers in the Palestinian Territories and other countries in the region are focused on instruction despite modest levels of student achievement, and lead us to examine what instructional practices are used. (ii) International Comparisons 42. The Stallings data from the Palestinian Territories can be interpreted in the context of other data recently published from Latin America (Bruns et al. 2014) and elsewhere. This is primarily a comparison of convenience, since while Stallings and related tools are widely used to observe teacher classroom practices around the globe, there are relatively few countries that have publically available data for comparison. Palestinian scores on TIMSS 8th Grade Math exceed those of several Latin American countries included in the Bruns data set. For example, Palestinian students achieved an average score of 404 on the 8th Grade Math section of TIMSS 2011, while Hondurans achieved an average score of 338,18 putting Honduras approximately 2/3 of a standard deviation below the Palestinian Territories, despite the fact that the two countries’ GDP per capita are very similar. Colombian students achieved a score of 380 on TIMSS 8th Grade Math in 2007, well below the Palestinian 2011 score, while its GDP per capita is almost three times larger.19 These differences in international student achievement correlate with 17 The “Good Practice Indicator” is a constructed benchmark for the United States – Bruns, Filmer and Patrinos 2011 18 In this instance 9th graders were tested in Honduras, putting them at a slight advantage in comparison to Palestinian students. 19 Colombia and Honduras have 2011 per capita GDP of $7,149 and $2,262, respectively, in current USD (WorldBank DataBank 2014), while Palestinian GDP per capita is $2,489 in current dollars, according to the Palestine Central Bureau of Statistics (2014). 19 differences in teacher classroom practices: the average amount of time spent on instruction by Palestinian Authority teachers exceeded all available measurements for countries in Latin America as well as the best practices guidelines, which likely explains a portion of why Palestinian students performed better than Latin American countries that participated in TIMSS. What we seek to uncover is given the magnitude of the difference in class time use, why aren’t Palestinian students doing even better? 43. The graph below (Figure 6) disagregates total classroom time off task into three components: (i) teacher social interaction with students, where the teachers are interacting with students about non-instructional issues; (ii) teacher social interactions with other staff or visitors; and, (iii) teacher absence from the classroom during class time. While the relatively high use of classroom time for instructional activities is consistent across low and high perfroming schools in the Palestinian context, a large variance in terms of student performance was observed. The key question examined below is wheather teachers in schools with higher levels of student performance use time allocated to teaching differently than those in schools with lower levels of student performance, and if so, how. Figure 6: International Comparison Teacher Time-on-Task 14% 13% Teacher out of 12% the Room 11% 10% 9% Teacher Social 8% Interaction or 7% Uninvolved 6% Teacher Social 5% Interaction with 4% Students 3% 2% 1% 0% Palestine Brazil Honduras Jamaica Colombia Peru Mexico D.F. (iii) Successful Practices within the Palestinian context 44. Not surprisingly, some of the variance in student performance is correlated with differences in observed teacher classroom practices. For example, in high-performing public school classrooms observed for this study, teachers dedicate 34 percent of class instructional time to the “discussion/debate/question and answer” category on average, while in low- performing public school classrooms the student discussion category is 19 percent (Figures 7 and 8). The average non-participation of students is 11 percent in high-performing public school 20 classrooms and 20 percent in low-performing public school classrooms. These and other teacher mediated factors, including the allocation time to the “practice and drill” category are correlated to and likely contribute to the variation in student achievement. Teachers who invest more classroom time in student discussion and are able to engage more of the students are, on average, more likely to have higher achieving students. This is true for the grade four data presented below in Figures 7 and 8, and some of the trends hold true for higher grades for which Stallings data was collected. However, it is not simply that teacher-centered approaches are ‘bad’ and student discussions and other student-centered approaches are ‘good’ in terms of raising student performance. Rather, it appears from this evidence at the 4th grade level that a mix of instructional practices biased in favor of student discussions and interaction is likely to produce the highest learning outcomes. We find support for this conclusion in other research which finds involving students in the class and in the construction of knowledge is associated with higher levels of learning, though the evidence is limited. The use of discussion as an instructional tool for science has been examined in other educational contexts (e.g. Newton et al. 2010, World Bank 2010), but there is very little evidence to show what types of discusions are the most effective in raising student achievement in different grades and subjects. Figure 7: Teacher Classroom Figure 8: Teacher Classroom Activities Low Activities High Performing Public Performing Public Grade 4 Grade 4 Reading, listening, repetition 14% 14% Discussion 32% 33% 11% 20% Tasks and chores 10% Non-participation 15% 19% Classroom Management 34% 45. The pattern of greater class time devoted to discussion and related activities in higher performing classrooms applies to higher grades in public schools as well (Figures 9, 10). For grade 10, teachers in low-performing classrooms use this approach 18 percent of the time, while teachers in high-performing classrooms use discussion 34 percent of the time. However, there is a convergence in student non-participation between low and high peforming classrooms at rates lower than either category at grade four; teachers in high and low-performing classrooms have an average student non-participation rate of 5 and 6 percent in grade 10, respectively. 21 Figure 9: Teacher Classroom Activities Figure 10: Teacher Classroom Activities High Low Performing Public School Performing Public School G10 Classrooms G10 Reading, listening, 5% 7% 6% repetition 6% Discussion 35% 47% 20% Tasks and chores 22% Non-participation 18% 34% Classroom Management 46. When the data is disagregated by subject, the differences for student non-participation and discussion variable become very large for Arabic, but not so for the other subjects. There are large observable differences in teacher classroom practices in Arabic language instruction across grades 4, 8 and 10, particularly in the amount of discussion (Figure 11). One interpretation of these trends is that greater use of discussion-based pedagogic approaches may decrease both student non-participation and decrease the amount of time required for classroom management. Figure 11: Teacher Classroom Activities Arabic Instruction 40% Low-performing Low classroom Performing Public School 35% Arabic Instruction Arabic (public school) High Performing Public High-performing School classroom 30% Arabic Instruction Arabic (public school) 25% 20% 15% 10% 5% 0% Reading, listening, Discussion Tasks and Non-participation Classroom repetition activities Management 47. The use of discussion as a pedagogic tool is correlated with higher peformance in Arabic classes across grades, but this does not hold true for other subjects, as there is much less variation in the allocation of teaching time between high and low-performing classrooms in the two other subjects examined, Science and Math (Figures 12 and 13). The reason for this is unclear and the directionality of some of the differences is the opposite of expected (discussion in science), while the magnitude of difference remains small. This finding may be related to the 22 complexity and specificity of teaching, where certain pedagogic approaches are effective for specific subjects but may be less effective for other subjects and may vary in effectiveness by grade. For example, high quality science instruction is assumed to include a significant protion laboratory and other practical work. An alternative explanatory factor could be lack of teacher mastery of content,20 a concern particularly in higher grades, though this study did not gather data on teacher subject knowledge. It is also possible that since Palestinian scores on national and international assessments are on average lower for math and science than Arabic, it may be that math and science are taught ineffectively even in the relatively high-performing classrooms. Finally, while not captured with these instruments, the quality of the discussion or other pedagogic activity is likely important in addition to the absolute quantity of that activity (World Bank 2010, among others). Put another way, low-quality discussions are likely to have as little impact on student learning as low-quality teacher-centered lectures; high quality lectures are likely preferable to low-quality student-centered discussions. 48. Other research in the region has found teacher-centered pedagogy to be prevalent. For example, when looking at UNWRA schools in West Bank, Gaza and Jordan, Hardman (2013) found that teaching practice was “largely teacher-fronted talk made up of teacher explanation, teacher-led recitation, cued elicitations and use of chalk/white board.” Elementary teachers were observed using predominantly teacher-centered lecture approaches, rarely asked open ended questions and only seven percent of observed teachers consistently provided feedback to students on oral responses (Hardman, 2013). Figure 12: Teacher Classroom Activities Math Figure 13: Teacher Classroom Activities Science Low Low PublicSchool Performing Public Low Performing Public Perfoming Public School School Math Classroom Low Performing School Science Instruction Classroom Instruction (Science) (Math) High Performing Public School Classroom High Performing Public School High Performing Public School Math High Performing Public School Science Classroom (Science) (Math) Instruction Instruction 40% 60% 35% 50% 30% 25% 40% 20% 30% 15% 20% 10% 5% 10% 0% 0% 20 Marshall & Sorto, 2012; Hanushek and Rivkin, 2006, Hill, Rowan, and Ball, 2005; Eide, Goldhaber, and Brewer, 2004; Metzler & Woessman, 2012 all find positive relationships between teacher subject knowledge and student performance, however, the size of this impact is generally less than 0.1 SD in student scores 23 49. In summary, although almost all Palestinian teachers teach during class time, the use of this time varies widely. There are large differences in the amount of time teachers allocate to discussion, and in the levels of student engagement, particularly in the lower grades. Some of these differences persist in upper grades, particularly in Arabic pedagogy. In the next section we expand our view beyond the classroom to examine the context in which teachers teach and how they are managed and supported. IV. School leadership, teachers and school facilities and their influences on student learning outcomes 50. There is a growing body of evidence on the impact and importance of school leadership (e.g. Grissom et al. 2013, Bruns et al. 2011). Using a survey tool, we investigated the perceptions and practices of school leaders and teachers across the domains of teacher interactions, preparedness, satisfaction and support by Supervisors and School Directors as well as the availability of learning materials and the quality of school facilities. In interpreting the results, we are mindful of the potential for reverse-causality and self-reinforcing relationships. (a) Teacher interactions 51. For the purposes of this study, teacher interaction encompasses three elements: i) teacher interactions with parents, ii) interactions between teachers, and iii) teacher interactions with students. When comparing practices in schools that house high and low-performing classrooms, we find large differences in behaviors and perceptions. This study was not designed to make causal claims, for example that higher teacher satisfaction causes improved student learning, since indeed improved student learning could be a cause for increased teacher satisfaction. Nonetheless, there are clear associations between particular behaviors and student learning outcomes which are likely to be useful in explaining differences in student performance in the Palestinian Territories. Interaction with parents and high parental involvement is correlated with higher student achievement 52. Not surprisingly, high parental involvement is positively associated with higher student learning outcomes. According to responses from teachers in high- Figure 14: Parental support to students performing classrooms, parental involvement in school activities is High performing Low performing reported to be at 45 percent, compared 80 with 11 percent parental involvement in 60 school activities at low-performing 40 classrooms. Directors of schools report 20 that when parents are informed about 0 school performance, school rules, or are Parental Parental support General parental given advice to support them when participation in for student support for helping their children with homework, school activities achievement students they are more likely to engage with their 24 children and encourage them to perform better. School directors with high-performing classrooms report other important areas for parental involvement including communication about children’s performance and behavior, workshops for parents and promoting parent-teacher conferences. Cooperative experiences and meaningful forms of teacher cooperation can impact student learning outcomes 53. Recent evidence from other contexts suggests that cooperative and collaborative experiences such as observing peers, mentoring, peer coaching or participation in a network of teachers with a common objective is a more effective form of teacher learning and development than simple exchanges of knowledge for example sharing lesson plans (OECD, 2012). Based on international data from the Teaching and Learning International Survey (TALIS) in 2008, more traditional forms of teacher cooperation, such as the exchange of knowledge, which includes attending workshops or seminars and exchange of general concerns of teachers, is most prevalent across education systems globally but is likely less effective in improving teacher performance in the classroom than other more collaborative approaches (OECD, 2013). 54. Based on the results of the teacher survey in the Palestinian territories, teachers largely cooperate through the less effective exchange of knowledge, which places them in line with the practices of many education systems, with some large differences in prevalence between high and low performers. Eighty-five percent of teachers in high-performing classrooms discuss and coordinate course homework with other teachers at least once a week compared to 20 percent of teachers in low-performing classrooms. Targeted cooperation for specific student age groupings happens at least once a week for 21 percent of high-performing teacher classrooms and only 4 percent in low-performing teacher classrooms, according to teacher self-reports. Figure 15: Teacher cooperation High performing Low performing 90 80 70 60 50 40 30 20 10 0 Teacher Teacher Cooperation Teacher Teacher Teachers who discussions cooperation between meetings exchange on discuss about based on teachers (attending at pedagogy (at selected student student age teaching the least 1 per least once topics in homework groups same grade week) per week) education (1 or subject to 3 times per week) 25 55. The results of our teacher survey demonstrate that teachers in high-performing classrooms invest more of their time interacting with fellow teachers through activities such as discussions on courses and teaching techniques, coordination of homework and co-teaching than teachers in low-performing classrooms. This suggests that teacher cooperation and collaboration has not become an institutionalized practice in Palestinian schools, and that the quality of the cooperation taking place could emphasize more cooperative and collaborative activities. Teacher satisfaction could potentially be an influencing factor in determining the level of teacher cooperation and interaction. 56. Evidence from teacher surveys reveals that teachers in high-performing classrooms are more likely to be satisfied (90 percent) than teachers in low-performing classrooms (70 percent). Additionally, only 45 percent of teachers in low-performing classrooms believe that teaching is a satisfactory profession, compared to 70 percent of teachers in high-performing classrooms. However, linkages between teacher job satisfaction and on the job cooperation were not examined within this study, making it difficult to draw any conclusions. Teacher education qualifications combined with training have the potential to improve student-learning outcomes 57. Higher levels of education for teachers and school directors were associated with higher student performance levels: low-performing classrooms had eight percent more directors and teachers without a Bachelor’s degree equivalent. While it is promising that many teachers and directors in the Palestinian Territories hold university and college level degrees, it is important to note that educational qualifications are only a Figure 16: Minimum Education Qualifications part of the equation High performing Low performing towards becoming an effective teacher. 50 Evidence from other 40 contexts suggests that the 30 role teacher certification 20 and additional 10 qualifications play in 0 improving student Teacher holds a Teacher holds a 2-year Principal holds a 2- outcomes is less university teaching college degree year college degree important than the quality certificate of these programs, particularly the inclusion of practical learning experiences (Boyd et al. 2009, Yoon et al. 2007). 26 58. As part of their professional development, teachers in the Figure 17: Teacher professional development Palestinian Territories participate in participation training activities but the level of participation varies between the High performing Low performing high and low-performing 100 classrooms. The best practice identified was in UNRWA schools 80 with high-performing classrooms, 60 where teachers consistently report higher teacher training enrollment 40 rates. For example, 68 percent of 20 UNRWA high-performing teachers enrolled in sessions on handling 0 individual student needs in the last General Student Individualized ICT Pedagogy two years compared to 33 percent Training Assessment Instruction of non-UNRWA low-performing teachers. The large variation in participation in trainings is also likely related to multiple factors including the availability of trainings, location of the trainings, teachers’ workload, directors’ attitudes towards trainings and the release time given for teachers to participate. Performance-based visits by Supervisors 59. The conceptualization of teacher supervisors is changing across many education systems. The role is shifting away from a purely authority-based relationship with a strong focus on accounting for progress though the official curriculum toward a relationship that includes mentorship, support and cooperative learning. This is the case in the Palestinian Territories, where the title of “Inspector” was changed recently to that of “Supervisor” to reflect this new conception. Our study found that supervisors visit schools with low-performing classrooms more often than high performers: 54 percent of teachers in low-performing schools report that they were visited 2-4 times per semester by a supervisor compared to 26 percent of teachers in high-performing schools. This appears to be an appropriate allocation of scarce resources to those schools most in need. Since supervisors visit both types of schools, they could be a conduit for transferring effective practices between classrooms and schools. Teacher Practices and Teacher Performance Standards 60. The classroom observations we conducted did not explicitly use the Palestinian Teacher Standards as a rubric. However, we thought it would be relevant to briefly link the current discussion of teacher classroom practices to these standards. A set of seven teacher performance standards have been identified by the National Institute for Educational Training (Table 3). Overall, our evidence suggests uneven compliance levels towards these standards, with higher achieving classrooms doing better than others. 27 Table 3: National Teacher Performance Standards21 National Teacher Performance Standards (date) 1 Planning for school improvement based on well-defined vision and mission 2 Engaging with internal and external stakeholders to facilitate the performance of school duties 3 Maintaining the school’s financial and material resources 4 Maintaining a high level of quality teaching and learning 5 Building a friendly school environment for the students to motivate learning and build stronger bonds to the school 6 Utilizing various assessment tools to support improved student learning 7 Utilizing technology in teaching, learning and school administration 61. The first of these Standards is planning for school improvement based on a well-defined vision and mission, where teachers are expected to first participate in the process of creating and publicizing the school’s vision and mission. Results from the surveys conducted indicate that few teachers, whether in high or low-performing classrooms, actively promote or participate in the process of school mission and vision development. Evidence we collected does suggest that teachers are spending time to establish internal and external relations (Standards Two and Three) that are ultimately helping the school improve performance. This is evident by teacher communication with parents, and by internal communication between peers, which is higher for teachers in higher-performing classrooms. 62. Teachers report significant time spent on collaboration and cooperation, especially in high-performing classrooms (Standard Four). Teachers are interested in professional development opportunities but may not necessarily translate that into updated pedagogy or effective teaching in the classroom. Building a school friendly environment for the students (Standard Five) is a task that begins with the school director and includes teachers and students. The resources and school materials need to be maintained and preserved throughout the years of usage, and the culture of creating a safe school environment is a challenge as teachers and students report issues with theft, misbehavior and vandalism. 63. Standard Six is related to student assessment and is supported through practices such as the national examination system, which was used to help identify classrooms for this study. At the classroom level, student assessment is considered essential to effective lesson plan design and is a key feedback mechanism for teacher performance. This area could be strengthened across grades and schools. Similarly, Standard Seven, the use of ICT in schools and by teachers is limited. Low-performing classrooms utilize technology at an average of two percent compared to five in high-performing classrooms. This is likely a reflection of the absolute availability of ICT resources in addition to pedagogical choices made by the teachers. 21 Authors’ translation from The National Institute for Educational Training (2010), School improvement based on standards: The Palestinian standards of effective schools. Ministry of Education and Higher Education. 28 (b) School Leaders School Leaders can be Instructional Leaders 64. School directors hold multiple roles that span human resources, financial management and academic leadership (Hans and Brewer, 2013, Grisson, Loeb, and Maseter, 2013, Branch G. F., Hanushek E. A., Rivkin S. G, 2012). This broad workload can mean that directors focus on immediate needs in the administrative side of their work while their role of academic or instructional leader is neglected (Darling-Hammond et. al, 2007). This trend is apparent in the Palestinian Territories, where school directors report that they are over-stretched and have little time to focus on quality classroom instruction. This is a challenge that is shared with many other school systems. In the Palestinian Territories, surveyed directors reported issues that they saw as limiting student learning outcomes, three of which are directly linked to their roles as school leaders: technical direction, external communication and administrative management. 65. First, directors report that their role as a technical director includes a range of tasks such as evaluating teacher performance, knowledge and understanding of subjects through evaluative measures, management of professional development courses, and the management of other non- academic activities. Second, directors have a role in assuring external communication with parents and other stakeholders includes informing parents about teachers’ accomplishments and the school’s academic level, and organization of external events. Third, school directors have an administrative management responsibility and are required to manage and allocate school resources. Based on this information, it is clear that School Directors have the potential to influence teacher performance and student learning outcomes in their schools. It also appears that there may be gains from delegating some of the responsibilities of School Directors to others (e.g. administrative functions), in favor of a focus on instructional leadership. 66. The greatest potential student learning gains are likely to come from increasing the instructional leadership element of school management. There is some evidence from other contexts that effective school leadership leads to higher student learning outcomes as directors increase the time and energy devoted to support improvement of instruction (e.g. Darling- Hammond et al, 2007, Reitzug et al., 2008). However, these types of changes need to be coupled with support to School Directors, both to train them on how to be effective instructional leaders and the authority and resources to delegate some of their other functions. School directors and student behavior 67. School directors indicated the following issues affecting student performance: attendance, cheating on exams, theft among students, verbal abuse among students, physical abuse among students, expulsion, schools seeking community assistance in resolving behavioral issues. Teachers report varying rates of disruptive students in the classroom, vandalism, and rule adherence. In all cases, lower student performance is associated with higher levels of student misbehavior. It is likely that each factor contributes to the other. 68. Student behavior was identified in the analysis as strongly negatively associated with high levels of student learning outcomes. Variations in student learning outcomes between high and low-performing classrooms in schools are apparent and school directors indicated that 29 misbehavior on school premises is connected to poor student learning outcomes across classroom types. In addition, directors in schools with low performing classrooms reported spending a larger amount of time managing student misconduct, which ranged from skipping school to vandalism of school property. 69. Classrooms with low levels of student performance are located in schools which suffer from a higher rate of disciplinary and behavioral problems among their student populations. Violence and the prevalence of physical assault, verbal abuse or derogatory language is almost double the rate present in schools with high-performing classrooms. It is not surprising that teachers in schools with lower performing classrooms are dedicating more of their time to classroom management and containment of problematic and disruptive behavior. We presume that the relationship between student performance and instruction is dynamic (e.g. low-quality instruction can contribute to student misbehavior and student misbehavior can contribute to low- quality instruction). Figure 18: Student behavioral issues High performing Low performing 90 80 70 60 50 40 30 20 10 0 Cheating Verbal abuse Physical Expulsion School seeks Disruptive Vandalism Students among abuse among community students in adhering to students students assistance the school rules with classroom behavioral issues Poor facilities and a lack of school resources are a distraction 70. School directors are unified in their dissatisfaction with the level of financial resources and physical facilities of their schools, and link these shortcomings with poor student learning outcomes on surveys. Nonetheless, the quality, availability and use of school resources and facilities vary across schools in the Palestinian Territories. Three key issues were identified in relation to school facilities. 71. First, schools that do not maintain regular and consistent communication with the MoEHE, particularly when placing a request for maintenance or new facilities, are less likely to have extra facilities such as a multipurpose room, space for physical education and other sporting activities, or green spaces within the school grounds. Second, teaching and learning tools used by schools varies across schools in association with the level of performance, as well as the presence of the school mission statement at the entrance and displays of student work on the walls. Third, higher performing classrooms are more likely to be located schools that maintain 30 their school facilities and show pride in student achievements by displaying the school ranking. (Annex 1: School Facility survey analysis). V. Conclusion: Policy implications and next steps 72. This study set out to examine areas for improvement in low-performing classrooms based on promising practices in high-performing classrooms to inform the Education Development Strategic Plan 2014-2019. Palestinian children have a school enrolment rate that is high relative to other developing countries and Palestinian teachers use instructional time better than other contexts for which comparable data is available. Why then are their achievement levels so modest? 73. This study found that more effective teachers tend to use more discussion in the classroom, have a higher proportion of students engaged in learning, receive greater support and conduct more outreach to parents. We found statistically significant relationships between classroom performance and the following parameters: teacher accountability, school culture, parental engagement, school facilities, school mission and student recognition, student involvement and student-centered pedagogy. While we make no causal claims, it is reasonable to predict that practices such as engaging more students in learning are more likely to lead to higher learning outcomes than not. 74. The findings of our study in and of themselves are not surprising, and align well with research from other countries and sources. What is different about our approach is that we have identified Palestinian teachers and school directors who already know how to implement these promising practices, and could share them with their colleagues. This contrasts with other approaches, many of which embrace an externally driven reform program. Instead of looking only to Shanghai or Helsinki, Palestinian educators can look to their peers for ways to improve student learning outcomes. Research suggests this peer-based localized knowledge is more likely to be used and translated into changed practices (Jackson et al. 2009, Birko 2004, Marsh et al. 2004). This is not to suggest that Palestinian educators have nothing to learn from international experience; on the contrary, there is a clear need to continually enrich teachers and school leaders. By combining what works locally with approaches for which there is good evidence internationally, Palestinian students should be able to significantly improve their learning outcomes. 75. The MoEHE has invested heavily in recent years to upgrade teacher skills, following the Palestinian Teacher Education Strategy (2008), which anchored the Palestinian Education Sector Development Plan (2008-2012). The plan focused on quality, with the core aim of improving teaching and learning in public schools through investments in pre-service and in-service teacher training. The policy response has included investments in these areas in recent years, for example the expansion of classroom experience during pre-service training, and a greater emphasis on practices as opposed to more theoretical approaches. However, teacher behavior is well established in research literature as difficult to change, and based on the data collected for this study, teacher classroom practices still have much room for improvement. We see the Teacher Education Strategy and prior Education Sector Development Plan as appropriate but as yet uncompleted agendas. The 2014-2019 strategy definition is an opportunity to strengthen and re-commit to the quality agenda through improved teacher practice and school leadership. 31 76. As a result of this and other research as well as their own experience, the Palestinian authorities are aware of the deficiencies in schools and classrooms. The current task is to design effective policy to implement changes that will positively impact student learning as soon as a part of the next Education Sector Development Plan. Based on the evidence collected and discussions with the MOEHE, we make the following recommendations: Teachers: I. Support teachers to use pedagogy that incorporates more active participation of students through high-quality discussions, debates and question and answer sessions, particularly in the lower grades and in Arabic language classes. Support this through school-wide initiatives that include peer-feedback between teachers and cross- observation of each other’s classrooms both within and between schools. Teachers should be skilled at a broad set of pedagogic approaches of which lecture and student- centered discussion are simply two examples. These approaches should be tailored to specific topics and student needs and the teacher should be able to choose among these instructional options to achieve maximum student learning. II. Organize school-based training of teachers and school directors responsible for promoting and facilitating cooperation among teachers within schools to support recommendation I, above. Mentoring could take place within and between schools for highly effective staff, and mentors and mentees could be incentivized through recognition and professional advancement. Prioritize lower performing schools for this assistance. III. Strengthen teachers’ knowledge and skills to design, conduct and analyze regular student assessment and to use this information for lesson planning and student support planning. Classroom assessment is the measurement of student learning. If students are not learning the material, classroom instruction should be adjusted in response. From this perspective, the assessment is not an end in itself, but a means to improve pedagogy. Encouragement and support for formative and summative classroom assessment can be provided through existing professional development programs, peer-to-peer teacher exchanges and supervisor evaluation and support. IV. Support active parent participation by requesting that key student assignments be shared with parents either before or after submission (with teacher feedback included). Design instruction to include parents, for example by asking students to include information gathered from family members. Reach out to parents for support and include them in student successes through notes, phone-calls or invitations to join occasional school-based activities. School Directors: V. Support school directors' to fulfil their instructional leadership role and shared responsibility for student learning, for example by providing for greater administrative support by other staff. School directors could be given greater autonomy to hire staff and assign responsibilities, and encouraged to conduct more parental and community outreach. At the same time, hold directors to a higher degree of accountability for student achievement, as measured by national exam scores and other school and student performance indicators. 32 VI. Promote the establishment of a school director's professional practice to stimulate joint learning and sharing of good practices. Directors should have personalized professional development plans, participate in professional development for teachers, as well as targeted training specifically for directors as part of a long-term professional development strategy. Stronger directors in higher performing schools could be paired with lower-performing ones in a peer-learning relationship. Directors should then be held accountable by the next level of administration for changing their practices as an outcome of this training and mentorship. VII. Special attention should be paid at the level of the classroom and school to the lowest performing students. School directors should be given the autonomy to allocate resources for tutoring and support of low-performing students though the use of existing teacher time, the use of school space for afterschool peer tutoring by other students, and by inviting parents or community members with appropriate expertise to undertake supervised tutoring of struggling students. Ministry: VIII. Improve MoEHE communication with low-performing schools through a program of regular meetings with school directors to review progress toward improvement. Mechanisms of improvement can include items presented above, as well as school improvement plans prepared by the school community (parents, students, teachers and school managers/administrators) under the guidance and leadership of the school director if funding is available. IX. Track school and student performance over time through national and international assessment to sustain accountability for results. Student performance should be measured through both national and international assessment, and results should be fed back to schools and teachers so they can be supported to adjust their practices to respond to specific performance gaps. X. Establish a school exchange program that focuses on cross-fertilization and sharing of good practices and joint learning between schools. This could be done within neighborhoods, so that lower and higher performing schools within the same geographic zone could learn from each other, and involve teachers as well as school directors. 77. One aspect highlighted in the MoEHE report is that of low-performing boy’s classrooms; 86% of low-performing classrooms were male-only. We were unable to address the gender issue on its own due to sample size limitations. Nonetheless, we believe it is likely that more effective classroom instructional practices will benefit boys as well as girls. A policy approach that prioritized low-performing students and schools would necessarily include more boys’ classrooms than girls’. 78. There are multiple ways to achieve the overall objective of improved student learning, and the research team is in dialogue with the MoEHE on how best to design implementation mechanisms using existing programs and available resources. The overarching goal is to improve student learning using locally-available expertise, augmented by international experience. There 33 is a role for lessons from elsewhere in the region and around the globe, as even the highest- performing Palestinian students do not match those of the highest ranked countries. Nonetheless, there are grounds for optimism that many of the solutions to the challenges of education already exist in the Palestinian context, and that student learning outcomes can be improved using locally available solutions. In order to do this, the policy actions chosen by the Government will need to be supported by continuous backing and follow-up at all levels of the MoEHE. They will also need to be incentivized for example through links to salary increases and professional advancement, as improvement will not happen by itself. 79. The central premise of this research is that some of the practices associated with improved student performance are transferrable between teachers and schools, and we identify multiple mechanisms to support their broader implementation throughout the Palestinian territories. At the same time we recognizing that “evidence on what to change is not the same as evidence on how to change,”22 and that some of these areas have been identified well prior to this research. The next step is to agree on the policy initiatives and to design effective change processes that will lead to improved student learning. 22 Hargreaves and Fullan (2012) p. 48 34 Annex 1 – Logistic and Factor Analysis Results Model 1(without Model 2 (Controlling for controlling for Socioeconomic status of socioeconomic status at students)23 school) Factor Exp(B) Sig. Exp(B) Sig. Low 0.34 0.001 0.42 0.001 accountability School culture 2.23 0.002 2.22 0.001 Communication 1.81 0.013 1.73 0.020 and parental involvement School facilities 2.23 0.001 2.30 0.001 School mission, 1.62 0.043 1.84 0.008 tools and student recognition Uninvolved 0.02 0.023 0.02 0.020 students Student-centered 3.41 0.016 3.5 0.013 pedagogy Low Accountability Factor: lack of teacher monitoring, lack of support and evaluation policies (teacher survey) School Culture Factor: treatment of school property by students, student behavior policies, teacher expectations of student performance, level of teacher satisfaction, parent participation, low-income student support services (teacher survey) Communication and Parental Engagement Factor: the level of communication between the school and parents including on student performance, and parents’ participation in a broad range school activities (leadership survey) School Facilities Factor: student access to playgrounds, classroom space, library, computers (leadership survey) School Mission, Tools and Student Recognition Factor: defined as prominently displaying a school’s mission statement, displaying student work and activities, presence in classrooms other educational tools such as pictures, maps and other visual and instructional aids (facilities survey) Uninvolved Student Factor: Based on the classroom observations, this is the proportion of students who are not engaged in the lesson and the amount of time they are not engaged Student-Centered Pedagogy Factor: Also from the classroom observations, this is the proportion of instructional time during which during which students participate in discussions, debates and question and answer activities 23 Using the same methodology as TIMSS, we asked school directors the proportion of low-income students at their school. Controlling for income by this method does not a have a large impact on the value of the exponent. 35 Factor analyses were conducted separately for the three survey tools: school director interviews, teacher interviews, and school facility surveys. The Stallings data was analyzed separately. The results of the three factor analyses are as follows: Survey Tool 1: Standardized Interviews with school directors First factor: Student behavior No. Factor 1 Density of factor 1 Theft 0.814 2 Students verbally abusing teachers or employees 0.812 3 Physical assault on teachers or employees 0.791 4 Cheating 0.77 5 Physical assault among students 0.716 6 Leaving school early 0.705 7 Vandalism 0.704 8 Late students attendance 0.688 9 Delays in filling vacancies 0.662 10 Student absence 0.658 11 Teacher absence 0.639 12 Verbal abuse among students 0.617 13 Late attendance or early departure by teachers 0.314 Second factor: School Facilities Number Factor 2 Density of factor 1 Inadequate playgrounds and open space 0.799 2 Shortage of school buildings 0.769 3 Inadequate lighting .776 4 Lack of classrooms .737 5 Lack of facilities .706 6 Lack of stationery .676 7 Shortage in library contents (books for science, Arabic .644 language, and mathematics) 8 Lack of computers to teach science, mathematics, and .644 Arabic language 9 Lack of books at the beginning of the school year .64 10 Lack of computer software to teach science, mathematics, .538 or Arabic language 11 Inadequate heating and cooling systems .383 36 Third factor: School Director Responsibilities Number Factor 3 Density of factors 1 Relationships between teachers and students in school .689 evaluation 2 Teachers’ knowledge and understanding of their majored .671 subject based on school evaluations 3 Interaction between teachers and with the principal .654 during school evaluations 4 Teachers’ understanding of pedagogy based on school .628 evaluation 5 Teachers’ classroom management ability based on school .602 evaluation 6 Professional development teachers took during school .507 evaluation 7 Nonacademic student outcomes .503 8 Creativity during class activities in school evaluation .49 9 Promotion of the school’s mission and vision .46 10 Students evaluation of the knowledge they attained in .447 school evaluation 11 Extracurricular activities .442 12 Direct evaluation through classroom visits .436 13 Promoting the curriculum .433 14 Students succeeding classes and not leaving school .403 during school evaluations 15 An essential part of my work is to monitor rules .385 enforcement 16 Student scores in school evaluation .355 Fourth factor: Communication and Parent Engagement Number Factor 4 Density of factor 1 Informing parents about school accomplishments .773 2 Informing parents about school rules .655 3 Supporting parents to help their children with homework .652 4 Informing parents about the school’s academic .626 achievements 5 Discussing with parents their level of involvement with .601 their children 6 Organizing workshops and lectures for parents on .563 teaching and learning 7 Serving in school committees .533 8 Volunteering in school projects, programs, or trips .528 37 9 Informing parents about student grades .423 10 Informing parents about their children’s school behavior .338 11 Visiting other schools, or attending educational .332 conferences for new ideas Fifth factor: Administrative management responsibilities Number Factor 5 Density of factor 1 Do you participate in decisions regarding promotions and .772 raises 2 Do you participate in discussions regarding teachers’ end .729 of service 3 Do you participate in discussions regarding allocation of .724 teachers’ starting salary 4 Do participate in setting disciplinary rules in school .585 5 I can influence decisions taken on a high administrative .541 level 6 Do you participate in accepting students into the school .505 7 Initiating educational, or developmental projects .469 8 Asking for intervention by the local community in cases -.432 of student misbehavior 9 Do you participate in setting a budget for teacher’s .379 professional development 10 Do you participate in setting the school budget .334 38 Survey Tool Two: Standardized Interviews with Teachers. First factor: Accountability Number Factor 1 Density of factor 1 The Principal enforces financial penalties on teachers -.883 2 Poor performing teachers are overlooked -.846 3 Contracts of poor performing teachers are terminated -.677 4 Frequency of student examinations -.673 5 Number of supervisory visits conducted by supervisors .670 during the year 6 There is a plan to improve the performance of teachers .650 7 Using successful methods by the school director to .571 distinguish good and bad teachers 8 Reviewing the work of teachers to meet administrative .496 needs Second factor: Interaction between teachers Number Factor 2 Density of factor 1 Exchange of teaching methods between teachers .639 2 Discussing how to teach a specific subject .629 3 Observing classes of other teachers, and giving them .619 feedback 4 Cooperation, planning, and preparing .619 5 Giving my educational experience to others .690 6 Cooperating with others to try new ideas .564 7 Attending meetings on students of the same age group .546 8 Participating in training activates .498 9 Discussions with colleagues who teach the same grade .487 10 Discussing and choosing educational tools .483 11 Discussing achievement issues for a specific student .464 12 Contributing to the enrichment of the curriculum .428 13 Visiting other colleagues to improve performance q19.4 .389 14 Discussion and coordination of home work among .384 different subjects, or the same subject a29.10 Third factor: Training Number Factor 3 Density of factor 1 Participation in professional to improve student .792 performance in critical thinking and problem solving 2 Participation in professional development programs on .695 handling students’ individual needs 3 Participation on professional development programs .670 39 on evaluating classroom subjects 4 Participation in professional development on subject .644 pedagogy 5 Participation in professional development on subject .469 content 6 Level of usage of computers during class .460 7 Participation in professional development programs on .444 using ICT in the classroom 8 Extent of computer usage in lesson preparation .370 Fourth factor: School Culture number Factor 4 Density of factor 1 Students preserving school property .630 2 Teachers’ expectations of student attainment .576 3 Job satisfaction of other teachers .556 4 Parents support for students’ educational attainment .519 5 Level of careless students in the classroom -.508 6 Your job satisfaction .507 7 Inability of a student to recall knowledge previously -.478 acquired 8 The existence of special services for students with low .430 income 9 Parents participating in school activities .409 10 Disruptive students in the classroom -.395 40 Fifth factor: Incentives Number Factor 5 Density of factor 1 I get a reward for improving my performance .874 2 I get a reward if I have an innovative idea related to the .871 teaching process 3 Distinguished teachers receive rewards .772 Survey Tool Three: Facility Survey 1. First factor: Resource availability and use Number Factor 1 Density of factor 1 Availability of multi-purpose rooms in school .718 2 Availability of space for sports .61 3 Aesthetic environment in school .536 4 Using the computer for purposes other than Technology .524 class Second factor: School Mission, Tools and Student Recognition Number Factor 2 Density of factor 1 Teaching and learning tools .776 2 Posters made by students on the announcement board .708 3 Display of the school mission at the front of the school .617 Third factor: Maintenance and display Number Factor 3 Density of factor 1 Display of the school’s achievement level .804 2 Major maintenance needed for school building .678 41 Annex 2 Models of Teacher Learning and Behavioral Change 80. There are at least three key teacher characteristics which influence student performance that have been the subject of considerable research and interest: teacher subject matter knowledge, teacher understanding of how students learn, and teacher instructional practice (including classroom management). Clearly, other teacher characteristics are relevant to student learning, however, this study focuses on teacher instructional practice. This is the focus of the classroom observation instrument, and requires a discussion of how teacher instructional practices change. There is an established literature on teacher training which will not be reviewed here (see for example Yoon et al, 2007 and Villegas-Reimers 2003). We offer some key observations linked to teacher training in the Palestinian Territories, below. 81. The standard model of teacher instructional practice change is that investments in teacher knowledge through training leads teachers to improve their classroom practices, which then leads to improved student learning outcomes (Figure A). However, there is evidence that training programs often fail to change teacher classroom behavior, and further that even when they are effective at changing teacher behavior, improved student learning doesn’t necessarily follow (Yoon et al., 2007 and Kennedy, 1998). Guskey (2000, 2002) proposes an inverted model where training programs allow for classroom practice experimentation. He suggests that if these experiments are effective in increasing student learning outcomes, teachers then adjust their beliefs and attitudes accordingly to accommodate these new modalities (Figure B). A hybrid model whereby teachers are learning new information, trying it out, discarding some elements, adding others and continuously experimenting to see how it affects student learning has been discussed by Piper (2013). This mixed model appears the most convincing to the team, given the limited evidence on sequencing and causality (Figure C). Standard Model (schematic) Change in Teacher Change in Teacher Improvements in Student Teacher Training Knowledge Learning Practices 42 Guskey Model (schematic) Improvements in Teacher Change in Teacher Change in Teacher Beliefs and Student Learning Training Practices Attitudes (if successful) Dynamic Model (schematic) Teacher Training Teacher Teacher Practices Knowledge and Beliefs Student Learning 82. In the dynamic model proposed here, teacher training has an impact on classroom practices as well as teacher knowledge and beliefs, which are both influenced by classroom experience and student success or failure. Ideally, teacher training is responsive to teacher needs as reflected in their practices and understanding. Rather than simply telling teachers what to do, trainers work with teachers to better understand how they can improve, and accompany them through that improvement process. Some of these approaches are incorporated into current Palestinian policy and practice, and one use of this study is to help inform the design of pre- service and in-service development programs. 43 INSTRUMENTS Due to the size of the files, the instruments could not be attached to the report. However, they can be found in the portal and will be disclosed along with the final report. The instruments are: 1) Observation Tool 2) Teacher Questionnaire 3) Principal Questionnaire 4) Facility Survey 44 References ANERA (2013). Early Childhood Development in the West Bank and Gaza. ANERA Reports on the Ground in the Middle East: 5. Assessment and Evaluation Department (2013). “Lessons Learned from High Achieving Schools in Palestine”. Ministry of Education and Higher Education. Barber, M., Mourshed, M. (2007) How The World’s Best-performing School Systems Come Out on Top (McKinsey and Company). Boyd, D., Grossman, P., Lankford, H., Loeb, S., & Wyckoff, J. (2009) Teacher Preparation and Student Achievement. Educational Evaluation and Policy Analysis 31(4): 416-440. Branch G. F., Hanushek E. A., Rivkin S. G.(2012). Estimating the effect of leaders on public sector productivity: The case of school principals (Working Paper No. 17803). Cambridge, MA: National Bureau of Economic Research. Brookfield, S. D., Preeskil, S. (2005). Discussion as a Way of Teaching, Second Edition. Wiley and Sons Inc., San Francisco. Bruns, B, Filmer, D, Patrinos, H. A. (2011). Making Schools Work: New Evidence on Accountability Reforms. World Bank. Claussen J, Kiernan M. and Gramshaug R. (2013) Public Expenditure Review. Education Sector In Palestine, September, 2013 Clotfelter, C. T., H. F. Ladd, and J. L Vigdor. “Teacher credentials and student achievement: Longitudinal analysis with student fixed effects.” Economics of Education Review 26, 2007: 673-682. Darling-Hammond, L., LaPointe, M., Meyerson, D., Orr. M. T., & Cohen, C. (2007). Preparing School Leaders for a Changing World: Lessons from Exemplary Leadership Development Programs. Stanford, CA: Stanford University, Stanford Educational Leadership Institute. Eide, E., Goldhaber, D., & Brewer, D. (2004). The Teacher Labour Market and Teacher Quality. Oxford Review of Economic Policy, 20, pp 230-244. Filmer D., Pritchett L.H., (2001). Estimating Wealth Effects Without Expenditure Data or Tears: An Application To Educational Enrolments in States in India. Demography, February 2001, 38:1: 115-132. Freire, P., (2006). Pedagogy of the Oppressed. Continuum International Publishing Group, New York. (1st edition 1970). Grissom, J., Loeb, S., Master, B., (2013). Effective Instructional Time Use for School Leaders: Longitudinal evidence from observations for principals. Education research 42: 433-444, first published on October 25, 2013. http://dx.doi.org/10.1787/9789264123540-en 45 Guskey, T. R., (2002). Professional Development and Teacher Change. Teachers and Teaching: theory and practice, Vol. 8, No. 3/4. Hanushek E., Rivkin S., (2006). Teacher Quality. Handbook of the Economics of Education, Volume 2, Amsterdam: North Holland Pages: 1052-1078 Hargreaves, A., Fullan, M., (2012) Professional Capital: Transforming Teaching in Every School, Teachers College Press, New York. Hill, H., Rowan, B., Ball, D., (2005). Effects of Teachers' Mathematical Knowledge for Teaching on Student Achievement. American Educational Research Journal, Summer 2005, Vol. 42, No. 2, pp. 371–406 Kennedy, M. (1998). Form and Substance in Inservice Teacher Education. Research Monograph. Madison: National Institute for Science Education, University of Wisconsin. Klar, W Hans and Curtis A. Brewer (2013). Successful Leadership in High-Needs Schools: An Examination of Core Leadership Practices Enacted in Challenging Contexts. Educational Administration Quarterly December 2013 49: 768-808, first published on March 29, 2013 Longitudinal Evidence from Observations of Principals. Educational Researcher Volume/Issue: 42(8) Pages: 433-444. Lung, F. and Ragatz, A. (2010). Inside Indonesia’s Mathematics Classrooms: A TIMSS video study of teaching practices and student achievement. World Bank Group. Marshall, J. H. & Sorto, M. A. (2012). The Effects of Teacher Mathematics Knowledge and Pedagogy on Student Achievement in Rural Guatemala. International Review of Education 58(2): Pages 173 – 197. Metzler, J., and Woessmann, L., (2002). "The impact of teacher subject knowledge on student achievement: Evidence from within-teacher within-student variation." Journal of Development Economics, 2012 Pages: 486-496. Newton, P., Driver R., Osborne J., The Place of Argumentation in the Pedagogy of School Science. International Journal of Science Education,Volume 21, Issue 5, 1999 OECD (2009). Creating Effective Teaching and Learning Environments: First Results from TALIS. OECD Publishing. OECD (2013/14). Fostering learning communities among teachers. Teaching in Focus, OECD Publishing. Pascale R., Sternin J., Sternin M., (2010) The Power of Positive Deviance: How unlikely innovators solive the world’s toughest problems. Harvard business review press. Piper, B., (2013). Presentation Aug 6, 2013 for USAID Education Summit. 46 Pont, Beatriz, Deborah Nusche, D., and Hunter Moorman, H.. (2008). Improving school leadership, Volume I: Policy and Practice. OECD Publishing. Reitzug, U. and Deborah L. West, and Roma Angel (2008). Conceptualizing Instructional Leadership: The Voices of Principals. Education and Urban Society: 40 Page: 694- 714, first published on May 28, 2008. Schleicher, A. (2011), Building a High-Quality Teaching Profession: Lessons from around the World, OECD Publishing. http://dx.doi.org/10.1787/9789264113046-en The National Institute for Educational Training (2010), School improvement based on standards: The Palestinian standards of effective schools. Ministry of Education and Higher Education. UNESCO, World Data on Education, 7th edition 2010/2011, retrieved on 28 April 2014, http://www.ibe.unesco.org/fileadmin/user_upload/Publications/WDE/2010/pdf- versions/Palestine.pdf. Vieluf S., et al. (2012), Teaching Practices and Pedagogical Innovation: Evidence from TALIS, OECD Publishing. Villegas-Reimers, E. (2003). Teacher professional development: An international review of the literature. Paris: International Institute for Educational Planning. Vishwanath, T., Blankespoor, B., Calandra, F., Krishnan, N., Mahadevan, M., Yoshida, M. (2014). Seeing is believing: poverty in the Palestinian territories. World Bank Group. Wei, R. C., Andree, A., Darling-Hammond, L. (2009). How nations invest in teachers: high- achieving nations treat their teachers are professionals. Educational Leadership. Yoon, K. S., Duncan, T., Lee, S. W.-Y., Scarloss, B., & Shapley, K. (2007). Reviewing the Evidence on How Teacher Professional Development Affects Student Achievement. National Center For Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Zeitlin, M. (1991). Nutritional resilience in a hostile environment: Positive Deviance in Child Nutrition. Nutirion Reviews: 49(9) Pages: 259-268. 47