WPS5643 Policy Research Working Paper 5643 Education Outcomes, School Governance and Parents’ Demand for Accountability Evidence from Albania Danila Serra Abigail Barr Truman Packard The World Bank Europe and Central Asia Region Human Development Economics Unit April 2011 Policy Research Working Paper 0 Abstract The extent to which teachers and school directors are measures are correlated to various education outputs, held to account may play a central role in determining although they tend lose statistical significance once education outcomes, particularly in developing and parent characteristics, school resources and top-down transition countries where institutional deficiencies can accountability indicators are considered. An in-depth distort incentives. This paper investigates the relationship analysis of participatory accountability within the between an expanded set of school inputs, including schools focuses on parents’ willingness to hold teachers proxies for the functionality of “top-down” and “bottom- to account. Here, the survey data are combined with up” accountability systems, and education outputs in data from lab-type experiments conducted with parents Albanian primary schools. The authors use data generated and teachers in the schools. In general, the survey data by an original survey of 180 nationally representative highlight problems of limited parental involvement and schools. The analysis shows a strong negative correlation lack of information about participatory accountability between measures of top-down accountability and structures. The experiments indicate that the lack of students’ rates of grade repetition and failure in parental participation in the school accountability system final examinations, and a strong positive correlation is owing to information constraints and weak institutions between measures of top-down accountability and that allow parent class representatives to be appointed by students’ excellence in math. Bottom-up accountability teachers rather than elected by parents. This paper is a product of the Human Development Economics Unit, Europe and Central Asia Region. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at tpackard@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team Education Outcomes, School Governance and Parents‟ Demand for Accountability: Evidence from Albania Danila Serra1 Abigail Barr2 Truman Packard3 JEL Classification: O1, I2, C8, C9 Keywords: Accountability, governance, education, lab experiment, Albania 1 Department of Economics, Florida State University, dserra@fsu.edu (corresponding author) 2 Nuffield College and the Department of International Development, University of Oxford, abarr@economics.ox.ac.uk 3 Human Development Economics, Europe and Central Asia Regional Department, World Bank, tpackard@worldbank.org Page ii Acknowledgements The authors would like to thank the Albanian Ministry of Education and Science (MoES) for their interest and assistance during the preparation of this work, in particular to Mimoza Agolli (MoES Economic Director), and Floriand Nurce (Budget Chief) who worked hard to gather and provide important statistical data. Thanks are also due to the Institute for Development Research Alternatives (IDRA), who collected the data, and especially to Auron Pasha, Enkelejda Pashaj and Adela Gjergjani for their technical excellence, professionalism, and willingness to explore and apply cutting-edge data collection techniques in difficult circumstances and against pressing deadlines. This paper is a product of the World Bank‟s Accountability for Better Governance Program (P107759) of analytical advisory assistance to the Government of Albania, and was supported with financial assistance from the Governance Partnership Facility (GPF). The team received support and input from Lorena Kostallari (Senior Operations Officer, Human Development, World Bank) and Gentjana Sula (Operations Officer, Human Development, World Bank) who also managed important discussions and exchanges of information with the Government from the World Bank‟s Office in Tirana. Andrea Guedes (Senior Operations Officer, Human Development, World Bank) provided guidance to ensure the operational relevance of this work. Administrative and logistical support was provided from Washington, by Katarina Timina (Program Assistant, ECSHD) and in Tirana by Elda Hafizi (Program Assistant, ECCAL). The team worked under the direction of Tamar Manuelyan Atinc and Mamta Murthi (Prior and Current Directors, Human Development, Europe and Central Asia, World Bank); Jane Armitage (Country Director, Western Balkans, World Bank ); Gordon Betcherman and Jesko Hentschel (Prior and Current Sector Managers for Human Development Economics, Europe and Central Asia, World Bank); Camille Lampart Nuamah and Kseniya Lvovsky (Prior and current Country Managers, Albania, World Bank), and Daniel Dulitzky (Country Sector Coordinator for the Western Balkans, Human Development, World Bank). The Peer Reviewers were, Magnus Lindelow (Senior Economist, Human Development, East Asia and Pacific Region, World Bank), Mattias Lundberg (Senior Economist, Human Development Network, World Bank), and Waly Wane (Senior Economist, Poverty Reduction and Economic Management, Africa Region, World Bank), who provided helpful guidance at the outset of the AAA program and held the team to a high technical standard of quality. Special thanks are extended to Clelia Rontoyanni (Public Sector Specialist, Poverty Reduction and Economic Management, World Bank) for collaboration with the team during preparation of the Albania Country Governance Review, and for her suggestions that have strengthened this report, and to Dena Ringold (Senior Economist, Office of the Chief Economist, Human Development Network, World Bank) for ensuring consistency between this report and wider efforts in the World Bank‟s Human Development Network to improve governance and accountability in the social sectors. Page 1 Part I. Introduction, Motivation & Summary of Findings I.a. The Importance of Accountability for Better Goveranance and Services 1. Strengthening governance in the social sectors is critical to improving performance and the impact of services on the welfare of families. Although poor governance in other areas of public administration –such as management of the wage bill, revenue collection and procurement- affects households as well, governance failures in the social sectors arguably impose a heavier burden. Households -particularly families with children- interact with health and education service providers more frequently than they do with other public services. Household demand for education and health is arguably more inelastic than demand for other services. And the burden of governance failures in the social sectors is likely to be heaviest on the poorest households, for whom the costs of privately-provided alternatives can be prohibitive. 2. Good governance requires key enabling conditions, the most important of which is a functioning structure of accountability – a public good that the state cannot provide on its own. “Accountability” refers to “the act of holding public officials and service providers answerable for processes and outcomes, and imposing penalties if specified outputs and outcomes are not delivered” (Lewis and Petterson, 2009). Effective accountability therefore requires the establishment of clear rules and responsibilities, monitoring, and actual enforcement of predetermined penalties when rules are not followed and responsibilities are not met. The arguments for the State taking an active role in enabling accountability are similar to those for the creation and maintenance of other public goods, such as national defense, security, a clean environment and price stability. However, accountability is the only public good that the state cannot provide on its own, and which requires citizens to be not only able but also willing to bring pressure to bear on government and service providers (Collier, 2009). 3. Where the links between the state and households are institutionally fragile, improvements in performance can be had by shortening the route of accountability. The World Bank‟s 2004 World Development Report showed conceptually and empirically how the distance between the recipients of services, providers and the national authorities can be very long, institutionally fragile and full of obstacles (see Figure 1). This causes accountability channels that attempt to span these distances to lose effectiveness or even break down. Figure 1. The Long and Short Routes of Accountability Source: World Bank WDR 2004 Page 2 4. Doctors and teachers paid by line ministries are really accountable to national authorities only, even if on paper they should also be accountable to local government, communities and the end-users of services. While governments have a primary role in setting the rules and responsibilities of service provision through formal institutional arrangements, monitoring and penalty-enforcement cannot be supplied only by governments. Conflicts of interest and the risk of collusion between the monitors and the monitored call for a more participatory approach to accountability - an approach that directly involves service recipients. 5. This is the rationale for shortening the “route of accountability” by establishing robust participatory mechanisms that engage citizens in holding service providers accountable. Under the participatory approach, governments put institutions in place to allow citizens to play an active role in monitoring and sanctioning providers when their performance is unsatisfactory. Yet in most developing and transition countries, these mechanisms are still missing, or where they exist, they work poorly. I.b. Improving Governance through Stronger Accountability in Albania’s Education Sector 6. The Government of Albania is making rapid progress in strengthening governance across the public sector, particularly in education. Considerable advances have been made in the education sector in particular, with reforms to the service compact and how education services are supplied (represented in the right-hand side of Figure 1). These measures have reduced the room for discretion and collusive practices, increased fairness, and have brought to bear the disciplining forces of the market and greater transparency to improve the performance of the sector. The education sector reforms have been positively assessed, including in a recent World Bank country report (World Bank 2011) which presents data from a specially designed education governance module that was deployed with the 2008 wave of the Albania Living Standards Measurement Survey (LSMS). 7. The Ministry of Education and Science (MoES) is seeking to further improve governance by strengthening parent and community engagement in participatory accountability structures (the left- hand side of Figure 1). The Ministry has implemented reforms designed to strengthen the independence of school boards and parent committees, and is considering measures to increase school autonomy as well as make education financing more responsive to the priorities of stakeholders. Currently there is little autonomy or flexibility given to school-stakeholders to determine their needs and to make investments decisions. “Conditional transfers” from the national budget are channelled to Regional Education Departments (REDs to finance major inputs like teacher salaries, training, teaching materials and a textbook subsidy scheme. Local governments have discretion over the resources they receive as “unconditional transfers” from Ministry of Finance , which cover the costs of utilities and maintenance. Local governments also manage funds to finance investments which they receive on a competitive basis from MoES. These resources are used for improvements to schools such as rehabilitations, extentions, new buildings, and furniture for classrooms. Figure 2 presents a sylized representation of the de jure service delivery and accountability relationships in Albania‟s pre-university education sector. However, until recently, parents and other community members did not have strong incentives nor effective structures to interact with school directors and teachers, in order to identify their particular issues and come up with solutions. 8. To create an enabling environment for greater parent and community engagement in school governance, the Ministry has given school boards greater responsibilities. The composition of these boards has been changed so that school principals should no longer be part of the boards, to give more representation to parents and community members (see Figure 2). The boards have been given greater responsibility for monitoring the actions of school administrators, and could eventually play a role in the selection of head teachers. There is some interest in providing block Page 3 grants to schools so that they can decide on their own spending priorities. A pilot School Improvement Grant (SIG) scheme is being considered by MoES to give greater responsibility directly to school stakeholders for decisions over resources. Figure 2. Service Delivery and Accountability Relationships in Albania’s Pre University Education Sector4 Government MoES RED holds accountable employs School director and teachers provides School services board to Parents Source: Serra, Barr and Packard (2011) 9. A key objective of the Accountability for Better Governance analytical program, initiated in the summer of 2008, is to measure household perceptions and experiences of governance in the education sector, and given this experience, their demand for accountability. Although planned improvements to education governance are ongoing, the analytical program provides important feedback to the government –in the form of responses to new household and facility user surveys- on its efforts thus far. Additionally, since further measures to improve the performance of school governance structures as well as to increase school autonomy are still being considered, the program offers an opportunity to rigorously and scientifically explore the impact of reforms piloted as part of the Albania Education Excellence and Equity Project (EEE-P). Towards this end, the program provides a baseline of data on the performance and functioning of pre-university primary schools that was not previously available to the government. Strengthening accountability mechanisms in order to improve governance is one of the core development objectives of the EEE- P, supported with co-financing from the government, the World Bank, the European Investment 4 The figure simplifies the existing accountability relationships in two ways, although all the essential elements are represented. First, the figure does not show local governments that are also subject to elections, but who have no power over REDs. Second, according to MoES guidelines, a the community level parent members of the School Boards should be elected parent class representatives, The parent class representatives of each grade are together members of the school‟s Parent School Council. Members of this Council elect from among themselves parent members of the School Board. Thus a necessary condition for the School Board to function as intended, is that its parent members are elected by elected parent class representatives, who themselves are chosen by parents participating in the accountability process. Page 4 Bank (EIB), and the Council of Europe Bank. The reforms that remain on the agenda for the education sector involve giving greater means for stakeholders in local schools to hold the government and providers accountable for the quality of education. 10. With the support of the Governance Partnership Facility (GPF) a special survey was designed to examine how parents, community members and teachers interact in school governance structures. The Albania School Stakeholder Survey and Lab-type Experiment was designed and deployed in 180 communities drawn randomly to represent the network of basic (primary) schools in Albania. Along with the survey, a set of controlled lab-type experiments (or “behavioral games”) was also conducted with parents and teachers in order to measure their demand for accountability. 11. This paper presents the findings of the survey and lab-type experiments, and is structured into three parts. The remainder of Part I summarizes the findings from analysis of the survey and lab-type experiments. Parts II and III provide greater technical details for interested readers who are statistically inclined. Appendicies explain the design and sampling strategy used to select schools, parents and other stakeholders for the survey and lab-type experiments, as well as more detailed data on school performance dissagregated by RED. I.c. Summary of Findings from the School Stakeholder Survey and Lab-type Experiments 12. The School Stakeholder Survey, collected in the autumn of 2009, was designed to provide policy makers with data on the functioning of schools and to inform the design of further reforms in the education sector. The survey was designed to: (i) quantify education resources and take stock of institutions for accountability, including both external (“top down”) and participatory (“bottom up”) structures; (ii) measure stakeholders‟ awareness of the schools‟ institutional framework in general, and participatory accountability institutions in particular; and (iii) gage actual parental involvement with the school. In what follows, we present descriptive statistics drawn from the survey to measure -or proxy- school inputs, school and teacher characteristics and effort, “top-down” accountability and transparency, participatory “bottom up” accountability and parental involvement, and parents‟ and community characteristics. The data reveal factors that are significantly correlated with school performance, as measured by student repetition rates, student drop-out rates, and students‟ performance on Albanian language and math exams. 13. Since the new school stakeholder data are drawn from a baseline survey, the analysis discussed in Part II is presented as a series of correlations rather than causal relationships. The results have to be interpreted cautiously as they are drawn from retrospective data collected with a cross-section survey, and are therefore vulnerable to biases due to omissions and endogeneity. Indeed the Albania School Stakeholder Survey was designed first and foremost to collect a baseline so that the impact of interventions the government is still considering can be measured. Nevertheless, the richness of the new data is unique, and allows researchers to investigate the relationship between school outcomes and: (1) school physical and human resources; (2) “top- down” accountability and transparency; (3) “bottom up” participatory accountability institutions and parental involvement; and (4) parents‟ and community characteristics. By employing statistical techniques at the level of RED and district, the problems of endogeneity can be attenuated. And while there is potential endogeneity of parental involvement, this is a greater risk where parents have a choice in determining in which school to enroll their children. In Albania parents appear to have this choice only in large urban areas.5 In order to limit the problem of possible multicollinearity among variables, principal factor analysis is used on sets of variables that are 5 When asked why they chose that specific school to enroll their children, 88% of parents in rural areas point to the vicinity to their house, and 12% mention the presence of good teachers as a reason. These percentages change to 65% and 35% respectively in urban areas. Page 5 meant to proxy for similar school, household or community characteristics and that appear highly correlated. 14. Part II presents indicators drawn from the school stakeholder survey in Albania that have been previously associated with school performance in other countries. The second part of the report starts by motivating the indicators selected from the survey with a review of the empirical literature that uses the “education production function” as its analytical foundation. Disparities in key “inputs” and “outputs” across schools are then presented, and with the important caveats above about the limitations of what can be concluded from retrospective data, statistically significant associations are discussed. Students‟ performance on final year exams, repetition rates, and drop- out rates are examined in relation to indicators of school resources (physical and human); teacher effort; parent and community characteristics that shape the context in which students learn; as well as “top down” and “bottom up” accountability structures. 15. Part III delves deeper into the role of local participatory governance structures and parents‟ demand for accountability in school performance. As discussed previously, better school performance can depend on parents holding teachers accountable through participatory structures. But even where these structures are in place, they are vulnerable to lack of parent capacity and capture by better informed teachers and school principals. Parent involvement depends on awareness of these institutions, and their willingness to actively participate to try and keep teachers accountable. For this reason, parents‟ demand for accountability is critical. However, it is difficult to measure parent demand for accountability with only subjective data on perceptions about problems in schools, attitudinal data about how things ought to be changed, or aspirational data about their willingness to do something about it. Techniques from experimental economics can overcome these limitations by drawing robust measures of demand for accountability that corroborate the subjective, attitudinal, and aspirational data from surveys. In Part III, the results from lab-type, controlled field experiments conducted with parents and teachers, are combined with survey data to investigate how parents‟ demand for accountability relate to their observable characteristics, such as education, income, occupation; existing school-level institutions for participatory accountability; parents‟ information about such institutions; and parents‟ actual involvement in the school. 1.c.1 Significant variations in performance and resources across Albania’s schools 16. Starting with school performance, the survey results show that schools in urban locations appear to outperform schools in rural locations in students‟ achievement in their final exams. The data discussed in this section are presented in Tables 1-7 of Part II. Overall, the proportion of students who do extremely well in math was generally low and especially low in rural areas, where only 6 percent of the students who took the 2009 leaving examination earned a 9 or a 10. The drop- out rate was significantly higher in rural locations. Conversely, schools in urban areas showed higher repetition rates (although the difference is not statistically significant). 17. Differences in student and school performance could reflect the wide disparities in physical and human resources found in the survey. Almost all schools in the sample reported problems of poor facilities and infrastructure. On average, as reported in the survey, 95 percent of the classrooms in the surveyed schools could not be used in poor weather conditions, and schools suffered power shortages for about four days in a typical month. Schools in rural locations were significantly smaller, more likely not to have a phone, to experience power shortages, and not to have a library. As for human resources, the student-teacher ratio was higher in urban locations, although more variation was found in rural schools, where there were up to 62 students per teacher. Teachers in urban schools had achieved higher levels of education and were more experienced; they Page 6 are paid slightly more than rural teachers (which could simply reflect their higher average educational attainment), but they did not appear to be significantly more motivated. 18. Differences in performance could also stem from variation in teacher effort, the extent of quality control by education authorities, and perceptions of school governance. The survey provides key indicators of teacher effort, perceptions of school governance, and a glimpse at the use of formal “top down” accountability structures. On a positive note, the survey responses indicated that teacher absenteeism does not seem to be a major problem in Albanian primary schools: on average, the number of days of unjustified absence in a typical month was only 0.18, with a minimum of 0 and a maximum of 5 days. There were no significant differences in teacher absenteeism between rural and urban schools. The number of inspections from RED representatives in the semester prior to the survey varied greatly across schools, from a minimum of 0 to a maximum of 36. Rural schools were significantly less likely to receive an RED inspection. Overall, in 20 percent of schools, school directors were not subject to any form of appraisal and sanction if they underperformed. Moreover, about half of the surveyed teachers reported that they were not subject to any penalty if they underperformed. Perceptions of irregularities in the appointment of school directors and teachers were quite high, especially in rural areas. As many as 32 percent and 22 percent of the surveyed school stakeholders believed that school directors and teachers were appointed to posts for their personal connections rather than qualifications “often” or “very often”. 19. The existance and quality of “bottom up” participatory accountability structures – particularly those the Ministry has tried to strengthen – could be important factors for performance. The survey captured previously unavailable information about the bodies that have been established for participatory “bottom up” accountability. The number of school board meetings in the semester prior to the survey (January – May 2009) varied from a minimum of zero to a maximum of five. There were no significant differences in this indicator between rural and urban schools. However, contrary to the MoES directives, in about 30 percent of schools, the school director participated in school board meetings and expressed his or her vote. But again, there were no significant differences in this indicator between rural and urban schools. In about 81 percent of the schools, according to the answers of parent school board members, parent representatives were elected by parents rather than appointed by teachers; and in 43 percent of the schools teacher school board members were elected by teachers rather than appointed by the school directors. These proportions were significantly higher in urban schools as compared to rural schools. 20. The survey found a worrying lack of information among parents about participatory accountability structures in their children‟s schools. About 43 percent of the parents did not know school boards existed, and 28 percent did not know that there were class parent representatives. Only about 80 percent of parents who were informed about the existence of the school boards knew that there were parent members, and of these, 72 percent did not know their names. The lack of information among parents captured by the survey was more severe in rural schools than in urban schools. Of the 72 percent of parents who were aware of the existence of parent class representatives, only half knew their names; and of the parents who knew that parent representatives existed and were elected by parents, 77 percent did not vote in the most recent election. 21. Parent involvement in participatory accountability structures varied greatly across schools. The survey measured levels of parent engagement in accountability structures. The average number of meetings that parents said they had with teachers in the previous semester (January-May 2009) was around 4, yet this indicator varied considerably across schools, and was significantly higher in urban schools. The proportion of parents who usually participate in parent-teacher meetings also varied greatly across schools, from a minimum of 34 percent to a maximum of 100 percent, with an Page 7 average of 72 percent. There was a significant difference between rural and urban schools in the proportion of parents that participated in parent-teacher meetings, albeit very small in magnitude. Lack of parental involvement was considered a problem by teachers in about 20 percent of the schools, but there were no significant differences between rural and urban locations in this indicator. 22. Household and community characteristics -significantly associated with school performance in other countries- also varied greatly across the surveyed stakeholders in Albania. Not surprisingly, the average years of schooling of both mothers and fathers of the students and the number of books owned by the household, were significantly higher in urban locations than in rural locations. Albanian parents have on average completed primary school and at least two years of secondary or vocational school. A household‟s economic status, as well as the proportion of employed parents -and the proportion of parents employed in occupations other than farming - were significantly higher in urban areas. About 90 percent of surveyed parents seemed to value the education of their children, but more so if the family lived in urban areas. Turning to community cohesion, urban areas were characterized by less community cohesion than rural areas. In urban areas there was less trust and cooperation than in rural areas -as shown by the significantly higher proportion of respondents who thought that “in their neighborhood there are people who want to take advantage”, and the smaller proportion of those who thought that “members of their community would be willing to help if needed”, and would come together to jointly solve a common problem such as possible water shortage. 1.c.2. Simple pair-wise correlations between education “inputs” and “outputs” 23. A school‟s physical and human resources were significantly correlated with the proportion of students who excelled in math and Albanian in the 2009 final examination. In particular, excellence in both math and language appeared more likely in urban schools, schools that have a phone (which could proxy the quality of local infrastructures), and schools that did not suffer as much from power shortages. The presence of a library in the school was also positively correlated with students‟ performance in the math test, but not in the Albanian test. Although no significant correlation was found between education outcomes and the characteristics of the school director, education of teachers and the student-teacher ratio, there was a significant, positive correlation between students‟ excellence in the language exam and teachers‟ years of experience, their motivation and wages. Teachers‟ motivation was also negatively correlated with students‟ drop-out rate. 24. Student and school performance were significantly correlated with perceptions of school governance, as well as with measures of “top-down” and participatory “bottom up” accountability structures. Perceptions of malfeasance in the appointment of teachers were positively correlated with the drop-out rate. The number of visits from RED inspectors was correlated positively with students‟ excellence in Albanian and negatively with the drop-out rate. Students‟ excellence in math and language was significantly more likely where parents were informed about participatory accountability structures; knew the names of the math and Albanian teachers; had more frequent meetings with teachers; and where lack of parental involvement was not considered to be a problem in the school. Parents‟ lack of information was also positively correlated with failure and repetition rates. Interestingly, the drop-out rate was found to be significantly higher in schools where there were fewer school board meetings, where the school director was participating and voting in these meetings, and where parents had fewer meetings with teachers. 25. Not surprisingly, family characteristics were also found to be important correlates of education outcomes. The average years of schooling of students‟ mothers and fathers; the number Page 8 of books owned by the household; and an index of household socioeconomic status, were all strongly and positively correlated with student performance on final examinations. 1.c.3. Findings from multivariate correlation analysis 26. The last section in Part II examines the robustness and relative importance of the correlations discussed above. For each measure of school performance – the drop-out rate, repetition rate, failure on exams, and student excellence in math and Albanian - four regressions were conducted, each including only one of four categories of independent variables: (1) school resources; (2) “top-down” accountability and transparency; (3) participatory “bottom up” accountability and parental involvement; and (4) family and community background. The correlation results are presented in Table 8. 27. School resources and top-down accountability structures appeared less significantly related to measures of school failure – drop-out rates and repetition - than parental involvement. There were very few correlates with drop-out, repetition and failure rates among school physical and human resources. The only significant results were a positive correlation between rural location and failure rate, and the negative relationship between teachers‟ motivation and drop-out rate. None of the top-down accountability and transparency measures appear to be correlated with drop-out, repetition and failure rates. In contrast, the number of school board meetings and whether the school director usually participates and votes in the meetings were, respectively, negatively and positively correlated with drop-out and repetition rates. Moreover, lack of parental involvement in the school, as assessed by all school stakeholders, was positively correlated with students‟ repetition rate. Among the proxies for the students‟ family background, the only significant result was a negative correlation between fathers‟ education and repetition rates. 28. School resources, teachers‟ characteristics and inspections, were more robust and significantly correlated with students‟ excellence in math and language. In particular, the percentage of students excelling in Albanian was significantly higher in urban schools which suffer fewer days with no electricity; where teachers are more motivated; and where they have more teaching experience. The percentage of students excelling in math was significantly higher in schools that had a telephone and in those with a library. More RED inspections and fewer days of teachers‟ absence were significantly associated with better performance of students in the Albanian exam. Teachers‟ absence had a significant negative association with student performance on the math exam, while the use of a register to record teacher absences was significantly and positively associated with student performance in math. 29. Parental involvement continued to show a robust, significant relationship with student achievement. A lack of parental involvement in the school was found to be negatively associated with students‟ excellence in both math and Albanian exams. Moreover, the proportion of students who excelled in math was higher the greater the number of meetings parents had with teachers and lower if parents had no information about participatory accountability structures. 30. However, some of the education “inputs” correlated differently with “outputs”. The indicators examined above were differently correlated with students‟ excellence in math than with student excellence in Albanian. For instance, the rural location of the school, and the experience and motivation of teachers were negatively correlated with students‟ performance in Albanian, but not in math. None of the proxies for top-down and participatory accountability, nor parental involvement or family background, were correlated with the proportion of students who got a 9 or a 10 in Albanian. The only variable that was significantly correlated with students‟ excellence in Page 9 Albanian (with the expected negative sign) was lack of cohesion within members of the community. Students‟ excellence in math, however, was correlated with a larger set of factors. The proportion of students excelling in math was lower in schools with poor facilities. While teachers‟ characteristics did not appear significant, teacher appraisal and transparency with respect to teachers‟ absence from school were; indeed, students‟ excellence in math was more likely in schools where teachers believed they would be penalized if they underperformed, and where a teacher absence register existed and was regularly used. Lack of parental involvement in the school also correlated negatively, with students‟ performance in the final math exam. A father‟s education was strongly and positively associated with children‟s performance in math. And the extent of divide (lack of perceived cohesion) in the community appears to matter by reducing the proportion of students excelling in both math and language. 1.c.4. Lab-type experiments and parents’ demand for accountability 31. The low levels of parent participation in school governance structures could reflect lack of information or parents‟ reluctance to hold teachers accountable. As discussed earlier, Albania has made concerted efforts to improve governance across the public sector. Its efforts in the education sector have been received positively by households. Seeking to complement governance improvements in education compact and the supply of education services, the Government has been investing in shortening the route to accountability, by strengthening participatory accountability structures (see again Figure 1). However, the data collected in the Albania School Stakeholder Survey indicate that parental participation in these structures is still very low. Is this lack of participation owing to poor institutions, a lack of knowledge and understanding, or a preference not to participate? 32. Parents appear seriously under-informed about accountability structures in their children‟s schools, and in many places these structures are not operating as they should. The findings summarized previously, highlight both a departure in de facto accountability structures from how the MoES intends them to function de jure, as well as a severe lack of information among parents about these structures. Contrary to MoES directives, in about 30 percent of schools, the school director participated and voted in school board meetings. In 81 percent of the schools, parent class representatives were elected by parents as intended, rather than appointed by teachers; and in only 43 percent of the schools were teacher school board members elected by teachers rather than appointed by the school directors. Only half of parents report participating in electing parent class representatives, and about 70 percent of those who did not participate were unaware either of the existence of parent class representatives or of the fact that they should be elected by parents. Moreover the survey identified a worrying proportion of parents who were unaware of the existence of the school board (43%), including a large proportion of elected parent class representatives (23%) and parents who had voted for parent class representatives (31%). This indicates that information problems are severe, even among parents who are participating in school governance in some way. 33. But if they had more information, would parents participate? The survey findings about the functioning of participatory accountability structures and parents‟ information, begs the question: If parents were informed, would they participate in holding schools and teachers to account? Part III presents the conclusions of an analytical attempt to directly measure parents‟ preferences for participation through specially designed lab-type experiments (LTEs) that were conducted with a random sample of teachers and parents who responded to the school stakeholder survey. 34. Experimental techniques allow a controlled simulation of the parent-teacher accountability relationship. Lab-type experiments (LTEs) facilitate the measurement of individuals‟ values, beliefs and preferences that cannot be captured in survey data. They have been used in a number of Page 10 countries and contexts, involving a wide variety of groups; from university students, and children, to adults without formal literacy or numeracy in foraging small-scale societies.6 The specific details of each game will vary, depending on the specific preferences that they are designed to capture. LTEs are always designed with a clear research objective in mind, and employ real money in order to recreate as closely as possible the incentives that individuals face in the real-life situations that the games aim to simulate and elucidate. 35. The games were designed to yield insights into parents‟ preferences for participation, i.e. their willingness to forgo personal payoffs in order to hold teachers to account. In the games, participants‟ roles were matched to their real life roles in the school. The games were designed to mimic key aspects of the school context: teachers having resources that they might or might not pass on to their pupils, and parents having the choice to forgo personal resources to hold teachers to account. In the games, a player (teacher) is invited to divide a sum of money between herself and another player (a parent), and a third player (another parent) can pay to have money subtracted from the first player‟s allocation.7 It is the third players‟ behavior that is of particular interest, as by choosing to “fine” they reveal both what they consider to be unfair behavior on the part of the first player, as well as their own willingness to pay to express this opinion. In other words, the third player is expressing their willingness to hold the first player to account for how the resources were divided (“fairly” or “unfairly”) between themselves and the passive second player. 36. However, the games were also designed to differ from the actual school context in several important ways, particularly with respect to the information parents have and how they can act on it. In the actual school context parents‟ decisions to hold school directors and teachers to account usually require that the parents expose themselves, whereas in the game parents were guaranteed at least some anonymity (teachers who were fined do not know which parents fined them). While in the school context the costs associated with participation may differ across parents, in the game the material cost of holding teachers to account was fixed and known by all. Finally, in the actual school context, parents may be uninformed about the existing participatory institutions and the actual performance of teachers, whereas in the game, parents were fully informed about how they can hold teachers to account and can condition their decisions on accurate measures of teacher performance. Therefore, the game can be viewed as a counterfactual to the existing situation in Albania‟s primary schools. In this counterfactual information is not constrained, parents wishing to participate in holding teachers to account are afforded some protection (anonymity), and the consequences of their participation are clearly defined. Most importantly, in the game the manner in which parents could hold teachers to account was simple and clear, and the parents were fully informed. This being the case, the games afforded an opportunity to take a first glimpse at the parents‟ preferences for participation in holding teachers to account. The findings discussed are presented in Tables 10-12 in Part III. 37. Parents‟ decisions in the game were significantly related to their wealth and levels of education; however, their participation in actual school accountability structures was not significant. Not surprisingly, the decision to forgo personal resources to hold teachers to account (to fine) in the game was more likely among wealthier and better educated parents. Unexpectedly, parents who participate in actual school accountability structures (as parent class representatives of on school boards) were no more likely to engage in fining in the game. Further parents who were less informed about the schools their children attended were no less willing to participate in fining in the games. 6 See Barr et al. (2009) for an example of a cross-cultural experimental study based on the implementation of similar experimental designs to the ones we use here in 15 different small-scale societies. 7 The LTE discussed is a version of Fehr and Fischbacher‟s (2004) Third Party Punishment Game (TPG). Page 11 38. How a school‟s participatory accountability structures were composed and functioned, as well as perceptions of school governance, were significantly associated with parent‟s decisions in the game. Critically, the extent to which participating parents held teachers accountable in the games depended significantly on whether the de jure participatory structures – parent class representatives and school boards - have been de facto compromised. Specifically, parent class representatives on the school boards that had been appointed to their position by teachers, rather than elected by parents, were significantly less demanding of teachers. Moreover, parents who perceived irregularities and malfeasance in the appointment of teachers were significantly more demanding of teachers in the game. 39. Parents‟ behavior in the game indicates that low participation in school accountability structures is likely to reveal constraints rather than parents‟ reluctance to hold teachers accountable. The lack of significant statistical correlation between the decision to participate in fining in the game, and parents‟ participation and information in the actual school context, could be an indication that the games were not externally valid, i.e. did not provide an accurate simulation of the actual parent-teacher accountability relationship. Alternatively, it could be an indication that the lack of participation and information in the actual school context are not the byproduct of distorted parents‟ preferences, but rather the result of the system‟s failure to generate de facto participatory institutions, in which parents are represented by individuals who are accountable to them rather than captured by the interests of teachers, and are made aware of the available means to participate. Keeping in mind the possible limitations in external validity of the games, when the findings from the game (that appointed parent representatives are less demanding of teachers) is set in context - with the extensive information problems that emerged from the survey- a cautious conclusion can be draw. Namely, that lack of parental participation in Albania is more likely to be a problem of external constraints than one of preferences. 40. These findings are important for policy makers and sector specialists to keep in mind as they look for participatory structures that effectively shorten the route of accountability. Important messages can be drawn from these findings.  First, the degree to which the parents of primary school students in Albania are willing to participate in the game bodes well for the government‟s objectives of increasing parent engagement in school accountability and governance structures. This is good news for the government, and supportive of the steps it has taken and is still planning to strengthen school governance and autonomy.  Second, the quality of accountability structures and how school governance is perceived matters to parents‟ decisions to become involved. This should underscore to the authorities the importance of monitoring implementation of the school governance reforms to ensure de facto departures from their design and intent are minimized.  Third, there is considerable scope for improving the degree to which parents are informed about the school governance reforms and how they can now assume a more active role in holding teachers and school administrators to account. This lack of information is found even among parents who are currently participating as elected class representatives. 41. The findings also show how future interventions can be designed and deployed to increase the likelihood of success. From a more technical and methodological angle, the results presented in this report shows that by making it possible to directly measure individuals‟ willingness to engage in holding teachers to account, lab-type experimental techniques could allow better calibration of participatory interventions. This could increase the likelihood of the interventions‟ success. Lab- type experimental techniques implemented in the field can provide useful insights into the chances Page 12 of success of alternative interventions, and/or allow for better tailoring of random controlled trials (RCTs) to different contexts. Moreover, if combined with surveys and RCTs, LTEs allow for useful and much needed investigations of the interaction between individuals‟ preferences and formal and informal institutions. Page 13 Part II. Accountability, Good Goveranance and School Performance II.a. Are There Recipies for School Success? Lessons from Prior Research 43. A large body of research is focused on identifying the determinants of student achievement in schools and school performance. The most commonly used approach for the study of educational outcomes is the so-called production function approach (Hanushek, 1996), which relates students‟ and schools‟ performance to a number of measurable educational inputs, some of which (for example, physical and human resources available in schools) can be controlled by policy-makers while others (such as children‟s innate ability and socioeconomic background) cannot (although they can be indirectly affected by certain specific policies). The school (physical) resources usually investigated within the production function framework are: the quality of building and infrastructures; the teacher-pupil ratio; and, when available, expenditure per pupil, and teachers‟ salaries. The most commonly used proxies for human resources are teachers‟ characteristics, such as years of schooling and teaching experience. 44. The empirical evidence on the effect of physical and human resources on students‟ achievement is mixed. Harbison and Hanushek (1992) provide a comprehensive review of 96 studies based on retrospective data, i.e. based on ordinary (i.e. non-experimental data collected as responses to surveys) variation across schools and households in developing and transition countries, and illustrate that none of the physical and human resources listed above has been shown to be consistently and significantly related to educational outputs. In other words, “there seems to be no clear and systematic relationship between key inputs and educational outputs” (Hanushek, 1996). Further investigations in Ghana (Glewwe and Jacoby, 1994), Jamaica (Glewwe et al, 1995), India (Kingdon, 1996) and the Philippines (Tan et al. 1997) confirm that the quest for a causal link between teacher characteristics and school resources on the one hand, and educational attainment on the other, is still far from complete. Indeed, in the studies cited most variables proved statistically insignificant, and while each investigation found some teacher and school characteristics to be statistically significant, the specific resources tested or found significant varied greatly across the different studies. 45. Krueger (1999) highlights that “much of the uncertainty in the literature derives from the fact that the appropriate specification of the education production function is uncertain”. Along the same line, Glewwe and Kremer (2005) stress that omitted variables at either the student, class, school, or local level may bias the estimates based on retrospective data either upward or downward. For example, unobservables such as children‟s innate ability and parents‟ motivation may be positively correlated with school overall performance, if parents enroll high-ability children in better performing schools. Similarly, teachers‟ unobservable characteristics, such as motivation, are likely to bias the estimates upward, if they are correlated with the observable characteristics included in the empirical specification.8 46. Researchers have tried to overcome these limitations by employing data generated by “natural experiments” or by conducting randomized control trials (RCTs). The resulting evidence on the impact of school resources on education outputs is still mixed. While studies exploiting natural experiments suggest, for instance, that reducing the pupil-teacher ratio increases test scores (Krueger, 1999; Angrist and Levy, 1999, among the others), RCTs conducted by Duflo et al. (2009) 8 The differences in the results obtained by different studies may therefore be the result of the different specifications used and, consequently, different omitted variables. Possible multicollinearity between the many school and teacher variables used with the attempt to reduce omitted variable bias, may also be responsible for the discrepancy in estimates across studies. See Glewwe and Kremer (2005) for a comprehensive discussion of all possible limitations of estimates of education production functions based on retrospective data. Section 2 will also provide further discussion on this. Page 14 in Kenya and by Banerjee et al (2010) in India provide null results. RCTs conducted in Kenya have also shown ineffectiveness of the use of flip-charts (Glewwe et al., 2004) and the provision of textbooks (Glewwe et al. 2009) to schools in improving education outcomes.9 47. Another explanation why school resources (physical and human) do not seem to generate the expected effects on education outputs is that resources intended for instruction may be diverted to other purposes or may not be efficiently used. In transition and developing countries, institutional deficiencies lead to accountability problems and incentives which result in poor performance. Teachers are frequently absent from the classroom.10 Even when they are present, some may not be doing their job adequately.11 School director and teachers‟ accountability is conditional on: (1) assignment of clear responsibilities; (2) being monitored while exercising these responsibilities; and (3) being subject to clear penalties when their performance falls short of predetermined targets (see Lewis and Petterson, 2009). The three conditions for accountability are usually all missing in developing and transition countries, resulting in teachers‟ absenteeism and/or low motivation, and, ultimately, poor education attainments, both in terms of quantity and quality. 48. Policy makers have tried to improve accountability using both “top-down” and “bottom-up” mechanisms. Top-down accountability typically takes the form of a monitoring system and a set of pre-established sanctions or rewards managed by higher levels of administrative authority in an institutional setting. In the case of the education sector, it often refers to school inspections from regional or local education offices, and resulting sanctions or rewards being dispensed to school director and teachers. Top-down accountability can also exist internally, within the school, in the form of the school director‟s efforts to regularly monitor the performance of teachers. However, empirical research shows top-down mechanisms are of limited effectiveness, as they are often subject to abuse and collusion between the “monitored” and the “monitor”.12 49. In response to the difficulties in implementing top-down accountability mechanisms effectively, bottom-up participatory programs have received increasing attention. The growing consensus is that while the government has a primary role in setting rules and responsibilities of service providers through formal institutional arrangements, monitoring and penalty-enforcement cannot be supplied by the government alone. In education, decentralization and local school management programs are advocated to capture the information advantage that parents and communities have with respect to the needs of their children; to harness their strong incentives to monitor the performance of teachers and school directors; and to exploit their comparative advantage in conducting the monitoring. 9 See Glewwe and Kremer (2005) and Kremer and Holla (2009) for comprehensive reviews of evaluations of the impact of school resources on education outcomes using RCTs. 10 Chaudhury et al. (2006) report results from surprise visits in primary schools in six developing countries; on average 19% of the teachers where found absent from school on the days of the visits – 27% in Uganda, 25% in India, 20% in Kenya, 16% in Bangladesh, 14% in Ecuador, and 11% in Peru. 11 Several researchers have found that even when teachers are in school, they may not necessarily teaching their students. In India, while 75 percent of teachers were present in the school, only about half of all teachers were actually teaching in the classroom when the enumerators arrived. 12 Several studies employing RCTs have. Kremer and Chen (2001) have implemented and evaluated the effectiveness of a top-down internal program that gave school directors the authority to give bonuses to teachers based on their presence at school. The program resulted highly ineffective in reducing teachers‟ absence, as school directors ended up giving bonuses to all teachers irrespective of attendance, which highlights one important limitation of top-down mechanisms: the possibility of collusion between the monitored and the monitor. Duflo et al. (2007) have implemented and evaluated a program implemented in NGO-run education centers in India, which provided bonuses to teachers based on an objective measure of their presence at school: photographs that teachers were required to take at the beginning and the end of the school day with a camera set with tamper-proof time and date. The program was highly successful in reducing teachers‟ absenteeism and improving test scores; however, it relied on an externally administered top-down system that would encounter strong resistance of teachers and teacher unions, had to be replicated in public schools. Page 15 50. Empirical research focused on parental involvement in education governance through participatory “bottom up” accountability structures, is still in its infancy, but promising. While measures of parental background have for a long time been included in the empirical specifications employed in retrospective studies (again, with mixed results13) attempts to measure and account for parental involvement in their children‟s education, and in participatory initiatives at the school level, are more recent. Retrospective studies conducted in the US (Houtenville and Conway, 2007) in Argentina (Eskeland and Filmer, 2007), in El Salvador (Jimenez and Sawada, 1999), in Mexico (Gertler et al., 2008; and Skoufias and Shapiro, 2006), among others, all show a significant impact of participatory programs and parental involvement on education outcomes.14 Jimenez and Sawada (1999) found evidence that a local management system improved students' language skills and attendance in El Salvador. In Mexico Gertler, Patrinos, Rubio-Codina (2008) found that providing small grants ($500 to $700) to parent associations -to be used for infrastructure or materials that they considered important- improved student performance and reduced grade repetition by about 5%.15 Skoufias and Shapiro (2006) investigated the effectiveness of the Mexican Quality Schools Program (PEC), within which parent associations developed and undertook school improvement plans funded by five-year grants of up to US$15,000. They found that participation in the program decreased drop-out, failure and repetition rates, and based on focus group interviews with parents and school directors, concluded that parents' involvement increased following the introduction of the program.1617 51. The evidence provided by studies based on RCTs is more mixed. In Kenya, alongside a program that allowed and funded public sector schools to hire extra teachers on short term contracts, some communities (both from the treatment and the control samples) were randomly selected for training in how to monitor teachers. The monitoring program had no impact on the attendance rates of the extra teachers and only a small, positive and insignificant effect on attendance rates of public sector teachers (Duflo et al. 2007). This was followed by Duflo, Dupas and Kremer (2009) implemented a controlled experiment in which 70 out of 140 Kenyan schools received an extra contract teacher, hired locally by a school committee made up of teachers and elected parent representatives. At the end of the year, the school committees had to decide whether to hire the same contract teacher or a different one. Further, in half of the 70 schools, school committee members, including parent representatives, were trained on how to monitor and evaluate the performance of the contract teacher. The authors found that in the short term students assigned to the contract teachers performed better. However, this improvement in performance persisted only in schools in which committee members were trained and encouraged to meet and reassess the teacher. Also in Kenya, a program designed to strengthen ties between school committees and local educational authorities through training and joint meetings and under which school committees could award prizes to teachers who performed well, increased committee activity and the level of education of those elected to the committees but had no effect on teacher attendance (de 13 Mother‟s education seems to be the household characteristic most consistently related to students‟ performance. 14 For instance, Jimenez and Sawada (1999), Gertler, Patrinos, Rubio-Codina (2008) and Skoufias and Shapiro (2006) investigate the impact of specific participatory institutions in El Salvador and in Mexico respectively. In an ex post evaluation of the EDUCO program, Jimenez and Sawada (1999) found evidence that a local management system improved students' language skills and attendance. The EDUCO program established community education associations, called ACEs, composed by elected parents, which are given authority over the management of funds provided by the ministry, the hiring and firing of teachers, and the physical maintenance of the school. This program is therefore an example of the shortest possible route to teachers‟ accountability. 15 Parent associations also received training in the management of the funds and in participatory skills to increase their involvement in school activities. Based on qualitative data, such as interviews with parents, teachers and school directors, the authors argue that the program was successful in increasing parental participation in school matters. 16 Note that, compared to the EDUCO program, both AGE and PEC did not provide parents with direct means to hold teachers accountable, i.e. the route to accountability was much longer in this case. 17 Along the same lines, it may be the case that having parent associations in place but without financial resources is not as effective, that is it may be necessary to have a mandate and the resources. Page 16 Laat et al. 2008). However, in Uttar Pradesh in India, neither informing communities about the quality of local schools, what they should expect from schools, local funding for schools, and the responsibilities of Village Education Committees (VECs), nor training communities and VECs in techniques for monitoring schools and assessing their children‟s progress, had any impact on school performance (Banerjee et al. 2009). II.b. Accountability in the School “Production Function” 52. Students‟ performance, and consequently school performance, can be measured on the basis of many different outcomes, e.g. students‟ average test scores, repetition rates, drop-out rates etc. Thus, the education production function can take the form qij = f(xi,, a1i, a2i , zij,, ηij ) 53. where qij is the performance of school i in terms of output j, and xi represents the level of educational resources, i.e. school and teachers‟ characteristics, available in school i. The set of accountability efforts made in school i can be divided into two components: a1i captures the top- down accountability efforts exercised by educational office inspectors and school director as well as transparency in school administration; and a2i refers to the parental and local community participation in holding school director and teachers to account, as well as parents‟ general involvement in their children education; zij comprises attributes of the students‟ parents and community. Finally, the term ηij is a random error, which may includes school, community and household unobservable characteristics. 54. However, a note of caution is needed. This investigation into school performance in Albania also employs retrospective data and is therefore not exempt from possible biases due to omitted variables and endogeneity. Nevertheless, the richness of the new data from the Albania School Stakeholder Survey is unique, and allows researchers to investigate the relationship between school outcomes and: (1) school physical and human resources; (2) top-down accountability and transparency; (3) participatory accountability institutions and parental involvement; (4) parents‟ and community characteristics. 55. Since the data are retrospective in nature, establishing causal links is problematic due to omitted variables bias and possible endogeneity of school resources, parental characteristics and accountability measures. Fully addressing endogeneity problems is virtually impossible given the retrospective nature of the data. Indeed the Albania School Stakeholder Survey is intended first and foremost as a baseline to measure the impact of interventions the Government is still considering. That said, by employing fixed effects at the level of the Regional Educational District (RED) and clustering the error terms by district, the problems of endogeneity of school resources and top-down accountability measures can be attenuated (although not entirely eliminated).18 As for the potential endogeneity of parental involvement, this is likely to be especially severe if parents have a choice in determining which school to enroll their children. In Albania this seems to be the case only in large urban areas.19 Nevertheless, it is a reason for concern.20 18 The REDs are local education administration entities (thirteen in total) that are most directly accountable to The Ministry of Education and Science (MoES). While the MoES is directly responsible for the development of curricula, the selection of textbooks, the structure of the academic year, the workload of teachers, the allocation of resources among local education institutions, and teacher training (World Bank, 2005), the RED are responsible for the delivery of primary and secondary education in their respective districts – 37 in total. 19 When asked why they chose that specific school to enroll their children, 88% of parents in rural areas point to the vicinity to their house, and 12% mention the presence of good teachers as a reason. These percentages change to 65% and 35% respectively in urban areas. Page 17 56. Conscious of the non-experimental nature of the data employed in this section of analysis, and unresolved endogeneity issues, in the sections that follow, care is taken to present results as correlations rather than causation. Moreover, in order to limit the problem of possible multicollinearity among the independent variables, principal factor analysis is used on sets of variables that are meant to proxy for similar school, household or community characteristics and that appear highly correlated. 57. The survey was designed to: (i) quantify educational resources and institutions for accountability, including both external (“top down”) and participatory (“bottom up”) mechanisms; (ii) measure the awareness of the different stakeholders of the school institutional framework in general, and participatory institutions in particular; and (iii) measure actual parental involvement with the school. In what follows, we present descriptive statistics drawn from the survey to measure or proxy school outputs, school and teacher characteristics, “top-down” accountability and transparency, participatory “bottom up” accountability and parental involvement, and parents‟ and community characteristics. In the first part of this section, we explain the measures that have been selected from the survey data. In the second part, we present descriptive statistics and the results of tests for significant differences between community groups of interest. 2.b.1. Measures of students’ performance at the school level 58. Until the 2006-2007 academic year primary school leaving examinations were designed and administered by school teachers. This system generated opportunities for irregularities and prevented MoES from carrying out objective comparisons of students‟ performance across schools. Starting from June 2008, the locally administered final year examinations were replaced by standardized tests, centrally administered by the division of the MoES that is also in charge of State Matura high school final year examinations. Therefore, since 2008 the performance of primary school students on leaving examinations has been comparable across schools. 59. We focus on five education outcomes at the school level. Three of them are generated by the leaving examinations corresponding to the academic year immediately preceding our data collection (the 2009 final year examinations). In each school, we look at: (1) the proportion of students who failed the final year examination; (2) the proportion of students who excelled (i.e. received a 9 or a 10) in the Albanian language test; and (3) the proportion of students who excelled (i.e. received a 9 or a 10) in the math test. In addition, we consider: (4) the drop-out rate (i.e. the proportion of students who were enrolled in the school in the 2008-2009 academic year and did not return to school the following year21); and (5) the repetition rate (i.e. the proportion of students repeating a year at the time of the survey). 2.b.2. Measures of school physical and human resources 60. The questionnaire directed at the schools‟ administrative representative provided extensive information about the status of the buildings and physical resources available in each school. Guided by the existing literature, we focus on the following school characteristics: the urban vs. rural location of the school; the distance (in kilometers) from the nearest town; whether the school can be reached by public transportation; the number of days with no electricity in a typical month; whether the school has a telephone; the proportion of classrooms in poor condition that cannot be 20 Gunnerson et al. (2009) use retrospective data to study the impact of parental participation and school outcomes on education outcomes in eight Latin American countries. They find that correcting for endogeneity substantially increase the estimated impact of parental participation on students‟ test scores. 21 Note that this may be also due to the family moving to a different location, or the student being transferred to a different school, where more than one are available. Page 18 used in certain times of the year; the proportion of desks in poor condition; and whether the school has a library. 61. Both the questionnaire directed at school administrative representatives and that directed to teachers generated many important human resources variables, whose impact on students‟ performance has been investigated also by previous studies. These are: the student-teacher ratio; the average years of schooling of teachers, the average years of experience of teachers, the proportion of “local” teachers (i.e. those who grew up in the location where the school is), and the average and lowest wages received by teachers in the school. Moreover, we have information about the school directors‟ years of experience (as a school director), whether he or she received any school management training, and whether he or she is a “local” (i.e. grew up in the village/town where the school is located). Finally, we attempted to control for teachers‟ motivation – which is often considered a possible omitted variable in previous studies – by asking teachers whether “lack of teacher motivation” is a major or moderate problem in the school, as opposed to a minor or a non- existent problem. In each school we compute the proportion of teachers who believe that teacher motivation is not a problem in the school. 2.b.3. Measures of “Top-down” accountability and transparency 62. Regional Education Directorates (REDs) have no authority over the amount of funds allocated to schools – these are centrally established by MoES – and their administrative heads are centrally appointed; they are however responsible for the appointment and transfer of teachers, the distribution of administrative materials to schools, inspections of schools, as well as planning and supervising the construction of new facilities and major rehabilitation of existing schools.22 Besides employing RED fixed effects in our regressions, we aimed to directly account for differences in top- down accountability across schools by asking school directors and teachers specific questions about monitoring and possible penalties. Our main variables of interest are: the number of visits of RED representatives in the previous academic year (January-May 2009); whether the school director is subject to penalties if he/she underperforms (self-reported by school directors); whether teachers are subject to penalties if they underperform (self-reported by teachers); and the days of unjustified absence of teachers in the month of May 2009, as reported in the teachers‟ absence registry, in the schools where such a registry is kept. 63. To measure transparency, we report whether the school keeps a teacher absence register and if it publicly displays teachers‟ absence and students‟ performance in the final year examination. We also generate a measure of corruption in the appointment of school directors and teachers based on perceptions of all the surveyed stakeholders. Each respondent was asked to evaluate how often school directors (teachers) are appointed to posts based on personal connections rather than their qualifications, using a 5-item Likert scale, ranging from “never” (1) to “always (5). In each school we consider the proportion of respondents that answered “often” or “always” to the above questions. 2.b.4. Parent Involvement in Participatory “Bottom up” Accountability Structures 22 The REDs are also responsible for teachers‟ salaries, which are paid using the total budget allocated by the MoES, and for the collection of information at the district level and transmitting this information to the Ministry. The local municipalities/communes have very limited responsibility for the delivery of education services. Under the current arrangements, they receive conditional transfers from the MoES, through the REDs, to cover teachers‟ salaries and scholarships for pupils in secondary schools. They also receive unconditional grants from the Ministry of Finance to cover operational and maintenance expenditures, including minor repairs for the maintenance of buildings, and recurring costs (water, electricity, sewage, heating, bussing, etc.). The grants, however, are not earmarked for education, and municipalities and communes are also expected to finance other public services from it. The size of the block grant is decided based on population, area, and an urbanization coefficient (see Tibi, et. al., 2004). Page 19 64. The institutional framework currently in place in Albania is centralized and top-down in nature, since it assigns the task of monitoring teachers and imposing penalties for under- performance to administrative units that are not accountable to service recipients. However, recently the MoES has been trying to improve teachers‟ performance and quality of education by promoting community engagement in local governance structures, such as parent committees and school boards. School boards have always existed within school administration units, but they have recently been given greater responsibilities for school governance. Moreover, in an attempt to reduce possible “elite capture” their composition has changed and the school principal is no longer allowed to be a member of the board. 65. Among the members of the school boards, there should be at least two parents, elected by elected representatives of parents of pupils in all classes, and at least one member of the community who should not be a parent in the school.23 One teacher representative – elected by all teachers – and one or two student representatives are also members of the board. Moreover, a representative of the Regional Educational Department/ Educational Office, and a representatives of the local government unit (municipality or commune), appointed by the mayor, should also be part of the board.24 66. The main functions of the school board are to examine and approve (by voting) the school‟s mid-term plan and annual plan submitted by the school director, as well as the school‟s annual financial report. The board also has decisional power for the adoption of curricula and textbooks, as well as the school‟s budget (if any) generated by public funds, contributions of the community or other donors, or revenues from school‟s activities. School board members may also discuss problems related to the performance of teachers, or the school director, in response to complaints of parents, students, or teachers. MoES has been debating whether the responsibilities of the board should be extended in the near future to hiring and firing of teachers. 67. Regarding the composition and functioning of the school board, we register the number of meetings of the board in the previous semester (January-May 2009)25; whether the school director usually participates in the meetings and votes; whether the parent representatives are elected by parents rather than appointed by teachers or the school director; and whether the teacher representatives are elected by teachers rather than appointed by the school director. 68. In order to account for the de facto functioning of participatory institutions, we ask parents whether they are informed about the existence of school board and parent class representatives. Conditional on positive answers, we also collect information about parents‟ participation in the election of parent representatives. To further investigate the extent of parental involvement, we register whether parents know the names of their children‟s math and language teachers, and the number of parent-teacher meetings they attended in the previous semester (January-May 2009). We also gather information from teachers about the proportion of parents that usually participate in parent-teacher meetings, and we ask parents whether they ever filed a complaint concerning education matters. Finally, we elicit all the opinions of all surveyed stakeholders with respect to the extent of parental involvement in the school; we report the proportion of respondents who believe that lack of parental involvement is a moderate or serious problem in the school. 23 No details about the selection of the member of the community have been officially provided. 24 The Guideline for “Establishing and Functioning of the School Board” (MoES, 17 October 2007) established that “when the Regional Educational Department/ Educational Office or the local government (municipality, commune) covers a great number of schools, they are represented in board by their delegate, who as a rule should be a member of the scholastic community”. 25 We also asked about meetings in the current semester (September –December 2009), but the comparability of answers across schools is compromised by the fact that the survey was administered in different schools at different times of the first semester of the year 2009-2010. Page 20 2.b.5. Parents’ and Community Characteristics 69. Following the existing practice, we control for the average level of education of fathers and mothers of the students enrolled in the school. We also report whether at least one parent reports being unemployed, and among the possible occupations of the employed parents, we give special attention to farming. In order to further account for parents‟ preferences for education we register the number of books kept in the households (self-reported by parents), and we ask parents whether they agree or disagree (on a 5-point Likert scale) with the following statement: “There is not much point in sending my children to school, because education won‟t help them to find a better job when they grow up.” We report the proportion of parents in each school who disagree or strongly disagree with the statement. Finally, following Filmer and Pritchett (2001) we construct an index of household socio-economic status based on parents‟ answer to 13 questions aimed at registering whether the household owned a number of assets, such as a washing machine, a fridge, an oven, a car, a mobile phone etc. We conduct principal factor analysis on the answers to the 13 questions and report the resulting first factor, which represents our index of household socioeconomic status.26 70. While ethnic fragmentation is not a problem in Albania – only 2.5% of the surveyed stakeholders speak a language different than Albanian (Greek) in their households – divisions within the community based on wealth, and/or education and/or landholding, and/or political affiliation may prevent parents from collectively participate in the school, possibly leading to lower education outcomes.27 In order to measure the extent of divide within the community we asked all surveyed stakeholders the following question: “There are often differences between people living in the same village/neighborhood. For example, difference in wealth, income, social status, ethnic or ethnic-linguistic background. There can also be differences in religious or political beliefs, or there can be differences due to age or sex. To what extent do any such differences divide your village/neighborhood?” 71. We then registered the answers of the different school stakeholders on a scale from 1 (to a very small extent) to 5 (to great extent) and computed the average across respondents per school. In order to proxy the value given to education in the community, or lack thereof, we report the proportion of stakeholders that agreed or strongly agreed with the statement “Parents in my community still do not understand the benefits of education for their children”. We aimed to directly measure individuals‟ propensities to cooperate with other members of the community by asking respondents whether they agreed or disagreed (on a 5-point Likert scale) with the following two statements: (1) “In this village /neighborhood, there are people who want to take advantage”; (2) “most people in this village/neighborhood are willing to help if you need it”. We also asked how likely it is that if there were, for instance, a water supply problem people would cooperate to try to solve the problem. Finally, we registered whether in the past year, the respondent or any household member “participated in any community activities, in which people came together to do some work 26 Principal factor analysis is a technique used to summarize the information contained in a large number of variables to a smaller number by creating a set of mutually uncorrelated components of the data. The first factor is the linear index of all the observed variables that captures the most common variation among them. In the case of our index of socioeconomic status, the first factor captures 52% of the common variation among the 13 asset variables. 27 Retrospective studies of the relationship between education inputs and outcomes do not usually account for community characteristics among the possible determinants of school performance. However, Bjorkman and Svensson (2010), in their study – based on RCTs – of the impact of participatory programs on health outcomes in Uganda, have shown that division within the community, and particularly ethnic fractionalization, may prevent collective action for improved service provision, resulting in lower outcomes than in more homogeneous societies. Galiani et al. (2008) in their study of decentralization efforts in Argentinean secondary schools (based on a “political natural experiment”), have shown that areas that are poorly administrated and characterized by populations that do not usually voice their preferences may be left behind and end up worse off rather than better off following decentralization. Page 21 for the benefit of the community”. We aggregate the answers of all the surveyed school stakeholders in order to generate measure of community trust and cooperation. II.c. Variation in Descriptive & Performance Indicators across Albania’s Primary Schools 72. Table 1 presents our measures of school outputs, and compares schools in urban and rural locations (with less than 3,000 inhabitants). In the last column the results of tests for significant differences between rural and urban schools are reported, and where significant differences are found, are shown in bold. Table A.2.1 in the Appendix compares education outputs of schools located in the different REDs. Table 1. School Performance Rural Full sample Schools in rural areas Schools in urban areas vs. urban Mean Min Max sd Mean Min Max sd Mean Min Max sd p-value School Outputs Drop-out rate (2009) 0.04 0.00 0.30 0.04 0.04 0.00 0.04 0.04 0.03 0.00 0.22 0.04 0.089 Repetition rate (2009) 0.23 0.00 0.79 0.18 0.19 0.00 0.70 0.16 0.30 0.00 0.79 0.20 0.250 Fail rate in the 2009 final exam 0.01 0.00 0.26 0.02 0.01 0.00 0.26 0.03 0.01 0.00 0.08 0.01 0.105 Students‟ excellence in math 0.08 0.00 0.45 0.11 0.06 0.00 0.45 0.09 0.13 0.00 0.40 0.11 0.000 Students‟ excellence in language 0.23 0.00 0.79 0.18 0.19 0.00 0.70 0.16 0.30 0.00 0.79 0.20 0.000 Note: The p-values reported in the last column are generated by one-tailed tests of equality of means. 73. Schools in an urban location appear to outperform schools in rural locations in terms of students‟ achievement in the final exam. Overall, the proportion of students who do extremely well in math seems to be generally low and especially low in rural areas, where only 6% of the students who took the 2009 leaving examination got a 9 or a 10. While there does not seem to be a significant difference between rural and urban location with respect to the proportion of students who failed the final examination, the drop-out rate is significantly higher in rural locations. Conversely, schools in urban areas show higher repetition rates, although the difference is not statistically significant. A comparison between REDs (see Table A.2.1) shows Gjirokaster as the best performing RED, with 17% and 40% of the final year students excelling respectively in math and Albanian. The worst performing RED seems to be Kukes, where only 2% and 15% of the final year students got a 9 or a 10 respectively in math and in Albanian in the leaving examination. 74. Table 2 presents selected measures of the schools‟ physical and human resources. Overall, almost all schools in the sample seem to have problems relating to poor facilities and infrastructure. On average, 95% of the classrooms in the surveyed schools cannot be used in poor weather conditions, and schools suffer power shortages for about four days in a typical month. Schools in rural locations are significantly smaller, more likely not to have a phone, to experience power shortages, and not to have a library. 75. As for human resources, the student-teacher ratio is on average higher in urban locations, although there seems to be more variation in rural schools, where there can be up to 62 students per teacher. Teachers in urban schools have achieved higher levels of education and are more experienced; they are paid slightly more than rural teachers, but they do not seem to be more motivated. We do not observe significant differences across rural and urban schools with respect to school directors‟ characteristics. Table A.2.2 in the Appendix compares school resources across the different REDs. Page 22 Table 2. School Resources Rural Full sample Schools in rural areas Schools in urban areas vs. urban Mean Min Max Sd Mean Min Max sd Mean Min Max sd p-value N. students 354 30 1559 289 220 30 1559 219 590 128 1218 243 0.000 Infrastructures Can be reached by public transp. 0.31 1.00 0.00 0.54 0.28 1.00 0.00 0.55 0.35 1.00 0.00 0.52 0.29 No electricity (days in a month) 3.76 0.00 31.00 7.89 4.72 0.00 31.00 8.98 2.06 0.00 25.00 5.10 0.015 No phone 0.66 0.00 1.00 0.47 0.90 0.00 1.00 0.31 0.25 0.00 1.00 0.43 0.000 % desks in poor conditions 0.14 0.00 0.80 0.17 0.15 0.00 0.80 0.19 0.12 0.00 0.65 0.14 0.122 % classrooms that cannot be used in 0.948 0.00 1.00 0.145 0.94 0.00 1.00 0.168 0.956 0.00 1.00 0.883 0.294 poor weather conditions Schools with a library 0.783 0.00 1.00 0.413 0.713 0.00 1.00 0.454 0.908 0.00 1.00 0.292 0.002 Human resources School director years of experience 3.784 0.00 19.00 3.877 3.557 0.00 19.00 3.599 4.187 0.00 19.00 4.329 0.149 School director received school 0.59 0.00 1.00 .49.23 0.59 0.00 1.00 0.49 0.60 0.00 1.00 0.49 0.545 management training School director is a “local” 0.661 0.00 1.00 0.474 0.643 0.00 1.00 0.481 0.692 0.00 1.00 0.465 0.254 Student-teacher ratio 15.18 3.20 62.30 6.73 13.16 3.20 62.30 6.58 18.80 5.82 38.92 5.39 0.000 % teachers with university degree 0.80 0.00 1.00 0.22 0.77 0.00 1.00 0.25 0.86 0.40 1.00 0.15 0.004 or more Teachers‟ average years of 14.94 7.29 18.50 1.18 14.78 7.29 18.50 1.33 15.22 12.38 17.00 0.80 0.008 education Teachers‟ average years of teaching 16.94 4.60 34.20 5.69 16.07 5.40 34.20 5.66 18.49 4.60 33.60 5.44 0.003 experience % of “local” teachers 0.569 0.00 1.00 0.285 0.576 0.00 1.00 0.294 0.557 0.00 1.00 0.270 0.337 Motivation of teachers‟ in the 0.00 1.00 0.246 0.00 1.00 0.251 0.00 1.00 0.237 0.150 0.742 0.756 0.717 schools (self-reported) Average salary paid to teachers 35245 27750 41000 2489 34999 28625 41000 2561 35690 27750 39900 2307 0.038 Lowest salary paid to teachers 30909 20000 41000 3453 30394 20000 41000 3744 31842 23000 36400 2633 0.004 Note: The p-values reported in the last column are generated by either one-tailed tests of equality of means or Chi-square tests. 76. Table 3 reports statistics for our measures of top-down accountability and transparency in school practices. The number of inspections from RED representatives in the previous semester varies greatly across schools, from a minimum of 0 to a maximum of 36; the comparison between rural and urban schools highlights that rural schools are significantly less likely to receive inspections. Overall, in 20% of schools school director are not subject to any form of appraisal and sanction if they underperform. Moreover, about half of the surveyed teachers report that they are not subject to any penalty if they underperform. Perceptions of irregularities in the appointment of school directors and teachers are quite high, especially in rural areas, where 32% and 22% of the surveyed school stakeholders believe that often or very often school directors and teachers, respectively, are appointed to posts due to personal connections rather than qualifications. Teacher absenteeism does not seem to be a major problem in Albanian schools; on average, the number of days of unjustified absence in a typical month is only 0.18, with a minimum of 0 and a maximum of 5 days; it also does not seem to be different across rural and urban schools. Page 23 Table 3. Top-down Accountability (School averages) Urban Full sample Schools in rural areas Schools in urban areas vs. rural Mean Min Max sd Mean Min Max sd Mean Min Max sd p-value Top-down accountability N. of inspections from RED 5.04 0.00 36.00 5.52 3.90 0.00 35.00 4.06 7.02 0.00 36.00 7.00 0.000 Underperforming school director is not subject to penalties (self- 0.20 0.00 1.00 0.40 0.21 0.00 1.00 0.41 0.18 0.00 1.00 0.39 0.698 reported by school director) Underperforming teachers are not subject to penalties (self-reported 0.49 0.00 1.00 0.26 0.48 0.00 1.00 0.27 0.51 0.00 1.00 0.25 0.233 by teachers) Teachers' days of absence in May 0.18 0.00 5.17 0.70 0.22 0.00 5.17 0.83 0.13 0.00 2.25 0.37 0.203 2009 Transparency School keeps a teacher absence 0.00 1.00 0.31 0.00 1.00 0.27 0.91 0.00 1.00 0.29 0.90 0.92 0.546 register Public display of teachers' absence 0.12 0.00 1.00 0.33 0.15 0.00 1.00 0.36 0.08 0.00 1.00 0.27 0.163 Public display of students' final 0.56 0.00 1.00 41.50 0.70 0.00 1.00 0.46 0.82 0.00 1.00 0.39 0.079 grades % respondents who perceive corruption in the appointment of 0.28 0.00 0.82 0.18 0.26 0.00 0.82 0.17 0.32 0.03 0.77 0.18 0.008 school directors % respondents who perceive corruption in the appointment of 0.20 0.00 0.64 0.15 0.19 0.00 0.64 0.15 0.22 0.00 0.64 0.15 0.099 teachers Note: The p-values reported in the last column are generated by either one-tailed tests of equality of means or Chi-square tests. 77. Simple pair-wise correlations between the different measures of accountability and transparency reveal that not keeping a teacher absence register is associated with underperforming school directors not being subject to penalties. Table A.2.3, in the Appendix, compares the schools in the different REDs with respect to accountability and transparency of school administrations. The data show that the number of RED inspections is on average significantly higher in the REDs of Fier and Shkoder, and significantly lower in Korce, where it is also most likely that a bad performing school director is not subject to penalties. Perceptions of corruption in the appointments of school director and teachers are most severe in Vlore, Diber and Elbasan. 78. To contrast with the indicators above, Table 4 provides information about the bodies that have been established for participatory “bottom up” accountability and parental involvement in the school. The top panel of the table reports statistics on the composition and meetings of the school board (SB). The number of SB meetings in the previous semester (January – May 2009) varies from a minimum of zero to a maximum of five, with no significant differences between rural and urban schools. Contrary to the MoES directives, in about 30% of schools, the school director participates in school board meetings and expresses his or her vote, again with no significant differences between rural and urban schools. In about 81 percent of the schools, according to the answers of parent SB members, parent class representatives are elected by parents rather than appointed by teachers; and in 43% of the schools teacher school board members are elected by teachers rather than appointed by the school directors. These proportions are significantly higher in urban schools as compared to rural schools. 79. The second panel of Table 4 reports the proportions of parents who are not informed about: the existence of the school board; the existence of parent class representatives; the name of their child‟s math teacher; and the name of their child‟s Albanian teacher. About 43% of the parents do Page 24 not know about the existence of the school board28 and 28% do not know that there exist class parent representatives. Lack of information is more severe in rural schools than in urban schools. A closer look at the data (not reported in the table) show that of the 72 percent of parents who are aware of the existence of parent class representatives, only half know their names; and of the parents who know that parent representatives exist and are elected by parents, 77 percent did not vote in the most recent election.29 Table 4. Participatory “Bottom Up” Accountability and Parent Involvement (School averages) Rural Full sample Rural areas Urban areas vs urban Mean Min Max sd Mean Min Max sd Mean Min Max sd p-value The School Board: N. of meetings in the past semester 1.92 0.00 5.00 0.76 1.89 0.00 4.00 0.71 1.98 0.00 5.00 0.83 0.215 School directors participates and votes 0.29 0.00 1.00 0.46 0.30 0.00 1.00 0.46 0.28 0.00 1.00 0.45 0.632 Parent representatives are elected by parents 0.81 0.00 1.00 0.39 0.78 0.00 1.00 0.42 0.86 0.00 1.00 0.35 0.113 Teacher representatives are elected by 0.43 0.00 1.00 0.50 0.37 0.00 1.00 0.48 0.54 0.00 1.00 0.50 0.016 teachers Lack of information about participatory organs and teachers: Parents not knowing that there is a SB 0.43 0.00 1.00 0.25 0.48 0.00 1.00 0.24 0.36 0.00 1.00 0.26 0.001 Parents not knowing that there are parent 0.28 0.00 1.00 0.24 0.34 0.00 1.00 0.24 0.16 0.00 0.70 0.18 0.000 class representatives Parents not knowing the name of the math 0.25 0.00 0.80 0.21 0.26 0.00 0.80 0.21 0.22 0.00 0.80 0.20 0.143 teacher Parents not knowing the name of the 0.23 0.00 0.80 0.20 0.25 0.00 0.70 0.21 0.20 0.00 0.80 0.19 0.055 language teacher Parental involvement: N. meetings with math teacher, reported by 4.22 1.40 8.50 1.48 3.99 1.60 8.40 1.42 4.62 1.40 8.50 1.50 0.003 parents (past semester) N. meetings with language teacher, reported 1.50 8.10 1.44 1.70 8.10 1.38 1.50 8.10 1.49 4.15 3.94 4.52 0.005 by parents (past semester) % Parents who participate in parent-teacher 0.72 0.34 1.00 0.11 0.72 0.34 1.00 0.12 0.74 0.43 0.90 0.10 0.092 meetings, reported by teachers Parents who filed a complaint in education 0.21 0.00 0.80 0.18 0.22 0.00 0.80 0.19 0.18 0.00 0.50 0.15 0.080 matters % respondents who think that lack of parental involvement is a major problem in 0.215 0.00 0.93 0.17 0.21 0.00 0.73 0.19 0.21 0.00 0.93 0.17 0.49 the school Note: The p-values reported in the last column are generated by either one-tailed tests of equality of means or Chi-square tests. 80. The bottom panel of Table 4 reports the number of meetings parents said they held with the math and the language teachers during the previous semester (January-May 2010), and the proportion of parents who usually participate in parent-teacher meetings, as reported by teachers. We also report the proportion of parents who filed a complaint concerning education matters at least once in their life. The average number of meetings that parents said they had with teachers in the previous semester (January-May 2009) is on average around 4, yet it varies considerably across schools, and is significantly higher in urban schools. The proportion of parents who usually participate in parent-teacher meetings also varies greatly across schools, from a minimum of 34% to a maximum of 100%, with an average of 72%, there seems to be a significant difference between rural and urban schools in the proportion of participating parents, albeit such difference is quite small in magnitude. Lack of parental involvement seems to be considered a problem in about 20% 28 Only about 80 percent of the parents informed about the existence of the SB know that there are parent SB members, and of these, 72 percent do not know their names. 29 The great majority of these parents (74 percent) did not vote because they were not informed about the date and location of the election. Page 25 of the schools, with no significant differences between rural and urban locations. Table A.2.4 in the Appendix presents statistics of our measures of participatory accountability and parental involvement across schools located in the different REDs. 81. The indicators in Table 5 confirm that as expected the average years of schooling of both the mother and the father of the students, and the number of books owned by the household, are significantly higher in urban locations than in rural locations. Albanian parents have on average completed primary school and at least two years of secondary or vocational school. The household economic status, as well as the proportion of employed parents, and the proportion of parents employed in occupations other than farming are also significantly higher in urban areas. About 90% of the surveyed parents seem to value the education of their children, and more so if they are located in urban areas. Table 5. Parents’ Characteristics (School averages) Rural Full Sample Schools in rural areas Schools in urban areas vs. urban Mean Min Max Sd Mean Min Max sd Mean Min Max sd p-value Average years of schooling of 6.89 14.80 1.66 7.20 11.80 0.90 6.89 14.80 1.67 10.14 9.31 11.62 0.000 mothers Average years of schooling of 6.67 15.78 1.54 7.60 12.80 1.04 6.67 15.78 1.60 10.40 9.74 11.57 0.000 fathers N. of books in the household 23.47 0.70 122.9 21.70 14.78 0.70 81.50 12.42 38.85 1.10 122.9 25.83 0.000 Never helps children with 0.16 0.00 0.70 0.16 0.16 0.00 0.60 0.17 0.14 0.00 0.70 0.14 0.215 homework Currently unemployed (self- 0.23 0.00 0.80 0.19 0.25 0.00 0.80 0.19 0.19 0.00 0.60 0.17 0.028 reported) Most common occupation of 0.07 0.00 0.60 0.11 0.11 0.00 0.60 0.12 0.01 0.00 0.30 0.04 0.000 parents: farmer (self-reported) Household economic status* 0.017 -1.291 1.693 0.596 -0.269 -1.291 0.831 0.388 0.522 -0.894 1.692 0.566 0.000 Disagree with:"There is not much point in sending my children to 0.40 1.00 0.13 0.40 1.00 0.14 0.60 1.00 0.11 0.90 0.88 0.93 0.009 school -education won‟t help them to find a better job" Note: * the index of household economic status is the principal factor obtained from principal factor analysis over a set of 13 variables describing the assets owned by the household. The p-values reported in the last column are generated by either one-tailed tests of equality of means or Chi-square tests. 82. Table 6 reports some salient characteristics of the community to which the schools in the sample belong. The extent of “divide” (as defined in the earlier section) within the community seems to be moderate, as suggested by the 2.6 average of our index computed on a scale of 1 to 5, across all the surveyed school stakeholders. Urban areas seem to be characterized by greater divide than rural areas. Furthermore, in urban areas we observe less trust and cooperation than in rural areas, as shown by the significantly higher proportion of respondents who think that in their neighborhood there are people who want to take advantage, and the fewer proportion of those who think that members of their community would be willing to help if needed, and would come together to jointly solve a common problem such as possible water shortage. Page 26 Table 6. Community Characteristics (Averages of answers of all school stakeholders) Full Sample Rural areas Urban areas Rural vs. urban Mean Min Max Sd Mean Min Max sd Mean Min Max sd p-value Divide within the village/neighborhood Extent of divide (1-low; 5- high) 2.67 1.55 3.89 0.48 2.56 1.55 3.77 0.43 2.88 1.83 3.89 0.50 0.000 Agree with: “Parents in my community still do not understand the benefits of 0.489 0.00 0.933 0.19 0.481 0.00 0.867 0.18 0.504 0.00 0.933 0.20 0.220 education for their children” Trust and cooperation Think that “In this village /neighborhood, there are people who want to take 0.32 0.03 0.77 0.17 0.29 0.03 0.72 0.16 0.36 0.05 0.77 0.18 0.010 advantage” Think that “most people in this village/neighborhood are willing to help if 0.65 0.13 1.00 0.19 0.68 0.13 1.00 0.19 0.59 0.18 0.90 0.18 0.000 you need it” Think that “village members are likely to 0.03 1.00 0.19 0.03 1.00 0.19 0.30 1.00 0.18 0.69 0.71 0.66 0.048 cooperate in the case of water shortage” Participated in community activities in the 0.24 0.00 0.73 0.17 0.23 0.00 0.73 0.17 0.25 0.00 0.67 0.17 0.249 last year Note: The p-values reported in the last column are generated by either one-tailed tests of equality of means or Chi-square tests. II.d. Correlates of School Performance 83. In this section we aim to identify the factors that are significantly correlated with school performance (as measured by the five education “outputs” identified earlier). In the first half simple pair-wise correlations are presented. These are followed by estimates from regression analysis. 84. Table 7 presents only the variables that are significantly correlated with at least one of the five education outputs. In other words, if a variable for which descriptive statistics presented earlier is not reported in Table 7 that indicates that the variable is not correlated with any of the five measures of school performance. The top panel of Table 7 shows that most proxies of school resources are correlated with the proportion of students who excelled in math and Albanian in the 2009 final examination. In particular, excellence in both math and language appears more likely in urban schools, schools that have a phone (which might proxy the quality of local infrastructures), and schools who do not tend to suffer from power shortages. The presence of a library in the school also seems to be positively correlated with students‟ performance in the math test, but not in the Albanian test. 85. As for the availability and quality of human resources, there is no significant correlation between education outcomes and: characteristics of the school director; education of teachers; and student-teacher ratio (not reported in the table). We do however find years of teaching experience, teachers‟ motivation and wages to be significantly and positively correlated with students‟ excellence in the language exam. Teachers‟ motivation seems also to be negatively correlated with students‟ drop-out rate. 86. The second panel of Table 7 focuses on measures of top-down accountability and transparency. The only factors found to be correlated with at least one of the education outcomes are the number of visits from RED inspectors, and perceptions of corruption in the appointment of teachers. The former is correlated positively with students‟ excellence in Albanian and negatively with drop-out rate; the latter is positively correlated with drop-out rate. Page 27 Table 7. Pair-wise Correlations Drop-out Repetition Excellence in Excellence in Fail rate rate rate Albanian math RESOURCES Rural location 0.104 0.051 0.096 -0.296*** -0.313*** Availability of public transportation 0.067 0.042 0.099 0.137* -0.004 Days with no electricity 0.045 -0.057 0.004 -0.201*** -0.163** No phone 0.134* 0.039 0.073 -0.280*** -0.384*** Library 0.046 -0.039 0.048 0.066 0.242*** Teachers‟ years of experience 0.063 0.018 0.083 0.137* 0.091 Teachers are motivated -0.196* -0.045 -0.078 0.228*** 0.088 Average wage of teachers 0.004 -0.059 0.024 0.132* 0.113 TOP-DOWN ACCOUNTABILITY AND TRANSPARENCY N. of RED inspections -0.167** -0.029 -0.063 0.209*** 0.111 Perceptions of corruption in the appointment of teachers 0.191** 0.028 0.050 0.103 -0.017 PARTICIPATORY ACCOUNTABILITY AND PARENTAL INVOLVEMENT N. of SB meetings (Jan-May 09) - 0.238*** -0.0805 - 0.045 0.043 0.045 School directors participates and votes 0.149* -0.069 0.027 -0.039 -0.070 % parents not knowing that there is a SB 0.102 0.073 0.187** -0.139* -0.184** % parents not knowing that there are parent class reps. 0.076 0.183** 0.136* -0.123* -0.189** % parents not knowing the name of the math teacher 0.068 -0.028 0.159** -0.236*** -0.175** % parents not knowing the name of the language teacher 0.121 -0.052 0.161** -0.190** -0.159** Number of meetings with the math teacher (Jan-May 09) -0.248*** -0.008 -0.149** 0.204*** 0.296*** Number of meetings with the math teacher (Jan-May 09) -0.251*** -0.014 -0.119 0.198*** 0.317*** “ lack of parental involvement in the school is a problem” 0.107 0.096 0.063 -0.132* -0.166** PARENTS’ CHARACTERISTICS Average mother education -0.227*** -0.184** -0.178** 0.322*** 0.358*** Average father education -0.225*** -0.215*** -0.178** 0.263*** 0.364*** Average n. of books in the household -0.150** -0.129* -0.126* 0.180** 0.186** % currently unemployed 0.111 0.048 0.024 -0.147* -0.109 % farmers 0.118 0.186** 0.184** -0.222*** -0.136* Average index of economic status -0.178** -0.148** -0.171** 0.282*** 0.306*** Note: ***significant at the 1% level, **significant at the 5% lever, * significant at the 10% level. 87. The third panel of Table 7 highlights that a large number of measures of participatory accountability and parental involvement are correlated with education outcomes. In particular, students‟ excellence in math and language appears significantly more likely where parents are informed about participatory organs, know the names of the math and Albanian teachers, have more frequent meetings with teachers, and where lack of parental involvement is not considered to be a problem in the school. Parents‟ lack of information is also positively correlated with failure and repetition rates. Interestingly, the drop-out rate seems to be higher in schools where there are fewer SB meetings, where the school director can participate and vote in such meetings, and where parents held fewer meetings with teachers. Page 28 88. The bottom panel of Table 7 confirms that family characteristics are important correlates of education outcomes. The average years of schooling of students‟ mother and father, the number of books owned by the household and the index of socioeconomic status are all strongly correlated, with the expected sign, with all five school performance variables. None of the community characteristics presented in Table 6 appear to be significantly correlated with school-level education outputs. 89. Following the exposition and data analysis structure in previous sections, for each measure of school performance four OLS regressions have been conducted, each including only one category of independent variables: (1) school resources; (2) top-down accountability and transparency; (3) participatory accountability and parental involvement; and (4) family and community background. The OLS regressions have then been run including all four categories, and the risk of multicollinearity bias has been lowered by using principal factor analysis. In all regressions RED fixed effects have been used, and the standard errors have been clustered by district to account for possible unobservable inter-dependencies between schools belonging to the same district. Table 8 and Table 9 report estimates for drop-out, repetition and failure rates, and for excellence in math and Albanian, employing one category of explanatory variables at a time. Table 10 reports estimates for all school performance measures, and including all categories of explanatory variables. 90. The top panel of Table 8 shows very few correlates with drop-out, repetition and failure rates among school resources. The only significant results seem to be the positive correlation between rural location and failure rate and the negative relationship between teachers‟ motivation and drop-out rate. None of the top-down accountability and transparency measures appear to be correlated with drop-out, repetition and failure rates. In contrast, the number of SB meetings and whether the school director usually participates and votes in the meetings are, respectively, negatively and positively correlated with drop-out and repetition rates. Moreover, lack of parental involvement in the school, as assessed by all school stakeholders, is positively correlated with students‟ repetition rate. 91. Note that, in the participatory accountability regressions (Panel 3), in order to reduce the bias generated by multicollinearity between the different proxies of parental awareness and involvement, principal factor analysis was conducted on three groups of highly related variables: (i) the measures of parents‟ lack of information with respect to the existence of class parent representatives and school board; (ii) the measures of parents‟ lack of information of the names of head teacher, math teacher, language teacher and school director; (iii) the measures of parents‟ participation in parent-teacher meetings, with different teachers, between January and May 2009. Thus, in the regression, the resulting indexes have been included: parents‟ awareness of participatory organs; parents‟ awareness of teachers‟ names; and meetings with teachers. Similarly, when testing for family background we do not control for the unemployment status of parents, nor their possible occupation as farmers, since both variables are highly correlated with the household index of socioeconomic status. Among the proxies for the students‟ family background, the only significant result is the negative correlation between fathers‟ education and repetition rate. Page 29 Table 8. Correlates with Drop-out, Repetition and Failure Rates Dependent variable: Drop-out rate Repetition rate Failure rate (final exam) (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) RESOURCES Rural school 0.00 -0.00 0.01* (0.009) (0.002) (0.008) Public transportation 0.01 0.00 0.01 (0.007) (0.002) (0.009) No electricity (days) 0.00 -0.00 0.00 (0.000) (0.000) (0.001) No telephone 0.01 0.00 0.00 (0.011) (0.005) (0.007) No library -0.01 0.00 -0.01 (0.008) (0.003) (0.012) Motivated teachers -0.04*** 0.01 -0.01 (0.012) (0.008) (0.017) Teacher experience -0.00 0.00 0.00 (0.001) (0.000) (0.001) Ln (teacher wage) 0.05 -0.02 -0.03 (0.046) (0.020) (0.045) TOP-DOWN ACCOUNTABILITY AND TRANSPARENCY RED inspections -0.00 -0.00 -0.00 (0.000) (0.000) (0.000) SD not subject to sanctions 0.00 -0.00 0.01 (0.009) (0.003) (0.012) Teachers not subject to sanctions -0.01 -0.00 0.00 (0.014) (0.004) (0.015) Teachers absence-May09 0.00 -0.00 -0.01 (0.003) (0.001) (0.005) Teacher absence register -0.01 0.00 0.01 (0.020) (0.004) (0.017) Corruption in teacher appointment 0.03 0.00 0.04 (0.023) (0.007) (0.033) PARTICIPATORY ACCOUNTABILITY AND PARENTS’ INVOLVEMENT N. SB meetings -0.01*** -0.00* -0.00 (0.004) (0.001) (0.005) Director can vote 0.01 -0.00** -0.00 (0.009) (0.002) (0.013) No info about participatory organs 0.01 0.01 -0.00 (0.014) (0.009) (0.010) No info about teachers‟ names -0.00 -0.01 0.02 (0.010) (0.006) (0.011) N. of meetings with teachers -0.01 0.00 -0.01 (0.005) (0.005) (0.016) Lack of parental involvement in school 0.03 0.02* 0.01 (0.032) (0.012) (0.029) PARENTS’ CHARACTERISTICS Mother yrs. of education 0.00 0.00 -0.00 (0.004) (0.002) (0.005) Father yrs. of education -0.01 -0.00* -0.00 (0.004) (0.002) (0.005) N. books in hh 0.00 0.00 0.00 (0.000) (0.000) (0.000) Economic status -0.00 -0.00 -0.02 (0.006) (0.006) (0.010) Constant -0.42 0.05** 0.06*** 0.10*** 0.26 0.02 0.02* 0.04** 0.31 0.04* 0.06*** 0.08* (0.483) (0.024) (0.015) (0.030) (0.216) (0.015) (0.013) (0.019) (0.479) (0.022) (0.020) (0.040) Observations 167 164 161 169 174 169 168 177 170 165 165 173 R-squared 0.218 0.176 0.231 0.199 0.093 0.068 0.130 0.102 0.116 0.117 0.131 0.124 Note: Robust standard errors, in parentheses, have been adjusted to account for clustering within districts. *** p<0.01, ** p<0.05, * p<0.1. Page 30 92. Table 9 reports results obtained by conducting the same empirical procedure, with the proportion of students who excelled in math (columns 1 to 4) and in language (columns 5 to 8) as dependent variables. The estimates in the top panel confirm that a number of school resources and teachers‟ characteristics are significantly correlated with students‟ excellence in math and language. In particular, the percentage of students excelling in Albanian is significantly higher in urban schools which suffer fewer days with no electricity, where teachers are motivated, and have more teaching experience. The percentage of students excelling in math is higher in schools that have a telephone and in those with a library. 93. Among the top-down accountability and transparency measures, more RED inspections and fewer days of teachers‟ absence are associated with better performance in the Albanian exam. Teachers‟ absence, and whether the school has a teacher absence register, seem to affect -negatively and positively, respectively- students‟ performance in the math exam. 94. The estimates presented in the third panel of Table 9 suggest that lack of parental involvement in the school is negatively associated with students‟ excellence in both math and Albanian. Moreover, the proportion of students who excelled in math is higher the higher the number of meetings parents had with teachers, and lower if parents have no information about participatory accountability bodies. 95. Among the family background proxies, fathers‟ education and household economic status seem to positively affect respectively students‟ excellence in math and excellence in Albanian language. Moreover, in communities where differences among community members create divisions to a larger extent, students are less likely to excel in both math and language. Page 31 Table 9. Correlates with Students’ Excellence in Albanian and Math Dependent variable: % students who excelled in Albanian % students who excelled in math (final exam 2009) (final exam 2009) (1) (2) (3) (4) (5) (6) (7) (8) RESOURCES Rural school -0.09** -0.01 (0.042) (0.025) Public transportation 0.03 -0.01 (0.031) (0.017) No electricity (days) -0.00*** -0.00 (0.001) (0.001) No telephone -0.02 -0.06*** (0.043) (0.020) No library -0.00 -0.04** (0.036) (0.017) Motivated teachers 0.23*** 0.06 (0.076) (0.037) Teacher experience 0.00** 0.00 (0.002) (0.002) Ln (teacher wage) -0.03 -0.04 (0.226) (0.127) TOP-DOWN ACCOUNTABILITY AND TRANSPARENCY RED inspections 0.01** 0.00 (0.004) (0.001) SD not subject to sanctions 0.02 0.03 (0.042) (0.023) Teachers not subject to sanctions -0.01 -0.03 (0.065) (0.031) Teachers absence-May09 -0.03** -0.01*** (0.012) (0.004) School has a teacher absence register 0.04 0.05* (0.056) (0.023) Corruption in teacher appointment 0.06 0.01 (0.128) (0.066) PARTICIPATORY ACCOUNTABILITY AND PARENTS’ INVOLVEMENT N. SB meetings 0.01 -0.00 (0.017) (0.009) Director can vote -0.02 -0.02 (0.048) (0.016) No info about participatory organs -0.02 -0.03** (0.033) (0.017) No info about teachers‟ names -0.04 -0.01 (0.032) (0.018) N. of meetings with teachers 0.06 0.05* (0.038) (0.023) Lack of parental involvement is a problem -0.15* -0.09** (0.089) (0.038) PARENTS’ AND COMMUNITY CHARACTERISTICS Mother yrs. of education 0.03 0.01 (0.021) (0.013) Father yrs. of education 0.00 0.03*** (0.019) (0.009) N. books in hh -0.00 -0.00 (0.001) (0.001) Economic status 0.08* 0.03 (0.028) (0.023) Extent of divide in the community -0.07** -0.03** (0.028) (0.014) Constant 0.58 0.13 0.22*** 0.00 0.40 0.05 0.12*** -0.18* (1.312) (0.078) (0.037) (0.175) (2.352) (0.037) (0.027) (0.088) RED fixed effects YES YES YES YES YES YES YES YES R-squared 0.250 0.204 0.200 0.254 0.325 0.153 0.218 0.253 Note: Robust standard errors, in parentheses, have been adjusted to account for clustering within districts. *** p<0.01, ** p<0.05, * p<0.1. Page 32 96. Table 10 reports regression estimates for all five school performance measures, controlling for all categories of independent variables. Therefore, the estimates presented in Table 10 check for the robustness of the results highlighted in Tables 7, 8 and 9. Table 10. Correlates with School-level Education Outcomes Dependent variables Drop-out Repetition Failure % students % students rate rate rate excelling in excelling in Albanian math (1) (2) (3) (4) (5) Rural school -0.00 -0.01 0.01 -0.07* -0.02 (0.015) (0.005) (0.013) (0.039) (0.024) Poor facility and resources (first factor) -0.01 0.00 -0.00 0.01 -0.02** (0.008) (0.002) (0.009) (0.023) (0.009) Teacher experience -0.00 0.00 0.00 0.01** -0.00 (0.001) (0.000) (0.001) (0.002) (0.002) Motivated teachers -0.02 0.02* -0.02 0.19** 0.03 (0.017) (0.009) (0.023) (0.086) (0.038) RED inspections -0.00 0.00 0.00 0.01 0.00 (0.000) (0.000) (0.001) (0.003) (0.002) SD not subject to sanctions 0.00 0.00 0.01 0.03 0.03 (0.011) (0.003) (0.012) (0.039) (0.018) Teachers not subject to sanctions -0.01 0.00 0.01 -0.01 -0.05* (0.015) (0.004) (0.014) (0.066) (0.028) Teachers absence-May09 0.00 -0.00** -0.01* -0.01 -0.00 (0.003) (0.001) (0.005) (0.013) (0.008) School has a teacher absence register -0.00 0.00 0.02 0.03 0.05** (0.020) (0.005) (0.016) (0.056) (0.021) Corruption in teachers‟ appointments 0.03 0.01 0.05* -0.00 -0.04 (0.028) (0.008) (0.030) (0.131) (0.064) N. of SB meetings -0.01** -0.00** -0.00 -0.02 -0.01 (0.004) (0.001) (0.007) (0.027) (0.012) No information about participatory organs (parents) 0.00 0.00 0.00 0.01 -0.02 (0.016) (0.008) (0.012) (0.032) (0.019) No information about names of teachers (parents) -0.01 -0.01 0.01 -0.03 -0.01 (0.009) (0.007) (0.014) (0.033) (0.020) N. of meetings with teachers -0.00 0.01 -0.01 0.04 0.03 (0.006) (0.007) (0.016) (0.033) (0.021) Lack of parental involvement in the school 0.02 0.03** 0.02 -0.09 -0.09** (0.034) (0.013) (0.029) (0.097) (0.042) Mother years of education 0.00 0.00 -0.00 0.02 0.00 (0.005) (0.002) (0.007) (0.020) (0.011) Father years of education -0.01 -0.00 0.00 -0.01 0.02* (0.005) (0.002) (0.006) (0.023) (0.010) N. of books in the household -0.00 -0.00 0.00 -0.00 -0.00 (0.000) (0.000) (0.000) (0.001) (0.000) Economic status -0.00 -0.01 -0.01 0.05 0.00 (0.010) (0.007) (0.012) (0.040) (0.029) Extent of divide in the community 0.00 -0.00 -0.01 -0.07** -0.04** (0.006) (0.004) (0.008) (0.031) (0.019) Constant 0.13** 0.02 0.03 0.12 0.00 (0.053) (0.020) (0.072) (0.244) (0.127) RED fixed effects YES YES YES YES YES R-squared 0.298 0.188 0.194 0.391 0.433 Note: Robust standard errors, in parentheses, have been adjusted to account for clustering within districts. *** p<0.01, ** p<0.05, * p<0.1. 97. The estimates for drop-out, repetition and failure rates are not very informative as most of the variables are rendered statistically insignificant a large set of controls are included. Where variable appear significant, their substantive relationship to outcomes is difficult to interpret. For example, teachers‟ motivation appears positively correlated with repetition rates, possibly suggesting that more motivated teachers are more demanding of students. Along the same lines, repetition and failure rates are lower in schools where teachers‟ absence was more severe in May 2009. Drop-out and repetition rates appear respectively higher and lower the higher the number of school board meetings; repetition rates are also higher in the case of lack of parental involvement in Page 33 the school. None of the proxies for family background seem to affect drop-out, repetition and failure rates. 98. Interestingly, different sets of variables seem to be correlated with students‟ excellence in math and in Albanian. For instance, the rural location of the school, and the experience and motivation of teachers are negatively correlated with students‟ performance in Albanian, but not in math. None of the proxies for top-down and participatory accountability, nor parental involvement or family background, are correlated with the proportion of students who got a 9 or a 10 in Albanian. The only variable that seems significantly correlated with students‟ excellence in Albanian, with the expected negative sign, is the extent of divide within members of the community. 99. The results obtained for students‟ excellence in math and Albanian are more enlightening. Students‟ excellence in math appears to be correlated with a larger set of factors. First of all, the proportion of students excelling in math is lower in schools characterized by poor school facilities.30 While teachers‟ characteristics do not seem to play a role, teacher appraisal and transparency with respect to teachers‟ absence from school do; indeed, students excellence in math is more likely in schools where teachers believe that they would be penalized if they underperform, and where a teacher absence register exists and is regularly used. Lack of parental involvement in the school also appears to be correlated, negatively, with students‟ performance in the final math exam. A father‟s education also seems to have strong, positive, impact on children‟s performance in math. Finally, the extent of divide in the community appears to matter by reducing the proportion of students excelling in both math and language. 30 This was measured by an index obtained through principal factor analysis to reduce multicollinearity bias. The index was constructed using: days with no electricity; absence of a telephone in the school; percentage of desks in poor conditions and absence of a library. Page 34 Part III. Parent Demand for Accountability and Better Governance III.a. The Importance of Parents’ Preferences 100. From a now widely accepted conceptual framework presented in earlier sections of this report, there are two routes of accountability: a long route and a short route. The long route connects the recipients of the services and policymakers in the ideal institutional setting, with well- functioning electoral systems, free from patronage and clientelism, and with perfect information on the standards that public service providers should meet. In such a setting, service recipients can hold policymakers accountable for the delivery of poor public services through the exercise of their votes. However, this ideal setting is rarely if ever the reality experienced by households, especially in developing and transition countries, where the route between service recipients and policymakers is often long, institutionally fragile, and strewn with obstacles. Decentralization may shorten the route to a certain extent, but electoral inefficiencies, elite capture and imperfect information can still hinder households from exercising accountability.31 In such contexts, the shorter “local” route to accountability, which connects service recipients directly with service providers, is likely to be more viable and effective (World Bank, 2004). 101. In the education sector, as with other areas of public service, improving accountability and governance requires that school directors and teachers are assigned clear responsibilities; are monitored while exercising these responsibilities; and are subject to penalties when their performance falls short of predetermined targets. In education, the most direct -and therefore shortest- route of accountability would be for all parents to be involved in both the formulation of the responsibilities and outcomes that enter the contracts signed by teachers, and the implementation of penalties that make these contracts binding.32 But because the active participation of all parents is logistically unfeasible, only a sub-group of parents, representing the interests of all parents, are given decisional power in schools as active members of a school board or some other structure. There is general agreement that the design of the specific participatory institutions adopted by schools should have the clear objective of inducing (directly or indirectly) the participation of all parents while minimizing the chances of elite capture, 33 and that the existence and functioning of such institutions should be known to all parents. 102. In Albania‟s basic schools, according to MoES guidelines, parent members of the School Boards should be elected parent class representatives. The parent class representatives of each grade are together members of the school‟s Parent School Council. Members of this Council elect from among themselves parent members of the School Board. Thus a necessary condition for the School Board to function as intended, is that its parent members are elected by parent class representatives, who themselves have been elected by parents participating fully in the accountability process. If the parent class representatives who are supposed to elect the parent members of the school board have conflict of interest because they have been appointed by teachers, the representation of parents' interests in the School Board is necessarily compromised. 31 See Oates (1972) and Azfar et al. (2001) for general discussion of the possible limitations of decentralization. 32 In practice, school-based management (SBM) programs may or may not give decisional power to the parents. Some programs give parents decisional power over the school budget only; others involve parents also in the choice of the curriculum and textbooks; others give parents responsibilities in the hiring and firing of teachers; others invite parents to participate in the development of a school improvement plan, and possibly in fund-raising. School-based management programs may also exclude parents from participating in the decision process. There seems to be three different models of SBM (see Wohlstetter and Odden, 1992). In the first model, the primary objective is to give the community control over decision-making; in the second model, teacher- and not parent- empowerment is the primary objective of the reform; finally, in the third model it is the principal to be given the most control lover decision-making. 33 It is for instance reasonable to believe that parents are more likely to feel empowered and thus get involved if the sub-group of parents who are assigned decisional power are elected by all parents rather than appointed to their position by the school director and/or the teachers, and/or the head of the community. Elected parent representatives are also less likely to be “captured” by the interests of school director and teachers. Page 35 103. But do parents get involved in participatory accountability structures? What are the factors that determine whether parents get involved? And if they do participate, does this have an impact on student and school performance? 104. As discussed in earlier sections of this report, the empirical literature exploring the impact of parental involvement in education is relatively new. The existing studies can be categorized generally into two types: those using retrospective survey data of the sort presented earlier; and those that use an experimental or randomized controlled trial (RCT) approach, which involves the random allocation of participatory institutions of various types to different schools. Retrospective studies focus mainly on the existence of participatory institutions and actual parental participation, and their association with school outcomes. As discussed earlier, while these retrospective, ex post evaluations provide encouraging findings, they can be compromised by endogeneity and omitted variables34 as schools with participatory institutions or more parental involvement in general may differ from other schools in a number of ways that may both drive the difference in parental participation and have an independent effect on school outcomes. This problem can, of course, be addressed by randomly allocating participatory programs to schools and measuring the effect by conducting ex ante and ex post surveys, but this approach can be costly. 105. Parents‟ willingness to participate in holding teachers to account depends on the actual and expected costs and benefits associated with participating. The actual costs include: transportation costs; the opportunity cost of time; and the internal psychological costs that parents may suffer when holding teachers to account, especially when they have personal relationships with teachers or feel somehow inferior or beholden to teachers. The expected benefits from participation include the better education that parents expect their children to receive if teachers are held to account and any intrinsic benefit associated with believing that holding teachers to account is “the right thing to do”. This section of the report discusses the use of experimental techniques to help measure these less tangible intrinsic costs and benefits to parents involvement in structures designed to help them hold service providers to account. 106. The experimental techniques that were deployed in parallel with the survey of school stakeholders in Albania, do not overcome the limitations of retrospective evaluations. Nor are they a substitute for the power of RCTs conducted with an ex-ante and ex-post evaluation strategy. However, they can provide valuable insights that help policymakers avoid what can be costly mistakes in the design and piloting of innovative interventions, particularly in country contexts where there is little or no prior information to guide their efforts. III.b. Controlled Lab Type Experiments: Just Playing Games? 107. Alongside the Albania School Stakeholder Survey presented in the part two of this report, parents and teachers were invited to participate in the specifically designed controlled, lab-type experiments (LTEs). The retrospective survey was designed to quantify the institutions for increasing accountability (including both external “top down” and participatory “bottom up” structures); to measure the awareness of stakeholders (especially parents), of the school institutional framework in general, and participatory institutions in particular; and to measure actual parental involvement with the school. To complement this valuable data, the LTE was designed to yield insights into parents‟ preferences for participation, i.e. their willingness to forgo personal payoffs in order to hold teachers to account.35 34 Gunnarsson et al. (2009), in their survey-based ex post evaluation of participatory institutions in eight Latin American countries, found that controlling for the possible endogeneity of school resources and parents‟ participation, substantially increases the estimated impact of parental participation on students‟ test scores. 35 It also provided a measure of teachers‟ levels of intrinsic motivation that will be analyzed elsewhere. Page 36 108. Lab-type experiments (LTEs) –sometimes referred to simply as “behavioral games”- facilitate the measurement of individuals‟ values, beliefs and preferences that cannot be captured in survey data. They have been used in a number of countries and contexts, involving a wide variety of groups; from university students, and children, to adults without formal literacy or numeracy in foraging small-scale societies.36 The specific details of each game will vary, depending on the specific preferences that they are designed to capture. LTEs are always designed with a clear research objective in mind, and employ real money in order to recreate as closely as possible the incentives that individuals face in the real-life situations that the games aim to simulate and elucidate. 109. Experimentally generated measures of individual preferences have been shown to correlate with individuals‟ behavior in a variety of contexts of direct interest to policy makers. Barr and Packard (2000 and 200#) show that experimental measures of risk aversion and time preference are significantly associated with the choices of workers whether to take up formal pension plans in Chile, and even when choosing between plans structured either as individual savings (“defined contribution”) or risk sharing (“defined benefit”) in Peru. Carpenter and Seki (forthcoming) show that Japanese fishermen‟s preferences for cooperation, as measured in a lab-type public goods game, predict their productivity when pooling their catches with other fishermen in everyday life. 37 Karlan et al. (2005) find that individual trustworthiness, measured using a lab-type trust game conducted with members of a non-profit “village banking” organization in Peru, predicts loan repayment. Barr and Serneels (2009) show that individual tendencies to reciprocate, as measured using a lab-type trust game played by employees of manufacturing firms in Ghana, predict both individual- and firm-level productivity. Serra, Serneels and Barr (forthcoming) show that the pro- social motivations of final year medical and nursing students in Ethiopia, as measured using a lab- type generalized trust game, predict their decision to work in non-profit health facilities rather than in the public or private for profit sectors. Finally, Barr and Zeitlin (2010) find that the decisions of Ugandan teachers towards their pupils‟ parents in games correlate strongly with the teachers‟ actual absenteeism rates. 110. With the exception of Barr and Zeitlin (2010), LTEs have not been incorporated into baseline surveys for RCTs aimed at improved accountability in the education sector. This is a pity, because specifically designed LTEs could help us address the main limitation of randomized evaluations, as highlighted by Bardhan (2005), i.e. their unsuitability for “deciphering the mechanisms through which certain outcomes are generated (the „why‟ and the „how‟) and the social dynamics that are involved” (p. 16). Moreover, they could provide early insights relating to important questions the full addressing of which would require costly and time-consuming RCTs. And, as such, they can lead to better RCT design and a corresponding increase in the information generated by time and money spent on RCTs. 111. In Albania, three LTEs were adapted and conducted with the school stakeholders who participated in the survey (see a fuller explanation in Appendix 3). In the LTE most directly related to parents‟ preferences for accountability, one player is invited to divide a sum of money between herself and another player, and a third player can pay to have money subtracted from the first player‟s allocation.38 It is the third players‟ behavior that is of particular interest here as by choosing to “fine” they reveal both what they consider to be unfair behavior on the part of the first player, as well as their own willingness to pay to express this opinion. In other words, the third player is 36 See Barr et al. (2009) for an example of a cross-cultural experimental study based on the implementation of similar experimental designs to the ones we use here in 15 different small-scale societies. 37 Similarly, preferences measured in the context of a public goods game have been found to predict the likelihood of Brazilian fishermen to over-exploit their common fishing ground (see Fehr and Leibbrandt, 2008). 38 The LTE discussed is a version of Fehr and Fischbacher‟s (2004) Third Party Punishment Game (TPG). Page 37 expressing their willingness to hold the first player to account for how the resources were divided (“fairly” or “unfairly”) between themselves and the passive second player. 112. The most important differences between the game as was conducted in Albania‟s primary schools, and how it has been used in other contexts are that: (a) the subjects are ordinary people, rather than university students in a lab; (b) the game roles were assigned to subjects with reference to and with the aim of matching the roles they (potentially) play in a real, ongoing accountability scenario. Specifically, teachers were invited to play in the role of “givers”, i.e. the individuals who have to decide how to divide an endowment between themselves and another subject. Parents were invited to play either in the roles of the “recipients” and “third party punishers”. The exact parameters and sequence of the games and important details about how they were conducted are explained in Appendix 3. 113. The objective when designing the game was to render observable the net effects of the parents‟ internal psychological costs and benefits associated with holding teachers to account. This is important as these intangible costs and benefits as correspond to parents‟ preferences for active participation in the teacher-parent accountability relationship. To this end, the game was designed to match the school context in terms of actual and possible relationships, while differing from it with respect to its simplicity and the availability of information to stakeholders. By matching the relational-space in the game to the actual or potential relational-space in the school context, the likelihood of parents‟ preferences that are salient when they are considering holding teachers to account also being salient in the game is increased. 114. There were, nevertheless, many important contextual differences between the relationships simulated in the game and the actual accountability relationship between parents and teachers outside of the game. Three of these are worthy of particular note. First, in the actual school context parents‟ decisions to hold school and teachers to account usually require that the parents expose themselves, whereas in the game parents were guaranteed at least some anonymity - teachers who were fined do not know which of the five parents fined them. Second, while in the school context the material costs associated with participation may differ across parents, in the game the material cost of holding teachers to account was fixed and known by all. Finally, in the actual school context, parents may be uninformed about the existing participatory institutions and the actual performance of teachers whereas in the game, parents were fully informed about how they can hold teachers to account and can condition their decisions on accurate measures of teacher performance. The only that was left ambiguous was how much teachers should allocate to parents, but prior studies employing this type of LTE on non-student subjects who are partially or fully integrated into monetized economies, provide a basis for expecting the 50:50 split to emerge as a reference point of “fairness” (the results presented below are consistent with this expectation). Therefore, assuming that the relational-space matching worked, the game can be viewed as a counterfactual to the existing situation in Albania‟s primary schools, in which information is not constrained, parents wishing to participate in holding teachers to account are afforded some protection (anonymity), and the consequences of their participation are clearly defined. 115. Given this LTE design, it is only under very specific conditions that we expect parents‟ behavior in the game to correlate with their behavior and attributes in the actual school context and this being the case, if such correlations are observed, they can be taken as evidence that the specific conditions apply. A series of conjectures are formulated here to guide the empirical analysis and the interpretation of the findings from the game.  If participation in fining in the game correlates with participation in the form of voting in the parent representative elections or being an elected representative, it indicates that the two types of participation are motivated by the same preferences and the LTE and school context Page 38 are well matched. Conversely, lack of correlation may indicate either that the LTE lacks external validity, i.e., that the preferences that are salient in the school context are not invoked in the LTE or that, parent‟s participation in the school context is subject to external constraints that are not present in the LTE;  If parents who participate in real life but in a compromised way -i.e. as parent representatives who have been appointed by teachers or school directors rather than being elected- are less willing to participate in fining in the LTE, it indicates precisely the sort of conflicts of interest that would arise in malfunctioning systems of accountability;  If participation in the experiment correlates with being informed about the participatory bodies existing in the school, it indicates that being informed and participating in holding teachers to account in the LTE are motivated by the same preferences. Conversely, if they are not correlated, it indicates that being informed is not driven by a preference to participate and is, instead, owing to some sort of external constraint or system failure. 116. Moreover, it is reasonable to expect parents‟ individual characteristics relating to the expected costs and benefits associated with participation -such as parents‟ economic status, education and perceptions of malpractice at school- play a significant role in their decision to participate in the game and/or real life. In particular, parents‟ economic status could matter because, even if in the game setting the cost of fining is fixed, poorer parents are likely to have higher marginal utility for the cash made available in the game, and are therefore less likely to participate in fining. As for parents‟ education, it could affect the value parents‟ give to their children‟s education, and, consequently, the intrinsic benefit parents gain from holding teachers to account. Both greater economic status and education are also likely to diminish feelings of inferiority of parents toward teachers, reducing non-monetary costs associated with holding teachers to account. Finally, perceptions of malpractice in the appointment of teachers might encourage greater participation by reducing non-monetary costs and increasing non-monetary pleasures from fining teachers. However, while economic status, education and perceptions of corruption in the appointment of teachers are likely to increase participation in the game, they will affect participation in real life only if de facto participation is possible in the school and parents are (or can take steps to become) informed about how to participate in accountability structures. III.c. Results from Lab-type Experiments with Albania’s School Stakeholders 117. Figure 1 summarizes the fining decisions made by the parents in the role of the third party observer with the power to punish teachers by fining if they consider the teachers‟ decisions selfish or unfair.39 The data points marked by chevrons indicate the percentage of parents (vertical axis) who said they would fine a teacher who made a specified allocation (horizontal axis) to the passive parent. Only 16 percent of the parents decided never to fine teachers, regardless of their behavior in the game, even when teachers decided to keep the entire endowment (2000 Lek). As expected, the decision to fine depends negatively on the amount teachers allocate to (passive) parents.40 39 Figure 2 in appendix shows allocations of teachers to (passive) parents, i.e. the orange players, in the third-party punishment game. The mode at 50:50 is unusually strong, especially compared to results obtained from student samples. There are quite a lot of offers higher than 50%. This would be cause for concern in an experiment where those fulfilling the offering and receiving role were drawn from the same pool, but in this case they are not and know that they are not. If a teacher had a strong prefernece for fairness and considered himself materially better off than the average parent in the receiving role he would give more than 50%. Note that a teacher with no preference for fairness but who believes that the finers will fine unfair offers will rarely make a high offer – to do so he would have to have quite extreme beliefs about the likelihood of being fined. 40 The strategies of all parents are consistent, i.e. there are not irregular switches in the fining decisions (such as for instance, not fining a parent if he/she keeps all the endowment, but fining him/her for giving a positive amount). This might seem unusual to Page 39 Figure 3. Parents’ Fining Behavior 90 80 % third-party parents who fined 70 60 50 40 30 20 10 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 Amount the teachers gave to the (passive) parent (% of endowment) Source: Serra, Barr and Packard (2011) 118. Fining the teacher is costly to the third party parent, and the decision to fine depends on the intrinsic benefit from imposing the fine outweighing the fixed cost. Assuming that the intrinsic benefit to the parent of fining increases as the gap between what the parent believes the teacher ought to allocate and what they actually allocate widens (with the latter below the former), then only when the gap gets sufficiently wide will the fine be levied. For this reason, the minimum unfined offer chosen by each third party parent gives us a direct measure of that parents‟ intrinsic benefit from fining, i.e., their preference for participating in holding teachers to account. 119. Table 11 shows that for 77% of parents the minimum unfined offer is less than half of the endowment, 21% of the parents chose a minimum unfined offer of precisely one half of the endowment , and only 2% chose minimum unfined offers greater than one half. It is worth highlighting, particularly for readers not familiar with experimental methods and the results obtained from the same game in other countries (e.g., Barr et al. 2009), that parents in Albania are revealing a comparatively strong willingness to pay to fine teachers who share unfairly within the game. researchers who are familiar with the implementation of strategy elicitation to non-students in lab-type experiments conducted in the field. However, the lack of inconsistencies can be explained by the fact that all parents were literate. The minimum years of schooling are 4 only for 1% of the sample. 44% of the parents completed 8 years of schooling and the remaining parents completed secondary school (27%) or even achieved an undergraduate or postgraduate degree (30%). Page 40 Table 11. Minimum Unfined Offer Minimum Un-fined Offer Percent Cum. 0 16.44 16.44 0.1 14.22 30.67 0.2 10.67 41.33 0.3 13.00 54.33 0.4 22.33 76.67 0.5 21.44 98.11 0.6 0.78 98.89 0.7 0.78 99.67 0.8 0.33 100 Source: Serra, Barr and Packard (2011) 120. To investigate the correlations between parents‟ behavior in game and their behavior and attributes in the school context two series of multivariate analyses are conducted. The first of these is a series of Probit analyses focusing on a dependent variable that equals one if a parent in the third-party role participates in fining and zero if the parent never fines (columns 1 to 3 in Table 12). The second is a series of linear regressions taking the minimum un-fined offer chosen by each parent in the third-party role as the dependent variable (columns 4 to 6 in Table 12). The sample upon which this analysis is based is restricted to the parents who fine at least once. Positive (negative) coefficients in the Probits identify variables that are associated with parents deciding to participate in (abstain from) fining. Positive (negative) coefficients in the linear regressions identify variables that are associated with a lower (higher) degree of tolerance of teachers‟ selfish behavior. 121. We start our empirical analysis by testing whether parent class representatives are more likely to participate in holding teachers to account in the game, while controlling for parents‟ individual characteristics, such as economic status, education, and perceptions of corruption in the appointment of teachers. We then look at the parent representatives who, contrary to the de jure institutional provisions, have been appointed by teachers. We then expand the empirical specification to control for the decision to vote in the most recent elections of parent class representatives. Finally, we include a measure of parents‟ lack of information with respect to the existing participatory accountability structures.41 41 In order to reduce the bias generated by multicollinearity between the various proxies of parental awareness and involvement, we conducted factor analysis on the measures of parents‟ lack of information with respect to the existence of class parent representatives and School Board; we include the first factor in the empirical specifications appearing in columns 3 and 6 of Table 12. Page 41 Table 12. Parents’ Preferences for Accountability Dep. Variable: Dep. Variable: Dummy equal to 1 if parent Minimum un-fined offer participates in fining Probit Probit Probit OLS OLS OLS (1) (2) (3) (4) (5) (6) Parent representative 0.02 0.03 0.03 -0.02 -0.02 -0.02 (0.025) (0.025) (0.024) (0.012) (0.013) (0.013) Parent representative appointed by a teacher 0.05 0.05 0.06 -0.09*** -0.09** -0.08** (0.069) (0.074) (0.065) (0.035) (0.035) (0.035) Voted in the last parent rep elections -0.03 -0.02 0.01 0.01 (0.022) (0.023) (0.011) (0.012) No information about participatory organs 0.02 0.01 (0.017) (0.008) Perceptions of corruption in the appointment of teachers -0.01 -0.01 -0.02 0.02* 0.02* 0.02* (0.024) (0.025) (0.025) (0.012) (0.012) (0.012) Economic status 0.03* 0.03* 0.03* -0.00 -0.00 0.00 (0.014) (0.014) (0.014) (0.007) (0.007) (0.007) Years of schooling 0.01* 0.01* 0.01** 0.00 0.00 0.00 (0.003) (0.003) (0.003) (0.001) (0.001) (0.001) Rural location 0.02 0.01 0.01 -0.01 -0.01 -0.02 (0.029) (0.029) (0.029) (0.014) (0.014) (0.015) Female 0.01 0.01 0.01 -0.01 -0.01 -0.01 (0.022) (0.022) (0.022) (0.010) (0.010) (0.010) % parents among friends and relatives in the session 0.04 0.05 0.05 -0.01 -0.01 -0.01 (0.032) (0.032) (0.032) (0.011) (0.011) (0.011) Constant 0.36*** 0.36*** 0.35*** (0.034) (0.034) (0.035) RED dummies YES YES YES YES YES YES Observations 900 900 900 752 752 752 R-squared (or pseudo) 0.16 0.16 0.16 0.31 0.31 0.31 Note: Robust standard errors, in parentheses, have been adjusted to account for clustering within sessions conducted in different schools. Columns 1 to 4 report marginal effects of continuous variables and the effect of a change from 0 to 1 for dichotomous variables. *** p<0.01, ** p<0.05, * p<0.1. Source: Serra, Barr and Packard (2011) 122. The estimates presented in Table 12 suggests that the decision to actively participate in the accountability system and -conditional on participating- the extent to which parents hold teachers accountable, are motivated by different sets of factors. Parents‟ decision to participate in fining in the game appears to depend primarily, and positively, on parents‟ economic status and education. Parent representatives, no matter whether they were elected or appointed, and parents who participated in voting are no more willing to participate in holding teachers accountable in the game; conversely uninformed parents are no less willing to participate in the game. 123. The results obtained from the linear regressions taking the minimum un-fined offer as the dependent variable highlight how effective participatory accountability also depends on the extent to which the existing institutions are de facto participatory. Indeed, the estimates in Columns 4 to 6 indicate that parent representatives that have been appointed to their position are relatively less demanding of teachers (i.e. their minimum un-fined offer is significantly lower). Although it is unclear if appointed parent representatives have become less demanding as a result of being appointed by teachers/school directors, or have been appointed because they are less demanding, the negative coefficient on the “Appointed” variable is evidence of a conflict of interest in parent representatives who are put in place in accordance with de facto institutions that do not match the de jure governmental provisions. This interpretation is supported by another important result: parents who believe that teachers are always or often appointed to positions owing to personal connections rather than their qualifications, are more demanding of teachers (i.e. their minimum unfined offer is significantly higher). 124. To summarize the results from the LTE, parents‟ willingness to take an active role in participatory accountability, as measured by the game, is positively associated with parents‟ economic status and education. Parents who are uninformed about the participatory accountability Page 42 institutions available at the school level, and parents who are not currently involved with the school, are no less willing to forgo personal resources to hold teachers to account in the game. Once parents decide to hold teachers to account, the rigor with which they do so is lower if they have acquired a role in the de facto accountability system in a compromised way, i.e. by being appointed parent class representatives by teachers or the school director, and higher if they believe that teachers have been appointed to their position in a compromised way. Page 43 References Angrist, J., and Lavy, V. (1997), “The effect of a change in the language of instruction on the return to schooling in Morocco”, Journal of Labor Economics 15 (1), S48–S76. Banerjee, A., P. Bardhan, K. Basu, R. Kanbur, and D. Mookherjee: “New Directions in Development Economics: Theory or Empirics?” A symposium in Economic and Political Weekly, 2005. Banerjee, A., Rukmini B., Duflo, E., Glennerster, R. And S. Khemani (2010), “Pitfalls of Participatory Programs: Evidence from a Randomized Evaluation in Education in India”, American Economic Journal: Economic Policy, Vol. 2, Issue 1: 1-30. Bardhan, P.: “Theory or Empirics in Development Economics”. A symposium in Economic and Political Weekly, 2005. Barr, A. And A. Zeitlin: “Dictator games in the lab and in nature: External validity tested and investigated in Ugandan primary schools”, CSAE Working paper WPS/2010-11, 2010. Barr, A. and D. Serra: “Corruption and Culture: An Experimental Analysis”. Journal of Public Economics, Vol. 94, Issues 11-12: pp. 862-869, 2010. Barr, A. and P. Serneels: "Reciprocity in the Workplace", Experimental Economics, 12(1), 99-112, 2009. Barr, A., C. Wallace et al. (2009) "Homo Æqualis: A Cross-Society Experimental Analysis of Three Bargaining Games", CSAE working paper WPS/2009-02, 2009. Barrera-Osorio, F., Fasih, T., Patrinos, H. A., and L. Santibanez: Decentralized Decision-Making in Schools. The theory and Evidence on School-based Management. Human Development, The World Bank, Washington, DC., 2009. Bird, R.M.: “Decentralizing infrastructure: For good or for ill?” in: A. Estache (Ed.) Decentralizing Infrastructures: Advantages and Limitations, World Bank Discussion papers 290, Washington D.C., 1995. Björkman, M and J. Svensson (2010), "When Is Community-Based Monitoring Effective? Evidence from a Randomized Experiment in Primary Health in Uganda," Journal of the European Economic Association, vol. 8(2-3), pages 571-581, 04-05. Brandts, J. and G. Charness: "Hot vs. cold: Sequential experimental games", Experimental Economics, 2: 227-238, 2000. Brandts, J., Charness, G.: "The strategy versus the direct-response method: A survey of experimental comparisons". Working paper, University of California Santa Barbara, 2009. Brosig, J., J. Weimann, and C.-L. Yang (2003) "The hot versus cold effect in a simple bargaining experiment", Experimental Economics, 6(1): 75-90. Carpenter, J. and E. Seki: “Do Social Preferences Increase Productivity? Field Experimental Evidence from Fishermen in Toyoma Bay”, Economic Inquiry, forthcoming. Cason, T. and V. Mui: "Social influence in the sequential dictator game", Journal of MathematicalPsychology 42: 248-265, 1998. Chaudhury, N., Hammer, J., Kremer, M., Muraldhiran, K., Rogers, H. (2006). “Missing in action: Teacher and health worker absence in developing countries”. Journal of Economic Perspectives 20 (1), 91–116. de Laat, Joost, Michael Kremer, and Christel Vermeersch (2008), “Teacher Incentives and Local Participation,” mimeo. Deininger, K. and P. Mpuga: “Does greater accountability improve the quality of public service delivery? Evidence from Uganda”, World development, Vol. 33, Issue 1, pp. 171-191, 2005. Duflo E, Dupas, P., and M. Kremer: “Peer Effects, Pupil-Teacher Ratios, and Teacher Incentives: Evidence from a Randomized Evaluation in Kenya”, MIT Mimeo, September 2009. Page 44 Duflo, Esther, Rema Hanna, and Stephan Ryan (2007), “Monitoring Works: Getting Teachers to Come to School,” NBER Working Paper No. 11880. Elster, J.: “Social Norms and Economic Theor”, Journal of Economic Perspectives, 3(4): 99-117, 1989. Fehr, E. and A. Leibbrandt: “Cooperativeness and Impatience in the Tragedy of the Commons.” IZA Discussion Paper No. 3625, 2008. Fehr, E. and U. Fischbacher: “Third-party punishment and social norms”, Evolution and Human Behavior 25, 63–87, 2004. Filmer D, Pritchett LH (2001): “Estimating wealth effect without expenditure data – or tears: an application to educational enrollments in states of India”, Demography, 38:115-32 Galiani S, Gertler P. and E. Schargrodsky (2008) “School Decentralization: Helping the good get better, but leaving the poor behind”, Journal of Public Economics. Gertler, Patrinos and Rubio-Codina (2008), “Empowering parents to improve education: Evidence from rural Mexico”, World Bank‟s Policy Research Working Paper Series. Glewwe P., Holla A. and M. Kremer (2008), “Teacher incentives in the developing world”, in M. Springer, ed., Performance Incentives: Their Growing Impact on American K-12 Education. Brookings Institution Press. 2009. Glewwe P., M. Kremer and S. Moulin (2009) “Many Children Left Behind? Textbooks and Test Scores in Kenya”, American Economic Journal: Applied Economics 1(1):112-135. Glewwe, P., Grosh, M., Jacoby, H., Lockheed, M. (1995). “An eclectic approach to estimating the determinants of achievement in Jamaican primary education”. World Bank Economic Review 9 (2), 231–258. Glewwe, P., Jacoby, H. (1994). “Student achievement and schooling choice in low-income countries: Evidence from Ghana”. Journal of Human Resources 29 (3), 843–864. Glewwe, P., Kremer, M., Moulin, S., Zitzewitz, E. (2004). “Retrospective vs. prospective analyses of school inputs: The case of flip charts in Kenya”. Journal of Development Economics 74, 251–268. Glewwe, Paul and Michael Kremer (2005), “Schools, Teachers, and Education Outcomes in Developing Countries,” Eric Hanushek and Finish Welch (eds.), Handbook of Economics of Education 1, Elsevier. Güth, W., S. Huck, and W. Mueller: "The relevance of equal splits in ultimatum games", Games and Economic Behavior, 37: 161-169, 2001. Gunnar S. E. and D. Filmer (2007), “Autonomy, participation, and learning in Argentine schools - findings and their implications for decentralization”, Education Economics. Gunnarsson, V., Orazem P.F, Sánchez M.A. and A. Verdisco (2009), “Does Local School Control Raise Student Outcomes? Evidence on the Roles of School Autonomy and Parental Participation”, Economic Development and Cultural Change. Gunnarsson, V., Orazem P.F, Sánchez M.A. and A. Verdisco: “Does Local School Control Raise Student Outcomes? Evidence on the Roles of School Autonomy and Parental Participation”, Economic Development and Cultural Change, 2009. Harbison, R., Hanushek, E. (1992). Educational Performance of the Poor: Lessons from Rural Northeast Brazil. Oxford University Press. Houtenville J. and K. S. Conway (2007), “Parental Effort, School Resources, and Student Achievement”, Journal of Human Resources. Houtenville J. and K. S. Conway: “Parental Effort, School Resources, and Student Achievement”, Journal of Human Resources, 2007. Jimenez E. and Y. Sawada (1999) “Do community-managed school work? An evaluation of El Salvador‟s EDUCO Program”, World Bank Economic Review, 13(3), 415-442. Jimenez E. and Y. Sawada: “Do community-managed school work? An evaluation of El Salvador‟s EDUCO Page 45 Program”, World Bank Economic Review, 13(3), 415-442, 1999. Karlan, D. : “Using Experimental Economics to Measure Social Capital and Predict Financial Decisions”, American Economic Review, 95(5), pp. 1688-1699, 2005. Kingdon, G. (1996). “The quality and efficiency of public and private schools: A case study of urban India”. Oxford Bulletin of Economics and Statistics 58 (1), 55–80. Kremer, M. and A. Holla (2009), “Improving Education in the Developing World: What Have We Learned from Randomized Evaluations?” The Annual Review of Economics, 1, 19.1–19.33. Kremer, M. and A. Holla: “Improving Education in the Developing World: What Have We Learned from Randomized Evaluations?” The Annual Review of Economics, 1, 19.1–19.33, 2009. Kremer, M. and D. Chen (2001), “An Interim Report on a Teacher Attendance Incentive Program in Kenya.” Mimeo, Harvard University. Kremer, M. and D. Chen: “An Interim Report on a Teacher Attendance Incentive Program in Kenya.” Mimeo, Harvard University, 2001. Krueger, A. (1999), "Experimental Estimates Of Education Production Functions," The Quarterly Journal of Economics, MIT Press, vol. 114(2), pages 497-532. Lewis, M. and G. Petterson (200): “Governance in Education: Raising Performance” mimeo. Lewis, M. and G. Petterson: “Governance in Education: Raising Performance” mimeo, 2009. Oates: Fiscal Federalism, Harcourt Brace Jovanovich, New York, 1972. Platteau, J. P.: “Behind the Market Stage Where Real Societies Exist – Part II : The Role of Moral Norms”, Journal of Development Studies, Vol. 30(4), pp. 386-422, 1994. Posner, R. A. and E. B. Rasmusen: “Creating and enforcing norms, with special reference to sanctions” International Review of law and economics 19, 369-382, 1999. Reinikka R. and J. Svensson “Working for God? Evaluating service delivery of (religious) not-for-profit health facilities in Uganda”, Journal of the European Economic Association, forthcoming. Reinnika R. and J. Svensson: “Fighting corruption to improve schooling: Evidence from a newspaper campaign in Uganda”, Journal of the European Economic Association, 3 (2-3): 259-267, 2005. Serra, D. , P. Serneels and A. Barr “Intrinsic motivations and the non-profit sector: Evidence from Ethiopia”, with A. Barr and P.Serneels, Personality and Individual Preferences, special issue on Economics and Psychology, forthcoming. Serra, D., A. Barr and T. Packard (2011) ““Participatory accountability in Albanian primary schools: A lab- type experimental study”, mimeo. Skoufias E. and J. Shapiro (2006) "Evaluating the Impact of Mexico's Quality Schools Program: The Pitfalls of Using Nonexperimental Data", World Bank‟s Policy Research Working Paper Series. Skoufias E. and J. Shapiro: "Evaluating the Impact of Mexico's Quality Schools Program: The Pitfalls of Using Nonexperimental Data", World Bank‟s Policy Research Working Paper Series, 2006. Tan, J.-P., Lane, J., Coustere, P. (1997), “Putting inputs to work in elementary schools: What can be done in the Philippines?”, Economic Development and Cultural Change 45 (4), 857–879. Wohlstetter and Odden, Educational Administration Quarterly November 1992, 28: 529-549. World Bank: Poverty and Education in Albania: Who Benefits from Public Spending? Human Development Unit, Report No. 31983-AL, 2005. World Bank: Poverty and Education in Albania: Who Benefits from Public Spending? Human Development Unit, Report No. 31983-AL, 2005. World Bank: World Development Report 2004: Making Services Work for Poor People, Oxford University Press, 2004. Page 46 World Bank: World Development Report 2004: Making Services Work for Poor People, Oxford University Press, 2004. Young, P. : "Social norms", in the New Palgrave of Economics, Second Edition, edited by Steven N. Durlauf and Lawrence E. Blume. London: Macmillan, 2008. Page 47 APPENDIX 1 Design and Sampling Strategy for the Albania School Stakeholder Survey and Field Experiment October – December 2009 This Appendix presents the details and data collected with the Albania School Stakeholder Survey and Field Experiments 2009. The appendix starts with a description of the survey sample, and tables showing selected variables collected in the survey. An explanation of the field experiments (or “games”) and results from analysis of the experiments are also presented. Although the survey and experiments offer a very rich source of information already, the nationally representative sample of Albania‟s basic schools and the methodology applied in the survey were designed specifically to provide a “baseline” from which the impact of school autonomy reforms being considered by the Government of Albania can eventually be rigorously evaluated. The Albania School Stakeholder Survey and Field Experiment collected data from community members with a stake in the performance of public schools: parents, teachers, members of school governance structures, and headmasters. Each category of stakeholder answered a specific questionnaire, and a separate questionnaire was directed at a school administrative representative to gather basic information about the school facility. Once the sample of schools was selected, to ensure a representation of primary schools in Albania, students were identified from target grade levels, and the respondents to the survey were selected with respect to these students. Students themselves were not subjects of the survey directly. The respondents from each category of stakeholder were as follows: Parents: 10 parents (of the 10 students chosen at random); - 3 parents of students enrolled in grade 3; - 7 parents of students enrolled in grade 6; Total number of parents to be surveyed: 2000 Teachers: 5 teachers (the teachers of the10 students chosen at random and, if these were less than 5, additional teachers of other students at the same school) Total number of teachers surveyed: 1000 School board members: One “teacher member”, one “parent member” and one “community member”; The parent and the teacher members were other than the parents and teachers already identified to be respondents; Total number of school board members: 600 School Principal: One per school Total number of school principals surveyed: 200 School administrative representative: One representative was chosen per school, who answered factual questions about the school and its facilities. This could also be the school principal or the head teacher. Total number of school administrative representatives surveyed: 200 Page 48 The data resulting from each of the different questionnaires has been compiled in separate databases that can be linked for analysis. The survey and field experiment was conducted between October and December 2009 in a representative sample of 180 public primary schools, offering grades one to nine. 20 additional school were surveyed to act as a back-up.42 There are 2691 public basic (i.e. primary) level schools in Albania, of which 12 percent are located in urban areas. Out of the 2691 primary schools, we defined as “eligible” those schools with five or more pupils in grade three, and ten or more students in grade seven. Only 1623 of the 2691 primary schools in the complete list met our eligibility criterion. However, only 9 percent of all Albanian public primary school pupils attend the ineligible schools. The list of eligible schools was then divided into 37 strata according to district and either six, four or two schools were randomly sampled from each strata, the number depending on the number of pupils attending primary schools in the district.43 In all REDs but two (Diber and Gjirokaster) the majority of surveyed schools where located in villages or hamlets with less than 3000 inhabitants. The average number of pupils per school in our sample is 327, although this number falls to 200 if we exclude the urban schools. The average surveyed school employs 20 teachers, although the number falls to 14 if we, once again, exclude the urban schools.44 For each school, we collected data from a stratified random sample of community members with a stake in the performance of the school, i.e. the school director, teachers, members of the school board and parents. Each type of stakeholder answered a specific questionnaire. In addition, a separate questionnaire was directed at a school administrative representative to gather basic information about the school facility. Once the sample of schools was randomly selected, three and seven students were randomly selected from grades three and seven, respectively. The teacher of the third graders and four teachers of the seventh graders were then randomly selected to participate in the survey, together with the parents of the selected pupils. Therefore, in each school we aimed to collect data from ten parents, five teachers, three members of the school boards and the school directors, leading to total sample sizes of 1800 parents, 900 teachers, 180 school directors and 540 school board members. However, administering the questionnaire to members of the school board proved challenging; in 13 schools we were only able to survey one or two rather than three school board members.45 Table A.1.1 provides summary statistics of the surveyed schools by RED. The different number of schools selected in each RED reflects both differences in the number of districts belonging to each RED, and differences in the proportions of students attending primary schools in each district. In all REDs but two (Diber and Gjirokaster) the majority of surveyed schools where located in a village or rural area with less than 3000 inhabitants. The average number of pupils per school is 327, yet this number falls to 200 if we exclude urban schools. Similarly, the average surveyed school employs 20 teachers, whereas an average school located in a village employs 14 teachers. 42 We collected data from 20 additional schools, which offer only grade one to five, and are dependent on another school in terms of administration (including the school director). Since in these schools we were unable to administer the questionnaires to all the stakeholders and conduct the LTEs with parents and teachers, we exclude them from the current analysis. 43 Since the district of Tirana contained twice the percentage of pupils as well as twice the number of independent eligible schools, it was divided in two strata, Tirana Municipality and Tirana District, leading to a total of 37 strata. 44 Summary statistics for the schools sampled in each RED can be found in Appendix Table A1. 45 In addition to the survey, parents and teachers were invited to participate in specifically designed lab-type experiments (LTEs). The LTE were designed to measure parents‟ preferences for participation, i.e. their willingness to forgo personal payoffs in order to hold teachers to account, as well as to provided a measure of teachers‟ levels of intrinsic motivation that will be analyzed elsewhere. See Serra, Barr and Packard (2010) for further details. Page 49 Table A.1.1. Sampling Summary % of Average Min. N. Max. N. Average N. Min. N. Max. N. Rural N. school total N. students students teachers teachers teachers location sample students RED Berat 15 8% 238 8 552 15 1 31 67% Diber 17 9% 315 23 923 19 3 50 50% Durres 13 6.5% 369 14 935 19 1 47 61% Elbasan 23 11.5% 281 9 823 18 2 39 74% Fier 20 10% 310 51 884 21 3 65 75% Gjirokaster 12 6% 283 38 681 20 2 33 42% Korce 19 9.5% 270 13 801 12 1 41 74% Kukes 16 8% 211 84 601 18 7 30 87% Lezhe 17 8.5% 393 91 1211 23 7 56 60% Shkoder 16 8% 304 27 1006 18 3 42 69% Tirane 20 10% 558 14 1559 29 1 84 60% Vlore 12 6% 365 84 1218 26 8 60 58% Total 200 100% 327 8 1559 20 1 84 67% Source: Albania School Stakeholder Survey and Field Experiment 2009 Page 50 APPENDIX 2 Statistics from the Albania School Stakeholder Survey, by RED Table A.2.1. School performance by RED Drop-out rate Repetition rate Fail rate Students’ excellence Students’ excellence in RED N. of schools (2009) (2009) (final exam 2009) in math language Berat 14 0.03 0.02 0.05 0.09 0.18 (0.03) (0.07) (0.08) (0.12) (0.18) Diber 13 0.03 0.00 0.01 0.10 0.31 (0.03) (0.01) (0.03) (0.16) (0.18) Durres 12 0.03 0.01 0.03 0.10 0.18 (0.02) (0.01) (0.06) (0.11) (0.17) Elbasan 20 0.06 0.01 0.02 0.06 0.21 (0.04) (0.01) (0.04) (0.07) (0.21) Fier 16 0.03 0.01 0.01 0.09 0.27 (0.03) (0.01) (0.03) (0.13) (0.18) Gjirokaster 12 0.03 0.00 0.02 0.17 0.40 (0.03) (0.00) (0.04) (0.12) (0.22) Korce 18 0.05 0.01 0.06 0.08 0.23 (0.11) (0.02) (0.11) (0.10) (0.18) Kukes 15 0.01 0.00 0.01 0.02 0.15 (0.02) (0.01) (0.02) (0.04) (0.16) Lezhe 16 0.04 0.00 0.04 0.08 0.18 (0.07) (0.00) (0.07) (0.11) (0.16) Shkoder 14 0.008 0.01 0.01 0.06 0.21 (0.02) (0.01) (0.02) (0.07) (0.13) Tirane 18 0.01 0.00 0.01 0.08 0.16 (0.02) (0.01) (0.02) (0.08) (0.15) Vlore 12 0.004 0.00 0.00 0.11 0.34 (0.01) (0.01) (0.02) (0.09) (0.17) Total 180 0.03 0.01 0.02 0.08 0.23 (0.06) (0.02) (0.06) (0.11) (0.18) Page 51 Table A.2.2. School resources by RED Director % No Average Average Distance received teacher electricity Student/ years of salary N. of Rural from No % desks school with a RED (# days in teacher teaching paid to schools location nearest telephone in poor management university typical ratio experience teacher town condition training degree month) Berat 14 0.64 14.57 4.50 0.50 0.12 0.57 14.88 0.79 18.13 34827.01 (0.50) (11.43) (8.83) (0.52) (0.15) (0.51) (6.10) (0.17) (6.98) (1917.75) Diber 13 (0.62 15.86 8.69 0.69 0.22 0.36 15.30 0.59 14.17 33453.71 (0.51) (15.23) (13.49) (0.48) (0.21) (0.50) (5.32) (0.24) (4.71) (3092.63) Durres 12 0.58 7.50 5.42 0.58 0.11 0.42 20.02 0.72 21.87 35160.98 (0.51) (9.75) (9.17) (0.51) (0.09) (0.51) (15.00) (0.25) (6.61) (1638.34) Elbasan 20 0.70 11.55 2.15 0.70 0.12 0.70 14.48 0.84 17.48 34227.59 (0.47) (11.21) (6.67) (0.47) (0.21) (0.47) (5.09) (0.22) (6.26) (2315.59) Fier 16 0.69 9.25 2.63 0.63 0.12 0.38 15.67 0.86 17.46 34838.63 (0.48) (8.00) (4.32) (0.50) (0.15) (0.50) (3.30) (0.12) (4.06) (1648.88) Gjirokaster 12 0.42 13.25 2.17 0.42 0.06 0.42 15.42 0.90 15.00 36401.30 (0.51) (10.09) (5.73) (0.51) (0.08) (0.51) (8.82) (0.16) (4.67) (994.65) Korce 18 0.72 7.56 1.28 0.78 0.15 0.61 12.47 0.94 17.31 36703.56 (0.46) (8.67) (2.59) (0.43) (0.23) (0.50) (5.82) (0.09) (6.30) (1214.67) Kukes 15 0.87 12.93 6.60 1.00 0.20 0.71 11.42 0.57 13.14 33368.28 (0.35) (12.13) (9.88) (0.00) (0.19) (0.47) (4.29) (0.28) (5.12) (3262.89) Lezhe 16 0.56 6.88 3.06 0.81 0.14 0.88 15.74 0.85 16.15 35729.63 (0.51) (8.00) (7.43) (0.40) (0.14) (0.34) (4.70) (0.15) (4.22) (1927.17) Shkoder 14 0.64 19.50 4.71 0.71 0.13 0.93 15.42 0.74 16.47 34303.64 (0.50) (25.65) (9.67) (0.47) (0.15) (0.27) (6.50) (0.30) (5.61) (3424.92) Tirane 18 0.56 5.33 4.17 0.56 0.15 0.56 19.24 0.86 18.73 36732.24 (0.51) (5.90) (8.34) (0.51) (0.21) (0.51) (4.58) (0.15) (4.65) (2433.33) Vlore 12 0.58 17.33 1.08 0.42 0.09 0.50 12.60 0.92 17.02 37103.78 (0.51) (15.91) (1.44) (0.51) (0.12) (0.52) (4.35) (0.10) (5.68) (1297.66) Total 180 0.64 11.43 3.76 0.66 0.14 0.59 15.17 0.80 16.94 35245.22 (0.48) (12.87) (7.89) (0.47) (0.17) (0.49) (6.73) (0.22) (5.69) (2489.37) Page 52 Table A.2.3. Top-down accountability and transparency by RED Respondents Respondents No risk of Public No risk of School has Public perceiving perceiving penalties for display of N. RED penalties for teacher display of corruption in corruption in underperform- students' inspections underperform- absence teachers' the the ing school final ing teachers register absence appointment appointment of directors grades RED of directors teachers % % respondents Mean % schools % teachers % schools % schools % schools respondents by schools by schools Berat 3.85 0.14 0.66 0.86 0.21 0.93 0.26 0.17 (2.67) (0.36) (0.30) (0.36) (0.43) (0.27) (0.15) (0.10) Diber 3.30 0.00 0.33 1.00 0.00 0.64 0.41 0.26 (2.16) (0.00) (0.23) (0.00) (0.00) (0.50) (0.18) (0.13) Durres 4.64 0.42 0.55 0.83 0.17 0.75 0.28 0.13 (3.61) (0.51) (0.32) (0.39) (0.39) (0.45) (0.13) (0.12) Elbasan 3.53 0.35 0.44 0.95 0.00 0.75 0.36 0.30 (4.25) (0.49) (0.26) (0.22) (0.00) (0.44) (0.15) (0.17) Fier 8.94 0.13 0.51 0.88 0.06 0.63 0.32 0.24 (10.07) (0.34) (0.25) (0.34) (0.25) (0.50) (0.25) (0.20) Gjirokaster 4.00 0.08 0.60 0.92 0.08 0.83 0.20 0.21 (2.45) (0.29) (0.26) (0.29) (0.29) (0.39) (0.13) (0.17) Korce 2.61 0.44 0.57 0.83 0.06 0.78 0.34 0.28 (1.69) (0.51) (0.18) (0.38) (0.24) (0.43) (0.19) (0.16) Kukes 5.21 0.14 0.39 0.86 0.21 0.71 0.23 0.17 (7.28) (0.36) (0.23) (0.36) (0.43) (0.47) (0.13) (0.11) Lezhe 5.13 0.13 0.56 0.94 0.31 0.63 0.20 0.12 (2.55) (0.34) (0.11) (0.25) (0.48) (0.50) (0.11) (0.09) Shkoder 9.14 0.07 0.43 0.93 0.21 0.79 0.15 0.12 (9.27) (0.27) (0.32) (0.27) (0.43) (0.43) (0.11) (0.10) Tirane 4.59 0.33 0.39 0.94 0.06 0.61 0.19 0.12 (3.91) (0.49) (0.25) (0.24) (0.24) (0.50) (0.13) (0.08) Vlore 5.58 0.00 0.48 0.92 0.17 0.92 0.41 0.30 (3.58) (0.00) (0.23) (0.29) (0.39) (0.29) (0.18) (0.11) Total 5.04 0.20 0.49 0.91 0.12 0.74 0.28 0.20 (5.52) (0.40) (0.26) (0.29) (0.33) (0.44) (0.18) (0.15) Note: Standard deviation in parentheses. Page 53 Table A.2.4. Bottom-up accountability and parents’ involvement by RED School Parents with Parents with no Parents who Parents who SB Parent SB teacher Parents who director no information do not know participate in representatives representatives do not know participates information about the the name of parent- are elected by elected by the name of and votes in about the existence of language teacher parents teachers math teacher SB meetings school board parent class reps teacher meetings % schools % schools % schools % parents % parents % parents % parents % parents Berat 0.14 0.92 0.31 0.54 0.28 0.16 0.14 4.58 (0.36) (0.29) (0.48) (0.21) (0.25) (0.14) (0.13) (1.09) Diber 0.14 0.93 0.50 0.37 0.17 0.10 0.12 3.39 (0.36) (0.27) (0.52) (0.27) (0.19) (0.10) (0.14) (1.02) Durres 0.18 0.60 0.60 0.48 0.31 0.28 0.28 4.57 (0.40) (0.52) (0.52) (0.27) (0.25) (0.29) (0.23) (1.72) Elbasan 0.50 0.79 0.71 0.48 0.33 0.17 0.12 3.96 (0.51) (0.42) (0.47) (0.23) (0.24) (0.14) (0.14) (1.03) Fier 0.00 1.00 0.36 0.43 0.26 0.21 0.23 5.11 (0.00) (0.00) (0.50) (0.22) (0.20) (0.17) (0.21) (1.46) Gjirokaster 0.17 0.73 0.25 0.33 0.33 0.15 0.16 4.98 (0.39) (0.47) (0.45) (0.19) (0.25) (0.14) (0.15) (1.30) Korce 0.67 0.88 0.24 0.42 0.27 0.30 0.28 3.12 (0.49) (0.33) (0.44) (0.30) (0.27) (0.24) (0.24) (1.20) Kukes 0.14 0.82 0.44 0.21 0.19 0.25 0.27 3.57 (0.36) (0.40) (0.53) (0.20) (0.22) (0.16) (0.18) (1.34) Lezhe 0.33 0.73 0.50 0.63 0.32 0.33 0.31 4.68 (0.49) (0.46) (0.52) (0.26) (0.25) (0.18) (0.21) (1.98) Shkoder 0.71 0.93 0.57 0.49 0.40 0.28 0.26 3.36 (0.47) (0.27) (0.51) (0.22) (0.28) (0.22) (0.20) (0.59) Tirane 0.17 0.53 0.53 0.38 0.19 0.39 0.39 4.44 (0.38) (0.51) (0.51) (0.21) (0.17) (0.24) (0.22) (0.97) Vlore 0.17 0.83 0.17 0.36 0.26 0.27 0.23 5.32 (0.39) (0.39) (0.39) (0.26) (0.26) (0.25) (0.20) (1.89) Total 0.29 0.81 0.43 0.43 0.28 0.25 0.23 4.22 (0.46) (0.39) (0.50) (0.25) (0.24) (0.21) (0.20) (1.48) Page 54 APPENDIX 3 Design and Application of the Field Lab-Type Experiments (LTEs) 1. In tandem with the Albania School Stakeholder Survey, controlled lab-type experiments (LTEs), or “games” of the type used by experimental economists to explore behavioral traits, were conducted with teachers and parents in all all of the sampled schools. The games were designed to provide a measure of parents‟ willingness to engage in activities aimed at holding school staff accountable. These games measure behavioral traits that cannot be reliably captured through the application of traditional survey instruments. 2. Games allow researchers another way to get closer look at peoples‟ preferences, as observed by their behavior. For example, if a sample of individuals is asked whether they are trustworthy, few if any will say that they are not. However, if they are engated in a game in which half are invited to trust members of the remaining half and the latter are then invited to reciprocate if they so wish; and if the game is being played for real money, a significant proportion of the individuals will not reciprocate, i.e., they will reveal themselves to be untrustworthy through the way that they play the game. Similarly, if an individual is asked if they would be willing to pay a bribe in exchange for an enhanced service even if they knew that, in so doing, they would cause harm to others, few would say that they would. However, if the person was engages in a game involving real money that simulates this situation, many acts of bribery would be observed. 3. Three specific LTEs were designed to accompany the Albania School Stakeholder Survey.  A Teacher-Parent Dictator Game: In a standard dictator game (Forsythe et al., 1994) experimental participants play in pairs, in the role of either “sender” (i.e. the “dictator”) or “recipient”. Each pair is allocated a given “pot” of money, and the sender has to decide how to divide the pot between herself and the recipient. The sender can allocate to the recipient any amount between zero and the full pot. The two players go home with whatever share of the pot that has been allocated to them by the “sender” in the game. In each surveyed school, teachers play in the role of “sender” and parents play in the role of “recipient”. Play is anonymous and one-shot. The game is designed to measure teachers‟ “other-regarding” or “altruistic” preferences toward parents.  A Teacher-Parent Third Party Punishment Game (TPG): The “third party punishment” game involves three players. As in the dictator game, a “sender” has to decide how to share a pot of money between herself and a “recipient”. The difference with a standard dictator game is that the TPG involves a third player who, if she wishes, can fine the “sender” at her own cost. In each surveyed school we play the TPG with teachers in the role of “sender” and parents in the role of either “recipient” or “third party”. The game is designed to see on the one hand whether teachers are more „other regarding‟ towards parents when they are being held accountable by other parents, and, on the other, whether parents are willing to forgo personal resources in order to hold teachers accountable. In this way, the game provides a measure of parents‟ “demand for accountability”, as well as how teachers react to when this demand is exercised.  A School Voluntary Contribution Mechanism (VCM) Game: In a standard VCM game individuals are given an endowment of tokens and they must decide how many tokens they wish to contribute toward the provision of the public good, and how many they wish to keep for “private use”. The return to the investment in the public good is a share of the total Page 55 amount contributed by all the participants, multiplied by some positive factor. In each school we conduct a VCM game involving all the surveyed teachers and parents. We implement the game in dichotomous format, i.e. each participant is given one token and the choice to either “invest the token in a group account” or use the token for private gain. Whatever is invested in the group account is tripled and redistributed to all participants in the game. The monetary value of the token is calibrated so that it is worth investing in the school only if a large enough number of participants decide to do so. This game simulates the dilemma over contributing private resources to ensure the well functioning of the school versus keeping private resources for individual use, where contribution (or investment) in the school is a public good. 4. The LTE/game employed most extensively in the measurement of parents‟ demand for accountability in Albania, was the third party punishment game (TPG), but with important alterations.46 As in any TPG, one player is invited to divide a sum of money between him or herself and another player, and a third player can pay to have money subtracted from the first player‟s allocation. It is the third players‟ behavior that is of particular interest here as by choosing to fine they reveal both what they consider to be fair behavior on the part of the first player and their willingness to pay to express this. In other words, they are expressing their willingness to hold others to account with respect to dividing a resource fairly between themselves and passive others. The most important difference between Fehr and Fischbacher‟s original experiment and the one implemented in Albanian primary schools is that the former involved student subjects, whereas here the game roles were assigned to non-student subjects with reference to and with the aim of matching the roles they (potentially) play in a real, ongoing accountability scenario. Specifically, we invited teachers to play in the role of “givers”, i.e. the individuals who have to decide how to divide an endowment between themselves and another subject. We invited parents to play in the roles of the “recipients” and “third party punishers”. 5. In each school, one experimental session was held involving fifteen subjects: the 5 teachers and the 10 parents selected for the survey. Each experimental session was conducted using a classroom large enough to seat all fifteen subjects and the two field researchers running the session. Two additional rooms were used as decision-making stations, to ensure complete privacy for one- on-one interviews during which actual teachers‟ and parents‟ decisions were recorded. 6. Each session proceeded as follows. On arrival, each subject was registered and asked to blindly pick a badge bearing a number and either a purple, an orange, or an orange with a blue t- shirt figure. Teachers were asked to pick a badge out of 5 bearing purple figures, parents were asked to pick a badge out of 10 bearing orange or orange with blue t-shirts figures. Parents who picked the orange figures were then automatically assigned the role of “recipients” in the game, and parents who picked orange with blue-t-shirt figures were assigned the role of “third-party”. Following registration, each subject was invited to sit at the desk bearing their player number. Once all the subjects had arrived and were seated, one of the field researchers, before describing the experimental games, invited all those with purple badges to raise their hands and then asked “Am I right in thinking that you are all teachers?” Having received confirmation, she or he said “During the workshop I will refer to you as purple players.” The presenter then went through the same procedure for orange and orange with blue-t-shirt badge wearers, i.e., parents. The aim of this exercise was to ensure that badge colors and the roles that the badge wearers assumed in their everyday lives and in the experiment were linked in the minds of the participants. This was the only atypical framing applied to the experiment. 46 The LTE discussed is a version of Fehr and Fischbacher‟s (2004) Third Party Punishment Game (TPG). Page 56 Figure A.3.1. Example of the visual aids used to explain the Third Party Punishment Game Source: Serra, Barr and Packard (2011) 7. Then the experiment was introduced.47 Wall mounted visual aids depicting purple, orange, and orange with blue t-shirt figures, and moving representations of real Albanian LEK (oversized plastic-coated pictures of bills with magnets on the back) were used to demonstrate the effects of various allocation and fining decisions on final take-home payoffs (see figure A.3.1). Once the subjects had been taught the game, the teachers were called to one-on-one meetings with a field researcher, were taken through the game again, tested, and then invited to express their chosen allocation by dividing the real-money stake between the purple figure, representing themselves, and the orange figure, representing the parent they had been anonymously paired with, on the table in front of them. 8. Each purple player (teacher) was invited to divide 2,000 Albanian LEK (presented as ten 200 notes) between him or herself and one of the five passive orange players (parents) in the session.48 Each blue-t-shirt player (parents) was endowed with 1000 Albanian LEK and could pay 200 LEK to have 600 LEK taken away from the purple player (teacher) with whom they were anonymously matched. The orange with blue t-shirt players‟ decisions were elicited using the strategy method. Each was asked, in one-on-one meetings, whether they would “do nothing and leave things as they are” or “pay to have the purple player fined”, in the event that the purple player allocated 0 to the orange player and 2000 to himself, 200 to the orange player and 1800 to himself, 400 to the orange player and 1600 to himself, and so on all the way to 1400 to the orange player and 600 to himself (eight choices in total). We did not ask what they would do if the purple player 47 First, the teachers, randomly and anonymously paired with parents with orange figure badges, played a standard Dictator Game (DG). The parents with blue-t-shirt badges were not directly involved in the DG. However, each was endowed with half of the stake that each teacher was given to allocate between him or herself and a parent. We did this to ensure that the parents with blue-t-shirts badges did not enter the third party punishment game feeling under-remunerated. After playing the third-party punishment game, parents and teachers played a voluntary contribution game (VCM). Here, we focus only on the TPG. 48 At the time of the experiment, 2000 LEK was worth just above $21 and corresponded to the pay for one and a half days of work for the average teacher in our sample. It corresponded to the pay of almost two days of work for the average parent in the sample. Page 57 allocated less than 600 to him or herself as this would have either potentially led to purple players having net earnings of less than zero or required some conditioning of the math of the game and corresponding further explanation. Mindful of the need to keep the games as simple as possible both for the sake of the subjects and the field researchers who conducted the experiments, we decided to obviate these complications by not allowing the orange with blue t-shirt, third party players to fine purple players who had allocated less than the value of the fine to themselves.49 9. The application of the strategy elicitation method to the third party punishers maximized comparability across parents as it ensured that each parent placed in the third party role responded to the same set of possible stimuli. Had their responses been directly elicited, the actual stimuli applied to each third party punisher would have varied in accordance with their purple player‟s (teacher‟s) decision. The fining strategy elicitation enabled us to identify both parents who would never fine a teacher, not even if the teacher kept the entire endowment, and the threshold allocation by a teacher to a passive parent above which the fining parent would not fine.50 10. After all purple players and all orange with blue t-shirt players had made their decisions in private, we randomly matched one purple, one orange and one orange with blue-t-shirt player, maintaining anonymity, and computed payoffs in accordance with the decisions made by purple players and blue t-shirt players in the one-on-one meetings. 49 This was not explained in the initial presentation of the game, but if players asked about it during one-on-one interviews, they were provided with the explanation set out here. 50 Whether and to what extent the strategy elicitation affects observed behavior is the subject of an ongoing debate. The empirical evidence is mixed. While Güth, Huck, and Mueller (2001), Shotter et al. (1994) and Brosig, Weimann and Yang (2003) find that strategy elicitation induces significantly different behavior to direct elicitation, using different experimental designs, Cason and Mui (1998), Brandts and Charness (2000) and Oxoby and McLeish (2004) find no differences. The complexity of the experiment may be a crucial factor: the difference increases with the complexity of the game (Brandts and Charness, 2000). Our game is simple so any effect is likely to be small. For a recent survey of experimental comparisons of strategy versus direct-response method, see Brandts and Charness (2009).