WPS5413 Policy Research Working Paper 5413 Performance of Fe y Alegría High School Students in Colombia Is it a Matter of Fe (Faith) or Alegría (Joy)? Juan Carlos Parra Osorio Quentin Wodon1 The World Bank Human Development Network Development Dialogue on Values and Ethics September 2010 Policy Research Working Paper 5413 Abstract Fe y Alegría is a catholic network of schools that started perform worse than other schools for all years in the operations in Colombia in 1971, and in 2009 served sample. However, Fe y Alegría schools also cater to poorer more than 72,000 students in 61 schools. This paper students who come from disadvantaged backgrounds. assesses the performance of Fe y Alegría secondary Once controls are included for student background, Fe schools in Colombia using test scores for Spanish and y Alegría schools actually often perform as well and in mathematics, as well as detailed information on the some cases better than other schools for mathematics and characteristics of the household to which students Spanish, thus partially reversing the previous finding. belong. Simple statistics suggest that Fe y Alegría schools This paper--a product of the Development Dialogue on Values and Ethics, Human Development Network--is part of a larger work program on the role of faith-based organizations in service delivery. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at jparraosorio@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team Performance of Fe y Alegría High School Students in Colombia: Is it a Matter of Fe (Faith) or Alegría (Joy)? Juan Carlos Parra Osorio and Quentin Wodon1 Keywords: Education, faith-based schools, public-private partnerships, Fe y Alegría, Colombia JEL categories: I2, L31, H43 Corresponding author: Juan Carlos Parra Osorio, World Bank; Email: jparraosorio@worldbank.org 1 The authors are with the Development Dialogue on Values and Ethics in the Human Development Network at the World Bank. The opinions presented in this paper are those of the authors only, and do not necessarily reflect the views of the World Bank, its Executive Directors, or the countries they represent. The paper was presented at a multi-country training event organized for Fe y Alegría staff in October 2009 in Peru in collaboration with the World Bank Institute and Magis Americas, and at a World Bank education seminar in February 2010 in Washington, DC. Comments from participants to both events and especially Felipe Barrera-Osorio are gratefully acknowledged. 1. Introduction An emerging body of evidence suggests that private schools, including faith-based schools, may provide better education services than public schools (e.g., Allcott and Ortega 2009; Altonji et al. 2005; Asadullah et al. 2009; Backiny-Yetna and Wodon, 2009; Barrera- Osorio, 2009; Chakrabarti and Peterson, 2009; Cox and Jimenez 1990; Evans and Schwab 1995; González and Arévalo 2005; Hoxby 1994; Hsieh and Urquiola 2006; Wodon and Ying, 2009). In the economic literature, several reasons have been advanced to explain the gains in performance associated with private schools (Epple and Romano 1998; LaRocque and Patrinos 2006; Nechyba 2000; Savas 2000). First, private schools may introduce competition in the education sector and thereby raise overall quality. Second, private providers may have more flexibility than public providers in the management of the schools. Third, to the extent that private providers of education are competitively selected, better providers would emerge in the private as opposed to the public sphere. Fourth, risk-sharing between the government and the private sector may also lead to better provision. Beyond these economic arguments for a role to be played by faith-based providers, there may also be a number of other potential advantages in having faith-based organizations providing education services. As noted by Belshaw (2005), FBOs have a long-term commitment to their communities and they often reach out to the poorest members of the community. Through links to sister organizations in other countries, they may benefit from outside funding and expertise. Faith-based schools often emphasize values of respect and consideration for others. In addition, religious leaders often have a moral authority that helps in mobilizing the community's resources around the schools. Finally, faith-based providers may be more dedicated or altruistic than other providers. As argued by Reinikka and Svensson (2010) in the case of health service provision in Uganda, faith-based providers may not be motivated only or mostly by profit or perks maximization--they seem to be "working for God." At the same time however faith-based schools may also suffer from weaknesses, especially if they place the pursuit of their religious mandate ahead of the needs of students in regard to what they need to learn to be successful in today's world. In this paper, we assess the performance of Fe y Alegría secondary schools in Colombia using a rich multi-year data set with test scores on a wide range of subjects as well as detailed information on the characteristics of the household to which students belong. Fe y Alegría is a catholic education network founded in 1955, serving almost one million students in 17 countries2. It targets excluded population and its work program, according to the organization's mission, is based on the Christian values of justice, participation, and solidarity. The network started operating in Colombia in 1971, and in 2006 it served more than 72,000 students in 61 schools3. Even though academic excellence is not explicitly mentioned in its mission or vision, it is one of the variables Fe y Alegría principals and leaders use for their internal evaluations. Most observers consider Fe y Alegría to be successful, but few rigorous evaluations have been undertaken until now. This paper was inspired by previous research on the performance of Fe y Alegría in Venezuela conducted by Allcott and Ortega (2009) who use propensity score matching methods to estimate the effects on standardized test scores of graduating from Fe y Alegría in comparison with public schools. They find an average treatment effect on the order of 0.1 standard deviation in mathematics, which is small, but nevertheless statistically significant. They argue that the better performance of Fe y Alegría stems not only from its labor contract flexibility and decentralized administrative structure, but also from the peculiar "family culture" of the schools. 2 In chronological order of start of operations: Venezuela, Ecuador, Panama, Peru, Bolivia, El Salvador, Colombia, Nicaragua, Guatemala, Brazil, Dominican Republic, Paraguay, Argentina, Honduras, Chile, Haiti, Spain, and Chad. There are no Fe y Alegría schools in Spain. 3 Only formal education is included. Building on Allcott and Ortega's (2009) work, the data set and methodology that we use in this paper enables us to test in a more robust way for the performance of Fe y Alegría schools than previous work for three main reasons. First, while Allcott and Ortega (2009) use data on test scores for one year only, we have five years of good data at our disposal, which enables us to test whether differences in performance are robust over time. Second, we have a larger number of subjects on which students are tested. Beyond test scores for mathematics and Spanish, we also look at data on test scores for scientific topics. Third, we have an especially good set of controls regarding the socio-economic characteristics of the families to which the children belong. Finally, we use alternative matching methods (propensity score matching as well as coarse exact matching) to ensure that the results obtained are robust to the choice of method. Our results confirm previous work suggesting that faith-based schools perform relatively well. Simple statistics on test scores suggest that Fe y Alegría schools perform worse than other schools for all years in the sample. However, Fe y Alegría schools also cater to poorer students. Once controls are included for student background, Fe y Alegría schools actually perform at least as well as, and in several cases better than other schools. Section 2 of the paper describes our data and methodology. Section 3 presents our empirical results. A conclusion follows. 2. Data and methodology The ICFES (Colombian Institute for the Promotion of Higher Education) test is a standardized test administered by the Ministry of Education that every student has to take during the last year of high school4. The test score is used mainly by colleges for admission purposes, and had no implications for high school graduation for the years used in this study5. The database provided by ICFES has test scores for every single student taking the test from 1998 to 2006. 4 The test can be retaken after graduation from high school. 5 It became a requirement for high school graduation in 2009. Scores for five subjects are analyzed: Spanish, mathematics, biology, chemistry, and physics. Socioeconomic variables at the student level are included in the database from 1998 to 20036. These variables include birth date, gender, level of education of parents, their occupation, number of people in the family, number of siblings, household income, dwelling ownership, and whether the students works. Information on tuition paid is available from 2000 to 2006. Because our analysis relies on controls and matching techniques to assess the performance of Fe y Alegría schools, we use the data from 1998 to 2003. We restrict the sample using the following criteria: (1) Schools in cities where Fe y Alegría has at least one school in the current year; (2) Students who graduate from high school in the current year or the year before; (3) Students between 15 and 20 years of age; and (4) Students who attended both morning and afternoon sessions of the test. The data for 2001 seems to be problematic because the number of observations is much smaller than in other years. This then gives us five years of data to work with, for 1998-2000 and 2002-2003. Table 1 contains the total number of observations for Fe y Alegría and other schools for each year in the sample, after imposing the four constraints above. We would have liked to explore different control groups, but there is not enough information in the database to identify, for example, public or private schools, or other school level variables like number of students, number of teachers, and qualification of teachers, among others. That is the reason why our control group includes all non Fe y Alegría schools. The percentage of students in the sample attending Fe y Alegría schools increases from 1.1 in 1998 to 1.5 in 2003; the share of Fe y Alegría schools grows from 1.8 percent in 1998 to 2.5 percent in 2003. We rely on both traditional propensity score matching and CEM (coarse exact matching) techniques for the estimations (Iacus et al., 2009; King and Stuart, 2007). CEM is a recent 6 Socioeconomic information was not collected from 2004 to 2006. method to form a better control group. It works by defining strata in which all observations have the same values for the observables X. Consider a sample of units drawn from a population of N. Let denote as an indicator variable for unit i that takes on value 1 if unit i is a member of the treated group (Fe y Alegría in our case) and 0 if i is a member of the control group (other schools). The observed outcome (test scores in our case) variable is = (1) + (1 - ) (0), where (0) is the potential outcome for observation i if the unit does not receive treatment and (1) is the potential outcome if the (same) unit receives treatment. For each observed unit, (0) is unobserved if i receives treatment and (1) is unobserved if i does not receive treatment; this is known as the "missing observation problem" in the evaluation literature. To compensate for the fact that treated and control groups are not necessarily identical before treatment, matching estimators aim to control for pre-treatment covariates. Denote = ( , , , ) as a k-dimensional data set, where each is a column vector of observed values of pre-treatment variable j for the n sample observations. That is, = , = 1, ... , , = 1, ... , ]. Let = {: = 1} be the set of indexes for the treated units and = # be a count of the elements of this set; similarly = {: = 0}, = # for the control units, with + = . Denote by and the number of treated and control units matched by CEM. Let and be the sets of indexes of the matched units in the two groups. After coarsening, the CEM algorithm creates a set of strata (cells), say , each with the same values of X. Observations in strata that contain at least one treated and one control unit are retained; all other observations are dropped from the sample. We denote by the treated units in stratum s, and by = # the number of treated units in the stratum, similarly for the control units, i.e. and = # . The number of matched units is, respectively for treated and controls, = and = . To each matched unit i in stratum s, CEM assigns the following weights: 1, = , Unmatched units receive weight = 0. After the final sample is constructed using CEM, we just compare, within each stratum, the test scores for treated observations with the average test score for the control observations. 3. Results Tables 2 and 3 provide basic statistics on the characteristics of students in both Fe y Alegría and other schools. On average, students in Fe y Alegría schools live in households with a lower income per capita, whose parents have lower education levels, have higher propensity to own their dwelling that is inhabited by slightly bigger households. All variables used in the estimations have been transformed into categorical variables. Several studies have found negative correlations between students' performance and lower per capita income, parents' education level, and higher household size. This is likely then to lead to lower average test scores for students in Fe y Alegría schools, and it is indeed what is observed for every subject and every year. Table 4 presents the average scores in Fe y Alegría schools and other schools. The results indicate that students in other schools do better than students in Fe y Alegría schools in every subject and for every year in the sample. The gap in Spanish and mathematics is at 1.0 to 4.5 percentage points, and the gaps for biology, chemistry, and physics are between 2.0 and 7.0 percentage points (see Figure 1). In order to control for student characteristics, we rely on matching techniques7. If test scores are independent of the decision of whether to attend a Fe y Alegría school given the observables, comparing scores between Fe y Alegría students and other schools using matching techniques will give an unbiased estimate of the difference in test scores attributable to the type of school attended. However, including all observations in the sample might make the results highly model-dependent if the control and the treatment groups are not similar enough. The ideal situation is to have both groups with identical distributions for all observable (and unobservable) characteristics. In order to improve the balance of our sample, that is to have a control group that is more similar to the treated group, we select the final sample used for our estimations using coarse exact matching or CEM (see Section 2; Iacus et al., 2009; King and Stuart, 2007). In order to run the CEM model, we include as control variables the parents' education level, the parents' occupation, household size, per capita income, whether the student works, dwelling ownership, student gender, student age, and location (city) dummies8. The CEM algorithm defines strata, each with the same value of the observables. Only strata with at least one observation from the treated group and one from the control group are used. All other observations are discarded9. As a test of robustness, we also use propensity score matching, with a probit regression model whose estimates are provided in annex. Key results are presented in table 5 for all subjects, as well as for mathematics and Spanish in Figures 2 and 3. Figure 2 provides the percentage difference in test scores based on CEM. The hollow bars represent differences that are not statistically different from zero. In Figure 3, the same is done with the results from propensity score matching. We tend to favor the 7 One way to control for observable variables like income, parents' education, household size, gender, among others, would be to include them in a simple regression model, such as OLS, with the regression including a dummy for Fe y Alegría as one of the explanatory variables. This however would assume that treatment effects are similar for the sample as a whole, which may not be appropriate. Using matching techniques relaxes that assumption. 8 We use level dummies for all variables to capture potential nonlinearities in the model. 9 On average, 40 percent of the observations in the control group (other schools) are dropped after CEM. results based on CEM, since the matching is more precise in that case, but the results are fairly similar using both methods. Consider first mathematics and Spanish, which are more commonly used for measuring performance (both in the literature and in Colombia) than scientific subjects. It appears that except in 2003, Fe y Alegría schools do as well or better than other schools. However, for physics, chemistry and biology, the performance of Fe y Alegría schools tends to be lower (the only exception is for physics in 1998 using the CEM method). Looking at the Figures and estimates, one may wonder whether there is a trend in the results for mathematics and Spanish, with better performance for Fe y Alegría schools in earlier years in comparison to other schools, no statistical difference in performance in the middle years, and lower performance in the last year of data. Because the sample of schools differs from year to year, with a larger number of Fe y Alegría schools over time, this could potentially be explained by the fact that perhaps new Fe y Alegría schools could perform less well than older schools, thereby pulling overall results for the network down over time. The data however does not confirm this hypothesis. We redid the estimations by restricting the school sample to remain the same in 2003 as per the 2000 sample (given that the number of schools in 1998 and 1999 is smaller), and this did not affect the results for 2003 in that they remained essentially unchanged, as well as statistically significant in terms of the difference between Fe y Alegría and other schools for that year. Thus we cannot explain the apparently lower performance for Fe y Alegría in 2003 by the change in the sample of schools covered over time, but given that we have only one year of data where Fe y Alegría schools perform at a lower level than other schools, we think that it would be premature to assert that there is indeed solid evidence for a trend towards a lower level of performance among Fe y Alegría schools over time. 4. Conclusion This paper quantified the differences in high school test scores between students in Fe y Alegría schools and other schools in Colombia between 1998 and 2003. We find that since Fe y Alegría caters, on average, to poorer students and students with lower-educated parents, simply comparing average test scores might lead to wrong conclusions regarding the performance of the schools. We used a recent method (coarse exact matching) to form a better control group (balancing) and found that for mathematics and Spanish, most negative gaps for Fe y Alegría students that were found across the years and subjects when comparing averages, either vanish or become gains after balancing in 4 out of 5 years. For robustness purposes, we also estimated performance using propensity score matching, and the results were similar. On the other hand, Fe y Alegría students tend to do less well in physics, chemistry and biology, although the difference after matching with other schools is significantly smaller than the difference before matching. References Allcott, H., and D. E. Ortega. (2009). "The Performance of Decentralized School Systems: Evidence from Fe y Alegría in Venezuela." In Emerging Evidence on Private Participation in Education: Vouchers and Faith-Based Providers, ed. F. Barrera-Osorio, H. A. Patrinos, and Q. Wodon, Washington, DC: The World Bank. Altonji, J. G., T. E. Elder, and C. R. Taber. (2005). "An Evaluation of Instrumental Variable Strategies for Estimating the Effects of Catholic Schooling", Journal of Human Resources, Vol. 40(4): 791­821. Asadullah, M. N., N. Chaudhury, and A. Dar. (2009). "Student Achievement in Religious and Secular Secondary Schools in Bangladesh." In Emerging Evidence on Private Participation in Education: Vouchers and Faith-Based Providers, ed. F. Barrera-Osorio, H. A. Patrinos, and Q. Wodon, Washington, DC: The World Bank. Backiny-Yetna, P., and Q. Wodon. (2009). "Comparing Faith-Based and Government Schools in the Democratic Republic of Congo." In Emerging Evidence on Private Participation in Education: Vouchers and Faith-Based Providers, ed. F. Barrera-Osorio, H. A. Patrinos, and Q. Wodon. Washington, DC: The World Bank. Barrera-Osorio, F., (2009). "The Concession Schools of Bogotá, Colombia." In School Choice International: Exploring Public-Private Partnerships, ed. R. Chakrabarti and P. E. Peterson, Cambridge: MIT Press. Belshaw, D. (2005). "Enhancing the Development Capability of Civil Society Organisations, with Particular Reference to Christian Faith-based Organisations (FBOs)." In Reclaiming Development: Assessing the Contributions of Non-Governmental Organisations to Development Alternatives, Economic and Social Research Council Global Poverty Research Group, London. Chakrabarti, R., and P. E. Peterson. (Eds.). (2009). School Choice International: Exploring Public-Private Partnerships, Cambridge: MIT Press. Cox, D., and E. Jimenez. (1990). "The Relative Effectiveness of Private and Public Schools: Evidence from two Developing Countries", Journal of Development Economics, Vol. 34(1­2): 99­121. Epple, D., and R. E. Romano. (1998). "Competition between Private and Public Schools, Vouchers, and Peer-Group Effects", American Economic Review, Vol. 88(1): 33-62. Evans, W., and R. Schwab. (1995). "Finishing High School and Starting College: Do Catholic Schools Make a Difference?", Quarterly Journal of Economics Vol. 110(4): 941­74. González, R. A., and G. Arévalo. (2005). "Subsidized Catholic Schools in Venezuela." In Private Education and Public Policy in Latin America, ed. L. Wolff, J. C. Navarro, and P. González. Washington, DC: Project For Educational Revitalization in the Americas. Hoxby, C. M. (1994). "Do Private Schools Provide Competition for Public Schools?" National Bureau of Economic Research Working Paper 4978. NBER, Cambridge, MA. Hsieh, C., and M. Urquiola. (2006). "The Effects of Generalized School Choice on Achievement and Stratification: Evidence from Chile's School Voucher Program", Journal of Public Economics, Vol. 90: 1477­1503. Iacus, S., King, G., and G. Porro. (2009). "Causal Inference Without Balance Checking: Coarsened Exact Matching." Manuscript, Harvard University. King, G., and E. Stuart. (2007). "Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference", Political Analysis, Vol. 15: 199-236. LaRocque, N., and H. Patrinos. (2006). "Choice and Contracting Mechanisms in the Education Sector." (processed) World Bank, Washington, DC. Nechyba, T.J. (2000). "Mobility, Targeting and Private School Vouchers", American Economic Review, Vol. 90(1): 130­46. Reinikka, R., and J. Svensson. (2010). "Working for God? Evidence From a Change in Financing of Nonprofit Health Care Providers in Uganda", Journal of the European Economic Association, Vol. 8(6). Savas, E. S. (2000). Privatization and Public-Private Partnerships, New York: Chatham House Publishers. Wodon, Q., and Y. Ying. (2009). "Literacy and Numeracy in Faith-Based and Government Schools in Sierra Leone." In Emerging Evidence on Private Participation in Education: Vouchers and Faith-Based Providers, ed. F. Barrera-Osorio, H. A. Patrinos, and Q. Wodon. Washington, DC: The World Bank. Figure 1: Percentage Difference Between Average Test Scores (Fe y Alegría minus Other schools) ­ Before Matching 0.0 -0.5 -1.0 -1.5 Percentage difference (FyA -Others) -2.0 -2.5 Spanish Math -3.0 -3.5 -4.0 -4.5 -5.0 1998 1999 2000 2002 2003 Source: Authors' estimation using ICFES data. Figure 2: Percentage Difference Between Average Test Scores (Fe y Alegría minus Other schools) - Propensity score matching 2.0 1.5 1.0 Percentage difference (FyA - Other) 0.5 Spanish Math 0.0 -0.5 -1.0 1998 1999 2000 2002 2003 Source: Authors' estimation using ICFES data. Figure 3: Percentage Difference Between Average Test Scores (Fe y Alegría minus Other schools) - CEM matching 1.5 1.0 0.5 Percentage difference (FyA - Other) 0.0 Spanish Math -0.5 -1.0 -1.5 1998 1999 2000 2002 2003 Source: Authors' estimation using ICFES data. Table 1: Total number of students and schools in the sample 1998 1999 2000 2001 2002 2003 Number of students Fe y Alegría 1,052 1,317 1,581 849 1,735 1,967 Other 98,368 109,576 113,374 77,054 121,589 127,572 Number of schools Fe y Alegría 23 24 30 29 37 41 Other 1,283 1,244 1,568 1,629 1,703 1,631 Source: Authors' estimation using ICFES data. Table 2: Mean value of control variables: Fe y Alegría schools 1998 1999 2000 2002 2003 Education of the father and mother Father completed primary or less 0.05 0.07 0.07 0.07 0.07 Father completed secondary 0.85 0.75 0.74 0.75 0.74 Father completed college 0.06 0.08 0.09 0.08 0.09 Father post-graduate education 0.04 0.10 0.09 0.10 0.09 Mother completed primary or less 0.04 0.04 0.07 0.04 0.07 Mother completed secondary 0.87 0.79 0.77 0.79 0.77 Mother completed college 0.06 0.09 0.10 0.09 0.10 Mother post-graduate education 0.03 0.08 0.07 0.08 0.07 Occupation of the father and mother Father manager/owner 0.06 0.09 0.07 0.09 0.07 Father employee 0.52 0.52 0.52 0.52 0.52 Father construction worker/other 0.39 0.34 0.34 0.34 0.34 Father homemaker, retiree, student 0.04 0.05 0.07 0.05 0.07 Mother manager/owner 0.01 0.03 0.02 0.03 0.02 Mother employee 0.18 0.26 0.25 0.26 0.25 Mother construction worker/other 0.11 0.10 0.11 0.10 0.11 Mother homemaker, retiree, student 0.70 0.61 0.61 0.61 0.61 Other household characteristics Household size: 1 to 3 0.08 0.10 0.10 0.10 0.10 Household size: 4 or 5 0.53 0.53 0.54 0.53 0.54 Household size: 6 or 7 0.32 0.28 0.28 0.28 0.28 Household size: More than 7 0.07 0.09 0.08 0.09 0.08 Per capita income: Less than 0.5 MW 0.85 0.82 0.84 0.82 0.84 Per capita income: 0.5 to 1 MW 0.12 0.10 0.09 0.10 0.09 Per capita income: 1 to 1.5 MW 0.02 0.04 0.04 0.04 0.04 Per capita income: 1.5 to 2 MW 0.00 0.02 0.02 0.02 0.02 Per capita income: More than 2 MW 0.00 0.02 0.01 0.02 0.01 Characteristics of the student Student works 0.05 0.04 0.05 0.04 0.05 Family owns dwelling 0.83 0.77 0.76 0.77 0.76 Female student 0.43 0.49 0.50 0.49 0.50 Student aged 15 0.01 0.01 0.02 0.01 0.02 Student aged 16 0.16 0.17 0.20 0.17 0.20 Student aged 17 0.37 0.42 0.46 0.42 0.46 Student aged 18 0.27 0.25 0.21 0.25 0.21 Student aged 19 0.13 0.11 0.09 0.11 0.09 Student aged 20 0.06 0.04 0.02 0.04 0.02 Geographic Location Armero 0.00 0.00 0.00 0.00 0.00 Barrancabermeja 0.00 0.00 0.00 0.00 0.00 Barranquilla 0.00 0.02 0.04 0.02 0.04 Bello 0.16 0.13 0.11 0.13 0.11 Bogotá 0.27 0.27 0.28 0.27 0.28 Cali 0.18 0.08 0.08 0.08 0.08 Cartagena 0.04 0.06 0.07 0.06 0.07 Ciénaga 0.00 0.00 0.00 0.00 0.00 Cúcuta 0.00 0.04 0.03 0.04 0.03 Dos Quebradas 0.03 0.01 0.01 0.01 0.01 Ibagué 0.02 0.02 0.02 0.02 0.02 Lérida 0.02 0.01 0.02 0.01 0.02 Los Patios 0.05 0.05 0.03 0.05 0.03 Manizales 0.00 0.03 0.05 0.03 0.05 Tierralta 0.06 0.03 0.04 0.03 0.04 Medellín 0.19 0.25 0.21 0.25 0.21 Source: Authors' estimation using ICFES data. Table 3: Mean value of control variables: Other schools 1998 1999 2000 2002 2003 Education of the father and mother Father completed primary or less 0.02 0.03 0.03 0.03 0.03 Father completed secondary 0.68 0.64 0.62 0.64 0.62 Father completed college 0.15 0.15 0.16 0.15 0.16 Father post-graduate education 0.15 0.18 0.20 0.18 0.20 Mother completed primary or less 0.02 0.03 0.02 0.03 0.02 Mother completed secondary 0.74 0.69 0.66 0.69 0.66 Mother completed college 0.14 0.15 0.17 0.15 0.17 Mother post-graduate education 0.09 0.13 0.15 0.13 0.15 Occupation of the father and mother Father manager/owner 0.12 0.13 0.13 0.13 0.13 Father employee 0.62 0.58 0.59 0.58 0.59 Father construction worker/other 0.24 0.25 0.23 0.25 0.23 Father homemaker, retiree, student 0.03 0.05 0.05 0.05 0.05 Mother manager/owner 0.04 0.05 0.06 0.05 0.06 Mother employee 0.32 0.32 0.35 0.32 0.35 Mother construction worker/other 0.07 0.07 0.07 0.07 0.07 Mother homemaker, retiree, student 0.57 0.55 0.52 0.55 0.52 Other household characteristics Household size: 1 to 3 0.14 0.13 0.14 0.13 0.14 Household size: 4 or 5 0.57 0.55 0.58 0.55 0.58 Household size: 6 or 7 0.24 0.26 0.22 0.26 0.22 Household size: More than 7 0.05 0.06 0.05 0.06 0.05 Per capita income: Less than 0.5 MW 0.63 0.66 0.65 0.66 0.65 Per capita income: 0.5 to 1 MW 0.30 0.22 0.22 0.22 0.22 Per capita income: 1 to 1.5 MW 0.06 0.08 0.08 0.08 0.08 Per capita income: 1.5 to 2 MW 0.00 0.03 0.03 0.03 0.03 Per capita income: More than 2 MW 0.00 0.02 0.02 0.02 0.02 Characteristics of the student Student works 0.06 0.06 0.05 0.06 0.05 Family owns dwelling 0.74 0.72 0.71 0.72 0.71 Female student 0.45 0.46 0.46 0.46 0.46 Student aged 15 0.01 0.02 0.02 0.02 0.02 Student aged 16 0.17 0.21 0.22 0.21 0.22 Student aged 17 0.35 0.37 0.39 0.37 0.39 Student aged 18 0.26 0.23 0.22 0.23 0.22 Student aged 19 0.14 0.12 0.10 0.12 0.10 Student aged 20 0.06 0.05 0.04 0.05 0.04 Geographic Location Armero 0.00 0.00 0.00 0.00 0.00 Barrancabermeja 0.00 0.00 0.00 0.00 0.00 Barranquilla 0.00 0.11 0.10 0.11 0.10 Bello 0.03 0.02 0.02 0.02 0.02 Bogotá 0.52 0.44 0.46 0.44 0.46 Cali 0.17 0.10 0.12 0.10 0.12 Cartagena 0.07 0.07 0.06 0.07 0.06 Ciénaga 0.00 0.00 0.00 0.00 0.00 Cúcuta 0.00 0.04 0.04 0.04 0.04 Dos Quebradas 0.01 0.01 0.01 0.01 0.01 Ibagué 0.05 0.04 0.04 0.04 0.04 Lérida 0.00 0.00 0.00 0.00 0.00 Los Patios 0.00 0.00 0.00 0.00 0.00 Manizales 0.00 0.03 0.03 0.03 0.03 Tierralta 0.00 0.00 0.00 0.00 0.00 Medellín 0.15 0.13 0.12 0.13 0.12 Source: Authors' estimation using ICFES data. Table 4: Average test scores 1998 1999 2000 2002 2003 Fe y Alegría schools Spanish 48.7 52.2 47.5 48.5 49.2 Mathematics 49.5 50.1 43.0 42.9 41.5 Biology 46.5 48.1 45.3 45.4 45.2 Chemistry 45.2 51.1 45.3 44.0 43.3 Physics 47.3 46.9 45.6 44.9 45.8 Other schools Spanish 51.1 54.7 48.3 50.0 51.2 Mathematics 51.3 52.5 43.4 43.9 42.5 Biology 50.2 51.0 46.4 46.6 46.8 Chemistry 47.6 54.1 46.3 45.5 45.0 Physics 49.6 49.8 45.9 46.1 47.3 Source: Authors' estimation using ICFES data. Table 5: Percentage difference in test scores after matching (Fe y Alegría minus Other schools) ­ CEM and PSM 1998 1999 2000 2002 2003 CEM Spanish 0.540 1.345** 0.199 -0.328 -0.747** Mathematics 1.548*** 0.083 -0.174 -0.093 -0.578** Biology -1.437** 0.239 -0.797*** 0.080 -0.877*** Chemistry -0.412 0.596 -0.354 -0.809*** -0.856*** Physics 1.362** -0.386 -0.130 -0.682** -1.308*** PSM Spanish -0.080 0.703 0.301 -0.399 -0.922*** Mathematics 1.023** -0.288 -0.104 -0.167 -0.701** Biology -1.969*** -0.370 -0.682** -0.121 -1.153*** Chemistry -0.668 0.013 -0.179 -0.925*** -0.936*** Physics 0.834 -0.999** 0.261 -1.068*** -1.352*** Source: Authors' estimation using ICFES data. * significant at 10% ** significant at 5% *** significant at 1% Annex: Probit regression for PSM Table A1: Probit regression for participation in Fe y Alegría (PSM) 1998 1999 2000 2002 2003 Owns dwelling 0.233*** 0.141*** 0.157*** 0.155*** 0.195*** Father completed primary or less (ref.) Father completed secondary -0.084 -0.142** -0.264*** -0.209*** -0.130*** Father completed college -0.340*** -0.321*** -0.456*** -0.372*** -0.322*** Father post-graduate education -0.399*** -0.313*** -0.444*** -0.372*** -0.286*** Mother completed primary or less (ref.) Mother completed secondary -0.040 -0.151** 0.006 -0.251*** -0.040 Mother completed college -0.162* -0.263*** -0.068 -0.339*** -0.201*** Mother post-graduate education -0.090 -0.202** 0.027 -0.345*** -0.184*** Father manager/owner (ref.) Father employee -0.048 -0.029 -0.041 -0.023 -0.038 Father construction worker/other 0.017 0.064 -0.044 0.030 0.077* Father homemaker, retiree, student -0.047 0.031 -0.131** 0.004 0.002 Mother manager/owner (ref.) Mother employee 0.152 0.225*** 0.106 0.062 0.069 Mother construction worker/other 0.362*** 0.331*** 0.184** 0.213*** 0.150** Mother homemaker, retiree, student 0.255** 0.225*** 0.069 0.074 0.040 Household size: 1 to 3 -0.065 -0.262*** -0.131** -0.169*** -0.048 Household size: 4 or 5 0.051 -0.126*** -0.056 -0.103** -0.029 Household size: 6 or 7 0.079 -0.075 -0.055 -0.052 -0.013 Household size: More than 7 (ref.) Per capita income: Less than 0.5 MW (ref.) Per capita income: 0.5 to 1 MW -0.298*** -0.228*** -0.295*** -0.321*** -0.292*** Per capita income: 1 to 1.5 MW -0.182** -0.010 -0.174*** -0.172*** -0.295*** Per capita income: 1.5 to 2 MW 0.010 0.278*** -0.005 0.015 -0.567*** Per capita income: More than 2 MW #N/A 0.102 0.038 -0.091 -0.574** Gender: female -0.015 0.039* 0.056*** 0.082*** 0.019 Age 15 (ref.) Age 16 -0.155 0.168*** -0.179* -0.056 0.544*** Age 17 -0.136 0.215*** -0.111 0.057 0.527** Age 18 -0.154 0.164*** -0.062 0.019 0.832*** Age 19 -0.238** 0.037*** -0.191* -0.059 0.536*** Age 20 -0.252** 0.037*** -0.343*** -0.125 0.222*** Student works -0.100* -0.110** -0.124** -0.065 -0.014 Armero 0.368 0.641*** 0.357 0.532** 1.005*** Barranquilla #N/A #N/A -1.000*** -0.595*** -0.629*** Bello 0.620*** 0.583*** 0.525*** 0.523*** 0.430*** Bogotá -0.306*** -0.572*** -0.438*** -0.370*** -0.392*** Cali -0.061 -0.163*** -0.337*** -0.326*** -0.126*** Cartagena -0.339*** -0.119*** -0.368*** -0.174*** 0.006 Ciénaga #N/A #N/A #N/A #N/A 0.294** Cúcuta #N/A #N/A -0.319*** -0.382*** -0.362*** Dos Quebradas 0.284*** -0.001 -0.165* -0.003 0.129 Ibagué -0.439*** -0.374*** -0.574*** -0.368*** -0.524*** Lérida 0.955*** 0.969*** 0.562*** 1.059*** 1.114*** Los Patios 1.065*** 1.037*** 1.088*** 0.868*** 0.759*** Manizales #N/A #N/A -0.332*** 0.096* 0.178*** Tierralta 1.412*** 1.243*** 1.011*** 1.391*** 1.064*** Medellin (ref.) Constant -2.219*** -1.785*** -1.700*** -1.443*** -1.734*** Source: Authors' estimation using ICFES data. * significant at 10% ** significant at 5% *** significant at 1%