Report No: ACS6932 . Republic of Colombia Improving Opportunities for Education II Final Progress Report . November 27, 2013 . LCSHE LATIN AMERICA AND CARIBBEAN . Page 1 of 31 Standard Disclaimer: . This volume is a product of the staff of the International Bank for Reconstruction and Development/ The World Bank. The findings, interpretations, and conclusions expressed in this paper do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. . Copyright Statement: . The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The International Bank for Reconstruction and Development/ The World Bank encourages dissemination of its work and will normally grant permission to reproduce portions of the work promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA, telephone 978-750-8400, fax 978-750-4470, http://www.copyright.com/. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA, fax 202-522-2422, e-mail pubrights@worldbank.org. Page 2 of 31 Colombia Programmatic Knowledge Services IMPROVING OPPORTUNITIES FOR EDUCATION Final Progress Report (Phase II – P132235) RESULTS STORY In October 2011, when the Colombia Improving Opportunities for Education PKS was launched, the Ministry of Education (MEN) faced a challenging situation. Net primary school enrolment had fallen from 94 percent (in 2000) to 90 percent. PISA test scores – which allow for an international comparison of language, math and science mastery by 15 year olds – lagged far behind those of the OECD, secondary school attendance rates were growing but tertiary school enrolment lagged behind that of Ecuador and on par with Paraguay. Recent policy initiatives, namely a tertiary education reform, were met by fierce student opposition in November 2011, resulting in the President pulling the proposed reform from Congress before it could be deliberated. In this context, the Bank was invited to provide support to the MEN to help get the education quality and coverage reforms back on track. The Bank provided a range of services including • Intensive accompaniment of the MEN to design an Upper Secondary Education Strategy. The Bank team provided peer review to the analytical pieces commissioned by the Ministry, hands-on technical assistance by a team of global experts, policy papers to provide guidance on best practices for the design of education systems and teacher improvement policies, an evaluation of a law intended to improve teacher quality, and concrete policy recommendations. • A book that reviewed the higher education system in Colombia and another that reviewed the system in the state of Antioquia. The books were presented at three separate events in Colombia, reaching more than 1500 people. The books’ findings were reported in more than 20 separate stories in the press. More than 3000 books were distributed and both have been translated into Spanish. The OECD, which co-authored both books, presented the results to President Santos. • Data collection and analysis to understand the education production process. The Bank team worked with the MEN, Secretary of Education of Antioquia, and civil society to observe hundreds of teachers in the classroom and to quantify their observations in terms of teacher time and materials use. The results were summarized in a country-level and state-level (Antioquia) report and had broader impact in the LAC region through their inclusion in a regional study that collected similar data in six other Latin American countries. To complement the policy recommendations, the team also prepared a detailed note on policies to improve teacher performance. The objective of the PKS was to share knowledge, but it ended up informing policy. By the end of the PKS series, the MEN is finalizing a bold new Secondary Education Reform Strategy. The tertiary education dialogue shifted to a small set of high quality reforms. Reforms to enhance teacher quality are under Page 3 of 31 Colombia Programmatic Knowledge Services IMPROVING OPPORTUNITIES FOR EDUCATION Final Progress Report (Phase II – P132235) November 27, 2013 Table of Contents Introduction ..................................................................................................................................... 6 Background, Context and Fit with Country Strategy...................................................................... 6 Development Objectives, Operational Impact and Indicators ........................................................ 9 Services and Products ................................................................................................................... 12 Dissemination and Quality Assurance .......................................................................................... 20 Team and Budget .......................................................................................................................... 20 Clients and Partnerships ................................................................................................................ 22 Risk Factors and Mitigation Measures ......................................................................................... 23 Lessons Learned............................................................................................................................ 24 Annex : Summary of PKS Activities, Phases I and II .................................................................. 25 Page 4 of 31 Acronyms ACCES Acceso con Calidad a la Educación Superior AAA Analytical and Advisory Services COLCIENCIAS Administrative Department of Science, Technology and Innovation CPS Country Partnership Strategy GoC Government of Colombia ICETEX Colombian Institute of Educational Credit and Technical Studies Abroad ICFES Colombian Institute for Educational Evaluation (Instituto Colombiano para la Evaluación de la Educación) LAC Latin America and Caribbean MEN Ministry of Education NDP National Development Plan PISA Programme for International Student Assessment PKS Programmatic and Knowledge Services R&D Research and Development STI Science, Technology and Innovation Page 5 of 31 Introduction 1. This note presents the achievements in Phase II of the Programmatic Knowledge Services (PKS) for Improving Opportunities for Education in Colombia and is the Final Report of the PKS series. The objective of this PKS was to provide evidence and related policy advice to enhance public sector capacity for improved education policy making for the primary, secondary, and tertiary levels and within the STI sector. The PKS was fully aligned with the country’s National Development Plan and with the Bank Country Partnership Strategy. The PKS was a multi-year program, covering FY12-FY14 and was implemented in two phases. This note presents the actions undertaken under Phase II (carried out in FY13) and closes the PKS series. The FY14 strategy for the Bank’s education program will focus on new lending. Background, Context and Fit with Country Strategy Background 1 2. Colombia’s education sector does not adequately prepare people for a competitive, global labor market. Primary school enrolment decreased from 94 percent in 2000 to 90 percent in 2011, short of the Latin America and Caribbean (LAC) regional average of 93.8 percent. 2 Secondary school enrolment shows a positive trend. Between 2002 and 2009, gross enrolment rates in secondary school rose from 57.4 to 75.6 percent. 3. The growing pool of secondary school graduates aspires to tertiary education. Between 2002 and 2010, gross enrollment at the tertiary undergraduate level rose from 24.4 percent to 37.1 percent and the percentage of undergraduates enrolled in technical and technology programs grew from 19.6 to 34.2. 3 Retention in tertiary is more of a problem, with dropout rates close to 46 percent in a cohort. Colombia has made strides in increasing every type of graduate enrolment. Between 2002 and 2010, Masters degree enrolment increased by 250 percent and Doctoral enrolment by nearly 550 percent. These gains are in line with the Government of Colombia’s (GoC) view that reforming and improving tertiary education 4 is vital to the country’s competitiveness, economic growth, and prosperity. Returns to tertiary education have been high in recent years and seem likely to remain so. 4. Learning gains, while significant, are not as rapid as needed to set Colombia on a greater path for innovation and growth. While Colombian student’s PISA (Programme for International Student Assessment) scores increased between 2006 and 2009, they continue to severely lag behind OECD countries. In the reading domain in 2009, Colombian students achieved a mean score of 413. This is 80 points below the OECD average of 493, indicating that at age 15 Colombia’s students lag behind students in an averagely-performing OECD country 1 This section presents the situation at the time Phase II of the PKS was initiated, thereby providing a benchmark for the review of Phase II. In most cases, the issues likely persist at the close of the PKS, though 2013 data are not available to confirm the assumption. 2 Source: EdStats. 3 ibid 4 This document will use the term “tertiary education” to encompass all the post-secondary education, which Colombians call “educación superior,” literally translated as “higher education.” This includes academic education leading to degree qualifications and vocational and technical education. Page 6 of 31 such as the United Kingdom, by the equivalent of two years of schooling. Colombia’s 15-year-olds performed less well than those in Chile (449), Uruguay (426) and Mexico (425); similarly to those in Brazil (412); and significantly better than those in Argentina (398), Panama (371) and Peru (370). Similar results emerged from the mathematics and science portions of the test. The Global Competitiveness Index ranks Colombia’s education system quality as 77th in the world, with quality of math and science ranked at 107th. Colombia’s ranks 66th in the innovation pillar of the World Economic Forum’s Global Competitiveness Index 2012-13, a significant drop from its 2011-2012 ranking of 57. 5 5. The Ministry of Education (MEN) has been undertaking efforts to improve the quality of education at all levels and to expand coverage of secondary and tertiary education. Since teachers are the main input to a quality education, the Ministry of Education is revisiting the 2002 Estatuto Docente (Teacher Statute) to determine if it, indeed, provides incentives to teachers to improve their performance through teacher evaluation linked to job promotion and continuation. To address quality and access to quality post-basic education, a tertiary education reform was presented to Congress in 2011, but due to student and professor dissatisfaction with the absence of a highly consultative process, the reform was pulled. Drawing on lessons from that experience, the MEN embarked on an upper secondary education reform process, which will be completed in early 2014. Over two years, the MEN has been preparing diagnostics and shaping a reform that will move the upper secondary school system into a 21st century education system. The issue of education quality and equitable access to tertiary education became a priority for ICETEX, the organization that manages ACCES, Colombia’s primary student loan program. The ACESS is a powerful tool to provide incentives to the tertiary education system to improve the quality of their services, both benefitting the student beneficiaries and enhancing the long-term viability of the model. Fit with Country Strategy 6. The Programmatic Knowledge Services (PKS) are an important component of the integrated program of financial, knowledge, and convening services put forward in the World Bank’s Country Partnership Strategy (CPS) (Report 60620-CO), which is designed to selectively support Colombia’s National Development Plan (NDP) (Prosperidad para Todos 2010-2014). In December of 2010, the Government of Colombia launched the NDP, with the overarching goals of increasing employment, reducing poverty and improving security. The NDP has three main objectives: (1) Sustainable Growth and Competitiveness; (2) Equality of Opportunities for Social Prosperity; and (3) Consolidation of Peace. The PKS supported the GoC’s NDP first two goals. 7. This work fits under two pillars of the CPS. The work carried out in this PKS contributed to the subtheme entitled, “Improved Productivity and Innovation” of CPS Pillar III “Inclusive Growth with Enhanced Productivity,” and the second subtheme, “Improved Opportunities in Education,” of Pillar I, “Expanding Opportunities for Social Prosperity.” The logic mirrors that of the NDP: increased human capital raises productivity and improves the 5 World Economic Forum (2012) The Global Competitiveness Report 2012-2013: Full Data Edition. Klaus Schwab and Xavier Sala-i-Martin (eds). (World Economic Forum: Geneva). Page 7 of 31 quality of life of those who possess it, but creating it requires a path where opportunity is provided as broadly and equitably as possible. 8. The World Bank has a long history of supporting quality education in Colombia. Between FY08 and FY11, the World Bank carried out a multi-year Program of Analytic and Advisory Activities (AAA) to support the Government of Colombia in improving the quality and relevance of its Basic and Secondary education programs and policies. Specifically, the “Education Quality Programmatic Advisory Knowledge Services” (P106710) provided technical expertise and policy advice – informed by analytical work and international experience – to contribute to consensus building among key actors in Colombia on policy reforms and actions. The Bank’s recent “Improving the Performance of Social Services PKS – Phase I” (P127472) was a joint ED-HNP effort to enhance public sector capacity in Colombia to use information for improved policy making through the development of coordination and accountability frameworks and tools in health, education and early childhood development. The on-going joint HD-SDN “Peace Consolidation PKS” (P144491) addresses the issue of peace consolidation in Colombia; the HD Component of that PKS focuses on education for peace in schools. 9. Three on-going operations and two new operations are closely related to the activities covered by this PKS. The Antioquia Secondary Education Project (P052608) supports student completion of secondary school and transition to tertiary through a mix of student loans, grants, flexible schooling models, and counseling services. The Rural Education APL II (P082908) seeks to improve rural education quality at pre-school, primary and secondary levels; develop mechanisms that promote equity among vulnerable population groups (including ethnic groups); and institutionalize rural education delivery capacity in departmental and municipal Secretaries of Education. The Strengthening the National System of Science, Technology and Innovation Project (P117590) focuses on COLCIENCIAS (Administrative Department of Science, Technology and Innovation) and aims to strengthen its capacity to promote human capital, R&D and innovation, and to raise awareness of science, technology and innovation. Two education operations are under preparation. The Student Loan Support Project II (P145782), which follows up on a recently closed project (P105164) supports ICETEX (Colombian Institute of Educational Credit and Technical Studies Abroad) to improve coverage and equity of tertiary education through student loans, as well as increasing ICETEX’s financial sustainability. The Improving Access and Quality of Upper Secondary Education Project (P145353) will support the MEN’s implementation of their new Upper Secondary Education Strategy. 6 For an overview of Bank’s recent engagement in Colombia’s education sector, see Figure 1. 10. The client played an active role in defining the PKS activities and in providing inputs as specific outputs/processes were realized. Three processes of client engagement were used. First, the Bank team agreed with the clients those activities that were initiated in Phase I and would be completed in Phase II. Second, the Bank team identified new client requests that fit within the PKS objectives and were within the Bank’s areas of expertise. Third, as products were being developed, the team was in constant dialogue with the client to ensure relevance of and client input into the products. 6 This project is under preparation. The specific components cannot be detailed at the time of the preparation of this Final Progress Report. Page 8 of 31 Figure 1: Bank Engagement on Education in Colombia in the Context of Long-Term Engagement on Human Development 2000-2008 2009-2011 2012-2015 General Policy across the Education Innovations for Learning Education for a Productive and Cycle Peaceful Colombia • Rural Education APL I (P050578) • Rural Education APL II (2008-2013) • Science, Technology, and Innovation • Social Sector Adjustment Loan (P069861) (P082908) Project (P117590) • Higher Education – Improving Access • Antioquia Upper Secondary Education • Student Loan Project APL II - Financial Services (P074138) Project (P052608) ICETEX (P145782) • Cundinamarca Education Quality • Student Loan Project APL I – ICETEX • Improving Access and Quality of Improvement (P077757) (P105164) Upper Secondary Education • Programmatic Labor Reform and Social (P145656) Structural Adjustment Loan I (P079060) • Programmatic Labor Reform and Social Structural Adjustment Loan II (P082865) • 3rd Programmatic Labor Reform and Social Development Policy Loan (P094097) • Education Strategy (P056495) • CO Education Quality Programmatic • Improving Opportunities for • Education Study (P085591) Advisory Knowledge Services II Education PKS (P129612 & Knowledge Services • Public Training Reform Issues (P074786) (P106710) P132235) • Higher Education Strategy (P077435) • Skills for Shared Growth (P123144) • Improving the Performance of Social • CO Education Quality Programmatic AAA • Empowering Young Women Affected by Services PKS (P127472) Phase I (P106634) Violence Pilot Project (P052608) • Peace Consolidation PKS (P144491) • Soccer Together Pilot Project (P125697) • Medium Term Effects of • Youth Reintegration Pilot Project Home-Based ECD (P133512) (P095598) • Enhancing Governance, Transparency and Accountability in Education (P125541) • Study-tour for high-ranking Peruvian • Improved ECD Action Plans in La Convening Services officials to learn from the Escuela Nueva Paz and El Alto (GFR # 10885) model in Colombia (2011) • Community of Practice in ECD Page 9 of 31 Development Objectives, Operational Impact and Indicators 11. The objective of the PKS program was to provide evidence and related policy advice to enhance public sector capacity for improved education policy making for the primary, secondary, and tertiary levels and within the Science, Technology, and Innovation (STI) sector. The objective of the Second Phase of this PKS was to provide analytical work, advisory services, and convening services to support the Ministry of Education’s efforts to identify bottlenecks to higher quality service delivery and to support Colciencia’s efforts to do the same for the STI initiative that it leads. 12. The common theme across the activities covered by this PKS was optimization of investment in human capital through improved quality at all levels of formal education. Improving opportunities for education should simultaneously increase the competitiveness of the Colombia economy and thereby its long-term growth potential by employing greater numbers of more capable individuals in the workforce. At the same time, those individuals benefit from the fruits of greater competitiveness through higher wages, more and better employment, and other corollaries. Making this happen requires, inter alia, policies that maximize equality of opportunity, especially educational opportunities. 13. The activities supported by the PKS during its Phase II had significant operational impact. In particular, the Bank’s work was crucial in the following areas: • Upper Secondary Education Reform. The Bank’s support to the MEN’s Reform process in Phase II created a strong, trusting relationship. The technical feedback and the expert advice were highly valued by the Minister. In November 2012, the DNP formally requested a Bank operation to support the reform • Tertiary education. The dissemination of two studies (written under PKS I) generated thoughtful debate, both within the Ministry and in the press. The PKS results were adopted by the Vice Ministries of education in their sub-sectoral stratetegies and the Antioquia results are informing the debate on the regionalization of higher education in that department. 14. The PKS largely met its objectives. Specifically, it provided evidence and policy advice to inform reforms at the primary, secondary, and tertiary education levels. The work in Phase II primarily benefitted the MEN’s efforts to develop a bold Upper Secondary Education Reform strategy. As discussed below, the PKS did not ultimately address the STI portion of the objective because STI work was found to fit better, and thus be adequately covered under, the Colombia Innovation and Competitiveness PKS (P144510). PKS results are presented in Table 1; the program results and indicators are those that were proposed in the Concept Note for Phase II. The results for each Component are presented after the Component description in the next section. Page 10 of 31 Table 1: Program Result Targets (presented in the CN) and End of PKS Status Program Result Indicator Baseline End of PKS Increased awareness Use of Phase I 0 Documents are being used by MEN, Vice of stakeholders of documents by Ministry of Basic & Secondary Education, the key issues in policymakers, the Vice Ministry of Tertiary Education, and upper secondary civil society, the Secretary of Education in Antioquia, as education leaders, and evidenced by referral to documents in policy teachers dialogue and in implementation of policy advice in the respective reform agendas Raise awareness Number of 0 Disseminated to 3000 stakeholders. among policymakers references to Incorporated into strategies at the national of the regional OECD reports and subnational levels. Reported in more dimensions in than 20 newspaper stories. Results shared tertiary education with an audience of more than 1500 people. policy Increased Number of 0 Activity moved to Innovation and inclusiveness of STI international Competitiveness PKS (P144510) in Colombia STI conferences that Colciencias participates in Raised stakeholder Resubmission 0 The tertiary education strategy is under awareness of key of tertiary development, with inputs from PKS issues in tertiary reform or activities (described below). The upper education announcement secondary education reform strategy of new strategy incorporates elements to better prepare students for the transition to tertiary education, including 12th year of school and a new curriculum. Increased knowledge Improved 0 Delayed until the Upper Secondary by education validity of curriculum reform is completed. Dialogue policymakers in SABER tests continues via the Bank’s new project for testing tertiary education (ICETEX). Greater Improvements 0 New algorithm to better target students from understanding by in ICETEX’ lower socioeconomic strata. ICETEX of how to measurement remove obstacles for and targeting students from lower socioeconomic strata to access tertiary education Page 11 of 31 Services, Products and Results by Component 15. The PKS Team delivered 8 of the 11 planned outputs in Phase I and added three new outputs. The criteria for adding new products were (i) high value, low cost due to the PKS budget constraints, and (ii) a response to client demand that had not been anticipated. The new outputs were: • Inclusion of the Colombia case study in a regional study of classroom time use • Preparation of a report on classroom time use in Antioquia • Policy Note of global best practices in teacher policy and application to Colombia 16. The criteria to drop outputs were (i) no longer of client demand due to a change in priorities, and (ii) greater economies of scale by carrying out the activity via another product or at another time. The three dropped outputs were • A Policy document based on “Review of National Policies for Education: Tertiary Education in Colombia.” The government used the full document so the team deemed this summary unnecessary to advance the dialogue. • Workshops with ICFES to develop technical capacity for constructing a new test. This activity is postponed until the new curriculum for upper secondary education is developed and implemented. Dialogue continues through the preparation of a new project for tertiary education. • COLCIENCIASs participation in international event. This activity better fits with the objectives of the Innovation and Competitiveness PKS and was thus moved to that PKS, as reported in the Concept Note for the PKS, which was approved May 2013. 7 17. Table 2 summarizes the outputs originally envisioned in Phase II, the outputs that were produced, and a brief explanation of any deviation between the planned and final outputs. Table 2: Proposed and Actual Outputs in Phase II Proposed Activities Actual Activities Comments Component I: Strengthening the Efficiency of Education Classroom Observation 3 outputs delivered: Paper The data were available to Analysis. Analytical paper analyzing Colombia data, produce 3 pieces, each analyzing classroom Paper analyzing Antioquia intended for a different observation data data, Inputs to LAC regional audience. The regional, study country, and state-level analyses allow benchmarking 7 The overall program development objective of the Innovation and Competitiveness PKS is to stimulate increased innovation and technology development in Colombia by building government and private sector capacity through a combination of knowledge and convening services. HD-team’s contributions include: (i) Just-in-time knowledge exchange to support client participation in international conferences and/or online training pertaining to innovation that may arise and be of interest to the client (COLCIENCIAS), and (ii) Advisory services and activities around how to use science, technology and innovation to address social challenges and meet development objectives in Colombia. Page 12 of 31 across different levels. Full School Day Convening Clients attended regional The workshop was a more Activities. 1 workshop, 1 conference on full day efficient means to meet the SSKE with other country schools client’s learning and working on the issue connecting needs Component II: Reforming the Upper Secondary Education System for Greater Quality and Relevance Assessing the impact of the 2 outputs delivered: The policy note was added to Estatuto Docente on Paper presenting the this task to more fully develop learning outcomes. evaluation results, Policy the paper’s policy Note on teacher reform recommendations. Review of upper secondary Paper produced, and As planned education policy in expanded to include Colombia. international lessons. Expert advice to support 1 workshop, 2 missions, As planned MEN’s Upper Secondary and 2 policy guidance Education Strategy papers produced Component III: Strengthening Access to and Quality of the Tertiary Education System Policy Note on Tertiary Dropped Before the team could begin Education in Colombia. A work on the document, the concise summary of priority Ministry was using the study findings and related policy results, making a policy options identified in the document unnecessary. (Phase I) OECD study Dissemination of the One large event Event was larger than OECD/WB review of the originally planned and with tertiary education system in good press coverage so the Colombia. 3 events team decided the remaining 2 planned events were not necessary. Book was also translated and printed in Spanish. The book was disseminated to more than 2000 stakeholder. Dissemination of the Delivered The events were carried out OECD/WB review of the as planned. Approximately tertiary education system in 1000 books (English) were Antioquia. 2 events disseminated and a Spanish translation was prepared Beyond SABER11 and Dropped The process is delayed until SABER PRO: Developing the new secondary education ICFES’ technical capacity curriculum is developed in the for constructing a new test. context of the on-going Upper 2 technical workshops. Secondary Education Reform Enhancing equity in tertiary Just in time advice The team deemed it more Page 13 of 31 education through better provided to ICETEX to efficient to carry out the measurement and targeting improve their poverty activity in the context of - 3 technical workshops. targeting mechanism project preparation with ICETEX rather than the originally planned events with ICFES, an institution that has little leverage on education equity. Addressing STI challenges Moved to Innovation and In May 2013, a PKS focused by strengthening Competitiveness PKS on innovation was developed. Colciencias – Participation in inclusive innovation conference Component 1: Strengthening the Efficiency of Education 18. In addition to coverage gaps, learning outcomes are far lower than hoped. The Ministry of Education is thus searching for sources of inefficiency in the current system. One potential source of inefficiency is a sub-optimal use of classroom time at both the primary and secondary school levels. Component 1 included two activities to address this issue. • Analysis of the classroom observation data. (analytical paper). One paper was envisioned but three outputs were produced. First, Colombia was included as a case study in the regional study “Building Better Teachers in Latin America and the Caribbean: New evidence on Strategies for Teacher Quality and Student Learning”. 8 The study finds that Colombian teachers spend a greater proportion of classtime in academic instruction as compared to the seven other countries in the sample (65 percent v. 60 percent). However, the 65 percent of time spent in classroom instruction is still far below the OECD recommended 85 percent, and far too much time is spent in administrative tasks (25 percent). Colombian teachers have the highest incidence in the LAC region of not using any materials in the classroom, such as blackboard, notebooks, textbooks, etc. The study is in its final phase of review, has been presented at an international conference at Harvard University, and will be shared with the MEN when completed. Second, a country-specific paper was produced as a background paper to the regional study. In Phase I of the PKS, some initial statistics were summarized in a PPT and share with the MEN. In Phase II, a full analysis and a complete research paper were prepared: “Colombia- Dinámicas en Clase Escolar: Informe Sobre Observaciones de Aula Utilizando el Método Stallings”. The main findings are that there is a great deal of heterogeneity in teacher time use in Colombia. While greater time spent in academic instruction is generally correlated with higher student test scores, those most in need of 8 The data were collected expressly for the purpose of this study. The data collection process was based on classroom observation in grades 5, 9, and 11 of a randomly selected sample of schools (representative at the urban national level). The data were analyzed using non-parametric summary statistics. Page 14 of 31 such intensive instruction are not receiving it. Specifically, teachers of students in the lowest academic performance quintile only spend 49 percent of classtime in academic instruction, the rest of the time the teacher is in non-teaching tasks (social time inside or outside of the classroom) or in administrative tasks (including discipline), as compared to 82 percent in the highest performing schools. The inequality is perhaps worse within schools. In the worst performing schools, there is a 60 percentage point gap in time usage in academic instruction (ranging from 18 to 78 percent) while in the highest performing schools, the gap is 34 percentage points (ranging from 63 to 97 percent). The report recommends training teachers on management of administrative tasks and on how to keep students engaged in academic activities. It also recommends greater use of in-classroom teacher training. Third, a region-specific paper was produced for the department of Antioquia. Using the same methodology as the LAC regional study and Colombia-specific study, “Antioquia, Colombia- Dinámicas en Clase Escolar: Informe Sobre Observaciones de Aula Utilizando el Método Stallings” finds that teachers dedicate 62 percent of their time to academic instruction, slightly below the national average and above other countries in the region. Antioquian teachers spend more time in administrative activities than Colombian teachers (by 2 percentage points). Fewer classrooms in Antioquia do not use any teaching materials (21 percent) as compared to the national average. The report recommendations are similar to those presented in the national report. The results from these studies have influenced policymaking in two specific ways. First, they reignited a discussion around the use of in-classroom training to teach pedagogical practice. The model had fallen out of use but the analytical work and discussion around it has reinitiated this model. Second, the MEN is planning to adapt the survey instrument used in the studies to evaluate the new in-service teacher training strategy, under implementation in the Rural Education program. • Knowledge sharing and convening services for “full school day” (convening). 9 The PKS originally envisioned two activities: (i) a workshop to share the results of a literature review with clients at the national and sub-national levels, and (ii) a South-South Knowledge exchange between Colombian clients and those working on this issue in other LAC Countries. Instead, the team took advantage of a regional conference on full time schools that the Bank hosted in the Dominican Republic in March 2013. The conference presented the full school day experiences of Brazil, Argentina, Mexico, Dominican Republic and Uruguay and touched on a range of topics including curriculum, teacher training, parental engagement, and decentralization. The PKS funded the conference participation of the Secretary of Education of Medellin and of the MEN Director for full time schools. 9 The idea of the full school day – as envisioned in Colombia – is to focus academic teaching during the normal school day and to engage the children/youth in activities to develop their socio-emotional behavioral skills and to address non-academic issues (environment, gender, rights, etc.) during the “extended period,” of two to three hours daily. Page 15 of 31 Following the conference, the MEN initiated a study of the full-time school model in its Rural Education project. The municipality of Medellin has asked the Bank to provide technical assistance on an impact evaluation of their on-going full-time school pilot. Component 2: Reforming the Upper Secondary Education System for Greater Quality and Relevance 19. The MEN is in the process of reforming their upper secondary education system. The unacceptably high dropout rates, low learning, and high unemployment rates among youth point to the failure of the current system to serve this part of the population. As a result, the MEN is engaged in a process to reform the system, which includes several years of analysis, culminating in a reform strategy. The MEN invited the Bank to provide knowledge and convening services on a range of topics. Component 2 includes several knowledge/convening activities: • Assess the impact of the Estatuto Docente on learning outcomes. (analytical paper) The PKS envisioned a study to estimate the effect of the 2002 Teacher Statute (Estatuto Docente) 10 on teacher quality, as proxied by learning outcomes. Using a difference in difference technique, “The New Statute of Teachers’ Professionalization Effects on Education Quality in Colombia” finds that students from schools with a higher proportion of new Statute teachers have lower scores on the national achievement test (SABER 11). To improve the results hoped for under the reformed Statute, the report recommends training principals to effectively implement the new evaluation system, using teacher evaluation data to provide feedback to teachers to improve their classroom practices and impact students’ achievement, and improved collection of critical data to allow for monitoring. A policy note was also prepared to more fully develop the policy options presented in the Teacher Statute paper. The policy note draws on the findings of the Teacher Statute analysis as well as the SABER Teachers evaluation, other impact analyses and analytical studies, and international examples from the regional study Building Better Teachers. Some of the specific policy options proposed include: strengthening adequate rubrics and standards for the use of comprehensive teacher evaluations in practice; supporting the use of classroom observation for in-service training and professional development activities such as feedback, coaching, and mentoring; strengthening the capacity of school directors to implement the reform by providing professional development to improve their managerial capacities, pedagogical leadership, and use of evaluation and performance management tools; and improving quality control of data and integrating databases across agencies to support reform monitoring. • A Review of Upper Secondary Education Policy in Colombia (just-in time advice, convening). The original plan was to (i) provide global expert advice to the clients, and (ii) prepare a paper that summarizes the findings from the client’s studies and provide 10 The Statute focuses on evaluation and professional development. See Concept Note for Phase II of the PKS for details. Page 16 of 31 policy options. The first output was produced. The second output was expanded to draw from the global secondary education literature and to provide observations to feed into Colombia’s reform process. Technical Assistance. The technical assistance consisted of TA missions with international experts to advise on the upper secondary education strategy that is under preparation by the MEN. The support took two forms. First, a team of international experts joined the MEN in a 2-day strategy workshop in October 2012. The Bank’s team reviewed and provided feedback on the background papers commissioned by the MEN. A short report based on the mission’s assessment was prepared by the Bank to share mission conclusions with the clients. The main conclusions were: (a) the current education system is designed for the needs of the past; a new system is needed for the future, (b) the upper secondary education reform requires extending the school cycle, delayed vocational education tracking, and an emphasis on quality of learning defined by competencies, and (c) the gap in upper secondary education legislation is an opportunity to design a system that serves these potential reform objectives. Second, the Bank fielded two missions to provide feedback on the strategy in closed MEN sessions. A Bank-sponsored team spent a week in April and another in June meeting with a range of stakeholders including Congressional members, the private sector, and students and providing TA to the Ministry of Education to help shape the Upper Secondary Education Reform process. After the first mission, a short report that summarized the team’s diagnosis of the problems and proposed solutions 11 was shared with the MEN. The report concluded that the Upper Secondary Education Reform is a priority since it is the main bottleneck to social inclusion. But the problems in the current system are multiple, ranging from a conceptualization of the sector’s role – tracking students early in the education cycle and graduating them too young – to a misconceived interpretation of school autonomy (total independence rather than central support to autonomous initiatives). The paper concludes the MEN should proceed with the reform, a multi-sectoral committee should be put together to conceptualize it, the process must develop a model to incorporate the private sector’s views, and the need for an education system that keeps students in the system for an adequate period to develop the skills that will serve them throughout their lives. Colombia in the international context. A review paper was produced to provide international evidence and best practice policy advice as an input to the MEN’s strategy. The main findings of “La Educación Media en Colombia: Estrategias para su Fortalecimiento, Mejoramiento y Modernización” are (i) Colombia has a shorter education cycle than other countries, which commonly have one more year of formal education, (ii) the Colombian system that tracks secondary school students into (often dead end) vocational or academic streams is leading to a generation of ill-prepared technical workers, (iii) expanding access to tertiary education can create incentives for secondary school completion, and (iv) initial and continual teacher training, in parallel with curriculum reform, should focus on enhancing teachers’ ability to more effectively 11 The mission’s analysis and results were based on prior knowledge of literature and past experience in strategy design; the team did not conduct empirical research. Page 17 of 31 teach basic academic and socio-emotional competencies. The paper findings have been discussed with the clients. As a result of the Bank’s support, MEN’s upper secondary education strategy is closer to world class standards, drawing from global experience brought to the discussion through these activities. Elements considered in the reform strategy include: reform of the structure of the upper secondary education system and the curriculum to ensure that all students graduate from secondary school with fundamental competencies (generic, academic and exploratory); eliminating the possibility of pursuing pure technical programs offered by the National Training Service (SENA), instead allowing for additional technical training after the standard school day or after completing secondary education (currently, grade 11); adding an additional grade to the education cycle; professionalizing the secondary teacher education track and focusing initial and continual training on teaching for learning outcomes; and reforming curriculum and testing to focus on academic and higher order cognitive skills (as opposed to testing for content) and socio-emotional skills. As proposed by the technical team, the MEN formed an inter-sectoral committee to guide the reform process, which includes representatives from sub-national Secretaries of Education, employers, and the tertiary education. The close collaboration with the client on these activities allowed for the Bank to leverage the client’s technical knowledge and increased the clients’ technical ability to assess and design education strategy. Component 3: Strengthening Access to and Quality of the Tertiary Education System 20. The GoC is conscious of the key contribution tertiary education can make to the country’s development and prosperity, and is committed to ensuring its quality and relevance. Colombia’s tertiary system must continue to actively address key challenges associated with quality, relevance and coverage in order to achieve inclusive growth with enhanced productivity while at the same time expanding opportunities for social prosperity. Component 3 included the following activities to support this end: • Dissemination of the “Review of National Policies for Education: Tertiary Education in Colombia”. (convening). Under Phase I of the PKS, The World Bank and OECD jointly produced a comprehensive overview of Colombia’s tertiary education system, at the request of the Ministry of Education. 12 Under Phase II, two activities were carried out. First, an event to launch the publication was held in Bogota in January 2013, jointly with the OECD and the Ministry of Education. More than 1000 people participated in the event. The event was positively reported in the media in more than 20 individual stories. The OECD report team met with President Santos to discuss the report. Second, the book was printed in English and was translated into Spanish and printed. The Spanish version was disseminated through direct mailings to more than 2000 people, 12 For a summary of study results, see the PKS Phase I Progress report. Page 18 of 31 mainly university rectors, libraries, and researchers and was distributed to a more general audience through book fairs. Several report findings have been adopted by the MEN. Specifically, the Vice Ministry of Higher Education is incorporating into its strategy the report’s recommendations on improving equity in higher education through improving financing mechanisms and in improving system efficiency through funding higher education based on performance rather than per capita. The Vice Ministry of Basic and Secondary Education has picked up several recommendations, including: a 12th year of school and modifying the ICFES such that it tests material taught in the curriculum. • Dissemination of the “Higher Education in Regional and City Development: Antioquia, Colombia. (convening) In Phase I of this PKS, the Bank again joined forces with the OECD to prepare a review of Antioquia’s tertiary system. In phase II, three activities were carried out. First, the report was launched in July 2012 in a public event that included departmental officials, municipal officials, the academic sector and the private sector. Second, a smaller event was hosted by EAFIT – the premier university in Medellin. Third, the document was printed as a book in English and disseminated to more than 1000 people; it has been translated into Spanish and is awaiting OECD clearance for publication. The report results have been used by the Secretary of Education of Medellin in the strategy to regionalize tertiary education. This includes the development of branch universities at the municipal level. • Enhancing equity in tertiary education through better measurement and targeting (analytical support). Originally the PKS envisioned that workshops would be held to assess current levels of enrolment across different levels of income, identify obstacles for the access of the socioeconomically disadvantaged youth to tertiary education, and strategies for their removal, and develop a strategy for better measurement of socioeconomic status and targeting of disadvantaged students. These issues emerged as central discussions in the preparation of the ICETEX project so the team focused on providing technical assistance through the loan preparation process rather than having a separate dialogue on the issue. As a results of this technical assistance, the algorithm used by ICETEX to allocate student loans has been changed to better target economically disadvantaged students, which is expected to increase the share of ACCES beneficiaries from disadvantaged background by 5 percentage points, equivalent to an additional 2500 students annually (from a base of 10,500 students annually). The client has greater capacity to assess options for future modifications to the algorithm. Further, the discussion highlighted the difficulty that students from lower socio-economic backgrounds have in transitioning to tertiary education, leading to a request from the Vice-Minister of Higher Education for Bank support in implementing programs to facilitate the transition. Page 19 of 31 Dissemination and Quality Assurance 21. The quality control, delivery, and dissemination mechanisms differed by product. The studies were subject to peer review while technical assistance, dissemination events, and workshops were not. The process followed for each knowledge product is presented in Table 3. The Classroom Observation Analysis book went through a Bank review process, where comments were provided by peer reviewers, a review meeting was chaired by the LAC Chief Economist, and after comments were incorporated the whole book was subject to an external review process managed by Stanford Press. The quality assurance for the task “Enhancing equity in tertiary education through better measurement and targeting” was through standard Bank project quality processes – peer review and written comments and discussion via a PCN review, QER, and Decision Meeting. The background papers for the Classroom Observation Analysis, Estatuto Docente evaluation, and Upper Secondary Education Policy paper were through written peer comments from recognized top experts in the respective topics. The comments were sufficiently straightforward to not merit a review meeting. Team and Budget 22. The core team comprised of Marcelo Becerra (Senior Economist), Raja Bentaouet Kattan (Senior Education Specialist), Elsa Coy (Tea Assistant), Maria Elena Paz Gutzalenko (Team Assistant), Alexandra Gonzalez (consultant), Martha Laverde (Senior Education Specialist), and Wendy Cunningham (TTL, Sector Leader LCSHD). Table 3: Quality Control and Dissemination of Knowledge Products (Phase II) Activity Quality Control Dissemination Component I: Strengthening the Efficiency of Education Classroom Formal review process in the context The regional study Observation Analysis of the regional study. The study dissemination strategy will was reviewed by five peer reviewers be determined by the and the review meeting was chaired Regional Leadership Team by the Chief Economist. The book (RLT). The country and is going through an external review state studies will be process. Colombia and Antioquia disseminated jointly with papers reviewed by Barbara Bruns the regional study. The (Lead Education Economist) study results have been informally shared with the MEN. Component II: Reforming the Upper Secondary Education System for Greater Quality and Relevance Assessing the impact Written comments provided by Report and Policy Note to of the Estatuto Juan Baron (Economist, LCSHE) be shared with the MEN Docente on learning and Halsey Rogers (Lead outcomes Economist, HDNED) Review of upper Written comments provided by Paper shared with the MEN secondary education Miguel Szekley (Director, during preparation policy in Colombia Instituto Tecnológico y de Estudios Superiores de Monterrey), Ernesto Cuadra Page 20 of 31 (Lead Education Specialist), Victor Manuel Moncayo (Universidad Nacional) Component III: Strengthening Access to and Quality of the Tertiary Education System Enhancing equity in The project peer reviewers: Implemented into the tertiary education Andreas Blom (Lead Education ICETEX project that is through better Economist), Maria Paulina under preparation measurement and Mogollon (FPD Specialist), targeting - 3 technical Francisco Marmolejo (Lead workshops. Education Specialist) 23. The Advisory Group, who oversees the quality of the PKS’s strategic direction, is composed of the following individuals: Rita Almeida (Senior Economist, LCSHE), Andreas Blom (Andreas Blom, AFTEE), and Maria Paulina Mogollon (Strategy and Operations Officer, FPDVP) reviewed the Final report and provided written comments. Some also reviewed individual products, namely the “Enhancing equity in tertiary education through better measurement and targeting” task. Additional reviewers with different skills sets were invited to review specific products; the reviewer for each product is identified in Table 3. 24. Phase II cost $159,000; $104,000 in BB, $50,000 in partner funding, and $5000 in TF. Funding for the activities was drawn from PKS budget, project budgets, a regional study, partner funding (OECD), and counterpart funding. Most of the activities were delivered by core team members (with limited use of consultants) due to the nature of tasks and the scarcity of VC. Table 4 presents the team leader, cost and funding source for each activity in Phase II. Table 4: Cost and funding (Phase II) Activity TTL Cost Funding Source Strengthening the Efficiency of Education Stallings studies on the use classroom time M. Laverde $30k BB Full School Day Convening Activities M. Laverde $5k BB (PKS) Reforming the Upper Secondary Education System for Greater Quality and Relevance Impact study of the Estatuto Docente on R.B. Kattan $5 TF learning outcomes $10 BB (PKS) Expert advice to the Ministry of Education M. Laverde $55k BB (PKS & project funds) Review of upper secondary education policy M. Laverde $14k BB (PKS) in Colombia Strengthening Access to and Quality of the Tertiary Education System Dissemination of the “Review of National M. Crawford/ $30k Counterpart funds Policies for Education: Tertiary Education in M. Laverde (events), partner funds Colombia” – events, translation and printing (printing and translation) Dissemination of the “Higher Education in M. Laverde $20k Counterpart funds Regional and City Development: Antioquia, (events, translation), Colombia” – events, translation and printing partner funds (events, Page 21 of 31 printing), BB (PKS) Enhancing equity in tertiary education M. Becerra/J. $10k BB (Project funds) through better measurement and targeting Entwistle 25. The budget for the 2-stages of the PKS was $1,236,000. The greatest expenses were incurred by the clients and partners, in the preparation of background studies for the Upper Secondary Education Reform Strategy (for which the Bank provided technical assistance and peer review) and the production and dissemination of the Tertiary Education studies (funded by the OECD), respectively. A summary of anticipated and the actual expenses for Phases I and II, by funding source, are presented in Table 5. Table 5: Anticipated and Actual Costs, by Funding Source, for both phases of the PKS Type Phase I (FY12) Phase II (FY13) Total Estimated Actual Estimated Actual Actual BB-WPA financed activities under PA/PKS n/a 80k 108k 49k 129k code Trust Funds n/a 0 15k 5k 5k Other stand-alone BB- n/a 167k 63k 75k 242k activities (SPN; IDF etc) Counterpart/Partner n/a 810k 0 50k 860k Total PA Engagement A budget was not 1.057k 186k 179k 1,236k prepared for phase I Clients and Partnerships 26. The main clients and counterparts for the Phase II of the PKS include the Ministry of Education; the Vice-Ministry of Higher Education (Viceministerio de Educación Superior); the Vice-Ministry of Pre-school, Primary and Secondary Education (Viceministerio de Educacción Preescolar, Básica y Media); ICETEX; and the Antioquia Education Secretary (Secretaria de Educación de Antioquia). Target audiences for consultations and dissemination included the immediate counterparts, as well as think tanks, universities, non-government organizations other line ministries, and state and local governments. Table 6 provides details. Table 6: Counterparts and Partners for Areas of PKS, Phase III Activity Client Partner Other Bank collaboration Component 1: Strengthening the Efficiency of Education Classroom Observation Analysis MEN, Secretary of MEN, Fundacion LAC Chief Education of Compartir, Economist Antioquia Antioquia Sec. of Office Education, CorpoEducación Full School Day Convening MEN, Secretary of LAC Ministries of LC3 HD team Page 22 of 31 Activities Education, Education Antioquia Component 2: Reforming the Upper Secondary Education System for Greater Quality and Relevance Impact of the Estatuto Docente on MEN Fedesarrollo, As peer learning outcomes Harvard Univ reviewers Review of upper secondary MEN Univ de los Andes As peer education policy in Colombia and Universidad reviewer Nacional Provide expert advice to the MEN’s MEN International ECA and Upper Secondary Education Strategy consultants MENA HD Strengthening Access to and Quality of the Tertiary Education System Dissemination of the “Review of Academia, think MEN, OECD HD Anchor National Policies for Education: tanks, MEN, Tertiary Education in Colombia” – Secretaries of events, translation and printing Education Dissemination of the “Higher Antioquia Sec. of MEN,OECD --- Education in Regional and City Education, Development: Antioquia, Colombia” Universidad de Antioquia, Universidad EAFIT Enhancing equity in tertiary ICETEX, MEN DNP As peer education through better reviewers measurement and targeting 27. Cross-sectoral collaboration. The team worked with various departments across the Bank. Specifically: the classroom observation analysis was in collaboration with the LAC Chief Economist’s Office, the full school day activities was hosted by LC3 HD; the technical assistance for the Upper Secondary Education reform included ECA and MENA HD staff; and the HD Anchor provided support to the dissemination of the tertiary education reports (Table 6). 28. Key partners included the OECD, the MEN, and the Secretary of Education of Antioquia. While the MEN was a client, it also was a partner, such as in hosting the dissemination events of the studies and managing the classroom time use data collection. Further, it prepared many of the analytical works that the Bank team reviewed. The Secretary of Education of Antioquia played a similar role. The OECD partnered on the two higher education studies in Phase I and in the dissemination events, translating, and publishing in Phase II (Table 6). Risk Factors and Mitigation Measures 29. Social Mobilization. In Colombia, social mobilization against government policy in education is not uncommon. The tertiary education reforms proposed in November 2011 by the Santos administration faced significant resistance from some student and faculty unions to the extent that the reform was pulled from Congress before it could be deliberated. Accordingly, there was a risk that knowledge work conducted under the PKS program could have been appropriated by interests groups or political parties in an effort to serve their own interests. This Page 23 of 31 risk was mitigated through (i) broad stakeholder engagement, (ii) team maintenance of close contact with management, EXT and key stakeholders; and (iii) careful management of dissemination events to focus on technical rather than political discussions. 30. Timing. The MEN is undertaking an ambitious reform agenda that was initiated two years into its four year term. There was a risk that there would not be sufficient time to develop the knowledge, plan the reform, and implement it before the next round of national elections. This risk was mitigated through (i) documenting knowledge outcomes to be shared with a possible future administration, (ii) focusing on key knowledge needs of the client as opposed to a broad range of knowledge products, and (iii) continued dialogue with sub-national Secretaries of Education, whose electoral cycle began in 2011 and lasts longer than that of the national government. Lessons Learned 31. The programmatic approach allowed for a long-term engagement that paralleled the timing of the MEN’s reform process. The client was on a 2.5 year timetable, not a Bank fiscal year schedule. The multi-year nature of the PKS allowed the Bank team to align with the client’s timeframe. When the client needed intense support, the Bank could quickly mobilize its resources and when the client needed time to work on its own, the Bank could pull back and be patient, not under pressures from internal short deadlines. Thus, a 2.5 year engagement was possible, with continuous budget and staffing to meet the client demands as they naturally arose through the process. 32. The close relationship between the client and the Bank kept the Bank engaged even when the MEN could have withdrawn in order to manage its own internal issues that threatened its reform efforts. This trust and partner-of choice status was achieved through a strong working relationship between the TTLs of each product – which was initiated before this PKS series-, constant collaboration and dialogue, and a consistent stream of high quality products. 33. A team of both field-based and HQ staff was a key to success. The field-based staff has a long-standing relationship with the clients, a deep knowledge of Colombia, an ear to the ground on recent developments that could affect our engagement, and an historical knowledge to guide the new work. They not only contributed technical inputs to the process, leading many of the tasks, they also were the Ambassadors for the whole project. The HQ-based staff contributed a deeper analysis of issues (the impact evaluation and classroom observation work) and a different skills mix with deep international expertise. International consultants also contributed to the latter. 34. A longer-term, client-driven agenda kept the PKS team focused on results. The MEN envisioned a three-step process, where the first step was analytical, the second was strategy development, and the third step was to operationalize the strategy. The MEN saw the Bank’s accompaniment along the three steps of the process, thus guiding the Bank on tasks to prepare at each step and tasks/teams to put in place for the next set of steps. Such a process also allowed Page 24 of 31 for the Bank to pull in international experts in anticipation of MEN requests or to prepare analytical work in preparation for later stages. 35. The PKS model allowed for flexibility in services provided to a client in need of technical assistance. Unlike a standard ESW, Phase II of the PKS was primarily technical assistance and convening services, which are more difficult to predict, and to commit to in a Concept Note, in terms of intensity, timing, and process, than studies are. Since the PKS allows for, and recognizes, these processes, the team was able to quickly adjust staff tasks and re-mobilize resources as required by the client’s own process. Further, the PKS model legitimized the convening and dialogue activities, thus providing resources for this “glue” that turned a series of products into an engagement with real results. 36. Finally, the flexibility in the instrument allowed the team to drop and substitute activities as the Colombia program shifted. Namely, when the Phase II Concept Note was prepared, it included STI activities, which were not a perfect fit. When the Innovation PKS was created, the activity could easily be shifted to a more relevant program. Or, when the Policy Note on tertiary education was no longer necessary, that activity could be easily substituted with an activity with higher value to the client. Future Work 37. Although the PKS series closes with this report, the experience points to areas of future work by the government, academia, or through future PKS. Some potential future areas of analysis include: • Classroom time use and learning outcomes. The studies supported by this PKS did not examine the learning outcomes of more or less teacher time spent on academic instruction. Current teaching methods, the quality of teaching, and an absence of pedagogical materials to support the learning process suggest that more academic instruction alone may not lead to the hoped for learning outcomes. Future work could use the SABER test scores, together with classroom observation time and use of materials to better understand if more academic time alone, more materials, a combination, or some other factor will lead to greater learning outcomes. • Understanding dropouts. The Secondary Education Strategy supported by this PKS includes actions to reduce the share of students who dropout of secondary school. Further analysis could examine how to retrofit those strategies to the specific needs of youth at risk of dropping out of school. For example, students who leave school due to school costs will require a different set of the new policies than students who dropout due to pregnancy. • ECD. The literature firmly establishes the role for ECD in reducing school dropouts, among a range of other negative youth behaviors. Further research could explore policy to bridge Colombia’s Upper Secondary and ECD strategies. • TVET. The PKS focus on upper secondary education did not allow for space to explore better the existing technical and vocational education (TVET) system. Building on a Bank report on the SENA, further research could examine (a) how to create a TVET Page 25 of 31 monitoring system, (b) programs for TVET instructors to participate in internships in the private sector, and (c) methods to strengthen the links between the private sector and TVET without introducing costs or negative incentives to the former Page 26 of 31 Annex I : Summary of PKS Activities, Phases I and II Phase I Completed Activities Phase II Completed Activities Component I: Strengthening the Efficiency of Education • Inside the Black Box of LAC’s Classrooms: • Classroom Observation Analysis – Using the Stallings Classroom Snapshot – analytical papers analyzing additional PPT presenting results of the Classroom aspects of the classroom observation data Observation studies in Colombia and in Colombia and in Antioquia and inputs to Antioquia a regional study • Dissemination of the Stallings Classroom • Full School Day Convening Activities – 1 Observation Study – 3 events regional convening activity Component II: Reforming the Upper Secondary Education System for Greater Quality and Relevance • Expert advice to support MEN’s • Assessing the impact of the Estatuto assessment of the Upper Secondary Docente on learning outcomes – Education System analytical paper and policy note • Debate on potential aspects of the reform – • Review of upper secondary education 3 workshops policy in Colombia – Review paper • Expert advice to support MEN’s assessment of the Upper Secondary Education System – 1 strategy workshop, 2 technical missions Component III: Strengthening Access to and Quality of the Tertiary Education System • Review of National Policies for • Dissemination of the comprehensive Education: Tertiary Education in OECD/WB review of the tertiary education Colombia – review paper system in Colombia - 1 large event, report • Higher Education in Regional and City translated into Spanish and printed Development: Antioquia, Colombia – • Dissemination of the comprehensive review paper OECD/WB review of the tertiary education • Innovation, Productivity and system in Antioquia – 2 events and report Employment: Evidence from translated to Spanish Colombian Manufacturing Firms – • Enhancing equity in tertiary education analytical paper through better measurement and targeting • International conferences addressing – technical support through ICETEX STI – 2 events project preparation. • Evaluation of the expansion of Ondas Program - review/strategy paper Page 27 of 31 ANNEX II: SUMMARY OF THE TABLES IN THE MAIN TEXT Activity Bank team Link to SOF/ Counter Delivered and Planned Quality Final Outcomes (ID) (Table 4) other budget part outputs/ dissemination Control (Section “Services, (Table 2) tasks (Table (Table 6) (Table 2) (Table 3) Products, and 4) Outcomes”) Classroom M. P12263 $30k MEN, Planned: Analytical Formal Peer Instrument is being used by Observati Laverde, B. 8 BB Secretary paper analyzing Review the MEN to evaluate the on Bruns P08290 of classroom observation process in the Rural Education Project. Analysis. 8 Educatio data context of the (P132235) P14535 n of regional Results incorporated into 3 Antioquia Actual: 3 outputs study. The the Secondary Education delivered - Paper study was Strategy analyzing Colombia data, reviewed by 5 Paper analyzing peer Antioquia data, Inputs to reviewers and LAC regional study the review meeting was The regional study chaired by the dissemination strategy Chief will be determined by the Economist. Regional Leadership The book is Team (RLT). The going through country and state studies an external will be disseminated review jointly with the regional process. study. The study results Colombia and have been informally Antioquia shared with the MEN. papers reviewed by B. Bruns (lead education economist) Full M. P05260 $5k BB MEN, Planned: 1 workshop, 1 n.a Following the conference, School Laverde, P. 8 Secretary SSKE with other country the MEN initiated a study of Day Holland P14535 of working on the issue the full-time school model in Convening 3 Educatio its Rural Education project. Activities. n, Actual: Clients attended The municipality of Medellin (P132235) Antioquia regional conference on has asked the Bank to full day schools provide technical assistance on an impact evaluation of Document of the World Bank their on-going full-time school pilot. Assessing R.B. Kattan P14535 $5k TF MEN Planned: Impact Study of Written As a result of the Bank’s the impact 3 $10k the Estatuto Docente on comments support, MEN’s upper of the BB learning outcomes provided by secondary education strategy is Estatuto Juan Baron closer to world class standards, Docente Actual: 2 outputs (Economist, drawing from global on delivered - Paper LCSHE) and experience brought to the learning presenting the evaluation Halsey discussion through these outcomes. results, Policy Note on Rogers (Lead activities. Elements (P132235) teacher reform Economist, considered in the reform HDNED) strategy include: reform of the Dissemination: Report structure of the upper and Policy Note to be secondary education system shared with the MEN and the curriculum to ensure that all students graduate from Expert M. P14535 $55k MEN Planned: Expert advice n.a secondary school with advice to Laverde, A. 3 BB to the Ministry of fundamental competencies support Rodriguez, Education (generic, academic and MEN’s J.M. exploratory); eliminating the Upper Moreno, J. Actual: 1 workshop, 2 possibility of pursuing pure Secondary D. Alonso missions, and 2 policy technical programs offered by Education guidance papers the National Training Service Strategy produced (SENA), instead allowing for (P132235) additional technical training Review of M. Laverde P14535 $14k MEN Planned: Review of Written after the standard school day or upper 3 BB Upper Secondary comments after completing secondary secondary Education Policy in provided by education (currently, grade education Colombia Miguel 11); adding an additional grade policy in Szekley to the education cycle; Colombia. Actual: Paper produced, (Director, professionalizing the (P132235) and expanded to include Instituto secondary teacher education international lessons Tecnológico y track and focusing initial and de Estudios continual training on teaching Dissemination: Paper Superiores de for learning outcomes; and shared with MEN during Monterrey), reforming curriculum and preparation Ernesto testing to focus on academic Cuadra (Lead and higher order cognitive Education skills (as opposed to testing for Specialist), content) and socio-emotional Page 29 of 31 Victor Manuel skills. As proposed by the Moncayo technical team, the MEN (Universidad formed an inter-sectoral Nacional) committee to guide the reform process, which includes representatives from sub-national Secretaries of Education, employers, and the tertiary education. The close collaboration with the client on these activities allowed for the Bank to leverage the client’s technical knowledge and increased the clients’ technical ability to assess and design education strategy. Dissemina F. P14535 $30k Academi Planned: 3 events n.a. The discussion has tion of the Marmolejo, 3 counter a, think highlighted the difficulty that OECD/WB M. P14578 -part tanks, Actual: One large event students from lower review of Laverde, A. 2 funds MEN, with more than 1000 socio-economic the tertiary Gonzalez (event), Secretari participants. Book was backgrounds have in education partner es of also translated and transitioning to secondary system in funds Educatio printed in Spanish. The school, leading to a request Colombia (printin n book was disseminated from the Vice-Minister of (P132235) g and to more than 2000 Higher Education for Bank translati stakeholder. support in implementing on) programs to facilitate the transition. Dissemina M. Laverde P14535 $20k Antioquia Planned: 2 events n.a. The report results have tion of the 3 counter Sec. of been used by the Secretary OECD/WB P05260 -part Education, Actual: 2 events. of Education of Medellin in review of 8 funds Universid Approximately 1000 the strategy to regionalize the tertiary (event), ad de books (English) were tertiary education. This education partner Antioquia, disseminated and a includes the development of system in funds Universid Spanish translation was branch universities at the Antioquia. (printin ad EAFIT prepared municipal level. (132235) g and translati Page 30 of 31 on), BB Enhancing M. P14578 $10k ICETEX, Planned: 3 technical The project As a results of this technical equity in Laverde, 2 BB MEN workshops. peer assistance, the algorithm tertiary M. reviewers: used by ICETEX to allocate education Becerra, O. Actual: Just in time Andreas Blom scholarships has been through Medina, J. advice provided to (Lead changed to better target better Entwistle ICETEX to improve their Education economically disadvantaged measurem poverty targeting Economist), students, which is expected ent and mechanism Maria Paulina to increase the share of targeting Mogollon ACCES beneficiaries from (P132235) Dissemination: (FPD disadvantaged background Implemented into the Specialist), by 5 percentage points, ICETEX project that is Francisco equivalent to an additional under preparation Marmolejo 2500 students annually (Lead (from a base of 10,500 Education students annually). The Specialist) client has greater capacity to assess options for future modifications to the algorithm. n.a. are those technical assistance and convening tasks that are not subject to a quality assurance process. Page 31 of 31