Document of The World Bank Report No: ICR00003070 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-72840) ON A LOAN 7284-CR IN THE AMOUNT OF US$30 MILLION TO THE REPUBLIC OF COSTA RICA EQUITY AND EFFICIENCY OF EDUCATION PROJECT June 27 , 2014 Education Sector Central America Country Management Unit Latin America and the Caribbean Region CURRENCY EQUIVALENTS (Exchange Rate Effective June 27, 2004) Currency Unit = Costa Rican Colones (CRC) US$ 1.00 = CRC 436.84 FISCAL YEAR ABBREVIATIONS AND ACRONYMS CAFTA Central American Free Trade Agreement CAS Country Assistance Strategy CGR Contraloría General de la República CR Costa Rica CSE Higher Education Council DEI Department for Intercultural Education DIEE Dirección de Infraestructura y Equipamiento Escolar DO Development Objective DPE Directorate of Primary Education DRE Regional Directorate for Education EDUCO El Salvador's Community-Managed Schools Programme (EDUCO: Educación con Participación de la Comunidad) ESMF Environmental and Social Management Framework FONABE Fondo Nacional de Becas IBRD International Bank for Reconstruction and Development ICR Implementation Completion Report ICT Information and Communication Technologies IDB The Inter-American Development Bank ILI Intensive Learning ICR ILO International Labor Organization IMAS Instituto Mixto de Asistencia Social IP Implementation Progress IPPF Indigenous Peoples Planning Framework IRE Índice de Rezago Educativo MEP Ministry of Public Education PAS Policy and Activity Schedule PCU Project Control Unit PDO Project Development Objective PIAD Programa de Informatización para el Alto Desempeño POA Project’s Annual Operational Plan PROMECE Programa de Mejoramiento de la Calidad de la Educación QAG Quality Assurance Group QEA Quality at Entry QSA Quality at Supervision SETENA Secretaría Técnica Nacional Ambiental SIDE Sistema de Información de Desarrollo Educativo SIGAF Sistema Integrado de Gestión de Administración Financiera SINAC Sistema Nacional de Areas de Conservación SIPO Sistema de Información de la Población Objetivo UNA Universidad Nacional de Costa Rica Vice President: Jorge Familiar Acting Country Director: Maryanne Sharp Sector Manager: Reema Nayar Project Team Leader: Marcelo Becerra ICR Team Leader: Marcelo Becerra COSTA RICA EQUITY AND EFFICIENCY OF EDUCATION PROJECT CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 6 3. Assessment of outcomes ........................................................................................... 14 4. Assessment of Risk to Development Outcome ......................................................... 22 5. Assessment of Bank and Borrower Performance ..................................................... 23 6. Lessons Learned........................................................................................................ 25 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners........... 26 Annex 1. Project Costs and Financing .............................................................................. 27 Annex 2. Outputs by Component...................................................................................... 28 Annex 3. Economic and Financial Analysis ..................................................................... 38 Annex 4. Bank Lending and Implementation Support/Supervision Processes................. 41 Annex 5. Beneficiary Survey Results ............................................................................... 43 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ......................... 45 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ........................... 60 Annex 9. List of Supporting Documents .......................................................................... 61 COSTA RICA EQUITY AND EFFICIENCY OF EDUCATION PROJECT Data Sheet A. Basic Information CR Equity and Country: Costa Rica Project Name: Efficiency of Education Project ID: P057857 L/C/TF Number(s): IBRD-72840 ICR Date: 06/27/2014 ICR Type: Core ICR Lending Instrument: SIL Borrower: Republic of Costa Rica Original Total USD 30.00M Disbursed Amount: USD 24.81M Commitment: Revised Amount: USD 30.00M Environmental Category: B Implementing Agencies: Ministry of Public Education Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 06/12/2003 Effectiveness: 12/21/2006 12/21/2006 08/23/2007 09/03/2008 Appraisal: 02/24/2004 Restructuring(s): 06/29/2011 09/25/2012 Approval: 03/29/2005 Mid-term Review: 12/06/2010 Closing: 06/30/2011 12/31/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Moderately Quality at Entry: Government: Satisfactory Unsatisfactory i Implementing Moderately Quality of Supervision: Satisfactory Agency/Agencies: Unsatisfactory Overall Bank Overall Borrower Moderately Satisfactory Moderately Satisfactory Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry Yes Satisfactory at any time (Yes/No): (QEA): Problem Project at any Quality of Yes None time (Yes/No): Supervision (QSA): DO rating before Moderately Closing/Inactive status: Satisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 20 20 Primary education 50 50 Secondary education 30 30 Theme Code (as % of total Bank financing) Education for all 67 67 Education for the knowledge economy 33 33 E. Bank Staff Positions At ICR At Approval Vice President: Jorge Familiar David de Ferranti Country Director: Maryanne Sharp Jane Armitage Sector Manager: Reema Nayar Eduardo Velez Bustillo Project Team Leader: Marcelo Becerra Joel E. Reyes ICR Team Leader: Marcelo Becerra ICR Primary Author: William Experton ii F. Results Framework Analysis Project Development Objectives (from the loan agreement) The objectives of the Project are to: (a) reduce education quality gaps in the Borrower’s primary and secondary education system in rural areas; and (b) improve the equity and efficiency in the allocation, administration and use of the Borrower’s education sector resources. Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Indicator Reducing Quality Gaps; Reduce education quality gaps in targeted areas PDO 1 : (percentage of students that pass the bachillerato-exams). NATIONAL LEVEL Value 62.7% 70% 69.8% (Quantitative or Qualitative) TARGETED AREAS Value 56.5% 60.3 70% (Quantitative or (14.1%) (40.5%) Qualitative) Date achieved 12/31/2008 12/31/2013 12/31/2013 6.2 9.3 GAP 0 (48.6) (29.3) Achieved. At restructuring, a calculation error was committed: the numbers for the baseline value (56.5%) and actual value (60.3%) included in the results framework and monitoring were those of overall rural areas instead of the targeted area. This error was corrected during the ICR mission. With the actual baseline value for the targeted areas of 14.1% and an actual value of 40.5%, the Comments gap between the national level and the targeted areas was reduced from 48.6 to (incl. % 29.3 percentage points. The greater progression of the percentage of students achievement) who passed the “bachillerato” (exam at the completion of secondary education) in the targeted area was due to the investments—infrastructure, curriculum and training—carried out by the project, which provided access to four years of quality secondary education (Liceos rurales) where this opportunity did not exist previously. Improving Equity: Reduce gaps in graduation rates in targeted areas Indicator (graduation rate measured as the ratio between 11th grade enrollment and 7th PDO 2 : grade enrollment). NATIONAL LEVEL : Value 36.6% 41.5% 43.6% (Quantitative or Qualitative) iii TARGETED AREAS Value 22% 27.5% 34.4% (Quantitative or Qualitative) Date achieved 12/31/2008 12/31/2013 12/31/2013 GAP 14.6 14 9.2 Surpassed. This indicator represents the proportion of the students who had the opportunity to access and complete secondary education (11th grade) from those who completed primary education (7th grade). The progression had been greater Comments in the targeted area and the primary-secondary completion gap substantially (incl. % reduced from 14.6 to 9.2%, along with an improved investment realized in the achievement) targeted areas for the provision of quality high schools. During the ICR mission, the baseline value for the targeted area was corrected to reflect 34% at national level and 20.6% in the targeted area. Indicator Improving Efficiency: Reduce administrative costs per beneficiary of MEP PDO 3 : Equity and Transfers Programs (scholarships, transportation, food). Value 10,855 6,574 7,013 (Quantitative or CR colones CR colones CR colones Qualitative) Date achieved 12/31/2008 12/31/2013 12/31/2013 94% achieved This indicator is calculated by dividing the administrative cost Comments (mostly salaries of the staff in charge in the Ministry of Public Education, MEP) (incl. % of managing the equity and transfers programs by the number of beneficiaries. achievement) The number of beneficiaries substantially increased while the number of staff remained stable. (b) Intermediate Outcome Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Values Completion or documents) Target Years Number of classrooms built and renovated, and provided with furniture and Indicator 1 : equipment, in the School Networks. Value 0 186 236 Date achieved 12/31/2008 12/31/2013 12/31/2013 Surpassed. 236 class rooms (of which 72% were new and 28% rehabilitated) Comments were constructed and furnished. The schools are fully functional except for the (incl. % computer laboratories which did not receive computers. The MEP plans to achievement) provide new computers in 2014. Number of school building facilities constructed and provided with equipment, Indicator 2 : including kitchens, teachers lounges, student dormitories and restrooms in the School Networks. Value 0 71 81 Date achieved 12/31/2008 12/31/2013 12/31/2013 Comments Surpassed. 81 schools were built and furnished with facilities including iv (incl. % kitchens, teachers’ lounges, student dormitories and restrooms. In some cases achievement) they are equipped with solar panels. The schools are fully functional except for the computer laboratories which did not receive computers. The MEP plans to provide new computers in 2014. Number of Common School Network (Liceos rurales/rural high schools) Indicator 3 : facilities constructed and equipped (technology centers; cultural, arts and physical education facilities, meeting rooms, rooms for visiting professors). Value 0 15 14 Date achieved 12/31/2008 12/31/2013 12/31/2013 Partially achieved. 14 “liceos rurales” were built and furnished, allowing them Comments to be fully functional. Each liceo rural is equipped with common school network (incl. % facilities (rooms for visiting professors, cultural centers, physical education achievement) facilities, meeting rooms). Only rooms for art and the computer laboratories did not receive equipment. The MEP plans to provide equipment in 2014. Percentage of schools and “liceos rurales” equipped with an education sector Indicator 4 : information system in the 15 Administrative Circuits targeted by the Project. Value 0 100% 15.47% Date achieved 12/31/2008 12/31/2013 12/31/2013 Comments Not achieved: Purchase and distribution of equipment is pending and is (incl. % expected in 2014. achievement) Number of staff, in MEP Technical Units that administer the Equity and Indicator 5 : Transfers Programs that have trained to improve supervision and evaluation of these programs. Value 0 600 80 Date achieved 12/31/2008 12/31/2013 12/31/2013 Not achieved: The 80 staff of the MEP technical unit in charge of the equity and Comments transfers programs were trained. However, personnel of the MEP at the regional (incl. % level in the DRE were not trained as well as the school council members that do achievement) not belong to the MEP. Number of beneficiaries of the MEP’s Equity and Transfers Programs at national Indicator 6 a) : level a) SCHOLARSHIP Value 68,453 100,299 164,311 Date achieved 12/31/2008 12/31/2013 12/31/2013 Comments Surpassed (incl. % Target is not cumulative achievement) Number of beneficiaries of the MEP’s Equity and Transfers Programs at Indicator 6 b): national level b) TRANSPORTATION Value 70,075 93,423 102,623 Date achieved 12/31/2008 12/31/2013 12/31/2013 v Comments Surpassed (incl. % Target is not cumulative achievement) Number of beneficiaries of the MEP’s Equity and Transfers Programs at national Indicator 6 c) : level c) FOOD PROGRAM Value 582,708 656,000 673,129 Date achieved 12/31/2008 12/31/2013 12/31/2013 Comments Surpassed (incl. % Target is not cumulative achievement) Value of transfers of funds through the Equity and Transfers Programs to Indicator 7 a) : beneficiaries and School Councils--A) Transfer to Schools Councils (include maintenance and Equity Programs of transportation and Food Program). Value 31.611 million CR 90.918 million 81.076 million CR colones CR colones colones Date achieved 12/31/2008 12/31/2013 12/31/2013 Comments (incl. % 89% achieved achievement) Value of transfers of funds through the Equity and Transfers Programs to Indicator 7 b) : beneficiaries and School Councils--B) Scholarships Value 38.883 million 28.320 million CR 6.861 million CR colones CR colones colones Date achieved 12/31/2008 12/31/2013 12/31/2013 Comments (incl. % 73% achieved achievement) Indicator 8 : Number of staff that have been trained under MEP institutional reform program. Value 0 5, 170 632 Date achieved 12/31/2008 12/31/2013 12/31/2013 12.2% achieved: As part of the strengthening of the DRE of Sula, three one- Comments day workshops were organized for a total of 320 participants dedicated to (incl. % services to the client (parents and students) and communication in a situation of achievement) crisis, as well as a one-day training workshop for 312 staff of the DRE on communication strategies. Indicator 9 : Number of studies and workshops financed by Component 2. Value 0 4 6 Date achieved 12/31/2008 12/31/2013 12/31/2013 Surpassed: The studies are the following: (i) Effects of the equity programs; (ii) Cost of the equity programs; (iii) Institutional analysis, update and Comments implementation of work processes; (iv) Management support; (v) Development (incl. % of a management model for scholarships, food programs, and student achievement) transportation; and (vi) Reorganization of the DPEQ (Equity Programs Department). vi Percentage of national advisers and regional trainers trained in intercultural Indicator 10 : education. Value 0 179 451 Date achieved 12/31/2008 12/31/2013 12/31/2013 Surpassed: 76 national advisors and 375 regional advisors/trainers (of a total 711) were trained and certified in a 60-hour training about “Contextualized teaching practices in an intercultural vision.” Additional workshops in Comments intercultural education were organized gathering teachers, advisors students, (incl. % members and representatives of the indigenous communities. An error was achievement) committed in the original drafting of the results framework: the target value expressed was estimated as the absolute number of trained advisors instead of as the percentage. Indicator 11 : Completion of an assessment of the needs and profile of rural teachers. In 2013 a participatory Value diagnostic was End of (quantitative None realized by the evaluation or Qualitative) Department of Education of the UNA Date achieved 12/31/2008 12/31/2013 12/31/2013 Partially achieved: In 2013, a participatory diagnostic was implemented by the DEI, in collaboration with the Department of Education of the UNA Comments (Universidad Nacional), to determine the content of a comprehensive training (incl. % program for secondary education teachers. The UNA is expected to complete the achievement) full proposal of the training curriculum for rural secondary education teachers by the end of 2014. Percentage of rural secondary teachers trained in the Liceos rurales modality and Indicator 12 : intercultural education. Value 0 100% 13.5% Date achieved 12/31/2008 12/31/2013 12/31/2013 Comments 13.5% achieved: 1,697 secondary education teachers have been trained out of (incl. % 12,854 total in intercultural education and the curricular modality used in the achievement) Liceos rurales. Indicator 13 : Number of education materials produced with an intercultural perspective. Value 0 9 11 Date achieved 12/31/2008 12/31/2013 12/31/2013 Achieved: The following education materials with an intercultural perspective have been produced and disseminated as a result of the Project: (i) Strategy and methodology for the implementation of intercultural education within the Comments curriculum; (ii) three training modules focused on intercultural education; (iii) (incl. % seven mini encyclopedias of indigenous territories; (iv) 4 manuals to teach achievement) English in an indigenous context; (v) video documentary of the battle of Sardinal and Trinidad; (vi) Booklets of pedagogical activities; (vii) didactic literature; (viii) calendars in an indigenous context; (ix) books, legends and historic vii traditions of the Cabecares; (x) planning studies (Mathematics, Spanish, Social sciences, Social studies in an indigenous context); and (xi) Strategy for revitalizing the Boruca and Terraba languages. In addition, students of the DRE Los Santos developed course materials. Development of a standard methodology to analyze student assessment and Indicator 14 : complementary training modules. Completion of Value the student (quantitative Not done learning Not done or Qualitative) assessment analysis Date achieved 12/31/2008 12/31/2013 12/31/2013 Comments (incl. % Not achieved achievement) Percentage of MEP staff and teachers trained on the standard methodology to Indicator 15 : analyze student learning assessments. Value 0 100 0 Date achieved 12/31/2008 12/31/2013 12/31/2013 Comments (incl. % Not achieved achievement) G. Ratings of Project Performance in ISRs Actual Date ISR Disbursement No. DO IP Archived s (USD million) 1 04/27/2005 Satisfactory Satisfactory 0.00 2 04/29/2005 Satisfactory Satisfactory 0.00 3 12/22/2005 Satisfactory Satisfactory 0.00 4 05/18/2006 Moderately Unsatisfactory Unsatisfactory 0.00 5 12/27/2006 Moderately Satisfactory Unsatisfactory 0.00 6 05/12/2007 Moderately Satisfactory Moderately Unsatisfactory 0.15 7 11/13/2007 Moderately Satisfactory Moderately Unsatisfactory 0.15 8 11/18/2007 Moderately Satisfactory Moderately Unsatisfactory 0.15 9 05/09/2008 Moderately Satisfactory Moderately Satisfactory 0.45 10 12/05/2008 Moderately Satisfactory Moderately Satisfactory 0.45 11 05/16/2009 Moderately Unsatisfactory Unsatisfactory 0.92 12 12/13/2009 Moderately Unsatisfactory Moderately Unsatisfactory 2.54 13 06/28/2010 Moderately Unsatisfactory Moderately Unsatisfactory 3.35 14 02/23/2011 Moderately Unsatisfactory Moderately Unsatisfactory 4.62 15 08/22/2011 Moderately Unsatisfactory Moderately Unsatisfactory 10.00 16 04/18/2012 Moderately Unsatisfactory Moderately Unsatisfactory 15.55 viii 17 10/03/2012 Moderately Unsatisfactory Moderately Satisfactory 16.80 18 11/06/2012 Moderately Satisfactory Moderately Satisfactory 16.80 19 05/27/2013 Moderately Satisfactory Moderately Satisfactory 19.83 20 12/09/2013 Moderately Satisfactory Moderately Satisfactory 24.81 21 01/08/2014 Moderately Unsatisfactory Moderately Unsatisfactory 24.28 ix H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD million (a) Introduce the May 2004 Procurement Guidelines and Consultant Guidelines; (b) include a new procurement method 08/23/2007 MS U 0.15 (Community Participation); and (c) change the disbursement percentage of Category 5 (Training, other than for Subprojects) from 90% to 100%. Increase authorized allocation to US$ 3 million and the disbursement percentages to 09/03/2008 N MS MS 0.45 100% in Categories (1), (2) and (3) of the disbursement table (Section A.1, Schedule 1 of the Loan Agreement). a) Revise Project components to align activities with current Government strategic priorities, including the elimination of some Project activities which overlapped with other activities and reconfigure more logically similar activities. b) Define the areas targeted by the Project. 06/29/2011 MU MU 16.24 c) Change the Results Framework in order to reflect the proposed changes to the components, and introduce indicators that better reflect Project objectives and outputs, modifications in the components, and are measurable through Ministry of Public Education (MEP) databases and include baselines. x ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD million d) Change the Project Coordination Unit (PCU) to align it with broader changes in MEP and to improve Project management and coordination. e) Update environmental assessment template and guidelines to ensure that resettlement impacts are screened out per OP/BP 4.12 and update the Indigenous Peoples Planning Framework (IPPF) to document the Project’s approach and work with Indigenous Peoples. f) Change to the categories of expenditures that are financed out of loan proceeds so that the categories are aligned with the revised Project components, and changes to 100 percent expenditures that are financed out of loan proceeds. g) Extend 18 months to December 31, 2012 to ensure sufficient time for revised targets to be met. Extended the closing date for 12 09/25/2012 MU MU 16.80 months from December 31, 2012 until December 31, 2013. xi I. Disbursement Profile xii 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. In 2003, the Ministry of Public Education (MEP) launched a Plan called "Relanzamiento de la Educación Costarricense" (Revamping Costa Rican Education) with the goals to: (i) close the rural-urban education attainment gaps which continue even at the primary education level; (ii) increase and sustain the participation of students from low-income families in the education sector; and (iii) improve secondary and technical education. To ensure equity, regions with low education outcomes as well as students and families from the lowest income quintiles were especially targeted, most of which were rural. Given the low secondary education indicators (gross coverage of 65.6% in 2002 with a completion rate of only 30%), the Government proposed to continue investing strongly at this level nationwide. These goals were consistent with the objectives of the CAS dated April 27, 2004. While noting important improvements in the social sectors, the CAS supported social development sustainability and, more specifically, closing the gaps that remained across regions and social groups with regards to learning quality, coverage and graduation rates in education. Despite undeniable progress, particularly to universalize primary education completion, Costa Rica faced several issues. First, a quality gap measured by low graduation rates at the primary level still subsisted between urban and rural areas and, more specifically, in the indigenous territories. At the secondary level, the gap was much worse (62.7% passed the final secondary exam or “Bachillerato” at national level, compared to 56% in rural area and only 14.1% in indigenous territories) due to a problem of access with a lack of facilities, combined with an issue of low quality and inadequate modalities of education (telesecundaria). 1 In addition, the equity programs including scholarships, transportation, and food needed to be better targeted to induce a greater education demand among the poorest families. 1.2 Original Project Development Objectives (PDO) and Key Indicators 2. The objectives of the Project are to: (a) reduce education quality gaps in the Borrower’s primary and secondary education system in rural areas; and (b) improve the 1 Telesecundaria was conceived as a low cost secondary education modality through distance education via TV for students in rural areas. The main limitation of this alternative method was the narrow coverage, proposed to serve only the 8th and 9th grades, when four years of education (10th and 11th grade) were necessary to prepare for the bachillerato. In addition, the 2006 assessment done by the MEP showed that “telesecundaria” schools: (i) lack of supervision from the MEP; (ii) had non-functional and deteriorated facilities, some with no electricity or connection for the TV set; (iii) received limited support of books and didactic materials; and (iv) had less than three teachers per school, most of whom lacked the right skills and profile for secondary education. 1 equity and efficiency in the allocation, administration and use of the Borrower’s education sector resources. 2 3. Key indicators for rating Project outcomes were: a) Reduce internal education efficiency gaps in primary education in the targeted macro-regions of the Project (composed of municipalities with low education indicators and indigenous and afro-descendent populations). The reduction of the educational regional gaps was measured by the following education efficiency indicators: (i) average percentage of overage students and (ii) dropout rates. b) Improved efficiency of non-traditional secondary education rural modalities (“telesecundaria”) and increased access in targeted rural areas. c) Increase equity of demand-based education programs and increase access of beneficiaries (including indigenous and afro-descendent populations) from the lowest income quintiles (quintile 1 and 2) in the targeted macro-regions. d) Improved cooperation among rural schools measured by the conformation of at least 60 Rural School Collaborative Networks and improved shared utilization of key education quality inputs within the Collaborative Network, mainly: (i) infrastructure; (ii) information and technology centers; and (iii) allocation of specialized teachers (second language, culture and values, physical education, etc.) Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 4. The PDO remained unchanged. The restructuring was done under the constraint that the PDO could not be changed since it would have implied Congressional approval, along with the risk that the delay for approval would be too long and that the Project would have to be closed. However, the 2011 restructuring made substantial changes to the Project: (i) the four original PDO indicators were revised and replaced with 3 new PDO indicators; and (ii) the targeted areas were defined as the 9 rural administrative territories (which span 10 indigenous territories out of 24 total in Costa Rica). This restructuring was carried out: (i) to better reflect the PDO and allow for the measurement of outcomes through the MEP database; and (ii) to identify the areas of Project activity to facilitate its implementation. a) Reducing Quality Gaps: Reduce education quality gaps in targeted areas (percentage of students that pass “bachillerato” exams). b) Improving Equity: Reduce gaps in the secondary graduation rate in targeted areas (graduation rate measured as the ratio between 11th grade enrollment and 7th grade enrollment). 2 The PDOs are from the loan agreement. 2 c) Improving Efficiency: Reduce administrative costs per beneficiary of MEP’s Equity Programs (scholarships, transportation, and food). 1.3 Main Beneficiaries 5. The Project was originally intended to serve in the Municipalities, communities and schools with the lowest education indicators which include the following direct beneficiaries in rural, indigenous/afro-descendent and low income communities: (i) 25,000 students from preschool to grade 6 in dispersed rural communities (mostly in multigrade schools); (ii) 6,000 students grade 7 to 11 (mostly in “telesecundaria” schools) in dispersed rural communities; (iii) 200,000 students in primary education schools; and (iv) 1,000 students in secondary schools. And the other beneficiaries included the technical units of the MEP, the Regional and District Offices, institutions implementing education demand–side programs and community-based school councils. 6. When the 2011 restructuring redefined the areas of project activity, the beneficiaries changed to: (i) students in primary education schools and secondary schools in 9 rural Administrative circuits, all of whom directly benefited from activities under Component 1 and measured as per PDOs indictor 2; (ii) the beneficiaries of the equity programs (as per shown in intermediate indicator 6, Component 2); (ii) the teachers assigned to rural areas; and (iii) the staff of the MEP’s technical units at central and regional level. 1.4 Original Components (as approved) 7. The Project consisted of three components: Component 1: The Quality and Equity of Rural Education Component aimed at financing the implementation of Annual Operational Subprojects (POA Subprojects) to improve rural education attainment (Rural Education Quality and Equity Subprojects) and institutional development (Institutional Development Subprojects). To orient the preparation of the POA Subprojects, a menu of strategies and expected results to increase the level, quality and equity of basic (up to 9th grade) rural education have been included in a Policy Activity Schedule, PAS. Component 2: Equity of Education Services. This component aimed at increasing the capacity of the MEP to reduce equity gaps by supporting activities to identify, reach and monitor the delivery of pertinent supply and demand education services to regions with low education indicators and students from low income households by (i) raising the capacity of various MEP units managing the MEP’s demand-based equity programs: scholarships, vouchers, transportation and school meals; and (ii) developing and implementing an integrated information system that can track the diverse education outcomes across municipalities, communities and schools (Sistema de Información de Desarrollo Educativo SIDE). 3 Component 3: Improved Institutional Efficiency. This component aimed at improving the institutional capacity of the MEP through inter-departmental integration strategies and working alliances across central, regional and school organizations. The component was expected: (i) to strengthen the capacity of the MEP’s staff (both of pedagogical and administration units) to conduct participatory sector diagnosis, plan strategically, and implement, monitor and evaluate education programs; (ii) to improve the efficiency of education services provided in the rural sector, by the integration and strengthening of the MEP units leading such services; and (iii) to increase capacity of the Project Coordination Unit for coordination, fiduciary, monitoring and evaluation activities of externally financed projects , which in turn should support the Technical Units of the MEP. 1.5 Revised Components 8. When it was clear that the Project with its original design could not progress and needed to be adapted to the Government’s policy, the Bank initiated the restructuring of the Project in 2009. However, the Bank came to an agreement with the Government to proceed with the formal restructuring on 06/29/2011 for the following reasons: (i) to finalize the operation manual, the environmental framework and the indigenous people plan; (ii) to evaluate at mid-term some activities, such as the information systems; and (iii) to proceed first with the institutional changes to resolve the conflict between PROMECE and the MEP. 9. To better align the activities with Government’s priorities and policy changes 3 , to clarify and simplify the content of each component, to focus the Component 1 infrastructure in targeted areas and to regroup more logically similar activities, the revised project consisted of three components: Component 1: Improving access to quality education through: (i) the renovation of existing classrooms and the construction of new classrooms, including the provision of furniture and equipment such as computers and software: (ii) School building facilities, such as construction and provision of furniture, and equipment for kitchens, teachers lounges, student dormitories, and restrooms in schools belonging to rural School Networks; (iii) the construction of common school network facilities, such as technology centers and cultural, arts and physical education facilities, including the provision of equipment as well as energy and connectivity facilities; and (iv) the provision of furniture and equipment for the Regional Indigenous Education Directorate of Sula. Component 2: Improving MEP’s Institutional Efficiency through: (i) the development and implementation of an education sector information system at the school level; (ii) the support for MEP’s Implementing Technical Units that manage the Equity Programs and the Transfers Program to improve planning and monitoring of such Programs; (iii) the 3 The policy changes are described in par 27 and 28. 4 support of implementation of MEP’s institutional reform; and (iv) the management of the implementation of the Project. Component 3: Strengthening the quality of primary and secondary education through: (i) the development and implementation of a training program in intercultural education for MEP staff at the central and regional level; (ii) a rural education quality improvement program adapted to the local and cultural context, including the development of a strategy to transform Telesecundarias to Liceos Rurales, the development of rural teaching methods that use ICTs, the provision of computer equipment and professional development programs for teachers assigned to rural areas; (iii) improvements in MEP staff and teacher capacity to analyze student learning assessments. 1.6 Other significant changes 10. An amendment to the Loan agreement was approved on August 23, 2007 to: (i) introduce the May 2004 Procurement and Consultant Guidelines; (ii) include a new procurement method (Community Participation); and (iii) change the disbursement percentage of category 5 from 90% to 100%. 11. A second amendment to the Loan agreement was approved on September 3, 2008 to increase the authorized allocation to US$3 million and the disbursement percentages to 100% in categories 1), 2) and 3). Permit taxes will be also be considered as eligible expenditures. 12. The June 29, 2011 restructuring: (i) reorganized project activities; (ii) redefined the targeted areas of project activities, discontinuing the four initial macro regions and focusing the construction on 10 indigenous territories; (ii) aligned the outcome indicators with the PDO and adjusted the monitoring indicators to the changes in project activities; (iii) reallocated the Loan proceeds among different categories; (iv) reflected changes in the disbursement percentages in all categories to 100 percent; (v) clarified the responsibilities between PROMECE, the MEP’s Office and MEP’s Technical Units; and (vi) extended the Project closing dates to December 31, 2012. 13. The September 25, 2012 restructuring extended the Project for 12 months (from December 31, 2012 until December 31, 2013). The extension was necessary to finance: (i) the construction of 34 schools and the full furnishing and equipping of all schools; and (ii) some key technical assistance activities related to institutional and curricular reform. 5 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry Strengths 14. The Project built on strong analytical work regarding: (i) social sector financing (Costa Rica: Social Spending and the Poor 2003); (ii) education, productive technology and competitiveness, with a focus on secondary education (Closing the Gap in Education and Technology); and (iii) economic analysis of the education sector (Diagnóstico Económico de la Educación en Costa Rica). This analytical work disaggregated data by region/poverty, included ethnic variable for schools, and needs adapted to the different school situations. This approach raised the Government’s attention about uneven education performance across regions and income quintiles and was particularly appropriate to address equity gaps and improve efficiency. 15. The Project was targeted to serve rural, indigenous/afro-descendants and low income and socially disadvantaged communities that had the lowest education indicators in terms of years of schooling and literacy rates. It engaged the borrower’s attention and commitment: (i) to address equity issues and close education gaps; (ii) to increase local participation and promote the role of civil society organizations; and (iii) to pioneer innovative approaches to improving educational efficiency. 16. Lessons from projects in Central 4 and South America dealing with education for rural dispersed communities were reflected in the Project design. In particular, those related to: (i) community participation and decentralization of school resources to school councils; (ii) well-managed multi-grade classrooms to improve student achievement; (iii) a flexible curriculum, tailored to individual needs of students; (iv) improved school organization, planning and management of resources; and (v) institutionalizing community participation within the legal and financial framework of public education system management. 17. The IDB and the World Bank formed a strategic alliance to coordinate efforts, wherein they determined that the Bank’s support for rural education would complement IDB’s support to secondary education nationwide. 18. The high counterpart funding of 40% indicated the strong commitment of the Costa Rican Government to the Project objectives. Weaknesses 19. The Project was too complex and lacked the clarity to be fully understood by those who had to implement it. First, because of the change in Government 4 In particular, the flagship project EDUCO in Salvador. 6 administrations, the MEP team who implemented the Project was different from the team who prepared it. Second, Component 1 of the Project included a Policy and Activity Schedule (PAS), which proposed a comprehensive and potentially effective menu of areas, objectives and results for the preparation of POA sub-projects. However, choices of activities and investments were vaguely defined and left to the discretion of local communities. Third, there was overlap in some activities among components causing confusion around the responsibility for implementation. Finally, the Project’s results framework had an unnecessary high level of details which added to the overall confusion. 20. The scope of the Project was too large in attempting to achieve three objectives: closing the education quality gap of the indigenous group and lower income students in rural areas, increasing efficiency of the education system and the efficiency in the equity of resource allocation in rural education. The Project was too ambitious in its expectation that it could change the organization and institutional culture of the MEP. For instance: strengthening the capacity of the DIEE (Direction of Infrastructure) to provide necessary technical assistance to the school councils to build school infrastructure. 21. Despite the wealth of information available, the selection of the targeted areas and schools to be financed by the Project was not accomplished during the Project’s preparation. Moreover, there was no clarity on the selection process. It was not clear if the 20-24 municipalities with the lowest education indicators would be selected within the targeted Macro-Regions (Norte, Atlántico, Puntarenas and Guanacaste) for the Project’s investment. To carry out the targeting, the MEP was supposed to develop the IRE (Índice de Rezago Educativo) which combined: (i) socio- economic indicators; (ii) provision of quality inputs by the MEP (teachers, technology centers and infrastructure); and (iii) participation of equity demands programs to correlate the allocation of education inputs and socio-economic gaps. The IRE should have been provided by the general information system SIDE (Sistema de Information del Desarrollo Educativo) to be developed within the Project, but was considered inconsistent with the sequencing of the investment. The institutional set-up and the complex decision-making process in Costa Rica are largely the root cause of the aforementioned issues. 22. The Project Implementation Arrangements and the responsibilities were insufficiently defined. The reliance upon coordinating mechanisms–Comité Superior Consultivo and Comité de Unidades Ejecutoras—neither guaranteed an efficient decision-making process nor secured the integration of the technical units, including common objectives and implementation strategies. Due to the insufficient definition of implementation arrangements and the complexity of the Project, the design of an Operations Manual was still not completed in 2008. The Project investments definition (about 60% of the Project) relied almost entirely on the participation of local communities and empowered the regional technical units of the MEP. This deliberate choice of the preparation team was based on good practices in other countries; however, the feasibility of this bottom up/demand-side approach was insufficiently analyzed in the Costa Rican Institutional context. 7 23. The critical risks were not properly identified, the assessment underestimated the risks and the measures that could mitigate these risks were inaccurate. First, the assessment completely ignored the broad country risk due to the political crisis that emerged in 2004 (prior to Board approval) and the limited power of the executive branch of Government versus the legislature, along with the length of constitutional review for the ratification of external loans. 5 In programming the Board presentation, the Bank failed to plan according to the cycle for project approval and project budgets. Second, it underestimated the risks related to the low capacity of the MEP to efficiently execute the Project and, more specifically, the accountability of the technical units of MEP regarding their respective components. Overall, PROMECE could have had a clearer and narrower role as fiduciary management unit, along with the possibility to support the institutionalization of Project management within MEP, given responsibility to the technical units (“Unidades Ejecutoras”). Third, the assessment underestimated the risk linked to the participatory process wherein the POA (Annual Operational Subprojects) was prepared based on the various inputs provided by local communities. There was no evidence that the procedures were clear and the processes and capacity where in place in the MEP to guarantee a participatory process. 2.2 Implementation Factors outside the control of the Government and the implementing agency 24. There was a 21-month delay in project effectiveness. Despite the government’s full commitment to the Project objectives, it was in the last year of its tenure and, therefore, had little influence in Congress to secure ratification. General elections were held on February 5, 2006 and the new Government which took office on May 8, 2006 had to re-enter the ratification of the loan in the Congressional agenda in June 2006, after which they received legislative approval and completed the legal formalities. The Project was declared effective on December 21, 2006. Given the late approval during the calendar year 2006, the 2007 Budget Law did not include the Project’s proposed investments; therefore, the Project required an “extraordinary” budget approval by the National Assembly. At the time, the Congressional agenda was polarized by CAFTA (Central American Free Trade Agreement) negotiations and the Law modifying the budget was not approved until August 2007. As a result, the first loan disbursement took place in March 2008. 25. The decision of Contraloría General de la República (CGR) regarding the Project's largest procurement contract (US$3.7 million) contradicted the Bank's rules and significantly affected Project completion. The bidding process for the procurement contract (to purchase computer equipment for teachers and students in the targeted area) involved several activities across the components and, therefore, impeded the full execution and disbursement of loan proceeds. Following a complaint to 5 The Government's collapse occurred during project preparation. Considering the risk, the Bank could have delayed completing project preparation and Board approval until new elections had been held. 8 the CGR, the procurement process failed and the contract was not awarded due to a difference in Bank procedures and country procurement rules, with the CGR favoring country rule—a decision that the Bank found unacceptable. The Bank, with the support of the Ministry of Finance, stated that an additional extension of the Project was possible if the case was resolved. After PROMECE cancelled and published the failed process, CGR allowed for an appeal from the affected firm, as a result the Bank did not proceed with the extension. Factors generally subject to implementing agency control 26. In April 2009, the High Education Council (CSE the “Consejo Superior de Educación”) decided to transform “telesecundaria” to “Liceos rurales” and approved a new curriculum from grade 7 to 11. This was a fundamental step to promote equity as this policy aimed to provide access to secondary school to all students in rural areas and prepare for their Bachillerato. This policy consisted in: (i) implementing the new curriculum (of 4 years instead of the 2 included in telesecundaria), with a minimum of 5 qualified professors in the main disciplines by Liceo (instead of 1 to 3 unqualified in telelesecundaria); (ii) rehabilitating and building school infrastructure; (iii) appointing a Director and an administrative staff member in each institution; (iv) adapting program contents to the context and providing specific books and pedagogical materials; (v) increasing the number of scholarships available; (vi) increasing enrollment in each institution; (vii) providing a wage premium of 50% to attract teachers in rural areas; and (viii) providing energy, connectivity and computer equipment. 27. In 2007, MEP initiated an institutional reform (Decree 34075) and implemented a large scale reorganization of its internal structure (Decree 35513 of September 25, 2009) to eliminate duplication of functions and inefficiencies. This decree established the guidelines for the decentralization in the Regional Directorates and their respective responsibility in implementing an education policy that recognizes cultural and ethnic diversity. That effort did not substantially increase the accountability of the technical units of the MEP to implement the Project’s technical components. However, there exist two notable exceptions: (i) the equity programs (except scholarships) were grouped under a new directorate of equity programs; and (ii) a Department for Intercultural Education (DEI) was established within the Directorate for Curriculum Development. At the time, DIEE was still under development and overwhelmed by the national investment program it managed. Therefore, the Government decided in 2010 that the infrastructure of component 1 would be managed by PROMECE 6. The PROMECE structure was revised in August 2010 and approved by the Ministry of Planning (Ministerial Decision 433). The restructuring of MEP was complemented in 2011 through the organization of the financial and budget department (decree 36451) and in 2014 by the organization of the central department (decree 38170). 6 PROMECE (Programa de Mejoramiento de la Calidad de Educación) was a Project Implementation Unit created in 1992 to manage all the exterior funded projects and in particular the one co-financed by IDB and the Bank. 9 28. The implementation of the Project was severely hampered by internal struggles between PROMECE and MEP regarding the management of the Project. Since 1992, PROMECE was in charge of executing all externally-funded education projects, both technically and fiduciary, to the point that it became a parallel structure that could challenge the power and decision-making role of the MEP. The original design of the Project changed this balance by reducing the role of PROMECE to a PCU in charge of the administrative, fiduciary and monitoring activities within the organizational structure of the MEP, while the technical and planning leadership remained under the purvey of MEP’s technical units (“Unidades Ejecutoras”). However, the responsibilities and the decision-making process were not defined until the 2011 restructuring. There was a lack of clear understanding of the competencies of PROMECE with respect to the implementing units and Office of the Minister, wherein PROMECE had an excessive presence in the management of the Project and insufficient coordination with the executing units of MEP. The 2011 restructuring confirmed that the role of the PROMECE was restricted to the fiduciary and the monitoring aspects of the Project and the infrastructure and equipment activities under component 1 of the Project. In October 2009, the PROMECE Director was dismissed on the base of an issue raised- not related to the Project - by the MEP´s Internal Audit, which took place in 2006, and he was replaced in December 2009. There was three successive PROMECE Directors in 2010. But the former Director brought his case to Court and then after different appeals to the Constitutional Court he was temporarily reinstated as PROMECE Director in September 2011 through the close of the project. While at the end the Court failed against the Director, the project was already in its final stages. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization Design: 29. In the initial Project, the Results Framework needed a better alignment of outcome indicators with the Project Development Objectives and a set of monitoring indicators that could be more easily measurable through the Ministry of Public Education’s Database. Many indicators which resulted from analytical work required an important overhaul of the MEP information systems to be useable for the Project. The SIDE (Sistema de Desarollo Educativo) which was supposed to link the database of the MEP to other databases, such as SIPO 7, and provide the indicators related to equity was a Project output and was not rendered operational to provide regular annual data for the monitoring. 30. At the 2011 restructuring, the team changed the results framework to better reflect Project objectives and outputs, ensure measurability through MEP databases and to adjust values to the new definition of targeted areas, wherein: i) the four original PDO indicators were dropped and replaced by three (see para 5 above); and ii) 7 SIPO is a database developed by IMAS (Instituto Mixto de Asistencia Social) to monitor and target social programs. 10 all 11 original intermediate indicators were dropped or revised and replaced by 15. Because the targeted areas were well identified, the design of the results framework was much clearer and the monitoring of outputs was substantially improved. However, since the PDOs were not modified, some shortcomings remained: (i) an indicator should have been added to measure the reduction of the gap in primary education; (ii) PDO 3, by measuring the reduction of administrative cost it captures the essence of Component 2 of the restructured Project, but fails to measure the efficiency of transfers and equity programs in reaching the low-income students and facilitating their access to education as initially intended in the original Project; (iii) the baseline and target values of PDO 1 were those of rural areas instead of targeted areas; (iv) the baseline values for PDO 2 were not accurate; and (v) for indicator 10, the target should have been a percentage instead of an absolute value. Implementation: 31. After the restructuring, the monitoring of Project activities and outcomes indicators improved dramatically. However, the results framework was not updated regularly and the supervision missions constantly struggled to provide sufficient reporting of Project implementation in the targeted areas. On the positive side, the workshops organized with representatives of the indigenous communities were carried out on a regular basis and allowed the team to measure the communities’ level of satisfaction and collect their recommendations. Also, the indigenous plan was updated every year; however, there was no systematic assessment of the impact of the Project, particularly of the impact on quality as a result of the pedagogical changes in the classroom. Utilization: 32. The results framework was limited to the activities of the Project and the data for the indicators were provided with delays. 2.4 Safeguard and Fiduciary Compliance Environmental: 33. The Project triggered OP/BP 4.01 (Environment). The team prepared an Environmental and Social Management Framework (ESMF) in December 2003, which included an environmental assessment template and environmental guidelines for contractors. The environmental assessment template included screening criteria to identify, evaluate and mitigate potential environmental or social impacts from school construction and rehabilitation. The template was updated in July 2009 to incorporate SETENA environmental guidelines and requirements for Environment Viability License applications. PROMECE prepared environmental impact files for each Project site. The Operations Manual also included specific requirements that define license or permit solicitation processes related to the regional National Areas Protected System (Sistema Nacional de Areas de Conservación, SINAC) to ensure that trees cut down for construction do not impact protected areas and forests. In addition, environmental 11 supervisors were hired for each cluster of school subprojects to ensure that construction complied with the requirements outlined in the Operations Manual. 34. The updated Operations Manual included screening criteria that ensures that all lands designated for school construction or refurbishment are free of outstanding legal issues and would not imply physical or economic displacement impacts, per OP/BP 4.12. Given the vast majority of schools have been constructed within the Indigenous Collective’s 12 Territories, land usage rights are transferred from the Indigenous Development Association (Asociaciones de Desarrollo Indigenas, ADIs), or legal representative of the Territory, to the Education or Administrative Council. A usage rights agreement was signed as one of the primary steps for Project eligibility. The assessment done by the Bank in 2011 revealed that: (i) of the 81 schools to be constructed or refurbished, 74 were located on the same site as an already establishes and operational education center; and (ii) for the 7 new properties, 4 had no current use, 2 sites were yet to be determined and 1 site required the Project to await the expiration of an agreement for agricultural use. All the outstanding land issues were resolved except one and, as a result, one school (the school of Vesta in the territory Cabeca Tayni) was not built. Indigenous: 35. The Project triggered OP/BP 4.10 (Indigenous Peoples) as the direct beneficiaries of this Project are overwhelmingly Indigenous, wherein 80 of the 81 school construction/refurbishment projects were located in Indigenous Territories, Component 2 supported the strengthening of the Indigenous Education System, and Component 3 supported the development of Intercultural Education. As part of Project Preparation in 2003, a Social Assessment and an Indigenous Peoples Planning Framework (IPPF) were prepared based on extensive consultations with Indigenous and rural education stakeholders. The Project’s implementation included all of the recommendations. As part of the restructuring, the IPPF was updated and validated after three workshops held on October 29 and 30 and November 9, 2012 to consult a select group of Indigenous leaders that represent Costa Rica’s different ethnicities and territories. Whereas the original Project included activities designed to take place in the Department of Limon— which is home to the majority of the Afro-Costa Rican population—these activities have been subsequently removed from the Project due to the provision of alternative financing for construction in this area. As a result, the updated IPPF only addressed issues related to Indigenous Peoples and not the Afro-Costa Rican population. 36. The IPPF itself served as the MEP’s Indigenous Peoples Plan, fully incorporating during implementation: (a) the principles and processes of free, prior and informed consultation with ADIs, which are the legal Indigenous representatives, relevant stakeholders (Indigenous parents and teachers) and traditional leaders; (b) documentation of broad support in each beneficiary community; and (c) socio-cultural adaptations in the implementation of consultation processes, work within Indigenous communities, and in the design of schools, legal reforms and processes to develop the subcomponent on intercultural education. 12 37. Finally, the MEP issued the Decree 35513 on August 2012, which reorganized the regional school administration, thereby recognizing the 24 indigenous territories and creating the Regional Directorate of Sula to represent 7 territories of Limon and 6 administrative circuits. In addition, 9 indigenous circuits were created and integrated with the Regional Directorate of Turrialba, Grande de Térraba and Coto. The Government also saw the Project as a key tool in helping Costa Rica to comply with its international obligation under International Labor Organization (ILO) 169, which calls for education systems to respect Indigenous cultures and participation in curriculum development, teaching and management. 38. The Project successfully established the indigenous and intercultural policy by: (i) fully renewing and developing the education system in in the majority of Costa Rica’s indigenous territory and providing new opportunities for youth and their families; and (ii) establishing the DRE of Sula specific to indigenous territories and permanent participatory and consultative processes with the communities; (iii) including the Department for Intercultural Education (DEI of 6 staff) in the MEP organization to put in place a full-scale intercultural education policy, including the development of didactic materials adapted to the cultural context, the organization of a series of trainings for teachers and regional advisors, the definition of infrastructure needs and design of schools through participatory methods that respect indigenous cultures. 39. Financial Management Performance throughout the project life was, in general, moderately satisfactory. In September 2012, significant delays in the submission of the IFRs and Audit Reports were noted. This issue was fixed in April 2013 and IFRs were delivered to the Bank in general with either minor or moderate delays throughout the life of the project. . During the last supervision to the project before its closure, some poor contract monitoring practices and ineligible expenditures were detected. The Bank performed a follow-up visit to the project during the grace period to support PROMECE in the closing processes, including ensuring that the reimbursement of the ineligible expenditures, the final cancellation of funds and the delivery of the final audit report were done on time.,. With the exception of the first and the last audit reports for the project, which were delivered timely, all other audits were delivered with important delays. 40. Procurement arrangements were appropriate. Three processes were used for construction: (i) community participation (the community built the school with the support of one engineer) for 18% of the cases; (ii) work subcontracted/delegated to firms which manage the entire process for 28% of the cases; and (iii) construction companies for 54% of the cases. These processes were recognized as very successful by the communities and authorities. Nevertheless, several procurement problems occurred. First, the procurement of 12 consultants to develop 30 participative diagnostics (for a cost of US$ 360,000) in 2009 was cancelled and investigated by the MEP Internal Audit Unit and by the “Contraloría General de la República” (CGR) MEP specifically requested two ex-post procurement reviews (the first in May 2009 and the second in November 2010). Both Bank reviews found the same misprocesses and mistakes, but the Bank finally considered there were not enough infringements to declare misprocurement. 13 Second, the procurement of computers (US$ 3.7 million) to equip the schools failed following a complaint to the CGR. The contract was not awarded due to a difference in Bank procedures and country procurement rules, with the CGR favoring country rules that the Bank found unacceptable. The Bank objected to the evaluation report, consequently PROMECE requested cancellation of the process on November 1, 2013. 2.5 Post-completion Operation/Next Phase 41. There is no specific operation or next phase programmed. Nevertheless, the experience gained in the targeted areas to develop quality and intercultural education will be used in the other indigenous territories. The ownership and improved autonomy of the indigenous communities through the Project and Decree 35513, which regulates indigenous and intercultural education, strongly indicate that the policy in favor of education in rural areas and for the indigenous population will be continued. 3. Assessment of outcomes 3.1 Relevance of Objectives, Design and Implementation Relevance: Pre – 2011 restructuring – Substantial/Post restructuring -Substantial Relevance of objectives: Substantial 42. The Project objectives are embedded in the National Development Plan 2010- 2014. It “guarantees the right for all to quality education, which is vital to economic dynamism, social cohesion and collective identity. This right will be consolidated using equity instruments for moving towards universal coverage in preschool, primary and secondary. This requires adequately funded and efficiently integrate equity programs and uses them to offset the effect of the various economic, ethnic, cultural, gender inequalities and other limiting access to equal opportunities in education.” This strongly indicates that the Government will continue to pursue the objective of increasing the graduation rate in secondary education as well as reducing the quality gaps in rural areas through investments and equity programs. The Project’s objectives are consistent with the Cluster 2 of the Country Partnership Strategy 2012-2015 (CPS) to improve efficiency and equity in social sectors. The CPS specifies that: (i) the Government will continue efforts to increase the number of secondary graduates with a special focus on the poor. (ii) The Government is strongly committed to education with the passage of legislation to increase the budget allocation to the sector for 6% to 8%. Relevance of design: Substantial 43. The Project design is highly relevant with the objective of reducing the quality gaps in rural area and moving forward to guarantee the right for all to quality education. Three main components can be considered best practices to achieve the objectives: : (i) to invest in “Liceos rurales,” which have proven to be the most efficient way to promote graduation at secondary education level; (ii) to adapt the 14 curriculum and teaching in primary and secondary education to cultural and local needs; and (iii) to expand and improve the targeting and efficiency of equity programs to ensure that low income students and students at risk can access all education levels (preschool, primary and secondary). 3.2 Achievement of Project Development Objectives Reduction of the education quality gaps in primary education in rural areas. Pre-restructuring rating: Negligible Post-restructuring rating: Substantial 44. Previously, there was no assessment of learning outcome through a standardized exam to measure quality in primary education in Costa Rica. Therefore, the Project the graduation rate in primary education used as a proxy for quality. 8 Primary education is universal in Costa Rica and almost all students complete 6th grade and graduate. The graduation gap in rural areas improved at a higher pace than the national level and it is remarkable to note that the graduation rate in rural areas is now better than at national level. The existence of pockets of poverty at the periphery of major cities, overcrowded class rooms and youth delinquency may explain the quality issue in urban areas and this consistent trend. In the targeted areas the graduation rate improved from 87.9% to 89.1%; nevertheless, with about 5 percentage points the graduation gap with the rest of the country remains significant, mostly because those areas still concentrate the highest levels of poor, illiterate and marginalized indigenous population. Table 1: Reduce Education Quality Gaps in Primary Education in Rural Areas Baseline value 2006 Formally revised Actual value 2013 target value 2011 Indicator : Graduation rate in primary education (6th grade) 1) National level 93.47% N/A 95.79% 2) Rural areas 92.27% N/A 96.12% 3) Targeted areas 87.9% N/A 89.1% Gap (1-2) 1.2 -0.33 45. The relative closing of the gap in rural area in primary education is considered largely a result of the expansion of the equity and transfers programs and the improvement of their respective targeting of low-income students (details are in table 4 and 5 and the corresponding paragraphs). Therefore, the Project contributed to foster access, participation and achievement of low-income students to primary education through the improvement of the efficiency and targeting of scholarships, transportation and food programs. 46. In the targeted areas, the Project contributed to the improvement of quality and graduation rates in primary education by improving the learning conditions 8 Graduation rates are used by the World Bank as a proxy indicator for quality in Education Sector typical results chain. 15 through: (i) the construction or rehabilitation of 63 primary schools; and (ii) the provision of furniture and equipment for the schools to be fully functional. The schools did not receive their computers but this was not detrimental to their functionality. In the DRE of Sula, the Project financed the training of: (i) 129 primary education teachers for teaching the new Spanish curriculum; and (ii) 23 primary education teachers and 3 advisors in intercultural teaching practices. Reduction of the education quality gaps in secondary education within rural areas Pre-restructuring rating: Negligible Post-restructuring rating: Substantial 47. At secondary education level the quality gap in rural areas measured by the percentage of students passing the “bachillerato” exam was slightly reduced from 9.7 percentage points to 9.5. The passing rate improved significantly in rural and urban areas. In urban areas, the passing rate nearly reached the target of 70%. Yet with 60.3%, the graduation rate in rural areas remained well below the 70% target, which was too ambitious. The improvement is considerable in the indigenous territories and, specifically, in the areas targeted by the Project due to a lack of school facilities offering the full secondary education curriculum to prepare the “bachillerato” in 2006. Table 2: Reduce Education Quality Gaps in Secondary Education in Rural Areas Indicator Baseline value 2006 Formally Revised Actual Value 2013 Target Values 2011 Indicator: Reduce education quality gaps in targeted areas (percentage of students that pass - bachillerato- exams). 1) National level 62.7% 70% 69.8% 2) Rural area 53% 70% 60.3% 3) 24 indigenous 11.4% NA 46.5% territories 4) Targeted areas: ( 9 administrative 14.1% NA 40.5% circuits/10 indigenous territories) 5) 14 “Liceos rurales” 0 NA 49.9% built by the project Gap (1) – (2) 9.7 9.5 48. The Project was instrumental to implement the MEP policy that transformed Telesecundarias to Liceos rurales (as described in par.26). In 2006, there were 139 telesecundarias in rural areas providing a low-quality secondary education which failed to provide the full curriculum to allow students to complete their “bachillerato.” In 2014, they were replaced by 109 Liceos rurales (of which 25 were located in indigenous territories) 9 providing the opportunity to students in rural areas to complete four years of secondary education and to prepare for the bachillerato exam. Enrollment in Liceos rurales tripled from 3,113 students in 2009 to 9,501 in 2013. 9 In 2014 only 32 Telesecundaria remain. 16 49. The Project contributed to the improvement of graduation rates at secondary education level through: (i) the construction of 14 Liceos rurales (their enrollment is growing fast and enrollments already exceed 120 students in 8 of them); and (ii) the provision of furniture and equipment for all these schools to be fully functional. The school infrastructure in the targeted areas was built to support a transformation process of the educational model through collaborative networks that respect the autonomy and cultural identity of the indigenous communities. The initial MEP infrastructure program was fully achieved with the exception of: (i) three schools (Vesta, Sharabata, and Jonkruhora) that were not built because of land and accessibility issues; and (ii) the school computer laboratories were not equipped with computers because the procurement failed, which was fortunately not detrimental to the schools’ operations. 50. The Project contributed to the development of School Networks to promote equity of access and quality of education in the targeted indigenous territories. The 14 Liceos rurales were put at the head of Networks of 66 primary schools. The Networks were organized: (i) to foster a sense a community and cultural identity among teachers, students and parents; (ii) to provide continuity in the educational process to increase access to secondary education; (iii) to organize training and workshops; and (iv) to establish common facilities (e.g. cultural and Art Centers or meeting rooms) for the benefit of the entire network. The policy was fully implemented in 9 school networks, including 41 schools from the DRE (Regional Directorate) of Sula, whose main activities included the following: (i) organizing participative diagnostics in the 9 Liceos rurales to identify main issues and make recommendations; (ii) developing a relevant pedagogic model for the liceos rurales and multi-grade schools; (iii) building plans (“planes de vida”) including objectives and indicators at short, medium and long-term; and (iv) organizing about 30 workshops for teachers and advisors to develop the organizational structure of the network and to install the intercultural pedagogic model and the curriculum adapted to the indigenous context. 51. The Project contributed to the quality of secondary education through the financing of professional development programs for teachers and advisors. At national level: (i) 1,697 teachers out of 12,854 were trained for the Liceos rurales; (ii) 76 national advisors and 375 regional advisors were trained and certified in a 60-hours training about “Adapted teaching practices in an intercultural vision” to support teachers; and (iii) 350 teachers were trained in the use of information technology for teaching. In the DRE of Sula, the Project financed the training of: (i) 28 secondary level teachers in the use of information technology to teach English; and (iii) 79 secondary level teachers in intercultural teaching practices. 52. To improve the quality of teaching in an intercultural context, the DEI designed and carried out a vast program to collect, through a participative approach, the traditions, customs and languages of indigenous cultures in Costa Rica in order to adapt curriculum contents and elaborate didactic materials. Based on this information and with the financing of the Project, DEI produced documents for teachers, pedagogic advisors and supervisors that were disseminated through training and workshops. The documents that were largely disseminated include: (i) strategy and 17 methodology for the implementation of intercultural education and curriculum (distributed to 105 teachers and advisors); (ii) three training modules of intercultural education (distributed to 5,153 teachers, supervisors and advisors); (iii) video documentary of the Battle of Sardinal and Trinidad 1856-57 (distributed to all 27 DRE to be used for teacher training); and (iv) 4 pedagogic materials related to literature, tradition, history and cultural events distributed to 200 teachers of the DRE of Turralbia. Some pedagogic materials were produced for the direct use of the students, in particular four manuals to teach English in an indigenous context that were produced, distributed and used by 5,630 III cycle students. The full list of documents is in Annex 9. Improve equity in the allocation of education sector resources. Pre restructuring rating: Negligible Post restructuring rating: Modest 53. Equity of access to education in Costa Rica, measured by the ratio between 11th grade enrollment and 7th grade enrollment, improved substantially. The ratio increased from 34 % to 43.6% at national and surpassed the target set by the project. The enrollment gap in targeted area was reduced from 13.4% to 9.2%. Equity measured by the distribution of 13 to 17 year-old students (age group corresponding to secondary education) by income quintile, showed the same improvement, particularly for students from low income families (7.4 percentage points for Q1 students and 12.1 p.p. for Q2 students and 10.5 p.p for Q3 students). This last result is consistent with the fact that there was a higher proportion of students from low income families in rural areas and even more in the targeted areas. Table: 3 Improving equity by reducing gaps in graduation rates Baseline value 2006 Formally revised Actual value 2013 target value 2011 Indicator : Improving equity: Reduce gaps in graduation rates measured as the ratio between 11th grade enrollment and 7th grade enrollment 1) National level 34% 42.5% 43.6% 2) Targeted areas 20.6% 30% 34.4% Gap (1-2) 13.4% 12.5% 9.2% Indicator: 13-17 years old students enrolled in formal education by income quintiles (2006 -2012) Quintile 1 69.7% NA 77.1% Quintile 2 70.8% NA 82.9% Quintile 3 75.9% NA 86.4% Quintile 4 84.7% NA 91.2% Quintile 5 95.3% NA 92.7% 54. These outcomes were achieved through Government policy, which increased investment in education considerably (between 2006 and 2011 the budget was multiplied by 2.4 and the budget of education as a percentage of GDP went from 5.2% to 7.1%) and translated into more school facilities, more teachers (teaching staff increase from 77,446 in 2007 to 88,548) and more resources for the equity and transfer programs. 55. Equity in the allocation of educational resources improved though the combination of: (i) a high concentration of scholarship distribution to 6-12 year-old 18 students toward the lowest income families 10(75% to quintiles I and II in 2010 and 76% in 2013); (ii) a significant increase of the number of beneficiaries of scholarships (the number more than doubled from 68,000 in 2006 to 164,311), transportation (70,075 to 102,623) and food (from 582,708 to 623,129): and (iii) an increase in FONABE amounts of the different categories of scholarships (agreement N 153-13 of May 7 2013). It is worth notice that the unit amount of scholarships received by indigenous students at primary and secondary education levels is 50% higher than for other students. Table 4: Improving equity in the allocation of resources (This indicator is provided through the household survey 2010-2013) Indicator Baseline value Target values Actual 2013 (2010) Indicator: Allocation of FONABE scholarships by quintile of revenue for 6 to 12th years old students Quintile 1 44% N/A 41% Quintile 2 31% N/A 35% Quintile 3 17% N/A 17% Quintile 4 6% N/A 5% Quintile 5 1% N/A 1% Country Total 100% 100% 100% 56. The Project contributed to the improvement of equity in the allocation of resources of the equity and transfer programs through the development of a targeting model, improvement in the allocation procedure and upgrade in the management system. This was possible through the financing of studies in a five-step process: (i) an analysis of the factors that have an impact on dropout and repetition rates and the definition of an education vulnerability indicator for targeting a transfer program that combines household ownership, access to technology, mother’s education level, distance to school, minority group status, social risk, health conditions in the district, school level in the district; (ii) the development of a model for calculating the cost of equity programs, according to various efficiency and transparency criteria for adapting them to regional and cultural patterns and for determining the budget needs according to enrollment (the model was implemented by the MEP since 2010 for transport and since 2013 for food programs 11 ); (iii) the development of an integrated model combining poverty index and education vulnerability for the scholarships (this model was used since 2012 by FONABE); (iv) the improvement of all the processes and procedures to manage the equity programs, with an emphasis on delegating decision-making power to the 10 The distribution of scholarships by income level is a proxy which measures only one dimension of the equity issue. The Government “Equity Programs” were better targeted through three set of criteria: (i) poverty level; (ii) education vulnerability (which include isolation and distance to the school); (iii) individual specific situations (pregnant teen student, student at risk, indigenous student, handicapped student). It does not fully reflect the improvement of the “fair”targeting of those programs to the ones who need the most. 11 The FONABE, (Fondo Nacional de Becas) Fund to finance the scholarships, is not directly administered by the MEP. 19 Regional Directorate; (v) the development and implementation on July 2013 of a fully computerized financial and accounting system (TCTE) that complements the SIGAF (from the Ministry of Finance) to manage, allocate and control the equity and transfers programs. The differences observed between 2010 and 2013 in the allocation of scholarships by income level as results of the implementation of the new model seem modest. However, more time and more detail analysis will be necessary to measure the full impact of a more pertinent targeting on students who need the most. Finally, the Project financed the training of 42 staff of the DRE in 2012 and 38 in 2013 to improve the supervision and assessment of equity and transfer programs. Improve efficiency in the administration and use of education sector resources. The outcome is substantial. Pre restructuring rating: Negligible Post restructuring rating: Substantial 57. The administrative cost per beneficiary of managing the equity and transfer programs was reduced 30% from 2006 to 2013 (from 10,855 to 7,013 CR colones). While the number of scholarship beneficiaries at a national level increased by 240%, the beneficiaries of transportation inputs increased by 46% and the beneficiaries of food programs increased by 15%, the number of MEP administrative staff to manage these programs remained about the same. The transfer of funds to school councils for transportation and food increased from 31.611 million CR colones in 2006 to 81.076 million CR colones in 2013 and the transfers of funds to scholarship beneficiaries from 6.861 million CR colones to 28.320 million CR colones. The equity and transfer programs represented 35.72% of the total 2013 education budget. Table 5: Improve efficiency in the administration and use of education resources Indicator Baseline value 2006 Formally Revised Actual Value 2013 Target Values 2011 Improving Efficiency: Reduce administrative costs per beneficiary of MEP Equity and Transfers Programs (scholarships, transportation, food). Value 10,855 CR colones 8,005 CR colones 7,013 CR colones Number of beneficiaries of scholarships 68,453 N/A 164,311 Number of beneficiaries of transportation 70,075 N/A 102,623 Number of beneficiaries of food programs 582,708 N/A 673,129 58. The Project contributed to the improvement of the efficiency in the administration and use of the equity and transfer programs through: (i) the improvement of the allocation procedures and the enhancement of the management information system (see par.56), which expedited and computerized system processes (to leverage staff time and capacity); and (ii) the reorganization of the MEP and 20 consolidation of all the dispersed services previously in charge of the equity and transfers programs under the responsibility of one Directorate (DPE) 12. 3.3 Efficiency Rating: Modest 59. The cost-benefit analysis performed for the restructuring (2011) provided a sound basis to estimate the economic results of the operation. Based on the same assumptions used in 2007, an updated of the economic analysis was carried out for the ICR using actual data for the different variables (Annex 3). In terms of the cost benefit analysis, the Project would yield a present value of net benefits, after investments, of US$15.1 million and produce an internal rate of return (IRR) of 29%, compared with 51% at appraisal. In terms of cost-effectiveness, the unit costs of school infrastructure were reasonable, taking into account the high specificity of the indigenous territories, in terms of remote locations, use of local materials and community involvement: cost by square meter of school construction was an average of US$350 for wood and US$490 for cement (“prefabricado”), in line with national benchmarking. From a fiscal point of view, the additional recurrent maintenance costs as a result of Project investments in infrastructure would represent approximately less than 1% of the Education Ministry Expenditures. However, main outcomes will be highly dependent on maintaining recurrent allocations for equity and transfer programs. 3.4 Justification of Overall Outcome Rating Rating: Moderately satisfactory 60. The Overall Outcomes uses a split rating to comply with ICR guidelines. Relevance ratings are “substantial” for both pre- and post-2011 restructuring periods. Design rating is “modest” before restructuring and “substantial” after restructuring. There is no evidence of progress against original PDO indicators, therefore the efficacy rating is “negligible” before restructuring; and “substantial” after restructuring. Finally, the efficiency rating is “modest” for both periods. The combination of these partial ratings is “unsatisfactory” for the pre-restructuring period and “satisfactory” for the post restructuring period. In turn, the overall ratings for the two periods are weighted by the disbursement percentages prior to and following the 2011 restructuring, yielding an overall weighted rating of 4.1, equivalent to “moderately satisfactory.” 12 The DPE was reorganized though the MEP decree 36451 of 2011 and a derogation of the MEP decree 38170 of February 14, 2014. 21 Table 6: Weighted Project rating Pre-restructuring Post- Overall (March 2005-June restructuring 2011) (June 2011- December 2013) Relevance Substantial Substantial Design Modest Substantial Reduce the education quality gaps in primary Negligible Substantial education in rural areas Reduce the education quality gaps in Negligible Substantial secondary education in rural areas Improve equity in the allocation of education Negligible sector resources Modest Improve efficiency in the administration and Negligible Substantial use of education resources Overall Efficacy Negligible Substantial Efficiency Modest Modest Overall outcome rating Unsatisfactory Satisfactory Rating value 3 5 Weight 30% 70% Weighted value 0.9 3.5 4.1 Final rating Moderately Satisfactory 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 61. The Project focused entirely on the indigenous population that is positioned at the lower end of the socio-economic spectrum and located in remote areas. The project was relevant not only from educational perspective by providing new opportunities at secondary level but also by helping disadvantaged students with food, transportation and scholarships. (b) Institutional change/ Strengthening 62. The main achievement of the Project was to establish the intercultural and indigenous policy within the organization of the MEP and the education policy of Costa Rica. 4. Assessment of Risk to Development Outcome Rating: Moderate 63. The risk that the education quality gaps in Costa Rica’s primary and secondary education levels would increase remains moderate: (i) with the significant effort realized in rural areas to improve school facilities, to establish the liceos rurales 22 and to train qualified teachers and advisors the graduation rates in secondary education should progressively reach the rates observed at national level like they did in primary education; (ii) in the targeted areas and more broadly in the indigenous territories, the trend should also be positive, the facilities built are sufficient to accommodate a strong increase in demand at the secondary education level, the curriculum was adapted to specific needs, the teachers trained in intercultural education practices and the decentralized administration of Sula remains committed to the implementation of this policy in indigenous territories. However, the dissemination of the books and intercultural documents (produced by DEI) in 2014 and 2015 are subject to the availability of a budget. 64. With regard to the improvement of equity and efficiency in the allocation of resources, the implementation of the model combining the poverty index and education vulnerability, along with the corresponding established procedures will ensure that the equity and transfer programs continue to target low income families. However, the processes to monitor and control the use of the funds still need to be improved. A proposal of a new regulatory framework has been submitted for consultation at the end of 2013 and should be approved in 2014. Also, a consulting firm should be contracted to produce the training materials to strengthen the School Councils. Despite the expected change in administration during May 2014, MEP is likely to continue this policy in rural areas and extend the policy initiated by the Project to the rest of the indigenous territories. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory 65. The Project built on strong analytical work and included lessons learned from several projects in Central and Latin America. It engaged the Borrower’s attention related to the most socially disadvantaged communities and proposed innovative approaches to increase local participation and to promote the role of civil society in education. However, the Project had significant shortcomings: (i) it was too complex and insufficiently clear to be implemented; (ii) its scope was too large and was insufficiently targeted for the resources available; (iii) it invested in an education model “Telesecundaria” that was inappropriate to improve quality and equity; (iv) the implementation arrangement and responsibilities were insufficiently defined, given the complex institutional situation; (v) the Results Framework required better alignment with the PDOs and a set of monitoring indicators that would be more easily measured; and (vi) the critical risks were not properly identified. 23 (b) Quality of Supervision Rating: Satisfactory 66. During the long period pending the ratification of the project, Bank’s team maintained involvement in project support activities providing advice and technical assistance to the PCU on fiduciary matters and to the MEP to ensure expeditious implementation upon the Project’s ratification. 67. The Bank ensured strong supervision despite the difficult conditions faced in Project implementation. The Bank maintained a close relationship with the Ministry of Finance to solve difficult issues. The presence in the field of an infrastructure/procurement specialist greatly helped. The Bank provided candid and realistic assessment of the Project’s situation in implementation status report (ISRs). The Bank’s role was highly regarded by MEP authorities for its commitment to the objectives of Costa Rica’s education policy as well as for its support and the useful solutions it provided. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 68. Since the quality at entry is “moderately unsatisfactory” and the supervision “satisfactory,” the outcome rating applies. Therefore, the overall Bank performance is “moderately satisfactory.” 5.2 Borrower Performance (a) Government Performance Rating: Satisfactory 69. The Government fully supported the Plan "Relanzamiento de la Educación Costarricense" providing the resources to the education sector necessary to reduce the rural-urban education gap. The Government provided counterpart funding, corresponding to 40% of the total Project Cost. The Government fully supported MEP policy to develop an intercultural education policy in indigenous territories. The Ministry of Finance closely monitored the Project and worked proactively to solve controversial issues with the Contraloría. (b) Implementing Agency or Agencies Performance Rating: Moderately unsatisfactory 70. Overall, the MEP and PROMECE were able to carry most activities of the Project to their full completion with the exception of three (the improvement of the 24 capacity to analyze student learning assessment, the development of an information system at school level and the piloting of the use of ICTs in rural education). Overall, almost 20% of the funds of the Project were left undisbursed due to the problem with Contraloría, but this was outside control of the implementing agency. However, MEP and PROMECE were never able to reconcile their institutional issues. Moreover, despite the strong leadership and commitment of the Minister, some technical units of the MEP were not fully committed with the Project. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately satisfactory 71. Since the Government performance is “satisfactory” and the implementing agency “moderately unsatisfactory” the outcome rating applies. 6. Lessons Learned General application 72. The risk analysis should be taken seriously. When a risk cannot be mitigated, the activity should be either dropped or substantially modified. Experiences or good practices from other countries can be transferred, provided two conditions are fulfilled: (i) there is Government ownership; and (ii) a careful institutional analysis demonstrates that the conditions for this transfer are favorable. It is extremely risky to carry out a project with a participatory approach in a complex institutional context, particularly when the project design does not define decision-making power regarding school investment and contents of the subprojects to be financed. An in-depth restructuring that simplifies the project, clarifying the components and targeting the investments, can be appropriate to re-focus the project, improve the policy dialogue with the Government and mobilize the staff around common goals. Project specifics lessons learned 73. In rural areas, it is possible to propose a pedagogical model and teaching conditions for secondary education which provide the same opportunities to children from low income families and promote equity. The participatory approach is appropriate to adapt the curriculum to the indigenous culture and infrastructure to the needs and local customs. 76. The environmental safeguards could be used as a tool and framework to implement and monitor a National indigenous policy. Construction processes through community participation or managed by a competitively-selected third party can be effective and cost-efficient. 25 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners a) Borrower/implementing agencies Please see Annex 7 b) Cofinanciers N/A c) (Other partners and stakeholders (e.g. NGOs/private sector/civil society) N/A 26 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Actual/Latest Appraisal Estimate Percentage of Components Estimate (USD (USD million) Appraisal million) Component 1: Quality and 34.24 20.5 60.7% Equity of Rural Education Component 2: Improved 3.86 Equity of Education Services 21.8 525% Component 3: Improved 10.20 1.9 19% Institutional Efficiency Total Baseline Cost 48.30 44.2 100% Physical Contingencies 0.72 0.0 Price Contingencies 0.68 0.00 Total Project Costs 49.70 0.00 Front-end fee IBRD 0.30 0.15 50% Total Financing Required 50.00 44.35 89% (b) Financing Appraisal Actual/Latest Type of Co- Estimate Estimate Percentage of Source of Funds financing (USD (USD Appraisal million) million) Borrower 20.00 20.00 100.00 International Bank for Reconstruction 30.00 24.2 80.00 and Development 27 Annex 2. Outputs by Component 1. The objectives of the Project are to: (a) reduce education quality gaps in the Borrower’s primary and secondary education system in rural areas; and (b) improve the equity and efficiency in the allocation, administration and use of the Borrower’s education sector resources. Component 1: Efficient and Equitable Access to Rural Education Classrooms: 2. The Project financed in the targeted areas : (i) the construction of 14 new Liceos rurales; (ii) the construction or rehabilitation of 63 primary schools; and (iii) the provision of furniture and equipment for all the schools. This corresponds to 236 classrooms of which 72% are new and 28% rehabilitated. (see table below) The initial objectives were fully met at the exception of three schools (Vesta, Sharabata, and Jonkruhora), which were not built because of land and accessibility issue. School name # students Network Territory District Col. Boca Cohen 141 Calveri 65 Moi 40 Bella Vista 63 Alto Cohen 72 Boca Cohen Cabécar, Tayní Limón Cerere 111 Jabuy 65 Isla Cohen 79 Boca Cohen 72 Col. Vesta 123 Valle de la Gavilán 115 Vesta Cabécar, Tayní Estrella Vesta 1/ - Col. Katsi 126 Bris 22 Namú Wokir 57 Tala Alto Katsi 17 Katsi Bribri, Talamanca manca Boca Urén 60 Dururpe 22 Katsi 58 Col. Sepecue 138 Sepecue 95 Santo Tomás 64 Sepecue Bribri, Talamanca Talamanca Sibodi 109 Mojoncito 103 Col. Coroma 72 Coroma Bribri, Talamanca Talamanca 28 School name # students Network Territory District Coroma 66 Col. Palmeras 75 Cabécar, Bajo Palmeras Talamanca Palmeras 85 Chirripó Col. Roca Quemada 88 Roca Cabécar, Bajo Turrialba Tsiniklari 75 Quemada Chirripó Col. Fila Carbón II 98 Fila Carbón II Cabécar, Bajo Paso Marcos 53 Turrialba / Alto Quetzal Chirripó Sharabata 1/ - Col. Térraba 204 San Antonio 89 La Fila 62 La Shamba 37 Buenos Bajo Veragua 23 Térraba Teribe, Térraba Aires Sabana 15 Bijagual 31 Ceibón 44 Alto Veragua 16 Col. La Casona 120 Ngobegue 190 Brus Mali 72 Betania 34 Ngöbe-Buglé, Coto La Casona Coto Brus La Chiva 45 Brus Villapalacios 45 Quiabdo 104 Jonkruhora 1/ - Col. San Rafael de 137 Cabagra Brazo de Oro 47 Las Delicias 37 Palmira 17 San Juan 65 San Rafael de Buenos Bribri. Cabagra Cabagra Aires San Rafael 163 Ska Dikol 49 Sikebata 12 Huacabata 29 Mrusara 14 Col. Yeri 65 Yeri 36 Río Azul 45 Buenos Yeri Bribri, Salitre Sipar 28 Aires Arturo Tinoco 99 El Puente 70 Col. San Rafael Bajo San Rafael Ngöbe-Buglé, 55 Corredores Los Indios Bajo Los Abrojo Montezuma 29 School name # students Network Territory District San Rafael Bajo Los Indios 68 Indios Col. Abrojo Guaymí - 73 Montezuma Abrojo Ngöbe-Buglé, Corredores Bajo Los Indios 18 Montezuma Abrojo Montezuma Cacoragua 15 Col. Yimba Kaj 125 Rey Curré 60 El Progreso 28 Bajos de Mamey 15 Buenos Yimba Kaj Boruca, Curré Aires Lagarto 4 Las Vegas 20 Zapotal 9 Katuir 25 Patiño 50 Keköldi Bribri, Keköldi Talamanca Keköldi 25 School building facilities: 3. The school infrastructure in the targeted areas was conceived as a process of transformation to implement an educational model through collaborative networks in full respect of the autonomy and cultural identity of the indigenous communities. Depending on the condition of accessibility, three processes were used for the construction: (i) community participation (the community build the school with the support of one engineer) for 18% of the cases; (ii) work subcontracted/delegated to firms which manage the entire process for 28% of the cases; and (iii) construction companies for 54% of the cases. All the schools were fully furnished with furniture and equipment for class rooms, teachers lounge, student dormitories, cafeteria, restrooms and kitchens. At the exception of the computers that could not be procured. Some schools received solar panels to provide electricity. Common school network facilities: 4. The 15 new Liceos rurales were put at the head of Networks of 66 primary schools. The objectives were: (i) to foster a sense a community and cultural identity among teachers, students and parents; (ii) to provide continuity in the educational process to increase access to secondary education; (iii) to organize training and workshops; and (iv) to put in common some facilities for the benefit of the entire network. Regional Indigenous Education Directorate: 5. The Regional Indigenous Education Directorate of Sula was constructed and furnished. All DRE staff was trained. 30 Component 2: Improving MEP’s Institutional Efficiency a. The development and implementation of an information system at school level 6. The Project did not finance any activity to develop and implement a new information system at school level. The two activities initially planned, consisting in developing the software for an information system at school level and installing the computer equipment and software necessary for information systems in the targeted schools, were not completed. First, the technical specifications for the development of the software were not finalized. The effort lacked coordination and failed to make the specifications compatible with the existing information system PIAD 13, that was in the development process, with various systems being used in the MEP departments (essentially for the equity programs and for the financial management). Second, the procurement process for computer equipment (contract of US$ 3.7 million) failed. Following a complaint to the Contraloría General de la República (CGR), the contract was not awarded due to a difference in Bank’s procedures and country procurement rules, with the CGR favoring country rules that the Bank found unacceptable. Instead, the DGEC (Directorate for the Management and Evaluation of the Quality) continued to develop the PIAD with the MEP resources. Software was developed for all levels of the education system and made available to all the schools. However, the computerized student record system is used only in about 25% of the education system (it is estimated that 70,000 teachers use it in about t 1000 schools out of 4,600); and only 38 secondary schools use the PIAD system on line as a pilot. The main statistical data on the education system are still not available to the users and to public on line. DGEC presented to MEP in 2013 a US$1 million budget to develop the information system and make it universal but this budget was rejected. b. Improving the design and strengthening the management of MEP’s Implementing Technical Units that administer the Equity and Transfers Programs. 7. The following activities contributed to the improvement of the efficiency and targeting of equity and transfers programs: (i) an analysis of the factors that have an impact on dropout and repetition rates and the definition of a vulnerability indicators to be used for targeting transfers program 14; (ii) the development of a model for calculating the cost of the equity programs according to various efficiency and transparency criteria for contextualizing them according to regions and cultural patterns and for determining the budget needs according to enrollment (the model is implemented by the MEP since 13 The PIAD (Programa de Informatización para el Alto Desempeño) was developed by the MEP since 1998 initially to register students. 14 The vulnerability indicator combines 8 indicators: household ownership, access to technology, mother education level, distance to school, belonging to a minority group, social risk, health condition in the district, school level in the district. 31 2010 for transport and since 2013 for canteens) 15; (iii) the development of an integrated model combining poverty index and education vulnerability for the scholarships (this model is used since 2012 by FONABE) (iii) the update of all the processes and procedure to manage the equity programs with an emphasis on decentralizing to Regional Directorate; (iv) the development and implementation on July 2013 of a fully computerized financial and accounting system TCTE complementing the SIGAF (from the Ministry of Finance) to manage, allocate and control the equity and transfers programs; (v) the training of 42 staff of the DRE in 2012 and 38 in 2013 to improve the supervision and assessment of equity and transfers programs; and (vi) the consolidation of all the equity programs and restructuring of all the services in charge of their management under the responsibility of one Directorate (DPE) 16 . However, the organization of the School Councils, the efficiency of the management of the funds delegated to them and the processes to monitor and control the use of the funds still need to be improved. A proposal of a new regulatory framework has been submitted for consultation at the end of 2013 and should be approved in 2014 and a consulting firm should be contracted to produce the training materials to strengthen the School Councils. 8. The administrative costs per beneficiary of managing the equity and transfers programs have been reduced by 30% from 2006 to 2013. While the number of beneficiaries of scholarships at national level increased by 240%, the beneficiaries of transportation by 46% and the beneficiaries of food programs by 15%. The transfers of funds to the school councils for transportation and food increased from 31.611 million CR colones in 2006 to 81.076 million CR colones in 2013 and the transfers of funds to beneficiaries of scholarships from 6.861 million CR colones to 28.320 million CR colones (the equity and transfers programs represent 35.72% of the total education budget). Indicator Baseline value Formally Revised Actual Value Achieved Target Values at Completion or Target Indicator PDO 3 : Improving Efficiency: Reduce administrative costs per beneficiary of MEP Equity and Transfers Programs (scholarships, transportation, food). Value 10,855 CR colones 8,005 CR colones 7013 CR colones Number of beneficiaries of scholarships 68,453 164,311 Number of beneficiaries of transportation 70,075 102,623 Number of beneficiaries of food programs 582,708 673,129 Date achieved 12/31/2006 12/31/2013 11/20/2013 15 The FONAB, Fund to finance the scholarships, is not directly administered by the MEP 16 The DPE was reorganized though the MEP decree 36451 of 2011 and a derogation MEP decree 38170 of February 14, 2014. 32 c. Strengthening and implementation of MEP’s institutional reform program at the central and regional level. 9. The Decree 34075 (November 5, 2007) that reorganized the MEP was complemented by the Decree 35513 (September 25, 2009), which established the guidelines for the decentralization in the Regional Directorates and their responsibility in implementing an education policy which recognized cultural and ethnic diversity. With regard to decentralization, the MEP implemented a communication plan in three steps: (i) in 2011 and 2012 a diagnostic was carried out in six DRE (Alajuela, Grande del Terraba, Peninsular, San Jose Central, San Jose Oeste y Sula) through surveys and workshops to identify issues related to services to users; (ii) in 2012 the design of the communication strategy; and (iii) the implementation in six Regional Directorates DRE (Heredia, Cartago, Aguirre, Guapiles, Cañas, Santa Cruz, and San Jose Norte). This implementation included: (i) a diagnostic of the DRE; (ii) the design and production of communication materials for the DRE; (iii) the organization of three one day workshops for a total of 320 participants dedicated to services to the client, executive protocol and communication in a situation of crisis; and (iv) a one day training for 312 staff of the DRE on communication strategy. This policy will be pursued in 7 regions in 2014 (with MEP resources). 10. With regard to intercultural education the Project gave the opportunity to the newly established Department for Intercultural Education (DEI of 6 staff) to put in place a full scale intercultural education policy including: (i) elaboration of contextualized didactic materials to be used in workshops and trainings; (ii) organization of a series of trainings for teachers and regional advisors to adapt curriculum contents and teaching methods to the needs of indigenous community17; (iii) definition of infrastructure needs and design of secondary schools through a participative methods to respect the indigenous culture in the targeted areas; and the (iv) design, organization and construction of the DRE of Sula entirely dedicated to education in the indigenous territories. 11. The subcomponent to improve the supervision process (definition of supervisors’ skills, normative, in service training of supervisors, supervision strategy) was not implemented. Because the bidding process launched in 2011 was delayed and not completed. However the Operation Manual for supervision was realized with MEP resources. This activity will be completed internally by the MEP in 2014 with its own resources. 12. The MEP wanted to improve the organization to manage conflicts at various levels of the education system. This activity was not realized because the bidding process was not successful. 17 A complete list of the training and staff involved is in annex. 33 Component 3: Quality of Education a. The development and implementation of a training program in intercultural education for MEP staff at the central and regional level. 13. In 2011, a unit “Departamento de Educación Intercultural” DEI of 6 staff was established in the MEP (Decree N° 36451-MEP) to be in charge of the program promoting intercultural education. (DEI substituted to the Department of Indigenous Education). The DEI elaborated: (i) the program to develop intercultural education; (ii) the content of all the training sessions; (iii) the documents and publications. This program was not fully implemented and a small number of teachers mostly from the DRE of Sula received a certified training. (see table below). All the advisers were trained to support the implementation of the intercultural policy in indigenous territories. Training in intercultural Date Primary Secondary Advisers Duration of the education (certified) Teachers Teachers training in hours Context and Relevance in 2012 379 60 Educational Practices: an intercultural vision 2012 26 2 3 120 Knowledge and Cultural feelings of Indigenous Peoples in dispersed rural areas in Latin America. Development of training 2012 28 40 materials for contextualized teaching English Third cycle in Rural High Schools in indigenous territories, with support of ICT. Update in the new Spanish 2013 129 40 program for the teachers and education centers of the DRE of Sulá Training in Cabécar language 2013 96 40 Methodology for the 2013 23 4 2 40 development of an Intercultural teaching practice in the classroom (MEP financing) Basic teaching skills to work in 2013 79 40 rural indigenous communities (MEP financing) 34 In addition about 30 workshops were organized to promote intercultural education through a participatory process. b. A rural education quality improvement program adapted to the local and cultural context (i) Improved model for rural secondary education 14. The Project was instrumental to implement the MEP strategy to transform Telesecundarias to Liceos rurales, a key policy to promote access and equity in the remote rural areas with dispersed population. In 2006, there were 139 Telesecundarias providing a low quality secondary education which failed to provide the full curriculum allowing the students to get the “bachillerato” level 18 . On April 2009, the Higher Education Council (CSE) approved the new curriculum of the Liceos rurales (April 2009) from grade 7th to 11th. The transformation consisted in: (i) implementing the new curriculum with a minimum of 5 professors by Liceos; (ii) rehabilitating and building schools infrastructure; (iii) appointing in each institution a Director and an administrative staff; (iv) adapting the program contents to the context and providing specific books and pedagogic materials; (v) increasing the number of scholarships available; (vi) increasing enrollment in each institutions; (vii) a wage premium of 50% to attract teachers; and (viii) providing energy, connectivity and computer equipment. To date: (i) there are 109 liceos rurales (of which 25 in indigenous territory) and only 32 Telesecundarias remain; and (ii) enrollment in Liceos rurales tripled from 3,113 students in 2009 to 9,501 in 2013. 15. The development of rural teaching methods that use ICT and the provision of computer equipment and software for the purpose of putting into use these rural teaching methods in all schools belonging to the Schools Networks under component 1 was not implemented: (i) the contract with “Fundación Omar Dengo” ($400,000) to develop teaching methods was not signed because the equipment were not purchased and there was no agreement on some conditions; and (ii) the bidding process for computer equipment and software was not finalized because of the legal issue with the “Contraloría General de la República.” Provisions have been made in the 2014 education budget to complete this activity. (ii) Professional development programs for teachers assigned to rural areas 16. Design and implementation of a teacher development program in the Regional Direction of Sula. In the Regional Direction of Sula was elaborated, experimented and implemented a comprehensive program to train all the administrative staff and teachers of 18 The 2006 assessment by the MEP shows that “telesecundaria” schools: (i) provide only 8 and 9th grades curriculum; (ii) lack of supervision from the MEP; (iii) have non functional and deteriorated facilities some with no electricity and connectivity for the TV set; (iv) receive few support of books and didactic materials; and (v) have less than three teachers per school, most of them without the right skills and profile for secondary education. 35 the region in intercultural education in four topics (language, territory, world view or cosmovisión, use of ICTs): (i) 175 teachers in primary and secondary education organized around 6 networks in 3 three days workshops; (ii) 35 preschool teachers; and (iii) 400 administrative staff of the region; (iii) 1697 teachers out of 12,854 (13.2%) secondary education teachers were trained in intercultural education and the modality “Liceos rurales.” In the targeted areas, 35 secondary education teachers were trained out of a total of 545 (24.7%). 17. Starting from scratch, the DEI initiated a vast program to collect, through a participatory approach, the traditions, customs and languages of indigenous cultures in order to renew curriculum contents and elaborate didactic materials. As a first phase, most of the documents produced are for the use of teachers, pedagogic advisors and supervisors and has been disseminated through training and workshops. The most relevant are (i) strategy and methodology for the implementation of intercultural education and curriculum (distributed to 105 teachers and advisors); (ii) three training modules of intercultural education (distributed to 5,153 teachers, supervisors and advisors); (iii) video documentary of the Battle of Sardinal and Trinidad 1856-57 (distributed to all 27 DRE to be used for teacher training); and (iv) 4 pedagogical materials related to literature, tradition, history and cultural events distributed to 200 teachers of the DRE of Turralbia. Few pedagogical materials were produced for the direct use of students. The most relevant are the 4 English language-learning manuals tailored to an indigenous context, which were distributed and used by 5,630 III cycle students. The target of the Project was met, but the main limitations for a larger dissemination were time, capacity and budget resources. Five additional documents were produced by DEI, but their dissemination in 2014 or 2015 is subject to budget availability. The full list of documents is presented in the following tables. Didactic materials produced and disseminated Production Dissemination Users For all indigenous territories Strategy and methodology for the 120 105 Teachers and implementation of intercultural education pedagogical and curriculum advisors Three training modules of intercultural 29400 5153 Teachers, education pedagogical advisors, supervisors in 27 DRE Seven Mini-encyclopedias of Indigenous 1500 1500 To be on the WEB Territories (DVDs) in 2014 4 manuals to learn English in an 12000 5630 Students in the III indigenous context cycle of Primary Video documentary of the Battle of 60 30 For Pedagogical Sardinal and Trinidad 1856-57 advisors to be used for teacher training Contextualized bilingual and intercultural 1 1 To be updated and Indigenous Education policy enriched in 2014 36 Strategy for revitalizing the Boruca and 1 1 For consultation by Térraba language the Boruca and Térraba community In the DRE of Turrialba Booklets of pedagogical activities 200 200 Teachers in the DRE of Turrialba Didactic Literature 150 150 Teachers Report of the meeting of administrative 1 1 To be consulted by circuits pedagogical advisors and DRE authorities Contextualized calendars 28 28 Strengthening of cultural identity Books , legends and historic traditions of 101 101 Teachers the Cabécares In the DRE of San Jose Oeste Contextualized planning (Mathematics, 4 4 Teachers Spanish Social Sciences, Social studies) Elaborated by students of the DRE Los Santos Projects for intercultural education for 3 3 Students in III and indigenous Ngöbes IV cycle Drawings about institutional ownerships 50 50 Students Didactic materials produced and not disseminated (will be subject to the availability of 2015 budget) Title Production Potential users Strategy for the elaboration of contextualized pedagogic 1 Teachers and materials pedagogical advisors A compendium of stories, customs and tradition of the 1 Teachers and Bribris and Cabécares pedagogical advisors A methodological guide for Cabecar Sa Nayuwo Sa Teachers and Siwawa for first grade pedagogical advisors Research and diagnostic about musical expression of 200 Teachers and indigenous Bribi and Cabécar pedagogical advisors Linguistic and cultural revitalization of the Cabécar 1 Teachers and language of the Ujarrás indigenous territory pedagogical advisors c. Improvements in MEP staff and teacher capacity to analyze student learning Assessments 18. Very little has been done to improve the capacity of MEP staff and teacher to analyze student learning assessment. In 2011, 2012 and 2013 seven trainings in the DRE of Sula were organized in which participated a total of 399 teachers and 24 MEP staff in the following topics: (i) regulation for repetition and drop out; (ii) integral process of learning assessment; and (iii) assessment in first grade. But the new standard methodology to analyze student learning assessment was not developed. 37 Annex 3. Economic and Financial Analysis 1. The cost-benefit analysis performed for the restructuring (2011) provided a sound basis to estimate the economic results of the operation. Based on the same assumptions used in 2007, an economic analysis was carried out for the ICR using actual data for the different variables. The updated economic analysis (EA) analyzes the following aspects: (a) the Project’s overall economic evaluation including cost-benefit analysis; and (b) fiscal impact of the Project. Since is this EA is based on actual data sensitivity analysis is not presented. 2. Project Interventions and Direct and Indirect Beneficiaries: The Education Equity and Efficiency Project aimed to close the growing gap between rural and urban education outcomes by financing the preparation of POA Activities in regions with low education indicators, mostly rural, which include: (i) rural education access, (ii) teacher staff development, (iii) pertinent pedagogical and academic modalities for rural contexts, (iv) targeting and monitoring of impact of demand-based equity Projects, and (v) capacity building of institutional actors involved delivering rural education services. Also, the Project supports targeting of education demand subsidies and finances capacity building strategies for institutions delivering rural education services. These interventions are expected to have quantifiable benefits that guarantee a positive internal return on Project investments. Project Interventions Yielding Direct Quantifiable Benefits Interventions Quantifiable Benefits • POA Activities • Reduced Repetition • Equity Programs • Reduced Dropouts • Improved Capacity and Efficiency of • Increased Primary Education Completion Technical Units of MEP managing the Rates provision of rural education services • Reduced Errors of Inclusion of Education Demand Subsidies • Increased lifetime earnings of 6th grade and 9th grade graduates 3. Assumptions for Economic Analysis of Equity and Education Project. The following parameters are considered relevant in estimating the economic benefits of the Project: (i) the length of the Project horizon (2007-2013), (ii) the time of impact of the Project in the students population, (iii) the size of the target population, (iv) the actual patterns of repetition and dropout rates in rural population, (v) the average number of years completed in rural areas; (vi) the existing patterns of errors of inclusion in the education demand subsidies; and (vii) the existing cost structure in the Education sector. Given the medium-to-long-term effect of the changes, the estimates presented in terms of reduced repetition and dropout rates and improved targeting of the education demand subsidies are conservative. Moreover, the analysis of the Project uses the following: (i) a discount rate of 10%; (ii) enrolled students are based in actual data during 2005-2013 period. 38 Cost-Benefit Analysis 4. Summary of Costs and Benefits. The criteria used to evaluate the economic efficiency of the Project include Net Present Value (NPV) and the Internal Rate of Return (IRR). The preliminary estimations took in consideration two sources of direct benefits. The first one is associated with the savings to the educative system from the reduction of repetition rates in rural areas. The second source of direct benefits considers the reduction in dropout rates that implies necessarily an amount of monetary resources saved to the system. Unitary costs per student in primary education for rural areas were estimated around US$616 and for the case of a student in secondary education the unitary costs are estimated around US$1,003. Two new sources of direct benefits will include: (i) savings generated by reducing the error of inclusion of education demand subsidies (scholarship, bonuses, transportation and meals); and (ii) increased future earnings of 6th grade graduates as opposed to non-graduates in rural areas. 5 The direct investment costs are distributed over an actual 7 year period (2007- 2013) and we made the assumption that the government will cover an additional recurrent cost to keep the continuing of the benefits achieved after the final year of the Project. This recurrent cost was assumed to be similar to the government investment realized during the life of the Project. The direct costs ere US$55.1 million investment costs associated with the Project. These costs would be incurred over the period 2005-2014. Given the expected disbursement profile, the present value of the is US$31.9 million when discounted with a ten percent discount rate. 6. The ratio benefits/costs, considering the full cost of the Project, would yield nearly US$1.4 of benefits for each dollar invested. The Project would yield a present value of net benefits, after investments, of US$15.1 million over ten years and produce an internal rate of return (IRR) of 29%. 7. The main results include: Year Investments Total Benefits Net Benefits 0 2005 0 -5,264,199 -5,264,199 1 2006 0 -7,824,023 -7,824,023 2 2007 4,650,000 -1,305,093 -5,955,093 3 2008 5,159,972 13,936,116 8,776,144 4 2009 6,000,312 21,836,330 15,836,018 5 2010 5,358,252 13,691,491 8,333,239 6 2011 12,865,915 11,050,702 -1,815,213 7 2012 6,630,936 19,258,453 12,627,517 8 2013 13,041,491 22,604,489 9,562,998 Total 53,706,878 87,984,266 34,277,388 Present Value Year Investments Total Benefits Net Benefits 0 2005 0 -5,264,199 -5,264,199 1 2006 0 -7,112,749 -7,112,749 2 2007 3,842,975 -1,078,589 -4,921,564 3 2008 3,876,763 10,470,410 6,593,647 4 2009 4,098,294 14,914,507 10,816,213 39 5 2010 3,327,053 8,501,339 5,174,286 6 2011 7,262,474 6,237,833 -1,024,640 7 2012 3,402,719 9,882,631 6,479,913 8 2013 6,083,952 10,545,161 4,461,209 Total 31,894,229 47,096,346 15,202,116 BC 1.4 IRR 29% Fiscal Impact of the Project 10. The first step in the analysis for the Project sustainability was to establish the base level expenditures prior to the implementation of the Project. After this evaluation, calculations were made on the additional costs that result from this Project. Type of Expenditure 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 Education Ministry Expenditures (EME) 1,164 1,066 1,059 1,150 1,225 1,393 1,765 1,852 2,024 2,106 Investment (US$ million) 0.15 0.66 1.50 1.36 8.87 2.63 9.04 Recurrent Costs (US$ Million) 4.50 4.50 4.50 4.00 4.00 4.00 4.00 Increase in EME -99 -7 92 74 168 372 87 172 82 Recurrent Costs as a % of EME 0.39% 0.37% 0.32% 0.23% 0.22% 0.20% 0.19% Recurrent Costs as a % of increase of EME 4.9% 6.1% 2.7% 1.1% 4.6% 2.3% 4.9% Investment and Recurrent Costs as a % of EME 0.4% 0.4% 0.4% 0.3% 0.7% 0.3% 0.6% Investment and Recurrent Costs as a % of increase of EME 5.1% 7.0% 3.6% 1.4% 14.9% 3.8% 16.0% 11. From a fiscal point of view, the total costs of the Project, including investment and recurrent costs represent approximately less than 1% of the MEP expenditures, which indicates that the implementation of the Project was highly viable though dependent on maintaining recurrent allocations for equity programs and infrastructure maintenance. 40 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Joel E. Reyes Sr. Institucional Dvlp. Specialist LCSHD Team Leader Helene Ribe Country Sector Leader LCSHD Sector Leader Manuel Salazar Social Protection Specialist LCSHD Operational Vicent Paqueo Lead Economist LCSHD Economics Raja Bentaouet Kattan Education Specialist HDNED Education Evelyn Villatoro Procurement Specialist LCOPR Procurement Manuel Vargas Financial Management Specialist LCOAA Financial Joshua Gallu Junior Professional Associate LCSHD Operations Aracelly Woodall Costing Specialist LCSHD Operations Christina Alquina Program Assistant LCSHD Operations Dalyn Meza Rural Education Specialist Consultant Education Ximena Traa Valarezo Social Evaluation Specialist Consultant Indigenous Peoples Denise Vaillant Teacher Development Specialist Consultant Education Jose Sucuc Intercultural Education Specialist Consultant Education Jose Luis Guzman Education Planning Specialist Consultant Education Francisco Esquivel Targeting Analysis Specialist Consultant Education Sandra De Barraza Education Management Specialist Consultant Education Sanigest Internacional Economic and Social Evaluation Consultant Economics Supervision/ICR Marcelo Becerra Lead Education Specialist LCSHE Team Leader Christel M. J. Vermeersch Senior Economist LCSHH Team Leader Tomas Socias Sr. Procurement Specialist LCSPR Procurement Maria Elena Paz Gutzalenko Program Assistant LCSHE Operational Fabienne Mroczka Financial Management Specialist LCSFM Financial Mgnt. Alvaro Larrea Senior Procurement Specialist LCSPT Procurement Martha Laverde Senior Education Specialist LCSHE Education Monica Lehnhoff Procurement Analyst LCSPT Procurement Dianna Pizarro Sr Social Development Specialist LCSDE Indigenous Peoples William Experton Consultant LCSHE ICR Primary Author Janet Entwistle Senior Operations Officer LCSHE Operational Advise Keisgner Alfaro Senior Procurement Specialist LCSPT Procurement Christina Alquinta Consultant ECSH2 Operations Jose Simon Rezk Senior Financial Management LCSFM Financial Mgnt. Antonio Blasco Sr. Financial Management Specialist LCSFM Finacial Mgnt. Sally L. Burningham Sector Manager LCSDE Operational Advisor Juan Luis Cordova Guirola Consultant LCCGT Economics Angela Demas Senior Education Specialist HDNED Education Emanuela Di Gropello Sector Leader AFTHD Education 41 Viviana A. Gonzalez Program Assistant LCSHH Operations Andrea C. Guedes Senior Operations Officer ECSH2 Education Suzana de Campos Abbott Consultant LCSWS Operations Advise Cristian Aedo Inostroza Senior Education Econ. ECSH2 Education Julia B. Nannucci Senior Program Assistant LCSHH Operations Alberto Pimentel Mata Consultant LCCGT Operations Laura B. Rawlings Lead Social Protection Special AFTSE Social Ximena B. Traa-Valarezo Consultant LCSSO Indigenous People Manuel Vargas Madrigal Lead Financial Management Spec MNAFM Financial Mgnt Gabriel Esteban Barrientos Team Assistant LCSHE Operations Evelyn Villatoro Senior Procurement Specialist EASR1 Procurement Aracelly G. Woodall Senior Program Assistant LCSTR Operations (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY99 0 8 FY00 8 46 FY01 0 0 FY02 9 171 FY03 31 199 FY04 9 45 Total: 57 469 Supervision/ICR FY04 3 17 FY05 6 39 FY06 13 100 FY08 17 103 FY09 36 164 FY10 27 166 FY11 17 122 FY12 3 79 FY13 5 76 FY14 5 65 Total: 132 931 42 Annex 5. Beneficiary Survey Results (if any) 43 Annex 6. Stakeholder Workshop Report and Results (if any) 44 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR 1. Introduction 1. The objectives of the Equity and Efficiency of Education Project, executed by the Ministry of Public Education (MPE), are to: (i) reduce rural education quality gaps in the Costa Rican education system; and (ii) improve equity and efficiency in the allocation, administration, and use of the borrower’s education sector resources. 2. Despite design limitations and organizational problems, the Project outcome was acceptable, as evidenced by improvement in systems that have a direct impact on equity, with benefits being focused on the indigenous population but with national coverage and by the significant improvement in the education model and conditions related to education materials for priority indigenous communities, along with the positioning and recognition of an indigenous education agenda, a situation that represents a qualitative leap forward relative to the situation that existed prior to the Project. 2. Project Objectives, Design, and Implementation Processes 2.1. Project Objectives 3. The objectives set forth in the Project are based on very sound commitments of the Costa Rican State. - The education policy entitled “Quality Schools as the Central Element of the Costa Rican Education System” [El Centro Educativo de Calidad como Eje de la Educación Costarricense], ratified on June 30, 2008, which sets forth the importance of the intercultural education approach in the Costa Rican education system. - The MEP’s institutional reform process, implemented by means of Executive Decree No. 34075-MEP of November 5, 2007, establishing a new administrative organization of the central offices. This reorganization (which was continued through subsequent partial reforms) strengthens the intercultural education strategy and converts the Department of Indigenous Education into the Department of Intercultural Education. - During the Laura Chinchilla Administration (2010), the Superior Council for Education Consejo Superior de Educación (CSE) identified four major national education objectives: (i) promoting education initiatives that increase equitable access to high quality and relevant education and training opportunities; (ii) strengthening the overall development of students through a solid and well- rounded curriculum; (iii) strengthening the mechanisms used to link education and training opportunities with the needs of national and local economies; and (iv) guaranteeing transparency and strengthening education sector administration through greater administrative efficiency and the optimal use of resources. - Under the “Jorge Manuel Dengo Obregón” National Development Plan (2006- 2010), the MEP identified ten strategic work guidelines to improve the Costa 45 Rican education system and thus to contribute to the development of the country. Three of these areas are directly linked to policies involving access to education by the population groups most impacted by social inequality and the efficiency with which these policies are executed. They involve (i) guaranteeing the right to education through measures to promote equity that facilitate progress toward the achievement of universal coverage at the preschool, primary and secondary levels; (ii) ensuring that schools and the administrative divisions of the MEP have appropriate, adequate, and well-adapted infrastructure and equipment for the proper functioning of the education system and promotion of the overall development of students; and (iii) ensuring that, in their administrative dealings with the MEP, the treatment extended to students, educators, officials, and the education community in general is timely, appropriate, flexible, efficient, and considerate. - In 2009, the MEP introduced an institutional reform program through Executive Decree No. 35513-MEP, aimed at enhancing the organizational structure of the MEP, the administrative management capacities of regional departments, the school supervision model, and promotion of intercultural education, all of which are fully consistent with the scope of the Project. - The “María Teresa Obregón Zamora” National Development Plan (2011-2014) is aimed at achieving shared and inclusive development, undertaking activities to meet the needs of the most vulnerable and disadvantaged population groups (which include communities with low social development levels, with this phenomenon being most marked in indigenous territories), taking advantage of the institutional structure and existing resources, and expanding and diversifying the social support services available, that is, enhancing efficiency in the allocation and use of resources. 2.2. Project Design and Implementation Processes 4. There were significant weaknesses in the original design owing to the scope of the goals and the complexity of the design. - The objectives were not very realistic given the intervention timeframe and investment amounts. Consequently, there was no correlation between the actual changes expected (improvement in education in all rural areas of the country) and the interventions to take place (activities over a five-year period and with an investment of US$30 million, covering institutional reforms, curricula, and physical structures). Even under a scenario of full achievement, only some of the areas could have been impacted, with the effects being weaker at the overall level. - Outputs, goals, and activities that were not defined or well calibrated, with unclear levels of authority (in terms of cause and effect) that raised questions regarding, inter alia, the rationale for and soundness of resource estimates and the identification of specific needs and demands, as well as the criteria related to viability and assessment. - The foregoing explains the limitation in the results framework, which was reformulated following the mid-term evaluation. At that time, a number of 46 framework elements were adjusted although some, for reasons related to restructuring viability as well as the lack of a sound information and monitoring system to provide alerts on inconsistencies, remained unchanged. - Design disconnected from the institutional situation. Under the Project, the work program and results framework designed did not, from the outset, take into account, the levels of interdependence required for execution. The disconnect began with the issue of the coordinating unit (PROMECE) which, by its mere existence, served to weaken the institutional framework, given that it was considered a unit, although structurally and functionally, it was at the same level as the MEP and had important legal capacity that provided it with a minimum level of deconcentration, although in reality this status was used by the unit and considered by administration and the World Bank as a status approximating decentralization. 5. This issue was partially addressed with the 2010 restructuring: - The alignment between activities and the Government’s strategic priorities was strengthened by addressing regional development disparities using a territorial development approach that transcends geography so as to adopt a comprehensive vision, with specific emphasis on the ethnic dimension. - The educational model applied in rural areas was redefined with a shift from distance- learning secondary schools (Telesecundarias), seen as a partial solution and as a way of perpetuating inequity, inasmuch as it does not allow for completion of the specialized education (ciclo diversificado) cycle, nor does it guarantee a well-rounded education or at a minimum, an education comparable to the national average, to a liceos rurales (rural high school) model, approved by the Superior Council for Education in 2009. This entails the introduction of a broader pedagogical approach, focusing on three areas: (i) teaching of core subjects; (ii) civic and intercultural education; and (iii) developing socio- productive projects. In the indigenous territories, the liceos rurales model incorporates local culture and links primary and secondary education by establishing a functional linkage between the networks of the two, using a collaborative teaching model and information and communication technologies. - The administration was reorganized to align functions and positions with the established standards, the institutional obligations of the PCU, and best practices in project management, and to identify a general process that links the series of related procedures (or specific processes) with the proper project implementation. 3. Project Results in Relation to Agreed Objectives 3.1. Project Development Objectives (PDOs) 6. On balance, the results obtained at project closing are satisfactory since the goals set for the three project development objectives were met, although there were a number of 47 weaknesses in the results and monitoring framework, as was noted and is reiterated in this section.  PDO1. Reduce quality gaps in project beneficiary areas 7. This objective is analyzed on the basis of the academic achievement of the student population receiving secondary education. Specifically, it examines the successful completion of this cycle by passing the national examinations to obtain a bachillerato (high school diploma). This involves taking a national standardized test that assesses a body of basic knowledge. Successful students obtain the bachillerato credential (along with a grade point average earned in the last two years of education). 8. It is important to note that this indicator has not been monitored consistently, partly because of shifting priority areas, which was not adequately reflected in the results framework or in the monitoring system. For the current assessment, the historical series was revised for the priority areas. 9. The national figures and those of the target area (school districts with direct project intervention) for 2006 stood at 62.7 percent and 14.1 percent, respectively, with a difference of 48.6 percent between the two. A positive trend can be noted, with a significant reduction in the gap beginning in 2008, which averaged roughly 27 percent (see diagram 1), compared to a stable trend at the national level. In 2013, the percentage of students who passed the exam at the national level stood at 69.8 percent, while in the priority area this figure was 40.5 percent, with a 29.3 percent gap between the two. Diagram 1. Percentage of students passing the bachillerato, 2006-2013 80 69.8 67.2 68.4 68 66.9 69.8 70 62.7 64.6 60 50 45 41.9 40.5 41.6 40.5 37.4 40 30 14.1 20 10 10.3 0 2006 2007 2008 2009 2012 2011 2012 2013 Nacional Área meta Spanish English Nacional National Área meta Target Area Source: Department of Academic Assessment and Certification, Director for Quality Management and Assessment , Ministry of Public Education (several years) 48 10. In other words, the indicator improved in both areas, with this improvement being much greater in the priority area, thus reflecting a significant narrowing of the gap.  PDO 2. Improve Equity 11. At the national level, the goal of the education system as a whole is to provide universal secondary education, with the successful movement of students through the system. The biggest challenge is in the rural areas and, even more so, in the indigenous territories where there have been longstanding problems in terms of access and opportunities for social, cultural, economic, and political participation by the population. Recognizing the difficulties associated with the retention of students in the formal education system, PDO 2 was defined as follows: Reduce the gaps in the secondary graduation rate in the project beneficiary areas. In other words, improve the ratio between the total number of students who enroll in secondary education and those who complete the cycle, measured by a comparison between enrollment at the eleventh and seventh grades nationally and in the Project beneficiary zones. 12. At the national level, the starting secondary graduation rate was 34 percent of students. In the beneficiary areas, this indicator stood at 20.6 percent. There was a significant gap, corresponding to low expectations of completion of the cycle. During the intervention period, there were improvements in both areas—a rate of 43.6 percent was registered at the national level and in the areas covered by the Project, this figure stood at 34.4 percent (see Diagram 2). The narrowing of the gap was greater than expected (a narrowing to 12.5 percent was expected; this gap narrowed to 9.2 percent). Diagram 2. Graduation rate at the national level and in the target areas, 2006-2013 50 40 30 20 10 0 2006 2007 2008 2009 2010 2011 2012 2013 Nacional Area meta Spanish English Nacional National Área meta Target Area Source: Department of Statistical Analysis, Ministry of Public Education (several years)  PDO 3. Improve institutional efficiency 49 13. Under the Project, a reduction in per capita administrative costs was achieved in the case of the beneficiaries of the MEP’s equity programs, which include scholarships, transportation, and school meals programs. 14. This objective yielded positive results, given that the indicator improved—falling from a base value of C 10,855 in 2009 to C 8,005 in 2013 (see Diagram 3). A general improvement in the trend started in 2011, which is in part explained (as will be discussed later) by the institutional improvements in the management of equity programs, which benefitted the entire population, particularly the most vulnerable sectors. Diagram 3. Administrative cost per beneficiary, equity programs 2009 - 2013 Nacional 15000 10000 5000 0 2009 2010 2011 2012 2013 Spanish English Nacional National Source: Office for Equity Programs, MEP, 2014. 15. This entails more efficient use of resources, so that they can be channeled toward the effective provision of subsidies for student scholarships, transportation, and school meals programs, thus reducing the amount provided to the organization for the provision of these services. Expectations were exceeded in this area during the period and the trend was very positive. 16. It should be also noted that there were problems in terms of the suitability of this indicator relative to intervention results. In addition, there were a number of deficiencies in determining attribution of the effect. This indicator already reflects a national trend; however, it does not reflect the efficiency of equity programs in the priority areas. 17. Similarly, it should be borne in mind that equity programs entail a direct relationship between the beneficiary and the central level of the MEP, without associated administrative costs at the regional or local management levels and does not take into account the work of school councils (Juntas de Educación y Administrativas), which is unpaid. Consequently, no assessment is being done of whether there was in fact improvement in the process that was targeted in the first place. 50 3.2. Intermediate Results 3.2.1. Component 1: Efficient and equitable access to rural education 18. The main achievements were as follows: - The positioning and activation of the 15 school networks based on the recognition of the relationship dynamics among the communities, primary schools, and secondary schools, and taking into account the primary goal of improving the level of secondary education. The school networks in Boca Cohen, Vesta, Katsi, Sepecue, Coroma, Palmeras, Roca Quemada, Alto Quetzal (or Fila Carbón II), Térraba, La Casona, San Rafael de Cabagra, Yeri, San Rafael Bajo Los Indios, Abrojo Guaymí – Montezuma, and Yimba Kaj were provided with infrastructure and the basic operating resources. - Construction or improvement of a total of 78 schools, among them 15 secondary schools accounting for 18 percent of the work, with the remaining 82 percent of work being done on primary schools. This entails a much greater level of achievement than projected, not taking into account the inability to complete construction of three schools (Vesta, Sharabata, and Jonkruhora) owing to access problems, which were compounded by weather-related conditions. All told, investments were made in the construction or improvement of 236 classrooms, 72 percent of which were new and 28 percent, rehabilitated. The work was done using a community participation model in 19 percent of the cases, a management firm in 28 percent of the cases, and a construction company in 53 percent of the cases. All schools were furnished (with community contributions in 26 cases). - Construction and furnishing of the Sulá Regional Education Directorate (DRE) and of the six administrative districts in Suretka, Amubri, Sepecue, San Gabriel, Gavilán, and Namaldí. - Furnishing of schools via the Ethics, Aesthetics, and Citizenship (Ética, Estética y Ciudadanía) program, by providing sporting and musical equipment to the 15 secondary schools covered by the Project. 19. Implementation of this component was satisfactory. All the objectives were met or exceeded, with the exception of the visual arts and computer equipment. The physical layout of the spaces for instruction/learning directly benefitted a group of close to 5,000 students (71 percent at the primary level and 29 percent at the secondary level). In addition, the activities under this component led to other benefits, such as (i) the elimination of the gap in infrastructure access and quality among the indigenous communities; (ii) responsible environmental and health management that takes indigenous culture into account; (iii) designs that were done taking into account community specificities and expectations; (iv) positive contributions to the community economies on a micro level as well as job creation; (v) strengthening of collaborative networks and social capital; and (vi) encouraging and legitimizing social participation and project ownership. 51 3.2.2. Component 2: Improving MEP’s Institutional Efficiency - Information system 20. During the first phase of project execution (2009), several steps were taken to implement the Business Objects IT system, a series of organizational tools to improve management through training, organization, and the dissemination of information related to the operating environment, which strengthens decision making using as a basis up-to- date and reliable information. The process was not continued as it was determined that the system was unsuited to the institutional features and needs of the MEP. 21. At the same time, a process was started to design and identify the Education Development Information System (Sistema de Información de Desarrollo Educativo SIDE), establishing the High Performance Education Information System (Programa de Informatización para el Alto Desempeño (PIAD) 19 as the basis for information. The model made SIDE the umbrella system and proposed viewing the PIAD as a locally- developed data input system (from schools). However, activities related to the procurement of computer equipment and technical advice for implementing the PIAD at priority area schools did not take place. 22. In light of this situation, the MEP has continued the process, with the assistance of the Quality Management and Evaluation Directorate (Dirección de Gestión y Evaluación de la Calidad DGEC), the entity responsible for coordinating the implementation and institutionalization of the PIAD. In 2013, more than 1,600 schools (primary and secondary) had adopted the PIAD, a figure that represents 32 percent of the total. 23. In conclusion, the Project’s limitation in the area of the development and implementation of an information system is clear and, with a mere 15.47 percent of primary schools and liceos rurales receiving IT equipment, the result is unsatisfactory. 19 The PIAD was designed in 1998 as a registration platform and information system to support teaching and administrative activities and to benefit the education situation (lower dropout rates and reduce the effects of poverty) by automating processes and providing inputs for decision making and improved management, using computer tools and updated and reliable information. The model entails recording data on students and family of origin, participation in equity programs, performance indicators (success rates, absenteeism, discipline, etc.), programming of courses and schedules, school output tables, and information on staff (training and education levels, constraints) among other variables that impact both the education system and results. . 52 - Support for the Units Executing Equity and Transfer Programs 24. The Project financed a series of consulting services designed specifically to strengthen the processes and capacity of the aforementioned units and contribute to the improvement of equity programs. It also aimed to promote a review and redefinition process for the management framework of the school councils. • Consulting services provided by Sanigest Solutions: in 2009, Sanigest Solutions obtained key information on the factors influencing student dropout and absenteeism and on the impact of equity programs in reducing these rates, by linking these indicators to the execution of activities and the resulting changes. The education vulnerability index (EVI) was designed to help anticipate the main factors associated with the probability of dropping out of school or academic failure at the primary and secondary levels. The EVI was used to design the integrated scholarship evaluation model [modelo integrado de calificación de becas MICB], which the National Scholarship Fund (FONABE) began using in 2012 for the processing of new applications. This method is based on the integrated poverty measurement model (MIB), which facilitates the combination of several models, such as the Poverty Activity and the education vulnerability index. • Consulting services provided by Comercio y Desarrollo S.A.: in 2009, this firm designed a model for calculating subsidies for the equity programs, addressing such areas as efficiency, rationale, comprehensiveness, transparency, specificities and contextualization, in accordance with cultural regions and norms, ongoing updating and linkages with financial systems, and regulation and oversight, providing added value in terms of legitimacy and sustainability. Implementation of the transportation cost model began in 2010, while the cost model for the student meal program was introduced in 2013. • Consulting services on organizational analysis, updating and implementation of work processes: Pricewaterhouse Coopers (2011) contributed specifically to the development of the DPE’s target operating model, documenting processes and procedures and producing the manual of job positions and recommendations for ongoing improvement. • Consulting services on management support: in 2012, Rocío López, a private consultant, created a needs map for automating school financial management for the Equity Programs in the short, medium, and long term, and prepared the terms of reference for the design of the system for the financial, accounting, and budget transactions of the School Councils. • Consulting services provided by Proyectica S.A. (2012) y TCTE and Proyectica S.A. (2013): these consulting services produced an IT system that automated the allocation of resources for the meal program (PANEA) and the Student Transportation Program. This model needs to be completely overhauled in order to ensure the organization, efficiency, transparency, and oversight of the management of transfers, through the operation of these online processes, and careful and close coordination between the DPE and the Financial Directorate. 53 - Support for Community-Based School Councils 25. The Project provided for a process to strengthen the community-based school councils. However the contract with the consulting firm was canceled. Owing to the needs and opportunities identified (in the context of alignment with the institutional reform of the MEP), a draft regulation was prepared and submitted for public consultation in late 2013. This draft regulation resulted in a new General Regulation for community school councils and is pending approval in 2014. It aims, among other things, to clearly outline areas of responsibility, regulate the use of public funds, promote accountability, and streamline processes. 26. Generally speaking, the strengthening of the Equity Programs Directorate was evident during the Project execution process, as ratified by Executive Decree No. 38170 of February 13, 2014 on the organization for the central offices of the Ministry of Public Education, which repealed Executive Decree No. 36451-MEP of 2011 on the administrative organization for the central offices of the Ministry of Public Education, with a view to providing a coherent and coordinated regulatory framework based on internal and context-driven changes. This paved the way for not only a programmatic integration of the equity programs into a single directorate, but also for clearly defined functions for the integrated management of the programs that had been disjointed and uncoordinated in the past. 27. With respect to the strengthening of the equity programs, significant achievements were made in the areas of institutional development, enhanced efficiency, and direct impact on users: - Training was provided to eighty persons from the MEP’s Technical Units that manage the Equity and Transfers Programs, with a view to improving monitoring and evaluation of these programs. The target number of 600 persons, which was supposed to include representatives from the school councils, was not met, owing to the prior implementation of the regulatory reform process for the councils. - The trend regarding the number of beneficiaries of the MEP’s Equity and Transfers Programs is extremely positive: (i) the number of scholarships awarded increased from 68,453 to 164,311 (exceeding the target of 100,299); (ii) the number of persons receiving transport subsidies increased from 70,075 to 102,423, exceeding the target (93,424); and (iii) the number of meal program beneficiaries increased from 582,708 to 673,129, exceeding the target (656,000). - With respect to the amount of fund transfers through Equity and Transfer Programs: (i) the amount of the transfers to school councils (meals and transportation) increased from C 31,611 million to C 81,076 million, a partial achievement (89 percent) of the target set; and (ii) the value of the transfers to beneficiaries (scholarships) increased from C 861 million to C 28,320 million, achieving 73 percent of the set target. The cost models and improvements in the process demonstrated a more efficient use of resources, in a manner that ensured an effective increase in the number of beneficiaries, a streamlined costing and allocation system and, overall, a greater number of students receiving benefits 54 from the equity programs, which does not require a proportional increase in allocated funds. Support with institutional reform 28. The MEP has been pushing for significant adjustments to the structure and processes aimed at adapting the institution to serve the public interest, in accordance with management criteria based on the principles of quality, efficiency, and equity. Executive Decree No. 34075-MEP of November 5, 2007 established a new administrative organization for the central offices of the Ministry of Public Education, which, among other things, provides for the conversion of the Indigenous Education Department into the current Intercultural Education Department composed of the Indigenous Education Unit and the Contextualization and Cultural Relevance Unit. The process is later finalized with the enactment of Executive Decree No. 35513-MEP of September 25, 2009, which established the guidelines for the decentralization of regional directorates and their responsibility in promoting intercultural education as a tool for contextualizing the education policy, along with the strengthening and recognition of our society as multicultural and multiethnic. 29. Among the various areas for action covered by the Project is the strengthening of the operating management capacity of the Sulá DRE, through the preparation of assessments and execution of development plans in the networks. It was also necessary to strengthen the capacity of all of the country’s regional directorates in order to hone the skills needed to perform the assigned functions, and thus generate the impact expected of active education entities in the regions. In this connection, a pilot organizational communication and corporate governance plan was implemented. The range of topics covered by the workshops organized to design communication strategies and support the training plan included media management and information protection, voice management, assertive communication and emotional intelligence, and customer service. A total of 312 persons from the regional offices of Aguirre, Cañas, Cartago, Guápiles, Heredia, Santa Cruz, and San José Norte attended these workshops. 30. Under the MEP’s institutional reform program, a total of 632 public employees from the regional directorates received training in communication and management and customer service capacity building, in light of the empowerment strategy designed for these directorates. This figure fell far short of the target of 5,170 public employees. Six studies and several workshops were conducted, thus helping exceed the target (4). Significant gains, which must be consolidated, were made in the management of the school councils, and there was a repositioning and strengthening of the regional education directorates, unpaid administrative counterparts with varying important roles, in the education system in order to ensure the institutional efficiency of the MEP. The Project’s contribution to the reform exercise was geared in principle toward equity programs. However, in practice there was some degree of disorganization in the processes and a number of execution discrepancies, attributable in large measure to the fact that the design was not based on an adequate assessment of needs, resources, and priorities, or on a comprehensive and dynamic vision of the processes. 55 3.2.3. Component 3: Quality of education - Intercultural education training program for MEP staff at the central and regional levels 31. The implementation of the Intercultural Course on Intercultural Education designed to build the capacity of MEP staff (advisers) at the central and regional levels was spearheaded by the Department of Indigenous Education (DEI). Several activities under the Continuing Education Program in the DREs also received financial support from the Project. In that connection, workshops and meetings for regional advisers (a total of 219 advisers trained) were conducted in the DREs located in Los Santos, Turrialba, Heredia, San José Oeste, San Cruz, Nicoya, Liberia, Canas, and Sarapiquí. All told, 451 national and regional advisers (63.4 percent) were trained during the period 2012-2013, which is lower than the target set (100 percent). - Rural education quality improvement program adapted to the local and cultural context 32. Steady progress was made in the effort to convert distance education secondary schools into liceos rurales. In 2006, there were 132 distance education secondary schools and by 2013, this number had fallen to 32. As of March 2014, there are only 28 distance secondary schools. The number of liceos rurales is increasing, jumping from 63 in 2010 to 109 in 2013. In indigenous territories, the conversion process is almost complete, as only one distance education secondary school (Telesecundaria Villa Hermosa) remains in the Grande del Térraba DRE. The number of liceos rurales in these territories increased from 17 in 2010 to 25 in 2013. In 2013, new reforms for the system were approved, thereby strengthening objectives, curricula, learning assessments, and socioproductive projects, with a view to providing more opportunities to male and female students. 33. A new organizational structure of school networks and nuclear schools in school districts in the Sulá DRE paved the way for the design of an education model that is appropriate for liceos rurales and multigrade schools and uses ICTs, and includes the core subjects and themes of history, language, the cultural knowledge and values of the Bribris and Cabécares peoples, and generates development plans for each network. The Keneblok Ñita/ Shkablö Ñara school network, Siwa Bla/Shkablö Kichá nuclear schools and the Siwa Ka network’s development plan all took shape. Consequently, in the Sulá DRE, implementation of the strategy is in the nascent stages in nine liceos rurales and 41 primary schools, that is, nine networks are being consolidated. - This process involved the execution of the following activities between 2012 and 2013: (i) introduction of participatory assessments in the nine liceos rurales in the Sulá DRE and theirs respective networks (Katsi, Yorkín, Namaldí, Boca Cohen, Coroma, China Kicha. Palmera, Usekla, and Alto Cohen); (ii) development of the draft education model relevant for the liceos rurales and multigrade schools; (iii) preparation of the development plans for the nine networks; and (iv) implementation of workshops for awareness building, training, and practical work for the development of an organizational structure and the operation of the 56 networks and nuclear schools, the use of digital technologies, and the implementation of education models (more than 16 teacher training workshops were conducted). - In addition, 4,555 out of a total of 14,835 teachers received training in intercultural relations, accounting for 31 percent of the total provided for, thus demonstrating that while steady progress has been made, a large number of teachers still require training. 34. In terms of its overall scope, the liceo rural model has increased access to education in dispersed rural areas and proved to be an attractive model for young people in these communities. A steady increase in enrollment over the past five years attests to this fact: total enrollment increased from 3,113 students in 2009 to 9,501 students in 2013. 20 35. The activity pertaining to the development of teaching methods using ICTs and the provision of IT equipment and software to support this activity was not executed. Therefore, outside the scope of the Project, the Technological Resources Directorate conducted training activities in the use of ICTs for 350 teachers from the liceos rurales and multigrade schools covered by the Project. 36. The MEP launched a consultative and participatory work process with a view to identifying the ideal qualifications of teachers and their training and education needs. In late 2013, the findings of the assessments were used to draft an Indigenous Education Policy paper that included the teacher profile. The Indigenous Manual of Job Positions was subsequently prepared and published in accordance with Resolution DG-151-2013 of September 26, 2013. This profile was recognized as the basis by which contracts, training plans and professional skills upgrading, as well as other institutional programs, which strengthen the framework approach through sustained professional practice, are being and will continue to be executed. 37. Institutional efforts are being coupled with teacher training plans at state universities. A project entitled “Academic program for training teachers in cycles III and IV who work in indigenous communities” was executed and involved the UNA, UCR, UNED, and the MEP. The project runs from 2014 to 2017 and seeks to achieve the following (i) in the short term, meet the main needs of teachers; and (ii) in the long term, design and implement a university training process that leads to a university degree. 38. The final activity for improving rural education involved support from the Project to generate materials for teachers and students, which support processes for awareness building and training in intercultural education. More than 12 educational materials— 20 Ministry of Public Education, 2013. Features of Liceos Rurales at which enrollment exceeds 125 students. Curriculum Development Directorate 57 course modules, videos, history compendium, encyclopedias, textbooks, educational strategy documents—with an intercultural perspective, were published and disseminated, exceeding the target set (9). - Improvements in the capacity to assess student learning 39. The Project provided for the hiring of a consulting firm to compile the information on school performance in the project areas and design a teacher training strategy that abides by the principles of intercultural education, thus transcending traditional models. The activity was not executed, although during the project period several training activities were conducted by IDP, using MEP resources, to at least partially meet needs in this area. A total of 399 teachers and 26 administrative staff from the MEP received training in 2011, 2012, and 2013, at various workshops that covered such topics as (i) policies governing promotion and repetition; (ii) technical delivery of the documents for the 200 [sic] National Assessment Plan; and (iii) learning assessment – a comprehensive process. 4. Sustainability of the project components 40. In terms of the project’s scope, based on an increased integration of processes related to the institutional agenda, it was established that all of the activities provided for have basic sustainability requirements, contingent on the allocation of financial and human resources to the MEP: - The education policy and the programs generated give continuity and even strengthen commitments regarding rural education, and, specifically with indigenous communities, in a framework that was completely overhauled during the project period. This policy also provides better prospects for coordination with academic training and professional development bodies, which are gradually being integrated into the new approaches. - Administrative organization, including reforms of existing and established procedures and manuals, is more consistent with an approach based on equity, interculturality, and inclusiveness as key factors for improving equity and efficiency in education. - The strategy for intercultural education and the strategy for the development of liceos rurales include consolidated institutional bases, clearly defined roles and objectives, and a basic resource package. However, these efforts require sustained investments to maintain the necessary momentum, and, especially, make further progress possible. The risk lies in the possibility that, faced with a slowdown caused by a shift in priorities, the education agenda for rural areas and indigenous territories will lose momentum, and the units and programmatic activities will be retained, albeit under a basic and anemic operating system. - The reorganization and strengthening of the Regional Education Directorates is a consolidated process that is underway, and will require subsequent follow-up that involves broad-based ownership and legitimacy. The outlook is favorable, given that the sustained provision of resources for operational and strategic management is the most important factor. 58 - The general suite of equity programs has been organized and streamlined, and has in particular improved their impact through costing and resource distribution mechanisms that are in line with other public administration systems, thus strengthening their legitimacy and increasing the possibility of their applicability in the future. - Investment in infrastructure is, without a doubt, the most complex component in terms of sustainability—given that, although a specific directorate is in place at the institutional level, this directorate is facing constraints with respect to financial and human resources and approaches—for ensuring the continuity of infrastructure projects as a part of a vision of educational development. - The size of the project portfolio being managed by the MEP, based on the demand for maintenance and new works, has historically exceeded institutional capacity, and this trend is unlikely to be reversed in the future. Particular attention and a commitment will be needed to ensure that investments made are adequately maintained and, despite the fact that an expansion of the Project-financed model is deemed unlikely, that the material base is sustained in accordance with the principles that governed the investment in the beneficiary communities. - The foregoing is contingent on continuity in the strategy to build the management capacities of the community school councils, which would boost institutional capacity, and include these councils among the key actors in the processes. The new Council Regulations, which are in the process of being approved and are underpinned by a consultative and participatory design process, are an important success factor that could boost sustainability expectations. 59 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders N/A 60 Annex 9. List of Supporting Documents MINISTERIO DE EDUCACION PUBLICA (MEP) “Informe de Cierre Proyecto Equidad y Eficiencia de la Educación”, April 30, 2014 MINISTERIO DE EDUCACION PUBLICA (MEP), “Documento Final Salvaguarda de los derechos de los pueblos indígenas en la ejecución del Proyecto Equidad y Eficiencia de la Educación” (Convenio de Préstamo 7284-CR), April 15, 2014 MINISTERIO DE EDUCACION PUBLICA (MEP) Estrategia de Educación Intercultural 2008-2014: Lineamientos y Resultados, May 22, 2014 World Bank (2004), Costa Rica: Country Assistance Strategy FY Report No. 28570-CR, April 20, 2004, Washington, D.C World Bank (2008), Costa Rica: Country Partnership Strategy FY Report No. 43.276 – CR, July 25, 2008, Washington, D.C Word Bank (2006), Costa Rica, Equity and Efficiency of Education Project Loan CR- 7284, Project Appraisal Document, Report 28.326-CR, March 4, 2005, Washington, D.C Word Bank (2006), Costa Rica, Equity and Efficiency of Education Project Loan CR- 7284, Loan Agreement, June 9, 2005, Washington, D.C Word Bank (2006), Costa Rica, Equity and Efficiency of Education Project Loan CR- 7284, Project Paper for Restructuring, Report 61819-CR, June 29, 2011, Washington, D.C 61 62